Positive proportion of logarithmic gaps between consecutive primes

I think I can establish the claim for all $\lambda < 1/4$. In principle one should be able to get up to $\lambda < 1.145\dots$ but as noted in other comments this would require upper bounds on the density of large intervals free of primes that do not appear to be known unconditionally.

For any $\lambda > 0$ and any $X \geq 1$, let $f_X(\lambda)$ be the proportion of $1 \leq x \leq X$ such that the interval $[x, x+\lambda \log X]$ is free of primes (one can replace $\log X$ here by $\log x$ if desired, it doesn't really change things). This is clearly a monotone decreasing function of $\lambda$ taking values in $[0,1]$. The problem is essentially equivalent to establishing a noticeable dip in the value of $f_X$. More precisely:

Proposition 1 Let $\lambda_0>0$. Then the following are equivalent:

  1. For all $\lambda < \lambda_0$, one has $S_\lambda(X) \gg_\lambda \frac{X}{\log X}$ for sufficiently large $X$.

  2. For all $\lambda < \lambda_0$, there exists $\Lambda>\lambda$ such that $f_X(\lambda) - f_X(\Lambda) \gg_\lambda 1$ for sufficiently large $X$.

Proof Suppose 1 holds and $\lambda < \lambda_0$. From the PNT and Markov's inequality one has $S_\Lambda(X) \ll \frac{1}{\Lambda} \frac{X}{\log X}$, thus for $\Lambda = O_\lambda(1)$ large enough one has $S_\lambda(X) - S_\Lambda(X) \gg_\lambda \frac{X}{\log X}$. Thus there are $\gg_\lambda \frac{X}{\log X}$ prime gaps in $[1,x]$ of size between $(\lambda-o(1)) \log X$ and $(\Lambda+o(1)) \log X$ (since $\log p_n = (1+o(1)) \log X$ for almost all primes $p_n \leq x$). This easily implies that $f_X(\lambda) - f_X(\Lambda) \gg_\lambda 1$ for sufficiently large $X$ (possibly after tweaking $\lambda$ and $\Lambda$ by $o(1)$ first).

Conversely, if 2 holds, then for a proportion $\gg_\lambda 1$ of $x \in [1,X]$, the interval $[x,x+\lambda \log X]$ is free of primes while the interval $[x,x+\Lambda \log X]$ has a prime. Let $A$ be large. The proportion of $x$ for which $[x,x+\Lambda \log X]$ has a prime but $[x-A\Lambda \log X, x]$ does not is $O(1/A)$, so by taking $A = O_\lambda(1)$ large enough, we see that for a proportion $\gg_\lambda 1$ of $x \in [1,X]$, the intervals $[x-A \Lambda \log X, x]$ and $[x+\lambda \log X,x+\Lambda \log X]$ contain primes but $[x,x+\lambda \log X]$ does not, thus $x$ lies in a prime gap of size between $\lambda \log X$ and $(A+1)\Lambda \log X$. From this one can establish $S_\lambda(X) \gg_\lambda \frac{X}{\log X}$ without difficulty. $\Box$

The claim now follows from this proposition and

Proposition 2 For any $\lambda > 0$, one has $$ 1 - \lambda+o(1) \leq f_X(\lambda) \leq 1 - \frac{\lambda}{1+4\lambda}+o(1)$$.

Proof Let $N_\lambda$ be the number of primes in $[x,x+\lambda \log X]$ where $x$ is drawn uniformly at random from $[1,X]$, so $f_X(\lambda)$ is the probability that $N_\lambda=0$. From the prime number theorem and the first moment method one has $$ {\bf E} N_\lambda = \lambda + o(1)$$ which by Markov's inequality gives that ${\bf P}(N_\lambda \geq 1) \leq \lambda + o(1)$, which gives the lower bound on $f_X(\lambda)$. For the upper bound we need to control the second moment $$ {\bf E} \binom{N_\lambda}{2}.$$ Gallagher's calculation assuming Hardy-Littlewood predicts this quantity to be $\lambda^2/2 + o(1)$ (and more generally $N_\lambda$ should be Poisson distributed with parameter $\lambda$). This is not known unconditionally, but if one uses the standard Selberg sieve estimate coming from Bombieri-Vinogradov (e.g. Theorem 7.16 of Opera del Cribro), which is an upper bound that is worse than Hardy-Littlewood by a factor of $4$, one gets an upper bound of $2\lambda^2 + o(1)$. In particular $$ {\bf E} N_\lambda^2 \leq \lambda + 4 \lambda^2 + o(1).$$ But from Cauchy-Schwarz one has $$ {\bf E} N_\lambda^2 \geq ({\bf E} N_\lambda)^2 / (1-f_X(\lambda))$$ and if one puts together all these inequalities one obtains the claim. $\Box$

If one could prove an upper bound on $f_X(\Lambda)$ that went to zero as $\Lambda \to \infty$ then we could use the above arguments to show that $S_\lambda(x) \gg_\lambda \frac{x}{\log x}$ for all $\lambda < 1$. The paper https://www.ams.org/journals/tran/2010-362-05/S0002-9947-09-05009-0/ linked to in the previous answer would extend this to $\lambda < 1.145...$ due to slightly improved lower bounds on $f_X(\lambda)$ for $\lambda$ near $1$.

One can probably improve the upper bound on $f_X(\lambda)$ a little bit by using the inequality $f_X(\lambda) \binom{-k}{2} \leq {\bf E} \binom{N_\lambda - k}{2}$ for any natural number $k$, then optimising in $k$, but I don't think this actually improves the $1/4$ threshold. The Elliott-Halberstam conjecture would let one replace $1/4$ with $1/2$ though. Meanwhile, the Maynard sieve should be able to slightly improve upon the Markov inequality lower bound for $f_X(\lambda)$ for any value of $\lambda>0$ and thus lead to modest improvement in the $1/4$ threshold (this may possibly already follow from the paper linked to above). On the other hand getting the upper bound for $f_X(\Lambda)$ below $1/2$ seems to require getting around the parity problem (in particular eliminating the scenario in which the integers split into long intervals, with the Liouville function on almost primes biased to be +1 on half of these long intervals (as measured by density), and biased to be -1 on the other half).


The inequality $S_\lambda (x)\gg_{\lambda} \frac{x}{\log x}$ should follow from the case $m=0$ of the paper https://www.cambridge.org/core/journals/bulletin-of-the-australian-mathematical-society/article/positive-proportion-of-short-intervals-containing-a-prescribed-number-of-primes/BE20070F6B2864B2EFD65C268267BC5A

at least for small values of $\lambda$. More precisely, the result should be that there exists a large constant $C>0$ such that for any $\lambda<\frac{1}{C^4\log^2C}$, we have the desired lower bound for $S_{\lambda}(x)$.

Although the above cited paper could be more useful to analyze the case of general gaps between primes, for gaps of consecutive primes I would like to refer you to the following result https://www.ams.org/journals/tran/2010-362-05/S0002-9947-09-05009-0/

in which the requested inequality may be shown for values of $\lambda<\lambda_0$, for a certain $\lambda_0>1$ explicitly computed in that paper.

I doubt it has already been proven such lower bound unconditionally for any $\lambda>0$.