Is Excel VBA's Rnd() really this bad?

To yield a better random generator and to make its performance faster, I modified your code like this:

Const N = 1000           'Put this on top of your code module
Sub ZoomRNG()

Dim RandXY(1 To N, 1 To 3) As Single, i As Single, x As Single, y As Single

For i = 1 To N
    Randomize            'Put this in the loop to generate a better random numbers
    Do
        x = Rnd
        y = Rnd
        If x > 0.5 And x < 0.51 Then
            If y > 0.5 And y < 0.51 Then
                RandXY(i, 1) = i
                RandXY(i, 2) = x
                RandXY(i, 3) = y
                Exit Do
            End If
        End If
    Loop
Next
Cells(1, 9).Resize(N, 3) = RandXY
End Sub

I obtain this after plotting the result

The result looks better than your code's output. Modifying the above code a little bit to something like this

Const N = 1000
Sub ZoomRNG()

Dim RandXY(1 To N, 1 To 3) As Single, i As Single, x As Single, y As Single

For i = 1 To N
    Randomize
    Do
        x = Rnd
        If x > 0.5 And x < 0.51 Then
            y = Rnd
            If y > 0.5 And y < 0.51 Then
                RandXY(i, 1) = i
                RandXY(i, 2) = x
                RandXY(i, 3) = y
                Exit Do
            End If
        End If
    Loop
Next
Cells(1, 9).Resize(N, 3) = RandXY
End Sub

yields a better result than the previous one

Sure the Mersenne Twister MT19937 in C++ is still better, but the last result is quite good for conducting Monte-Carlo simulations. FWIW, you might be interested in reading this paper: On the accuracy of statistical procedures in Microsoft Excel 2010.

That seems like it would take on average 1000 * 100 * 100 iterations to complete and VBA is usually a bit slower than native Excel formulas. Consider this example

Sub ZoomRNG()
    t = Timer
    [a1:a1000] = "=ROW()"
    [b1:c1000] = "=RAND()/100+0.5"
    [a1:c1000] = [A1:C1000].Value
    Debug.Print CDbl(Timer - t) ' 0.0546875 seconds
End Sub

Update

It's not that bad at all! This will work too even without Randomize

Sub ZoomRNGs() ' VBA.Rnd returns Single
    t = Timer
    For i = 1 To 1000
        Cells(i, 1) = i
        Cells(i, 2) = Rnd / 100 + 0.5
        Cells(i, 3) = Rnd / 100 + 0.5
    Next i
    Debug.Print Timer - t ' 0.25 seconds
End Sub

Sub ZoomRNGd() ' the Excel Function RAND() returns Double
    t = Timer
    For i = 1 To 1000
        Cells(i, 1) = i
        Cells(i, 2) = [RAND()] / 100 + 0.5
        Cells(i, 3) = [RAND()] / 100 + 0.5
    Next i
    Debug.Print Timer - t ' 0.625 seconds
End Sub

and Single has about half of the precision of Double :

s = Rnd: d = [RAND()]
Debug.Print s; d; Len(Str(s)); Len(Str(d)) ' " 0.2895625  0.580839555868045  9  17 "

Update 2

I found C alternative that is as fast as VBA Rnd.
C:\Windows\System32\msvcrt.dll is the Microsoft C Runtime Library:

Declare Function rand Lib "msvcrt" () As Long ' this in a VBA module

and then you can use it like this x = rand / 32767 in your code:

Sub ZoomRNG()
    t = Timer
    Dim i%, x#, y#, Found As Boolean
    For i = 1 To 1000
        Found = False
        Do
            x = rand / 32767 ' RAND_MAX = 32,767
            y = rand / 32767
            If ((x > 0.5) And (x < 0.51)) Then
                If ((y > 0.5) And (y < 0.51)) Then
                    ' Write if both x & y in a narrow range
                    Cells(i, 1) = i
                    Cells(i, 2) = x
                    Cells(i, 3) = y
                    Found = True
                End If
            End If
        Loop While (Not Found)
    Next i
    Debug.Print Timer - t ' 2.875 seconds
End Sub