Count the squares

Haskell, 71 70 65 63 62 61 58 56 bytes

Thanks to @xnor for a some ingenious improvements!

(n#m)e|e>n*m*1e3=0|n<m=m#n$e|d<-n-m=(d#m)e+1
n!m=n#m$n*m

Try it online!


JavaScript (ES6), 59 58 bytes

f=(m,n=!(k=m/1e3,c=0))=>m*n<k?c:(c++,m-=n)<n?f(n,m):f(m,n)

Test

f=(m,n=!(k=m/1e3,c=0))=>m*n<k?c:(c++,m-=n)<n?f(n,m):f(m,n)

console.log(f(1.002))
console.log(f(1.003))
console.log(f(1.004))
console.log(f(1.005))
console.log(f(1.006))
console.log(f(1.007))
console.log(f(1.008))
console.log(f(1.009))


Mathematica, non-non-competing (21 bytes)

This answer is non-competing because it doesn't answer the actual question asked! But it does answer a variant of the question, and provides an excuse to highlight some interesting math.

Tr@*ContinuedFraction

Symbolic function taking a positive rational number as input (whose numerator and denominator represent the dimensions of the original paper) and returning a positive integer. For example, Tr@*ContinuedFraction[1350/1000] returns 10. (ContinuedFraction acts differently on floating-point numbers due to precision issues, which is why a rational number is needed as input in this context.)

An interesting interpretation of the geometric procedure described in the problem (cutting squares off a rectangle repeatedly) is that it's an implementation of the Euclidean algorithm for finding greatest common divisors! Consider the example in the question itself, with ratio 1.35, which could be modeled by having a piece of paper with dimensions (1350,1000). Every time a square is cut off, the smaller number is subtracted from the larger number; so the resulting rectangles in this example have dimensions (350,1000), then (350,650), then (350,300), then (50,300), then (50,250) and (50,200) and (50,150) and (50,100) and (50,50), and also (50,0) once we take away the last square from itself. This is exactly how the Euclidean algorithm operates (modulo the difference between division and repeated subtraction), and indeed we see that 50 is indeed the GCD of 1350 and 1000.

Typically in the Euclidean algorithm, one keeps track of these intermediate dimensions and discards the number of subtractions; however, one can alternately record how many times we subtracted one number from the other before the difference becomes too small and we have to switch what we're subtracting. That method of recording is precisely the continued fraction of a rational number. (Continued fractions of irrational numbers, which never terminate, are also super cool, but not relevant here.) For example, in the example 1350/1000, we subtracted 1000 1 time, then 350 2 times, then 300 1 time, then 50 6 times; therefore the continued fraction of 1350/1000 is {1,2,1,6}. Mathematically, we've rewritten 1350/1000 as 1+1/(2+1/(1+1/6)), which you can verify.

So for this problem, if you don't stop when the squares get smaller than a certain threshhold, but simply count all the finitely many squares before it stops, then the total number of squares equals the total number of subtractions, which is to say the sum of all the integers in the continued fraction—and that is precisely what the composition of functions Tr@*ContinuedFraction computes! (For the given example 1.35, it gets the answer the OP desires, because the final square is large enough that all squares were counted. But Tr@*ContinuedFraction[1001/1000], for example, yields 1001, since it counts the one huge square and all 1000 of the small 1x1000 squares.)