What made Einstein think that gravity was caused by the curvature of spacetime?
To be exact Einstein made a claim that it is gravity that curves space-time.
You can follow his reasoning in his "Relativity: The Special and General Theory." Einstein started off with comparing acceleration caused by gravity to acceleration in a lift (assuming it moves with accelerated motion) going up. He claimed that these two accelerations are indistinguishable from each other - see chapter 20. Later in chapter 22 he said:
... we learn that a body which is in a state of uniform rectilinear motion with respect to K (in accordance with the law of Galilei) is executing an accelerated and in general curvilinear motion with respect to the accelerated reference-body K' (chest). This acceleration or curvature corresponds to the influence on the moving body of the gravitational field prevailing relatively to K'. It is known that a gravitational field influences the movement of bodies in this way, so that our consideration supplies us with nothing essentially new.
After that he concluded that light as seen from such an accelerated lift must also be curvilinear as compared to an outside inertial frame.
Einstein then proceeds with his line of reasoning that finally leads to the conclusion that space must by curved (he describes some other thought experiments, such as a spinning disk with clocks located in its center and on its edge, and also as introduces Gaussian coordinates to prove his point).
I think this book is quite easily digestible for almost anybody and worth the time.
According to the excellent and very well researched scientific biography "Subtle is the Lord" by A. Pais, as late as 1912 Einstein was still assuming a flat Euclidean space (at that point he had been working on the general theory for 5 years). Then (in 1912)
Some time between August 10 and August 16, it became clear to Einstein that Riemannian geometry is the correct mathematical tool for what we now call general relativity theory. The impact of this abrupt realization was to change his outlook on physics and physical theory for the rest of his life.
In Pais' analysis of what happened in the period leading up to this realization, he quotes from a talk by Einstein in 1922
If all [accelerated] systems are equivalent, then Euclidean geometry cannot hold in all of them. To throw out geometry and keep [physical] laws is equivalent to describing thoughts without words. We must search for words before we can express thoughts. What must we search for at this point? This problem remained insoluble to me until 1912, when I suddenly realized that Gauss's theory of surfaces holds the key for unlocking this mystery. I realized that Gauss's surface coordinates had a profound significance. However, I did not know at that time that Riemann had studied the foundations of geometry in an even more profound way.
as well as a remark by Einstein in 1923 about that same period:
I had the decisive idea of the analogy between the mathematical problem of the theory [of general relativity] and the Gaussian theory of surfaces only in 1912, however, after my return to Zurich, without being aware at that time of the work of Riemann, Ricci, and Levi-Civita. This [work] was first brought to my attention by my friend Grossmann when I posed to him the problem of looking for generally covariant tensors whose components depend only on derivatives of the coefficients $g_{\mu\nu}$.
To which Pais adds:
We learn from these two statements that even during his last weeks in Prague Einstein already knew that he needed the theory of invariants and covariants associated with the differential line element $ds^2 = g_{\mu\nu}dx^\mu dx^\nu$ in which the ten quantities $g_{\mu\nu}$ are to be considered as dynamic fields which in some way describe gravitation.