Is energy really conserved?

The topic of "Energy Conservation" really depends on the particular "theory", paradigm, that you're considering — and it can vary quite a lot.

A good hammer to use to hit this nail is Noether's Theorem: see, e.g., how it's applied in Classical Mechanics.

The same principle can be applied to all other theories in Physics, from Thermodynamics and Statistical Mechanics all the way up to General Relativity and Quantum Field Theory (and Gauge Theories).

Thus, the lesson to learn is that Energy is only conserved if there's translational time symmetry in the problem.

Which brings us to General Relativity: in several interesting cases in GR, it's simply impossible to properly define a "time" direction! Technically speaking, this would imply a certain global property (called "global hyperbolicity") which not all 4-dimensional spacetimes have. So, in general, Energy is not conserved in GR.

As for quantum effects, Energy is conserved in Quantum Field Theory (which is a superset of Quantum Mechanics, so to speak): although it's true that there can be fluctuations, these are bounded by the "uncertainty principle", and do not affect the application of Noether's Theorem in QFT.

So, the bottom line is that, even though energy is not conserved always, we can always understand what this non-conservation mean via Noether's Theorem. ;-)

Then I learned that nuclear reactions allow energy to be converted into mass.

That would be the opposite and in any case, mass is energy (and energy is mass), so converting one into the other does conserve energy.

Then I also heard that apparently energy can spontaneously appear in quantum mechanics.

For a very short time, given by Heisenberg uncertainty principle. And that's not a violation of the conservation of energy.

So, are there any other caveats with the conservation of energy?

Why "other" ? There's not any problem with the conservation of energy.