Neutrons reflective materials in nuclear reactors
In case of overheating and melting of the core, the control rods may not be inserted any more leading to a disaster.
I have to interject that it's a little more complicated than that, although I like to discourage people from thinking about a reactor is a super complicated system. It's not super complicated, it's just a high-order system. The temperature of the fuel is proportional to the heat content of the fuel. As you put in more heat, the temperature goes up. But that makes power the derivative of temperature. If the thermal heat production rate equals the rate of removal, then temperature isn't changing either way. But if it's not balanced power is the derivative of temperature. Our goal is always to keep temperature low so that no part of it melts.
This is why reactivity trips people up. Reactivity is the derivative of power, via neutron flux levels. If reactivity is balanced, neutron flux stays constant, if it's not, it changes over time geometrically (by that I mean it changes by percent, like 10% per minute). This means that reactivity is the 2nd derivative of temperature.
As if that wasn't enough, decay heat leaves a constant heat source even after the chain reaction has been shut off. If you can safely shut things down, then you need a pathway for the heat to get out. If you're still worried about criticality, then you have two objectives, and sometimes they work against each other. This is generally the point that people start screaming "call in the nuclear engineers!"
A subcritical mass can be made critical by placing neutron reflective material all around it. So my question is why nuclear reactors are not operated with this kind of reflective shielding instead of control rods?
Yes, of course, like this:
You can see Reflector control labeled there. That is a piece of neutron reflector that can move up and down. The nuclear core is bunt from one end to the other.
Nuclear fuel burnup is another concept, but please don't let that scare you off either. Burnup only takes place over a long period of time because nuclear fuel holds a lot of energy in total. As we keep fissioning atoms, we will eventually run out of the atoms that can fission. In terms of our mental picture, this inserts negative reactivity. The reflector inserts positive reactivity. To start the reactor, you can move the reflector into place and it will start burning. But then over time that part of the reactor you moved it to will putter out. At that point, you move the reflect a little bit further along the core, and then new fuel comes to life.
This is a "candle" type, but that's not exclusive to reflector-controlled reactors. There are also control drums. Those are big cylinders that sit to the side of the reactor, which contain neutron reflectors on one side, and something on the other side that either doesn't reflect or eats neutrons on the other side. The only difference is that this moves the reflector to the side as opposed to up or down. Everything is pretty physically obvious. You could create a viable control system with any given way you could think of to move the reflector in place.
The shield could be designed in order to support a maximum temperature before crashing/burning/melting driving the reactor far from the criticality automatically in a totally passive way.
Obviously we want that sort of thing to happen, but it's a little vague as you've proposed it so far. In the image I posted above, they very likely planned for overheating to cause the moderator ring to drop.
In practice, they'll be more conservative than designing fail-safe when temperature goes out of control. When they can no longer input an active control signal to the reactor, it will drop the control elements to shut it off. This is actually done with control rods, but it would basically work the same with a moving reflector, except that control rods move in, reflector moves out to shut down the reactor.
In summary: "yeah, sure"
The safety concern for a nuclear reactor is removing decay heat to prevent meltdown of the reactor core, not a run-away nuclear reaction. Decay heat is from radioactive decay of the products of fission and cannot be stopped by control rods or any other process. Decay heat is about 5% of full reactor power when the fission process is terminated. (As an aside, a reflector would reflect neutrons back into the core.)