How to prevent peltier power supply from oscillating

Basic problem

You have too much gain within the feedback loop and it's pushing the phase margin to the point where it becomes an oscillator.

Longer answer

Get rid of IC1 and rewire as per the purple change below: -

IC1 is bringing nothing to the party other than woe. There is enough open-loop gain within a normal op-amp (IC2) for this to work very effectively. Adding more loop gain is asking for trouble (especially if you were using a MOSFET instead of the darlington transistors).

If you have to adjust the DAC output use a resistor potential divider to produce the "new" demand voltage.

Take care to choose an op-amp that doesn't introduce problematic errors due to input offset voltage. Also take great care in ensuring that the 0 volts net is not carrying load currents between the op-amp and the DAC. Load currents and poor 0 volt net impedances can also cause oscillation and will significantly make precision control much worse.