What exactly was the "classical model" of black-body radiation, and what assumption about it made it wrong?

Let me add something to anna v's answer. The classical model of blackbody radiation is based on:

1. an exact recasting of the Maxwell equations describing electromagnetic radiation in a cavity, which shows that this physical system can be described as an infinite set of classical harmonic oscillators (normal modes) whose frequencies start from zero and are not upper bounded;
2. the hypothesis that there is an effective mechanism which allows thermal equilibrium of the radiation field;
3. that statistical mechanics can be applied to such a system to calculate equilibrium properties.

All together, these hypotheses have as a consequence that:

1. due to the equipartition theorem, each normal mode should contribute to the internal energy with the same average energy: ( $$k_BT$$ );
2. in a given volume and interval of frequencies $$d\nu$$, the number of normal modes grows like $$\nu^2$$;
3. therefore the total energy per unit volume diverges, due to the unintegrable growth of normal modes at high frequencies (from here, the name ultraviolet catastrophe).

Up to here, these are the facts textbooks refer to. However, there are a few facts that could be useful to know to put things in the historical perspective and, more important, to learn a lesson which could be useful in other contexts even today.

The actual weight of the ultraviolet catastrophe argument on the historical development of quantum physics is usually over-exagerated. It looks compelling for us, but for contemporaries of Rayleigh and Jeans, it was not a clear indication that something was wrong with classical mechanics. Statistical mechanics was in its infancy and not everybody was convinced of its general validity. Remember that Boltzmann had a hard time convincing the scientific community of the truth of his ideas. In particular, the general validity of equipartition theorem was not acknowledged by everybody.

Somewhat connected with this observation, reading the first paper where Planck derived his distribution makes clear that he was not concerned by any ultraviolet catastrophe (which is not mentioned in any place in his two main contributions to the blackbody radiation). On the contrary, his main concern was the disagreement beteween new experiments by Pringsheim and Lummer and Wien's energy distribution at low frequencies (long wavelengths). A readable account of the real history of Planck's contribution to the blackbody problem can be found in the paper Klein, M. J. (1961). Max Planck and the beginnings of the quantum theory. Archive for History of Exact Sciences, 1(5), 459-479.

A final comment on hypothesis n.2 (see above). 20-th century research on dynamical systems has shown that when oscillators with very different frequencies are weakly coupled, equilibration times could easily exceed any reasonable experimental time. In a way, it has been a lucky circumstance for the birth of Quantum Mechanics that such a result was not clearly known at the beginning of the century.

This link clearly shows the classical and quantum derivations

Blackbody radiation" or "cavity radiation" refers to an object or system which absorbs all radiation incident upon it and re-radiates energy which is characteristic of this radiating system only, not dependent upon the type of radiation which is incident upon it. The radiated energy can be considered to be produced by standing wave or resonant modes of the cavity which is radiating.

The amount of radiation emitted in a given frequency range should be proportional to the number of modes in that range. The best of classical physics suggested that all modes had an equal chance of being produced, and that the number of modes went up proportional to the square of the frequency.

But the predicted continual increase in radiated energy with frequency (dubbed the "ultraviolet catastrophe") did not happen. Nature knew better.

The crucial point is that by demanding quantized modes with energy $$hν$$ the high frequencies become less probable.