Why does confinement apply only to quarks, and not to nucleons?

If you take an electron and a proton there is a strong electromagnetic force between them because the electron has a charge of $-e$ and the proton has a charge of $+e$. However suppose you combine the electron and proton into a hydrogen atom. The hydrogen atom has a net charge of zero so there is no strong electromagnetic force between two hydrogen atoms.

However the cancellation of the electron and proton charge isn't complete because they don't occupy exactly the same point in space. On average the spacing between the electron and proton is the Bohr radius $a_0$, so if you have two hydrogen atoms the electron-electron distance can differ from the proton-proton distance by around $a_0$. The result is that there is a relatively weak force between the two hydrogen atoms called the London dispersion force.

The point of all this is that something basically similar happens in nuclei. It's more complicated because quarks have three types of charge but basically while the strong force acts between two quarks, hadrons have a zero net colour charge so there is no strong force acting between two hadrons.

However, just like a hydrogen atom, the quarks have a non-zero average separation and as a result there is a weaker force that acts between two hadrons. This is what we call the strong nuclear force. It's a somewhat confusing terminology due an accident of history, but the strong force is the force acting between two objects with a non-zero colour charge while the strong nuclear force acts between two hadrons with a (net) zero colour charge. The strong nuclear force is the hadron equivalent of the London dispersion force.

And finally, the strong nuclear force does get weaker with distance, and in fact it gets weaker very rapidly with distance. While the EM force falls off as $r^{-2}$ the strong nuclear force falls as $e^{-ar}$ (for some constant $a$). That's why the neutron can escape from a helium-5 nucleus.

This is related to the so-called "colour charge" carried by particles involved in strong interaction. Although at first it was proposed to allow same quarks exist inside the baryons (despite Pauli exclusion principle), it is also used to describe the ability of hadrons to be free of confinement — only colourless particles (or white) can be free.

This is not a full-blown explanation of the phenomenon — it is rather a model that describes well all the variety of known hadrons based on their symmetry. As fas as I know, the problem of implementation of confinement itself remains difficult.

There are 3 colours (red, green, blue) and the associated anticolours. For example, meson contain pairs of colour-anticolour quarks while baryons contain 3 different colours that together also are colourless.

See more here: https://www.wikiwand.com/en/Color_charge

Regarding the second part of the question. Basically, yes, nucleons and mesons are stable entities with respect to QCD. Nuclei are well described as conglomerations of nucleons with small discrepancies. At the same time, mesons for a long time are considered the carriers of the strong interaction — not unlike photons in QED.

However, this "stability" is possible only below the QCD crossover energy scale when hadrons deconfine and form a quark-qluon plasma. In some sense, one may consider an analogy to atoms that are themselves electrically neutral (but consist of charged particles) and are being dissociated at high temperatures.