How long does a permanent magnet remain a magnet?

If a permanent magnet could "decay" at the rate given in Rook's answer above there would be none found in geological strata.

A permanent magnet has a permanent orientation of the magnetic moments in a specific vectorially additive direction depending on small crystal domains. To change, i.e. be demagnetized, the magnetic moments have to be randomized by either an external magnetic field or excess heat/melting or vibrations possibly. If nothing like that happens it should be stable. Little magnets in a box left undisturbed would not change magnetisation unless a random magnetic field was in the area .

Non magnetic iron left undisturbed will acquire a field from the magnetic field of the earth, so some change in the orientation of the field could happen to these little magnets, depending on how they lay with respect to the weak field of the earth.

Some magnets, e.g. AlNiCo, require a keeper (essentially, a piece of iron placed between the poles) to keep magnetic flux lines concentrated inside the magnet, to keep from spontaneously demagnetizing (the material reaches an unstable point in the intrinsic B-H curve). They can also be demagnetized by mechanical means (e.g., by being dropped). I think this is because the shock provides energy necessary to change domain boundaries.