3D Billiards problem inside a torus

These so-called "whispering gallery modes" are familiar from studies of microcavity lasers; they can trap the light indefinitely, only limited by diffraction; this web site by Jens Nöckel nicely summarizes the issues; an efficient way to untrap the trajectory is to introduce flattened portions in the boundary (in 2D this would be a stadium rather than a circle, in 3D it could be an ellipsoidal shape). A research article on these issues, with many pointers to the literature, is Chaotic light: a theory of asymmetric resonant cavities.

As to the fact that this improbable scenario arose from chaos, it seems that numerical error really was at fault (my bisection code was faulty).

This behavior is more to be expected.

EDIT: According to the literature, as Carlo Beenakker pointed out, such "whispering gallery modes" wherein trajectories can stay arbitrarily close to the surface can only exist if they trace out a curve with strictly concave curvature everywhere -- a condition often violated in a torus. This, together with $C^6$ regularity, are a sufficient condition for the existence of such caustics. Quite an improvement over the previous result by Lazutkin demanding $C^{553}$ regularity.

Source: https://mat-web.upc.edu/people/rafael.ramirez/res/pdf/oberwolfach10.pdf