Sporadic java.lang.Exception errors since Spring '16 release

I started encountering this since the release of Spring '16.

Weird #Salesforce Apex TypeException where an inner class that extends Exception can't go into a Map of <,Exception> Source

The intermittent nature makes it hard to track down. Trying to create a managed package is a bit of a role of the dice.

There is now a known issue for it - Collection store exception thrown when adding an Exception to a apex collection that is currently scheduled for the Spring '16 Patch 11.0 release.