Tomcat intermittent slowdowns, unresponsive, hangs

Step one, as stated above, is to change the tomcat start script to add

-verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails  

When you have your slowdown, look for things in catalina.out like "FullGC" or many GCs...

I'd note, if you haven't already done so, bump up the tomcat heap size to somewhere around 1/2 to 3/4 available memory assuming this box JUST runs tomcat. For example, to set the max heap to 768 megabytes, you would add:

-Xmx768M

to JAVA_OPTS

If you're using ubuntu 10.04, these settings would generally be located in /etc/default/tomcat6.

We've had this happen when a good chunk of the memory in the "tenured" generation of the Java heap has been swapped out to disk because it is garbage and hasn't been used in a while. When a full collection is needed, that memory must be swapped back in.

In this case, your answer is somewhat counter-intuitive: REDUCE the size of the Java heap, or figure out what other stuff is using RAM that is causing the swapping. In our case, some nightly batch jobs used a bunch of RAM, causing the old generation to get swapped out to disk. So, then, the first Full GC needed the next morning took FOREVER (180+ seconds, much as you are seeing).

You might also try the concurrent-mark sweep collector, which reduces Full GC times by doing much of the work in parallel. This is the best documentation I've seen; there are also some good Sun blogs on the subject: http://www.oracle.com/technetwork/java/gc-tuning-5-138395.html