- !! This page is under construction !!*
For some time Tomcat has had some means of protection against memory leaks when stopping or redeploying applications. This page tries to list them, and shows the situations where leaks can be detected and fixed.
Diagnose a classloader leak upon request
Starting with tomcat 6.0.25, the manager webapp has a new "Find Leaks" button. When triggered, it displays a list of webapps (their context path) that have been stopped (this includes undeployed and redeployed ones) but whose classloader failed to be GCed.
If a leaking webapp is redeployed several times, it will appear as many times as it actually leaked.
Caution: This diagnosis calls System.gc()
which may not be desirable in production environments.
Different types of leaks that Tomcat can detect (or not)
When a webapp execution is stopped (this encompassed redeploy and undeploy), tomcat tries to detect and fix leaks.
Starting with tomcat 6.0.24, messages are logged to indicate the kind of leak that was detected.
ThreadLocal leaks
Classloader leaks because of uncleaned ThreadLocal
variables are quite common. Depending on the use cases, they can be detected or not.
Custom ThreadLocal class
Suppose we have the following 3 classes in our webapp :
public class MyCounter { private int count = 0; public void increment() { count++; } public int getCount() { return count; } } public class MyThreadLocal extends ThreadLocal<MyCounter> { } public class LeakingServlet extends HttpServlet { private static MyThreadLocal myThreadLocal = new MyThreadLocal(); protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { MyCounter counter = myThreadLocal.get(); if (counter == null) { counter = new MyCounter(); myThreadLocal.set(counter); } response.getWriter().println( "The current thread served this servlet " + counter.getCount() + " times"); counter.increment(); } }
If the LeakingServlet
is invoked at least once and the Thread that served it is not stopped, then we created a classloader leak !
The leak is caused because we have a custom class for the ThreadLocal
instance, and also a custom class for the value bound to the Thread. Actually the important thing is that both classes were loaded by the webapp classloader.
Hopefully tomcat 6.0.24 can detect the leak when the application is stopped: each Thread in the JVM is examined, and the internal structures of the Thread and ThreadLocal
classes are introspected to see if either the ThreadLocal
instance or the value bound to it were loaded by the WebAppClassLoader
of the application being stopped.
In this particular case, the leak is detected, a message is logged and internal structures of the JDK (ThreadLocalMap
) are modified to remove the reference to the ThreadLocal
instance.
Mar 16, 2010 11:47:24 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [test.MyThreadLocal] (value [test.MyThreadLocal@4dbb9a58]) and a value of type [test.MyCounter] (value [test.MyCounter@57922f46]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Note: this particular leak was actually already cured by previous versions of tomcat, because static references of classes loaded by the webappclassloader are nullified (see later).
Webapp class instance as ThreadLocal value
Suppose that we have the following class in the common classpath (for instance in a jar in tomcat/lib) :
public class ThreadScopedHolder { private final static ThreadLocal<Object> threadLocal = new ThreadLocal<Object>(); public static void saveInHolder(Object o) { threadLocal.set(o); } public static Object getFromHolder() { return threadLocal.get(); } }
And those 2 classes in the webapp :
public class MyCounter { private int count = 0; public void increment() { count++; } public int getCount() { return count; } } public class LeakingServlet extends HttpServlet { protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { MyCounter counter = (MyCounter)ThreadScopedHolder.getFromHolder(); if (counter == null) { counter = new MyCounter(); ThreadScopedHolder.saveInHolder(counter); } response.getWriter().println( "The current thread served this servlet " + counter.getCount() + " times"); counter.increment(); } }
If the servlet is invoked at least once, the webapp classloader would not be GCed when the app is stopped: since the classloader of ThreadScopedHolder
is the common classloader, it remains forever which is as expected. But its ThreadLocal
instance has a value bound to it (for the non-terminated thread(s) that served the sevlet), which is an instance of a class loaded by the webapp classloader...
Here again, tomcat 6.0.24 will detect and fix the leak by removing this ThreadLocal
reference from each Thread.
Mar 17, 2010 10:23:13 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@44676e3f]) and a value of type [test.leak.threadlocal.value.MyCounter] (value [test.leak.threadlocal.value.MyCounter@62770d2e]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
Webapp class instance indirectly held through a ThreadLocal value
Suppose we have the same ThreadScopedHolder
class (in the common classloader) and MyCounter
class in the webapp, but with the following servlet :
public class LeakingServlet extends HttpServlet { protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { List<MyCounter> counterList = (List<MyCounter>) ThreadScopedHolder .getFromHolder(); MyCounter counter; if (counterList == null) { counter = new MyCounter(); ThreadScopedHolder.saveInHolder(Arrays.asList(counter)); } else { counter = counterList.get(0); } response.getWriter().println( "The current thread served this servlet " + counter.getCount() + " times"); counter.increment(); } }
We have more or less the same kind of leak as the previous one, but this time tomcat 6.0.24 does not detect the leak when stopping the application (and does not fix it). The problem is that when it inspects the entries of ThreadLocalMap
, it checks whether either the key or the value is an instance of a class loaded by the webapp classloader. Here the key is an instance of ThreadLocal
, and the value is an instance of java.util.ArrayList
.
The "Find leaks" button in tomcat manager will report the leak when asked :
The following web applications were stopped (reloaded, undeployed), but their classes from previous runs are still loaded in memory, thus causing a memory leak (use a profiler to confirm): /testWeb
But it does not give any clue about what caused the leak, we would need to make a heapdump and analyse it with some tool like Eclipse MAT.
ThreadLocal pseudo-leak
Threads ContextClassLoader
Threads spawned by JRE classes
Threads spawned by classes loaded by the common classloader
Threads spawned by webapps
Child classloaders
static class variables
LogFactory
JavaBean Introspector cache
JDBC driver registration
RMI target
Summary matrix
References
Mark Thomas interview on DZone
Related issues
- 48837 - Memory leaks protection does not cure leaks triggered by JSP pages code