Eliot blogged about getting OutOfMemory errors after some days of running a web application.

This happened to me too and I spent almost a week trying to track down the leak with a whole arsenal of tools ... and it turned out there was no leak.

Most JVMs allocate memory on the heap for almost everything, except for reflective data. That is put in a separate location which is a section of the heap that is reserved for permanent generation. This gets easily filled up when you dynamically load an unload classes, or have a large number of classes. I simply added the following options to the java executable and all my worries were gone:

-XX:PermSize=256M -XX:MaxPermSize=256M

My entire command for Sun's JVM looks like this now:

java -server -Djava.awt.headless=true -Xms1024M -Xmx1024M
     -XX:PermSize=256M -XX:MaxPermSize=256M

Hope this helps someone.

For a while now I've been wondering why RIFE applications couldn't be re-deployed on the servlet containers we use (Tomcat and Resin). The changed classes were just never picked up. Initially I blamed the servlet containers themselves since I assumed they wrongly implemented some things in their classloaders. As is turned out, they weren't to blame at all, but the JDK was!

Yesterday I finally took the time to track it down since this is becoming an increasing burden for our production applications. I traced all classloader hierarchies, followed 'round class instances, etc etc. However, they were all newly created as they should. The last thing I hadn't checked, was if the new classes were actually created from the new class files. RIFE uses a custom classloader to on-the-fly modify the byte-code of classes that use continuations. This classloader delegates as much as possible to the parent classloaders, but it always loads the classes that are part of the web application. To do this, it needs to actually find the class files and read their content with code similar to this:

URL         class_resource = getResource(class_filename);
InputStream input_stream = class_resource.openStream();
// code that reads the bytes from the input stream

How foolish I was to expect that this would reliably provide me with the latest content of that resource. As it turns out, underneath, URLConnection is used, which by default caches the content it reads. Nowhere in the URL javadocs is this caching behavior mentioned. and the URLConnection javadocs don't say that it's the default behavior. The result is thus that the above code will not pick up contents of changed jar files after a re-deployment of the web application, since they are cached, even if the related classloader hierarchy is replaced.

The code that did the trick was:

URL           class_resource = getResource(class_filename);
URLConnection connection = class_resource.openConnection();
connection.setUseCaches(false);
InputStream   input_stream = connection.getInputStream();
// code that reads the bytes from the input stream

I hope this post will save some people the trouble I went through to track this behavior down.

The good thing is that with the next RIFE release, your applications will be re-deployable.