JDK version : 1.6.0_14

Note : this issue was found when investigating the PKCS11 issue for OpenSSO's J2EE Agent on GlassFish v21.b60e (final). We were performing stability tests with the PKCS11 provider removed from the list in java.security.

While running some stability / stress tests with JDK 1.6.0_14, we noticed that after about 17hrs the JVM would crash with OOME. By looking more closely at the java heap and the class histograms throughout a test run we observed the following :

1) time spent in GC is more important over time (from 0.05s to 0.15s or more) and the java heap space freed by a major GC (not a full GC) is less important over time.

2) the process size grows over time -- this growth is usually observed when the java heap size gets close to the max heap size defined. After some time the process size gets very close to the maximum 32 bits process size (when the OOME eventually happens).

3) the number of Finalizer objects when the occupied heap size is close to max heap size is very important (sometimes well over 100,000 objects). We also noticed that a lot of the references held by those Finalizer objects pointed to java.net.SocksSocketImpl, which in turn referenced java.io.FileDescriptor, and other socket objects (such as com.sun.net.ssl.internal.ssl.SSLSocketImpl).

We're not sure which part of our code is generating those specific objects that need finalization, but we know that there's a huge number of HTTPS connections being opened from the JVM (as shown by netstat), that seem to be inline with the number of file descriptors objects waiting on finalization -- those connections are in the TIME_WAIT state.

We currently think that those objects might be coming from the part of our code that does :

url = new URL(url, url.toExternalForm(), null);
HttpURLConnection conn = (HttpURLConnection)url.openConnection();

Even if we use the disconnect() method, we still see the memory growth (throughput is lower when disconnect() is called --- probably because keep alive isn't used then ?).

This is a big problem for us as it impacts the stability of our product (potential crash in production).