JDK-8136353 : ClassValue preventing class unloading
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.lang
  • Affected Version: 8,9
  • Priority: P3
  • Status: Resolved
  • Resolution: Not an Issue
  • OS: linux_ubuntu
  • CPU: x86_64
  • Submitted: 2015-09-09
  • Updated: 2018-03-02
  • Resolved: 2016-11-08
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 9
9Resolved
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
any JDK supporting ClassValue

FULL OS VERSION :
Linux 3.13.0-29-generic #53~precise1-Ubuntu SMP Wed Jun 4 22:06:25 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
ClassValue internal structures seem to prevent the garbage collection of classes, if those keep a reference to the ClassValue instance. I have tested these against several JDK 7, 8 and 9 versions, including for example JDK9 b78. And all of them fail to unload the classes from the supplied jar. It does not matter what class is used to attach a value to (my test uses Integer.TYPE ), nor does it matter if the field is static or not. If the field is changed to use a WeakReferece for example, garbage collection will work. 

This bug is quite problematic for Gradle an Groovy

THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Yes

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
I run the  provided CVTest program with very low memory settings for total memory/permgen. I tested with 4m and got a OOME afer less then 100 iterations. For this there has to be a "t/t.jar" with the test classes Dummy and MyClassValue.

EXPECTED VERSUS ACTUAL BEHAVIOR :
Since the controlling program CVTest does not keep any references to the class from t.jar or the ClassValue, all of this must be due for garbage collection at some point. Instead an OOME is thrown.
REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------

public class CVTest {

    public static void main(String[] args) throws Throwable {
        for (long i = 0; i<10000000; i++) {
            File dir = new File("t/t.jar");
            URLClassLoader classLoader = new URLClassLoader(new URL[]{dir.toURI().toURL()});
            ClassValue cv = (ClassValue) classLoader.loadClass("MyClassValue").newInstance();
            Object value = cv.get(Integer.TYPE);
            assert value !=null;
            assert value.getClass().getClassLoader() == classLoader;
            classLoader.close();
        }

    }
} 

And for "t/t.jar":

public class MyClassValue extends ClassValue {
    protected Object computeValue(Class type) {
        Dummy ret = new Dummy();
        Other.o = this;
        return ret;
    }
}

class Dummy {
  static Object o;
} 


---------- END SOURCE ----------


Comments
I disagree with this assessment. In my case, the ClassValue is held by a utility object used for our Java integration. That utility object has to live somewhere, so it's held by the JRuby runtime instance. There's a strong reference chain leading to the ClassValue. The value is a Ruby representation of the class, with reflected methods parsed out and turned into Ruby endpoints. Obviously, the value also references the class, either directly or indirectly through reflected members. So there's a reference chain from the value to the class. This Ruby class wrapper is only hard referenced directly if there's an instance of the object live and moving through JRuby. It may be referenced indirectly through inline caches. Otherwise, the only reference to it lives in the ClassValue. I do not believe this should prevent collection of the class associated with the ClassValue. The value referenced in the ClassValue should not constitute a hard reference. If it is alive *only* because of its association with a given class, that should not be enough to root either the object or the class. ClassValue should work like ThreadLocal. If the Thread associated with a value goes away, the value reference goes away. ThreadLocal does nothing prevent it from being collected. If the Class associated with a Value goes away, the same should happen to that Value and it should be collectable once all other hard references are gone.
02-03-2018

This was definitely a problem with Groovy (holding ClassInfo instances is a static set), and in this case unrelated to ClassValue. If associated values hold a strong reference to the ClassValue instances there will be an issue, but values can hold a strong reference to the class itself and there should be no issue with that, since the value has no reference to the key utilized by ClassValue.ClassValueMap. This bug can be closed as not an issue. I will open another issue to add an api note to ClassValue warning users that values should not hold references to ClassValue instance.
08-11-2016

AFAICT Groovy fixed the problem: https://issues.apache.org/jira/browse/GROOVY-7683 https://github.com/apache/groovy/commit/a8fb776023253ebc2da35538f25eccd7d59997ed Groovy's ClassValue implementation computed instances of ClassInfo that held a strong reference to the class associated with the computed value. The commit, linked to above, modified things such that ClassInfo now holds a weak reference to the class associated with the computed value. Groovy, when computing associated values, also added instances of ClassInfo to a "global" (static) set. Thus strong refs to computed values were retained for the life time of Groovy's ClassInfo class.
08-11-2016

From Jesse Glick: "While investigating a leak of `ClassLoader`s in Groovy 2.x due to the default behavior of `GroovyClassValueFactory`, I ran into https://bugs.openjdk.java.net/browse/JDK-8136353. Offhand that does not look like a bug to me (rather a mistake in Groovy), though my longstanding RFE https://bugs.openjdk.java.net/browse/JDK-6389107 would probably allow it to be fixed more easily."
29-08-2016

It's not obvious that this is a GC weak reference problem. While running the repro, the Full GCs don't clear the weak references, so it seems more likely a problem with the reproducer or the ClassValue. core-libs, could you take a look at the reproducer and the ClassValue class and figure out if this is the expected behavior or not.
04-04-2016

I just realized that the problem is probably that MyClassValue is strongly reachable from Dummy. We have the following reference chain: Dummy instance -(via class pointer)-> Dummy.class -(via Dummy.o static)-> MyClassValue instance -(via ClassValue.identity field)-> ClassValue$Identity instance The ClassValue class uses a WeakHashmap to semantically map between classes and values. Instances of ClassValue$Identity are used as 'keys' and the Dummy instances are used as the 'values'. Since the 'values' in WeakHashMap are held strongly, the keys are also strongly reachable and will not be cleared.
04-04-2016

The heap keeps growing if you run without -Xmx4m. The following command also shows that we leak the MyClassValue instances and instances of associated classes: $ jcmd CVTest GC.class_histogram For example: $ jcmd CVTest GC.class_histogram | grep Identity 45: 171212 2739392 java.lang.ClassValue$Identity
04-04-2016

You need to remove t.jar from the classpath to get the reproducer to "leak" memory: $ find -type f | sort ./CVTest.class ./CVTest.java ./t/t.jar/MyClassValue.class ./t/t.jar/MyClassValue$Dummy.class ./t/t.jar/MyClassValue.java $ java -Xmx4m -XX:+PrintGC -cp . CVTest
04-04-2016

Anyone succeeded to reproduce this issue(OOME) with attached reproducer? I couldn't reproduce with JDK9ea b78(the reporter said that he could see OOME), b80( same as Abhijit [~aroy] mentioned above) ,b111 and b112. I tried with: java -Xmx4m -cp ./t/t.jar:. CVTest
31-03-2016

Additional information: http://mail.openjdk.java.net/pipermail/compiler-dev/2016-January/009895.html > The source code in the issue description is incorrect; it doesn't > compile. Could you please attach the working test cases to the issue, so > future testers and developers can reproduce the problem? Here are links > to the 2 source code files: > https://issues.apache.org/jira/secure/attachment/12765900/CVTest.java > https://issues.apache.org/jira/secure/attachment/12765901/MyClassValue.java Attached CVTest.java and MyClassValue.java to the bug.
08-01-2016

ILW - M (possible bug in ClassValue prevents class unloading?) L (depends on use of ClassValue) H (none?) -> P4
23-09-2015

Test result: ************* Os: Oracle Linux 7- 64 bit 8.0 b132: Pass 8u60 b27: Pass 9ea b80 : Pass
18-09-2015