JDK-8168069 : X509TrustManagerImpl causes ClassLoader leaks with unparseable extensions
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: 8,9
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2016-10-08
  • Updated: 2024-04-03
  • Resolved: 2019-02-21
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 13
11-poolUnresolved 13 b10Fixed
Description
FULL PRODUCT VERSION :
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [Version 10.0.14393]

A DESCRIPTION OF THE PROBLEM :
When using the javax.net.ssl package together with a keystore that contains certificates that have unparseable extensions, a strong reference path is create from the static field sun.security.ssl.SSLContextImpl$DefaultSSLContext.defaultImpl all the way to the sun.security.x509.UnparseableExtension which in turn may have a Throwable with a backtrace containing strong references to classes that we may want to be gargate collected for example during a redeploy in a servlet environment.

Credits go to "CptS" @ GitHub / stackoverflow.com

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Download keystore from https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/test/resources/spi-cacert-2008.keystore (containing https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/test/resources/spi-cacert-2008.crt )

Set system property javax.net.ssl.trustStore to path of above keystore.

Invoke javax.net.ssl.SSLContext.getDefault() (or javax.net.ssl.SSLSocketFactory.getDefault() which calls the former)

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
In a servlet environment, the ClassLoader of the code calling javax.net.ssl.SSLContext.getDefault() should be available for garbage collection after a redeploy
ACTUAL -
Garbage collection is prevented by strong reference path from GC root. (Dump and analyze heap to verify.)

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Error message for parsing the certificate:

Unparseable certificate extensions: 1
[1]: ObjectId: 2.5.29.18 Criticality=false
Unparseable IssuerAlternativeName extension due to
java.io.IOException: No data available in passed DER encoded value.


GC root path available in image here: http://stackoverflow.com/questions/39741147/classloader-leak-because-of-invalid-ssl-certificate-in-solrj-httpclient-jvm-or

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
System.setProperty("javax.net.ssl.trustStore", "/path/to/downloaded/https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/test/resources/spi-cacert-2008.keystore");

javax.net.ssl.SSLContext.getDefault();

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
The ClassLoader Leak Prevention library provides a workaround by unsetting the "why" field using reflection. See https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/main/java/se/jiderhamn/classloader/leak/prevention/cleanup/X509TrustManagerImplUnparseableExtensionCleanUp.java
Comments
Git URL: https://github.com/openjdk/jdk/commit/e6f6863cbcf5e913c1035a7834907689f5889747
03-04-2024
URL: http://hg.openjdk.java.net/jdk/jdk/rev/c925e36a8e6d User: xuelei Date: 2019-02-21 02:46:47 +0000
21-02-2019
Please take care of two factors in the description: 1. the static field sun.security.ssl.SSLContextImpl$DefaultSSLContext.defaultImpl 2. sun.security.x509.UnparseableExtension has a Throwable with a backtrace containing strong references to classes. It's a result to cache the trust certificates for performance. Will consider a memory friendly enhancement in JDK 9.1.
05-01-2017
Attached the project and the profiler trace showing the issue.
02-01-2017
Mail to submitter: I am unable to reproduce the issue with the github project using the cacerts provided in the project. Here is the screenshot of the metaspace usage . After un-deploying the project, there isn’t any instances left as seen in heap dump.
02-01-2017
From submitter: -------------------- I'm assuming you're running "CptS":s project at https://github.com/CptS/solrj-classloader-leak (rather than my project at https://github.com/mjiderhamn/classloader-leak-prevention/ ). I don't think I actually ran that myself, so I cannot answer for it, but I know the Solr server must be responding for the leak to be triggered by that code. The simplified steps detailed in the bug report, or my test case at https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/test/java/se/jiderhamn/classloader/leak/prevention/cleanup/X509TrustManagerImplUnparseableExtensionCleanUpTest.java , should allow you to reproduce the issue however. If it helps, I can also provide a heap dump with the leak in it. (Possibly I could even write a stand alone application that fills metaspace with leaked classloaders).
21-12-2016