United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-8168069 : X509TrustManagerImpl causes ClassLoader leaks with unparseable extensions

Details
Type:
Bug
Submit Date:
2016-10-08
Status:
Open
Updated Date:
2017-01-05
Project Name:
JDK
Resolved Date:
Component:
security-libs
OS:
generic
Sub-Component:
javax.net.ssl
CPU:
generic
Priority:
P3
Resolution:
Unresolved
Affected Versions:
8,9
Targeted Versions:
9.1

Related Reports

Sub Tasks

Description
FULL PRODUCT VERSION :
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [Version 10.0.14393]

A DESCRIPTION OF THE PROBLEM :
When using the javax.net.ssl package together with a keystore that contains certificates that have unparseable extensions, a strong reference path is create from the static field sun.security.ssl.SSLContextImpl$DefaultSSLContext.defaultImpl all the way to the sun.security.x509.UnparseableExtension which in turn may have a Throwable with a backtrace containing strong references to classes that we may want to be gargate collected for example during a redeploy in a servlet environment.

Credits go to "CptS" @ GitHub / stackoverflow.com

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Download keystore from https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/test/resources/spi-cacert-2008.keystore (containing https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/test/resources/spi-cacert-2008.crt )

Set system property javax.net.ssl.trustStore to path of above keystore.

Invoke javax.net.ssl.SSLContext.getDefault() (or javax.net.ssl.SSLSocketFactory.getDefault() which calls the former)

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
In a servlet environment, the ClassLoader of the code calling javax.net.ssl.SSLContext.getDefault() should be available for garbage collection after a redeploy
ACTUAL -
Garbage collection is prevented by strong reference path from GC root. (Dump and analyze heap to verify.)

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Error message for parsing the certificate:

Unparseable certificate extensions: 1
[1]: ObjectId: 2.5.29.18 Criticality=false
Unparseable IssuerAlternativeName extension due to
java.io.IOException: No data available in passed DER encoded value.


GC root path available in image here: http://stackoverflow.com/questions/39741147/classloader-leak-because-of-invalid-ssl-certificate-in-solrj-httpclient-jvm-or

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
System.setProperty("javax.net.ssl.trustStore", "/path/to/downloaded/https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/test/resources/spi-cacert-2008.keystore");

javax.net.ssl.SSLContext.getDefault();

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
The ClassLoader Leak Prevention library provides a workaround by unsetting the "why" field using reflection. See https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/main/java/se/jiderhamn/classloader/leak/prevention/cleanup/X509TrustManagerImplUnparseableExtensionCleanUp.java


                                    

Comments
Please take care of two factors in the description:
1. the static field sun.security.ssl.SSLContextImpl$DefaultSSLContext.defaultImpl
2. sun.security.x509.UnparseableExtension has a Throwable with a backtrace containing strong references to classes.

It's a result to cache the trust certificates for performance.   Will consider a memory friendly enhancement in JDK 9.1.
                                     
2017-01-05
Attached the project and the profiler trace showing the issue.
                                     
2017-01-02
From submitter:
--------------------
I'm assuming you're running "CptS":s project at https://github.com/CptS/solrj-classloader-leak (rather than my project at https://github.com/mjiderhamn/classloader-leak-prevention/ ). I don't think I actually ran that myself, so I cannot answer for it, but I know the Solr server must be responding for the leak to be triggered by that code.

The simplified steps detailed in the bug report, or my test case at https://github.com/mjiderhamn/classloader-leak-prevention/blob/master/classloader-leak-prevention/classloader-leak-prevention-core/src/test/java/se/jiderhamn/classloader/leak/prevention/cleanup/X509TrustManagerImplUnparseableExtensionCleanUpTest.java , should allow you to reproduce the issue however. If it helps, I can also provide a heap dump with the leak in it. (Possibly I could even write a stand alone application that fills metaspace with leaked classloaders). 


                                     
2016-12-21
Mail to submitter:
I am unable to reproduce the issue with the github project using the cacerts provided in the project. Here is the screenshot of the metaspace usage . After un-deploying the project, there isn???t any instances left as seen in heap dump.
                                     
2016-11-22



Hardware and Software, Engineered to Work Together