JDK-8058547 : Memory leak in ProtectionDomain cache
  • Type: Bug
  • Component: security-libs
  • Sub-Component: java.security
  • Affected Version: 7,9
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2012-07-23
  • Updated: 2017-12-14
  • Resolved: 2014-10-20
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8 JDK 9
8u60Fixed 9 b37Fixed
Related Reports
Relates :  
Relates :  
Relates :  
Relates :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.7.0_04"
Java(TM) SE Runtime Environment (build 1.7.0_04-b20)
Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Linux 2.6.35.14-106.fc14.x86_64 #1 SMP Wed Nov 23 13:07:52 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux


A DESCRIPTION OF THE PROBLEM :
This memory leak is caused by the incorrect use of a WeakHashMap stored in the field sun.security.provider.PolicyFile$PolicyInfo.pdMapping. This WeakHashMap is used as an internal cache. The problem is that under certain circumstances there can be a strong reference chain from a value of a map entry to the associated key. This strong reference chain keeps this entry reachable forever.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
* Compile the four classes attached to this ticket.
* In the current directory, move the class AppManager into a subdirectory 'manager'.
* In the current directory, move the class App into a subdirectory 'app'.
* In the current directory, move the classes SomePermission and PerAppLibrary into a subdirectory 'per-app-lib'.
* Copy the attached file 'system.policy' to the current directory.
* Adapt the absolute code source paths in system.policy so that they point to the directories 'manager', 'app' and 'per-app-lib'.
* In the current directory: 'java -cp manager -Djava.security.policy=system.policy AppManager'
* After waiting for a while, you will see an 'OutOfMemoryError'.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The free memory values that are printed to the console should remain stable.
ACTUAL -
The free memory values that are printed to the console decrease, eventually resulting in an OutOfMemoryError.

ERROR MESSAGES/STACK TRACES THAT OCCUR :

Running app in java.net.URLClassLoader@60ec2ea8
Free memory: 53MB
Running app in java.net.URLClassLoader@5bf825cc
Free memory: 49MB
Running app in java.net.URLClassLoader@6094cbe2
Free memory: 45MB
Running app in java.net.URLClassLoader@1d8806f7
Free memory: 41MB
Running app in java.net.URLClassLoader@5e34d46a
Free memory: 37MB
Running app in java.net.URLClassLoader@576f8789
Free memory: 33MB
Running app in java.net.URLClassLoader@2b54ae5a
Free memory: 29MB
Running app in java.net.URLClassLoader@48129552
Free memory: 25MB
Running app in java.net.URLClassLoader@40128cad
Free memory: 21MB
Running app in java.net.URLClassLoader@7e6bc5aa
Free memory: 17MB
Running app in java.net.URLClassLoader@2994363b
Free memory: 13MB
Running app in java.net.URLClassLoader@7ffe9999
Free memory: 9MB
Running app in java.net.URLClassLoader@76a14c53
Free memory: 5MB
Running app in java.net.URLClassLoader@54b749a5
Free memory: 36MB
Running app in java.net.URLClassLoader@10c042ab
Free memory: 32MB
Running app in java.net.URLClassLoader@1704ebb
Free memory: 28MB
Running app in java.net.URLClassLoader@2073b879
Free memory: 24MB
Running app in java.net.URLClassLoader@1c898b41
Free memory: 20MB
Running app in java.net.URLClassLoader@20e1ed5b
Free memory: 16MB
Running app in java.net.URLClassLoader@c3b5587
Free memory: 12MB
Running app in java.net.URLClassLoader@29af45f4
Free memory: 8MB
Running app in java.net.URLClassLoader@11a40fff
Free memory: 8MB
Running app in java.net.URLClassLoader@21e8bf76
Free memory: 8MB
Running app in java.net.URLClassLoader@131b92e6
Free memory: 8MB
Running app in java.net.URLClassLoader@44cae5b8
Free memory: 8MB
Running app in java.net.URLClassLoader@5faecf45
Free memory: 8MB
Running app in java.net.URLClassLoader@504c2683
Free memory: 8MB
Running app in java.net.URLClassLoader@45e4d960
Free memory: 8MB
Running app in java.net.URLClassLoader@4706e02e
Free memory: 8MB
Running app in java.net.URLClassLoader@49deadf0
Free memory: 8MB
Running app in java.net.URLClassLoader@8e3cf25
Free memory: 8MB
Running app in java.net.URLClassLoader@1471cb25
Free memory: 8MB
Running app in java.net.URLClassLoader@e1641c0
Free memory: 8MB
Running app in java.net.URLClassLoader@38b72ce1
Free memory: 9MB
Running app in java.net.URLClassLoader@1817fe89
Free memory: 9MB
Running app in java.net.URLClassLoader@6a998c1
Free memory: 9MB
Running app in java.net.URLClassLoader@3b280492
Free memory: 9MB
Running app in java.net.URLClassLoader@5f154718
Free memory: 9MB
Running app in java.net.URLClassLoader@16fdf48d
Free memory: 9MB
Running app in java.net.URLClassLoader@2d0651eb
Free memory: 9MB
Running app in java.net.URLClassLoader@5cfe174
Free memory: 9MB
Running app in java.net.URLClassLoader@3be4d6ef
Free memory: 9MB
Running app in java.net.URLClassLoader@2f0f94a0
Free memory: 9MB
Running app in java.net.URLClassLoader@10c1c428
Free memory: 9MB
Running app in java.net.URLClassLoader@634e3372
Free memory: 5MB
Running app in java.net.URLClassLoader@3d949a87
Free memory: 5MB
Running app in java.net.URLClassLoader@183d1102
Free memory: 5MB
Running app in java.net.URLClassLoader@6f77e5d4
Free memory: 5MB
Running app in java.net.URLClassLoader@16a7bdcd
Free memory: 5MB
Running app in java.net.URLClassLoader@15dcc3ca
Free memory: 5MB
Running app in java.net.URLClassLoader@275cea3
Free memory: 5MB
Running app in java.net.URLClassLoader@3c626028
Free memory: 5MB
Running app in java.net.URLClassLoader@7bc2f501
Free memory: 5MB
Running app in java.net.URLClassLoader@79f5910e
Free memory: 5MB
Running app in java.net.URLClassLoader@6c10a234
Free memory: 6MB
Running app in java.net.URLClassLoader@3ddb8962
Free memory: 6MB
Running app in java.net.URLClassLoader@124ef122
Free memory: 6MB
Running app in java.net.URLClassLoader@1ca53e68
Free memory: 6MB
Running app in java.net.URLClassLoader@15e1df6e
Free memory: 6MB
Running app in java.net.URLClassLoader@4a6a00ca
Free memory: 6MB
Running app in java.net.URLClassLoader@526a4268
Free memory: 6MB
Running app in java.net.URLClassLoader@10be6858
Free memory: 6MB
Running app in java.net.URLClassLoader@7de534cb
Free memory: 6MB
Running app in java.net.URLClassLoader@2be44538
Free memory: 6MB
Running app in java.net.URLClassLoader@76c27ba
Free memory: 6MB
Running app in java.net.URLClassLoader@5c3eeab3
Free memory: 6MB
Running app in java.net.URLClassLoader@f6b7e0e
Free memory: 6MB
Running app in java.net.URLClassLoader@15136019
Free memory: 6MB
Running app in java.net.URLClassLoader@3ec2ccac
Free memory: 6MB
Running app in java.net.URLClassLoader@6a754384
Free memory: 6MB
Running app in java.net.URLClassLoader@1ab2f2d6
Free memory: 6MB
Running app in java.net.URLClassLoader@4eafccbe
Free memory: 6MB
Running app in java.net.URLClassLoader@7000bcbc
Free memory: 6MB
Running app in java.net.URLClassLoader@d6132c4
Free memory: 7MB
Running app in java.net.URLClassLoader@3918d722
Free memory: 7MB
Running app in java.net.URLClassLoader@490eb6ae
Free memory: 7MB
Running app in java.net.URLClassLoader@5f2e712f
Free memory: 7MB
Running app in java.net.URLClassLoader@6a3449a8
Free memory: 7MB
Running app in java.net.URLClassLoader@7d206f0
Free memory: 7MB
Running app in java.net.URLClassLoader@7ea06d25
Free memory: 7MB
Running app in java.net.URLClassLoader@2d11f5f1
Free memory: 7MB
Running app in java.net.URLClassLoader@203b4f0e
Free memory: 7MB
Running app in java.net.URLClassLoader@120773a1
Free memory: 7MB
Running app in java.net.URLClassLoader@7a4014a0
Free memory: 7MB
Running app in java.net.URLClassLoader@4e5ced83
Free memory: 7MB
Running app in java.net.URLClassLoader@3c818737
Free memory: 68MB
Running app in java.net.URLClassLoader@1a08777c
Free memory: 64MB
Running app in java.net.URLClassLoader@3e2f1b1a
Free memory: 60MB
Running app in java.net.URLClassLoader@c39a867
Free memory: 56MB
Running app in java.net.URLClassLoader@6e905272
Free memory: 52MB
Running app in java.net.URLClassLoader@1d209d56
Free memory: 48MB
Running app in java.net.URLClassLoader@6cd737e3
Free memory: 44MB
Running app in java.net.URLClassLoader@17feafba
Free memory: 40MB
Running app in java.net.URLClassLoader@63c5d81c
Free memory: 36MB
Running app in java.net.URLClassLoader@6facdcb9
Free memory: 32MB
Running app in java.net.URLClassLoader@6c3b0b1e
Free memory: 28MB
Running app in java.net.URLClassLoader@705063a5
Free memory: 24MB
Running app in java.net.URLClassLoader@7b6b340a
Free memory: 20MB
Running app in java.net.URLClassLoader@75ecda50
Free memory: 16MB
Running app in java.net.URLClassLoader@431d00cf
Free memory: 12MB
Running app in java.net.URLClassLoader@5fd1358f
Free memory: 8MB
Running app in java.net.URLClassLoader@1e69dff6
Free memory: 8MB
Running app in java.net.URLClassLoader@759f6a57
Free memory: 8MB
Running app in java.net.URLClassLoader@78952527
Free memory: 8MB
Running app in java.net.URLClassLoader@20c92ed6
Free memory: 8MB
Running app in java.net.URLClassLoader@79f1d448
Free memory: 8MB
Running app in java.net.URLClassLoader@5371ef98
Free memory: 8MB
Running app in java.net.URLClassLoader@77c5b2de
Free memory: 9MB
Running app in java.net.URLClassLoader@2f566b7d
Free memory: 9MB
Running app in java.net.URLClassLoader@4679cf8c
Free memory: 9MB
Running app in java.net.URLClassLoader@8bfc25c
Free memory: 9MB
Running app in java.net.URLClassLoader@3654919e
Free memory: 9MB
Running app in java.net.URLClassLoader@5c3f1224
Free memory: 9MB
Running app in java.net.URLClassLoader@4f0ab3f2
Free memory: 9MB
Running app in java.net.URLClassLoader@773de2bd
Free memory: 9MB
Running app in java.net.URLClassLoader@1a3a2a52
Free memory: 9MB
Running app in java.net.URLClassLoader@a5ae1e7
Free memory: 9MB
Running app in java.net.URLClassLoader@74904497
Free memory: 9MB
Running app in java.net.URLClassLoader@22dd0f87
Free memory: 9MB
Running app in java.net.URLClassLoader@48899e6a
Free memory: 9MB
Running app in java.net.URLClassLoader@433c8540
Free memory: 5MB
Running app in java.net.URLClassLoader@28562791
Free memory: 5MB
Running app in java.net.URLClassLoader@4678f83a
Free memory: 5MB
Running app in java.net.URLClassLoader@3eb217d5
Free memory: 5MB
Running app in java.net.URLClassLoader@24748417
Free memory: 5MB
Running app in java.net.URLClassLoader@2830803a
Free memory: 5MB
Running app in java.net.URLClassLoader@57543bc5
Free memory: 5MB
Running app in java.net.URLClassLoader@759f31de
Free memory: 5MB
Running app in java.net.URLClassLoader@2f57816a
Free memory: 6MB
Running app in java.net.URLClassLoader@19cd1d94
Free memory: 6MB
Running app in java.net.URLClassLoader@326cbb2d
Free memory: 6MB
Running app in java.net.URLClassLoader@108543aa
Free memory: 6MB
Running app in java.net.URLClassLoader@39df3255
Free memory: 6MB
Running app in java.net.URLClassLoader@10e80317
Free memory: 6MB
Running app in java.net.URLClassLoader@7d0d4ade
Free memory: 6MB
Running app in java.net.URLClassLoader@6c4cc81c
Free memory: 6MB
Running app in java.net.URLClassLoader@6162c87a
Free memory: 6MB
Running app in java.net.URLClassLoader@2c4dd413
Free memory: 6MB
Running app in java.net.URLClassLoader@306f7492
Free memory: 6MB
Running app in java.net.URLClassLoader@770f3801
Free memory: 6MB
Running app in java.net.URLClassLoader@441743be
Free memory: 6MB
Running app in java.net.URLClassLoader@7ba65cf7
Free memory: 6MB
Running app in java.net.URLClassLoader@740f5f97
Free memory: 6MB
Running app in java.net.URLClassLoader@77be91c8
Free memory: 6MB
Running app in java.net.URLClassLoader@35d3694a
Free memory: 6MB
Running app in java.net.URLClassLoader@6326c98
Free memory: 6MB
Running app in java.net.URLClassLoader@99ffac2
Free memory: 6MB
Running app in java.net.URLClassLoader@2e9e53ed
Free memory: 6MB
Running app in java.net.URLClassLoader@22b1656f
Free memory: 7MB
Running app in java.net.URLClassLoader@4ffb06c9
Free memory: 7MB
Running app in java.net.URLClassLoader@32114a41
Free memory: 7MB
Running app in java.net.URLClassLoader@2ce62a39
Free memory: 7MB
Running app in java.net.URLClassLoader@2d51e135
Free memory: 7MB
Running app in java.net.URLClassLoader@39556aec
Free memory: 7MB
Running app in java.net.URLClassLoader@778a5d49
Free memory: 7MB
Running app in java.net.URLClassLoader@13d5e499
Free memory: 7MB
Running app in java.net.URLClassLoader@66a75004
Free memory: 7MB
Running app in java.net.URLClassLoader@6d15a113
Free memory: 7MB
Running app in java.net.URLClassLoader@165e6c89
Free memory: 7MB
Running app in java.net.URLClassLoader@4be07f4b
Free memory: 7MB
Running app in java.net.URLClassLoader@47b6617
Free memory: 7MB
Running app in java.net.URLClassLoader@78ea5d87
Free memory: 7MB
Running app in java.net.URLClassLoader@bab5e85
Free memory: 7MB
Running app in java.net.URLClassLoader@5d3f79f7
Free memory: 7MB
Running app in java.net.URLClassLoader@138532dc
Free memory: 6MB
Running app in java.net.URLClassLoader@fe9e47
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at AppManager.runAppInNewClassLoader(AppManager.java:50)
	at AppManager.main(AppManager.java:27)
Caused by: java.lang.OutOfMemoryError: Java heap space
	at PerAppLibrary.<clinit>(PerAppLibrary.java:3)
	at App.run(App.java:7)
	... 6 more


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
public class App
{  
  public void run()
  {
    System.out.println("Running app in " + getClass().getClassLoader());
    PerAppLibrary.doSomeSomething();
  }
}

***********************************************************************************

import java.io.File;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;


public class AppManager
{

  public static final String APP_DIR = "app";
  public static final String LIB_DIR = "per-app-lib";

  public static void main(String[] args) throws MalformedURLException,
                                                ClassNotFoundException,
                                                InstantiationException,
                                                IllegalAccessException,
                                                SecurityException,
                                                NoSuchMethodException,
                                                IllegalArgumentException,
                                                InvocationTargetException
  {
    System.setSecurityManager(new SecurityManager());
    while (true)
      {
        runAppInNewClassLoader();
        printFreeMemory();
      }
  }


  // Helpers

  private static void runAppInNewClassLoader() throws MalformedURLException,
                                                      ClassNotFoundException,
                                                      InstantiationException,
                                                      IllegalAccessException,
                                                      SecurityException,
                                                      NoSuchMethodException,
                                                      IllegalArgumentException,
                                                      InvocationTargetException
  {
    URL[] urls = { new File(LIB_DIR).toURI().toURL(),
                   new File(APP_DIR).toURI().toURL() };
    ClassLoader loader = new URLClassLoader(urls);
    Class<?> clazz = loader.loadClass("App");
    Object app = clazz.newInstance();
    Method run = clazz.getDeclaredMethod("run");
    run.invoke(app);
  }


  private static void printFreeMemory()
  {
    Runtime rt = Runtime.getRuntime();
    rt.gc();
    System.out.println("Free memory: " + rt.freeMemory()/ (1024 * 1024) + "MB");
  }

}

*********************************************************************************************************

public class PerAppLibrary
{
  private  static int[] largeArray = new int[1024 * 1024];

  public static void doSomeSomething()
  {
    SecurityManager s = System.getSecurityManager();
    if (s != null)
      {
        s.checkPermissio


( This report has more than 16,000 characters and has been truncated. )

Comments
Fix failed and had to be backed out in another changeset. For JDK 9, it was backed out as part of JDK-8055753. For 8u60, it was backed out in JDK-8077734.
15-01-2016

The memory leak is in the WeakHashMap in ProtectionDomain which is a cache that maps each ProtectionDomain to its PermissionCollection. Some PermissionCollections such as PropertyPermissionCollection can have strong references leading back to a ProtectionDomain, thereby preventing entries from being removed. After wrapping the PermissionCollection entries in a WeakReference, the memory leak was fixed.
22-09-2014

system.policy was missing from the test case (got truncated). However, I was able to re-create it as follows: grant codeBase "file:${user.home}/tests/8058547/manager/" { permission java.util.PropertyPermission "user.dir", "read"; permission java.io.FilePermission "${user.home}/tests/8058547/-", "read"; permission java.io.FilePermission "${user.home}/tests/8058547/per-app-lib/-", "read"; permission java.io.FilePermission "${user.home}/tests/8058547/app/*", "read"; permission java.lang.RuntimePermission "createClassLoader"; permission java.lang.RuntimePermission "accessDeclaredMembers"; permission SomePermission "foo"; }; grant codeBase "file:${user.home}/tests/8058547/per-app-lib/" { permission SomePermission "foo"; }; grant codeBase "file:${user.home}/tests/8058547/app/" { permission SomePermission "foo"; }; I also eventually got an OutofMemoryError, it took longer and there was clearly some memory being released as the free memory went back up a few times, but I eventually got OutofMemoryError. I need to look into this some more, but it sounds like there is an overall leak somewhere...
19-09-2014

Hoping Dev can help evaluate. Unfortunately the testcase src got truncated. The same WeakHashMap seems to be in use for JDK 9.
16-09-2014