FULL PRODUCT VERSION :
Java HotSpot 64bit 14.2-b01, 1.6.0_16
ADDITIONAL OS VERSION INFORMATION :
OS: Linux hn743c2n2 2.6.9-68.9.ELmsdw.2smp #1 SMP Fri Jan 16 14:43:40 EST 2009 x86_64 (AURORA 4.0.1-x86_64)
A DESCRIPTION OF THE PROBLEM :
Below are the threads stacks of the deadlocked threads. This portion of the stack for the gsf-thread-1 MacLoad thread,
at com.msdw.dpg.etktools.db.replication.ReplicationSync.getReplicatedDsn
initializes some classes from mailapi.jar, which is signed and appears to cause the deadlock.
Deadlock log:
Found one Java-level deadlock:
=============================
"pool-4-thread-1":
waiting to lock monitor 0x0000002c29a2cff8 (object 0x0000002aa9a95cd8, a sun.misc.Launcher$AppClassLoader),
which is held by "gsf-thread-1 MacLoad"
"gsf-thread-1 MacLoad":
waiting to lock monitor 0x0000002c22402cc8 (object 0x0000002a9e0190c0, a java.lang.Class),
which is held by "pool-4-thread-1"
Java stack information for the threads listed above:
===================================================
"pool-4-thread-1":
at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:188)
- waiting to lock <0x0000002aa9a95cd8> (a sun.misc.Launcher$AppClassLoader)
at sun.security.jca.ProviderList.loadAll(ProviderList.java:264)
at sun.security.jca.ProviderList.removeInvalid(ProviderList.java:281)
at sun.security.jca.Providers.getFullProviderList(Providers.java:158)
- locked <0x0000002a9e0190c0> (a java.lang.Class for sun.security.jca.Providers)
at java.security.Security.getProviders(Security.java:421)
at sun.security.jgss.ProviderList.<init>(ProviderList.java:116)
at sun.security.jgss.GSSManagerImpl.<init>(GSSManagerImpl.java:65)
at org.ietf.jgss.GSSManager.getInstance(GSSManager.java:130)
at com.sybase.jdbc3.tds.KerberosSessionContext.a(Unknown Source)
at com.sybase.jdbc3.tds.KerberosSessionContext.<init>(Unknown Source)
at com.sybase.jdbc3.tds.Tds.login(Unknown Source)
at com.sybase.jdbc3.jdbc.SybConnection.a(Unknown Source)
at com.sybase.jdbc3.jdbc.SybConnection.handleHAFailover(Unknown Source)
at com.sybase.jdbc3.jdbc.SybConnection.<init>(Unknown Source)
at com.sybase.jdbc3.jdbc.SybConnection.<init>(Unknown Source)
at com.sybase.jdbc3.jdbc.SybDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:154)
[snip propriatery code]
- locked <0x0000002ab3677b70> (a com.msdw.dpg.etktools.cache.CacheKey)
[snip propriatery code]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
"gsf-thread-1 MacLoad":
at sun.security.jca.Providers.beginThreadProviderList(Providers.java:207)
- waiting to lock <0x0000002a9e0190c0> (a java.lang.Class for sun.security.jca.Providers)
at sun.security.jca.Providers.startJarVerification(Providers.java:86)
at sun.security.util.SignatureFileVerifier.<init>(SignatureFileVerifier.java:79)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:256)
at java.util.jar.JarVerifier.update(JarVerifier.java:188)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:321)
at java.util.jar.JarFile.getInputStream(JarFile.java:386)
- locked <0x0000002aa9b63320> (a java.util.jar.JarFile)
at sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:689)
at sun.misc.Resource.cachedInputStream(Resource.java:59)
- locked <0x0000002ab367a010> (a sun.misc.URLClassPath$JarLoader$2)
at sun.misc.Resource.getByteBuffer(Resource.java:154)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:249)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
- locked <0x0000002aa9a95cd8> (a sun.misc.Launcher$AppClassLoader)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
- locked <0x0000002aa9a95cd8> (a sun.misc.Launcher$AppClassLoader)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
- locked <0x0000002aa9a95cd8> (a sun.misc.Launcher$AppClassLoader)
[snip propriatery code]
- locked <0x0000002ab367a0d0> (a java.util.HashMap)
[snip propriatery code]
- locked <0x0000002ab367abd8> (a java.lang.Object)
[snip propriatery code]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
[snip propriatery code]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Found 1 deadlock.
Possible related bugs:
- http://bugs.sun.com/view_bug.do?bug_id=6850402
- http://bugs.sun.com/view_bug.do?bug_id=6440846
- http://bugs.sun.com/view_bug.do?bug_id=4944382
- http://www-01.ibm.com/support/docview.wss?uid=swg1IZ22372
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
We do not have a step by step reproduction, but the deadlock stack should give a pretty good idea of the issue.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
no deadlock
ACTUAL -
deadlock
ERROR MESSAGES/STACK TRACES THAT OCCUR :
Found one Java-level deadlock:
=============================
"pool-4-thread-1":
waiting to lock monitor 0x0000002c29a2cff8 (object 0x0000002aa9a95cd8, a sun.misc.Launcher$AppClassLoader),
which is held by "gsf-thread-1 MacLoad"
"gsf-thread-1 MacLoad":
waiting to lock monitor 0x0000002c22402cc8 (object 0x0000002a9e0190c0, a java.lang.Class),
which is held by "pool-4-thread-1"
Java stack information for the threads listed above:
===================================================
"pool-4-thread-1":
at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:188)
- waiting to lock <0x0000002aa9a95cd8> (a sun.misc.Launcher$AppClassLoader)
at sun.security.jca.ProviderList.loadAll(ProviderList.java:264)
at sun.security.jca.ProviderList.removeInvalid(ProviderList.java:281)
at sun.security.jca.Providers.getFullProviderList(Providers.java:158)
- locked <0x0000002a9e0190c0> (a java.lang.Class for sun.security.jca.Providers)
at java.security.Security.getProviders(Security.java:421)
at sun.security.jgss.ProviderList.<init>(ProviderList.java:116)
at sun.security.jgss.GSSManagerImpl.<init>(GSSManagerImpl.java:65)
at org.ietf.jgss.GSSManager.getInstance(GSSManager.java:130)
at com.sybase.jdbc3.tds.KerberosSessionContext.a(Unknown Source)
at com.sybase.jdbc3.tds.KerberosSessionContext.<init>(Unknown Source)
at com.sybase.jdbc3.tds.Tds.login(Unknown Source)
at com.sybase.jdbc3.jdbc.SybConnection.a(Unknown Source)
at com.sybase.jdbc3.jdbc.SybConnection.handleHAFailover(Unknown Source)
at com.sybase.jdbc3.jdbc.SybConnection.<init>(Unknown Source)
at com.sybase.jdbc3.jdbc.SybConnection.<init>(Unknown Source)
at com.sybase.jdbc3.jdbc.SybDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:154)
[snip propriatery code]
- locked <0x0000002ab3677b70> (a com.msdw.dpg.etktools.cache.CacheKey)
[snip propriatery code]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
"gsf-thread-1 MacLoad":
at sun.security.jca.Providers.beginThreadProviderList(Providers.java:207)
- waiting to lock <0x0000002a9e0190c0> (a java.lang.Class for sun.security.jca.Providers)
at sun.security.jca.Providers.startJarVerification(Providers.java:86)
at sun.security.util.SignatureFileVerifier.<init>(SignatureFileVerifier.java:79)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:256)
at java.util.jar.JarVerifier.update(JarVerifier.java:188)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:321)
at java.util.jar.JarFile.getInputStream(JarFile.java:386)
- locked <0x0000002aa9b63320> (a java.util.jar.JarFile)
at sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:689)
at sun.misc.Resource.cachedInputStream(Resource.java:59)
- locked <0x0000002ab367a010> (a sun.misc.URLClassPath$JarLoader$2)
at sun.misc.Resource.getByteBuffer(Resource.java:154)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:249)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
- locked <0x0000002aa9a95cd8> (a sun.misc.Launcher$AppClassLoader)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
- locked <0x0000002aa9a95cd8> (a sun.misc.Launcher$AppClassLoader)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
- locked <0x0000002aa9a95cd8> (a sun.misc.Launcher$AppClassLoader)
[snip propriatery code]
- locked <0x0000002ab367a0d0> (a java.util.HashMap)
[snip propriatery code]
- locked <0x0000002ab367abd8> (a java.lang.Object)
[snip propriatery code]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
[snip propriatery code]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Found 1 deadlock.
REPRODUCIBILITY :
This bug can be reproduced occasionally.
---------- BEGIN SOURCE ----------
none.
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
workaround was to delay loading the classes from that signed jar.