Bug ID: JDK-6965671 fatal error: acquiring lock JNIGlobalHandle_lock/16 out of order with lock CodeCache

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 6	JDK 7	Other
6u21pFixed	7Fixed	hs19Fixed

Bit of log (with classpath and command line, but it fails both client/server and on all platforms).

[2010-06-30T02:56:27.73] # Actual: CLASSPATH=/export/local/36309.JDK7.NIGHTLY.VM+linux-amd64_server_mixed_nsk.jvmti.testlist_129869D6F22/results/ResultDir/genevents001:/export/local/common/testbase/6/vm/vm/bin/classes:/export/local/common/jdk/baseline/linux-amd64/lib/tools.jar[2010-06-30T02:56:27.73] ${JAVA} ${JAVA_OPTS} ${EXECUTE_CLASS} ${TEST_ARGS}
[2010-06-30T02:56:27.73] # Actual: /export/local/common/jdk/baseline/linux-amd64/bin/java -d64 -server -Xmixed -Dsun.jvm.hotspot.runtime.VM.disableVersionCheck=1 -XX:-UseCompressedOops -XX:DefaultMaxRAMFraction=8 -XX:-PrintVMOptions -XX:+StartAttachListener -agentlib:genevents001=-waittime=5 nsk.jvmti.GenerateEvents.genevents001
[2010-06-30T02:56:27.73]  Locks owned:
[2010-06-30T02:56:29.12] Mutex: [0xad77bd8/0x1] CodeCache_lock - owner: 0x2aaac8045000
[2010-06-30T02:56:29.12] # To suppress the following error report, specify this argument
[2010-06-30T02:56:29.12] # after -XX: or in .hotspotrc:  SuppressErrorAt=/mutex.cpp:1293
[2010-06-30T02:56:29.12] #
[2010-06-30T02:56:29.12] # A fatal error has been detected by the Java Runtime Environment:
[2010-06-30T02:56:29.12] #
[2010-06-30T02:56:29.12] #  Internal Error (/tmp/jprt/P1/B/160437.coleenp/source/src/share/vm/runtime/mutex.cpp:1293), pid=29708, tid=1108306240
[2010-06-30T02:56:29.12] #  fatal error: acquiring lock JNIGlobalHandle_lock/16 out of order with lock CodeCache_lock/1 -- possible deadlock
[2010-06-30T02:56:29.12] #
[2010-06-30T02:56:29.12] # JRE version: 7.0-b99
[2010-06-30T02:56:29.12] # Java VM: OpenJDK 64-Bit Server VM (19.0-b02-201006281604.coleenp.rt-merge-fastdebug mixed mode linux-amd64 )
[2010-06-30T02:56:29.12] # An error report file with more information is saved as:
[2010-06-30T02:56:29.12] # /export/local/36309.JDK7.NIGHTLY.VM+linux-amd64_server_mixed_nsk.jvmti.testlist_129869D6F22/results/ResultDir/genevents001/hs_err_pid29708.log
nsk/jvmti/GenerateEvents/genevents001
  nsk/jvmti/scenarios/sampling/SP06/sp06t001
  nsk/jvmti/scenarios/sampling/SP06/sp06t002
  nsk/jvmti/scenarios/sampling/SP06/sp06t003

EVALUATION http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/65b0c03b165d

16-07-2010

EVALUATION 6965671: fatal error: acquiring lock JNIGlobalHandle_lock/16 out of order with lock CodeCache_lock/1 Reviewed-by: kvn, dcubed The jmethodID fix created a deadlock because of lock ordering when walking the CodeCache to generate CompiledMethodLoad events. The walk is performed with the CodeCache_lock held but we need to acquire the JNIGlobalHandle_lock to make a jmethodID. I considered switching back to collecting the methodOop instead of the jmethodID under the lock but inspection of the code made it clear that there's a preexisting problem with GC of the methodOops stored into the nmethodDesc. Instead I switched to a simpler implementation that holds the CodeCache_lock while walking the CodeCache but releases the lock to actually perform the notify. This simplifies the code greatly as well as fixing the bug. Tested with NSK jvmti,jdi,jdb,hprof,jit,regression and JDI_REGRESSION tests.

07-07-2010

EVALUATION http://hg.openjdk.java.net/jdk7/hotspot-comp/hotspot/rev/65b0c03b165d

03-07-2010

Relates :	JDK-6656830 - assert((*p)->is_oop(),"expected an oop while scanning weak refs")
Relates :	JDK-6968336 - VM crash guarantee(!nm->is_zombie()) failed: cannot lock a zombie method