JDK-8277213 : CompileTask_lock is acquired out of order with MethodCompileQueue_lock
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 18
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2021-11-16
  • Updated: 2021-11-25
  • Resolved: 2021-11-19
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 18
18 b25Fixed
Related Reports
Relates :  
JDK-8277042 - add test for 8276036 to compiler/codecache
Relates :  
JDK-8273917 - Remove 'leaf' ranking for Mutex
Description
The test taken from the currently open PR of JDK-8277042 (https://github.com/openjdk/jdk/pull/6364) fails with the following assertion.
 
To reproduce:
1. Run CodeCacheFullCountTest.java normally with JTreg
2. cd JTwork/scratch
3. Run the command line of the child VM:
java -cp [...] -XX:ReservedCodeCacheSize=2496k -XX:-UseCodeCacheFlushing CodeCacheFullCountTest WasteCodeCache

This fails intermittently (1 out of 20) on my local machine but is reliably triggered in tier testing

[2.709s][warning][codecache] CodeCache is full. Compiler has been disabled.
[2.709s][warning][codecache] Try increasing the code cache size using -XX:ReservedCodeCacheSize=
Java HotSpot(TM) 64-Bit Server VM warning: CodeCache is full. Compiler has been disabled.
Java HotSpot(TM) 64-Bit Server VM warning: Try increasing the code cache size using -XX:ReservedCodeCacheSize=
CodeCache: size=2496Kb used=2484Kb max_used=2488Kb free=11Kb
 bounds [0x00007f9153a98000, 0x00007f9153d08000, 0x00007f9153d08000]
 total_blobs=1753 nmethods=1014 adapters=661
 compilation: disabled (not enough contiguous free space left)
              stopped_count=0, restarted_count=0
 full_count=1
Java HotSpot(TM) 64-Bit Server VM warning: C2 initialization failed. Shutting down all compilers
 Locks owned:
Mutex: [0x00007f914c0240f0] MethodCompileQueue_lock - owner: 0x00007f914c20fce0 safepoint
# To suppress the following error report, specify this argument
# after -XX: or in .hotspotrc:  SuppressErrorAt=/mutex.cpp:444
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/opt/mach5/mesos/work_dir/slaves/ff806ead-2cac-495d-9cbc-62116f99bf14-S13706/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/42974813-fa1f-4628-8b00-f1c20979f765/runs/338c8ac4-37b2-4196-9ab6-b7102b51007b/workspace/open/src/hotspot/share/runtime/mutex.cpp:444), pid=146467, tid=146480
#  assert(false) failed: Attempting to acquire lock CompileTask_lock/safepoint out of order with lock MethodCompileQueue_lock/safepoint -- possible deadlock
#
# JRE version: Java(TM) SE Runtime Environment (18.0+24) (fastdebug build 18-ea+24-1591)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 18-ea+24-1591, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0x15641fd]  Mutex::check_rank(Thread*)+0x29d
.............
Command Line: -XX:ReservedCodeCacheSize=2496k -XX:-UseCodeCacheFlushing CodeCacheFullCountTest WasteCodeCache
.............
Current thread (0x00007f914c20fce0):  JavaThread "C2 CompilerThread0" daemon [_thread_in_vm, id=146480, stack(0x00007f913c80c000,0x00007f913c90d000)]

Stack: [0x00007f913c80c000,0x00007f913c90d000],  sp=0x00007f913c90bbc0,  free space=1022k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x15641fd]  Mutex::check_rank(Thread*)+0x29d
V  [libjvm.so+0x15649f1]  Mutex::lock(Thread*)+0x31
V  [libjvm.so+0xa88dc8]  CompileQueue::free_all()+0x48
V  [libjvm.so+0xa8c25c]  CompileBroker::shutdown_compiler_runtime(AbstractCompiler*, CompilerThread*)+0xac
V  [libjvm.so+0xa9569f]  CompileBroker::compiler_thread_loop()+0x50f
V  [libjvm.so+0x191456a]  JavaThread::thread_main_inner()+0x25a
V  [libjvm.so+0x191c900]  Thread::call_run()+0x100
V  [libjvm.so+0x15fe854]  thread_native_entry(Thread*)+0x104
Comments
Changeset: f34f1190 Author: Tobias Hartmann <thartmann@openjdk.org> Date: 2021-11-19 07:13:05 +0000 URL: https://git.openjdk.java.net/jdk/commit/f34f119080b4e8baf396fb26c21d572dd432fd91
19-11-2021
Verified that it's a regression in JDK 18 b18. This can only happen if compilation is disabled forever (i.e., if compiler initialization failed or if the code cache is full and sweeping is disabled).
17-11-2021
I think this might be a regression from JDK-8176393/JDK-8273917. I'll have a look.
17-11-2021
ILW = Assertion due to possible deadlock, rare, possibly use -XX:+UseCodeCacheFlushing and use high enough ReservedCodeCacheSize = HLL = P4
16-11-2021