JDK-8230943 : False deadlock detection with -XX:+CIPrintCompileQueue after JDK-8163511
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 11,13,14
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2019-09-13
  • Updated: 2022-06-27
  • Resolved: 2019-09-17
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 13 JDK 14
11.0.6Fixed 13.0.2Fixed 14 b15Fixed
Related Reports
Relates :  
JDK-8214315 - G1: fatal error: acquiring lock SATB_Q_FL_lock/1 out of order with lock tty_lock/0
Relates :  
JDK-8163511 - Allocation of compile task fails with assert: "Leaking compilation tasks?"
Relates :  
JDK-8231269 - JNIHandles::is_weak_global_handle is not cheap
Description
## Symptom
---------------------------------------------------
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error 
(/home/fool/workspace/jdk-dev/src/hotspot/share/runtime/mutex.cpp:399), 
pid=3842, tid=3846
#  fatal error: acquiring lock JNI weak alloc lock/7 out of order with 
lock tty_lock/3 -- possible deadlock
#
# JRE version: OpenJDK Runtime Environment (14.0) (slowdebug build 
14-internal+0-adhoc.fool.jdk-dev)
# Java VM: OpenJDK 64-Bit Server VM (slowdebug 
14-internal+0-adhoc.fool.jdk-dev, compiled mode, sharing, tiered, 
compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0xff529c] Mutex::set_owner_implementation(Thread*)+0x19a
#
---------------------------------------------------

## Reproduce
---------------------------------------------------
# just run with debug VMs
java -Xcomp -XX:+UnlockDiagnosticVMOptions -XX:+CIPrintCompileQueue
---------------------------------------------------
Comments
Thank you so much, Christoph Langer.
25-09-2019
I'll sponsor it for both, 11u and 13u. Will run it through our nightly testing and push it (for 13u when approved).
24-09-2019
Fix request for jdk11u, jdk13u This bug can still be reproduced in jdk11u and jdk13u. It would be better to fix it. Patch can be applied cleanly with hg import. The risk is low since it only affects the dumping of the compile queue. And I need a sponsor. Thanks a lot.
24-09-2019
URL: https://hg.openjdk.java.net/jdk/jdk/rev/2f301425af62 User: thartmann Date: 2019-09-17 13:07:44 +0000
17-09-2019
RFR: https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-September/035026.html
14-09-2019
I think stringstream is a good solution to this specific issue. Thanks Kim.
14-09-2019
print functions are expected to be fairly trivial in general, but of course it will all depend.
13-09-2019
Moving to compiler subcomponent. Holding the (very low ranked) tty_lock across non-trivial code is wrong. To avoid intermixing of messages, build a message in a stringstream and write that to the tty.
13-09-2019
Thanks David for your valuable comments.
13-09-2019
Unclear from the stack which team should try to investigate this: Stack: [0x00007f13da075000,0x00007f13da176000], sp=0x00007f13da16bdd8, free space=987k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x13a124a] Mutex::set_owner_implementation(Thread*)+0x3ea V [libjvm.so+0x1425267] OopStorage::allocation_status(oop const*) const+0x47 V [libjvm.so+0xf66d2e] JNIHandles::is_weak_global_handle(_jobject*)+0x3e V [libjvm.so+0xa131c8] CompileTask::print(outputStream*, char const*, bool, bool)+0x68 V [libjvm.so+0xa053c8] CompileQueue::print(outputStream*)+0x58 V [libjvm.so+0xa055aa] CompileQueue::add(CompileTask*)+0x10a V [libjvm.so+0xa0abe6] CompileBroker::compile_method_base(methodHandle const&, int, int, methodHandle const&, int, CompileTask::CompileReason, bool, Thread*)+0x656 V [libjvm.so+0xa0c072] CompileBroker::compile_method(methodHandle const&, int, int, methodHandle const&, int, CompileTask::CompileReason, DirectiveSet*, Thread*)+0x8c2 V [libjvm.so+0xa0c1b0] CompileBroker::compile_method(methodHandle const&, int, int, methodHandle const&, int, CompileTask::CompileReason, Thread*)+0x80 V [libjvm.so+0x9e4713] CompilationPolicy::compile_if_required(methodHandle const&, Thread*)+0x1a3 V [libjvm.so+0x11bec62] LinkResolver::resolve_static_call(CallInfo&, LinkInfo const&, bool, Thread*)+0x1a2 V [libjvm.so+0x1361955] MethodHandles::resolve_MemberName(Handle, Klass*, bool, Thread*)+0xf75 V [libjvm.so+0x1361a73] MHN_resolve_Mem+0x103 we have compiler code calling GC code which calls runtime code. :) CompileQueue::add calls print_tty() which uses a ttyLocker. OopStorage::allocation_status then locks _allocation_mutex I'm going to re-assign to GC initially. But there has always been problems with the ttyLocker in some circumstances. Given tracing/logging can be turned on in arbitrary paths, you can't assign a lock ranking that will always work.
13-09-2019