JDK-8246676 : monitor list lock operations need more fencing
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 15
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • CPU: ppc,x86_64
  • Submitted: 2020-06-05
  • Updated: 2024-10-17
  • Resolved: 2020-07-15
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 15 JDK 16
15 b33Fixed 16Fixed
Related Reports
Relates :  
JDK-8153224 - Monitor deflation prolong safepoints
Relates :  
JDK-8235795 - replace monitor list mux{Acquire,Release}(&gListLock) with spin locks
Description
We've observed this guarantee once when a thread was about to exit:
#  Internal Error (synchronizer.cpp:1677), pid=47037, tid=47409
#  guarantee(l_om_in_use_count == in_use_count) failed: in-use counts don't match: l_om_in_use_count=2, in_use_count=1

Stack: [0x00003ffe13000000,0x00003ffe13200000],  sp=0x00003ffe131fe180,  free space=2040k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xe5e450]  ObjectSynchronizer::om_flush(Thread*)+0x710
V  [libjvm.so+0xeb0cac]  Threads::remove(JavaThread*, bool)+0x3c
V  [libjvm.so+0xeb6840]  JavaThread::exit(bool, JavaThread::ExitType)+0x830
V  [libjvm.so+0xeb6aa0]  JavaThread::post_run()+0x30
V  [libjvm.so+0xeb3d88]  Thread::call_run()+0x198
V  [libjvm.so+0xcb11d4]  thread_native_entry(Thread*)+0x154
C  [libpthread.so.0+0x8a64]  start_thread+0xf4

Hg tip: http://hg.openjdk.java.net/jdk/jdk/rev/755ab9edca97
Test: proprietary
Reproducer: not available at present
Platform: linuxppc64le (Power8)
Comments
URL: https://hg.openjdk.java.net/jdk/jdk15/rev/40fcff600a24 User: dcubed Date: 2020-07-15 21:09:20 +0000
15-07-2020
I ran the serviceability/threads/TestFalseDeadLock.java on my Linux-X64 server in the same config as the Mach5 sighting $ elapsed_times start_runs doit_fastdebug.log start_runs 0 seconds doit_fastdebug.log 12 days 19 hours 41 minutes 14 seconds $ tail doit_fastdebug.do_test_loop_forever.log Run #150490...PASS Run #150491...PASS Run #150492...PASS Run #150493...PASS Run #150494...PASS Run #150495...PASS Run #150496...PASS Run #150497...PASS Run #150498...PASS 150499 runs done; pass_count=150499; fail_count=0 So 150K runs over 12+ days including execution during jdk-15+28 and jdk-15+29 stress testing without any failures. It's safe to say that this failure isn't easy to reproduce on my Linux-X64 server.
01-07-2020
Spotted in the jdk-16+3-40-tier3 CI job set: serviceability/threads/TestFalseDeadLock.java Here's a snippet from the log file: #section:main ----------messages:(5/277)---------- command: main TestFalseDeadLock reason: User specified action: run main/othervm TestFalseDeadLock Mode: othervm [/othervm specified] Additional options from @modules: --add-modules java.base --add-exports java.base/jdk.internal.misc=ALL-UNNAMED elapsed time (seconds): 15.542 ----------configuration:(4/111)---------- Boot Layer add modules: java.base add exports: java.base/jdk.internal.misc ALL-UNNAMED ----------System.out:(458/4672)---------- For random generator using seed: 1824364290680712035 To re-run test with same seed value please add "-Djdk.test.lib.random.seed=1824364290680712035" to command line. Hullo <snip> Hullo Hullo # To suppress the following error report, specify this argument # after -XX: or in .hotspotrc: SuppressErrorAt=/synchronizer.cpp:1676 # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S95/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/eb8f2428-096c-4084-9a45-7b21c08b0923/runs/9974367d-a652-4600-b70c-23bbbfc06422/workspace/open/src/hotspot/share/runtime/synchronizer.cpp:1676), pid=25790, tid=25929 # guarantee(l_om_in_use_count == in_use_count) failed: in-use counts don't match: l_om_in_use_count=2, in_use_count=1 # # JRE version: Java(TM) SE Runtime Environment (16.0+3) (fastdebug build 16-ea+3-40) # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 16-ea+3-40, mixed mode, sharing, tiered, compressed oops, parallel gc, linux-amd64) # Problematic frame: # V [libjvm.so+0x164b448]Hullo ObjectSynchronizer::om_flush(Thread*)+0x718 # # Core dump will be written. Default location: Core dumps may be processed with "/opt/core.sh %p" (or dumping to /opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/scratch/0/core.25790) # # An error report file with more information is saved as: # /opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/scratch/0/hs_err_pid25790.log # # If you would like to submit a bug report, please visit: # https://bugreport.java.com/bugreport/crash.jsp # ----------System.err:(0/0)---------- ----------rerun:(38/6221)*---------- Here's the crashing thread's stack: --------------- T H R E A D --------------- Current thread (0x00007f0d980d12f0): JavaThread "Thread-81" [_thread_in_vm, id=25929, stack(0x00007f0ccf8f9000,0x00007f0ccf9fa000)] Stack: [0x00007f0ccf8f9000,0x00007f0ccf9fa000], sp=0x00007f0ccf9f8a40, free space=1022k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x164b448] ObjectSynchronizer::om_flush(Thread*)+0x718 V [libjvm.so+0x16c6995] Threads::remove(JavaThread*, bool)+0x15 V [libjvm.so+0x16ce362] JavaThread::exit(bool, JavaThread::ExitType)+0x8f2 V [libjvm.so+0x16ce966] JavaThread::post_run()+0x16 V [libjvm.so+0x16c8a8d] Thread::call_run()+0x16d V [libjvm.so+0x13ccac6] thread_native_entry(Thread*)+0x116 Here's the test arguments info: VM Arguments: jvm_args: -Dtest.vm.opts=-XX:MaxRAMPercentage=6 -Dtest.tool.vm.opts=-J-XX:MaxRAMPercentage=6 -Dtest.compiler.opts= -Dtest.java.opts=-XX:+CreateCoredumpOnCrash -XX:+UseParallelGC -XX:+UseNUMA -Dtest.jdk=/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/linux-x64-debug.jdk/jdk-16/fastdebug -Dcompile.jdk=/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/linux-x64-debug.jdk/jdk-16/fastdebug -Dtest.timeout.factor=4.0 -Dtest.nativepath=/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/linux-x64-debug.test/hotspot/jtreg/native -Dtest.root=/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/src.full/open/test/hotspot/jtreg -Dtest.name=serviceability/threads/TestFalseDeadLock.java -Dtest.file=/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/src.full/open/test/hotspot/jtreg/serviceability/threads/TestFalseDeadLock.java -Dtest.src=/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/src.full/open/test/hotspot/jtreg/serviceability/threads -Dtest.src.path=/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/src.full/open/test/hotspot/jtreg/serviceability/threads:/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/src.full/open/test/lib -Dtest.classes=/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/classes/3/serviceability/threads/TestFalseDeadLock.d -Dtest.class.path=/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/classes/3/serviceability/threads/TestFalseDeadLock.d:/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/classes/3/test/lib -Dtest.class.path.prefix=/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/classes/3/serviceability/threads/TestFalseDeadLock.d:/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/src.full/open/test/hotspot/jtreg/serviceability/threads:/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/classes/3/test/lib -Dtest.modules=java.base/jdk.internal.misc --add-modules=java.base --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED -XX:MaxRAMPercentage=6 -XX:+CreateCoredumpOnCrash -XX:+UseParallelGC -XX:+UseNUMA -Djava.library.path=/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/linux-x64-debug.test/hotspot/jtreg/native java_command: com.sun.javatest.regtest.agent.MainWrapper /opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/serviceability/threads/TestFalseDeadLock.d/main.0.jta java_class_path (initial): /opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/classes/3/serviceability/threads/TestFalseDeadLock.d:/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/src.full/open/test/hotspot/jtreg/serviceability/threads:/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S264/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/4af97b51-6eda-43c7-a73f-b1c4fbb13a8e/runs/e9baf92b-c49e-4f04-8214-2c8decd71978/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_serviceability/classes/3/test/lib:/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+3-40/src.full/open/test/lib:/opt/mach5/mesos/work_dir/jib-master/install/jtreg/5.1/b01/bundles/jtreg_bin-5.1.zip/jtreg/lib/javatest.jar:/opt/mach5/mesos/work_dir/jib-master/install/jtreg/5.1/b01/bundles/jtreg_bin-5.1.zip/jtreg/lib/jtreg.jar Launcher Type: SUN_STANDARD [Global flags] intx CICompilerCount = 4 {product} {ergonomic} Number of compiler threads to run bool CreateCoredumpOnCrash = true {product} {command line} Create core/mini dump on VM fatal error size_t InitialHeapSize = 524288000 {product} {ergonomic} Initial heap size (in bytes); zero means use ergonomics size_t MaxHeapSize = 2006974464 {product} {ergonomic} Maximum heap size (in bytes) size_t MaxNewSize = 668991488 {product} {ergonomic} Maximum new generation size (in bytes), max_uintx means set ergonomically uint64_t MaxRAM = 33444835328 {pd product} {ergonomic} Real memory size (in bytes) used to set maximum heap size double MaxRAMPercentage = 6.000000 {product} {command line} Maximum percentage of real memory used for maximum heap size size_t MinHeapDeltaBytes = 524288 {product} {ergonomic} The minimum change in heap space due to GC (in bytes) size_t MinHeapSize = 8388608 {product} {ergonomic} Minimum heap size (in bytes); zero means use ergonomics size_t NewSize = 174587904 {product} {ergonomic} Initial new generation size (in bytes) uintx NonNMethodCodeHeapSize = 5839372 {pd product} {ergonomic} Size of code heap with non-nmethods (in bytes) uintx NonProfiledCodeHeapSize = 122909434 {pd product} {ergonomic} Size of code heap with non-profiled methods (in bytes) size_t OldSize = 349700096 {product} {ergonomic} Initial tenured generation size (in bytes) uintx ProfiledCodeHeapSize = 122909434 {pd product} {ergonomic} Size of code heap with profiled methods (in bytes) uintx ReservedCodeCacheSize = 251658240 {pd product} {ergonomic} Reserved code cache size (in bytes) - maximum code cache size bool SegmentedCodeCache = true {product} {ergonomic} Use a segmented code cache size_t SoftMaxHeapSize = 2006974464 {manageable} {ergonomic} Soft limit for maximum heap size (in bytes) bool UseCompressedClassPointers = true {lp64_product} {ergonomic} Use 32-bit class pointers in 64-bit VM. lp64_product means flag is always constant in 32 bit VM bool UseCompressedOops = true {lp64_product} {ergonomic} Use 32-bit object references in 64-bit VM. lp64_product means flag is always constant in 32 bit VM bool UseNUMA = false {product} {command line} Use NUMA if available bool UseParallelGC = true {product} {command line} Use the Parallel garbage collector.
18-06-2020
Discussing the race with David H., Erik O. and Robbin. This is related to this fix that was pushed in jdk-15+9: JDK-8235795 replace monitor list mux{Acquire,Release}(&gListLock) with spin locks along with modifications to the monitor list algorithm in jdk-15+26: JDK-8153224 Monitor deflation prolong safepoints
17-06-2020
This failure mode has only been seen on Platform: linuxppc64le (Power8) so far so I've set the OS and CPU values.
09-06-2020
ILW = HLM = P3
09-06-2020
Dropping to P3. Only a single occurrence has been seen to date.
05-06-2020