Bug ID: JDK-8256359 AArch64: runtime/ReservedStack/ReservedStackTestCompiler.java fails

JDK-8256359 : AArch64: runtime/ReservedStack/ReservedStackTestCompiler.java fails

Type: Bug
Component: hotspot
Sub-Component: runtime
Affected Version: 16

Priority: P2
Status: Closed
Resolution: Fixed
OS: linux
CPU: aarch64

Submitted: 2020-11-13
Updated: 2021-04-21
Resolved: 2020-11-26

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 11	JDK 13	JDK 16
11.0.11-oracleFixed	13.0.8Fixed	16 b27Fixed

Sub Tasks

JDK-8256803 :

ProblemList runtime/ReservedStack/ReservedStackTestCompiler.java on linux-aarch64 - Resolved

Description

The following test failed in the JDK16 CI:

runtime/ReservedStack/ReservedStackTestCompiler.java

Here's a snippet from the log file:

#section:main
----------messages:(5/622)----------
command: main -XX:-TieredCompilation -XX:-Inline -XX:CompileCommand=exclude,java/util/concurrent/locks/AbstractOwnableSynchronizer.setExclusiveOwnerThread ReservedStackTest
reason: User specified action: run main/othervm -XX:-TieredCompilation -XX:-Inline -XX:CompileCommand=exclude,java/util/concurrent/locks/AbstractOwnableSynchronizer.setExclusiveOwnerThread ReservedStackTest 
Mode: othervm [/othervm specified]
Additional options from @modules: --add-modules java.base --add-exports java.base/jdk.internal.misc=ALL-UNNAMED --add-exports java.base/jdk.internal.vm.annotation=ALL-UNNAMED
elapsed time (seconds): 16.907
----------configuration:(5/193)----------
Boot Layer
  add modules: java.base                            
  add exports: java.base/jdk.internal.misc          ALL-UNNAMED
               java.base/jdk.internal.vm.annotation ALL-UNNAMED

----------System.out:(62/3935)----------
CompileCommand: exclude java/util/concurrent/locks/AbstractOwnableSynchronizer.setExclusiveOwnerThread
Command line: [/scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+25-1586/linux-aarch64-debug.jdk/jdk-16/fastdebug/bin/java -cp /scratch/opt/mach5/mesos/work_dir/slaves/47535081-0322-4d83-bb78-f7e11abb86d0-S47885/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/2d9383f7-be9a-46ca-bd9e-a682713fb404/runs/711ce683-d382-44e6-9de8-416483e6778b/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_tier2_runtime/classes/0/runtime/ReservedStack/ReservedStackTestCompiler.d:/scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+25-1586/src.full/open/test/hotspot/jtreg/runtime/ReservedStack:/scratch/opt/mach5/mesos/work_dir/slaves/47535081-0322-4d83-bb78-f7e11abb86d0-S47885/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/2d9383f7-be9a-46ca-bd9e-a682713fb404/runs/711ce683-d382-44e6-9de8-416483e6778b/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_tier2_runtime/classes/0/test/lib:/scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+25-1586/src.full/open/test/lib:/scratch/opt/mach5/mesos/work_dir/jib-master/install/jtreg/5.1/b01/bundles/jtreg_bin-5.1.zip/jtreg/lib/javatest.jar:/scratch/opt/mach5/mesos/work_dir/jib-master/install/jtreg/5.1/b01/bundles/jtreg_bin-5.1.zip/jtreg/lib/jtreg.jar -XX:StackReservedPages=1 -version ]
[2020-11-13T19:11:40.898003850Z] Gathering output for process 32117
StackReservedPages=1 log: [java version "16-ea" 2021-03-16
Java(TM) SE Runtime Environment (fastdebug build 16-ea+25-1586)
Java HotSpot(TM) 64-Bit Server VM (fastdebug build 16-ea+25-1586, mixed mode)
]
[2020-11-13T19:11:41.134126825Z] Waiting for completion for process 32117
[2020-11-13T19:11:41.134535725Z] Waiting for completion finished for process 32117
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
PASSED
# To suppress the following error report, specify this argument
# after -XX: or in .hotspotrc:  SuppressErrorAt=/frame_aarch64.cpp:326
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/opt/mach5/mesos/work_dir/slaves/47535081-0322-4d83-bb78-f7e11abb86d0-S45708/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/fee1c1bd-3152-42f9-ad8c-fc28266cc5ff/runs/01291963-05c7-4ec1-8b60-6f112a23df10/workspace/open/src/hotspot/cpu/aarch64/frame_aarch64.cpp:326), pid=32094, tid=32196
#  assert(sp() <= (intptr_t*) result) failed: monitor end should be above the stack pointer
#
# JRE version: Java(TM) SE Runtime Environment (16.0+25) (fastdebug build 16-ea+25-1586)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 16-ea+25-1586, mixed mode, compressed oops, g1 gc, linux-aarch64)
# Problematic frame:
# V  [libjvm.so+0xb33508]  frame::interpreter_frame_monitor_end() const+0x68
#
# Core dump will be written. Default location: Core dumps may be processed with "/opt/core.sh %p" (or dumping to /scratch/opt/mach5/mesos/work_dir/slaves/47535081-0322-4d83-bb78-f7e11abb86d0-S47885/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/2d9383f7-be9a-46ca-bd9e-a682713fb404/runs/711ce683-d382-44e6-9de8-416483e6778b/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_tier2_runtime/scratch/2/core.32094)
#
# An error report file with more information is saved as:
# /scratch/opt/mach5/mesos/work_dir/slaves/47535081-0322-4d83-bb78-f7e11abb86d0-S47885/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/2d9383f7-be9a-46ca-bd9e-a682713fb404/runs/711ce683-d382-44e6-9de8-416483e6778b/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_tier2_runtime/scratch/2/hs_err_pid32094.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
#
----------System.err:(33/5874)----------
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method java.util.concurrent.locks.ReentrantLock$Sync.lock()V [1]
----------rerun:(40/7723)*----------


Here's snippets from the hs_err_pid file:

#  Internal Error (/opt/mach5/mesos/work_dir/slaves/47535081-0322-4d83-bb78-f7e11abb86d0-S45708/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/fee1c1bd-3152-42f9-ad8c-fc28266cc5ff/runs/01291963-05c7-4ec1-8b60-6f112a23df10/workspace/open/src/hotspot/cpu/aarch64/frame_aarch64.cpp:326), pid=32094, tid=32196
#  assert(sp() <= (intptr_t*) result) failed: monitor end should be above the stack pointer
#
# JRE version: Java(TM) SE Runtime Environment (16.0+25) (fastdebug build 16-ea+25-1586)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 16-ea+25-1586, mixed mode, compressed oops, g1 gc, linux-aarch64)
# Problematic frame:
# V  [libjvm.so+0xb33508]  frame::interpreter_frame_monitor_end() const+0x68

---------------  T H R E A D  ---------------

Current thread (0x0000ffff84007870):  GCTaskThread "GC Thread#2" [stack: 0x0000ffff8c720000,0x0000ffff8c920000] [id=32196]

Stack: [0x0000ffff8c720000,0x0000ffff8c920000],  sp=0x0000ffff8c91d810,  free space=2038k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xb33508]  frame::interpreter_frame_monitor_end() const+0x68
V  [libjvm.so+0xb2fecc]  frame::oops_interpreted_do(OopClosure*, RegisterMap const*, bool) const+0x2ec
V  [libjvm.so+0x15f96a8]  JavaThread::oops_do_frames(OopClosure*, CodeBlobClosure*) [clone .part.0]+0x88
V  [libjvm.so+0x15fc648]  Thread::oops_do(OopClosure*, CodeBlobClosure*)+0xc4
V  [libjvm.so+0x1605908]  Threads::possibly_parallel_oops_do(bool, OopClosure*, CodeBlobClosure*)+0x1e4
V  [libjvm.so+0xc3fc90]  G1RootProcessor::process_java_roots(G1RootClosures*, G1GCPhaseTimes*, unsigned int)+0x80
V  [libjvm.so+0xc405b0]  G1RootProcessor::evacuate_roots(G1ParScanThreadState*, unsigned int)+0x70
V  [libjvm.so+0xb81744]  G1EvacuateRegionsTask::scan_roots(G1ParScanThreadState*, unsigned int)+0x24
V  [libjvm.so+0xb823b0]  G1EvacuateRegionsBaseTask::work(unsigned int)+0x60
V  [libjvm.so+0x1752aa8]  GangWorker::run_task(WorkData)+0x98
V  [libjvm.so+0x1752bf4]  GangWorker::loop()+0x44
V  [libjvm.so+0x16077c8]  Thread::call_run()+0xf8
V  [libjvm.so+0x1337980]  thread_native_entry(Thread*)+0x120
C  [libpthread.so.0+0x7d40]  start_thread+0xb4

JavaThread 0x0000ffff540d8ee0 (nid = 32194) was being processed
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.util.concurrent.locks.ReentrantLock$Sync.lock()V+12 java.base@16-ea
j  java.util.concurrent.locks.ReentrantLock.lock()V+4 java.base@16-ea
j  ReservedStackTest$ReentrantLockTest.lockAndCall(I)V+18
j  ReservedStackTest$ReentrantLockTest.lockAndCall(I)V+25
j  ReservedStackTest$ReentrantLockTest.lockAndCall(I)V+25

<snip a bunch of frame repeats>

j  ReservedStackTest$ReentrantLockTest.lockAndCall(I)V+25
j  ReservedStackTest$ReentrantLockTest.lockAndCall(I)V+25
j  ReservedStackTest$ReentrantLockTest.run()V+2
J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c2d8 [0x0000ffffa5a2c240+0x0000000000000098]
J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c288 [0x0000ffffa5a2c240+0x0000000000000048]
J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c288 [0x0000ffffa5a2c240+0x0000000000000048]
J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c288 [0x0000ffffa5a2c240+0x0000000000000048]

<snip thousands? of frame repeats>

J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c288 [0x0000ffffa5a2c240+0x0000000000000048]
J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c288 [0x0000ffffa5a2c240+0x0000000000000048]
J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c288 [0x0000ffffa5a2c240+0x0000000000000048]
J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c288 [0x0000ffffa5a2c240+0x0000000000000048]
J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c288 [0x0000ffffa5a2c240+0x0000000000000048]
J 113 c2 ReservedStackTest$RunWithSOEContext.recursiveCall()V (65 bytes) @ 0x0000ffffa5a2c288 [0x0000ffffa5a2c240+0x0000000000000048]
j  ReservedStackTest$RunWithSOEContext.run()V+21
j  java.lang.Thread.run()V+11 java.base@16-ea
v  ~StubRoutines::call_stub


Starting this bug at P3 since this is a Tier2 test failure and
starting in hotspot/runtime since this is a Runtime test.
The failing build-ID, jdk-16+25-1586-tier2, only contains
test fixes in an unrelated area so I'm labeling this failure
as intermittent.

Comments

Fix Request (13u) Patch required for parity with jdk11. Applies cleanly except backout of JDK-8256803 is not required. tier1 tested
19-04-2021
Patch applies cleanly, fixes a rare crash if e.g. GC happens during the time we're throwing a delayed StackOverflowException.
26-11-2020
Changeset: 4e43b288 Author: Andrew Haley <aph@openjdk.org> Date: 2020-11-26 17:52:54 +0000 URL: https://git.openjdk.java.net/jdk/commit/4e43b288
26-11-2020
ILW = HLH = P2
24-11-2020
> There is a similar loading of esp at the end of method TemplateTable::branch(bool is_jsr, bool is_wide): That should be fine. It doesn't call into the VM and allow a safepoint.
23-11-2020
There is a similar loading of esp at the end of method TemplateTable::branch(bool is_jsr, bool is_wide): ... // remove activation // get sender esp __ ldr(esp, Address(rfp, frame::interpreter_frame_sender_sp_offset * wordSize)); // remove frame anchor __ leave(); // Ensure compiled code always sees stack at proper alignment __ andr(sp, esp, -16); // and begin the OSR nmethod __ ldr(rscratch1, Address(r19, nmethod::osr_entry_point_offset())); __ br(rscratch1); } } }
23-11-2020
If I force a GC in InterpreterRuntime::throw_delayed_StackOverflowError, then the test will crash every time. If I then change "esp" to "rscratch2" in remove_activation, the test passes every time. I'm moving this from compiler to runtime.
22-11-2020
As far as I can tell this is an old bug in InterpreterMacroAssembler::remove_activation(). It does the following to read the sender sp into esp (instead of a temporary register): 682 // remove activation 683 // get sender esp 684 ldr(esp, 685 Address(rfp, frame::interpreter_frame_sender_sp_offset * wordSize)); so when we take the throw_delayed_StackOverflowError path, that's the value that gets stored into the thread anchor. During object allocation for the exception, we trigger a GC, and in the case I looked at, a different GC thread hits the assert when scanning thread oops because we haven't really popped the frame yet, just changed the sp. Reading sender_sp into a temporary register seems like the right fix here. I don't see anything pointing to a problem with the JIT or deoptimization. [~aph] You may be the most familiar with this code. Does my diagnosis sound correct?
21-11-2020
Thank you David. Reassigning to Compiler sub-component for further investigation.
18-11-2020
Note this is the C2 version of the test that is failing. So passing on to compiler team may be the best starting point.
18-11-2020
We should augment the assert to print all the values involved.
18-11-2020
ILW = HLH = P2
17-11-2020