JDK-8233576 : [Graal] guarantee(oopDesc::is_oop_or_null(obj)) failed: invalid oop
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 11.0.7,14,15
  • Priority: P3
  • Status: Resolved
  • Resolution: Duplicate
  • Submitted: 2019-11-05
  • Updated: 2020-01-10
  • Resolved: 2020-01-10
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 14
14Resolved
Related Reports
Duplicate :  
JDK-8231515 - [Graal] Crash during exception throwing in InterpreterRuntime::resolve_invoke
Relates :  
JDK-4998314 - compute_compiled_exc_handler() called with pending exception
Description
Failure in JDK 14 CI testing.
(first failure in jdk-14+22-941-tier3)

test - compiler/c2/4998314/Test.java

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (t:/workspace/open/src/hotspot/share/oops/oop.cpp:129), pid=43216, tid=86184
#  guarantee(oopDesc::is_oop_or_null(obj)) failed: invalid oop: 0x0000008b77ccee80
#
# JRE version: Java(TM) SE Runtime Environment (14.0+22) (fastdebug build 14-ea+22-941)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 14-ea+22-941, mixed mode, sharing, tiered, jvmci, jvmci compiler, compressed oops, g1 gc, windows-amd64)
# Core dump will be written. Default location: T:\testoutput\test-support\jtreg_closed_test_hotspot_jtreg_hotspot_not_fast_compiler\scratch\0\hs_err_pid43216.mdmp
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
#

---------------  S U M M A R Y ------------

Command Line: -Dtest.class.path.prefix=T:\testoutput\test-support\jtreg_closed_test_hotspot_jtreg_hotspot_not_fast_compiler\classes\2\compiler\c2\4998314\Test.d;C:\ade\mesos\work_dir\jib-master\install\jdk-14+22-941\src.full\closed\test\hotspot\jtreg\compiler\c2\4998314 -Dtest.src=C:\ade\mesos\work_dir\jib-master\install\jdk-14+22-941\src.full\closed\test\hotspot\jtreg\compiler\c2\4998314 -Dtest.src.path=C:\ade\mesos\work_dir\jib-master\install\jdk-14+22-941\src.full\closed\test\hotspot\jtreg\compiler\c2\4998314 -Dtest.classes=T:\testoutput\test-support\jtreg_closed_test_hotspot_jtreg_hotspot_not_fast_compiler\classes\2\compiler\c2\4998314\Test.d -Dtest.class.path=T:\testoutput\test-support\jtreg_closed_test_hotspot_jtreg_hotspot_not_fast_compiler\classes\2\compiler\c2\4998314\Test.d -Dtest.vm.opts=-XX:MaxRAMPercentage=3 -Dtest.tool.vm.opts=-J-XX:MaxRAMPercentage=3 -Dtest.compiler.opts= -Dtest.java.opts=-XX:+CreateCoredumpOnCrash -ea -esa -server -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -XX:+UseJVMCICompiler -Djvmci.Compiler=graal -XX:+TieredCompilation -Dtest.jdk=c:\ade\mesos\work_dir\jib-master\install\jdk-14+22-941\windows-x64-debug.jdk\jdk-14\fastdebug -Dcompile.jdk=c:\ade\mesos\work_dir\jib-master\install\jdk-14+22-941\windows-x64-debug.jdk\jdk-14\fastdebug -Dtest.timeout.factor=10.0 -Dtest.root=C:\ade\mesos\work_dir\jib-master\install\jdk-14+22-941\src.full\closed\test\hotspot\jtreg -Dtest.nativepath=c:\ade\mesos\work_dir\jib-master\install\jdk-14+22-941\windows-x64-debug.test\hotspot\jtreg\native -XX:MaxRAMPercentage=3 -XX:+CreateCoredumpOnCrash -ea -esa -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -XX:+UseJVMCICompiler -Djvmci.Compiler=graal -XX:+TieredCompilation -Djava.library.path=c:\ade\mesos\work_dir\jib-master\install\jdk-14+22-941\windows-x64-debug.test\hotspot\jtreg\native -XX:-Inline -XX:CompileOnly=Test com.sun.javatest.regtest.agent.MainWrapper T:\testoutput\test-support\jtreg_closed_test_hotspot_jtreg_hotspot_not_fast_compiler\compiler\c2\4998314\Test.d\main.0.jta

Host: inst-ltxfp-Win2, AMD EPYC 7551 32-Core Processor                , 16 cores, 63G,  Windows Server 2012 R2 , 64 bit Build 9600 (6.3.9600.19358)
Time: Fri Nov  1 12:40:17 2019 /GM elapsed time: 21 seconds (0d 0h 0m 21s)

---------------  T H R E A D  ---------------

Current thread (0x0000008b76666000):  VMThread "VM Thread" [stack: 0x0000008b76df0000,0x0000008b76ef0000] [id=86184]

Stack: [0x0000008b76df0000,0x0000008b76ef0000]
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0xad7561]  os::platform_print_native_stack+0xf1  (os_windows_x86.cpp:369)
V  [jvm.dll+0xcdb9bb]  VMError::report+0xf0b  (vmerror.cpp:725)
V  [jvm.dll+0xcdd26e]  VMError::report_and_die+0x8ae  (vmerror.cpp:1533)
V  [jvm.dll+0xcdd964]  VMError::report_and_die+0x64  (vmerror.cpp:1317)
V  [jvm.dll+0x502e32]  report_vm_error+0x102  (debug.cpp:264)
V  [jvm.dll+0xabaf2a]  VerifyOopClosure::do_oop+0x6a  (oop.cpp:132)
V  [jvm.dll+0x5b7957]  InterpreterFrameClosure::offset_do+0x97  (frame.cpp:710)
V  [jvm.dll+0xac03aa]  InterpreterOopMap::iterate_oop+0x6a  (oopmapcache.cpp:208)
V  [jvm.dll+0x5b883d]  frame::oops_interpreted_do+0x7bd  (frame.cpp:890)
V  [jvm.dll+0xc7d807]  JavaThread::oops_do+0x2a7  (thread.cpp:2966)
V  [jvm.dll+0xc82669]  Threads::verify+0x79  (thread.cpp:5057)
V  [jvm.dll+0xca630d]  Universe::verify+0x27d  (universe.cpp:1086)
V  [jvm.dll+0xcde27a]  VM_Exit::doit+0x4a  (vmoperations.cpp:468)
V  [jvm.dll+0xcdeade]  VM_Operation::evaluate+0xbe  (vmoperations.cpp:68)
V  [jvm.dll+0xce3c98]  VMThread::evaluate_operation+0xb8  (vmthread.cpp:406)
V  [jvm.dll+0xce47fa]  VMThread::loop+0x5ba  (vmthread.cpp:567)
V  [jvm.dll+0xce5178]  VMThread::run+0xd8  (vmthread.cpp:305)
V  [jvm.dll+0xc76e52]  Thread::call_run+0x192  (thread.cpp:403)
V  [jvm.dll+0xad5d2e]  thread_native_entry+0x10e  (os_windows.cpp:465)
C  [ucrtbase.DLL+0x203ba]
C  [KERNEL32.DLL+0x13f2]
C  [ntdll.dll+0x154f4]

JavaThread 0x0000008b767f3000 (nid = 23404) was being processed
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.lang.String.toString()Ljava/lang/String;+1 java.base@14-ea
j  Test.foo(Ljava/lang/String;)V+1
J 82 jvmci Test.test()V (33 bytes) @ 0x0000008b0a3b237c [0x0000008b0a3b22c0+0x00000000000000bc]
j  Test.run()V+9
v  ~StubRoutines::call_stub
VM_Operation (0x0000008b77bce410): Exit, mode: safepoint, requested by thread 0x0000008b767d8000
Comments
I was able to reproduce the stub deopt problem by having the stub force deoptimization of the caller, so if that's the only issue with compiler/c2/4998314/Test.java then my fix should take care of it. I'll close this as a dup.
10-01-2020
I didn't have any luck reproducing it until I did the following: 1) call deopt_caller at the end of JVMCIRuntime::throw_and_post_jvmti_exception 2) perform a VM_ForceSafepoint on entry to JVM_ENTRY* 3) { ResourceMark rm; Threads::verify(); } in VMThread::loop() to force a lot of verifies 3) seemed to help the most.
09-01-2020
It's not really valid to build the FrameState for that to work. A rethrow exception FrameState must have the exception to be thrown on the top of stack and we're in the middle of a call which is producing the value to be thrown. I think we need to explicitly detect the deopt and dispatch to the unpack_with_exception entry point in the deopt blob from the exception blob. I've got a fix for this that I'm testing now included in my fix for JDK-8231515 since some FrameState verification logic I added detected both problems and they both have the same root cause. We should dup this one up against JDK-8231515. What's the best way to reproduce this particular failure?
07-01-2020
I'm trying to set up a FrameState with "rethrow exception" set. Would that also solve the problem in JDK-8231515?
07-01-2020
Yes that's one of several FrameState problems in JDK-8231515. I suspect this stub needs special handling to dispatch to the proper unpack_with_exception deopt entry point. That was the point I reached just before Christmas break. I'm deciding how to handle this now. So I think this bug might be a duplicate of what I'm fixing in JDK-8231515 with bad FrameStates from BytecodeExceptionNodes.
06-01-2020
[~never] How are stubs like NullPointerExceptionStub.createNullPointerException() supposed to work if we get a deopt during JVMCIRuntime::throw_and_post_jvmti_exception()? The stub will return the exception object in $rax, but it clears _pending_exception in the thread. When we return from the stub, the exception seems to be lost when we deopt, and the frame state seems wrong for continuing in the interpreter (we try to pop something off an empty stack).
01-01-2020
All of the failures with the assert (macOS and Windows) seem to follow the same pattern. The test has processed all 100 ThreadDeaths, and is trying to exit. The following trueInDebug logic then triggers extra verification, which sometimes fails: 449 void VM_Exit::doit() { 450 451 if (VerifyBeforeExit) { 452 HandleMark hm(VMThread::vm_thread()); 453 // Among other things, this ensures that Eden top is correct. 454 Universe::heap()->prepare_for_verify(); 455 // Silent verification so as not to pollute normal output, 456 // unless we really asked for it. 457 Universe::verify(); 458 } Something different seems to be happening on Linux. Instead of 100 "KILL THREAD!" messages, there are exactly 19.
16-12-2019
I'm looking at the crash from Dec 3, and $rsp is not aligned and it doesn't even belong to the current thread's stack region. It's off by more than 400M. So I guess a stack adjustment went wrong, maybe during deoptimization? In the Nov 15 crash, we apparently crashed at java.io.FileOutputStream.write([BII)V+16, but the instructions there are all 0's and don't disassemble into anything meaningful. I don't see the assert happening on Linux, just macOS and Windows.
12-12-2019
That bug would only happen if there's a JVMTI agent and I don't seem to see one in the command line for the test. I would guess this is just a problem with our handling async exceptions and deopt.
09-12-2019
[~never] Does this look like it could be the same as JDK-8231515?
06-12-2019
I see a number of different failure modes in test crashes that have been linked to this bug report, but it is not at all clear that they are all the same issue. This report and all the Windows failures are a failure during "verify on exit".
25-11-2019