Bug ID: JDK-8200195 serviceability/jvmti/FieldAccessWatch/FieldAccessWatch.java crashes with "assert(thread->thread_state() == _thread_in_native) failed: coming from wrong thread state"

JDK-8200195 : serviceability/jvmti/FieldAccessWatch/FieldAccessWatch.java crashes with "assert(thread->thread_state() == _thread_in_native) failed: coming from wrong thread state"

Type: Bug
Component: hotspot
Sub-Component: jvmti
Affected Version: 11

Priority: P3
Status: Resolved
Resolution: Fixed

Submitted: 2018-03-23
Updated: 2024-07-10
Resolved: 2018-04-10

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 11
11 b10Fixed

Related Reports

Relates :

JDK-8193369 - post_field_access does not work for some functions, possibly related to fast_getfield

Description

serviceability/jvmti/FieldAccessWatch/FieldAccessWatch.java crashes with assertion:

#  Internal Error (/home/katya/work/JavaSE/Hotspot/ws/jdk.hs/open/src/hotspot/share/runtime/interfaceSupport.inline.hpp:159), pid=21324, tid=21338
#  assert(thread->thread_state() == _thread_in_native) failed: coming from wrong thread state

Current CompileTask:
JVMCI:   1049  156       4       java.lang.String::hashCode (49 bytes)

Stack: [0x00007fc7b632c000,0x00007fc7b642d000],  sp=0x00007fc7b6428080,  free space=1008k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x180be52]  VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x162
V  [libjvm.so+0x180ccef]  VMError::report_and_die(Thread*, char const*, int, char const*, char const*, __va_list_tag*)+0x2f
V  [libjvm.so+0xb1e66d]  report_vm_error(char const*, int, char const*, char const*, ...)+0xdd
V  [libjvm.so+0x90387f]  ThreadInVMfromNative::ThreadInVMfromNative(JavaThread*)+0x7f
V  [libjvm.so+0xfa5757]  jni_GetFieldID+0xa7
C  [libFieldAccessWatch.so+0xbaf]  handleNotification+0x20f
V  [libjvm.so+0x11665fa]  JvmtiExport::post_field_access(JavaThread*, Method*, unsigned char*, Klass*, Handle, _jfieldID*)+0x1fa
V  [libjvm.so+0xe9a083]  InterpreterRuntime::post_field_access(JavaThread*, oopDesc*, ConstantPoolCacheEntry*)+0x383
j  java.util.ArrayList$Itr.<init>(Ljava/util/ArrayList;)V+19 java.base@11-internal
j  java.util.ArrayList.iterator()Ljava/util/Iterator;+5 java.base@11-internal
j  java.lang.invoke.InvokerBytecodeGenerator.cpPatches([B)[Ljava/lang/Object;+14 java.base@11-internal
j  java.lang.invoke.InvokerBytecodeGenerator.loadMethod([B)Ljava/lang/invoke/MemberName;+3 java.base@11-internal
j  java.lang.invoke.InvokerBytecodeGenerator.generateCustomizedCode(Ljava/lang/invoke/LambdaForm;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/MemberName;+29 java.base@11-internal
j  java.lang.invoke.LambdaForm.compileToBytecode()V+69 java.base@11-internal
j  java.lang.invoke.LambdaForm.prepare()V+21 java.base@11-internal
j  java.lang.invoke.MethodHandle.<init>(Ljava/lang/invoke/MethodType;Ljava/lang/invoke/LambdaForm;)V+33 java.base@11-internal
j  java.lang.invoke.BoundMethodHandle.<init>(Ljava/lang/invoke/MethodType;Ljava/lang/invoke/LambdaForm;)V+3 java.base@11-internal
j  java.lang.invoke.BoundMethodHandle$Species_LL.<init>(Ljava/lang/invoke/MethodType;Ljava/lang/invoke/LambdaForm;Ljava/lang/Object;Ljava/lang/Object;)V+3 java.base@11-internal
j  java.lang.invoke.BoundMethodHandle$Species_LL.make(Ljava/lang/invoke/MethodType;Ljava/lang/invoke/LambdaForm;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/invoke/BoundMethodHandle;+8 java.base@11-internal
j  java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+16 java.base@11-internal
j  java.lang.invoke.BoundMethodHandle$Species_L.copyWithExtendL(Ljava/lang/invoke/MethodType;Ljava/lang/invoke/LambdaForm;Ljava/lang/Object;)Ljava/lang/invoke/BoundMethodHandle;+19 java.base@11-internal
j  java.lang.invoke.LambdaFormEditor.bindArgumentL(Ljava/lang/invoke/BoundMethodHandle;ILjava/lang/Object;)Ljava/lang/invoke/BoundMethodHandle;+55 java.base@11-internal
j  java.lang.invoke.BoundMethodHandle.bindArgumentL(ILjava/lang/Object;)Ljava/lang/invoke/BoundMethodHandle;+7 java.base@11-internal
j  java.lang.invoke.MethodHandle.bindArgumentL(ILjava/lang/Object;)Ljava/lang/invoke/BoundMethodHandle;+6 java.base@11-internal
j  java.lang.invoke.MethodHandle.bindTo(Ljava/lang/Object;)Ljava/lang/invoke/MethodHandle;+15 java.base@11-internal
j  java.lang.invoke.MethodHandleImpl.makePairwiseConvertByEditor(Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;ZZ)Ljava/lang/invoke/MethodHandle;+94 java.base@11-internal
j  java.lang.invoke.MethodHandleImpl.makePairwiseConvert(Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;ZZ)Ljava/lang/invoke/MethodHandle;+18 java.base@11-internal
j  java.lang.invoke.MethodHandleImpl.makePairwiseConvert(Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;Z)Ljava/lang/invoke/MethodHandle;+4 java.base@11-internal
j  java.lang.invoke.MethodHandle.asTypeUncached(Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/MethodHandle;+51 java.base@11-internal
j  java.lang.invoke.MethodHandle.asType(Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/MethodHandle;+24 java.base@11-internal
j  java.lang.invoke.MethodHandleImpl$BindCaller.prepareForInvoker(Ljava/lang/invoke/MethodHandle;)Ljava/lang/invoke/MethodHandle;+20 java.base@11-internal
j  java.lang.invoke.MethodHandleImpl$BindCaller.bindCaller(Ljava/lang/invoke/MethodHandle;Ljava/lang/Class;)Ljava/lang/invoke/MethodHandle;+39 java.base@11-internal
j  java.lang.invoke.MethodHandleImpl.bindCaller(Ljava/lang/invoke/MethodHandle;Ljava/lang/Class;)Ljava/lang/invoke/MethodHandle;+2 java.base@11-internal
j  java.lang.invoke.MethodHandles$Lookup.maybeBindCaller(Ljava/lang/invoke/MemberName;Ljava/lang/invoke/MethodHandle;Ljava/lang/Class;)Ljava/lang/invoke/MethodHandle;+36 java.base@11-internal
j  java.lang.invoke.MethodHandles$Lookup.getDirectMethodCommon(BLjava/lang/Class;Ljava/lang/invoke/MemberName;ZZLjava/lang/Class;)Ljava/lang/invoke/MethodHandle;+255 java.base@11-internal
j  java.lang.invoke.MethodHandles$Lookup.getDirectMethodNoSecurityManager(BLjava/lang/Class;Ljava/lang/invoke/MemberName;Ljava/lang/Class;)Ljava/lang/invoke/MethodHandle;+14 java.base@11-internal
j  java.lang.invoke.MethodHandles$Lookup.unreflect(Ljava/lang/reflect/Method;)Ljava/lang/invoke/MethodHandle;+114 java.base@11-internal
j  org.graalvm.compiler.serviceprovider.JDK9Method.lookupMethodHandle(Ljava/lang/Class;Ljava/lang/String;[Ljava/lang/Class;)Ljava/lang/invoke/MethodHandle;+9 jdk.internal.vm.compiler@11-internal
j  org.graalvm.compiler.serviceprovider.JDK9Method.<clinit>()V+116 jdk.internal.vm.compiler@11-internal
v  ~StubRoutines::call_stub
V  [libjvm.so+0xebded1]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x7e1
V  [libjvm.so+0xe73522]  InstanceKlass::call_class_initializer(Thread*)+0x1c2
V  [libjvm.so+0xe73a37]  InstanceKlass::initialize_impl(Thread*)+0x457
V  [libjvm.so+0xe73e3b]  InstanceKlass::initialize(Thread*)+0x7b
V  [libjvm.so+0x11f2475]  LinkResolver::resolve_field(fieldDescriptor&, LinkInfo const&, Bytecodes::Code, bool, Thread*)+0xc65
V  [libjvm.so+0x11f2b5a]  LinkResolver::resolve_field_access(fieldDescriptor&, constantPoolHandle const&, int, methodHandle const&, Bytecodes::Code, Thread*)+0xca
V  [libjvm.so+0xe9ff56]  InterpreterRuntime::resolve_get_put(JavaThread*, Bytecodes::Code)+0x206
V  [libjvm.so+0xea085b]  InterpreterRuntime::resolve_from_cache(JavaThread*, Bytecodes::Code)+0xfb
j  org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.onSelection()V+66 jdk.internal.vm.compiler@11-internal
j  jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory()Ljdk/vm/ci/runtime/JVMCICompilerFactory;+185 jdk.internal.vm.ci@11-internal
j  jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.<init>()V+230 jdk.internal.vm.ci@11-internal
j  jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.<init>(Ljdk/vm/ci/hotspot/HotSpotJVMCIRuntime$1;)V+1 jdk.internal.vm.ci@11-internal
j  jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.<clinit>()V+13 jdk.internal.vm.ci@11-internal
v  ~StubRoutines::call_stub
V  [libjvm.so+0xebded1]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x7e1
V  [libjvm.so+0xe73522]  InstanceKlass::call_class_initializer(Thread*)+0x1c2
V  [libjvm.so+0xe73a37]  InstanceKlass::initialize_impl(Thread*)+0x457
V  [libjvm.so+0xe73e3b]  InstanceKlass::initialize(Thread*)+0x7b
V  [libjvm.so+0x11edd64]  LinkResolver::resolve_static_call(CallInfo&, LinkInfo const&, bool, Thread*)+0xd4
V  [libjvm.so+0x11f5035]  LinkResolver::resolve_invoke(CallInfo&, Handle, constantPoolHandle const&, int, Bytecodes::Code, Thread*)+0x135
V  [libjvm.so+0xe9dfe2]  InterpreterRuntime::resolve_invoke(JavaThread*, Bytecodes::Code)+0x232
V  [libjvm.so+0xea084b]  InterpreterRuntime::resolve_from_cache(JavaThread*, Bytecodes::Code)+0xeb
j  jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime()Ljdk/vm/ci/hotspot/HotSpotJVMCIRuntime;+3 jdk.internal.vm.ci@11-internal
v  ~StubRoutines::call_stub
V  [libjvm.so+0xebded1]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x7e1
V  [libjvm.so+0xe73522]  InstanceKlass::call_class_initializer(Thread*)+0x1c2
V  [libjvm.so+0xe73a37]  InstanceKlass::initialize_impl(Thread*)+0x457
V  [libjvm.so+0xe73e3b]  InstanceKlass::initialize(Thread*)+0x7b
V  [libjvm.so+0x11edd64]  LinkResolver::resolve_static_call(CallInfo&, LinkInfo const&, bool, Thread*)+0xd4
V  [libjvm.so+0x11f5035]  LinkResolver::resolve_invoke(CallInfo&, Handle, constantPoolHandle const&, int, Bytecodes::Code, Thread*)+0x135
V  [libjvm.so+0xe9dfe2]  InterpreterRuntime::resolve_invoke(JavaThread*, Bytecodes::Code)+0x232
V  [libjvm.so+0xea084b]  InterpreterRuntime::resolve_from_cache(JavaThread*, Bytecodes::Code)+0xeb
j  jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime()Ljdk/vm/ci/hotspot/HotSpotJVMCIRuntime;+3 jdk.internal.vm.ci@11-internal
v  ~StubRoutines::call_stub
V  [libjvm.so+0xebded1]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x7e1
V  [libjvm.so+0xebc5a2]  JavaCalls::call_static(JavaValue*, Klass*, Symbol*, Symbol*, Thread*)+0x212
V  [libjvm.so+0x10c1a76]  JVMCIRuntime::callStatic(char const*, char const*, char const*, JavaCallArguments*, Thread*)+0x166
V  [libjvm.so+0x10c2260]  JVMCIRuntime::initialize_HotSpotJVMCIRuntime(Thread*)+0xe0
V  [libjvm.so+0x10c33e3]  JVM_GetJVMCIRuntime+0xe3
j  jdk.vm.ci.runtime.JVMCI.initializeRuntime()Ljdk/vm/ci/runtime/JVMCIRuntime;+0 jdk.internal.vm.ci@11-internal
j  jdk.vm.ci.runtime.JVMCI.<clinit>()V+2 jdk.internal.vm.ci@11-internal
v  ~StubRoutines::call_stub
V  [libjvm.so+0xebded1]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x7e1
V  [libjvm.so+0xe73522]  InstanceKlass::call_class_initializer(Thread*)+0x1c2
V  [libjvm.so+0xe73a37]  InstanceKlass::initialize_impl(Thread*)+0x457
V  [libjvm.so+0xe73e3b]  InstanceKlass::initialize(Thread*)+0x7b
V  [libjvm.so+0x11edd64]  LinkResolver::resolve_static_call(CallInfo&, LinkInfo const&, bool, Thread*)+0xd4
V  [libjvm.so+0xebc4ee]  JavaCalls::call_static(JavaValue*, Klass*, Symbol*, Symbol*, Thread*)+0x15e
V  [libjvm.so+0x10c1a76]  JVMCIRuntime::callStatic(char const*, char const*, char const*, JavaCallArguments*, Thread*)+0x166
V  [libjvm.so+0x10c32d7]  JVMCIRuntime::initialize_JVMCI(Thread*)+0x147
V  [libjvm.so+0x10c3615]  JVMCIRuntime::get_HotSpotJVMCIRuntime(Thread*)+0x15
V  [libjvm.so+0x105b76d]  JVMCICompiler::compile_method(methodHandle const&, int, JVMCIEnv*)+0x18d
V  [libjvm.so+0xa90017]  CompileBroker::invoke_compiler_on_method(CompileTask*)+0xca7
V  [libjvm.so+0xa90401]  CompileBroker::compiler_thread_loop()+0x281
V  [libjvm.so+0x175f4ea]  JavaThread::thread_main_inner()+0x21a
V  [libjvm.so+0x175f6f4]  JavaThread::run()+0x144
V  [libjvm.so+0x148e69a]  thread_native_entry(Thread*)+0xfa

Comments

URL: http://hg.openjdk.java.net/jdk/jdk/rev/4062c2c5f7d7 User: jwilhelm Date: 2018-04-17 16:01:04 +0000
17-04-2018
URL: http://hg.openjdk.java.net/jdk/hs/rev/4062c2c5f7d7 User: amenkov Date: 2018-04-10 22:33:49 +0000
10-04-2018
This is the test issue - it uses cached JNIEnv value instead using a value passed to the callbacks
09-04-2018
That seems a major bug! We should always be operating on the current thread, even if we pass it as an arg to speed access to it. ??? The JNIEnv should always be for the current thread, it should be impossible to get a different thread from it.
06-04-2018
Debugging shown that the cause of the issue is using incorrect JavaThread object. JvmtiExport::post_field_access gets gets JavaThread object as arg (and it's the same as returned by JavaThread::current()) jni_GetFieldID (as other JNI functions - see JNI_ENTRY/JNI_ENTRY_NO_PRESERVE macros) gets JavaThread object by calling JavaThread::thread_from_jni_environment(env) We get the crash when 2 concurrent threads call jni_GetFieldID - 2nd thread calls JavaThread::thread_from_jni_environment(env) and gets JavaThread corresponding to the 1st thread (so it has "_thread_in_vm" state)
05-04-2018
I'm trying to reproduce the issue locally (was not able to perpoduce win-x64 release, now building debug build) Tracing/debugging is the next step. To me using of JvmtiJavaThreadEventTransition looks ok - lifetime of the object is clear accordingly C++ rules.
04-04-2018
All of these failure modes should be "impossible" if the thread-state transition helpers are working correctly and the constructors and destructors executing exactly as we expect. I note this bug was filed around the time that we were making changes to include files and .inline.hpp files, and I have to wonder if we have broken something.
04-04-2018
Thanks Alex, I missed: JvmtiJavaThreadEventTransition jet(thread); <= uses ThreadInVMFromNative It seems odd to have a RAII object positioned where it is - ie the block scope is not completely clear and I'm not 100% sure I'd trust the compiler to keep the code in the required order. We need to add tracing to see exactly what the thread state actually is at each step to see where things go wrong.
04-04-2018
Attached hs_err.log for Win-x64 (win_x64_hs_err_pid5344.log)
04-04-2018
rerun the tests: http://java.se.oracle.com:10065/mdash/jobs/amenkov-hs_3-20180404-1802-17341?search=result.status%3AEXECUTED_WITH_FAILURE got the same results
04-04-2018
JvmtiExport::post_field_access DOES change the state before calling callback: JvmtiJavaThreadEventTransition jet(thread); <-- change the state jvmtiEventFieldAccess callback = env->callbacks()->FieldAccess; if (callback != NULL) { (*callback)(env->jvmti_external(), jem.jni_env(), jem.jni_thread(), jem.jni_methodID(), jem.location(), field_jclass, field_jobject, field); }
04-04-2018
The test fails on linux/macos/windows, passes on Solaris On Linux and MACOSX it fails with the same "assert(thread->thread_state() == _thread_in_native)" On Windows the failure is different: # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (t:\\workspace\\open\\src\\hotspot\\share\\runtime/jniHandles.inline.hpp:54), pid=43656, tid=25268 # assert(!current_thread_in_native()) failed: must not be in native # Unfortunately hs_err_*.log file is not available
04-04-2018
I agree with David's analysis, this seems to be a bug in the JVMTI code. Since the test was recently added (JDK-8193369), it's probably just bad luck that this shows up during Graal testing only. Moving to hotspot/jvmti.
26-03-2018
initial ILW = 'wrong thread state' assert failure; only with graal as jit, reported with jvmti test; disable graal! = HLM = P3
26-03-2018
I can't see how this test can ever pass: V [libjvm.so+0xfa5757] jni_GetFieldID+0xa7 <= requires _thread_in_native C [libFieldAccessWatch.so+0xbaf] handleNotification+0x20f <= C code is native and oblivious to thread state V [libjvm.so+0x11665fa] JvmtiExport::post_field_access(JavaThread, Method, unsigned char, Klass, Handle, _jfieldID)+0x1fa <= doesn't change state before calling agent library V [libjvm.so+0xe9a083] InterpreterRuntime::post_field_access(JavaThread, oopDesc, ConstantPoolCacheEntry)+0x383 <= sets state _thread_in_vm via IRT entry. The JVMTI call path seems to be missing the change from _thread_in_VM to _thread_in_native before calling out to the agent library.
26-03-2018
The issue is observed when running only with Graal as JIT compiler. Steps to reproduce: > jtreg -jdk:JDK-HS_fastdebug -nativepath:jdk.hs/build/linux-x64/images/test/hotspot/jtreg/native -vmoptions:-XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -XX:+TieredCompilation -XX:+UseJVMCICompiler -Djvmci.Compiler=graal serviceability/jvmti/FieldAccessWatch/FieldAccessWatch.java
23-03-2018