JDK-8302350 : JfrThreadSampler failed with "assert((is_native() && bci == 0) || (!is_native() && 0 <= bci && bci < code_size())) failed: illegal bci: 0 for non-native method"
  • Type: Bug
  • Component: hotspot
  • Sub-Component: jfr
  • Affected Version: 17,21
  • Priority: P4
  • Status: Closed
  • Resolution: Duplicate
  • OS: linux
  • CPU: aarch64
  • Submitted: 2023-02-13
  • Updated: 2025-05-30
  • Resolved: 2025-05-30
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 25
25Resolved
Related Reports
Blocks :  
JDK-8321098 - Cooperative JFR Sampling
Blocks :  
JDK-8352251 - Implement JEP 518: JFR Cooperative Sampling
Duplicate :  
JDK-8352251 - Implement JEP 518: JFR Cooperative Sampling
Relates :  
JDK-8303153 - Native interpreter frame missing mirror
Description
The following test failed in the JDK21 CI:

applications/kitchensink/Kitchensink.java

Here's a snippet from the log file:

[2023-02-13T03:48:04.177560956Z] Gathering output for process 368870
[2023-02-13T03:48:10.366975074Z] Waiting for completion for process 368870
[2023-02-13T03:48:10.367574516Z] Waiting for completion finished for process 368870
Output and diagnostic info for process 368870 was saved into 'pid-368870-output.log'
[stress.process.out] #
[stress.process.out] # A fatal error has been detected by the Java Runtime Environment:
[stress.process.out] #
[stress.process.out] #  Internal Error (/opt/mach5/mesos/work_dir/slaves/91e16c40-06d4-468a-9fc3-7198a5bb7d5a-S115958/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/d33156e2-7f16-4ec2-b8c1-417dd13cf6b9/runs/14ec3492-7550-4e7a-a6a2-ecca47ce0ffd/workspace/open/src/hotspot/share/oops/method.cpp:364), pid=365129, tid=365229
[stress.process.out] #  assert((is_native() && bci == 0) || (!is_native() && 0 <= bci && bci < code_size())) failed: illegal bci: 0 for non-native method
[stress.process.out] #
[stress.process.out] # JRE version: Java(TM) SE Runtime Environment (21.0+10) (fastdebug build 21-ea+10-LTS-732)
[stress.process.out] # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 21-ea+10-LTS-732, mixed mode, sharing, tiered, compressed class ptrs, z gc, linux-aarch64)
[stress.process.out] # Problematic frame:
[stress.process.out] # V  [libjvm.so+0x1418518]  Method::bcp_from(int) const+0x74
[stress.process.out] #
[stress.process.out] # Core dump will be written. Default location: Core dumps may be processed with "/opt/core.sh %p" (or dumping to /opt/mach5/mesos/work_dir/slaves/0c72054a-24ab-4dbb-944f-97f9341a1b96-S139143/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/c1354abf-61a1-4c38-b27d-37fe7f5e43d3/runs/97979103-550d-420c-980a-60a037689058/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_kitchensink_Kitchensink_java/scratch/0/core.365129)
[stress.process.out] #
[stress.process.out] # JFR recording file will be written. Location: /opt/mach5/mesos/work_dir/slaves/0c72054a-24ab-4dbb-944f-97f9341a1b96-S139143/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/c1354abf-61a1-4c38-b27d-37fe7f5e43d3/runs/97979103-550d-420c-980a-60a037689058/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_kitchensink_Kitchensink_java/scratch/0/hs_err_pid365129.jfr
[stress.process.out] #
[stress.process.out] Unsupported internal testing APIs have been used.
[stress.process.out] 
[stress.process.out] # An error report file with more information is saved as:
[stress.process.out] # /opt/mach5/mesos/work_dir/slaves/0c72054a-24ab-4dbb-944f-97f9341a1b96-S139143/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/c1354abf-61a1-4c38-b27d-37fe7f5e43d3/runs/97979103-550d-420c-980a-60a037689058/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_kitchensink_Kitchensink_java/scratch/0/hs_err_pid365129.log
[stress.process.out] [thread 365180 also had an error]
[stress.process.out] [thread 369572 also had an error]
[stress.process.out] #
[stress.process.out] # If you would like to submit a bug report, please visit:
[stress.process.out] #   https://bugreport.java.com/bugreport/crash.jsp
[stress.process.out] #
[2023-02-13T03:49:10.372692689Z] Gathering output for process 369577
[2023-02-13T03:49:10.380694914Z] Waiting for completion for process 369577
[2023-02-13T03:49:10.380823315Z] Waiting for completion finished for process 369577

Here's the crashing thread's stack:

---------------  T H R E A D  ---------------

Current thread (0x0000fffd002e3e70):  JfrThreadSampler "JFR Thread Sampler" [stack: 0x0000fffc95f6f000,0x0000fffc9616f000] [id=365229] _threads_hazard_ptr=0x0000fffc84089b10, _nested_threads_hazard_ptr_cnt=0

Stack: [0x0000fffc95f6f000,0x0000fffc9616f000],  sp=0x0000fffc9616c8b0,  free space=2038k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1418518]  Method::bcp_from(int) const+0x74  (method.cpp:364)
V  [libjvm.so+0x1418658]  Method::validate_bci_from_bcp(unsigned char*) const+0x78  (method.cpp:359)
V  [libjvm.so+0xbfae0c]  frame::is_interpreted_frame_valid(JavaThread*) const+0x2bc  (frame_aarch64.cpp:528)
V  [libjvm.so+0xf093e4]  JfrGetCallTrace::find_top_frame(frame&, Method**, frame&)+0xe4  (jfrCallTrace.cpp:55)
V  [libjvm.so+0xf09e50]  JfrGetCallTrace::get_topframe(void*, frame&)+0x80  (jfrCallTrace.cpp:113)
V  [libjvm.so+0xfbce08]  OSThreadSampler::protected_task(SuspendedThreadTaskContext const&)+0x178  (jfrThreadSampler.cpp:192)
V  [libjvm.so+0x16cd384]  SuspendedThreadTask::internal_do_task()+0x44  (signals_posix.cpp:1836)
V  [libjvm.so+0x17b1d44]  SuspendedThreadTask::run()+0x14  (suspendedThreadTask.cpp:30)
V  [libjvm.so+0xfbcf6c]  JfrThreadSampleClosure::sample_thread_in_java(JavaThread*, JfrStackFrame*, unsigned int)+0x78  (jfrThreadSampler.cpp:209)
V  [libjvm.so+0xfbded8]  JfrThreadSampleClosure::do_sample_thread(JavaThread*, JfrStackFrame*, unsigned int, JfrSampleType)+0x3e8  (jfrThreadSampler.cpp:407)
V  [libjvm.so+0xfc20c4]  JfrThreadSampler::task_stacktrace(JfrSampleType, JavaThread**) [clone .constprop.1]+0x244  (jfrThreadSampler.cpp:624)
V  [libjvm.so+0xfc2a68]  JfrThreadSampler::run()+0x1c8  (jfrThreadSampler.cpp:558)
V  [libjvm.so+0x1818338]  Thread::call_run()+0xf8  (thread.cpp:224)
V  [libjvm.so+0x152e764]  thread_native_entry(Thread*)+0x100  (os_linux.cpp:737)
C  [libc.so.6+0x86478]  start_thread+0x2d4
Comments
# # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (/System/Volumes/Data/mesos/work_dir/slaves/8c556dc5-c20e-43f0-95f7-56473b5aa3a5-S2014/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/aaec6476-f0b5-4271-b059-264d8416fdda/runs/a1cae671-67ef-4995-acdb-9c579b6e3e7c/workspace/open/src/hotspot/share/oops/method.cpp:362), pid=43048, tid=48159 # assert((is_native() && bci == 0) || (!is_native() && 0 <= bci && bci < code_size())) failed: illegal bci: 0 for non-native method # # JRE version: Java(TM) SE Runtime Environment (25.0+9) (fastdebug build 25-ea+9-LTS-906) # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 25-ea+9-LTS-906, mixed mode, tiered, compressed class ptrs, z gc, bsd-aarch64) # Core dump will be written. Default location: core.43048 # # JFR recording file will be written. Location: /System/Volumes/Data/mesos/work_dir/slaves/8c556dc5-c20e-43f0-95f7-56473b5aa3a5-S2333/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1dc04df3-a213-4bb1-8b77-1e659f5922ad/runs/1b71d416-0f9f-4abf-9f24-1ff76b0c6ec2/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_runthese_RunThese30M_java/scratch/0/hs_err_pid43048.jfr # # If you would like to submit a bug report, please visit: # https://bugreport.java.com/bugreport/crash.jsp # --------------- S U M M A R Y ------------ Command Line: -Xbootclasspath/a:/System/Volumes/Data/mesos/work_dir/slaves/8c556dc5-c20e-43f0-95f7-56473b5aa3a5-S2333/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1dc04df3-a213-4bb1-8b77-1e659f5922ad/runs/1b71d416-0f9f-4abf-9f24-1ff76b0c6ec2/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_runthese_RunThese30M_java/scratch/0/wb.jar -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -XX:MaxRAMPercentage=6.25 -Dtest.boot.jdk=/System/Volumes/Data/mesos/work_dir/jib-master/install/jdk/23/37/bundles/macos-aarch64/jdk-23_macos-aarch64_bin.tar.gz/jdk-23.jdk/Contents/Home -Djava.io.tmpdir=/System/Volumes/Data/mesos/work_dir/slaves/8c556dc5-c20e-43f0-95f7-56473b5aa3a5-S2333/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1dc04df3-a213-4bb1-8b77-1e659f5922ad/runs/1b71d416-0f9f-4abf-9f24-1ff76b0c6ec2/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_runthese_RunThese30M_java/tmp -XX:+UseZGC -XX:MaxRAMPercentage=50 -Djava.net.preferIPv6Addresses=false -XX:+DisplayVMOutputToStderr -Xlog:gc*,gc+heap=debug:gc.log:uptime,timemillis,level,tags -XX:+DisableExplicitGC -XX:+StartAttachListener -XX:CompileCommand=memlimit,*.*,0 -Xlog:monitorinflation=info:file=../monitorinflation.log::filesize=500m -Djava.io.tmpdir=/System/Volumes/Data/mesos/work_dir/slaves/8c556dc5-c20e-43f0-95f7-56473b5aa3a5-S2333/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1dc04df3-a213-4bb1-8b77-1e659f5922ad/runs/1b71d416-0f9f-4abf-9f24-1ff76b0c6ec2/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_runthese_RunThese30M_java/scratch/0/java.io.tmpdir -Duser.home=/System/Volumes/Data/mesos/work_dir/slaves/8c556dc5-c20e-43f0-95f7-56473b5aa3a5-S2333/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1dc04df3-a213-4bb1-8b77-1e659f5922ad/runs/1b71d416-0f9f-4abf-9f24-1ff76b0c6ec2/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_runthese_RunThese30M_java/scratch/0/user.home -agentpath:/System/Volumes/Data/mesos/work_dir/jib-master/install/jdk-25+9-906/macosx-aarch64-debug.test/hotspot/jtreg/native/libJvmtiStressModule.dylib -Xverify:all -javaagent:redefineagent.jar -XX:NativeMemoryTracking=detail -Djdk.test.lib.random.seed=-2288451355969663145 applications.kitchensink.process.stress.Main /System/Volumes/Data/mesos/work_dir/slaves/8c556dc5-c20e-43f0-95f7-56473b5aa3a5-S2333/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/1dc04df3-a213-4bb1-8b77-1e659f5922ad/runs/1b71d416-0f9f-4abf-9f24-1ff76b0c6ec2/testoutput/test-support/jtreg_closed_test_hotspot_jtreg_applications_runthese_RunThese30M_java/scratch/0/kitchensink.final.properties Host: "Macmini9,1" arm64, 8 cores, 16G, Darwin 21.6.0, macOS 12.6.2 (21G317) Time: Mon Feb 3 17:35:53 2025 GMT elapsed time: 2834.408421 seconds (0d 0h 47m 14s) --------------- T H R E A D --------------- Current thread (0x00000001234f2010): JfrThreadSampler "JFR Thread Sampler" [id=48159, stack(0x0000000176a3c000,0x0000000176c3f000) (2060K)] _threads_hazard_ptr=0x000060001f0f2510, _nested_threads_hazard_ptr_cnt=0 Stack: [0x0000000176a3c000,0x0000000176c3f000], sp=0x0000000176c3df50, free space=2055k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.dylib+0x1161554] VMError::report(outputStream*, bool)+0x1aac (method.cpp:362) V [libjvm.dylib+0x1164b40] VMError::report_and_die(int, char const*, char const*, char*, Thread*, unsigned char*, void const*, void const*, char const*, int, unsigned long)+0x548 V [libjvm.dylib+0x57820c] print_error_for_unit_test(char const*, char const*, char*)+0x0 V [libjvm.dylib+0xd8b658] Method::validate_bci_from_bcp(unsigned char*) const+0x194 V [libjvm.dylib+0x67bef4] frame::is_interpreted_frame_valid(JavaThread*) const+0x128 V [libjvm.dylib+0x8d0070] JfrGetCallTrace::find_top_frame(frame&, Method**, frame&)+0x144 V [libjvm.dylib+0x8d037c] JfrGetCallTrace::get_topframe(void*, frame&)+0x8c V [libjvm.dylib+0x947e14] OSThreadSampler::protected_task(SuspendedThreadTaskContext const&)+0xc8 V [libjvm.dylib+0xf97d98] SuspendedThreadTask::internal_do_task()+0x44 V [libjvm.dylib+0x10473f4] SuspendedThreadTask::run()+0x14 V [libjvm.dylib+0x948100] JfrThreadSampleClosure::sample_thread_in_java(JavaThread*, JfrStackFrame*, unsigned int)+0x7c V [libjvm.dylib+0x9488ec] JfrThreadSampleClosure::do_sample_thread(JavaThread*, JfrStackFrame*, unsigned int, JfrSampleType)+0x154 V [libjvm.dylib+0x9495e0] JfrThreadSampler::task_stacktrace(JfrSampleType, JavaThread**)+0x364 V [libjvm.dylib+0x949230] JfrThreadSampler::run()+0x120 V [libjvm.dylib+0x10aeb08] Thread::call_run()+0xf4 V [libjvm.dylib+0xe48624] thread_native_entry(Thread*)+0x138 C [libsystem_pthread.dylib+0x726c] _pthread_start+0x94
03-02-2025
I think this bug could be solved by the proposed patch in JDK-8321098
04-12-2023
[~eosterlund] mentioned that when a load barrier slow path is taken, a JRT_LEAF call will be made, but before that, a lot of registers are being pushed before the call. See ZBarrierSetAssembler::load_at(). The issue here seems to be that there is an enter() that is executed on AARCH64, but not on x86, before the registers are saved. The enter() updates the fp, i.,e. constructing a new frame, but the instruction pointer is still in the interpreter. When the sampler code finds the ip from the CPU context, it is an address in the interpreter.. As such it looks at the fp (also from CPU context), to build a frame from that address, but the enter() that was issued has updated the fp, so a frame that is not an interpreter frame (it's just the register saves before the VM call), is now treated as such. This probably works ok on x86 because there is no enter() there, and so the correct frame would be constructed (from the unchanged fp, even though the sp has moved).
30-05-2023
The reason this problem occurs in the interpreter code on AARCH64 and RISC is because of XBarrierSetAssembler::load_at(): Implementation on those platforms contains an enter() and a leave() that creates another frame, i.e. most importantly, it destroys the frame pointer (which, up to that point, is pointing correctly at the interpreter frame fp). Note that the x86 implementation has no enter() and leave(). It's ok to save these registers before calling into a LEAF_CALL as long as you leave the fp intact, i.e. you are running in an interpreter frame; if we find an address from the CPU context that is in the interpreter, the next step is to start inspecting the interpreter frame which is always done from the fp (now fetched from the CPU context). So the solution to this problem is not creating that additional frame on AARCH64 and RISC. At least when it comes to the interpreter. I don't know how much sharing there is between the compiler and the interpreter for this barrier code.
30-05-2023
Spoke to [~mgronlun] and he promised to have a look.
23-05-2023
There is no way of fixing this in the stackwalker code. The premise is that the nmethod declares how large the frame is. If you need to spill additional registers before a JRT_LEAF call, you must do so in a separate stub frame.
15-05-2023
So the stack walker makes assumptions that nobody is spilling registers before calling JRT_LEAF functions (that by definition are not supposed to be walkable). The ZGC code spills some registers before making a JRT_LEAF call. We can of course argue that the ZGC code should change to comply with the expectations of the stack walker. But honestly, it seems like the stack walker is making a very fragile assumption that you can't spill registers before JRT_LEAF calls. I suspect we will find that ZGC isn't the only one spilling registers before a JRT_LEAF call. Therefore, I think the fix should really go into the stack tracer code.
09-05-2023
Please see attachment Stack_Weird.jpg for details.
23-02-2023
Moving this to the GC team for more analysis and inspection.
23-02-2023
// Core dig out Pool holder (InstanceKlass): (gdb) p &*(*this)->_constMethod->_constants->_pool_holder->_name._body $31 = (u1 *) 0x80051c57e "java/util/stream/Stream" Method name: p &(*(*this)->_name._body) $52 = (u1 *) 0x8005212be "filter" Full name: java/util/stream/Stream.filter(Ljava/util/function/Predicate;)Ljava/util/stream/Stream; Access flags: p (*this)->_access_flags $63 = {_flags = 1025} == 0x0000401 // where [0x400 == JVM_ACC_ABSTRACT] An abstract method has also a _code_size == 0. p (*this)->_constMethod->_code_size $64 = 0 How come the stack walking code on aarch64 looked up and returned an abstract method?
23-02-2023
How strange is this? assert((is_native() && bci == 0) || (!is_native() && 0 <= bci && bci < code_size())) failed: illegal bci: 0 for non-native method So it's not a native method, the bci is 0. The only way for that branch to be taken is if the code_size() == 0 or is negative.
22-02-2023