JDK-6471769 : Error: assert(_cur_stack_depth == count_frames(),"cur_stack_depth out of sync")
  • Type: Bug
  • Component: hotspot
  • Sub-Component: jvmti
  • Affected Version: hs24,6,7
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2006-09-18
  • Updated: 2014-05-20
  • Resolved: 2014-03-01
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 9
9 b06Fixed
Related Reports
Duplicate :  
Relates :  
Relates :  
Description
The following test was failed in the serviceability nightly:
  nsk/jdi/ReferenceType/defaultStratum/defaultStratum004

Info from Dan:
--------------------------------------------------------------------------
New nsk.quick_jdi failures (from 2006.09.12)
   nsk/jdi/ReferenceType/defaultStratum/defaultStratum004
        This test failed the following assertion:
            Internal Error (src/share/vm/prims/jvmtiThreadState.cpp, 273)
            assert(_cur_stack_depth == count_frames(),
                   "cur_stack_depth out of sync")
        on Solaris AMD64 Server VM (machine vm-v20z-5). This test
        failed the same way in both runs and passed in the 2006.09.11
        run on macine vm-v20z-8. There has been no change in the test,
        the JDK or the VM since the 2006.09.11 run. 
--------------------------------------------------------------------------

I was able to reproduce this bug on the vm-v20z-5.sfbay machine.
Please, see full contents of the files defaultStratum004.out and
hs_err_pid9561.log.hs_err in attachments.

This is the hs_err log output below:

ss45998@vm-v20z-5 hs_err hs_err_pid9561.log | c++filt
#
# An unexpected error has been detected by Java Runtime Environment:
#
#  Internal Error (/net/prt-solamd64-q1-2/PrtBuildDir/workspace/src/share/vm/prims/jvmtiThreadState.cpp, 273), pid=9561, tid=2
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20060914135846.dcubed.service_hs_b02_merge.2-debug compiled mode)
#
# Error: assert(_cur_stack_depth == count_frames(),"cur_stack_depth out of sync")
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x000000000044f800):  JavaThread "SDEDebuggee_mainThread" [_thread_in_vm, id=2]

Stack: [0xfffffd7ffd06a000,0xfffffd7ffd16a000),  sp=0xfffffd7ffd168550,  free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1217206];;  void VMError::report_and_die()+0x606
V  [libjvm.so+0x667841];;  void report_assertion_failure(const char*,int,const char*)+0x61
V  [libjvm.so+0xca9557];;  int JvmtiThreadState::cur_stack_depth()+0x6e7
V  [libjvm.so+0xc1caaa];;  void JvmtiExport::post_method_exit(JavaThread*,methodOop,frame)+0x21ba
V  [libjvm.so+0x8385de];;  void InterpreterRuntime::post_method_exit(JavaThread*)+0x21e
j  java.io.ObjectInputStream$BlockDataInputStream.getBlockDataMode()Z+4
j  java.io.ObjectInputStream.readObject0(Z)Ljava/lang/Object;+4
j  java.io.ObjectInputStream.readObject()Ljava/lang/Object;+19

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.io.ObjectInputStream$BlockDataInputStream.getBlockDataMode()Z+4
j  java.io.ObjectInputStream.readObject0(Z)Ljava/lang/Object;+4
j  java.io.ObjectInputStream.readObject()Ljava/lang/Object;+19
J  nsk.share.jpda.SocketConnection.doReadObject()Ljava/lang/Object;
J  nsk.share.jpda.SocketConnection.readObject()Ljava/lang/Object;
j  nsk.share.jpda.IOPipe.readln()Ljava/lang/String;+15
j  nsk.share.jpda.AbstractDebuggeeTest.doTest()V+55
J  nsk.share.jpda.AbstractDebuggeeTest.doTest([Ljava/lang/String;)V
j  nsk.jdi.ReferenceType.defaultStratum.defaultStratum004.defaultStratum004a.main([Ljava/lang/String;)V+8
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------
.........................<snip>......................

VM Arguments:
jvm_args: -Xcomp -XX:-PrintVMOptions -XX:CompileOnly=nsk -DHANGINGJAVA12468 -Xdebug -Xrunjdwp:transport=dt_socket,address=vm-v20z-5:62901,suspend=y
java_command: nsk.jdi.ReferenceType.defaultStratum.defaultStratum004.defaultStratum004a -testClassPath /export/gtee/Work/exec/nsk.quick_jdi-13-NIGHTLY-Serv_Baseline-ServerVM-comp-64BITSOLARIS-AMD64-2006-09-13-19-13-24/run2/gridadm.Solaris.x86/defaultStratum004 -arch=solaris-amd64 -waittime=2 -debugee.vmkind=java -transport.address=dynamic -debugee.vmkeys=-d64 -server -Xcomp -XX:-PrintVMOptions -XX:CompileOnly=nsk -DHANGINGJAVA12468 -pipe.port=62900
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=/net/gtee.sfbay/export/nightly/mustang/JDK/service_hs_baseline/jdk1.6/solaris-amd64
CLASSPATH=/export/gtee/Work/exec/nsk.quick_jdi-13-NIGHTLY-Serv_Baseline-ServerVM-comp-64BITSOLARIS-AMD64-2006-09-13-19-13-24/run2/gridadm.Solaris.x86/defaultStratum004:/net/gtee.sfbay/export/gtee/suites/testbase_vm.1.6/vm/bin/classes:/net/gtee.sfbay/export/nightly/mustang/JDK/service_hs_baseline/jdk1.6/solaris-amd64/lib/tools.jar
PATH=/net/gtee.sfbay/export/nightly/mustang/JDK/service_hs_baseline/jdk1.6/solaris-amd64/bin:/bin:/usr/bin:/net/gtee.sfbay/export/nightly/mustang/JDK/service_hs_baseline/jdk1.6/solaris-amd64/jre/bin:/mksnt
LD_LIBRARY_PATH=/net/gtee.sfbay/export/nightly/mustang/JDK/service_hs_baseline/jdk1.6/solaris-amd64/jre/lib/amd64/server:/net/gtee.sfbay/export/nightly/mustang/JDK/service_hs_baseline/jdk1.6/solaris-amd64/jre/lib/amd64:/net/gtee.sfbay/export/nightly/mustang/JDK/service_hs_baseline/jdk1.6/solaris-amd64/jre/../lib/amd64:/net/gtee.sfbay/export/nightly/mustang/JDK/service_hs_baseline/jdk1.6/solaris-amd64/jre/lib/amd64:/net/gtee.sfbay/export/nightly/mustang/JDK/service_hs_baseline/jdk1.6/solaris-amd64/jre/lib/amd64/server
SHELL=/usr/bin/sh
DISPLAY=vmsqe.sfbay:0.0

Signal Handlers:
SIGSEGV: [libjvm.so+0x1218320], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGBUS: [libjvm.so+0x1218320], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGFPE: [libjvm.so+0xe6b450], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGPIPE: [libjvm.so+0xe6b450], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGILL: [libjvm.so+0xe6b450], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGHUP: [libjvm.so+0xe66550], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGINT: [libjvm.so+0xe66550], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGQUIT: [libjvm.so+0xe66550], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0xe66550], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGQUIT: [libjvm.so+0xe66550], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0xe66550], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIG39: [libjvm.so+0xe6b460], sa_mask[0]=0x00000000, sa_flags=0x00000008
SIG40: [libjvm.so+0xe6b450], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c


---------------  S Y S T E M  ---------------

OS:                        Solaris 10 10/05 s10x_u1wos_11 X86
           Copyright 2005 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                             Assembled 20 July 2005

uname:SunOS 5.10 Generic_Patch_118844-30 i86pc  (T2 libthread)
rlimit: STACK 10240k, CORE 0k, NOFILE 65536, AS infinity
load average:0.36 0.10 0.03

CPU:total 2 amd64 3dnow

Memory: 4k page, physical 4127784k(3289664k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (20060914135846.dcubed.service_hs_b02_merge.2) for solaris-amd64, built on Sep 14 2006 14:38:05 by "PRT" with unknown Workshop:0x580
Edit out description from a different bug.
Here is the assert from the hs_err file from a more recent failure to
make the new failure matching feature happier:

http://sqeweb.sfbay/nfs/tools/gtee/results/JDK7/NIGHTLY/VM/2010-08-31/RT_Baseline_closed/vm/solaris-sparcv9/server/mixed/solaris-sparcv9_vm_server_compd_nsk.quick-jdi.testlist/analysis.html

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/tmp/jprt/P1/B/221616.phh/source/src/share/vm/prims/jvmtiThreadState.cpp:287), pid=1342, tid=2
#  assert(_cur_stack_depth == count_frames()) failed: cur_stack_depth out of sync
#
# JRE version: 7.0
# Java VM: Java HotSpot(TM) 64-Bit Server VM (19.0-b06-201008312216.phh.hotspot-rt-closed-push-fastdebug compiled mode solaris-sparc )
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
In my baseline testing on Solaris X64 for JDK8-B26, I observed this
failure with the following VM/NSK test:

     nsk/jdi/EventRequestManager/stepRequests/stepreq001

Here is a snippet from the hs_err_pid file:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/export/HUDSON/workspace/jdk8-2-build-solaris-amd64-product/jdk8/hotspot/src/share/vm/prims/jvmtiThreadState.cpp:293), pid=26993, tid=2
#  assert(_cur_stack_depth == count_frames()) failed: cur_stack_depth out of sync
#
# JRE version: 8.0-b26
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.0-b15-fastdebug compiled mode s
olaris-amd64 compressed oops)
# Core dump written. Default location: /work/shared/test_results/jdk8/baseline/s
olaris-x64/b26/vm-jdi-prod-server-fast-comp.solaris-x64/dcubed.SunOS.x86/stepreq
001/core or core.26993
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x0000000000441000):  JavaThread "main_thr" [_thread_in_vm, id=2
, stack(0xfffffd7ffbb7f000,0xfffffd7ffbc7f000)]

Stack: [0xfffffd7ffbb7f000,0xfffffd7ffbc7f000],  sp=0xfffffd7ffbc7c300,  free sp
ace=1012k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x286e53c]  void VMError::report(outputStream*)+0x8c8
V  [libjvm.so+0x286f6ad]  void VMError::report_and_die()+0x4fd
V  [libjvm.so+0xf1ac1f]  void report_vm_error(const char*,int,const char*,const
char*)+0x55f
V  [libjvm.so+0x1d38ef4]  int JvmtiThreadState::cur_stack_depth()+0x6b8
V  [libjvm.so+0x1b18f1d]  void JvmtiExport::post_method_exit(JavaThread*,methodO
op,frame)+0x2b4d
V  [libjvm.so+0x15d6941]  void InterpreterRuntime::post_method_exit(JavaThread*)
+0x4b1
j  java.io.ObjectInputStream$BlockDataInputStream.getBlockDataMode()Z+4
j  java.io.ObjectInputStream.readObject0(Z)Ljava/lang/Object;+4
j  java.io.ObjectInputStream.readObject()Ljava/lang/Object;+19
J  nsk.share.jpda.SocketConnection.doReadObject()Ljava/lang/Object;

[error occurred during error reporting (printing native stack), id 0xe0000000]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.io.ObjectInputStream$BlockDataInputStream.getBlockDataMode()Z+4
j  java.io.ObjectInputStream.readObject0(Z)Ljava/lang/Object;+4
j  java.io.ObjectInputStream.readObject()Ljava/lang/Object;+19
J  nsk.share.jpda.SocketConnection.doReadObject()Ljava/lang/Object;
J  nsk.share.jpda.SocketConnection.readObject()Ljava/lang/Object;
J  nsk.share.jpda.SocketIOPipe.readln()Ljava/lang/String;
j  nsk.jdi.EventRequestManager.stepRequests.stepreq001t.runThis([Ljava/lang/Stri
ng;)I+366
J  nsk.jdi.EventRequestManager.stepRequests.stepreq001t.main([Ljava/lang/String;
)V
v  ~StubRoutines::call_stub

Comments
The suggested fix is: http://javaweb.sfbay.sun.com/java/svc/ss45998/webrevs/2014/hotspot/6471769-JVMTI-DEPTH.1
25-02-2014

I've deleted my 4 last comments. They are incorrect as they were based on the same mistake. I have to investigate this issue a little bit more.
04-02-2014

My last comment above is wrong. The JvmtiEnv::is_thread_fully_suspended() checks if the target thread is current JavaThread.
31-01-2014

I can not reproduce this issue yet but have some good guess why this could happen. The JVMTI GetFrameCount spec tells this: " If this function is called for a thread actively executing bytecodes (for example, not the current thread and not suspended), the information returned is transient." But the JvmtiThreadState::count_frames() threats it a little bit differently. This is the assert from the function: assert(SafepointSynchronize::is_at_safepoint() || JvmtiEnv::is_thread_fully_suspended(get_thread(), false, &debug_bits), "at safepoint or must be suspended"); The assert above misses the condition: javaThread::current() == get_thread(). Counting frames at a safepoint should be Ok in general, but it is a way to get the count out-of-sync with the current thread. Still investigating this issue to get a better picture of what is really happening.
14-01-2014

RULE com/sun/jdi/ConnectedVMs.java Crash assert(_cur_stack_depth == count_frames()) failed: cur_stack_depth out of sync
08-05-2013

Is there an evaluation for this bug?
07-05-2013