United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-6956958 assert(is_clean() || is_call_to_compiled() || is_call_to_interpreted() || is_optimized() || is_megam
JDK-6956958 : assert(is_clean() || is_call_to_compiled() || is_call_to_interpreted() || is_optimized() || is_megam

Details
Type:
Bug
Submit Date:
2010-05-28
Status:
Closed
Updated Date:
2011-04-23
Project Name:
JDK
Resolved Date:
2011-04-23
Component:
hotspot
OS:
generic
Sub-Component:
compiler
CPU:
generic
Priority:
P2
Resolution:
Fixed
Affected Versions:
hs19
Fixed Versions:
hs19 (b04)

Related Reports
Backport:
Backport:
Backport:
Backport:
Duplicate:
Relates:
Relates:
Relates:

Sub Tasks

Description
The following VM/NSK test failed an assertion in the 2010.05.27 nightly:

    nsk/monitoring/ThreadMXBean/ThreadInfo/TimedWaitingThread/TimedWaitingThread002

Here is a snippet of the assertion failure:

    Internal Error (src/share/vm/code/compiledIC.cpp:617), pid=11439, tid=12
    assert(is_clean() || is_call_to_compiled() || is_call_to_interpreted() ||
        is_optimized() || is_megamorphic()) failed: sanity check

This is just a bug/failure sighting. The assertion failure was only seen
in one test in one configuration. I have no idea (yet) how reproducible
this failure is.

Here is a link to the failing configuration:

http://sqeweb.sfbay.sun.com/nfs/results/vm/gtee/JDK7/NIGHTLY/VM/2010-05-27/RT_Baseline/vm/solaris-sparcv9/server/comp/solaris-sparcv9_server_comp_nsk.quick-monitoring.testlist/analysis.html


Here is a snippet from the hs_err file:

---------------  T H R E A D  ---------------

Current thread (0x00000001005b2800):  JavaThread "CompilerThread1" daemon [_thread_in_vm, id=12, stack(0xffffffff60c00000,0xffffffff60d00000)]

Stack: [0xffffffff60c00000,0xffffffff60d00000],  sp=0xffffffff60cfef50,  free space=3fbffffffff7e4cfa40k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xeb10dc];;  __1cHVMErrorOreport_and_die6M_v_+0x6bc
V  [libjvm.so+0x504280];;  __1cPreport_vm_error6Fpkci11_v_+0x78
V  [libjvm.so+0x47723c];;  __1cKCompiledICGverify6M_v_+0xa4
V  [libjvm.so+0xb6b1d8];;  __1cHnmethodVcleanup_inline_caches6M_v_+0x2c0
V  [libjvm.so+0xd8ae40];;  __1cONMethodSweeperQsweep_code_cache6F_v_+0x8a0
V  [libjvm.so+0xd8a578];;  __1cONMethodSweeperOpossibly_sweep6F_v_+0x90
V  [libjvm.so+0x467b28];;  __1cMCompileQdDueueDget6M_pnLCompileTask__+0x20
V  [libjvm.so+0x46bb38];;  __1cNCompileBrokerUcompiler_thread_loop6F_v_+0x270
V  [libjvm.so+0xe033fc];;  __1cKJavaThreadRthread_main_inner6M_v_+0x174
V  [libjvm.so+0xe03268];;  __1cKJavaThreadDrun6M_v_+0x310
V  [libjvm.so+0xbc8f2c];;  java_start+0x184
Here is another failure with a very similar call trace. This failure
crashed instead of failing an assert(), but it crashed in the code
that is part of the assert. The VM/NSK test is:

    nsk/jdi/MonitorWaitRequest/addThreadFilter

The failing config is Linux AMD64 Server VM -Xcomp:

Here is a link to the failing config:

http://sqeweb.sfbay.sun.com/nfs/results/vm/gtee/JDK7/NIGHTLY/VM/2010-05-27/RT_Baseline/vm/linux-amd64/server/comp/linux-amd64_server_comp_nsk.quick-jdi.testlist/analysis.html


Here is a snippet of the stack trace from the hs_err file:

Stack: [0x00007f3b98313000,0x00007f3b98414000],  sp=0x00007f3b98412a20,  free space=3fe0000000000000018k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x440876];;  _ZNK10CompiledIC22is_call_to_interpretedEv+0xc6
V  [libjvm.so+0x44132a];;  _ZN10CompiledIC6verifyEv+0xaa
V  [libjvm.so+0x85a192];;  _ZN7nmethod21cleanup_inline_cachesEv+0x222
V  [libjvm.so+0x9cb30b];;  _ZN14NMethodSweeper15process_nmethodEP7nmethod+0x3eb
V  [libjvm.so+0x9cb4de];;  _ZN14NMethodSweeper16sweep_code_cacheEv+0xfe
V  [libjvm.so+0x9cb743];;  _ZN14NMethodSweeper14possibly_sweepEv+0x73
V  [libjvm.so+0x439083];;  _ZN12CompileQueue3getEv+0x13
V  [libjvm.so+0x43d454];;  _ZN13CompileBroker20compiler_thread_loopEv+0x194
V  [libjvm.so+0xa1cc06];;  _ZN10JavaThread17thread_main_innerEv+0xf6
V  [libjvm.so+0x894f00];;  _ZL10java_startP6Thread+0xf0
nsk/jdwp/ClassType/SetValues/setvalues001
nsk/jdwp/Event/THREAD_DEATH/thrdeath001
nsk/jdwp/Event/THREAD_START/thrstart001
nsk/jdwp/ObjectReference/IsCollected/iscollected001
nsk/jdwp/ThreadReference/OwnedMonitorsStackDepthInfo/ownedMonitorsStackDepthInfo002
nsk/jdwp/VirtualMachine/SetDefaultStratum/setdefstrat001

Also affects these tests in rt_baseline.  Too bad that sweeper change seemed cool.
Test
  java/util/PriorityQueue/ForgetMeNot.java
failes with the same message

                                    

Comments
EVALUATION

I think there's a hole in the logic for cleaning up caches as it appears we're dying because an inline cache points at an nmethod which has already been reclaimed.
                                     
2010-06-11
EVALUATION

http://hg.openjdk.java.net/jdk7/hotspot-comp/hotspot/rev/ff38d05ea86f
                                     
2010-06-19
EVALUATION

6956958: assert(is_clean() || is_call_to_compiled() || is_call_to_interpreted() || is_optimized() || is_megam
Reviewed-by: kvn

When the sweeper scans the code cache it divides it up into
NMethodSweepFraction chunks and processes one chunk per sweep.  Some
changes late in the review of the concurrent sweeper changed the
interation to skip by nmethods instead of CodeBlobs and during a
change late in the review some code which was only supposed to make
sure we had stopped on an nmethod was changed into skipping to the
next nmethod.  This meant that sweeper skipped processing the last
nmethod of each chunk.  Each nmethod is processed multiple times so
most of the time this was fine but occasionally the stars aligned and
the same nmethod was skipped every time.  This could leave it pointing
at a dead nmethod which could cause failures of the sort we were
seeing.  Logging code I added for a debug build showed precisely these
events occurring when the crash happened.  The fix is simply to remove
the extra call to next.

While investigating this I found several other issues that are
potential problems but they all appeared pretty benign.  I'm going to
fix those issues another another bug since they are somewhat larger
changes.
                                     
2010-07-12



Hardware and Software, Engineered to Work Together