United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-7018355 JSR 292: VM crash in DefNewGeneration::copy_to_survivor_space
JDK-7018355 : JSR 292: VM crash in DefNewGeneration::copy_to_survivor_space

Details
Type:
Bug
Submit Date:
2011-02-09
Status:
Closed
Updated Date:
2012-02-01
Project Name:
JDK
Resolved Date:
2011-05-10
Component:
hotspot
OS:
generic
Sub-Component:
compiler
CPU:
generic
Priority:
P2
Resolution:
Fixed
Affected Versions:
hs20
Fixed Versions:
hs21 (b11)

Related Reports
Backport:
Duplicate:
Duplicate:
Duplicate:
Duplicate:
Relates:

Sub Tasks

Description
VM crashed during JDK7 b126 promotion testing:

#  SIGSEGV (0xb) at pc=0xb7892e22, pid=20387, tid=3039271824
#
# JRE version: 7.0-b126
# Java VM: Java HotSpot(TM) Client VM (20.0-b06 mixed mode, sharing linux-x86 )

Stack: [0xb51f9000,0xb527a000],  sp=0xb5278b80,  free space=510k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1ade22]  DefNewGeneration::copy_to_survivor_space(oopDesc*)+0x12;;  DefNewGeneration::copy_to_survivor_space(oopDesc*)+0x12
V  [libjvm.so+0x1ae28c]  FastScanClosure::do_oop(oopDesc**)+0x4c;;  FastScanClosure::do_oop(oopDesc**)+0x4c
V  [libjvm.so+0x201549]  HandleArea::oops_do(OopClosure*)+0x39;;  HandleArea::oops_do(OopClosure*)+0x39
V  [libjvm.so+0x3e90df]  Thread::oops_do(OopClosure*, CodeBlobClosure*)+0x3f;;  Thread::oops_do(OopClosure*, CodeBlobClosure*)+0x3f
V  [libjvm.so+0x3ea999]  JavaThread::oops_do(OopClosure*, CodeBlobClosure*)+0x29;;  JavaThread::oops_do(OopClosure*, CodeBlobClosure*)+0x29
V  [libjvm.so+0x3ead64]  Threads::oops_do(OopClosure*, CodeBlobClosure*)+0x34;;  Threads::oops_do(OopClosure*, CodeBlobClosure*)+0x34
V  [libjvm.so+0x389c44]  SharedHeap::process_strong_roots(bool, bool, SharedHeap::ScanningOption, OopClosure*, CodeBlobClosure*, OopsInGenClosure*)+0x214;;  SharedHeap::process_strong_roots(bool, bool, SharedHeap::ScanningOption, OopClosure*, CodeBlobClosure*, OopsInGenClosure*)+0x214
V  [libjvm.so+0x1f29eb]  GenCollectedHeap::gen_process_strong_roots(int, bool, bool, bool, SharedHeap::ScanningOption, OopsInGenClosure*, bool, OopsInGenClosure*)+0x5b;;  GenCollectedHeap::gen_process_strong_roots(int, bool, bool, bool, SharedHeap::ScanningOption, OopsInGenClosure*, bool, OopsInGenClosure*)+0x5b
V  [libjvm.so+0x1af398]  DefNewGeneration::collect(bool, bool, unsigned int, bool)+0x228;;  DefNewGeneration::collect(bool, bool, unsigned int, bool)+0x228
V  [libjvm.so+0x1f460f]  GenCollectedHeap::do_collection(bool, bool, unsigned int, bool, int)+0x55f;;  GenCollectedHeap::do_collection(bool, bool, unsigned int, bool, int)+0x55f
V  [libjvm.so+0x163930]  GenCollectorPolicy::satisfy_failed_allocation(unsigned int, bool)+0xb0;;  GenCollectorPolicy::satisfy_failed_allocation(unsigned int, bool)+0xb0
V  [libjvm.so+0x41a866]  VM_GenCollectForAllocation::doit()+0x76;;  VM_GenCollectForAllocation::doit()+0x76
V  [libjvm.so+0x421cd1]  VM_Operation::evaluate()+0x41;;  VM_Operation::evaluate()+0x41
V  [libjvm.so+0x42092a]  VMThread::evaluate_operation(VM_Operation*)+0x7a;;  VMThread::evaluate_operation(VM_Operation*)+0x7a
V  [libjvm.so+0x420ef2]  VMThread::loop()+0x172;;  VMThread::loop()+0x172
V  [libjvm.so+0x421245]  VMThread::run()+0x85;;  VMThread::run()+0x85
V  [libjvm.so+0x3474f9]  java_start(Thread*)+0xf9;;  _ZL10java_startP6Thread+0xf9
C  [libpthread.so.0+0x573b]  abort@@GLIBC_2.0+0x573b

                                    

Comments
EVALUATION

One oddity is that running the test in a debug VM results in this exception:

### TRACE 0: RNG seed = -1409668440578980855
# ERROR: Exception in thread Stresser 686
# ERROR: java.dyn.WrongMethodTypeException: cannot convert to (MethodHandle,Object,Object,Object,Object,Object,Object,Object,Object,Object,Object)Object: invoke(MethodHandle,long,float,char,int,short,double,long,int,float,double)Object
# ERROR:        at java.dyn.MethodHandles.convertArguments(MethodHandles.java:1079)
# ERROR:        at sun.dyn.Invokers.genericInvoker(Invokers.java:86)
# ERROR:        at java.dyn.MethodHandle.invokeWithArguments(MethodHandle.java:420)
# ERROR:        at vm.mlvm.meth.share.MHSequenceGenerator.callSequence(MHSequenceGenerator.java:102)
# ERROR:        at vm.mlvm.meth.stress.compiler.sequences.Test$1.run(Test.java:41)

Using -XX:+UnlockDiagnosticVMOptions -XX:+VerifyMethodHandles with a product VM triggers the same exception.
                                     
2011-02-14
EVALUATION

Sometimes it possible to reproduce the following assert with JDK b126:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/tmp/workspace/jdk7-2-build-solaris-i586-product/jdk7/hotspot/src/share/vm/runtime/handles.cpp:46), pid=8882, tid=215
#  assert(SharedSkipVerify || obj->is_oop()) failed: sanity check
#
# JRE version: 7.0-b126
# Java VM: Java HotSpot(TM) Server VM (20.0-b06-fastdebug mixed mode solaris-x86 )
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x083e9400):  JavaThread "Stresser 204" [_thread_in_vm, id=215, stack(0xb0a1b000,0xb0a6b000)]

Stack: [0xb0a1b000,0xb0a6b000],  sp=0xb0a69bb0,  free space=314k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1d37fec]  void VMError::report(outputStream*)+0x85c
V  [libjvm.so+0x1d391b1]  void VMError::report_and_die()+0x555
V  [libjvm.so+0xafa06a]  void report_vm_error(const char*,int,const char*,const char*)+0x53a
V  [libjvm.so+0xd56cc5]  Handle::Handle #Nvariant 1(Thread*,oop)+0x719
V  [libjvm.so+0x170d2ff]  void MethodHandles::resolve_MemberName(Handle,Thread*)+0x1093
V  [libjvm.so+0x1733819]  MHI_resolve_Mem+0x789
j  sun.dyn.MethodHandleNatives.resolve(Lsun/dyn/MemberName;Ljava/lang/Class;)V+0
j  sun.dyn.MemberName$Factory.resolveInPlace(Lsun/dyn/MemberName;ZLjava/lang/Class;)Z+173
j  sun.dyn.MemberName$Factory.resolveOrNull(Lsun/dyn/MemberName;ZLjava/lang/Class;)Lsun/dyn/MemberName;+11
j  sun.dyn.MemberName$Factory.resolveOrFail(Lsun/dyn/MemberName;ZLjava/lang/Class;)Lsun/dyn/MemberName;+4
j  java.dyn.MethodHandles$Lookup.resolveOrFail(Ljava/lang/Class;Ljava/lang/String;Ljava/dyn/MethodType;Z)Lsun/dyn/MemberName;+38
j  java.dyn.MethodHandles$Lookup.findVirtual(Ljava/lang/Class;Ljava/lang/String;Ljava/dyn/MethodType;)Ljava/dyn/MethodHandle;+5
j  sun.dyn.Invokers.exactInvoker()Ljava/dyn/MethodHandle;+23
j  sun.dyn.Invokers.genericInvoker()Ljava/dyn/MethodHandle;+1
j  java.dyn.MethodHandle.invokeWithArguments([Ljava/lang/Object;)Ljava/lang/Object;+47
j  vm.mlvm.meth.share.MHSequenceGenerator.callSequence(Lvm/mlvm/meth/share/transform/MHTransformationsList;)Ljava/lang/Object;+20
j  vm.mlvm.meth.stress.compiler.sequences.Test$1.run()V+28
v  ~StubRoutines::call_stub
V  [libjvm.so+0xef1d0d]  void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x84d
V  [libjvm.so+0x18714e8]  void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x18
V  [libjvm.so+0xeef994]  void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*)+0x458
V  [libjvm.so+0x1151c11]  void thread_entry(JavaThread*,Thread*)+0x26d
V  [libjvm.so+0x1c24149]  void JavaThread::thread_main_inner()+0x179
V  [libjvm.so+0x1c23df9]  void JavaThread::run()+0x619
V  [libjvm.so+0x1861013]  java_start+0x6f3
C  [libc.so.1+0xa71d0]  _thr_setup+0x4e
C  [libc.so.1+0xa74c0]  __moddi3+0x60

This could be duplicate of 6991556 but 6991556 (which itself is a duplicate of 7007377) was delivered in HS20b06.
                                     
2011-02-14
EVALUATION

The bug can be reproduced up to HS20 b07 (JDK7 b128) but it is fixed from HS21 b01 (JDK7 b129) onward.  The push for 6990754 (which was the first one after the HS21 fork) seems to fix the issue, although I'm not sure if it really fixes the problem or just hides it.
                                     
2011-03-17
EVALUATION

Indeed a similar crash can be reproduced using JDK7 b129 and the mlvm deoptimize test.
                                     
2011-03-17
EVALUATION

I think we first need to take care of the exception when using +VerifyMethodHandles.  Maybe the crashes in a product VM are related to what a debug VM finds and bails out.  Are we sure the test is correct?
                                     
2011-03-22
EVALUATION

1. The test creates sequences of method handles in a random manner, so it may hit other bugs as well.

To reproduce the exact sequence used in particular test run, please supply -seed <n> argument to the test, where <n> is the value from the test log:

### TRACE 0: RNG seed = -1409668440578980855

Otherwise you get different sequence on every run.
                                     
2011-03-24
EVALUATION

I just had a similar crash on vmsqe-t2000c (solaris-sparcv9-product) with vm.mlvm.indy.stress.java.loopsAndThreads.INDIFY_Test:

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0xffffffff7e15b9b8, pid=27668, tid=3
#
# JRE version: 7.0-b134
# Java VM: Java HotSpot(TM) 64-Bit Server VM (21.0-b04 mixed mode solaris-sparc compressed oops)
# Problematic frame:
# V  [libjvm.so+0x55b9b8]  void FastScanClosure::do_oop(oopDesc**)+0x20


Stack: [0xffffffff76800000,0xffffffff76900000],  sp=0xffffffff768fe740,  free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x55b9b8]  void FastScanClosure::do_oop(oopDesc**)+0x20
V  [libjvm.so+0x1dbc84]  void InterpreterOopMap::iterate_oop(OffsetClosure*)+0x74
V  [libjvm.so+0x5aa6bc]  void frame::oops_interpreted_do(OopClosure*,const RegisterMap*,bool)+0x4c4
V  [libjvm.so+0xa1805c]  void Threads::oops_do(OopClosure*,CodeBlobClosure*)+0x20c
V  [libjvm.so+0x984800]  void SharedHeap::process_strong_roots(bool,bool,SharedHeap::ScanningOption,OopClosure*,CodeBlobClosure*,OopsInGenClosure*)+0x108
V  [libjvm.so+0x5e3378]  void GenCollectedHeap::gen_process_strong_roots(int,bool,bool,bool,SharedHeap::ScanningOption,OopsInGenClosure*,bool,OopsInGenClosure*)+0
xb8
V  [libjvm.so+0x55d078]  void DefNewGeneration::collect(bool,bool,unsigned long,bool)+0x2d0
V  [libjvm.so+0x5e2b54]  void GenCollectedHeap::do_collection(bool,bool,unsigned long,bool,int)+0x7f4
V  [libjvm.so+0x4f1afc]  HeapWord*GenCollectorPolicy::satisfy_failed_allocation(unsigned long,bool)+0x19c
V  [libjvm.so+0xa67c74]  void VM_GenCollectForAllocation::doit()+0xa4
V  [libjvm.so+0x27268c]  void VM_Operation::evaluate()+0xa4
V  [libjvm.so+0xa6b9f8]  void VMThread::evaluate_operation(VM_Operation*)+0x138
V  [libjvm.so+0xa6bf8c]  void VMThread::loop()+0x424
V  [libjvm.so+0x2f1514]  void VMThread::run()+0xa4
V  [libjvm.so+0x907ee0]  java_start+0x280

JavaThread 0x00000001005ce800 (nid = 284) was being processed
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
v  ~RuntimeStub::_complete_monitor_locking_Java
J  vm.mlvm.share.MlvmTest.trace(ILjava/lang/String;)V
j  vm.mlvm.indy.stress.java.loopsAndThreads.INDIFY_Test.bootstrap(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+38
j  sun.dyn.FilterGeneric$F5.invoke_F2(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+20
j  sun.dyn.CallSiteImpl.makeSite(Ljava/dyn/MethodHandle;Ljava/lang/String;Ljava/dyn/MethodType;Ljava/lang/Object;Lsun/dyn/MemberName;I)Ljava/dyn/CallSite;+104
j  sun.dyn.MethodHandleNatives.makeDynamicCallSite(Ljava/dyn/MethodHandle;Ljava/lang/String;Ljava/dyn/MethodType;Ljava/lang/Object;Lsun/dyn/MemberName;I)Ljava/dyn
/CallSite;+8
v  ~StubRoutines::call_stub
j  vm.mlvm.indy.stress.java.loopsAndThreads.INDIFY_Test.runThread(I)Z+13967
j  vm.mlvm.share.MultiThreadedTest$1.run()V+21
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub
VM_Operation (0xfffffffe1c0fc408): GenCollectForAllocation, mode: safepoint, requested by thread 0x0000000100930000
                                     
2011-03-25
EVALUATION

The following seeds triggered the bug with -stressThreadsFactor=100:

vmsqe-amd-02: ### TRACE 0: RNG seed = -1508065852318538130
vmsqe-amd-12: ### TRACE 0: RNG seed = 1835564045439005768
                                     
2011-03-25
SUGGESTED FIX

CheckUnhandledOops found three issues.  Additionally I'd like to change a couple of other oop usages for the sake of safety (e.g. when new code gets added without taking care of the naked oops).
                                     
2011-04-05
EVALUATION

I found a bug in MethodHandles::resolve_MemberName.  The suggested fix fixes at least the compiler/sequences test but I still get the same error with loopsAndThreads.  It seems the is another, most likely, similiar bug somewhere.
                                     
2011-04-05
EVALUATION

Reproducible every time on intelsdv33 with a linux-i586-jvmg build using JDK 7 b136:

intelsdv33:/export/home/twisti$ java -showversion -XX:+ShowMessageBoxOnError -server -Xmixed -XX:+UseSerialGC -verbosegc -XX:+VerifyBeforeGC -XX:+VerifyAfterGC -XX:+PrintHeapAtGC -XX:+UnlockDiagnosticVMOptions -XX:-VerifyMethodHandles -XX:+UnlockExperimentalVMOptions -XX:+EnableInvokeDynamic -cp vm7-jsr292-release-2011-04-07.jar vm.mlvm.indy.stress.java.loopsAndThreads.INDIFY_Test -stressIterationsFactor 1000000 -stressThreadsFactor 1000 -stressTime 600

...

[Verifying threads permgen tenured generation def new generation remset ref_proc syms strs zone dict hand C-heap code cache ]
java version "1.7.0-ea"
Java(TM) SE Runtime Environment (build 1.7.0-ea-b136)
Java HotSpot(TM) Server VM (build 21.0-b05-internal-jvmg, mixed mode)

...

{Heap before GC invocations=9 (full 0):
 def new generation   total 19648K, used 18421K [0xaf430000, 0xb0980000, 0xc4980000)
  eden space 17472K, 100% used [0xaf430000, 0xb0540000, 0xb0540000)
  from space 2176K,  43% used [0xb0760000, 0xb084d4f0, 0xb0980000)
  to   space 2176K,   0% used [0xb0540000, 0xb0540000, 0xb0760000)
 tenured generation   total 43712K, used 0K [0xc4980000, 0xc7430000, 0xef430000)
   the space 43712K,   0% used [0xc4980000, 0xc4980000, 0xc4980200, 0xc7430000)
 compacting perm gen  total 16384K, used 2449K [0xef430000, 0xf0430000, 0xf3430000)
   the space 16384K,  14% used [0xef430000, 0xef6944e8, 0xef694600, 0xf0430000)
No shared spaces configured.
 VerifyBeforeGC:[Verifying threads # To suppress the following error report, specify this argument
# after -XX: or in .hotspotrc:  SuppressErrorAt=/genOopClosures.hpp:178
==============================================================================
Unexpected Error
------------------------------------------------------------------------------
Internal Error at genOopClosures.hpp:178, pid=28791, tid=2934336368
guarantee(obj->is_oop_or_null()) failed: invalid oop: 0xb0761ce8

Do you want to debug the problem?

To debug, run 'gdb /proc/28791/exe 28791'; then switch to thread -1360630928 (0xaee66b70)
Enter 'yes' to launch gdb automatically (PATH must include gdb)
Otherwise, press RETURN to abort...
==============================================================================
                                     
2011-04-07
EVALUATION

I will push the bare oops fixes I have found under this CR to finally get these out of our way.  Any failures, like the one with vm/mlvm/indy/stress/java/loopsAndThreads, will be tracked under another CR, e.g. 6990266, 7018365, or 7018374.
                                     
2011-04-13
EVALUATION

http://hg.openjdk.java.net/jdk7/hotspot-comp/hotspot/rev/2a23b1b5a0a8
                                     
2011-04-18
EVALUATION

http://hg.openjdk.java.net/jdk7/hotspot-rt/hotspot/rev/2a23b1b5a0a8
                                     
2011-05-05



Hardware and Software, Engineered to Work Together