JDK-8062672 : JVM crashes during GC on various asserts which checks that HeapWord ptr is an oop
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 7u80
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: linux
  • CPU: sparc
  • Submitted: 2014-10-31
  • Updated: 2015-09-29
  • Resolved: 2014-12-11
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8 JDK 9
7u80 b04Fixed 8u60Fixed 9Fixed
Related Reports
Relates :  
Description
JVM crashes during GC on various asserts which checks that HeapWord ptr is an oop.
Issues could be intermittently reproduced with G1 & CMS.
I was able to reproduce only on linux-sparc hosts and only w/ 7u80, but there is an JI bug (JI-9011043) which was filed against very similar crash happened on solaris-sparc w/ 7u51.

Here are example of failed asserts:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/HUDSON/workspace/7u-2-build-linux-sparcv9/jdk7u-dev/1740/hotspot/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp:1012), pid=19426, tid=18446735281952250128
#  assert(k->is_oop(true )) failed: Should be klass oop
#
# JRE version: Java(TM) SE Runtime Environment (7.0_80-b03) (build 1.7.0_80-ea-fastdebug-langtools-nightly-h1740-20141030-b03-b03)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.80-b07-fastdebug mixed mode linux-sparc compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0xfffff80104024000):  GCTaskThread [stack: 0xfffff801025de000,0xfffff801026de000] [id=19437]

Stack: [0xfffff801025de000,0xfffff801026de000],  sp=0xfffff801026da360,  free space=1008k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xe34fa0]  VMError::report_and_die()+0x24c
V  [libjvm.so+0x6daa28]  report_vm_error(char const*, int, char const*, char const*)+0x6c
V  [libjvm.so+0x631894]  CompactibleFreeListSpace::block_size(HeapWord const*) const+0x27c
V  [libjvm.so+0x44fdf4]  BlockOffsetArrayNonContigSpace::block_start_unsafe(void const*) const+0x474
V  [libjvm.so+0x6287ac]  CompactibleFreeListSpace::block_start_const(void const*) const+0xe0
V  [libjvm.so+0x633ec0]  Space::block_start(void const*)+0x18
V  [libjvm.so+0xc1ea54]  CardTableModRefBS::process_chunk_boundaries(Space*, DirtyCardToOopClosure*, MemRegion, MemRegion, signed char**, unsigned long, unsigned long)+0xc8
V  [libjvm.so+0xc1fdc0]  CardTableModRefBS::process_stride(Space*, MemRegion, int, int, OopsInGenClosure*, CardTableRS*, signed char**, unsigned long, unsigned long)+0x458
V  [libjvm.so+0xc20188]  CardTableModRefBS::non_clean_card_iterate_parallel_work(Space*, MemRegion, OopsInGenClosure*, CardTableRS*, int)+0x15c
V  [libjvm.so+0x557028]  CardTableModRefBS::non_clean_card_iterate_possibly_parallel(Space*, MemRegion, OopsInGenClosure*, CardTableRS*)+0xb8
V  [libjvm.so+0x55a84c]  CardTableRS::younger_refs_in_space_iterate(Space*, OopsInGenClosure*)+0x8c
V  [libjvm.so+0x8077c8]  Generation::younger_refs_in_space_iterate(Space*, OopsInGenClosure*)+0x3c
V  [libjvm.so+0x68cfbc]  ConcurrentMarkSweepGeneration::younger_refs_iterate(OopsInGenClosure*)+0x34
V  [libjvm.so+0x55a074]  CardTableRS::younger_refs_iterate(Generation*, OopsInGenClosure*)+0x2c
V  [libjvm.so+0x7f17dc]  GenCollectedHeap::gen_process_strong_roots(int, bool, bool, bool, SharedHeap::ScanningOption, OopsInGenClosure*, bool, OopsInGenClosure*)+0xe8
V  [libjvm.so+0xc29314]  ParNewGenTask::work(unsigned int)+0x1cc
V  [libjvm.so+0xe759a8]  GangWorker::loop()+0x358
V  [libjvm.so+0xe743f4]  GangWorker::run()+0x24
V  [libjvm.so+0xc01914]  java_start(Thread*)+0x164
C  [libpthread.so.0+0x8a08]  start_thread+0xc8

or 

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/HUDSON/workspace/7u-2-build-linux-sparcv9/jdk7u-dev/1725/hotspot/src/share/vm/memory/space.cpp:822), pid=43076, tid=8796149045520
#  assert(p == current_top || oop(p)->is_oop()) failed: p (0x0000000147b36c00) is not a block start - current_top: 0x0000000147c00000, is_oop: false
#
# JRE version: Java(TM) SE Runtime Environment (7.0_80-b03) (build 1.7.0_80-ea-fastdebug-langtools-nightly-h1725-20141027-b03-b03)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.80-b07-fastdebug mixed mode linux-sparc compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x0000080004046800):  GCTaskThread [stack: 0x000008000346e000,0x000008000356e000] [id=43101]

Stack: [0x000008000346e000,0x000008000356e000],  sp=0x0000080003565a40,  free space=990k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xe34fa0]  VMError::report_and_die()+0x24c
V  [libjvm.so+0x6daa28]  report_vm_error(char const*, int, char const*, char const*)+0x6c
V  [libjvm.so+0xd2e040]  ContiguousSpace::block_size(HeapWord const*) const+0x2e4
V  [libjvm.so+0x78dc74]  G1BlockOffsetArrayContigSpace::block_start_unsafe(void const*)+0x26c
V  [libjvm.so+0x83fb90]  G1OffsetTableContigSpace::block_start(void const*)+0x38
V  [libjvm.so+0xd2aaa0]  DirtyCardToOopClosure::do_MemRegion(MemRegion)+0xc0
V  [libjvm.so+0x7d17a8]  ScanRSClosure::doHeapRegion(HeapRegion*)+0x448
V  [libjvm.so+0x799118]  G1CollectedHeap::collection_set_iterate_from(HeapRegion*, HeapRegionClosure*)+0x204
V  [libjvm.so+0x7cf5e8]  G1RemSet::scanRS(OopsInHeapRegionClosure*, CodeBlobToOopClosure*, int)+0xbc
V  [libjvm.so+0x7cf9c0]  G1RemSet::oops_into_collection_set_do(OopsInHeapRegionClosure*, CodeBlobToOopClosure*, int)+0xfc
V  [libjvm.so+0x7a0710]  _ZN15G1CollectedHeap23g1_process_strong_rootsEbN10SharedHeap14ScanningOptionEP10OopClosureP23OopsInHeapRegionClosureP16OopsInGenClosureib.clone.11+0x4ac
V  [libjvm.so+0x7b8a20]  G1ParTask::work(unsigned int)+0x6a4
V  [libjvm.so+0xe759a8]  GangWorker::loop()+0x358
V  [libjvm.so+0xe743f4]  GangWorker::run()+0x24
V  [libjvm.so+0xc01914]  java_start(Thread*)+0x164
C  [libpthread.so.0+0x8a08]  start_thread+0xc8


Note that in case of crash in CompactibleFreeListSpace::block_size, p point to:
(gdb) p *p
$1 = {i = 0xbaadbabebaadbabe <Address 0xbaadbabebaadbabe out of bounds>}

And in case of crash in ContiguousSpace::block_size, p point to:
(gdb) p *p
$1 = {i = 0xdeafbabedeafbabe <Address 0xdeafbabedeafbabe out of bounds>}
Comments
verified by $JAVA_HOME/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:+PrintFlagsFinal -version | grep UseMemSetInBOT for b03: bool UseMemSetInBOT = true {experimental} for b04: bool UseMemSetInBOT = false {experimental}
19-01-2015

ILW => HML => P2 Impact: High, crash. Likelihood: Medium, reproducible but takes time. Workaround: Low, explicitly set experimental option UseMemSetInBOT to false.
04-12-2014

Vladimir checked in the change above for Volker and it affects GC. This should be fixed by the GC or compiler group, not runtime.
02-12-2014

Comment regarding the patch: Re-wrote the file-read method so it could be re-used and added a detect-method for blkinit.
02-12-2014

Moving this to runtime. They might have other ideas on how to check which cpu features to enable, but I've attached a fix-proposal. A simple way to verify the patch is to run: fastdebug/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:+PrintFlagsFinal -version | grep UseMemSetInBOT bool UseMemSetInBOT = true {experimental} java version "1.7.0_80-ea-fastdebug" Java(TM) SE Runtime Environment (build 1.7.0_80-ea-fastdebug-b04) Java HotSpot(TM) 64-Bit Server VM (build 24.80-b07-fastdebug, mixed mode After the fix, when running on sparc-linux UseMemSetInBOT should be false when run with G1 or CMS: fastdebug/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:+PrintFlagsFinal -version | grep UseMemSetInBOT bool UseMemSetInBOT = false {experimental} java version "1.7.0_80-ea-fastdebug" Java(TM) SE Runtime Environment (build 1.7.0_80-ea-fastdebug-b04) Java HotSpot(TM) 64-Bit Server VM (build 24.80-b07-internal-fastdebug, mixed mode)
02-12-2014

These failures are cause by change: http://hg.openjdk.java.net/jdk7u/jdk7u-dev/hotspot/rev/6547c22e85df That change relies on blk_init() returning true to turn off UseMemSetInBOT for G1 and CMS, but this feature is not correctly setup on sparc-linux, so checking it will always return false when running sparc-linux. We need to add support for this feature (blk_init_instructions_m) in vm_version_linux_sparc.cpp.
01-12-2014