JDK-8020599 : G1: VM crashed with OOME on solaris with large heap
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: hs24,hs25
  • Priority: P2
  • Status: Closed
  • Resolution: Duplicate
  • OS: solaris_11
  • CPU: x86
  • Submitted: 2013-07-16
  • Updated: 2013-09-18
  • Resolved: 2013-09-02
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
hs25Resolved
Related Reports
Duplicate :  
Duplicate :  
Relates :  
Relates :  
Description
Tests 

nsk/stress/except/except001
nsk/stress/except/except004
nsk/stress/except/except006
nsk/stress/except/except007
nsk/stress/except/except008
nsk/stress/except/except009

crashed with options -d64 -server -Xcomp -XX:+UseG1GC

;; Using jvm: "/bpool/local/aurora/sandbox/sca/vmsqe/jdk/pit/2013-07-13-001338.amurillo.hs25-b41-jdk8-b99-control/fastdebug/solaris-amd64/jre/lib/amd64/server/libjvm.so"
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 4088 bytes for AllocateHeap
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (/opt/jprt/T/P1/001338.amurillo/s/hotspot/src/share/vm/memory/allocation.inline.hpp:62), pid=5464, tid=24
#
# JRE version: Java(TM) SE Runtime Environment (8.0) (build 1.8.0-internal-fastdebug-jprtadm_2013_07_12_17_28-b00)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.0-b41-fastdebug compiled mode solaris-amd64 compressed oops)
# Core dump written. Default location: /bpool/local/aurora/sandbox/results/ResultDir/except001/core or core.5464
#

---------------  T H R E A D  ---------------

Current thread (0x0000000000481000):  GCTaskThread [stack: 0xfffffd7ffcbfe000,0xfffffd7ffccfe000] [id=24]

Stack: [0xfffffd7ffcbfe000,0xfffffd7ffccfe000],  sp=0xfffffd7ffccf5920,  free space=990k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x2774b48]  void VMError::report(outputStream*)+0x92c;;  __1cHVMErrorGreport6MpnMoutputStream__v_+0x92c
V  [libjvm.so+0x2775fb0]  void VMError::report_and_die()+0x56c;;  __1cHVMErrorOreport_and_die6M_v_+0x56c
V  [libjvm.so+0xf6a5ab]  void report_vm_out_of_memory(const char*,int,unsigned long,VMErrorType,const char*)+0x497;;  __1cXreport_vm_out_of_memory6FpkciLnLVMErrorType_1_v_+0x497
V  [libjvm.so+0x11a01a3]  StarTask*Stack<StarTask,(unsigned short)1280>::alloc(unsigned long)+0x83;;  __1cFStack4nIStarTask_vbxG_Falloc6ML_pn0A__+0x83
V  [libjvm.so+0x1180ff8]  void G1ParPushHeapRSClosure::do_oop(unsigned*)+0x6fc;;  __1cWG1ParPushHeapRSClosureGdo_oop6MpI_v_+0x6fc
V  [libjvm.so+0x11df94d]  void FilterIntoCSClosure::do_oop(unsigned*)+0x20d;;  __1cTFilterIntoCSClosureGdo_oop6MpI_v_+0x20d
V  [libjvm.so+0x20de0c4]  int ObjArrayKlass::oop_oop_iterate_v_m(oop,ExtendedOopClosure*,MemRegion)+0x4ac;;  __1cNObjArrayKlassToop_oop_iterate_v_m6MnDoop_pnSExtendedOopClosure_nJMemRegion__i_+0x4ac
V  [libjvm.so+0x13158da]  void HeapRegionDCTOC::walk_mem_region_with_cl(MemRegion,HeapWord*,HeapWord*,ExtendedOopClosure*)+0x856;;  __1cPHeapRegionDCTOCXwalk_mem_region_with_cl6MnJMemRegion_pnIHeapWord_3pnSExtendedOopClosure__v_+0x856
V  [libjvm.so+0x246b778]  void Filtering_DCTOC::walk_mem_region(MemRegion,HeapWord*,HeapWord*)+0xb0;;  __1cPFiltering_DCTOCPwalk_mem_region6MnJMemRegion_pnIHeapWord_3_v_+0xb0
V  [libjvm.so+0x1323943]  void HeapRegionDCTOC::walk_mem_region(MemRegion,HeapWord*,HeapWord*)+0x23;;  __1cPHeapRegionDCTOCPwalk_mem_region6MnJMemRegion_pnIHeapWord_3_v_+0x23
V  [libjvm.so+0x246b1ed]  void DirtyCardToOopClosure::do_MemRegion(MemRegion)+0x155;;  __1cVDirtyCardToOopClosureMdo_MemRegion6MnJMemRegion__v_+0x155
V  [libjvm.so+0x11de501]  bool ScanRSClosure::doHeapRegion(HeapRegion*)+0x531;;  __1cNScanRSClosureMdoHeapRegion6MpnKHeapRegion__b_+0x531
V  [libjvm.so+0x11475fa]  void G1CollectedHeap::collection_set_iterate_from(HeapRegion*,HeapRegionClosure*)+0xae;;  __1cPG1CollectedHeapbBcollection_set_iterate_from6MpnKHeapRegion_pnRHeapRegionClosure__v_+0xae
V  [libjvm.so+0x11dbc7b]  void G1RemSet::oops_into_collection_set_do(OopsInHeapRegionClosure*,int)+0x53f;;  __1cIG1RemSetbBoops_into_collection_set_do6MpnXOopsInHeapRegionClosure_i_v_+0x53f
V  [libjvm.so+0x1155eab]  void G1CollectedHeap::g1_process_strong_roots(bool,SharedHeap::ScanningOption,OopClosure*,OopsInHeapRegionClosure*,G1KlassScanClosure*,int)+0x3db;;  __1cPG1CollectedHeapXg1_process_strong_roots6MbnKSharedHeapOScanningOption_pnKOopClosure_pnXOopsInHeapRegionClosure_pnSG1KlassScanClosure_i_v_+0x3db
V  [libjvm.so+0x1166ee3]  void G1ParTask::work(unsigned)+0x747;;  __1cJG1ParTaskEwork6MI_v_+0x747
V  [libjvm.so+0x27d2c4b]  void GangWorker::loop()+0x533;;  __1cKGangWorkerEloop6M_v_+0x533
V  [libjvm.so+0x2183182]  java_start+0x1ce;;  java_start+0x1ce
C  [libc.so.1+0x12257d]  _thrp_setup+0xa5;;  _thrp_setup+0xa5
C  [libc.so.1+0x122820]  _lwp_start+0x0;;  _lwp_start+0x0


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x0000000002632000 JavaThread "Service Thread" daemon [_thread_blocked, id=64, stack(0xfffffd7f9c52a000,0xfffffd7f9c62a000)]
  0x00000000023fb000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=63, stack(0xfffffd7f9c62b000,0xfffffd7f9c72b000)]
  0x00000000023f8000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=62, stack(0xfffffd7f9c72c000,0xfffffd7f9c82c000)]
  0x00000000023f5800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=61, stack(0xfffffd7f9c82d000,0xfffffd7f9c92d000)]
  0x00000000023f3000 JavaThread "Surrogate Locker Thread (Concurrent GC)" daemon [_thread_blocked, id=60, stack(0xfffffd7f9c92e000,0xfffffd7f9ca2e000)]
  0x00000000023a4800 JavaThread "Finalizer" daemon [_thread_blocked, id=59, stack(0xfffffd7f9ca2f000,0xfffffd7f9cb2f000)]
  0x0000000002398800 JavaThread "Reference Handler" daemon [_thread_blocked, id=58, stack(0xfffffd7f9cb30000,0xfffffd7f9cc30000)]
  0x0000000000441800 JavaThread "main" [_thread_blocked, id=2, stack(0xfffffd7ffed3f000,0xfffffd7ffee3f000)]

Other Threads:
  0x0000000002394000 VMThread [stack: 0xfffffd7f9d5fe000,0xfffffd7f9d6fe000] [id=57]
  0x00000000026a7000 WatcherThread [stack: 0xfffffd7f9c429000,0xfffffd7f9c529000] [id=65]

=>0x0000000000481000 (exited) GCTaskThread [stack: 0xfffffd7ffcbfe000,0xfffffd7ffccfe000] [id=24]

VM state:at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x000000000043f7d8] Threads_lock - owner thread: 0x0000000002394000
[0x000000000043fff8] Heap_lock - owner thread: 0x0000000000441800

Heap:
 garbage-first heap   total 20119552K, used 7196160K [0x0000000046400000, 0x0000000512400000, 0x0000000800000000)
  region size 1024K, 982 young (1005568K), 104 survivors (106496K)
 Metaspace total 4494K, used 3636K, reserved 110592K
  data space     4108K, used 3319K, reserved 8192K
  class space    386K, used 316K, reserved 102400K
Comments
This test does not fail on this particular machine after the fixes for JDK-8019902 and JDK-8013938. These fixes decreases the risk of getting native out of memory on Solaris x86, but they don't completely remove the risk. I have verified that the current test on the current machine does not fail after these fixes.
02-09-2013

Ok, I just looked at the error message and saw 'allocation.inline.hpp' and made a hasty conclusion, please ignore this extra info if it seems wrong.
26-07-2013

Ingemar, the run you linked to is not using G1GC. We've implemented a work-around for our troubles with running out of native memory on Solaris for G1. This bug is to track cases where that work around is not sufficient. The failure you linked to also does not have a core file. It is practically impossible to debug a native oome on Solaris without a core file or at least a virtual memory map output (pmap).
25-07-2013

This also happened in Promotion testing of JDK7u40_b34 (hs24) http://vmsqe-app.russia.sun.com/surl/1D Are we sure this problem was introduced in hs25 (instead of just first discovered in a thes of hs25)? This also looks a lot like INTJDK-7604667
25-07-2013