United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-8001591 NMT: assertion failed: assert(rec->addr() + rec->size() <= cur->base()) failed: Can not overlap in memSnapshot.cpp
JDK-8001591 : NMT: assertion failed: assert(rec->addr() + rec->size() <= cur->base()) failed: Can not overlap in memSnapshot.cpp

Details
Type:
Bug
Submit Date:
2012-10-25
Status:
Closed
Updated Date:
2013-07-10
Project Name:
JDK
Resolved Date:
2012-11-06
Component:
hotspot
OS:
os_x
Sub-Component:
runtime
CPU:
x86,sparc
Priority:
P2
Resolution:
Fixed
Affected Versions:
Fixed Versions:
hs25 (b09)

Related Reports
Backport:
Backport:
Backport:
Backport:

Sub Tasks

Description
After CR 7199092's fix, NMT encountered assertion failures:

A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/tmp/jprt/P1/234841.cphillim/s/src/share/vm/services/memSnapshot.cpp:103), pid=21133, tid=7
#  assert(rec->addr() + rec->size() <= cur->base()) failed: Can not overlap
#

and 

#  Internal Error (C:\jprt\T\P1\234841.cphillim\s\src\share\vm\services\memSnapshot.cpp:75), pid=6864, tid=4244
#  assert(cur->base() > rec->addr()) failed: Just check: locate()

on following test cases:

Solaris v9
closed/gc/4950157/BubbleUpRef.java

Windows x64:
nsk/jvmti/SetJNIFunctionTable/setjniftab001
nsk/jvmti/scenarios/jni_interception/JI05/ji05t001

Windows x32:
nsk/jvmti/SetJNIFunctionTable/setjniftab001


                                    

Comments
VM 	Java HotSpot(TM) Client VM 25.0 b7 (25.0-b07-internal-201210292336.yqi.hotspot-fastdebug)
Options 	-client -Xcomp -XX:DefaultMaxRAMFraction=8 -XX:NativeMemoryTracking=detail -XX:ReservedCodeCacheSize=256M
Arch 	32
Info 	compiled mode, sharing
Type/Group/Date 	Tue Oct 30 15:54:22 GMT+04:00 2012
Additional info 	

;; Using jvm: "/export/local/common/jdk/baseline/solaris-sparc/jre/lib/sparc/client/libjvm.so"
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/tmp/jprt/P1/233612.yqi/s/src/share/vm/services/memSnapshot.cpp:75), pid=2422, tid=3
#  assert(cur->base() > rec->addr()) failed: Just check: locate()
#
# JRE version: Java(TM) SE Runtime Environment (8.0-b62)
# Java VM: Java HotSpot(TM) Client VM (25.0-b07-internal-201210292336.yqi.hotspot-fastdebug compiled mode, sharing solaris-sparc )
# Core dump written. Default location: /export/local/119185.JAVASE.NIGHTLY.VM.RT_Baseline.2012-10-30.solaris-sparc_vm__client_comp_nsk.jvmti.testlist.runTests/results/ResultDir/ji05t001/core or core.2422
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x000d2c00):  Thread [stack: 0xfd370000,0xfd3f0000] [id=3]

Stack: [0xfd370000,0xfd3f0000],  sp=0xfd3ef6f8,  free space=509k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xbb0b0c]  void VMError::report_and_die()+0x7e8;;  __1cHVMErrorOreport_and_die6M_v_+0x7e8
V  [libjvm.so+0x45559c]  void report_vm_error(const char*,int,const char*,const char*)+0x74;;  __1cPreport_vm_error6Fpkci11_v_+0x74
V  [libjvm.so+0x8b8234]  bool VMMemPointerIterator::add_reserved_region(MemPointerRecord*)+0xfc;;  __1cUVMMemPointerIteratorTadd_reserved_region6MpnQMemPointerRecord__b_+0xfc
V  [libjvm.so+0x8bbd9c]  bool MemSnapshot::promote_virtual_memory_records(MemPointerArrayIterator*)+0x68c;;  __1cLMemSnapshotbEpromote_virtual_memory_records6MpnXMemPointerArrayIterator__b_+0x68c
V  [libjvm.so+0x8bad88]  bool MemSnapshot::promote()+0x180;;  __1cLMemSnapshotHpromote6M_b_+0x180
V  [libjvm.so+0x8c1edc]  void MemTrackWorker::run()+0x2a4;;  __1cOMemTrackWorkerDrun6M_v_+0x2a4
V  [libjvm.so+0x9694a4]  java_start+0x254;;  java_start+0x254


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x001b7c00 JavaThread "Service Thread" daemon [_thread_blocked, id=11, stack(0xecdb0000,0xece30000)]
  0x0009ac00 JavaThread "C1 CompilerThread0" daemon [_thread_in_native, id=10, stack(0xecca0000,0xecd20000)]
  0x00098400 JavaThread "Thread-1" [_thread_in_native, id=9, stack(0xecd30000,0xece2c000)]
  0x00192c00 JavaThread "Thread-0" [_thread_in_native, id=8, stack(0xfd020000,0xfd11c000)]
  0x0018a800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=7, stack(0xfd120000,0xfd1a0000)]
  0x000e1400 JavaThread "Finalizer" daemon [_thread_blocked, id=6, stack(0xfd1c0000,0xfd240000)]
  0x000d8c00 JavaThread "Reference Handler" daemon [_thread_blocked, id=5, stack(0xfd250000,0xfd2d0000)]
  0x0004d800 JavaThread "main" [_thread_blocked, id=2, stack(0xfedb0000,0xfee30000)]

Other Threads:
  0x000d4c00 VMThread [stack: 0xfd2e0000,0xfd360000] [id=4]
  0x0023a800 WatcherThread [stack: 0xecc10000,0xecc90000] [id=12]

=>0x000d2c00 (exited) Thread [stack: 0xfd370000,0xfd3f0000] [id=3]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 def new generation   total 4928K, used 618K [0xe4600000, 0xe4b50000, 0xe70a0000)
  eden space 4416K,  14% used [0xe4600000, 0xe469aac0, 0xe4a50000)
  from space 512K,   0% used [0xe4a50000, 0xe4a50000, 0xe4ad0000)
  to   space 512K,   0% used [0xe4ad0000, 0xe4ad0000, 0xe4b50000)
 tenured generation   total 10944K, used 0K [0xe70a0000, 0xe7b50000, 0xec600000)
   the space 10944K,   0% used [0xe70a0000, 0xe70a0000, 0xe70a0200, 0xe7b50000)
 Metaspace total 3232K, used 156K, reserved 8192K
  data space     3072K, used 129K, reserved 6144K
  class space    160K, used 27K, reserved 2048K

Card table byte_map: [0xec7b0000,0xec7f2000] byte_map_base: 0xec08d000

Polling page: 0xfdaf8000

Code Cache  [0xed000000, 0xed1e8000, 0xfd000000)
 total_blobs=473 nmethods=352 adapters=66 free_code_cache=260205Kb largest_free_block=266450688

Compilation events (10 events):
Event: 5.210 Thread 0x0009ac00 nmethod 345 0xed1dd788 code [0xed1dd880, 0xed1dd934]
Event: 5.210 Thread 0x0009ac00  34
Event: 5.211 Thread 0x0009ac00 nmethod 346 0xed1dd988 code [0xed1dda80, 0xed1ddb34]
Event: 5.211 Thread 0x0009ac00  34
Event: 5.238 Thread 0x0009ac00 nmethod 347 0xed1ddbc8 code [0xed1ddf80, 0xed1def6c]
Event: 5.241 Thread 0x0009ac00  34
Event: 5.244 Thread 0x0009ac00 nmethod 348 0xed1e0188 code [0xed1e02e0, 0xed1e07bc]
Event: 5.245 Thread 0x0009ac00  34
Event: 5.248 Thread 0x0009ac00 nmethod 349 0xed1e0a08 code [0xed1e0b40, 0xed1e0d28]
Event: 5.248 Thread 0x0009ac00  35

GC Heap History (0 events):
No events

Deoptimization events (0 events):
No events

Internal exceptions (2 events):
Event: 0.275 Thread 0x0004d800 Threw 0xe460c740 at /tmp/jprt/P1/233612.yqi/s/src/share/vm/prims/jni.cpp:3996
Event: 4.548 Thread 0x0004d800 Threw 0xe4651db0 at /tmp/jprt/P1/233612.yqi/s/src/share/vm/prims/jvm.cpp:1168

Events (10 events):
Event: 4.032 Executing VM operation: Deoptimize
Event: 4.033 Executing VM operation: Deoptimize done
Event: 4.053 Thread 0x0004d800 DEOPT PACKING pc=0xed18fa74 sp=0xfee2d7d0
Event: 4.053 Thread 0x0004d800 DEOPT UNPACKING pc=0xed0b9c04 sp=0xfee2d5c8 mode 0
Event: 4.275 Executing VM operation: Deoptimize
Event: 4.275 Executing VM operation: Deoptimize done
Event: 4.343 loading class 0x8058da58
Event: 4.343 loading class 0x8058da58 done
Event: 4.348 loading class 0x8043deb0
Event: 4.350 loading class 0x8043deb0 done


Dynamic libraries:
0x00010000 	/export/local/common/jdk/baseline/solaris-sparc/bin/java
0xff36a000 	/lib/libthread.so.1
0xff330000 	/export/local/common/jdk/baseline/solaris-sparc/bin/../jre/lib/sparc/jli/libjli.so
0xff35c000 	/lib/libdl.so.1
0xff1a0000 	/lib/libc.so.1
0xfdc00000 	/export/local/common/jdk/baseline/solaris-sparc/jre/lib/sparc/client/libjvm.so
0xff150000 	/lib/libsocket.so.1
0xff17c000 	/usr/lib/libsched.so.1
0xff130000 	/lib/libm.so.1
0xff0f0000 	/usr/lib/libCrun.so.1
0xff11c000 	/lib/libdoor.so.1
0xff0b0000 	/usr/lib/libdemangle.so.1
0xff090000 	/lib/libkstat.so.1
0xfeeb0000 	/lib/libm.so.2
0xfeff0000 	/lib/libnsl.so.1
0xfefc0000 	/lib/libmd.so.1
0xfef80000 	/lib/libmp.so.2
0xfdbc0000 	/export/local/common/jdk/baseline/solaris-sparc/jre/lib/sparc/libverify.so
0xfdb80000 	/export/local/common/jdk/baseline/solaris-sparc/jre/lib/sparc/libjava.so
0xfdb40000 	/lib/libscf.so.1
0xfdb20000 	/lib/libuutil.so.1
0xfdb00000 	/lib/libgen.so.1
0xfdae0000 	/lib/libnvpair.so.1
0xfda80000 	/export/local/common/testbase/8/vm/vm/bin/lib/solaris-sparc/nsk/jvmti/scenarios/jni_interception/JI05/libji05t001.so
0xfda10000 	/export/local/common/jdk/baseline/solaris-sparc/jre/lib/sparc/libzip.so

VM Arguments:
jvm_args: -Xcomp -XX:DefaultMaxRAMFraction=8 -XX:NativeMemoryTracking=detail -XX:ReservedCodeCacheSize=256M -agentlib:ji05t001=-waittime=5 
java_command: nsk.jvmti.scenarios.jni_interception.JI05.ji05t001
java_class_path (initial): /export/local/119185.JAVASE.NIGHTLY.VM.RT_Baseline.2012-10-30.solaris-sparc_vm__client_comp_nsk.jvmti.testlist.runTests/results/ResultDir/ji05t001:/export/local/common/testbase/8/vm/vm/bin/classes:/export/local/common/jdk/baseline/solaris-sparc/lib/tools.jar
Launcher Type: SUN_STANDARD

Environment Variables:
CLASSPATH=/export/local/119185.JAVASE.NIGHTLY.VM.RT_Baseline.2012-10-30.solaris-sparc_vm__client_comp_nsk.jvmti.testlist.runTests/results/ResultDir/ji05t001:/export/local/common/testbase/8/vm/vm/bin/classes:/export/local/common/jdk/baseline/solaris-sparc/lib/tools.jar
PATH=/export/local/common/jdk/baseline/solaris-sparc/bin:/bin:/usr/bin
LD_LIBRARY_PATH=/export/local/common/jdk/baseline/solaris-sparc/jre/lib/sparc/client:/export/local/common/jdk/baseline/solaris-sparc/jre/lib/sparc:/export/local/common/jdk/baseline/solaris-sparc/jre/../lib/sparc:/export/local/common/testbase/8/vm/vm/bin/lib/solaris-sparc/nsk/jvmti/scenarios/jni_interception/JI05:/export/local/common/jdk/baseline/solaris-sparc/jre/lib/sparc/client
SHELL=/usr/bin/bash
DISPLAY=

Signal Handlers:
SIGSEGV: [libjvm.so+0xbb1f08], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGBUS: [libjvm.so+0xbb1f08], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGFPE: [libjvm.so+0x973a14], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGPIPE: [libjvm.so+0x973a14], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGXFSZ: [libjvm.so+0x973a14], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGILL: [libjvm.so+0x973a14], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGQUIT: [libjvm.so+0x96d2f0], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGHUP: [libjvm.so+0x96d2f0], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGINT: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGTERM: [libjvm.so+0x96d2f0], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIG39: [libjvm.so+0x973a34], sa_mask[0]=0x00000000, sa_flags=0x00000008
SIG40: [libjvm.so+0x973a14], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c


---------------  S Y S T E M  ---------------

OS:                          Oracle Solaris 11 11/11 SPARC
  Copyright (c) 1983, 2011, Oracle and/or its affiliates.  All rights reserved.
                            Assembled 18 October 2011

uname:SunOS 5.11 11.0 sun4v
  (T2 libthread)
rlimit: STACK 8192k, CORE infinity, NOFILE 65536, AS infinity
load average:3.48 3.00 2.86

CPU:total 8 v9, popc, vis1, vis2, vis3, blk_init, cbcond, sun4v, niagara_plus

Memory: 8k page, physical 18350080k(7329128k free)

vm_info: Java HotSpot(TM) Client VM (25.0-b07-internal-201210292336.yqi.hotspot-fastdebug) for solaris-sparc JRE (1.8.0), built on Oct 29 2012 17:06:05 by "jprtadm" with Sun Studio 12u1

time: Wed Oct 31 10:18:12 2012
elapsed time: 5 seconds

# Host info: SunOS spb23070 5.11 11.0 sun4v sparc sun4v
                                     
2012-10-31
On Windows, the assertions indicate overlapped reserved memory ranges. Apparently, they are the cases with thread stacks, following is one case:

Java Threads: ( => current thread )
  0x0000000009bb5000 JavaThread "Service Thread" daemon [_thread_blocked, id=8644, stack(0x000000000c900000,0x000000000ca00000)]        <====
  0x0000000009a88800 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=2812, stack(0x000000000c780000,0x000000000c880000)]
  0x0000000009a84800 JavaThread "C2 CompilerThread0" daemon [_thread_in_vm, id=7796, stack(0x000000000ca20000,0x000000000cb20000)]
  0x0000000009bb4800 JavaThread "Thread-1" [_thread_in_native, id=6328, stack(0x000000000c8b0000,0x000000000c9b0000)]                            <====
  0x0000000009b38800 JavaThread "Thread-0" [_thread_in_native, id=9116, stack(0x000000000c5f0000,0x000000000c6f0000)]
  0x0000000009b28000 JavaThread "Attach Listener" daemon [_thread_blocked, id=9700, stack(0x000000000c4b0000,0x000000000c5b0000)]
  0x0000000009b1f000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=10224, stack(0x000000000c2f0000,0x000000000c3f0000)]
  0x0000000009ab3800 JavaThread "Finalizer" daemon [_thread_blocked, id=3140, stack(0x000000000c130000,0x000000000c230000)]
  0x0000000009aa1800 JavaThread "Reference Handler" daemon [_thread_blocked, id=7736, stack(0x000000000bfd0000,0x000000000c0d0000)]
  0x0000000001edf000 JavaThread "main" [_thread_blocked, id=9832, stack(0x0000000002140000,0x0000000002240000)]
                                     
2012-10-26
I split up this bug into 2, since they are total different issues.

The assertion failures on Windows are due to overlapping stacks, which have nothing to do with NMT. 

BubbleUpRef.java case is due to NMT not handling overlapped commit, which should be allowed within the reserved range, and it will be addressed by this bug.
                                     
2012-10-27
Add link to nightly failures:
http://aurora.ru.oracle.com/functional/faces/RunDetails.xhtml?names=117229.JAVASE.NIGHTLY.VM.RT_Baseline.2012-10-25-300#JT_HS%20%28jtreg%29_closed/gc/4950157/BubbleUpRef.java


                                     
2012-10-29
The failures:

nsk/jvmti/SetJNIFunctionTable/setjniftab001 
nsk/jvmti/scenarios/jni_interception/JI05/ji05t001

could be explained by Attached threads not being alive anymore. However the failure in

closed/gc/4950157/BubbleUpRef.java 

cannot, there is no jni/jvmti lib loaded and looking at the hs_err it doesn't seem like it thinks any of the stacks overlap. This particular failure looks unrelated to the overlapping stack issue.
                                     
2012-11-02
URL:   http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/69ad7823b1ca
User:  zgu
Date:  2012-11-06 00:00:13 +0000

                                     
2012-11-06
URL:   http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/69ad7823b1ca
User:  amurillo
Date:  2012-11-09 19:30:16 +0000

                                     
2012-11-09
Verification: 

JDK:      jdk8 build 97
OS/ENV:   ruger.us.oracle.com (Solaris 10, x86_64)
CMD:      ../jdk/jdk8_b97/jdk1.8.0/fastdebug/bin/java -jar ../tools/jtreg/lib/jtreg.jar ../tests/test/closed/gc/4950157/BubbleUpRef.java
RUNS:     3
RESULT:   PASS

                                     
2013-07-08
JDK:      jdk8 build 96
OS/ENV:   Windows Server 2008 R2, 64bit (slcaf439.us.oracle.com)
CMD:      /cygdrive/s/tools/gtee/bin/ute -jdk /cygdrive/c/local/reproduce/b96 -test nsk/jvmti/SetJNIFunctionTable/setjniftab001 -component vm -tlists empty -env ENABLE_LOCAL_TESTBASE=1 -vmopts '-XX:NativeMemoryTracking=detail'
	  /cygdrive/s/tools/gtee/bin/ute -jdk /cygdrive/c/local/reproduce/b96 -test nsk/jvmti/scenarios/jni_interception/JI05/ji05t001 -component vm -tlists empty -env ENABLE_LOCAL_TESTBASE=1 -vmopts '-XX:NativeMemoryTracking=detail'

RUNS:     3
RESULT:   PASS

                                     
2013-07-10



Hardware and Software, Engineered to Work Together