JDK-8008071 : Crashed in promote_malloc_records() with Kitchensink after 19 days
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: hs24
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2013-02-12
  • Updated: 2013-07-18
  • Resolved: 2013-02-21
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8 Other
7u40Fixed 8Fixed hs24Fixed
Description
Got the following crash with Kitchensink after 19 days of uptime:

00000000`15a1e4e0 00000000`6e696c5a jvm!VMError::report_and_die+0x171 [c:\jdk7u2_64p\jdk7u14\hotspot\src\share\vm\utilities\vmerror.cpp @ 853]
00000000`15a1e6d0 00000000`6e99f98f jvm!topLevelExceptionFilter+0x35a [c:\jdk7u2_64p\jdk7u14\hotspot\src\os\windows\vm\os_windows.cpp @ 2555]
00000000`15a1e770 00000000`72fc1213 jvm!`java_start'::`1'::filt$0+0xf [c:\jdk7u2_64p\jdk7u14\hotspot\src\os\windows\vm\os_windows.cpp @ 422]
00000000`15a1e7a0 00000000`6e99f0a9 msvcr100!__C_specific_handler+0x97
00000000`15a1e810 00000000`771858dd jvm!__GSHandlerCheck_SEH+0x75
00000000`15a1e840 00000000`771896d7 ntdll!RtlpExecuteHandlerForException+0xd
00000000`15a1e870 00000000`77196e08 ntdll!RtlDispatchException+0x20c
00000000`15a1ef10 00000000`6e6763a1 ntdll!KiUserExceptionDispatcher+0x2e
00000000`15a1f4b0 00000000`6e676cac jvm!MemSnapshot::promote_malloc_records+0x1a1 [c:\jdk7u2_64p\jdk7u14\hotspot\src\share\vm\services\memsnapshot.cpp @ 529]
00000000`15a1f510 00000000`6e677ef1 jvm!MemSnapshot::promote+0x5c [c:\jdk7u2_64p\jdk7u14\hotspot\src\share\vm\services\memsnapshot.cpp @ 490]
00000000`15a1f580 00000000`6e69775e jvm!MemTrackWorker::run+0x101 [c:\jdk7u2_64p\jdk7u14\hotspot\src\share\vm\services\memtrackworker.cpp @ 121]
00000000`15a1f5c0 00000000`72f71db7 jvm!java_start+0x9e [c:\jdk7u2_64p\jdk7u14\hotspot\src\os\windows\vm\os_windows.cpp @ 421]

Had a debugger attached to track handle leaks so the memory dump failed. Re-running to see if I can reproduce this. 

Managed to create a callstack only dump with some information in it:

Debug session time: Tue Feb 12 17:47:27.000 2013 (UTC + 1:00)
System Uptime: not available
Process Uptime: 19 days 2:46:52.000


Local var @ 0x15a1e4a0 Type _EXCEPTION_POINTERS
   +0x000 ExceptionRecord  : 0x00000000`15a1f3e0 _EXCEPTION_RECORD
      +0x000 ExceptionCode    : 0n-1073741819
      +0x004 ExceptionFlags   : 0
      +0x008 ExceptionRecord  : (null) 
      +0x010 ExceptionAddress : 0x00000000`6e6763a1 Void
      +0x018 NumberParameters : 2
      +0x020 ExceptionInformation : [15] 0
   +0x008 ContextRecord    : 0x00000000`15a1ef10 _CONTEXT
      +0x000 P1Home           : 0x15a1f3e0
      +0x008 P2Home           : 0x20706f98
      +0x010 P3Home           : 0xfffffa60`00000000
      +0x018 P4Home           : 0x6e470000
      +0x020 P5Home           : 0
      +0x028 P6Home           : 0
      +0x030 ContextFlags     : 0x10001f
      +0x034 MxCsr            : 0x1fa0
      +0x038 SegCs            : 0x33
      +0x03a SegDs            : 0x2b
      +0x03c SegEs            : 0x2b
      +0x03e SegFs            : 0x53
      +0x040 SegGs            : 0x2b
      +0x042 SegSs            : 0x2b
      +0x044 EFlags           : 0x10246
      +0x048 Dr0              : 0
      +0x050 Dr1              : 0
      +0x058 Dr2              : 0
      +0x060 Dr3              : 0
      +0x068 Dr6              : 0
      +0x070 Dr7              : 0
      +0x078 Rax              : 0xc930
      +0x080 Rcx              : 0
      +0x088 Rdx              : 0x3edeb
      +0x090 Rbx              : 0x20706f98
      +0x098 Rsp              : 0x15a1f4b0
      +0x0a0 Rbp              : 0x15a1f4f0
      +0x0a8 Rsi              : 0x13a59ea0
      +0x0b0 Rdi              : 0xc930
      +0x0b8 R8               : 0x20510040
      +0x0c0 R9               : 0xc930
      +0x0c8 R10              : 0x139360d0
      +0x0d0 R11              : 0
      +0x0d8 R12              : 0x15a1f530
      +0x0e0 R13              : 0x13a59ea0
      +0x0e8 R14              : 0
      +0x0f0 R15              : 0
      +0x0f8 Rip              : 0x6e6763a1

Comments
ILW: HML => P2 I: H, JVM crashes L: M W: L, turn off NMT
19-02-2013

Yes, the problem here is that that promote_malloc_records() needs to handle this.
19-02-2013

Seems perfectly legitimate for peek_next() to return null if the matched record was the last entry.
19-02-2013

From memSnapshot.cpp: // an arena record can be followed by a size record, we need to remove both if (matched_rec->is_arena_record()) { MemPointerRecord* next = (MemPointerRecord*)malloc_snapshot_itr.peek_next(); if (next->is_arena_memory_record() && next->is_memory_record_of_arena(matched_rec)) { peek_next() seems to have returned null, Zhengyu agrees that this looks like a bug. Assigning to Zhengyu.
18-02-2013

Managed to reproduce after almost 3 days, crashing in promote_malloc_records: 0 00000000`1337f530 00000000`6e676cac jvm!MemSnapshot::promote_malloc_records+0x1a1 [c:\jdk7u2_64p\jdk7u14\hotspot\src\share\vm\services\memsnapshot.cpp @ 529] 01 00000000`1337f590 00000000`6e677ef1 jvm!MemSnapshot::promote+0x5c [c:\jdk7u2_64p\jdk7u14\hotspot\src\share\vm\services\memsnapshot.cpp @ 490] 02 00000000`1337f600 00000000`6e69775e jvm!MemTrackWorker::run+0x101 [c:\jdk7u2_64p\jdk7u14\hotspot\src\share\vm\services\memtrackworker.cpp @ 121] 03 00000000`1337f640 00000000`73a21db7 jvm!java_start+0x9e [c:\jdk7u2_64p\jdk7u14\hotspot\src\os\windows\vm\os_windows.cpp @ 421] 04 00000000`1337f880 00000000`73a21e53 msvcr100!_callthreadstartex+0x17 [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c @ 314] 05 00000000`1337f8b0 00000000`76d5b22d msvcr100!_threadstartex+0x7f [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c @ 292] 06 00000000`1337f8e0 00000000`77176861 kernel32!BaseThreadInitThunk+0xd 07 00000000`1337f910 00000000`00000000 ntdll!RtlUserThreadStart+0x1d This time I have a complete mdmp and an hs_err file.
18-02-2013