JDK-8208091 : SA: jhsdb jstack --mixed throws UnmappedAddressException on i686
  • Type: Bug
  • Component: hotspot
  • Sub-Component: svc-agent
  • Affected Version: 8u172,11
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • CPU: x86
  • Submitted: 2018-07-23
  • Updated: 2022-10-06
  • Resolved: 2018-08-24
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 12 JDK 8
11.0.2Fixed 12 b09Fixed 8u202Fixed
Related Reports
Relates :  
JDK-6310967 - SA: jstack -m produce failures in output
Relates :  
JDK-8294920 - Missing SP value in Linux x86_32 thread context
Description
On 32-bit x86 and Linux we are seeing UnmappedAddressException on jhsdb jstack --mixed:

$ cat Foo.java 
public class Foo{
  public static void main(String[] arg){
     System.out.println("Hello");
     try {
       while(true)
         Thread.sleep(1000*60);
     }catch(InterruptedException ex){
     }
     System.out.println("Bye");
  }
}
$ ./build/linux-x86-normal-server-release/images/jdk/bin/java Foo &
$ ./build/linux-x86-normal-server-release/images/jdk/bin/jps
26872 Jps
26733 Foo
$ ./build/linux-x86-normal-server-release/images/jdk/bin/jhsdb jstack --pid 26733 --mixed
Attaching to process ID 26733, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 11-internal+0-adhoc.root.openjdk
Deadlock Detection:

No deadlocks found.

----------------- 26734 -----------------
"main" #1 prio=5 tid=0xf620ac00 nid=0x686e waiting on condition [0xf63a8000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
   JavaThread state: _thread_blocked
0xf7768430              ????????
----------------- 26735 -----------------
0xf7768430              ????????
sun.jvm.hotspot.debugger.UnmappedAddressException: f0
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.PageCache.checkPage(PageCache.java:208)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.PageCache.getData(PageCache.java:63)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.DebuggerBase.readBytes(DebuggerBase.java:225)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:494)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:462)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:429)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:69)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:165)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:58)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:53)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:259)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:450)
----------------- 26736 -----------------
0xf7768430              ????????
----------------- 26737 -----------------
0xf7768430              ????????
sun.jvm.hotspot.debugger.UnmappedAddressException: f0
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.PageCache.checkPage(PageCache.java:208)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.PageCache.getData(PageCache.java:63)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.DebuggerBase.readBytes(DebuggerBase.java:225)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:494)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:462)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:429)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:69)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:165)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:58)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:53)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:259)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:450)
----------------- 26738 -----------------
0xf7768430              ????????
----------------- 26739 -----------------
0xf7768430              ????????
----------------- 26740 -----------------
0xf7768430              ????????
----------------- 26741 -----------------
"Reference Handler" #2 daemon prio=10 tid=0xa392f000 nid=0x6875 waiting on condition [0xa457c000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_blocked
0xf7768430              ????????
----------------- 26742 -----------------
"Finalizer" #3 daemon prio=8 tid=0xa3932000 nid=0x6876 in Object.wait() [0xa437c000]
   java.lang.Thread.State: WAITING (on object monitor)
   JavaThread state: _thread_blocked
0xf7768430              ????????
----------------- 26743 -----------------
"Signal Dispatcher" #4 daemon prio=9 tid=0xa393a800 nid=0x6877 runnable [0x00000000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_blocked
0xf7768430              ????????
sun.jvm.hotspot.debugger.UnmappedAddressException: f0
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.PageCache.checkPage(PageCache.java:208)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.PageCache.getData(PageCache.java:63)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.DebuggerBase.readBytes(DebuggerBase.java:225)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:494)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:462)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:429)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74)
        at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:69)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:165)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:58)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:53)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:259)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:450)
----------------- 26744 -----------------
"C2 CompilerThread0" #5 daemon prio=9 tid=0xa393c000 nid=0x6878 waiting on condition [0x00000000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_blocked
0xf7768430              ????????
----------------- 26745 -----------------
"C1 CompilerThread0" #7 daemon prio=9 tid=0xa393e800 nid=0x6879 waiting on condition [0x00000000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_blocked
0xf7768430              ????????
----------------- 26746 -----------------
"Sweeper thread" #8 daemon prio=9 tid=0xa3940000 nid=0x687a runnable [0x00000000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_blocked
0xf7768430              ????????
----------------- 26747 -----------------
"Service Thread" #9 daemon prio=9 tid=0xa398bc00 nid=0x687b runnable [0x00000000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_blocked
0xf7768430              ????????
----------------- 26748 -----------------
0xf7768430              ????????
----------------- 26749 -----------------
"Common-Cleaner" #10 daemon prio=8 tid=0xa3993000 nid=0x687d in Object.wait() [0xa33aa000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
   JavaThread state: _thread_blocked
0xf7768430              ????????
----------------- 26733 -----------------
0xf7768430              ????????
Comments
Fix Request: Please approve this fix for JDK 11u. It's a local fix for linux/windows i686 (32 bit) serviceability agent. It does not affect other architectures. It's been in JDK 12 for over a month and hasn't caused issues so far. Testing: new regression test on an affected system. Risk is low since it's i686 only. JDK 12 patch applies as is.
02-10-2018
URL: http://hg.openjdk.java.net/jdk/jdk/rev/2e98c7737d8f User: sgehwolf Date: 2018-08-24 12:31:27 +0000
24-08-2018
This got reviewed[1], but isn't going to get pushed until August 24, 2018. That's what Chris asked me to do. HG exported patch: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8208091/JDK-8208091.final.export.patch [1] http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-August/024720.html
10-08-2018
Latest webrev with JNI compilation issue fixed: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8208091/webrev.03/
06-08-2018
Latest webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8208091/webrev.02/ There is still an issue with getting the JNI lib compiled with -fomit-frame-pointer.
03-08-2018
windows 32 bit is likely facing similar issues. From src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp: #undef REG_INDEX #ifdef _M_IX86 #define REG_INDEX(x) sun_jvm_hotspot_debugger_x86_X86ThreadContext_##x context.ContextFlags = CONTEXT_FULL | CONTEXT_DEBUG_REGISTERS; ptrIDebugAdvanced->GetThreadContext(&context, sizeof(CONTEXT)); ptrRegs[REG_INDEX(GS)] = context.SegGs; ptrRegs[REG_INDEX(FS)] = context.SegFs; ptrRegs[REG_INDEX(ES)] = context.SegEs; ptrRegs[REG_INDEX(DS)] = context.SegDs; ptrRegs[REG_INDEX(EDI)] = context.Edi; ptrRegs[REG_INDEX(ESI)] = context.Esi; ptrRegs[REG_INDEX(EBX)] = context.Ebx; ptrRegs[REG_INDEX(EDX)] = context.Edx; ptrRegs[REG_INDEX(ECX)] = context.Ecx; ptrRegs[REG_INDEX(EAX)] = context.Eax; ptrRegs[REG_INDEX(FP)] = context.Ebp; ptrRegs[REG_INDEX(PC)] = context.Eip; ptrRegs[REG_INDEX(CS)] = context.SegCs; ptrRegs[REG_INDEX(EFL)] = context.EFlags; ptrRegs[REG_INDEX(SP)] = context.Esp; So the change in Java code to use SP over ESP seems correct, fixing both Linux 32bit and windows 32bit SA.
03-08-2018
One clarification: The issue arises in stacks of native code which haven't been compiled with -fno-omit-frame-pointer AND excecution of the target program is caught at "pc() + offset % 4 == 0". Otherwise another framepointer validity check kicks in not showing the issue. From LinuxX86CFrame.java: // Check alignment of ebp if ( dbg.getAddressValue(ebp) % ADDRESS_SIZE != 0) { return null; } The test will check for this interesting condition in a loop and expects in this interesting case that no UnmappedAddressException is being thrown. If it doesn't hit the interesting condition in 20 iterations and no UnmappedAddressException is being thrown for other reasons the test passes too.
03-08-2018
Review-thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-July/024556.html
23-07-2018
webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8208091/webrev.01/
23-07-2018
The fix is a one-liner like this: diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/x86/LinuxX86CFrame.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/x86/LinuxX86CFrame.java --- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/x86/LinuxX86CFrame.java +++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/x86/LinuxX86CFrame.java @@ -55,7 +55,7 @@ public CFrame sender(ThreadProxy thread) { X86ThreadContext context = (X86ThreadContext) thread.getContext(); - Address esp = context.getRegisterAsAddress(X86ThreadContext.ESP); + Address esp = context.getRegisterAsAddress(X86ThreadContext.SP); if ( (ebp == null) || ebp.lessThan(esp) ) { return null;
23-07-2018
The issue arises in stacks of native code which haven't been compiled with -fno-omit-frame-pointer. In that case a simple frame pointer validity check should kick in, but that checking code has a subtle bug. The native code filling in register values and the Java code of the frame pointer check is not in sync. From src/jdk.hotspot.agent/linux/native/libsaproc/LinuxDebuggerLocal.c around line 404 we have this code: #ifdef i386 #define REG_INDEX(reg) sun_jvm_hotspot_debugger_x86_X86ThreadContext_##reg regs[REG_INDEX(GS)] = (uintptr_t) gregs.xgs; regs[REG_INDEX(FS)] = (uintptr_t) gregs.xfs; regs[REG_INDEX(ES)] = (uintptr_t) gregs.xes; regs[REG_INDEX(DS)] = (uintptr_t) gregs.xds; regs[REG_INDEX(EDI)] = (uintptr_t) gregs.edi; regs[REG_INDEX(ESI)] = (uintptr_t) gregs.esi; regs[REG_INDEX(FP)] = (uintptr_t) gregs.ebp; regs[REG_INDEX(SP)] = (uintptr_t) gregs.esp; regs[REG_INDEX(EBX)] = (uintptr_t) gregs.ebx; regs[REG_INDEX(EDX)] = (uintptr_t) gregs.edx; regs[REG_INDEX(ECX)] = (uintptr_t) gregs.ecx; regs[REG_INDEX(EAX)] = (uintptr_t) gregs.eax; regs[REG_INDEX(PC)] = (uintptr_t) gregs.eip; regs[REG_INDEX(CS)] = (uintptr_t) gregs.xcs; regs[REG_INDEX(SS)] = (uintptr_t) gregs.xss; #endif /* i386 */ In LinuxX86CFrame from file src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/x86/LinuxX86CFrame.java around line 58 we see this frame pointer validity check - ebp.lessThan(esp): public CFrame sender(ThreadProxy thread) { X86ThreadContext context = (X86ThreadContext) thread.getContext(); Address esp = context.getRegisterAsAddress(X86ThreadContext.ESP); if ( (ebp == null) || ebp.lessThan(esp) ) { return null; } So the Java code uses X86ThreadContext.ESP, but the native code only sets X86ThreadContext.SP or X86ThreadContext.UESP, hence the check returns a false negative and later on this code implicitly throws the exception: Address nextEBP = ebp.getAddressAt( 0 * ADDRESS_SIZE);
23-07-2018
Downstream bug where issue was originally discovered: https://bugzilla.redhat.com/show_bug.cgi?id=1602008
23-07-2018