JDK-7104960 : JSR 292: +VerifyMethodHandles in product JVM can overflow buffer
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: hs23
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: solaris_10
  • CPU: x86
  • Submitted: 2011-10-26
  • Updated: 2012-03-22
  • Resolved: 2012-01-23
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8 Other
7u4Fixed 8Fixed hs23Fixed
On Oct 25, 2011, at 12:23 PM, Tom Rodriguez wrote:

Thanks for the pointer.  I checked in crashes, where they are normally captured and didn't see anything.  I didn't expect the bundle to end up with that data.

I tried adding some code to verify that the boundaries on the code blobs are being respected but didn't see any problems.  Looking more closely at the core file and a regular run I think I see what's happening though.  The location of mappings in the JVM that crashed are slightly worse from a code generation perspective so we have to emit 8 instructions to emit some constants.  This causes code expansion which causes us to overflow the buffer.  Most of the time the values are smaller so that's why it's hard to reproduce the crash.

An interim fix that could be pushed into the JDK would be to remove the VerifyMethodHandles option from that test and maybe others in that java/lang/invoke suite.  I'm not actually sure why that's a product flag in the first place.  That's causes a bunch extra assembly to be generated that we wouldn't normally allow in a product JVM.  Turning it off should bring the total space required down to a safe level.

I think what's happening is that a stub is running off the end its code blob and overwriting space which is on the free list.  If search_freelist looks at it after that then it crashes.  The free list pointer appears to be 0x7fffffff78067900.

(dbx) x 0x7fffffff78067900/4lx 
0x7fffffff78067900:      0xd875000a9a10000e 0x1f1fffff9e13e3ff
0x7fffffff78067910:      0x9f2bf00c9e13e780 0x9f2bf00a9e13e014
(dbx) p -flx $l7               
$l7 = 0x9f2bf00c9e13e780
(dbx) dis 0x7fffffff78067900         
0x7fffffff78067900:     stx      %o4, [%l4 + %o2]
0x7fffffff78067904:     mov      %sp, %o5
0x7fffffff78067908:     sethi    %hi(0x7ffffc00), %o7
0x7fffffff7806790c:     bset     0x00000000000003ff, %o7        ! 0x7fffffff
0x7fffffff78067910:     sllx     %o7, 12, %o7

disassembly further back looks like this:

dbx) dis 0x7fffffff78067890         
0x7fffffff78067890:     ldx      [%o0 + 24], %o2
0x7fffffff78067894:     stx      %o2, [%g4 + 24]
0x7fffffff78067898:     ldx      [%o0 + 16], %o2
0x7fffffff7806789c:     stx      %o2, [%g4 + 16]
0x7fffffff780678a0:     ldx      [%o0 + 8], %o2
0x7fffffff780678a4:     stx      %o2, [%g4 + 8]
0x7fffffff780678a8:     ldx      [%o0], %o2
0x7fffffff780678ac:     stx      %o2, [%g4]
0x7fffffff780678b0:     and      %l5, 255, %o2
0x7fffffff780678b4:     add      %l4, -8, %o3
(dbx) dis
0x7fffffff780678b8:     cmp      %o2, 0
0x7fffffff780678bc:     be,pn    %icc,0x7fffffff780678f4        ! 0x7fffffff780678f4
0x7fffffff780678c0:     nop      
0x7fffffff780678c4:     sllx     %o2, 3, %o4
0x7fffffff780678c8:     add      %l4, %o4, %o0
0x7fffffff780678cc:     add      %l4, 0, %o4
0x7fffffff780678d0:     cmp      %o4, %o0
0x7fffffff780678d4:     bcc,pn   %xcc,0x7fffffff780678f4        ! 0x7fffffff780678f4
0x7fffffff780678d8:     nop      
0x7fffffff780678dc:     ldx      [%o4], %g5
(dbx) dis
0x7fffffff780678e0:     stx      %g5, [%o4 - 8]
0x7fffffff780678e4:     inc      8, %o4
0x7fffffff780678e8:     cmp      %o4, %o0
0x7fffffff780678ec:     bcs,pt   %xcc,0x7fffffff780678dc        ! 0x7fffffff780678dc
0x7fffffff780678f0:     nop      
0x7fffffff780678f4:     mov      %o3, %l4
0x7fffffff780678f8:     clr      %o4
0x7fffffff780678fc:     sllx     %o2, 3, %o2
0x7fffffff78067900:     stx      %o4, [%l4 + %o2]
0x7fffffff78067904:     mov      %sp, %o5

I thought it was an arraycopy stub at first but I think it's one of the stubs for method handles.  I'll take a look and see what's happening.


EVALUATION http://hg.openjdk.java.net/lambda/lambda/hotspot/rev/44ce519bc3d1


EVALUATION http://hg.openjdk.java.net/hsx/hotspot-emb/hotspot/rev/44ce519bc3d1

EVALUATION http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/44ce519bc3d1

EVALUATION 7104960: JSR 292: +VerifyMethodHandles in product JVM can overflow buffer Reviewed-by: kvn, jrose, twisti The code buffers on limits on sparc didn't properly account for space required then the code cache and libjvm.so were farther apart than an int so it would slightly overflow it's buffer. It would crash because the CodeCache became corrupted instead of reporting that it overflowed. I modified the overflow detection logic to work even in product. It doesn't check on every operation as it does in debug mode, it just vierifies at the end that it hasn't overflowed. I also extended the ForceUnreachable logic to work on sparc and turned it into a diagnostic so it can be tested in product. I added logging code to report buffer usage and reset the values on sparc and x86. If left about 4k of buffer in product mode and 20k in debug mode.

EVALUATION http://hg.openjdk.java.net/hsx/hotspot-comp/hotspot/rev/44ce519bc3d1