Bug ID: JDK-4703989 String.compareTo may walk one char past the end of array

Type: Bug
Component: hotspot
Sub-Component: compiler
Affected Version: 1.3.1,1.3.1_03,1.3.1_06,1.4.2

Priority: P3
Status: Resolved
Resolution: Fixed
OS: solaris_8
CPU: sparc

Submitted: 2002-06-18
Updated: 2002-11-01
Resolved: 2002-08-28

Other	Other	Other	Other
1.3.1_07 07Fixed	1.4.0_04Fixed	1.4.1_02Fixed	1.4.2Fixed

This crash occurred while doing a J2EE build with jdk1.3.1_03 on solaris sparc.

Date: Mon, 17 Jun 2002 16:09:20 -0700 (PDT)
From: Janet Koenig <###@###.###>
Subject: HotSpot Crash
To: ###@###.###, ###@###.###, ###@###.###
Cc: ###@###.###, ###@###.###

Does this look like a C1 crash to you guys?  It happens more frequently
on a big server (E250, E450).  Less frequently on Ultra 5:


Unexpected Signal : 11 occurred at PC=0xfb030734
Function name=compareTo (compiled Java code)
Library=(N/A)

Current Java thread:

Dynamic libraries:
0x10000 	
/net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/bin/../bin/sparc/nativ
e_threads/java
0xff350000 	/usr/lib/libthread.so.1
0xff390000 	/usr/lib/libdl.so.1
0xff200000 	/usr/lib/libc.so.1
0xff330000 	/usr/platform/SUNW,Ultra-250/lib/libc_psr.so.1
0xfe480000 	
/net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/client/l
ibjvm.so
0xff2d0000 	/usr/lib/libCrun.so.1
0xff1e0000 	/usr/lib/libsocket.so.1
0xff100000 	/usr/lib/libnsl.so.1
0xff0d0000 	/usr/lib/libm.so.1
0xff300000 	/usr/lib/libw.so.1
0xff0b0000 	/usr/lib/libmp.so.2
0xff080000 	
/net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/native_t
hreads/libhpi.so
0xff050000 	
/net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/libverif
y.so
0xfe440000 	
/net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/libjava.
so
0xff020000 	
/net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/libzip.s
o
0xfe230000 	
/net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/libioser
12.so

Local Time = Mon Jun 17 14:17:52 2002
Elapsed Time = 4
#
# HotSpot Virtual Machine Error : 11
# Error ID : 4F530E43505002BD 01
# Please report this error at
# http://java.sun.com/cgi-bin/bugreport.cgi
#
# Java VM: Java HotSpot(TM) Client VM (1.3.1_03-b03 mixed mode)
#

CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: 1.3.1_07 1.4.0_04 1.4.1_02 mantis FIXED IN: 1.3.1_07 1.4.0_04 1.4.1_02 mantis INTEGRATED IN: 1.3.1_07 1.4.0_04 1.4.1_02 mantis
14-06-2004
SUGGESTED FIX * /tmp/geta19798 Thu Jun 20 11:27:49 2002 --- c1_CodeEmitter_sparc.cpp Thu Jun 20 11:27:37 2002 *********** * 3545,3551 **** __ br(Assembler::notZero, false, Assembler::pt, Ldone); assert(chr0 == result, "result must be pre-placed"); __ delayed()->inccc(limit, sizeof(jchar)); ! __ br(Assembler::notZero, false, Assembler::pt, Lloop); __ delayed()->lduh(base0, limit, chr0); } --- 3545,3551 ---- __ br(Assembler::notZero, false, Assembler::pt, Ldone); assert(chr0 == result, "result must be pre-placed"); __ delayed()->inccc(limit, sizeof(jchar)); ! __ br(Assembler::notZero, true, Assembler::pt, Lloop); __ delayed()->lduh(base0, limit, chr0); }
11-06-2004
EVALUATION There's a load in the delay slot of the main compare loop that should be annulled. This is very hard to reproduce since you have to compare a String whose char[] is the last object in the tenured generation. The fix is simple but there isn't really a test case. ###@###.### 2002-06-18 Date: Tue, 18 Jun 2002 09:59:05 -0700 (PDT) From: Thomas Rodriguez <###@###.###> Subject: Re: HotSpot Crash To: ###@###.###, ###@###.### Cc: ###@###.###, ###@###.###, ###@###.###, ###@###.### I think I know what it is. We emit a special implemenation of String.compareTo and whoever wrote it put a load in the delay slot of a branch without annulling it, so it loads one char past the end of the array. The string being compared appears to be right at the end of a generation, so it's actually loading memory outside of any allocated space. This used to be harmless but one of the changes we put back into 1.3.1_03 was to modify os::uncommit_memory to use PROT_NONE so that touching uncommitted memory would result in a crash and I think that's what's happening from looking at the core file. I think you can make the problem disappear by picking some different heap parameters since that should perturb the allocation locations and put the string somewhere else. We should probably fix this in 1.3.1_04 and in mantis since I think we just added to PROT_NONE change to mantis a couple weeks ago. I've tried to write a test case for this but haven't had much luck, which is good I guess. I'll file a bug for this. tom > Date: Mon, 17 Jun 2002 16:40:38 -0700 (PDT) > From: Rajiv Mordani <###@###.###> > X-Sender: mode@nine > To: Thomas Rodriguez <###@###.###> > cc: ###@###.###, ###@###.###, ###@###.###, ###@###.### > Subject: Re: HotSpot Crash > > The core file is at /net/wampum/export/mode/wspack/core > > > - Rajiv > -- > :wq > > On Mon, 17 Jun 2002, Thomas Rodriguez wrote: > > > It's dying in compiled code so it's most likely a C1 bug. It's rare for runtime > > bugs to cause crashes in compiled code. Is there a test case or core file? > > > > tom > > > > > Date: Mon, 17 Jun 2002 16:09:20 -0700 (PDT) > > > From: Janet Koenig <###@###.###> > > > Subject: HotSpot Crash > > > To: ###@###.###, ###@###.###, > > ###@###.### > > > Cc: ###@###.###, ###@###.### > > > > > > Does this look like a C1 crash to you guys? It happens more frequently > > > on a big server (E250, E450). Less frequently on Ultra 5: > > > > > > > > > Unexpected Signal : 11 occurred at PC=0xfb030734 > > > Function name=compareTo (compiled Java code) > > > Library=(N/A) > > > > > > Current Java thread: > > > > > > Dynamic libraries: > > > 0x10000 > > > > > /net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/bin/../bin/sparc/nativ > > > e_threads/java > > > 0xff350000 /usr/lib/libthread.so.1 > > > 0xff390000 /usr/lib/libdl.so.1 > > > 0xff200000 /usr/lib/libc.so.1 > > > 0xff330000 /usr/platform/SUNW,Ultra-250/lib/libc_psr.so.1 > > > 0xfe480000 > > > > > /net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/client/l > > > ibjvm.so > > > 0xff2d0000 /usr/lib/libCrun.so.1 > > > 0xff1e0000 /usr/lib/libsocket.so.1 > > > 0xff100000 /usr/lib/libnsl.so.1 > > > 0xff0d0000 /usr/lib/libm.so.1 > > > 0xff300000 /usr/lib/libw.so.1 > > > 0xff0b0000 /usr/lib/libmp.so.2 > > > 0xff080000 > > > > > /net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/native_t > > > hreads/libhpi.so > > > 0xff050000 > > > > > /net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/libverif > > > y.so > > > 0xfe440000 > > > > > /net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/libjava. > > > so > > > 0xff020000 > > > > > /net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/libzip.s > > > o > > > 0xfe230000 > > > > > /net/koori.sfbay/a/v01/jdk/1.3.1_03/fcs/binaries/solsparc/jre/lib/sparc/libioser > > > 12.so > > > > > > Local Time = Mon Jun 17 14:17:52 2002 > > > Elapsed Time = 4 > > > # > > > # HotSpot Virtual Machine Error : 11 > > > # Error ID : 4F530E43505002BD 01 > > > # Please report this error at > > > # http://java.sun.com/cgi-bin/bugreport.cgi > > > # > > > # Java VM: Java HotSpot(TM) Client VM (1.3.1_03-b03 mixed mode) > > > # > > > > > > ###@###.### 2002-06-18 I realized that the other condition for seeing this bug is that the end of the tenured generation must also be exactly at the end of page, otherwise you won't see this failure. So basically it requires comparing a string whose char array is the last object in tenured and tenured ends exactly on a page boundary. A fairly amazing confluence of events. I also updated suggested fix to correspond to 1.3.1 sources instead of 1.4.1. ###@###.### 2002-06-20 As someone pointed out it doesn't seem like an amazing confluence of events when it happens to you every time but it is. It's just very deterministic as well so if you have a program which see it your are likely to see it in the future. ###@###.### 2002-07-16
16-07-2002
WORK AROUND You can create a .hotspot_compiler file in the directory where you run your program containing the line exclude java/lang/String compareTo This will keep us from compiling this method. You'll get extra output as well though, looking kind of like this. smite ~ % cat .hotspot_compiler exclude java/lang/String compareTo smite ~ % /export/ws/hopper/sparc/jdk1.4/bin/java stringcompare CompilerOracle: exclude java/lang/String compareTo ### Excluding compile: java.lang.String::compareTo You may also be able to tweak your initial heap size using -Xms to perturb the location of the string which happens to end up at the end of the heap. ###@###.### 2002-07-16
16-07-2002