JDK-6454444 : SEGV on solaris-i586 -server running ConcurrentQueueLoops
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 6
  • Priority: P1
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2006-07-30
  • Updated: 2010-12-07
  • Resolved: 2006-08-16
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other JDK 6
5.0u12Fixed 6 b96Fixed
Related Reports
Duplicate :  
JDK-6454475 - NPE on solaris-i586 -server running ConcurrentQueueLoops
Relates :  
JDK-5041651 - server compiler crash in b49; assertion in output.cpp:918
Relates :  
JDK-6450705 - NullPointerException in LinkedBlockingQueue.peek
Relates :  
JDK-6617341 - JVM crash at symbolOopDesc::as_klass_external_name
Description
I started seeing intermittent failures running jtreg test
test/java/util/concurrent/ConcurrentLinkedQueue/ConcurrentQueueLoops.java 
which started appearing since mustang b93 only when running -server

These can be most reliably reproduced on machine mho as follows:

 $ pwd; uname -a; repeat 100 (/java/re/jct-tools/2.1.6/archive/fcs/binaries/solaris/bin/jtreg -automatic -jdk:"/java/re/jdk/6/promoted/all/b93/binaries/solaris-i586" ConcurrentQueueLoops.java 2>&1 | tee jtreg.out; if grep -q failed jtreg.out; then cat ./JTwork/java/util/concurrent/ConcurrentLinkedQueue/ConcurrentQueueLoops.jtr; fi)
/u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue
SunOS mho 5.10 Generic_118855-14 i86pc i386 i86pc
test results: passed: 1
Report written to /u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/JTreport/report.html
Results written to /u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/JTwork
test results: failed: 1
Report written to /u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/JTreport/report.html
Results written to /u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/JTwork
Error: some tests failed or other problems occurred
#Test Results (version 2)
#Sun Jul 30 10:33:09 PDT 2006
#checksum:3c9cef4fcbc2b825
#-----testdescription-----
$file=/u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/ConcurrentQueueLoops.java
$root=/u/martin/ws/mustang/test
keywords=bug4486658
run=USER_SPECIFIED compile -source 1.5 ConcurrentQueueLoops.java\nUSER_SPECIFIED main/timeout=230 ConcurrentQueueLoops\n
source=ConcurrentQueueLoops.java
title=Checks that a set of threads can repeatedly get and modify items

#-----environment-----

#-----testresult-----
description=file:///u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/ConcurrentQueueLoops.java
end=Sun Jul 30 10:33:09 PDT 2006
environment=regtest
execStatus=Failed. Unexpected exit from test [exit code: 6]
javatestOS=SunOS 5.10 (x86)
javatestVersion=2.1.6
script=com.sun.javatest.regtest.RegressionScript 
sections=script_messages compile build main
start=Sun Jul 30 10:33:06 PDT 2006
status=Failed. Unexpected exit from test [exit code: 6]
test=java/util/concurrent/ConcurrentLinkedQueue/ConcurrentQueueLoops.java
work=/u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/JTwork/java/util/concurrent/ConcurrentLinkedQueue

#section:script_messages
----------messages:(4/209)----------
JDK under test: (/java/re/jdk/6/promoted/all/b93/binaries/solaris-i586)
java version "1.6.0-rc"
Java(TM) SE Runtime Environment (build 1.6.0-rc-b93)
Java HotSpot(TM) Server VM (build 1.6.0-rc-b93, mixed mode)

#section:compile
----------messages:(3/225)----------
command: compile /u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/ConcurrentQueueLoops.java 
reason: User specified action: run compile -source 1.5 ConcurrentQueueLoops.java 
elapsed time (seconds): 1.694
----------System.out:(0/0)----------
----------System.err:(0/0)----------
result: Passed. Compilation successful

#section:build
----------messages:(3/107)----------
command: build ConcurrentQueueLoops 
reason: Named class compiled on demand
elapsed time (seconds): 0.0010
result: Passed. No need to compile: ConcurrentQueueLoops.java

#section:main
----------messages:(3/139)----------
command: main ConcurrentQueueLoops
reason: User specified action: run main/timeout=230 ConcurrentQueueLoops 
elapsed time (seconds): 1.678
----------System.out:(15/486)----------
Warmup...
Threads: 1	:#
# An unexpected error has been detected by Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0xfb457079, pid=25526, tid=12
#
# Java VM: Java HotSpot(TM) Server VM (1.6.0-rc-b93 mixed mode)
# Problematic frame:
# J  ConcurrentQueueLoops$Stage.call()Ljava/lang/Integer;
#
# An error report file with more information is saved as hs_err_pid25526.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
----------System.err:(0/0)----------
result: Failed. Unexpected exit from test [exit code: 6]


test result: Failed. Unexpected exit from test [exit code: 6]


-----------------------------------------------------
-----------------------------------------------------
An hs_err log (from machine "bonsai") follows:

#
# An unexpected error has been detected by Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0xfb457079, pid=21867, tid=12
#
# Java VM: Java HotSpot(TM) Server VM (1.6.0-rc-b94 mixed mode)
# Problematic frame:
# J  ConcurrentQueueLoops$Stage.call()Ljava/lang/Integer;
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x08184c00):  JavaThread "pool-1-thread-1" [_thread_in_Java, id=12]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0xceb28306

Registers:
EAX=0x00000000, EBX=0xf32d8220, ECX=0x00000000, EDX=0xf32d8220
ESP=0xf2e769a0, EBP=0xf7416f58, ESI=0xf32d820c, EDI=0xf32d8220
EIP=0xfb457079, EFLAGS=0x00010246

Top of Stack: (sp=0xf2e769a0)
0xf2e769a0:   f3926680 08184c00 f32e0020 08184c00
0xf2e769b0:   f32e0010 f3926ec0 f32d8210 f32e0010
0xf2e769c0:   000186a0 000041a7 f32d8220 00000000
0xf2e769d0:   fb402c00 00000000 f3926680 fb40a877
0xf2e769e0:   08184ce8 f2e76a1c f2e76a18 fb402d67
0xf2e769f0:   fb402d67 f32695e0 f2e769f8 f741a6ed
0xf2e76a00:   f2e76a20 f741a938 00000000 f741a700
0xf2e76a10:   f2e769f4 f2e76a20 f2e76a48 fb403243 

Instructions: (pc=0xfb457079)
0xfb457069:   00 75 05 bb 01 00 00 00 8b ca c1 e9 09 c6 81 00
0xfb457079:   20 b6 fa 00 85 db 0f 84 87 00 00 00 8b 4c 24 2c 

Stack: [0xf2e27000,0xf2e77000),  sp=0xf2e769a0,  free space=318k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J  ConcurrentQueueLoops$Stage.call()Ljava/lang/Integer;


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
=>0x08184c00 JavaThread "pool-1-thread-1" [_thread_in_Java, id=12]
  0x08183c00 JavaThread "MainThread" [_thread_blocked, id=11]
  0x0814bc00 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=9]
  0x0814a400 JavaThread "CompilerThread1" daemon [_thread_blocked, id=8]
  0x08149000 JavaThread "CompilerThread0" daemon [_thread_blocked, id=7]
  0x08148000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=6]
  0x08133000 JavaThread "Finalizer" daemon [_thread_blocked, id=5]
  0x08132800 JavaThread "Reference Handler" daemon [_thread_blocked, id=4]
  0x08071000 JavaThread "main" [_thread_blocked, id=2]

Other Threads:
  0x0812fc00 VMThread [id=3]
  0x0814dc00 WatcherThread [id=10]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 def new generation   total 960K, used 879K [0xf3200000, 0xf3300000, 0xf3910000)
  eden space 896K,  97% used [0xf3200000, 0xf32db738, 0xf32e0000)
  from space 64K,   2% used [0xf32e0000, 0xf32e0758, 0xf32f0000)
  to   space 64K,   0% used [0xf32f0000, 0xf32f0000, 0xf3300000)
 tenured generation   total 4096K, used 153K [0xf3910000, 0xf3d10000, 0xf7200000)
   the space 4096K,   3% used [0xf3910000, 0xf3936580, 0xf3936600, 0xf3d10000)
 compacting perm gen  total 16384K, used 2182K [0xf7200000, 0xf8200000, 0xfb200000)
   the space 16384K,  13% used [0xf7200000, 0xf74218f0, 0xf7421a00, 0xf8200000)
No shared spaces configured.

Dynamic libraries:
0x08050000 	/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/bin/java
0xfefa0000 	/lib/libthread.so.1
0xfef80000 	/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/bin/../jre/lib/i386/jli/libjli.so
0xfefb0000 	/lib/libdl.so.1
0xfee90000 	/lib/libc.so.1
0xfe800000 	/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/jre/lib/i386/server/libjvm.so
0xfee50000 	/lib/libsocket.so.1
0xfee70000 	/usr/lib/libsched.so.1
0xfe7d0000 	/lib/libm.so.1
0xfe7a0000 	/usr/lib/libCrun.so.1
0xfe780000 	/lib/libdoor.so.1
0xfe6f0000 	/lib/libnsl.so.1
0xfe680000 	/lib/libm.so.2
0xfe650000 	/lib/libscf.so.1
0xfe620000 	/lib/libuutil.so.1
0xfe600000 	/lib/libmd5.so.1
0xfe5e0000 	/lib/libmp.so.2
0xfe520000 	/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/jre/lib/i386/native_threads/libhpi.so
0xfe4e0000 	/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/jre/lib/i386/libverify.so
0xfe490000 	/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/jre/lib/i386/libjava.so
0xfe460000 	/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/jre/lib/i386/libzip.so
0xfb230000 	/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3

VM Arguments:
jvm_args: -Dtest.src=/u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue -Dtest.classes=/u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/JTwork/classes/java/util/concurrent/ConcurrentLinkedQueue
java_command: com.sun.javatest.regtest.MainWrapper /u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/JTwork/classes/java/util/concurrent/ConcurrentLinkedQueue/ConcurrentQueueLoops.jta
Launcher Type: SUN_STANDARD

Environment Variables:
CLASSPATH=/u/martin/jct-tools/2.1.6/javatest.jar:/u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue/JTwork/classes/java/util/concurrent/ConcurrentLinkedQueue:/u/martin/ws/mustang/test/java/util/concurrent/ConcurrentLinkedQueue:/java/re/jdk/6/nightly/binaries/solaris-i586/latest/classes:/java/re/jdk/6/nightly/binaries/solaris-i586/latest/lib/classes:/java/re/jdk/6/nightly/binaries/solaris-i586/latest/lib/classes.jar:/java/re/jdk/6/nightly/binaries/solaris-i586/latest/lib/rt.jar:/java/re/jdk/6/nightly/binaries/solaris-i586/latest/lib/i18n.jar:/java/re/jdk/6/nightly/binaries/solaris-i586/latest/lib/classes.zip:/java/re/jdk/6/nightly/binaries/solaris-i586/latest/lib/tools.jar
PATH=/bin:/usr/bin
LD_LIBRARY_PATH=/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/jre/lib/i386/server:/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/jre/lib/i386:/net/mizu.sfbay/n/mustang/jdk6.0/binaries/solaris-i586/b94-2006-07-30/jdk1.6.0/jre/../lib/i386
DISPLAY=localhost:11.0

Signal Handlers:
SIGSEGV: [libjvm.so+0x4cf374], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGBUS: [libjvm.so+0x4cf374], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGFPE: [libjvm.so+0xf17c8], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGPIPE: [libjvm.so+0xf17c8], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGILL: [libjvm.so+0xf17c8], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGHUP: [libjvm.so+0x43ed88], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGINT: [libjvm.so+0x43ed88], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGQUIT: [libjvm.so+0x43ed88], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0x43ed88], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGQUIT: [libjvm.so+0x43ed88], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0x43ed88], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIG39: [libjvm.so+0x4401d4], sa_mask[0]=0x00000000, sa_flags=0x00000008
SIG40: [libjvm.so+0xf17c8], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c


---------------  S Y S T E M  ---------------

OS:                      Solaris 10 6/06 s10x_u2wos_nightly X86
           Copyright 2006 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                             Assembled 15 June 2006

uname:SunOS 5.10 Generic_118855-14 i86pc  (T2 libthread)
rlimit: STACK 16384k, CORE 10240k, NOFILE 65536, AS infinity
load average:0.30 0.13 0.07

CPU:total 2 family 15, cmov, cx8, fxsr, mmx, sse, sse2, mmxext, 3dnowext, 3dnow

Memory: 4k page, physical 2096700k(246264k free)

vm_info: Java HotSpot(TM) Server VM (1.6.0-rc-b94) for solaris-x86, built on Jul 30 2006 00:57:39 by "" with unknown Workshop:0x580
Comments
SUGGESTED FIX Webrev: http://analemma.sfbay.sun.com/net/prt-archiver.sfbay/data/archived_workspaces/main/c2_baseline/2006/20060803141100.kvn.6454444/workspace/webrevs/webrev-2006.08.03/index.html
14-08-2006
SUGGESTED FIX Fix the end offset of last call calculation in Step 1 in Compile::Shorten_branches(). Add asserts to check displacement for short jumps.
03-08-2006
EVALUATION It is the same case as 6454475. The problem was introduced in Tiger in the 5041651 fix. The check for adjacent call and safepoint in Compile::Shorten_branches() is not valid since the offset from last call is modified before the check. But during Step 3 the offset is calculated correctly and a nop is inserted for adjacent call and safepoint. This invalidate short jumps causing jumps to wrong addresses during execution.
01-08-2006
EVALUATION Problems running ConcurrentQueueLoops can be reproduced on previous JDK builds as well. On bonsai (SunOS bonsai 5.11 snv_43 i86pc i386 i86pc) I ran the test on b92 2000 times in parallel shells (1000 iterations each) I got the reported SIGSEGV 24 times out of 2000. Further testing reveals that the SIGSEGV can be reproduced back to b90, but not in b89. Observed frequencies: 14/1000 on bonsai, 3/1000 on mho. But only on b93 do I see the following failure: ==> javac -source 1.6 -Xlint:all ConcurrentQueueLoops.java ==> java -esa -ea ConcurrentQueueLoops Warmup... Threads: 1 :java.lang.NullPointerException at java.util.concurrent.ConcurrentLinkedQueue.poll(ConcurrentLinkedQueue.java:220) at ConcurrentQueueLoops$Stage.call(ConcurrentQueueLoops.java:62) at ConcurrentQueueLoops$Stage.call(ConcurrentQueueLoops.java:44) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.Error: Call loop failed at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) at java.util.concurrent.FutureTask.get(FutureTask.java:83) at ConcurrentQueueLoops.oneRun(ConcurrentQueueLoops.java:105) at ConcurrentQueueLoops.main(ConcurrentQueueLoops.java:39) Caused by: java.lang.Error: Call loop failed at ConcurrentQueueLoops$Stage.call(ConcurrentQueueLoops.java:85) at ConcurrentQueueLoops$Stage.call(ConcurrentQueueLoops.java:44) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) Command java -esa -ea ConcurrentQueueLoops failed: rc=1 And only on b93 have I seen the test get wedged (hang for a while) after printing: ==> javac -source 1.6 -Xlint:all ConcurrentQueueLoops.java ==> java -esa -ea ConcurrentQueueLoops Warmup... Threads: 1 :
31-07-2006