JDK-6849574 : VM crash using NonBlockingHashMap (high_scale_lib)
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 7
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: solaris_10
  • CPU: sparc
  • Submitted: 2009-06-09
  • Updated: 2011-03-08
  • Resolved: 2011-03-08
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7 Other
6u18Fixed 7Fixed hs14Resolved
Description
FULL PRODUCT VERSION :
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
Java HotSpot(TM) Client VM (build 11.3-b02, mixed mode, sharing)

FULL OS VERSION :
SunOS s10qa1 5.10 Generic_120011-14 sun4u sparc SUNW,Sun-Fire-V210

EXTRA RELEVANT SYSTEM CONFIGURATION :
-bash-3.00$ psrinfo -v
Status of virtual processor 0 as of: 03/24/2009 15:24:51
  on-line since 02/19/2009 15:49:57.
  The sparcv9 processor operates at 1336 MHz,
        and has a sparcv9 floating point processor.



A DESCRIPTION OF THE PROBLEM :
This program (which potentially has spurious use of Unsafe) causes a VM crash very reliably (but not always). My testing seems to indicate that it only happens on solaris and that the problem was introduced between 6u07 and 6u10



THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: No

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: No

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Compile and run the attached program source. You'll need the high-scale-lib (http://high-scale-lib.sourceforge.net/). I used version 1.1.1 of that library (http://voxel.dl.sourceforge.net/sourceforge/high-scale-lib/high-scale-lib-v1.1.1.zip)

IMPORTANT: Run with -client

java -client -classpath bin:lib/high-scale-lib.jar test.Test



EXPECTED VERSUS ACTUAL BEHAVIOR :
expect no output, actual is VM crash
ERROR MESSAGES/STACK TRACES THAT OCCUR :
#
# An unexpected error has been detected by Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0xfebdff04, pid=4125, tid=15
#
# Java VM: Java HotSpot(TM) Client VM (11.3-b02 mixed mode, sharing solaris-sparc)
# Problematic frame:
# V  [libjvm.so+0x3dff04]
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x000f9800):  JavaThread "pool-1-thread-6" [_thread_in_vm, id=15, stack(0xfb600000,0xfb680000)]

siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00300164

Registers:
 O0=0x21000021 O1=0x0030003c O2=0xfb67e748 O3=0x00000000
 O4=0x000f9800 O5=0x000fa014 O6=0xfb67e7c0 O7=0xd8c005e0
 G1=0x00000007 G2=0xd8c001e0 G3=0x00000004 G4=0x00300038
 G5=0x00000091 G6=0x00000000 G7=0xfc313000 Y=0x00000000
 PC=0xfebdff04 nPC=0xfebdff08


  Top of Stack: (sp=0xfb67e7c0)
0xfb67e7c0:   000f9800 fee34000 000fa030 fee34000
0xfb67e7d0:   00000001 002542c8 000fa034 000fa03c
0xfb67e7e0:   fb67f304 fb67e990 fb67e98c d8c00000
0xfb67e7f0:   fb67e984 00000001 fb67e848 fe8e11b0
0xfb67e800:   00000004 0000000a 01000101 00000000
0xfb67e810:   00000000 00000000 00000000 00000000
0xfb67e820:   00000000 00000000 00000000 00000000
0xfb67e830:   00000000 00000000 00000000 00000000

Instructions: (pc=0xfebdff04)
0xfebdfef4:   10 80 00 04 92 00 e0 00 c8 04 a0 00 92 01 00 03
0xfebdff04:   f6 02 61 28 80 a6 e0 00 12 80 00 04 84 10 20 00

Stack: [0xfb600000,0xfb680000],  sp=0xfb67e7c0,  free space=505k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x3dff04]
V  [libjvm.so+0xe11b8]
V  [libjvm.so+0xd7f74]
V  [libjvm.so+0x14afdc]
V  [libjvm.so+0x16e830]
V  [libjvm.so+0x4640c0]
v  ~RuntimeStub::ic_miss_stub
J  org.cliffc.high_scale_lib.NonBlockingHashMap.get_impl(Lorg/cliffc/high_scale_lib/NonBlockingHashMap;[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
J  org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotV.next()Ljava/lang/Object;
J  org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotE.next()Ljava/lang/Object;
J  test.Test$1.run()V
j  java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;+4
j  java.util.concurrent.FutureTask$Sync.innerRun()V+30
j  java.util.concurrent.FutureTask.run()V+4
j  java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Ljava/lang/Runnable;)V+59
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+28
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub
V  [libjvm.so+0xd6e64]
V  [libjvm.so+0x2d5fc8]
V  [libjvm.so+0xe9230]
V  [libjvm.so+0xe91b0]
V  [libjvm.so+0x4b5770]
V  [libjvm.so+0x41c188]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
v  ~RuntimeStub::ic_miss_stub
J  org.cliffc.high_scale_lib.NonBlockingHashMap.get_impl(Lorg/cliffc/high_scale_lib/NonBlockingHashMap;[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
J  org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotV.next()Ljava/lang/Object;
J  org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotE.next()Ljava/lang/Object;
J  test.Test$1.run()V
j  java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;+4
j  java.util.concurrent.FutureTask$Sync.innerRun()V+30
j  java.util.concurrent.FutureTask.run()V+4
j  java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Ljava/lang/Runnable;)V+59
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+28
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x000fc400 JavaThread "pool-1-thread-8" [_thread_blocked, id=17, stack(0xfb400000,0xfb480000)]
  0x000fac00 JavaThread "pool-1-thread-7" [_thread_blocked, id=16, stack(0xfb500000,0xfb580000)]
=>0x000f9800 JavaThread "pool-1-thread-6" [_thread_in_vm, id=15, stack(0xfb600000,0xfb680000)]
  0x000f8000 JavaThread "pool-1-thread-5" [_thread_in_Java, id=14, stack(0xfb700000,0xfb780000)]
  0x000f6c00 JavaThread "pool-1-thread-4" [_thread_blocked, id=13, stack(0xfb800000,0xfb880000)]
  0x000f5800 JavaThread "pool-1-thread-3" [_thread_in_Java, id=12, stack(0xfb900000,0xfb980000)]
  0x000f4000 JavaThread "pool-1-thread-2" [_thread_blocked, id=11, stack(0xfba00000,0xfba80000)]
  0x000f3000 JavaThread "pool-1-thread-1" [_thread_blocked, id=10, stack(0xfbb00000,0xfbb80000)]
  0x00069c00 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=8, stack(0xfbe80000,0xfbf00000)]
  0x00066400 JavaThread "CompilerThread0" daemon [_thread_blocked, id=7, stack(0xfbf80000,0xfc000000)]
  0x00065000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=6, stack(0xfe500000,0xfe580000)]
  0x00052000 JavaThread "Finalizer" daemon [_thread_blocked, id=5, stack(0xfc080000,0xfc100000)]
  0x00050800 JavaThread "Reference Handler" daemon [_thread_blocked, id=4, stack(0xfc180000,0xfc200000)]
  0x00030000 JavaThread "main" [_thread_blocked, id=2, stack(0xfe700000,0xfe780000)]

Other Threads:
  0x0004f000 VMThread [stack: 0xfc280000,0xfc300000] [id=3]
  0x0006b400 WatcherThread [stack: 0xfbd80000,0xfbe00000] [id=9]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 def new generation   total 1984K, used 210K [0xd0400000, 0xd0620000, 0xd0b10000)
  eden space 1792K,   1% used [0xd0400000, 0xd0404b60, 0xd05c0000)
  from space 192K, 100% used [0xd05f0000, 0xd0620000, 0xd0620000)
  to   space 192K,   0% used [0xd05c0000, 0xd05c0000, 0xd05f0000)
 tenured generation   total 4096K, used 3289K [0xd0b10000, 0xd0f10000, 0xd4400000)
   the space 4096K,  80% used [0xd0b10000, 0xd0e465f8, 0xd0e46600, 0xd0f10000)
 compacting perm gen  total 12288K, used 192K [0xd4400000, 0xd5000000, 0xd8400000)
   the space 12288K,   1% used [0xd4400000, 0xd4430100, 0xd4430200, 0xd5000000)
    ro space 8192K,  81% used [0xd8400000, 0xd8a8a740, 0xd8a8a800, 0xd8c00000)
    rw space 12288K,  60% used [0xd8c00000, 0xd934e8c8, 0xd934ea00, 0xd9800000)

Dynamic libraries:
0x00010000 	/opt/dev/teck/jdk1.6.0_13/bin/java
0xff398000 	/lib/libthread.so.1
0xff370000 	/opt/dev/teck/jdk1.6.0_13/bin/../jre/lib/sparc/jli/libjli.so
0xff340000 	/lib/libdl.so.1
0xff200000 	/lib/libc.so.1
0xff320000 	/platform/SUNW,Sun-Fire-V210/lib/libc_psr.so.1
0xfe800000 	/opt/dev/teck/jdk1.6.0_13/jre/lib/sparc/client/libjvm.so
0xff1d0000 	/lib/libsocket.so.1
0xff310000 	/usr/lib/libsched.so.1
0xff1a0000 	/lib/libm.so.1
0xff170000 	/usr/lib/libCrun.so.1
0xff150000 	/lib/libdoor.so.1
0xff080000 	/lib/libnsl.so.1
0xfef80000 	/lib/libm.so.2
0xff050000 	/lib/libscf.so.1
0xfef60000 	/lib/libuutil.so.1
0xfef40000 	/lib/libgen.so.1
0xfef10000 	/lib/libmd.so.1
0xfeef0000 	/platform/SUNW,Sun-Fire-V210/lib/libmd_psr.so.1
0xfeed0000 	/lib/libmp.so.2
0xfe7e0000 	/opt/dev/teck/jdk1.6.0_13/jre/lib/sparc/native_threads/libhpi.so
0xfe6d0000 	/opt/dev/teck/jdk1.6.0_13/jre/lib/sparc/libverify.so
0xfe690000 	/opt/dev/teck/jdk1.6.0_13/jre/lib/sparc/libjava.so
0xfe790000 	/opt/dev/teck/jdk1.6.0_13/jre/lib/sparc/libzip.so
0xfe5e0000 	/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3

VM Arguments:
java_command: test.Test
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=/export1/dev/teck/jdk1.6.0_13/
PATH=/export1/dev/teck/jdk1.6.0_13//bin:/bin:/home/teck/.installs/ant/bin:/home/teck/.installs/maven/bin:/usr/bin:/usr/sbin:/sbin:.:/export2/homes/teck/bin
LD_LIBRARY_PATH=/opt/dev/teck/jdk1.6.0_13/jre/lib/sparc/client:/opt/dev/teck/jdk1.6.0_13/jre/lib/sparc:/opt/dev/teck/jdk1.6.0_13/jre/../lib/sparc
SHELL=/bin/bash

Signal Handlers:
SIGSEGV: [libjvm.so+0x4fdf20], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGBUS: [libjvm.so+0x4fdf20], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGFPE: [libjvm.so+0x18ac88], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGPIPE: [libjvm.so+0x18ac88], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGXFSZ: [libjvm.so+0x18ac88], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGILL: [libjvm.so+0x18ac88], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGQUIT: [libjvm.so+0x41e3a0], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGHUP: [libjvm.so+0x41e3a0], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGINT: [libjvm.so+0x41e3a0], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0x41e3a0], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIG39: [libjvm.so+0x421568], sa_mask[0]=0x00000000, sa_flags=0x00000008
SIG40: [libjvm.so+0x18ac88], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c


---------------  S Y S T E M  ---------------

OS:                       Solaris 10 6/06 s10s_u2wos_09a SPARC
           Copyright 2006 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                             Assembled 09 June 2006

uname:SunOS 5.10 Generic_120011-14 sun4u  (T2 libthread)
rlimit: STACK 8192k, CORE infinity, NOFILE 65536, AS infinity
load average:3.04 1.61 1.04

CPU:total 1 has_v8, has_v9, has_vis1, has_vis2, is_ultra3

Memory: 8k page, physical 1572864k(1002696k free)

vm_info: Java HotSpot(TM) Client VM (11.3-b02) for solaris-sparc JRE (1.6.0_13-b03), built on Mar  9 2009 01:07:54 by "" with Workshop 5.8

time: Tue Mar 24 15:24:03 2009
elapsed time: 9 seconds


REPRODUCIBILITY :
This bug can be reproduced often.

---------- BEGIN SOURCE ----------
package test;

import java.util.Iterator;
import java.util.Random;
import java.util.Map.Entry;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicBoolean;

import org.cliffc.high_scale_lib.NonBlockingHashMap;

public class Test {

    private static final AtomicBoolean stop = new AtomicBoolean(false);

    public static void main(String[] args) {
        int num = 8;

        if (args.length > 0) {
            num = Integer.parseInt(args[0]);
        }

        NonBlockingHashMap<String, Object> map = new NonBlockingHashMap<String, Object>();

        ExecutorService exec = Executors.newFixedThreadPool(num);
        for (int i = 0; i < num; i++) {
            exec.submit(loadTask(map));
        }

        long end = System.currentTimeMillis() + 30000L;
        while (System.currentTimeMillis() < end) {
            try {
                Thread.sleep(1000);
            } catch (InterruptedException ie) {
                ie.printStackTrace();
            }
        }

        stop.set(true);
        exec.shutdown();
    }

    private static Runnable loadTask(
            final NonBlockingHashMap<String, Object> map) {
        return new Runnable() {
            private final Random r = new Random();
            private int i = 0;

            @Override
            public void run() {
                while (!stop.get()) {
                    i = ++i % 3;
                    switch (i) {
                    case 0: {
                        for (int i = 0; i < 5; i++) {
                            map.put(r.nextInt() + "", new Object());
                        }
                        break;
                    }
                    case 1: {
                        Iterator<Entry<String, Object>> iter = map.entrySet()
                                .iterator();

                        for (int i = 0; i < 5; i++) {
                            if (iter.hasNext()) {
                                iter.next();
                                iter.remove();
                            } else {
                                break;
                            }
                        }
                        break;
                    }
                    case 2: {
                        int size = map.size();
                        if (size > 0) {
                            map.get(r.nextInt(size) + "");
                        }
                        break;
                    }
                    default:
                        System.err.println("i=" + i);
                        System.exit(1);
                    }
                }
            }
        };
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Run with -server

Release Regression From : 6u8
The above release value was the last known release where this 
bug was not reproducible. Since then there has been a regression.

Comments
EVALUATION Verified.
13-07-2009

EVALUATION http://hg.openjdk.java.net/jdk7/hotspot-comp/hotspot/rev/c6386080541b
10-06-2009

EVALUATION There were some changes as part of 6459804 that went into 7b15 having more to do with adding G1 support that changed the card mark in compareAndSwapObject from a precise card mark to an imprecise one. It would continue to work right for normal objects since they normally use imprecise card marks but when used with an array it won't mark the right card so some updates will be missed. The fix is simply to restore the precise card mark code. note that this is sparc only since the x86 code didn't change for some reason.
09-06-2009