JDK-8207912 : runtime/Metaspace/DefineClass.java fails intermittently
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 11,12,13
  • Priority: P3
  • Status: Resolved
  • Resolution: Cannot Reproduce
  • OS: solaris_11
  • CPU: x86_64
  • Submitted: 2018-07-19
  • Updated: 2021-01-07
  • Resolved: 2020-12-01
Related Reports
Relates :  
JDK-8205611 - Improve the wording of LinkageErrors to include module and class loader information
Description
The following test failed due to a crash on Solaris X64
in the release config using jdk11+19 bits:

runtime/Metaspace/DefineClass.java

The test only failed in 1 of 3 'release' bits runs so I'm
tagging this bug as intermittent. It did not fail at all in the
'fastdebug' or 'slowdebug' bits runs.

Here is a snippet from the stdout of the log file: 

----------System.out:(20/1146)----------
sizeof(DefineClass.class) == 8919
parallelCapable : true
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fffff4e2900, pid=4936, tid=28
#
# JRE version: Java(TM) SE Runtime Environment (11.0) (build 11-internal+0-2018-06-26-1917180.dcubed.jdk11exp)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (11-internal+0-2018-06-26-1917180.dcubed.jdk11exp, mixed mode, tiered, compressed oops, g1 gc, solaris-amd64)
# Problematic frame:
# V  [libjvm.so+0x1ce2900]  char*Symbol::as_C_string()const+0x10
#
# Core dump will be written. Default location: /work/shared/bug_hunt/thread_SMR_stress/jdk11_exp/build/solaris-x86_64-normal-server-release/test-support/jtreg_open_test_hotspot_jtreg_tier1/runtime/Metaspace/DefineClass/core or core.4936
#
# An error report file with more information is saved as:
# /work/shared/bug_hunt/thread_SMR_stress/jdk11_exp/build/solaris-x86_64-normal-server-release/test-support/jtreg_open_test_hotspot_jtreg_tier1/runtime/Metaspace/DefineClass/hs_err_pid4936.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#
----------System.err:(0/0)----------
----------rerun:(27/3623)*----------

Here's the crashing thread's stack:

---------------  T H R E A D  ---------------
        
Current thread (0x0000000000f8a000):  JavaThread "Thread-9" [_thread_in_vm, id=28, stack(0x00007fffdbef7000,0x00007fffdbff7000)]
        
Stack: [0x00007fffdbef7000,0x00007fffdbff7000],  sp=0x00007fffdbff5c70,  free space=1019k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1ce2900]  char*Symbol::as_C_string()const+0x10
V  [libjvm.so+0x14eb357]  const char*java_lang_ClassLoader::describe_external(oopDesc*)+0x57
V  [libjvm.so+0x1cf20de]  void SystemDictionary::check_constraints(unsigned,InstanceKlass*,Handle,bool,Thread*)+0x12e
V  [libjvm.so+0x1cf02ea]  void SystemDictionary::define_instance_class(InstanceKlass*,Thread*)+0x12a
V  [libjvm.so+0x1cf0850]  InstanceKlass*SystemDictionary::find_or_define_instance_class(Symbol*,Handle,InstanceKlass*,Thread*)+0x1f0 
V  [libjvm.so+0x1cee809]  InstanceKlass*SystemDictionary::resolve_from_stream(Symbol*,Handle,Handle,ClassFileStream*,Thread*)+0x1d9
V  [libjvm.so+0x161dc89]  _jclass*jvm_define_class_common(JNIEnv_*,const char*,_jobject*,const signed char*,int,_jobject*,const char*,Thread*)+0x379
V  [libjvm.so+0x161e1eb]  JVM_DefineClassWithSource+0x13b
C  [libjava.so+0x145de]  Java_java_lang_ClassLoader_defineClass1+0x20e
j  java.lang.ClassLoader.defineClass1(Ljava/lang/ClassLoader;Ljava/lang/String;[BIILjava/security/ProtectionDomain;Ljava/lang/String;)Ljava/lang/Class;+0 java.base@11-internal
j  java.lang.ClassLoader.defineClass(Ljava/lang/String;[BIILjava/security/ProtectionDomain;)Ljava/lang/Class;+27 java.base@11-internal
j  test.DefineClass$MyParallelClassLoader.myDefineClass(Ljava/lang/String;[BII)Ljava/lang/Class;+7
j  test.DefineClass$ParallelLoadingThread.run()V+34
v  ~StubRoutines::call_stub
V  [libjvm.so+0x14d7393]  void JavaCalls::call_helper(JavaValue*,const methodHandle&,JavaCallArguments*,Thread*)+0x1f3
V  [libjvm.so+0x14d6739]  void JavaCalls::call_virtual(JavaValue*,Klass*,Symbol*,Symbol*,JavaCallArguments*,Thread*)+0x149
V  [libjvm.so+0x14d67d2]  void JavaCalls::call_virtual(JavaValue*,Handle,Klass*,Symbol*,Symbol*,Thread*)+0x52
V  [libjvm.so+0x163e044]  void thread_entry(JavaThread*,Thread*)+0xb4
V  [libjvm.so+0x1d3300f]  void JavaThread::thread_main_inner()+0xdf
V  [libjvm.so+0x1d32f0a]  void JavaThread::run()+0x2da
V  [libjvm.so+0x1b11e00]  thread_native_entry+0x240
C  [libc.so.1+0x125221]  _thrp_setup+0xa5
C  [libc.so.1+0x1254c0]  _lwp_start+0x0

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.lang.ClassLoader.defineClass1(Ljava/lang/ClassLoader;Ljava/lang/String;[BIILjava/security/ProtectionDomain;Ljava/lang/String;)Ljava/lang/Class;+0 java.base@11-internal
j  java.lang.ClassLoader.defineClass(Ljava/lang/String;[BIILjava/security/ProtectionDomain;)Ljava/lang/Class;+27 java.base@11-internal
j  test.DefineClass$MyParallelClassLoader.myDefineClass(Ljava/lang/String;[BII)Ljava/lang/Class;+7
j  test.DefineClass$ParallelLoadingThread.run()V+34
v  ~StubRoutines::call_stub
Comments
Thanks Ioi for the update. I agree that we can close this bug as not reproducible. java_lang_ClassLoader::describe_external() where the crash occurred was removed by me in JDK-8205611 in order to use the new method Klass::class_in_module_of_loader for more consistency across error messages. This work was completed in the JDK12 timeframe and backported by Goetz Lindenmaier in JDK-8223440. So this crash in describe_external() should no longer be an issue.
01-12-2020
I searched the latest mach5 results database and found no recorded crashes with the following stack (from the Description of this bug) V [libjvm.so+0x1ce2900] char*Symbol::as_C_string()const+0x10 V [libjvm.so+0x14eb357] const char*java_lang_ClassLoader::describe_external(oopDesc*)+0x57 V [libjvm.so+0x1cf20de] void SystemDictionary::check_constraints(unsigned,InstanceKlass*,Handle,bool,Thread*)+0x12e V [libjvm.so+0x1cf02ea] void SystemDictionary::define_instance_class(InstanceKlass*,Thread*)+0x12a V [libjvm.so+0x1cf0850] I found only 2 crashes related to Symbol::as_C_string, but they happened only in the valhalla repo with the following stack V [jvm.dll+0x73e27a] Symbol::as_C_string+0xa (symbol.cpp:230) V [jvm.dll+0x73e379] Symbol::as_klass_external_name+0x9 (symbol.cpp:292) V [jvm.dll+0x5304e4] Klass::external_name+0x164 (klass.cpp:727) V [jvm.dll+0x64af84] ObjArrayKlass::copy_array+0x194 (objArrayKlass.cpp:260) V [jvm.dll+0x6d7ea0] SharedRuntime::slow_arraycopy_C+0x70 (sharedRuntime.cpp:2067) I think we can probably close this bug as no reproducible.
01-12-2020
This test has failed 71 times since Aug 15, 2018, on all of windows/linux/mac/solaris, in both debug and release builds. So I am upping it to a P3. On Windows, the typical error message is: ----------System.out:(36/1492)*---------- sizeof(DefineClass.class) == 8883 Our pid is = 46896 Loading Java Agent. # # A fatal error has been detected by the Java Runtime Environment: # # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007fff00e01342, pid=46896, tid=16296 # # JRE version: OpenJDK Runtime Environment (13.0) (build 13-lworldea+0-jdk13-valhalla.29) # Java VM: OpenJDK 64-Bit Server VM (13-lworldea+0-jdk13-valhalla.29, mixed mode, sharing, compressed oops, g1 gc, windows-amd64) # Problematic frame: # V [jvm.dll+0x611342]BAAAAAAA CAAAAAAA DAAAAAAA EAAAAAAA FAAAAAAA GAAAAAAA HAAAAAAA IAAAAAAA JAAAAAAA KAAAAAAA
08-02-2019
Here are the logs for my jdk11+19 sightings: $ unzip -l jdk11_migrate.8207912.zip Archive: jdk11_migrate.8207912.zip Length Date Time Name --------- ---------- ----- ---- 119032 06-27-2018 13:35 jdk11_migrate_3/DefineClass.hs_err_pid.release 28329 06-27-2018 13:39 jdk11_migrate_3/DefineClass.jtr.release --------- ------- 147361 2 files
19-07-2018