JDK-8162553 : Crash in class unloading due to null CLD having a zero _keep_alive value
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 9
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2016-07-26
  • Updated: 2019-09-06
  • Resolved: 2016-08-11
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 9
9 b133Fixed
Related Reports
Duplicate :  
Duplicate :  
Duplicate :  
Duplicate :  
Relates :  
Relates :  
Relates :  
Description
#  SIGSEGV (0xb) at pc=0x00007fca89364643, pid=24616, tid=24693
#
# JRE version: Java(TM) SE Runtime Environment (9.0) (fastdebug build 9-internal+0-2016-07-25-230029.daholme.jdk9-hs-dev2)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0x11f6643]  MarkSweep::IsAliveClosure::do_object_b(oop)+0x3
#
Comments
Yes, I believe it is related.
11-08-2016

Could this also explain this crash? # assert(!(is_the_null_class_loader_data() && _unloading)) failed: The null class loader can never be unloaded I got it trying to reproduce 8158183.
09-08-2016

Thanks Mikael. I'm reassigning to Lois who added the code.
28-07-2016

I managed to reproduce the cause of this crash by asserting that the keep alive value of the null class loader must never reach zero. The calls to inc_keep_alive and dec_keep_alive should be balanced and since the initial value is 1 for _the_null_class_loader_data the value 0 is clearly invalid. Stack: [0xf7747000,0xf7798000], sp=0xf7796710, free space=317k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x14b5967] VMError::report_and_die(int, char const*, char const*, char*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned int)+0x137 V [libjvm.so+0x14b6660] VMError::report_and_die(Thread*, char const*, int, char const*, char const*, char*)+0x30 V [libjvm.so+0x932e40] report_vm_error(char const*, int, char const*, char const*, ...)+0x60 V [libjvm.so+0x80c0d5] ClassLoaderData::dec_keep_alive()+0x65 V [libjvm.so+0x10dfe38] ModuleEntryTable::patch_javabase_entries(Handle)+0x3b8 V [libjvm.so+0x10ea0e5] define_javabase_module(_jobject*, _jstring*, _jstring*, _jobjectArray*, Thread*)+0x1f65 V [libjvm.so+0x10ea531] Modules::define_module(_jobject*, _jstring*, _jstring*, _jobjectArray*, Thread*)+0x3a1 V [libjvm.so+0xde036a] JVM_DefineModule+0xea C [libjava.so+0xaf6b] Java_java_lang_reflect_Module_defineModule0+0x2b j java.lang.reflect.Module.defineModule0(Ljava/lang/reflect/Module;Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;)V+0 java.base@9-internal j java.lang.reflect.Module.<init>(Ljava/lang/reflect/Layer;Ljava/lang/ClassLoader;Ljava/lang/module/ModuleDescriptor;Ljava/net/URI;)V+139 java.base@9-internal j java.lang.reflect.Module.<init>(Ljava/lang/reflect/Layer;Ljava/lang/ClassLoader;Ljava/lang/module/ModuleDescriptor;Ljava/net/URI;Ljava/lang/reflect/Module$1;)V+6 java.base@9-internal j java.lang.reflect.Module$1.defineModule(Ljava/lang/ClassLoader;Ljava/lang/module/ModuleDescriptor;Ljava/net/URI;)Ljava/lang/reflect/Module;+9 java.base@9-internal j jdk.internal.module.Modules.defineModule(Ljava/lang/ClassLoader;Ljava/lang/module/ModuleDescriptor;Ljava/net/URI;)Ljava/lang/reflect/Module;+6 java.base@9-internal j jdk.internal.module.ModuleBootstrap.boot()Ljava/lang/reflect/Layer;+93 java.base@9-internal j java.lang.System.initPhase2()V+0 java.base@9-internal v ~StubRoutines::call_stub V [libjvm.so+0xc94761] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x761 V [libjvm.so+0x117e419] os::os_exception_wrapper(void (*)(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*), JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x19 V [libjvm.so+0xc934ce] JavaCalls::call_static(JavaValue*, KlassHandle, Symbol*, Symbol*, Thread*)+0x1fe V [libjvm.so+0x141c248] Threads::create_vm(JavaVMInitArgs*, bool*)+0x908 V [libjvm.so+0xd95faa] JNI_CreateJavaVM+0x8a C [libjli.so+0x2ed6] JavaMain+0x86 C [libpthread.so.0+0x6b2c] start_thread+0xcc C [libc.so.6+0xf75fe] clone+0x5e
28-07-2016

The class loader data is the_null_class_loader_data (gdb) print this $19 = (const ClassLoaderData * const) 0x7fca8044ebf0 (gdb) print *this $20 = {<CHeapObj<(MemoryType)1>> = {<AllocatedObj> = { _vptr.AllocatedObj = 0x7fca89f5e7b0 <vtable for ClassLoaderData+16>}, <No data fields>}, static _the_null_class_loader_data = 0x7fca8044ebf0, _class_loader = {_o = 0x0}, _dependencies = {_list_head = {<oop> = {_o = 0x101c04ad8}, <No data fields>}}, _metaspace = 0x7fca80474510, _metaspace_lock = 0x7fca8044eca0, _unloading = false, _is_anonymous = false, _keep_alive = 0, _claimed = 0, _handles = 0x7fca804bc870, _klasses = 0x80005ce18, _packages = 0x7fca8049b6b0, _modules = 0x7fca80492980, _jmethod_ids = 0x7fca805833a0, _deallocate_list = 0x7fca804c1460, _next = 0x0, _shared_class_loader_id = -1, static _ro_metaspace = 0x0, static _rw_metaspace = 0x0} At this point in do_unloading, shouldn't this be 'claimed' ? (gdb) print *this->_klasses $24 = {<Metadata> = {<MetaspaceObj> = {<No data fields>}, _vptr.Metadata = 0x7fca89f83a50 <vtable for InstanceKlass+16>, _valid = 0}, _layout_helper = 24, _super_check_offset = 56, _name = 0x7fca840941a8, _secondary_super_cache = 0x0, _secondary_supers = 0x7fc95c530090, _primary_supers = { 0x800000fb0, 0x80005ce18, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, _java_mirror = {_o = 0x101d2c458}, _super = 0x800000fb0, _subklass = 0x0, _next_sibling = 0x800060830, _next_link = 0x80005cbf8, _class_loader_data = 0x7fca8044ebf0, _modifier_flags = 8, _access_flags = {_flags = 2097184}, _trace_id = 70975488, _last_biased_lock_bulk_revocation_time = 0, _prototype_header = 0x1, _biased_lock_revocation_count = 0, _vtable_len = 21, _modified_oops = 1 '\001', _accumulated_modified_oops = 0 '\000', _shared_class_path_index = -1, static _lh_array_tag_type_value = 4294967295} The mirror looks like junk (if gdb is showing me the right stuff, everything else looks okay) (gdb) print *this->_klasses->_java_mirror._o $25 = {_mark = 0x2aafb23c01, _metadata = {_klass = 0x1ff0, _compressed_klass = 8176}, static _bs = 0x7fca80098a20} (gdb) print Universe::_narrow_ptrs_base $27 = (address) 0x0 Everything else looks legit except for the zero _keep_alive field.
28-07-2016

The last change in this related code was checked into hs Date: 2016-04-21 15:05:37 +0000 and integrated in Date: 2016-05-25 17:36:48 +0000. I can't reproduce this failure (looked like Mikael couldn't either) and since it never failed between when the code was checked in and now, I think it's a narrow race that already exists in this code. From source inspection there's no obvious bug and it'll take time to track down. This shouldn't be an integration blocker so I will remove the label. I'm still trying to download the core file.
27-07-2016

Linked bug where code to increment and decrement _keep_alive was added, but still not new.
27-07-2016

ok, first question I'll investigate is how would this be new.
27-07-2016

Coleen - can you take a look at this integration_blocker? Class unloading is involved so I thought of you... :-)
27-07-2016

I've run some tests on an instrumented build where I try to catch cases of the _keep_alive value of the_null_class_loader_data going below 1. No luck so far so this seems to be intermittent. Since GC is not calling dec_keep_alive on ClassLoaderData I'm reassigning this to runtime.
27-07-2016

These crashes are caused by _the_null_class_loader_data having a _keep_alive value of 0 when a full GC occurs. Because of the 0 value we go ahead and pass the NULL _class_loader value to a GC is_alive_closure which attempts to dereference the NULL pointer and causes the SEGV. Since GC code does not modify the _keep_alive value of CLDs and the value of the _the_null_class_loader_data _keep_alive is changed a bunch of times during module system initialization I suspect that the cause of the problem lies somewhere in that area. Both of the core files I opened show the GC being triggered from code running in the actual java application, so this is not a case of a GC being triggered during module system initialization or something to that end.
27-07-2016

Joe, since you're the JDK9-hs GC GK for July, please find someone to checkout this integration_blocker.
26-07-2016