JDK-8212205 : VM asserts after CDS archive has been unmapped
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 12
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2018-10-15
  • Updated: 2019-10-03
  • Resolved: 2018-10-29
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 12
11.0.4Fixed 12 b18Fixed
Related Reports
Relates :  
JDK-8178351 - Simplify MetaspaceShared::is_in_shared_space and MetaspaceObj::is_shared
Description
Test gc/arguments/TestUseCompressedOopsErgo.java#id0 fails intermittently on windows with 
# after -XX: or in .hotspotrc:  SuppressErrorAt=t:/workspace/open/src/hotspot/share/oops/klassVtable.cpp:1049
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (t:/workspace/open/src/hotspot/share/oops/klassVtable.cpp:1049), pid=20268, tid=8588
#  assert(_method == m) failed: sanity
#
# JRE version:  (12.0+15) (fastdebug build )
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 12-ea+15, mixed mode, tiered, compressed oops, parallel gc, windows-amd64)
# Core dump will be written. Default location: T:\\testoutput\\jtreg\\JTwork\\scratch\\2\\hs_err_pid20268.mdmp
#
# An error report file with more information is saved as:
# T:\\testoutput\\jtreg\\JTwork\\scratch\\2\\hs_err_pid20268.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#
Comments
Fix Request: This fix shall be backported to jdk11u because it fixes an regresion issue that was originally introduced during jdk11 development. The patch applies cleanly and the risk seems to be low. Patch will be run through test system at SAP before pushing.
20-03-2019
No crashes observed in JDK 12 CI since the fix. VERIFIED
05-01-2019
Thank you [~iklam]!
25-10-2018
The bug was introduced in JDK-8178351 (JDK 11). The fix should be backported to 11.
25-10-2018
The error can be reproduced on linux by hacking this code in metaspace.cpp: if (UseSharedSpaces /* hack && !can_use_cds_with_metaspace_addr(metaspace_rs.base(), cds_base) */) { FileMapInfo::stop_sharing_and_unmap( "Could not allocate metaspace at a compatible address"); } It can no longer be reproduced with the proposed fix: http://cr.openjdk.java.net/~iklam/jdk12/8212205-crash-after-cds-unmap.v01/
25-10-2018
Analysis: This is most likely a bug related to this code -- _shared_metaspace_{top,base} was not properly cleared after the CDS archive has been unmapped. class MetaspaceObj { bool is_shared() const { // If no shared metaspace regions are mapped, _shared_metaspace_{base,top} will // both be NULL and all values of p will be rejected quickly. return (((void*)this) < _shared_metaspace_top && ((void*)this) >= _shared_metaspace_base); } } From the hs_err log files, CDS is not enabled. Note that "sharing" is missing from the version string. Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 12-ea+15, mixed mode, tiered, compressed oops, parallel gc, windows-amd64) Also, from one of the crashes, there's clear evidence that CDS is not enabled -- SystemDictionary::resolve_wk_klasses_until eventually calls ClassLoader::load_class to parse class files. If CDS is enabled, this will not happen. V [jvm.dll+00b6df21] = 00b6ce30+ 241 os::platform_print_native_stack V [jvm.dll+00d8f65c] = 00d8d7e0+3708 VMError::report V [jvm.dll+00d91024] = 00d8f7d0+2132 VMError::report_and_die V [jvm.dll+00d916e4] = 00d90680+ 100 VMError::report_and_die V [jvm.dll+0059344e] = 005923d0+ 126 report_vm_error V [jvm.dll+00aeb7fe] = 00aea7a0+ 94 Method::set_interpreter_entry V [jvm.dll+00ae2869] = 00ae1780+ 233 Method::Method V [jvm.dll+004a84f6] = 004a5b20+6614 ClassFileParser::parse_method V [jvm.dll+004a8e55] = 004a7a90+ 965 ClassFileParser::parse_methods V [jvm.dll+004a9d4a] = 004a80a0+3242 ClassFileParser::parse_stream V [jvm.dll+00497ac8] = 004963c0+1800 ClassFileParser::ClassFileParser V [jvm.dll+0099efe2] = 0099de10+ 466 KlassFactory::create_from_stream V [jvm.dll+004b414e] = 004b2f80+ 462 ClassLoader::load_class V [jvm.dll+00ce65a9] = 00ce5330+ 633 SystemDictionary::load_instance_class V [jvm.dll+00ce874f] = 00ce6d60+2543 SystemDictionary::resolve_instance_class_or_null V [jvm.dll+00ce9c19] = 00ce89d0+ 585 SystemDictionary::resolve_wk_klasses_until V [jvm.dll+00ce8de9] = 00ce7ce0+ 265 SystemDictionary::resolve_preloaded_classes V [jvm.dll+00ce4cc3] = 00ce3ac0+ 515 SystemDictionary::initialize V [jvm.dll+00d5669b] = 00d55370+ 811 Universe::genesis V [jvm.dll+00d595e8] = 00d585c0+ 40 universe2_init V [jvm.dll+0074c7ec] = 0074b720+ 204 init_globals V [jvm.dll+00d2a534] = 00d28ef0+1604 Threads::create_vm V [jvm.dll+00830631] = 0082f560+ 209 JNI_CreateJavaVM_inner V [jvm.dll+00833eef] = 00832ed0+ 31 JNI_CreateJavaVM MetaspaceObj::_shared_metaspace_{base,top} is initialized MetaspaceShared::map_shared_spaces. However, afterwards, the CDS archive could be unmapped by this code in metaspace.cpp: if (UseSharedSpaces && !can_use_cds_with_metaspace_addr(metaspace_rs.base(), cds_base)) { FileMapInfo::stop_sharing_and_unmap( "Could not allocate metaspace at a compatible address"); } In this case, _shared_metaspace_{base,top} must be reset to zero. This is not done today. This happens rarely, only when Address Space Layout Randomization causes the failure of dynamically allocating the runtime class metaspace above the mapped CDS address space. This explains the intermittent nature of this bug.
25-10-2018
In another failed run from the same batch done by Leonid, it hits the following assert: ----------System.err:(42/2343)*---------- stdout: [# To suppress the following error report, specify this argument # after -XX: or in .hotspotrc: SuppressErrorAt=\\oops/method.hpp:501 # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (t:\\workspace\\open\\src\\hotspot\\share\\oops/method.hpp:501), pid=22196, tid=3816 # assert(!is_shared()) failed: shared method's interpreter entry should not be changed at run time # # JRE version: (12.0+15) (fastdebug build ) # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 12-ea+15, mixed mode, tiered, compressed oops, parallel gc, windows-amd64) # Core dump will be written. Default location: T:\\testoutput\\jtreg\\JTwork\\scratch\\3_1\\hs_err_pid22196.mdmp # # An error report file with more information is saved as: # T:\\testoutput\\jtreg\\JTwork\\scratch\\3_1\\hs_err_pid22196.log # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # ]; void set_interpreter_entry(address entry) { assert(!is_shared(), "shared method's interpreter entry should not be changed at run time"); if (_i2i_entry != entry) { _i2i_entry = entry; } if (_from_interpreted_entry != entry) { _from_interpreted_entry = entry; } } Possibly a bug in the CDS adapter_trampoline?
17-10-2018
CDS problem: #ifdef ASSERT if (MetaspaceShared::is_in_shared_metaspace((void*)&_method) && !MetaspaceShared::remapped_readwrite()) { // At runtime initialize_itable is rerun as part of link_class_impl() // for a shared class loaded by the non-boot loader. // The dumptime itable method entry should be the same as the runtime entry. assert(_method == m, "sanity"); } #endif
17-10-2018
Test starts failing in JDK 12+15 (haven't��tested in 14), failed in ~ 0.5% of runs. The full trace is: # A fatal error has been detected by the Java Runtime Environment: # Internal Error (t:/workspace/open/src/hotspot/share/oops/klassVtable.cpp:1049), pid=20268, tid=8588 assert(_method == m) failed: sanity # JRE version: (12.0+15) (fastdebug build ) Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 12-ea+15, mixed mode, tiered, compressed oops, parallel gc, windows-amd64) Core dump will be written. Default location: T:\testoutput\jtreg\JTwork\scratch\2\hs_err_pid20268.mdmp # If you would like to submit a bug report, please visit: http://bugreport.java.com/bugreport/crash.jsp # --------------- S U M M A R Y ------------ Command Line: -XX:+UseParallelGC -XX:-UseParallelOldGC -XX:CompressedClassSpaceSize=2147483648 -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI DetermineMaxHeapForCompressedOops Host: Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz, 8 cores, 60G, Windows Server 2012 R2 , 64 bit Build 9600 (6.3.9600.17415) Time: Sun Oct 14 07:35:13 2018 ric elapsed time: 0 seconds (0d 0h 0m 0s) --------------- T H R E A D --------------- Current thread (0x000000088116a800): JavaThread "Unknown thread" [_thread_in_vm, id=8588, stack(0x0000000880810000,0x0000000880910000)] Stack: [0x0000000880810000,0x0000000880910000] Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [jvm.dll+0xb70591] os::platform_print_native_stack+0xf1 (os_windows_x86.cpp:373) V [jvm.dll+0xd8f2aa] VMError::report+0xe6a (vmerror.cpp:698) V [jvm.dll+0xd90bc4] VMError::report_and_die+0x854 (vmerror.cpp:1445) V [jvm.dll+0xd91284] VMError::report_and_die+0x64 (vmerror.cpp:1269) V [jvm.dll+0x595fde] report_vm_error+0x7e (debug.cpp:233) V [jvm.dll+0x9a4aa7] itableMethodEntry::initialize+0x77 (klassvtable.cpp:1049) V [jvm.dll+0x9a534b] klassItable::initialize_itable_for_interface+0x34b (klassvtable.cpp:1267) V [jvm.dll+0x9a4f52] klassItable::initialize_itable+0x1a2 (klassvtable.cpp:1111) V [jvm.dll+0xd5779c] initialize_itable_for_klass+0x2c (universe.cpp:553) V [jvm.dll+0x627157] Dictionary::classes_do+0x67 (dictionary.cpp:326) V [jvm.dll+0x4bb984] ClassLoaderDataGraph::dictionary_classes_do+0xb4 (classloaderdatagraph.cpp:422) V [jvm.dll+0xd59563] universe_post_init+0x163 (universe.cpp:970) V [jvm.dll+0x7517ae] init_globals+0x11e (init.cpp:143) V [jvm.dll+0xd29a04] Threads::create_vm+0x644 (thread.cpp:3726) V [jvm.dll+0x835791] JNI_CreateJavaVM_inner+0xd1 (jni.cpp:3937) V [jvm.dll+0x83905f] JNI_CreateJavaVM+0x1f (jni.cpp:4028) C [jli.dll+0x52bf] JavaMain+0x113 (java.c:414) C [ucrtbase.DLL+0x1d885] C [KERNEL32.DLL+0x13d2] C [ntdll.dll+0x154f4]
15-10-2018