JDK-8354119 : Missing C2 proper allocation failure handling during initialization (during generate_uncommon_trap_blob)
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 7u351,8u341,11,17,21,24,25
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • CPU: x86_64
  • Submitted: 2025-04-09
  • Updated: 2025-05-08
  • Resolved: 2025-04-28
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 25
25 b21Fixed
Related Reports
Duplicate :  
JDK-8355088 - Crash at Assembler::prefixq_and_encode(int, bool)
Relates :  
JDK-8347406 - [REDO] C1/C2 don't handle allocation failure properly during initialization (RuntimeStub::new_runtime_stub fatal crash)
Relates :  
JDK-8354404 - [lworld] compiler/startup/StartupOutput.java crashes during AdapterHandlerLibrary::initialize() due to too little CodeCacheSize
Description
From JDK-8347406 ([~kvn]): Crash during `generate_uncommon_trap_blob()` call when running StartupOutput.java:

#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ff8abb3500d, pid=131416, tid=46000
#
# JRE version: Java(TM) SE Runtime Environment (25.0) (fastdebug build 25-internal-LTS-2025-04-09-0101299.vladimir.kozlov.jdkgit)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 25-internal-LTS-2025-04-09-0101299.vladimir.kozlov.jdkgit, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# V [jvm.dll+0x35500d] Assembler::subq+0x5d

Command Line: -XX:InitialCodeCacheSize=1189K -XX:ReservedCodeCacheSize=1223k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0x35500d] Assembler::subq+0x5d (assembler_x86.cpp:16896)
V [jvm.dll+0xebe356] OptoRuntime::generate_uncommon_trap_blob+0xc6 (runtime_x86_64.cpp:76)
V [jvm.dll+0xeb6f84] OptoRuntime::generate+0x14 (runtime.cpp:180)
V [jvm.dll+0x4c3d96] C2Compiler::initialize+0x1a6 (c2compiler.cpp:111)
V [jvm.dll+0x5dab46] CompileBroker::init_compiler_runtime+0xe6 (compileBroker.cpp:1762)
V [jvm.dll+0x5d8f8a] CompileBroker::compiler_thread_loop+0x11a (compileBroker.cpp:1903)
V [jvm.dll+0x932622] JavaThread::thread_main_inner+0x282 (javaThread.cpp:773)

CodeCache: size=1280Kb used=1280Kb max_used=1280Kb free=0Kb
 bounds [0x000001db79ed0000, 0x000001db7a010000, 0x000001db7a010000]
 total_blobs=270, nmethods=5, adapters=199, full_count=1
Compilation: disabled (not enough contiguous free space left), stopped_count=1, restarted_count=0
Comments
Changeset: 1f228e55 Branch: master Author: Damon Fenacci <dfenacci@openjdk.org> Date: 2025-04-28 06:18:53 +0000 URL: https://git.openjdk.org/jdk/commit/1f228e5539a5faa3b28e12548f8ad97eeacf3298
28-04-2025
Also reported via JDK-8355088.
22-04-2025
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk/pull/24549 Date: 2025-04-09 14:04:35 +0000
22-04-2025
> Actually it is different: it is missing space for adapters: This seems to be coming from Method::make_adapters if there is no mode code cache space and basic initialization has not yet completed (before compiler initialization has started).
22-04-2025
I was able to reproduce it on linux-x64 with a slowdebug build (with rr, so I could debug it): it looks indeed as if CodeBuffer::initialize returns before initialising _code_start (rightly since blob is null). Then later on, the first time it tries to write into the address (in `emit_int8`) it crashes (full call stack in hs_err_pid2273812.log).
10-04-2025
> `buffer` is not null - it is in C heap. buffer->_blob is null. Of course! Thanks Vladimir! On the other hand possibly _code_start is not initialised to nullptr, since CodeBuffer::initialize returns before initialising it: void CodeBuffer::initialize(csize_t code_size, csize_t locs_size) { ... set_blob(BufferBlob::create(_name, total_size)); if (blob() == nullptr) { // The assembler constructor will throw a fatal on an empty CodeBuffer. return; // caller must test this } // Set up various pointers into the blob. initialize(_total_start, _total_size); ... So, perhaps that's what's causing AbstractAssembler() not to exit the VM.
10-04-2025
ILW = Same as JDK-8347406 = P4
10-04-2025
Yes, all generated C2 blobs are affected.
09-04-2025
>> Actually AbstractAssembler() should have exit VM with OOM error in this case: >> >> Why it does not? > I think it returned earlier because of > if (code == nullptr) return; `buffer` is not null - it is in C heap. buffer->_blob is null.
09-04-2025
May be we missing check that compilation is disabled when we generated C2 blobs.
09-04-2025
I attached hs_err file which show flags.
09-04-2025
Yes, this is intermittent. C1 compilations filed up CodeCache while C2 tread just started and begin generated its own runtime code blobs. In this case uncommon_trap_blob.
09-04-2025
> Actually AbstractAssembler() should have exit VM with OOM error in this case: > > Why it does not? I think it returned earlier because of if (code == nullptr) return;
09-04-2025
[~kvn] comments in JDK-8347406: This happens because we don't check result of buffer allocation at the beginning of code: UncommonTrapBlob* OptoRuntime::generate_uncommon_trap_blob() { // Allocate space for the code ResourceMark rm; // Setup code generation tools const char* name = OptoRuntime::stub_name(OptoStubId::uncommon_trap_id); CodeBuffer buffer(name, 2048, 1024); <<<<<<<<<<<<<<<<<<< this one. MacroAssembler* masm = new MacroAssembler(&buffer); This CodeBuffer constructor calls BufferBlob::create(const char* name, uint buffer_size) which does not check result of allocation. CompileBroker::handle_full_code_cache() did reported full CodeCache before crash: stdout: [[0.253s][warning][codecache] CodeCache is full. Compiler has been disabled. [0.253s][warning][codecache] Try increasing the code cache size using -XX:ReservedCodeCacheSize= CodeCache: size=1280Kb used=1278Kb max_used=1278Kb free=1Kb bounds [0x000001db79ed0000, 0x000001db7a010000, 0x000001db7a010000] total_blobs=268, nmethods=3, adapters=199, full_count=1 Compilation: disabled (not enough contiguous free space left), stopped_count=1, restarted_count=0 # # A fatal error has been detected by the Java Runtime Environment: Actually AbstractAssembler() should have exit VM with OOM error in this case: AbstractAssembler::AbstractAssembler(CodeBuffer* code) { if (code == nullptr) return; CodeSection* cs = code->insts(); cs->clear_mark(); // new assembler kills old mark if (cs->start() == nullptr) { vm_exit_out_of_memory(0, OOM_MMAP_ERROR, "CodeCache: no room for %s", code->name()); } Why it does not?
09-04-2025
[~kvn] I guess the crash is very intermittent but are there possibly some specific flags/apps/code cache sizes that you used to trigger it (e.g. -XX:InitialCodeCacheSize=1189K -XX:ReservedCodeCacheSize=1223k)?
09-04-2025
The issue seems to originate from OptoRuntime::generate_uncommon_trap_blob(): if buffer == null then masm don't get initialized (in AbstractAssembler::AbstractAssembler) and it is referenced plenty of times later in the method (as __ ): ... #define __ masm-> //------------------------------generate_uncommon_trap_blob-------------------- UncommonTrapBlob* OptoRuntime::generate_uncommon_trap_blob() { // Allocate space for the code ResourceMark rm; // Setup code generation tools const char* name = OptoRuntime::stub_name(OptoStubId::uncommon_trap_id); CodeBuffer buffer(name, 2048, 1024); MacroAssembler* masm = new MacroAssembler(&buffer); assert(SimpleRuntimeFrame::framesize % 4 == 0, "sp not 16-byte aligned"); address start = __ pc(); // Push self-frame. We get here with a return address on the // stack, so rsp is 8-byte aligned until we allocate our frame. __ subptr(rsp, SimpleRuntimeFrame::return_off << LogBytesPerInt); // Epilog! ... Probably it would be a good thing to return nullptr as soon as a buffer cannot be allocated.
09-04-2025
The same problem could happen in OptoRuntime::generate_exception_blob as well: ExceptionBlob* OptoRuntime::generate_exception_blob() { ... // Allocate space for the code ResourceMark rm; // Setup code generation tools const char* name = OptoRuntime::stub_name(OptoStubId::exception_id); CodeBuffer buffer(name, 2048, 1024); MacroAssembler* masm = new MacroAssembler(&buffer); address start = __ pc(); // Exception pc is 'return address' for stack walker __ push(rdx); __ subptr(rsp, SimpleRuntimeFrame::return_off << LogBytesPerInt); // Prolog ...
09-04-2025