JDK-8344068 : Windows x86-64: Out of CodeBuffer space when generating final stubs
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 24
  • Priority: P2
  • Status: Resolved
  • Resolution: Fixed
  • OS: windows
  • CPU: x86_64
  • Submitted: 2024-11-12
  • Updated: 2024-12-16
  • Resolved: 2024-12-09
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 24 JDK 25
24Unresolved 25 b02Fixed
Related Reports
Relates :  
JDK-8331341 - secondary_super_cache does not scale well: C1 and interpreter
Description
Test: compiler/c2/TestLWLockingCodeGen.java

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (c:\sb\prod\1731376771\workspace\open\src\hotspot\share\asm/codeBuffer.hpp:200), pid=26820, tid=48168
#  assert(allocates2(pc)) failed: not in CodeBuffer memory: 0x000002040c2f33e0 <= 0x000002040c300041 <= 0x000002040c300040
#
# JRE version:  (24.0+24) (fastdebug build )
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 24-ea+24-2902, compiled mode, sharing, compressed class ptrs, z gc, windows-amd64)
# Core dump will be written. Default location: C:\sb\prod\1731424610\testoutput\test-support\jtreg_open_test_hotspot_jtreg_hotspot_compiler\scratch\3\hs_err_pid26820.mdmp
#
#

---------------  S U M M A R Y ------------

Command Line: -Dtest.vm.opts=-XX:MaxRAMPercentage=4.16667 -Dtest.boot.jdk=c:\ade\mesos\work_dir\jib-master\install\jdk\23\37\bundles\windows-x64\jdk-23_windows-x64_bin.zip\jdk-23 -Djava.io.tmpdir=c:\sb\prod\1731424610\testoutput\test-support\jtreg_open_test_hotspot_jtreg_hotspot_compiler\tmp -XX:+CreateCoredumpOnCrash -Dtest.tool.vm.opts=-J-XX:MaxRAMPercentage=4.16667 -J-Dtest.boot.jdk=c:\ade\mesos\work_dir\jib-master\install\jdk\23\37\bundles\windows-x64\jdk-23_windows-x64_bin.zip\jdk-23 -J-Djava.io.tmpdir=c:\sb\prod\1731424610\testoutput\test-support\jtreg_open_test_hotspot_jtreg_hotspot_compiler\tmp -J-XX:+CreateCoredumpOnCrash -Dtest.compiler.opts= -Dtest.java.opts=-XX:+UseZGC -Dtest.jdk=c:\ade\mesos\work_dir\jib-master\install\jdk-24+24-2902\windows-x64-debug.jdk\jdk-24\fastdebug -Dcompile.jdk=c:\ade\mesos\work_dir\jib-master\install\jdk-24+24-2902\windows-x64-debug.jdk\jdk-24\fastdebug -Dtest.timeout.factor=10.0 -Dtest.nativepath=c:\ade\mesos\work_dir\jib-master\install\jdk-24+24-2902\windows-x64-debug.test\hotspot\jtreg\native -Dtest.root=C:\ade\mesos\work_dir\jib-master\install\jdk-24+24-2902\src.full\open\test\hotspot\jtreg -Dtest.name=compiler/c2/TestLWLockingCodeGen.java -Dtest.file=C:\ade\mesos\work_dir\jib-master\install\jdk-24+24-2902\src.full\open\test\hotspot\jtreg\compiler\c2\TestLWLockingCodeGen.java -Dtest.src=C:\ade\mesos\work_dir\jib-master\install\jdk-24+24-2902\src.full\open\test\hotspot\jtreg\compiler\c2 -Dtest.src.path=C:\ade\mesos\work_dir\jib-master\install\jdk-24+24-2902\src.full\open\test\hotspot\jtreg\compiler\c2 -Dtest.classes=C:\sb\prod\1731424610\testoutput\test-support\jtreg_open_test_hotspot_jtreg_hotspot_compiler\classes\1\compiler\c2\TestLWLockingCodeGen.d -Dtest.class.path=C:\sb\prod\1731424610\testoutput\test-support\jtreg_open_test_hotspot_jtreg_hotspot_compiler\classes\1\compiler\c2\TestLWLockingCodeGen.d -Dtest.class.path.prefix=C:\sb\prod\1731424610\testoutput\test-support\jtreg_open_test_hotspot_jtreg_hotspot_compiler\classes\1\compiler\c2\TestLWLockingCodeGen.d;C:\ade\mesos\work_dir\jib-master\install\jdk-24+24-2902\src.full\open\test\hotspot\jtreg\compiler\c2 -XX:MaxRAMPercentage=4.16667 -Dtest.boot.jdk=c:\ade\mesos\work_dir\jib-master\install\jdk\23\37\bundles\windows-x64\jdk-23_windows-x64_bin.zip\jdk-23 -Djava.io.tmpdir=c:\sb\prod\1731424610\testoutput\test-support\jtreg_open_test_hotspot_jtreg_hotspot_compiler\tmp -XX:+CreateCoredumpOnCrash -XX:+UseZGC -Djava.library.path=c:\ade\mesos\work_dir\jib-master\install\jdk-24+24-2902\windows-x64-debug.test\hotspot\jtreg\native -XX:+ShowMessageBoxOnError -Xcomp -XX:-TieredCompilation -XX:CompileOnly=TestLWLockingCodeGen::sync com.sun.javatest.regtest.agent.MainWrapper C:\sb\prod\1731424610\testoutput\test-support\jtreg_open_test_hotspot_jtreg_hotspot_compiler\compiler\c2\TestLWLockingCodeGen.d\main.0.jta

Host: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz, 12 cores, 23G,  Windows Server 2019 , 64 bit Build 17763 (10.0.17763.4974)
Time: Tue Nov 12 15:26:01 2024 Etc elapsed time: 0.070776 seconds (0d 0h 0m 0s)

---------------  T H R E A D  ---------------

Current thread (0x0000020403a21680):  JavaThread "Unknown thread" [_thread_in_vm, id=48168, stack(0x000000f5a1100000,0x000000f5a1200000) (1024K)]

Stack: [0x000000f5a1100000,0x000000f5a1200000]
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0xd112b1]  os::win32::platform_print_native_stack+0x101  (os_windows_x86.cpp:235)
V  [jvm.dll+0xfde14d]  VMError::report+0x148d  (vmError.cpp:1011)
V  [jvm.dll+0xfe07cd]  VMError::report_and_die+0x80d  (vmError.cpp:1846)
V  [jvm.dll+0xfe0ed4]  VMError::report_and_die+0x64  (vmError.cpp:1611)
V  [jvm.dll+0x5a3b8b]  report_vm_error+0x5b  (debug.cpp:193)
V  [jvm.dll+0x2cdb67]  CodeSection::set_end+0x97  (codeBuffer.hpp:200)
V  [jvm.dll+0x2b88ca]  Assembler::movdqu+0x12a  (assembler_x86.cpp:3430)
V  [jvm.dll+0xbe4808]  MacroAssembler::movdqu+0xb8  (macroAssembler_x86.cpp:2639)
V  [jvm.dll+0x103ddca]  ZRuntimeCallSpill::restore+0x7a  (zBarrierSetAssembler_x86.cpp:122)
V  [jvm.dll+0x103d93e]  ZBarrierSetAssembler::load_at+0x2ae  (zBarrierSetAssembler_x86.cpp:280)
V  [jvm.dll+0xbc6dd7]  MacroAssembler::access_load_at+0x127  (macroAssembler_x86.cpp:6039)
V  [jvm.dll+0xbe0ca8]  MacroAssembler::load_heap_oop+0x98  (macroAssembler_x86.cpp:6058)
V  [jvm.dll+0xe6d916]  StubGenerator::generate_upcall_stub_load_target+0x1a6  (stubGenerator_x86_64.cpp:3814)
V  [jvm.dll+0xe6b78e]  StubGenerator::generate_final_stubs+0x3ce  (stubGenerator_x86_64.cpp:3983)
V  [jvm.dll+0xe5f62b]  StubGenerator_generate+0x5b  (stubGenerator_x86_64.cpp:4256)
V  [jvm.dll+0xec6afa]  initialize_stubs+0xda  (stubRoutines.cpp:248)
V  [jvm.dll+0xec6971]  final_stubs_init+0x41  (stubRoutines.cpp:302)
V  [jvm.dll+0x7fde40]  init_globals2+0x70  (init.cpp:192)
V  [jvm.dll+0xf4e0bb]  Threads::create_vm+0x44b  (threads.cpp:576)
V  [jvm.dll+0x927962]  JNI_CreateJavaVM_inner+0x82  (jni.cpp:3596)
V  [jvm.dll+0x92b60f]  JNI_CreateJavaVM+0x1f  (jni.cpp:3687)
C  [jli.dll+0x5377]  JavaMain+0x113  (java.c:488)
C  [ucrtbase.dll+0x2268a]  (no source info available)
C  [KERNEL32.DLL+0x17ac4]  (no source info available)
C  [ntdll.dll+0x5a4e1]  (no source info available)
Comments
Changeset: 830173fc Branch: master Author: Andrew Haley <aph@openjdk.org> Date: 2024-12-09 11:05:25 +0000 URL: https://git.openjdk.org/jdk/commit/830173fcb08b004ea3932d47cb522c589feec0b5
09-12-2024
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk/pull/22516 Date: 2024-12-03 14:38:39 +0000
03-12-2024
OK, I looked. As far as I can see this is to do with AVX512 stubs, which are voluminous indeed.
03-12-2024
OK, thanks. So, the delta of the memory used (in an x86 fastdebug build) by final stubs between Windows and Linux running the exact same test is not +2000 but in fact +22149, which is perhaps surprising. I'm quite happy simply to submit a PR which bumps the Windows delta to something which more accurately reflects reality, but I wonder if a Windows expert should voice an opinion.
21-11-2024
Here's the -Xlog:stubs=info output on the problematic "Windows_Server_2022_Standard - VM.Standard3.Flex" system *without* your changes: [0.011s][info][stubs] StubRoutines (initial stubs) [0x000001e5c82e0ee0, 0x000001e5c82e6228] used: 19879, free: 1441 [0.038s][info][stubs] StubRoutines (continuation stubs) [0x000001e5c82e68e0, 0x000001e5c82e71f0] used: 1447, free: 873 CompileCommand: compileonly TestLWLockingCodeGen.sync bool compileonly = true [0.059s][info][stubs] StubRoutines (final stubs) [0x000001e5c83633e0, 0x000001e5c8370040] used: 51883, free: 437 [0.099s][info][stubs] StubRoutines (compiler stubs) [0x000001e5c84025e0, 0x000001e5c8413c00] used: 54502, free: 16698 There is not much space left, as expected.
21-11-2024
[~aph] I'll re-run with -Xlog:stubs=info and will share the output here. Update: Here's the output: [0.012s][info][stubs] StubRoutines (initial stubs) [0x000001bcd3500ee0, 0x000001bcd3506228] used: 19879, free: 1441 [0.046s][info][stubs] StubRoutines (continuation stubs) [0x000001bcd35068e0, 0x000001bcd35071f0] used: 1447, free: 873 CompileCommand: compileonly TestLWLockingCodeGen.sync bool compileonly = true # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (workspace\\open\\src\\hotspot\\share\\asm/codeBuffer.hpp:200), pid=30416, tid=45580 # assert(allocates2(pc)) failed: not in CodeBuffer memory: 0x000001bcd35833e0 <= 0x000001bcd3590041 <= 0x000001bcd3590040 In the passing runs (it's intermittent), it looks like this: [0.010s][info][stubs] StubRoutines (initial stubs) [0x000001ee977b0ee0, 0x000001ee977b6228] used: 17993, free: 3327 [0.041s][info][stubs] StubRoutines (continuation stubs) [0x000001ee977b68e0, 0x000001ee977b71f0] used: 1447, free: 873 CompileCommand: compileonly TestLWLockingCodeGen.sync bool compileonly = true [0.059s][info][stubs] StubRoutines (final stubs) [0x000001ee97832e60, 0x000001ee9783fac0] used: 39019, free: 13301 [0.095s][info][stubs] StubRoutines (compiler stubs) [0x000001ee978cff60, 0x000001ee978e1580] used: 53833, free: 17367 Looks like it's only/mostly failing on our "Windows_Server_2022_Standard - VM.Standard3.Flex": Intel_R__Xeon_R__Platinum_8358_CPU___2.60GHz Windows_Server_2022_Standard fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht pni vmx ssse3 fma cx16 pdcm sse4 1 sse4 2 x2apic movbe popcnt aes xsave osxsave avx f16c rdrand hypervisor lahf lm arat xsaveopt fsgsbase tsc adjust bmi1 avx2 smep bmi2 erms invpcid avx512f rdseed adx smap clflushopt avx512cd
21-11-2024
I've tested on Linux. JDK-8331341 adds 512 bytes to the final stubs. There's nothing Windows-specific in that change as far as I can see, so I guess it also adds 512 bytes to the WIndows final stubs. However, we are nowhere near the limit for Linux/x86-64/ZGC: [3.359s][info][stubs] StubRoutines (final stubs) [0x00007fffe8502b60, 0x00007fffe850eff0] used: 29734, free: 20586 So, it looks like we have ton of space left, and Windows adds 2000 bytes to that limit. It could be that AVX512 makes a big difference here, but it sounds implausible. Are you in a position to re-run that test with -J-Xlog:stubs=info ? Maybe the Windows increment of 2000 bytes is too little? Alternatively, I'm happy to just submit a patch to do this, if you prefer. 1000 bytes to the 64-bit baseline should be enough to offset JDK-8331341. --- a/src/hotspot/cpu/x86/stubRoutines_x86.hpp +++ b/src/hotspot/cpu/x86/stubRoutines_x86.hpp @@ -38,7 +38,7 @@ enum platform_dependent_constants { // AVX512 intrinsics add more code in 64-bit VM, // Windows have more code to save/restore registers _compiler_stubs_code_size = 20000 LP64_ONLY(+46000) WINDOWS_ONLY(+2000), - _final_stubs_code_size = 10000 LP64_ONLY(+20000) WINDOWS_ONLY(+2000) ZGC_ONLY(+20000) + _final_stubs_code_size = 10000 LP64_ONLY(+21000) WINDOWS_ONLY(+2000) ZGC_ONLY(+20000) }; class x86 {
20-11-2024
OK, I'll have a look.
20-11-2024
ILW = HMM = P2
19-11-2024
It might be a bit intermittent and first showed up only in tier6.
14-11-2024
JDK-8331341 was integrated a week ago and the problem only started showing up yesterday. ??
14-11-2024
Andrew, would you have time to look at this?
13-11-2024
Test is from JDK-8329726. Looks like a ZGC specific issue. Bisection points to JDK-8331341 which modified stubs. I assume that we need to increase the _final_stubs_code_size on Windows: https://github.com/openjdk/jdk/blob/2eeaa57b19780723ad7c74b1a62dea491241b686/src/hotspot/cpu/x86/stubRoutines_x86.hpp#L41
13-11-2024