JDK-8358129 : compiler/startup/StartupOutput.java runs into out of memory on Windows after JDK-8347406
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 25
  • Priority: P4
  • Status: In Progress
  • Resolution: Unresolved
  • OS: windows
  • Submitted: 2025-05-30
  • Updated: 2025-06-09
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 25
25Unresolved
Related Reports
Causes :  
Sub Tasks
JDK-8358259 :  
Description
The test compiler/startup/StartupOutput.java  starts 200 VMs in a loop , this can lead to resource shortages on some machines.
We see errors like this, probably the number of started JVMs should be somewhat reduced e.g. to 50 or 100, and the resource consumption of those started JVMs should be adjusted too.

Example error :
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 4194304 bytes. Error detail: G1 virtual space
# Possible reasons:
#   The system is out of physical RAM or swap space
#   This process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
#   JVM is running with Zero Based Compressed Oops mode in which the Java heap is
#     placed in the first 32GB address space. The Java Heap base address is the
#     maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
#     to set the Java Heap base and to place the Java Heap above 32GB virtual address.
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_windows.cpp:3556), pid=19372, tid=46104

We see an
InitialHeapSize                          = 536870912   (~500m plus other memory needed by the JVM)
this is probably too much and leads to the resource shortage.
Comments
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk/pull/25582 Date: 2025-06-02 10:57:22 +0000
02-06-2025

Sorry, I think my previous comment wasn't very clear: what I meant is that the loop in StartupOutput.java starts 200 VMs concurrently but doesn't need to do it concurrently. I think what you are referring to is the case of multiple tests that should run non-concurently, e.g. using "exclusiveAccess.dirs".
02-06-2025

> There is no real need to run those tests concurrently I think it is the other way around, if the test has to run 'lonely'/ non-concurrently it has to be specified ; I think there are a couple of such tests already with this requirement.
02-06-2025

There is no real need to run those tests concurrently (they are short and basically check that the VM doesn't crash with limited code cache). I think running them sequentially should be OK (and would avoid running out of memory).
02-06-2025

ILW = Test failure due to running out of memory, single test and problemlisted, no workaround = MLH = P4 We should fix it in JDK 25 but since it's a test issue, it can still be fixed in RDP 1 even though this is a P4.
02-06-2025

>This issue is also causing other tests to fail with the same kinds of memory issues - >presumably they are trying to run concurrently on the same machine! Yes [~dholmes] we observed the same - concurrently running tests are influenced by the temporary resource shortage caused by starting many JVMs 'holding' each too much memory.
02-06-2025

Damon, please have a look.
02-06-2025

Will likely need to ProblemList the misbehaving test.
01-06-2025

This issue is also causing other tests to fail with the same kinds of memory issues - presumably they are trying to run concurrently on the same machine!
01-06-2025

We also see this in our CI test system.
01-06-2025

On the system where we see most failures we have 32G RAM. Maybe reduce the number of VMs started from 200 to 100, and set a 'good' Xmx value for them? The ergo - values calculated are probably not good for this kind of test. Maybe something like Xmx = 32G / (#procs * 2) , so we would end up with an Xmx value of ~ 150M for the procs started by the test?
30-05-2025

The main problem seems to be that on Windows there is no real overcommit - the memory must be backed by the swapfile as far as I know. And this test starts 200 VMs concurrently in a tight loop without heap settings. Perhaps the test can be changed to not run the VMs concurrently or define a heap size, or both. I am also not sure whether "out.stdoutShouldNotContain("# A fatal error")" will wait for VM termination to check - otherwise we could miss crashes, because the VM startup is to slow. The "out.getExitValue()" call after this will at least wait for process termination to get the exit code.
30-05-2025

Hi [~azeller] maybe you have further valuable input ?
30-05-2025