JDK-8272364 : Parallel GC adaptive size policy may shrink the heap below MinHeapSize
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 17,18,21,22
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2021-08-12
  • Updated: 2024-07-09
  • Resolved: 2024-05-03
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 23
23 b22Fixed
Related Reports
Relates :  
Relates :  
Description
[~rkennke] reports that the heap size can become lower than -Xms / MinHeapSize.

https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2021-August/036307.html

---
I have an example that shows the 'problem':

http://cr.openjdk.java.net/~rkennke/Example.java

Reproduce it using:

java -Xms1g -Xmx4g -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Xlog:gc*=info,gc=debug,gc+heap*=trace,gc+ergo*=trace:file=gc.log:time,uptimemillis,level,tags Example 300

Check the heap log for heap size entries:

grep "Pause Young" gc.log

This yields the following:

http://cr.openjdk.java.net/~rkennke/gc-heapsize.txt

It shows that the heap is indeed shrinking well below -Xms, e.g. ~600MB instead of 1024MB here:

GC(12) Pause Young (Allocation Failure) 559M->589M(592M) 4,797ms 
---

The test case:
---
import java.util.*;

public class Example {

    // private static int LOOP_COUNT = Integer.MAX_VALUE;
    private static int LOOP_COUNT = 100;

    private static int DEFAULT_LIST_SIZE = 300; // default value (300 = 300MB)

    public static void main(String... args) {
        Example example = new Example();

        int listSize = DEFAULT_LIST_SIZE;
        if (args.length > 0) {
            listSize = Integer.parseInt(args[0]);
        }

        for (int i = 0; i < LOOP_COUNT; i++) {
            example.test(listSize);
            try {
                // Thread.sleep(100L);
                Thread.sleep(500L);
                // Thread.sleep(1000L);
            } catch (Exception ignore) {}
        }
    }

    // specify enough size to listSize (depending on your heap setting),
    // so the ArrayList can be promoted to old gen and can trigger full GC.
    private void test(int listSize) {
        List list = new ArrayList();
        for (int i = 0; i < listSize; i++) {
            list.add(new byte[1024 * 1024]); // add 1MB byte array to ArrayList
        }
        // clear ArrayList at the end, so it can be GCed at full GC cycle
        list = new ArrayList();
    }
}
---

The heap size should stay within MinHeapSize and MaxHeapSize, so this seems to be a bug.
Comments
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk22u/pull/217 Date: 2024-05-20 18:25:59 +0000
09-07-2024

May I get code review for jdk22u backport? https://git.openjdk.org/jdk22u/pull/217 Thanks,
29-05-2024

A pull request was submitted for review. URL: https://git.openjdk.org/jdk22u/pull/214 Date: 2024-05-17 22:29:40 +0000
17-05-2024

Changeset: 6bef0474 Author: Zhengyu Gu <zgu@openjdk.org> Date: 2024-05-03 00:28:18 +0000 URL: https://git.openjdk.org/jdk/commit/6bef0474c8b8773d0d20c0f25c36a2ce9cdbd7e8
03-05-2024

A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/18877 Date: 2024-04-20 15:05:55 +0000
20-04-2024

It looks like another miscalculation of ergonomics. Run attached test case and log has: [0.014s][debug][gc,heap] Minimum heap 1073741824 Initial heap 1073741824 Maximum heap 4294967296 [0.014s][trace][gc,heap] 1: Minimum young 1572864 Initial young 357564416 Maximum young 1431306240 [0.014s][trace][gc,heap] Minimum old 524288 Initial old 716177408 Maximum old 2863661056 The Minimum young size seems low. Probably the same issue as JDK-8329223.
18-04-2024

One other thing to keep in mind when looking at the logs is that the capacity of the heap for Serial and Parallel doesn't include "to-space" so here the heap is still 1G even if it looks like it is not: --- [13,546s][info ][gc ] GC(40) Pause Young (Allocation Failure) 598M->597M(1004M) 21,535ms [13,546s][info ][gc,cpu ] GC(40) User=0,50s Sys=0,00s Real=0,02s [13,546s][debug][gc,heap ] GC(40) Heap after GC invocations=41 (full 17): [13,546s][debug][gc,heap ] GC(40) PSYoungGen total 329216K, used 19520K [0x00000007aab00000, 0x00000007c0000000, 0x0000000800000000) [13,546s][debug][gc,heap ] GC(40) eden space 309248K, 0% used [0x00000007aab00000,0x00000007aab00000,0x00000007bd900000) [13,546s][debug][gc,heap ] GC(40) from space 19968K, 97% used [0x00000007bd900000,0x00000007bec10130,0x00000007bec80000) [13,546s][debug][gc,heap ] GC(40) to space 19968K, 0% used [0x00000007bec80000,0x00000007bec80000,0x00000007c0000000) [13,546s][debug][gc,heap ] GC(40) ParOldGen total 699392K, used 592596K [0x0000000700000000, 0x000000072ab00000, 0x00000007aab00000) [13,546s][debug][gc,heap ] GC(40) object space 699392K, 84% used [0x0000000700000000,0x00000007242b5278,0x000000072ab00000) [13,546s][debug][gc,heap ] GC(40) Metaspace used 6126K, committed 6272K, reserved 1056768K [13,546s][debug][gc,heap ] GC(40) class space used 516K, committed 576K, reserved 1048576K ---
24-08-2021

I took a look at this since I made a change in this area back in 2014 and it might be the cause for this. The problem seem to be that the adaptive size policy acts on the generations alone without looking at the whole heap. This can lead to the used heap being smaller than Xms. The change made back in 2014 was to set the minimum old generation size to GenAlignment, since this is the smallest possible value. The sizing code is pretty brittle and changing values might have some side effects because there are always a few different ways to change a generation size, ratios or explicit sizes. I don't think just reverting the calculation for MinOldSize will work for all scenarios. Comparing Parallel to Serial I see that the generations created for Serial uses the initial generation size for both min and initial size, while for Parallel the actual minimum size is used. Changing parallel to also use initial as minimum seems to work, but not sure if that will come with any unexpected side-effects. If going with that approach we should probably also look at if removing MinNewSize and MinOldSize from GenArguments is possible.
24-08-2021

Does not reproduce with G1 and serial gc. Seems to be an artifact of Parallel GC's adaptive size policy.
13-08-2021

I tested and reproduced this issue with the latest openjdk/jdk and -XX:+UseParallelGC.
12-08-2021

What version does this issue apply to? -XX:+UseParallelOldGC was obsoleted in JDK 15, and removed in JDK 16.
12-08-2021