JDK-8244500 : jtreg test error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 15
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • CPU: unknown
  • Submitted: 2020-05-06
  • Updated: 2022-12-21
  • Resolved: 2020-06-22
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 15 JDK 16 JDK 8 Other
11.0.16Fixed 15.0.4Fixed 16 b03Fixed 8u371Fixed openjdk8u372Fixed
Related Reports
Duplicate :  
JDK-8244109 - error in test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java
Relates :  
JDK-8250984 - Memory Docker tests fail on some Linux kernels w/o cgroupv1 swap limit capabilities
Relates :  
JDK-8231111 - Cgroups v2: Rework Metrics in java.base so as to recognize unified hierarchy
Relates :  
JDK-8247863 - Unreachable code in OperatingSystemImpl.getTotalSwapSpaceSize()
Relates :  
JDK-8242480 - Negative value may be returned by getFreeSwapSpaceSize() in the docker
Relates :  
JDK-8236617 - jtreg test containers/docker/TestMemoryAwareness.java fails after 8226575
Description
I noticed  issues  on  one of our Linux  test machines  with jtreg test   test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java
After  JDK-8231111: Cgroups v2: Rework Metrics in java.base so as to recognize unified hierarchy  .
we see errors on one of our Linux test machines in jtreg test   test/hotspot/jtreg/containers/docker/TestMemoryAwareness.java .

> Error:
> test result: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: 'OperatingSystemMXBean.getTotalSwapSpaceSize: 0' missing from stdout/stderr
> Test output reports:
> OperatingSystemMXBean.getTotalSwapSpaceSize: 16106061824
> OperatingSystemMXBean.getFreeSwapSpaceSize: 15958736896
> We also see this output:
> WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
>

Maybe a check for the  warning message “Your kernel does not support swap limit capabilities”  should be added to the test.
Comments
For reference, the actual code change for OpenJDK 8u got included with the 8u backport of JDK-8250984.
12-12-2022
Fix request (8u) Please consider this for jdk8u-dev as part of cgroups v2 backports support. It's a test-only change and has reduced to a functional no-op but merging will keep things in sync context-wise for further backports.
12-12-2022
A pull request was submitted for review. URL: https://git.openjdk.org/jdk8u-dev/pull/142 Date: 2022-10-17 08:17:20 +0000
17-10-2022
[~ebaron] Please only integrate this after the first batch of cgroups v2 changes get in. See JDK-8230305. Thanks!
22-03-2022
Fix Request 11u Requesting a backport to 11u as part of CGroups v2 support. The patch did not apply cleanly due to one hunk made obsolete by JDK-8250984. PR was reviewed by sgehwolf.
21-03-2022
A pull request was submitted for review. URL: https://git.openjdk.java.net/jdk11u-dev/pull/920 Date: 2022-03-18 22:14:20 +0000
18-03-2022
Fix request (15u) Requesting backport to 15u as a prerequisite for JDK-8250984. This is test-only fix, applies cleanly. Tested with container tests, the affected tests passed.
07-05-2021
Changeset: 732d8865 Author: Severin Gehwolf <sgehwolf@openjdk.org> Date: 2020-05-20 21:12:20 +0000 URL: https://git.openjdk.java.net/lanai/commit/732d8865
02-07-2020
Changeset: 732d8865 Author: Severin Gehwolf <sgehwolf@openjdk.org> Date: 2020-05-20 21:12:20 +0000 URL: https://git.openjdk.java.net/panama-foreign/commit/732d8865
02-07-2020
Changeset: 732d8865 Author: Severin Gehwolf <sgehwolf@openjdk.org> Date: 2020-05-20 21:12:20 +0000 URL: https://git.openjdk.java.net/amber/commit/732d8865
02-07-2020
Changeset: 732d8865 Author: Severin Gehwolf <sgehwolf@openjdk.org> Date: 2020-05-20 21:12:20 +0000 URL: https://git.openjdk.java.net/mobile/commit/732d8865
02-07-2020
URL: https://hg.openjdk.java.net/jdk/jdk/rev/8d722fb14093 User: sgehwolf Date: 2020-06-22 11:59:02 +0000
22-06-2020
This was originally fixed for the old cgroups implementation with JDK-8236617. The old impl returned 0 for systems where underlying cgroup files were missing. The implementation supporting cgroups v1 AND cgroups v2 returns -1 when required files don't exist.
18-06-2020
Suggested fix (tested by Matthias on the affected system): http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8244500/03/webrev/ RFR: http://mail.openjdk.java.net/pipermail/hotspot-dev/2020-June/042006.html
08-06-2020
So we do get a memory limit value, just not for swap. The trouble is that we might get the same behaviour for some systems which specify unlimited swap via --memory-swap=-1 $ cat MetricsTest.java import jdk.internal.platform.Metrics; public class MetricsTest { public static void main(String[] args) { Metrics m = Metrics.systemMetrics(); if (m != null) { System.out.println("m.getMemoryLimit() = " + m.getMemoryLimit()); System.out.println("m.getMemoryAndSwapLimit() = " + m.getMemoryAndSwapLimit()); } } } Running this program in a docker container like so: $ sudo docker run -ti --rm --memory=100M --memory-swap=-1 -v $(pwd)/build/linux-x86_64-server-release/images/jdk:/jdk fedora:32 /bin/bash [root@edabdc335b21 /]# /jdk/bin/java -cp /jdk --add-exports=java.base/jdk.internal.platform=ALL-UNNAMED MetricsTest m.getMemoryLimit() = 104857600 m.getMemoryAndSwapLimit() = -1 So if we were to change OperatingSystemMXBean to recognize this pattern - returning 0, say, we would potentially be incorrect. The pattern being -1 for getMemoryAndSwapLimit() and > 0 for getMemoryLimit() that is. Note however, podman doesn't seem to allow -1 for --memory-swap, despite the man page saying (works for docker): ------------ --memory-swap=number [...] Set number to -1 to enable unlimited swap. -------------
04-06-2020
Matthias reports that he gets this as diagnostic output on the affected system: Metrics.getMemoryAndSwapLimit() == -1 Metrics.getMemoryLimit() == 104857600 Metrics.getMemoryAndSwapUsage() == -1 Metrics.getMemoryUsage() == 30670848
04-06-2020
This seems to be a case where host system values - over container values - are now being returned. Note that there is a conditional on line 63 of OperatingSystemImpl.java which seems to evaluate to false because limit < 0 (I suspect it to be -1, but I've got no way of verifying it). long limit = containerMetrics.getMemoryAndSwapLimit(); long memLimit = containerMetrics.getMemoryLimit(); if (limit >= 0 && memLimit >= 0) { [...] // return container limit } // return host value
18-05-2020