JDK-8275713 : TestDockerMemoryMetrics test fails on recent runc
  • Type: Bug
  • Component: core-libs
  • Affected Version: openjdk8u312,11.0.13,13
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • CPU: generic
  • Submitted: 2021-10-21
  • Updated: 2022-12-05
  • Resolved: 2021-10-28
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 Other
11.0.14 b01Fixed openjdk8u372Fixed
Related Reports
Relates :  
Relates :  
Description
This bug affects JDKs < 15.

With recent runc as OCI runtime, --kernel-memory option is being ignored. I.e. even if you specify a value it'll not populate. The Linux kernel deprecated the feature and runc did so to. 'crun', another runtime is about to ignore the option too. References:
runc: https://github.com/opencontainers/runc/pull/2840
podman PR to remove (hide) the option: https://github.com/containers/podman/pull/12048

The effect of this is that container test TestDockerMemoryMetrics.java fails on recent docker (which uses runc by default). It also fails when run with podman and runc as runtime.

Failure looks like this:
========== NEW TEST CASE:      testKernelMemoryLimit, value = 100m
[COMMAND]
docker run --tty=true --rm --volume /disk/openjdk/upstream-sources/git/jdk11u/JTwork/classes/jdk/internal/platform/docker/TestDockerMemoryMetrics.d:/test-classes/ --kernel-memory=100m jdk-internal:test-metrics-memory /jdk/bin/java -cp /test-classes/ --add-exports java.base/jdk.internal.platform=ALL-UNNAMED -Djdk.test.docker.image.name=fedora -Djdk.test.docker.image.version=34 MetricsMemoryTester kernelmem 100m
[ELAPSED: 854 ms]
[STDERR]
WARNING: Specifying a kernel memory limit is deprecated and will be removed in a future release.

[STDOUT]
[kernelmem, 100m]
Exception in thread "main" java.lang.RuntimeException: Kernel Memory limit not equal, expected : [104857600], got : [-1]
        at MetricsMemoryTester.testKernelMemoryLimit(MetricsMemoryTester.java:113)
        at MetricsMemoryTester.main(MetricsMemoryTester.java:38)


The stand-alone Docker/podman equivalent is:
$ docker run --kernel-memory 200m --rm -ti fedora:34 cat sys/fs/cgroup/memory/memory.kmem.limit_in_bytes
WARNING: Specifying a kernel memory limit is deprecated and will be removed in a future release.
9223372036854771712

Note the huge value instead of the (previously) expected value of 209715200 for the kernel memory limit.

It doesn't affect JDK mainline as it has cgroups v2 support and the follow-up fix of that, JDK-8240189, included a fix for this issue. JDK-8240189 landed in JDK 15. In particular this hunk from JDK-8240189 needs backporting to older releases:

 public class MetricsMemoryTester {
+
+    public static final long UNLIMITED = -1;
+
     public static void main(String[] args) {
         System.out.println(Arrays.toString(args));
         switch (args[0]) {
@@ -115,13 +118,13 @@ public class MetricsMemoryTester {
             System.out.println("TEST PASSED!!!");
             long limit = getMemoryValue(value);
             long kmemlimit = mCgroupV1.getKernelMemoryLimit();
-            if (kmemlimit != 0 && limit != kmemlimit) {
+            if (kmemlimit != UNLIMITED && limit != kmemlimit) {
                 throw new RuntimeException("Kernel Memory limit not equal, expected : ["
                         + limit + "]" + ", got : ["
                         + kmemlimit + "]");
             }

Comments
A pull request was submitted for review. URL: https://git.openjdk.org/jdk8u-dev/pull/121 Date: 2022-09-21 10:12:07 +0000
01-11-2022

Changeset: fd22289a Author: Severin Gehwolf <sgehwolf@openjdk.org> Date: 2021-10-28 08:20:57 +0000 URL: https://git.openjdk.java.net/jdk11u/commit/fd22289a851a645c09f63229469decfb7b3ee450
03-11-2021

Changeset: fd22289a Author: Severin Gehwolf <sgehwolf@openjdk.org> Date: 2021-10-28 08:20:57 +0000 URL: https://git.openjdk.java.net/jdk11u-dev/commit/fd22289a851a645c09f63229469decfb7b3ee450
28-10-2021

Fix Request (OpenJDK 11u): Please approve this JDK 11 fix for the container tests (Testbug). It accounts for recent container runtimes which start to ignore --kernel-memory switch. This patch makes the test only fail if the returned kernel memory is not "unlimited" (as that is the value returned when the setting is ignored). Test only, no risk. Test fails prior this patch, passes after. See JDK-8275735 where I propose to remove this Linux only API entirely for JDK mainline.
27-10-2021