JDK-8239559 : Cgroups: Incorrect detection logic on some systems
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2020-02-20
  • Updated: 2022-05-03
  • Resolved: 2020-02-25
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 15
11.0.16Fixed 15 b12Fixed
Related Reports
Relates :  
Relates :  
Relates :  
Relates :  
After JDK-8231111 there are some reports of failures on some cgroups v1 systems (e.g. SLES 11/12). In particular, systems which support cgroups v1, but don't have controllers mounted on a cgroups v1 hierarchy.

Failure looks like:
java.lang.InternalError: java.lang.reflect.InvocationTargetException
                at java.base/jdk.internal.platform.Metrics.systemMetrics(Metrics.java:65)
                at java.base/jdk.internal.platform.Container.metrics(Container.java:43)
                at jdk.management/com.sun.management.internal.OperatingSystemImpl.<init>(OperatingSystemImpl.java:48)
                at jdk.management/com.sun.management.internal.PlatformMBeanProviderImpl.getOperatingSystemMXBean(PlatformMBeanProviderImpl.java:281)
                at jdk.management/com.sun.management.internal.PlatformMBeanProviderImpl$3.nameToMBeanMap(PlatformMBeanProviderImpl.java:198)
                at java.management/sun.management.spi.PlatformMBeanProvider$PlatformComponent.getMBeans(PlatformMBeanProvider.java:195)
                at java.management/java.lang.management.ManagementFactory.getPlatformMXBean(ManagementFactory.java:686)
                at java.management/java.lang.management.ManagementFactory.getOperatingSystemMXBean(ManagementFactory.java:388)
                at com.sun.javatest.regtest.config.OS.<init>(OS.java:158)
                at com.sun.javatest.regtest.config.OS.current(OS.java:59)
                at com.sun.javatest.regtest.config.RegressionContext.<init>(RegressionContext.java:62)
                at com.sun.javatest.regtest.config.RegressionContext.getDefault(RegressionContext.java:45)
                at com.sun.javatest.regtest.config.RegressionTestFinder.<init>(RegressionTestFinder.java:93)
                at com.sun.javatest.regtest.config.RegressionTestSuite.createTestFinder(RegressionTestSuite.java:100)
                at com.sun.javatest.regtest.config.RegressionTestSuite.<init>(RegressionTestSuite.java:82)
                at com.sun.javatest.regtest.config.RegressionTestSuite.open(RegressionTestSuite.java:65)
                at com.sun.javatest.regtest.config.TestManager.getTestSuites(TestManager.java:164)
                at com.sun.javatest.regtest.tool.Tool.run(Tool.java:1076)
                at com.sun.javatest.regtest.tool.Tool.run(Tool.java:1027)
                at com.sun.javatest.regtest.tool.Tool.main(Tool.java:143)
                at com.sun.javatest.regtest.Main.main(Main.java:58)
Caused by: java.lang.reflect.InvocationTargetException
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.base/java.lang.reflect.Method.invoke(Method.java:564)
                at java.base/jdk.internal.platform.Metrics.systemMetrics(Metrics.java:61)
                ... 20 more
Caused by: java.lang.ExceptionInInitializerError
                at java.base/jdk.internal.platform.CgroupSubsystemFactory.create(CgroupSubsystemFactory.java:94)
                at java.base/jdk.internal.platform.CgroupMetrics.getInstance(CgroupMetrics.java:163)
                ... 25 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 4 out of bounds for length 1
                at java.base/jdk.internal.platform.cgroupv2.CgroupV2Subsystem.initSubsystem(CgroupV2Subsystem.java:71)
                at java.base/jdk.internal.platform.cgroupv2.CgroupV2Subsystem.<clinit>(CgroupV2Subsystem.java:42)
                ... 27 more

The reason for this is that CgroupSubsystemFactory incorrectly detects such a system as cgroups v2

$ grep cgroup /proc/self/mountinfo
$ cat /proc/cgroups
#subsys_name    hierarchy       num_cgroups     enabled
cpuset  0       1       1
cpu     0       1       1
cpuacct 0       1       1
memory  0       1       1
devices 0       1       1
freezer 0       1       1
net_cls 0       1       1
blkio   0       1       1
perf_event      0       1       1
Fix Request (OpenJDK 11u): I would like to backport this to 11u. The patch applies cleanly with no changes and the added test passes. At this time, the additional test fix requested by @sgehwolf cannot be added as it requires additional JDK changes to compile.

A pull request was submitted for review. URL: https://git.openjdk.java.net/jdk11u-dev/pull/970 Date: 2022-03-31 13:03:47 +0000

[~stooke] Please only add the approval request label once the patch has been reviewed and has been posted. Also, make sure you add a "Fix Request" comment as per: https://openjdk.java.net/projects/jdk-updates/approval.html

When this gets backported to JDK 11u for the cgroups v2 support, then the test fix included in jdk head commit here: https://github.com/openjdk/jdk/commit/4d6593ce0243457e7431a5990957a8f880e0a3fb should get included as well as JDK-8272124 is already in OpenJDK 11u.

[~neato] Same here? Why the components change? Change affects classes in java.base module. Not hotspot.

URL: https://hg.openjdk.java.net/jdk/jdk/rev/9e359ab51ce6 User: sgehwolf Date: 2020-02-25 19:00:27 +0000

From 'man cgroups': /proc/cgroups (since Linux 2.6.24) This file contains information about the controllers that are compiled into the kernel. An example of the contents of this file (reformatted for readability) is the following: #subsys_name hierarchy num_cgroups enabled cpuset 4 1 1 cpu 8 1 1 cpuacct 8 1 1 blkio 6 1 1 memory 3 1 1 devices 10 84 1 freezer 7 1 1 net_cls 9 1 1 perf_event 5 1 1 net_prio 9 1 1 hugetlb 0 1 0 pids 2 1 1 The fields in this file are, from left to right: 1. The name of the controller. 2. The unique ID of the cgroup hierarchy on which this controller is mounted. If multiple cgroups v1 controllers are bound to the same hierarchy, then each will show the same hierarchy ID in this field. The value in this field will be 0 if: a) the controller is not mounted on a cgroups v1 hierarchy; b) the controller is bound to the cgroups v2 single unified hierarchy; or c) the controller is disabled (see below). The heuristic uses case b) currently in order to figure out if we have cgroups v2. However, this bug is caused by case a), which can be detected by looking at /proc/self/mountinfo. In case of a) no cgroup filesystems are mounted at all and it makes little sense to continue. Even if the factory would detect it as cgroups v1, creating a Metrics instance will fail, thus Metrics == null. Best to fail early in the factory in that case.

RFR: http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-February/064870.html

I've created JDK-8239785 for the hotspot work.

Candidate webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239559/01/webrev/ I need to do some more testing on real systems, but with the given mountinfo/cgroups files I was able to reproduce the issue of detecting the wrong cgroup type in a test.

Apparently reproducible on RHEL 6 too. Affects hotspot similarly with this trace: ./images/jdk/bin/java -Xlog:os+container=trace -version [0.004s][trace][os,container] OSContainer::init: Initializing Container Support [0.005s][trace][os,container] Mount point for cgroupv2 not found in /proc/self/mountinfo openjdk version "" 2020-02-20