JDK-8239559 : Cgroups: Incorrect detection logic on some systems
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2020-02-20
  • Updated: 2023-07-20
  • Resolved: 2020-02-25
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 15 JDK 8 Other
11.0.16Fixed 15 b12Fixed 8u361Fixed openjdk8u372Fixed
Related Reports
Duplicate :  
JDK-8302716 - Cgroups : detection logic broken on systems with v1 controllers
Relates :  
JDK-8231111 - Cgroups v2: Rework Metrics in java.base so as to recognize unified hierarchy
Relates :  
JDK-8245543 - Cgroups: Incorrect detection logic on some systems (still reproducible)
Relates :  
JDK-8258954 - CGroups - invalid detection of cgroupv2
Relates :  
JDK-8284102 - [TESTBUG] [11u] Retroactively add regression test for JDK-8272124
Relates :  
JDK-8239785 - Cgroups: Incorrect detection logic on old systems in hotspot
Relates :  
JDK-8302716 - Cgroups : detection logic broken on systems with v1 controllers
Description
After JDK-8231111 there are some reports of failures on some cgroups v1 systems (e.g. SLES 11/12). In particular, systems which support cgroups v1, but don't have controllers mounted on a cgroups v1 hierarchy.

Failure looks like:
java.lang.InternalError: java.lang.reflect.InvocationTargetException
                at java.base/jdk.internal.platform.Metrics.systemMetrics(Metrics.java:65)
                at java.base/jdk.internal.platform.Container.metrics(Container.java:43)
                at jdk.management/com.sun.management.internal.OperatingSystemImpl.<init>(OperatingSystemImpl.java:48)
                at jdk.management/com.sun.management.internal.PlatformMBeanProviderImpl.getOperatingSystemMXBean(PlatformMBeanProviderImpl.java:281)
                at jdk.management/com.sun.management.internal.PlatformMBeanProviderImpl$3.nameToMBeanMap(PlatformMBeanProviderImpl.java:198)
                at java.management/sun.management.spi.PlatformMBeanProvider$PlatformComponent.getMBeans(PlatformMBeanProvider.java:195)
                at java.management/java.lang.management.ManagementFactory.getPlatformMXBean(ManagementFactory.java:686)
                at java.management/java.lang.management.ManagementFactory.getOperatingSystemMXBean(ManagementFactory.java:388)
                at com.sun.javatest.regtest.config.OS.<init>(OS.java:158)
                at com.sun.javatest.regtest.config.OS.current(OS.java:59)
                at com.sun.javatest.regtest.config.RegressionContext.<init>(RegressionContext.java:62)
                at com.sun.javatest.regtest.config.RegressionContext.getDefault(RegressionContext.java:45)
                at com.sun.javatest.regtest.config.RegressionTestFinder.<init>(RegressionTestFinder.java:93)
                at com.sun.javatest.regtest.config.RegressionTestSuite.createTestFinder(RegressionTestSuite.java:100)
                at com.sun.javatest.regtest.config.RegressionTestSuite.<init>(RegressionTestSuite.java:82)
                at com.sun.javatest.regtest.config.RegressionTestSuite.open(RegressionTestSuite.java:65)
                at com.sun.javatest.regtest.config.TestManager.getTestSuites(TestManager.java:164)
                at com.sun.javatest.regtest.tool.Tool.run(Tool.java:1076)
                at com.sun.javatest.regtest.tool.Tool.run(Tool.java:1027)
                at com.sun.javatest.regtest.tool.Tool.main(Tool.java:143)
                at com.sun.javatest.regtest.Main.main(Main.java:58)
Caused by: java.lang.reflect.InvocationTargetException
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.base/java.lang.reflect.Method.invoke(Method.java:564)
                at java.base/jdk.internal.platform.Metrics.systemMetrics(Metrics.java:61)
                ... 20 more
Caused by: java.lang.ExceptionInInitializerError
                at java.base/jdk.internal.platform.CgroupSubsystemFactory.create(CgroupSubsystemFactory.java:94)
                at java.base/jdk.internal.platform.CgroupMetrics.getInstance(CgroupMetrics.java:163)
                ... 25 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 4 out of bounds for length 1
                at java.base/jdk.internal.platform.cgroupv2.CgroupV2Subsystem.initSubsystem(CgroupV2Subsystem.java:71)
                at java.base/jdk.internal.platform.cgroupv2.CgroupV2Subsystem.<clinit>(CgroupV2Subsystem.java:42)
                ... 27 more

The reason for this is that CgroupSubsystemFactory incorrectly detects such a system as cgroups v2

Note:
$ grep cgroup /proc/self/mountinfo
<nothing>
$ cat /proc/cgroups
#subsys_name    hierarchy       num_cgroups     enabled
cpuset  0       1       1
cpu     0       1       1
cpuacct 0       1       1
memory  0       1       1
devices 0       1       1
freezer 0       1       1
net_cls 0       1       1
blkio   0       1       1
perf_event      0       1       1
Comments
Fix Request (8u) Please consider this for 8u as part of cgroups v2 support. It wasn't clean: adjustments needed for Optional.isEmpty not present in 8u. Thanks
13-12-2022
A pull request was submitted for review. URL: https://git.openjdk.org/jdk8u-dev/pull/136 Date: 2022-10-13 09:34:05 +0000
13-10-2022
Fix Request (OpenJDK 11u): I would like to backport this to 11u. The patch applies cleanly with no changes and the added test passes. At this time, the additional test fix requested by @sgehwolf cannot be added as it requires additional JDK changes to compile.
31-03-2022
A pull request was submitted for review. URL: https://git.openjdk.java.net/jdk11u-dev/pull/970 Date: 2022-03-31 13:03:47 +0000
31-03-2022
[~stooke] Please only add the approval request label once the patch has been reviewed and has been posted. Also, make sure you add a "Fix Request" comment as per: https://openjdk.java.net/projects/jdk-updates/approval.html
31-03-2022
When this gets backported to JDK 11u for the cgroups v2 support, then the test fix included in jdk head commit here: https://github.com/openjdk/jdk/commit/4d6593ce0243457e7431a5990957a8f880e0a3fb should get included as well as JDK-8272124 is already in OpenJDK 11u.
22-03-2022
[~neato] Same here? Why the components change? Change affects classes in java.base module. Not hotspot.
09-08-2021
URL: https://hg.openjdk.java.net/jdk/jdk/rev/9e359ab51ce6 User: sgehwolf Date: 2020-02-25 19:00:27 +0000
25-02-2020
From 'man cgroups': /proc/cgroups (since Linux 2.6.24) This file contains information about the controllers that are compiled into the kernel. An example of the contents of this file (reformatted for readability) is the following: #subsys_name hierarchy num_cgroups enabled cpuset 4 1 1 cpu 8 1 1 cpuacct 8 1 1 blkio 6 1 1 memory 3 1 1 devices 10 84 1 freezer 7 1 1 net_cls 9 1 1 perf_event 5 1 1 net_prio 9 1 1 hugetlb 0 1 0 pids 2 1 1 The fields in this file are, from left to right: 1. The name of the controller. 2. The unique ID of the cgroup hierarchy on which this controller is mounted. If multiple cgroups v1 controllers are bound to the same hierarchy, then each will show the same hierarchy ID in this field. The value in this field will be 0 if: a) the controller is not mounted on a cgroups v1 hierarchy; b) the controller is bound to the cgroups v2 single unified hierarchy; or c) the controller is disabled (see below). The heuristic uses case b) currently in order to figure out if we have cgroups v2. However, this bug is caused by case a), which can be detected by looking at /proc/self/mountinfo. In case of a) no cgroup filesystems are mounted at all and it makes little sense to continue. Even if the factory would detect it as cgroups v1, creating a Metrics instance will fail, thus Metrics == null. Best to fail early in the factory in that case.
24-02-2020
RFR: http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-February/064870.html
21-02-2020
I've created JDK-8239785 for the hotspot work.
21-02-2020
Candidate webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239559/01/webrev/ I need to do some more testing on real systems, but with the given mountinfo/cgroups files I was able to reproduce the issue of detecting the wrong cgroup type in a test.
20-02-2020
Apparently reproducible on RHEL 6 too. Affects hotspot similarly with this trace: ./images/jdk/bin/java -Xlog:os+container=trace -version [0.004s][trace][os,container] OSContainer::init: Initializing Container Support [0.005s][trace][os,container] Mount point for cgroupv2 not found in /proc/self/mountinfo openjdk version "15.0.0.1-internal" 2020-02-20
20-02-2020