JDK-8239559 : Cgroups: Incorrect detection logic on some systems
  • Type: Bug
  • Component: core-libs
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2020-02-20
  • Updated: 2020-12-21
  • Resolved: 2020-02-25
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 15
15 b12Fixed
Related Reports
Relates :  
Relates :  
Relates :  
Description
After JDK-8231111 there are some reports of failures on some cgroups v1 systems (e.g. SLES 11/12). In particular, systems which support cgroups v1, but don't have controllers mounted on a cgroups v1 hierarchy.

Failure looks like:
java.lang.InternalError: java.lang.reflect.InvocationTargetException
                at java.base/jdk.internal.platform.Metrics.systemMetrics(Metrics.java:65)
                at java.base/jdk.internal.platform.Container.metrics(Container.java:43)
                at jdk.management/com.sun.management.internal.OperatingSystemImpl.<init>(OperatingSystemImpl.java:48)
                at jdk.management/com.sun.management.internal.PlatformMBeanProviderImpl.getOperatingSystemMXBean(PlatformMBeanProviderImpl.java:281)
                at jdk.management/com.sun.management.internal.PlatformMBeanProviderImpl$3.nameToMBeanMap(PlatformMBeanProviderImpl.java:198)
                at java.management/sun.management.spi.PlatformMBeanProvider$PlatformComponent.getMBeans(PlatformMBeanProvider.java:195)
                at java.management/java.lang.management.ManagementFactory.getPlatformMXBean(ManagementFactory.java:686)
                at java.management/java.lang.management.ManagementFactory.getOperatingSystemMXBean(ManagementFactory.java:388)
                at com.sun.javatest.regtest.config.OS.<init>(OS.java:158)
                at com.sun.javatest.regtest.config.OS.current(OS.java:59)
                at com.sun.javatest.regtest.config.RegressionContext.<init>(RegressionContext.java:62)
                at com.sun.javatest.regtest.config.RegressionContext.getDefault(RegressionContext.java:45)
                at com.sun.javatest.regtest.config.RegressionTestFinder.<init>(RegressionTestFinder.java:93)
                at com.sun.javatest.regtest.config.RegressionTestSuite.createTestFinder(RegressionTestSuite.java:100)
                at com.sun.javatest.regtest.config.RegressionTestSuite.<init>(RegressionTestSuite.java:82)
                at com.sun.javatest.regtest.config.RegressionTestSuite.open(RegressionTestSuite.java:65)
                at com.sun.javatest.regtest.config.TestManager.getTestSuites(TestManager.java:164)
                at com.sun.javatest.regtest.tool.Tool.run(Tool.java:1076)
                at com.sun.javatest.regtest.tool.Tool.run(Tool.java:1027)
                at com.sun.javatest.regtest.tool.Tool.main(Tool.java:143)
                at com.sun.javatest.regtest.Main.main(Main.java:58)
Caused by: java.lang.reflect.InvocationTargetException
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.base/java.lang.reflect.Method.invoke(Method.java:564)
                at java.base/jdk.internal.platform.Metrics.systemMetrics(Metrics.java:61)
                ... 20 more
Caused by: java.lang.ExceptionInInitializerError
                at java.base/jdk.internal.platform.CgroupSubsystemFactory.create(CgroupSubsystemFactory.java:94)
                at java.base/jdk.internal.platform.CgroupMetrics.getInstance(CgroupMetrics.java:163)
                ... 25 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 4 out of bounds for length 1
                at java.base/jdk.internal.platform.cgroupv2.CgroupV2Subsystem.initSubsystem(CgroupV2Subsystem.java:71)
                at java.base/jdk.internal.platform.cgroupv2.CgroupV2Subsystem.<clinit>(CgroupV2Subsystem.java:42)
                ... 27 more

The reason for this is that CgroupSubsystemFactory incorrectly detects such a system as cgroups v2

Note:
$ grep cgroup /proc/self/mountinfo
<nothing>
$ cat /proc/cgroups
#subsys_name    hierarchy       num_cgroups     enabled
cpuset  0       1       1
cpu     0       1       1
cpuacct 0       1       1
memory  0       1       1
devices 0       1       1
freezer 0       1       1
net_cls 0       1       1
blkio   0       1       1
perf_event      0       1       1
Comments
URL: https://hg.openjdk.java.net/jdk/jdk/rev/9e359ab51ce6 User: sgehwolf Date: 2020-02-25 19:00:27 +0000
25-02-2020

From 'man cgroups': /proc/cgroups (since Linux 2.6.24) This file contains information about the controllers that are compiled into the kernel. An example of the contents of this file (reformatted for readability) is the following: #subsys_name hierarchy num_cgroups enabled cpuset 4 1 1 cpu 8 1 1 cpuacct 8 1 1 blkio 6 1 1 memory 3 1 1 devices 10 84 1 freezer 7 1 1 net_cls 9 1 1 perf_event 5 1 1 net_prio 9 1 1 hugetlb 0 1 0 pids 2 1 1 The fields in this file are, from left to right: 1. The name of the controller. 2. The unique ID of the cgroup hierarchy on which this controller is mounted. If multiple cgroups v1 controllers are bound to the same hierarchy, then each will show the same hierarchy ID in this field. The value in this field will be 0 if: a) the controller is not mounted on a cgroups v1 hierarchy; b) the controller is bound to the cgroups v2 single unified hierarchy; or c) the controller is disabled (see below). The heuristic uses case b) currently in order to figure out if we have cgroups v2. However, this bug is caused by case a), which can be detected by looking at /proc/self/mountinfo. In case of a) no cgroup filesystems are mounted at all and it makes little sense to continue. Even if the factory would detect it as cgroups v1, creating a Metrics instance will fail, thus Metrics == null. Best to fail early in the factory in that case.
24-02-2020

RFR: http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-February/064870.html
21-02-2020

I've created JDK-8239785 for the hotspot work.
21-02-2020

Candidate webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239559/01/webrev/ I need to do some more testing on real systems, but with the given mountinfo/cgroups files I was able to reproduce the issue of detecting the wrong cgroup type in a test.
20-02-2020

Apparently reproducible on RHEL 6 too. Affects hotspot similarly with this trace: ./images/jdk/bin/java -Xlog:os+container=trace -version [0.004s][trace][os,container] OSContainer::init: Initializing Container Support [0.005s][trace][os,container] Mount point for cgroupv2 not found in /proc/self/mountinfo openjdk version "15.0.0.1-internal" 2020-02-20
20-02-2020