JDK-8226575 : OperatingSystemMXBean should be made container aware
  • Type: Enhancement
  • Component: core-svc
  • Sub-Component: java.lang.management
  • Affected Version: 8,11,13,14
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2019-06-21
  • Updated: 2024-06-04
  • Resolved: 2019-12-12
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 13 JDK 14 JDK 15 JDK 8 Other
11.0.9Fixed 13-poolResolved 14 b27Fixed 15Fixed 8u261Fixed openjdk8u272Fixed
Related Reports
CSR :  
JDK-8228428 - OperatingSystemMXBean should be made container aware
Duplicate :  
JDK-8211848 - OperatingSystemMXBean should be made Container Aware
Relates :  
JDK-8250627 - Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics
Relates :  
JDK-8231111 - Cgroups v2: Rework Metrics in java.base so as to recognize unified hierarchy
Relates :  
JDK-8258241 - [8u] Missing doPrivileged() hunks from JDK-8226575
Relates :  
JDK-8242480 - Negative value may be returned by getFreeSwapSpaceSize() in the docker
Relates :  
JDK-8236617 - jtreg test containers/docker/TestMemoryAwareness.java fails after 8226575
Relates :  
JDK-8199944 - Add Container MBean to JMX
Relates :  
JDK-8242430 - Correct links in javadoc of OperatingSystemMXBean
Relates :  
JDK-8250984 - Memory Docker tests fail on some Linux kernels w/o cgroupv1 swap limit capabilities
Relates :  
JDK-8255934 - JConsole 14 and greater fails to connect to older JVM
Relates :  
JDK-8203357 - Container Metrics
Relates :  
JDK-8235681 - Remove unnecessary workarounds in UnixOperatingSystem.c
Sub Tasks
JDK-8236876 :  
Release Note: OperatingSystemMXBean Methods Inside a Container Return Container Specific Data - Closed
Description
When Java is running in a container many of the OperatingSystemMXBean accessor methods return host based information rather than container specific data.

jdk/open/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java contains these native methods which extract Host specific data.  These methods should be examine to determine if there is a container specific source of this information.

    private native long getCommittedVirtualMemorySize0();
    private native long getFreePhysicalMemorySize0();
    private native long getFreeSwapSpaceSize0();
    private native long getMaxFileDescriptorCount0();
    private native long getOpenFileDescriptorCount0();
    private native long getProcessCpuTime0();
    private native double getProcessCpuLoad0();
    private native double getSystemCpuLoad0();
    private native long getTotalPhysicalMemorySize0();
    private native long getTotalSwapSpaceSize0();

These Java methods should first try to extract this information from the internal container Metrics API, if available.  Otherwise it should call these existing native methods.

Here are the Container Metrics implementation classes:

jdk/open/src/java.base/share/classes/jdk/internal/platform/Container.java
jdk/open/src/java.base/share/classes/jdk/internal/platform/Metrics.java
Comments
Fix request (13u) Requesting backport to 13u for parity with 11u. The patch applies cleanly, but it requires some modifications. The original patch added new methods to OperatingSystemMXBean according to CSR JDK-8228428. Backporting them to 13u is not suitable. Similar to 11u/8u, existing methods have been changed to return the container values instead. CSR for 13u is filed and approved: JDK-8255834, it's similar to 11u/8u CSR JDK-8248804. Tested with tier1 and container tests on Linux and Windows, the only failure is containers/docker/TestMemoryAwareness.java which is fixed by follow-up JDK-8236617, the plan is to push them together. RFR approval: http://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-November/004248.html
26-11-2020
Fix Request (OpenJDK 8u): Please approve backporting this to OpenJDK 8u. The JDK 11 patch doesn't apply cleanly as tests are split between hotspot and jdk. What's more the patch from JDK 14 introduced new methods on OperatingSystemMXBean which isn't suitable for a backport to an LTS release. Instead, existing methods have been changed to return the container limit over the host system. This has been clarified in the javadoc and a CSR has been filed and got approved. The patch got reviewed by Andrew Hughes. I intend to follow up with a backport of JDK-8236617 so as to not break testing (on some systems). Risk should be fairly low since it only affects Linux and then only when run in a container. In cases such as wrongly detected container one can use -XX:-UseContainerSupport to turn the feature off (see JDK-8250627). Testing: Included container tests on Linux x86_64, tier 1 tests. Builds on various platforms. webrev: jdk: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8226575/jdk8/02/webrev/ hotspot: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8226575/jdk8/01/hotspot/webrev/ CSR: https://bugs.openjdk.java.net/browse/JDK-8248804 RFR: http://mail.openjdk.java.net/pipermail/jdk8u-dev/2020-July/012178.html Approval: http://mail.openjdk.java.net/pipermail/jdk8u-dev/2020-August/012542.html
25-08-2020
Fix Request (OpenJDK 11u): Please approve backporting this to OpenJDK 11u. The plan is to also backport this to OpenJDK 8u (for Oracle JDK 8 parity) and it would be bad if we'd regress on the feature front in later JDKs. Thus, requesting a backport here too. The patch doesn't apply cleanly. What's more the patch from JDK 14 introduced new methods on OperatingSystemMXBean which isn't suitable for a backport to an LTS release. Instead, existing methods have been changed to return the container limit over the host system. This has been clarified in the javadoc and a CSR has been filed and got approved. The patch got reviewed by Andrew Dinn. I intend to push this with JDK-8236617 so as to not break testing. Risk should be fairly low since it only affects Linux and then only when run in a container. Testing: Included container tests on Linux x86_64. webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8226575/jdk11/02/webrev/ Corresponding CSR: https://bugs.openjdk.java.net/browse/JDK-8248804 RFR: http://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-July/003495.html
17-07-2020
URL: https://hg.openjdk.java.net/jdk/jdk/rev/899254bd233b User: dtitov Date: 2019-12-12 03:22:51 +0000
12-12-2019
Please see the CSR request - JDK-8228428 - for how this change evolved.
22-11-2019
Updated comment as there are two OperatingSystemMXBean interfaces: - java.lang.management.OperatingSystemMXBean - com.sun.management.OperatingSystemMXBean (a subtype of the former) What the methods of these two interface report depends on exactly how the method has been specified. If it refers to something "available to the JVM (or current process)" then it needs to be container aware. If it refers to the "whole system" or something "physical", then that should report information for the whole system. Arguably some of the whole-system/physical APIs are misguided and reflect a history where containers and virtualized environments were not envisaged. But changing them is a compatability issue and so it may be better to add new methods that report more interesting/relevant information for the process's environment. Either way a CSR request will need to be filed and the changes examined from a compatibility perspective. Some of the APIs may not be "well formed" in the first place - for example getSystemLoadAverage(): "Returns the system load average for the last minute. The system load average is the sum of the number of runnable entities queued to the available processors and the number of runnable entities running on the available processors averaged over a period of time. " This refers to the "system" but specifies the calculation is based on "available processors" - which is a contradiction as "available" != "system".
18-07-2019
The spec for java.lang.management.OperatingSystemMXBean::getAvailableProcessors: Returns the number of processors available to the Java virtual machine. This method is equivalent to the Runtime.availableProcessors() method. com.sun.management.OperatingSystemMXBean API provides the metrics for the operating system on which the Java virtual machine is running. I believe this issue may require spec clarification or update.
21-06-2019
Originally reported here: http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-June/028439.html
21-06-2019
Assigning to me on behalf of Andrew Azores. He'll be working on getting this fixed.
21-06-2019