JDK-8170888 : [linux] Experimental support for cgroup memory limits in container (ie Docker) environments
  • Type: Enhancement
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 8,9
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • Submitted: 2016-12-08
  • Updated: 2018-02-08
  • Resolved: 2016-12-13
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8 JDK 9 Other
8u121Fixed 9 b150Fixed openjdk7uFixed
Related Reports
Relates :  
JDK-8146115 - Improve docker container detection and resource configuration usage
Relates :  
JMC-5522 - Rule for memory settings when running in a container
Description
We have an open RFE targeted to JDK 10 to get the JVM to play nicely with memory limits imposed by a Docker container (through the Linux cgroups facility): JDK-8146115

There has been a non-Oracle submission to get some basic support for this into 9:

http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-November/025404.html

then:

http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-December/025406.html

It is proposed for this RFE that we add an experimental VM option, such as -XX:+UseCGroupMemoryLimitForHeap, to opt-in to using the value in /sys/fs/cgroup/memory/memory.limit_in_bytes as the value for phys_mem as suggested in the submission.

This will allow us to get quick feedback from the community and customers as to how things work, and allow us to shape the full RFE for JDK 10.
Comments
Okay - a request for enhancement backport approval for 8u has been sent to jdk8u-dev@openjdk.java.net
19-12-2016
First of all that's cumbersome, and secondly it will blow up for naive users if there is no memory.limit_in_bytes file. This fix is very localized and will not hurt anyone and will save our customer's some headaches.
16-12-2016
In looking at the backport to 8u I realized that this effort served very little purpose. The user can just as easily specify: -XX:MaxRAM=`cat /sys/fs/cgroup/memory/memory.limit_in_bytes` to get the same results.
14-12-2016
Example logging output when running unlimited: ./jdk-9/fastdebug/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Xlog:gc+heap -version [0.002s][info][gc,heap] Setting phys_mem to the min of cgroup limit (8796093022207MB) and initial phys_mem (64353MB) [0.016s][info][gc,heap] Heap region size: 4M java version "9-internal" Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2) Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode)
09-12-2016
Example of enabling the flag on a system with no cgroups: > ./build/linux-x64-debug/images/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -version Java HotSpot(TM) 64-Bit Server VM warning: Unable to open cgroup memory limit file /sys/fs/cgroup/memory/memory.limit_in_bytes (No such file or directory) java version "9-internal" Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2) Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode)
09-12-2016
Example of logging output: root@e3f0f811cd91:/# ./jdk-9/fastdebug/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Xlog:gc+heap -version [0.003s][info][gc,heap] Setting phys_mem to the min of cgroup limit (128MB) and initial phys_mem (64353MB) [0.016s][info][gc,heap] Heap region size: 1M java version "9-internal" Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2) Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode)
09-12-2016
Here's an example run in a container with memory limited to 256M. First without using the flag: root@2f2664c92737:/# ./jdk-9/fastdebug/bin/java -XX:+PrintFlagsFinal -version | grep MaxHeap uintx MaxHeapFreeRatio = 70 {manageable} {default} size_t MaxHeapSize = 16873684992 {product} {ergonomic} java version "9-internal" Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2) Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode) And then with: root@2f2664c92737:/# ./jdk-9/fastdebug/bin/java -XX:+PrintFlagsFinal -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -version | grep MaxHeap uintx MaxHeapFreeRatio = 70 {manageable} {default} size_t MaxHeapSize = 132120576 {product} {ergonomic} java version "9-internal" Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2) Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode) so we can see the affect on the heap size.
09-12-2016
FC Extension Request ---------------------------- As noted we had planned a full RFE for 10 when we had more time to fully explore working with cgroup limits. But there is growing demand for the JVM to be able to function within Docker containers that impose memory limits, today. By providing an experimental option to allow this interaction we can address this current request and also gather feedback and information to help shape the full RFE for 10. As an opt-in experimental flag there is zero risk for anyone not running with the flag set. Testing of this flag will be manual and ad-hoc, there is no proposal to include this as a testing mode for an existing test processes.
08-12-2016