JDK-8296671 : [JFR] jdk.ContainerConfiguration event should include host total memory
  • Type: Enhancement
  • Component: hotspot
  • Sub-Component: jfr
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • CPU: generic
  • Submitted: 2022-11-09
  • Updated: 2024-02-01
  • Resolved: 2022-11-24
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 20
20 b26Fixed
Related Reports
Relates :  
Relates :  
Description
The documentation of the jdk.PhysicalMemory event says:

/**
 *  PhysicalMemory event represents:
 *
 *  @totalSize == The amount of physical memory (hw) installed and reported by the OS, in bytes.
 *  @usedSize  == The amount of physical memory currently in use in the system (reserved/committed), in bytes.
 *
 *  Both fields are systemwide, i.e. represents the entire OS/HW environment.
 *  These fields do not include virtual memory.
 *
 *  If running inside a guest OS on top of a hypervisor in a virtualized environment,
 *  the total memory reported is the amount of memory configured for the guest OS by the hypervisor.
 */

Yet, when run in a container, it reports the container memory rather than the host value. Given that jdk.ContainerConfiguration reports the container values, it would make sense to report the memory of the physical host.

Reproducer:
$ git clone https://github.com/Karm/mandrel-integration-tests.git
$ cd mandrel-integration-tests/apps/debug-symbols-smoke
$ mvn package
$ sudo podman run --memory=1G --memory-swap=1G --rm -ti -v /disk/openjdk/upstream-sources/git/jdk-jdk/build/linux-x86_64-server-fastdebug/images/jdk:/opt/jdk:z -v $(pwd)/target:/opt/target:z fedora:37
[root@4b003616eab8 /]# /opt/jdk/bin/java -XX:StartFlightRecording=filename=flight-java.jfr -Xlog:jfr -jar /opt/target/debug-symbols-smoke.jar
[1.030s][info][jfr] Flight Recorder initialized
[1.030s][info][jfr] Repository base directory: /tmp
[1.259s][info][jfr] Started recording "1" (1) {dumponexit=true, filename=/flight-java.jfr}
[1.260s][info][jfr,startup] Started recording 1. No limit specified, using maxsize=250MB as default.
[1.260s][info][jfr,startup] 
[1.260s][info][jfr,startup] Use jcmd 26 JFR.dump name=1 to copy recording data to file.
Q to quit
Q
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
[3.585s][info][jfr        ] Stopped recording "1" (1). Reason "Dump on exit".
[3.671s][info][jfr        ] Transferred 222333 bytes from the disk repository
[3.671s][info][jfr        ] Wrote recording "1" (1) to /flight-java.jfr
[3.672s][info][jfr        ] Closed recording "1" (1)
[3.681s][info][jfr        ] Removed repository /tmp/2022_11_09_13_53_17_26
[root@4b003616eab8 /]# /opt/jdk/bin/jfr print --events jdk.ContainerConfiguration flight-java.jfr
jdk.ContainerConfiguration {
  startTime = 13:53:17.658 (2022-11-09)
  containerType = "cgroupv1"
  cpuSlicePeriod = 100 ms
  cpuQuota = -0.00100 ms
  cpuShares = -1
  effectiveCpuCount = 12
  memorySoftLimit = -1 byte
  memoryLimit = 1.0 GB
  swapMemoryLimit = 1.0 GB
  eventThread = "main" (javaThreadId = 1)
}

[root@4b003616eab8 /]# /opt/jdk/bin/jfr print --events jdk.PhysicalMemory flight-java.jfr
jdk.PhysicalMemory {
  startTime = 13:53:17.648 (2022-11-09)
  totalSize = 1.0 GB
  usedSize = 130.3 MB
}

jdk.PhysicalMemory {
  startTime = 13:53:19.991 (2022-11-09)
  totalSize = 1.0 GB
  usedSize = 171.8 MB
}

Unfortunately, with this behaviour it's not easy to know the (container) hosts' total memory which is useful for some monitoring use-cases. Note that hotspot container trace logging (`os+container`) shows this info after JDK-8292083:

[root@4b003616eab8 /]# /opt/jdk/bin/java -Xlog:os+container=trace -version
[0.007s][trace][os,container] OSContainer::init: Initializing Container Support
[0.007s][debug][os,container] Detected optional pids controller entry in /proc/cgroups
[0.007s][debug][os,container] Detected cgroups hybrid or legacy hierarchy, using cgroups v1 controllers
[0.007s][trace][os,container] Path to /memory.use_hierarchy is /sys/fs/cgroup/memory/memory.use_hierarchy
[0.008s][trace][os,container] Use Hierarchy is: 1
[0.010s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
[0.010s][trace][os,container] CPU Quota is: -1
[0.010s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
[0.010s][trace][os,container] CPU Period is: 100000
[0.010s][trace][os,container] OSContainer::active_processor_count: 12
[0.010s][trace][os,container] CgroupSubsystem::active_processor_count (cached): 12
[0.010s][trace][os,container] total physical memory: 67174506496
[0.010s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
[0.010s][trace][os,container] Memory Limit is: 1073741824
[0.028s][trace][os,container] CgroupSubsystem::active_processor_count (cached): 12
[0.050s][trace][os,container] total physical memory: 67174506496
[0.050s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
[0.050s][trace][os,container] Memory Limit is: 1073741824
[0.155s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
[0.155s][trace][os,container] CPU Quota is: -1
[0.155s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
[0.155s][trace][os,container] CPU Period is: 100000
[0.155s][trace][os,container] OSContainer::active_processor_count: 12
openjdk version "20-internal" 2023-03-21
OpenJDK Runtime Environment (fastdebug build 20-internal-adhoc.sgehwolf.jdk-jdk)
OpenJDK 64-Bit Server VM (fastdebug build 20-internal-adhoc.sgehwolf.jdk-jdk, mixed mode, sharing)

Edit:
After some discussion https://mail.openjdk.org/pipermail/hotspot-jfr-dev/2022-November/004360.html it seems this is expected behaviour. But since there is no way to know the host total memory via JFR the proposal is to amend the jdk.ContainerConfiguration event with host total memory field.
Comments
Changeset: 3c4d5204 Author: Severin Gehwolf <sgehwolf@openjdk.org> Date: 2022-11-24 10:05:18 +0000 URL: https://git.openjdk.org/jdk/commit/3c4d5204ff96280b123f42a8cfbaef308e470b69
24-11-2022

A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/11143 Date: 2022-11-14 20:19:29 +0000
14-11-2022

os::Linux::physical_memory() would be the API to use on Linux. os::physical_memory() returns the container memory.
09-11-2022