JDK-8222043 : [TESTBUG] Create JCMD tests for Docker and JFR
  • Type: Enhancement
  • Component: hotspot
  • Sub-Component: jfr
  • Affected Version: 13
  • Priority: P3
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2019-04-05
  • Updated: 2021-12-23
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbdUnresolved
Related Reports
Blocks :  
JDK-8228343 - JCMD and attach fail to work across Linux Container boundary
Relates :  
JDK-8193710 - jcmd -l and jps commands do not list Java processes running in Docker containers
Relates :  
JDK-8227122 - [TESTBUG] Create Docker sidecar test cases
Relates :  
JDK-8221711 - [TESTBUG] create more tests for JFR in container environment
Relates :  
JDK-8195809 - [TESTBUG] jps and jcmd -l support for containers is not tested
Description
A common scenario where a Monitored VM runs inside a Docker container, while the observer tool/process is on a host system. The observer should be able to control JFR recording via JCMD (JFR.start/stop/check/dump).

 Important notes:
 -- jcmd -l (list) running outside of the container (on a host system) does not seem to work (investigation in progress)
 -- use 'ps' command to discover the observed JVM process
 -- in order for JCMD and attach (com.sun.tools.attach.VirtualMachine.attach) to work, the user ID of the monitored VM should match or be accessible by the user of the observer tool/process; for that, use the following:
     - run "id" on the host system, record user ID of the 'test' user (aka observer)
     - when starting docker, use --user=<UID> (E.g.: docker run --user=123456)
Comments
[~bobv] Bob, thank you for pointing out the "JDK-8193710". As I understand, with that issue fixed, the "jcmd -l" and "jps" should be able to see and list JVM processes running across the container boundary, if the jcmd/jps is launched from the host system. I understand that the username and UID should match between the user running jcmd/jps and the JVM under observation. In my prototype I matched the user (not in the attached tests, but my prototype in progress); I also ran containers with --cap-add=SYS_PTRACE, but it still does not work for me. I will continue to investigate, perhaps I missed something in my test setup. If it still does not work, I will attach the updated test case and probably file a bug. Let me know if you have any suggestions for me.
26-04-2019
The comment in the description is incorrect: " -- jcmd -l (list) running outside of the container (on a host system) will not list JVM processes running inside the system (a known fact, by design, see JDK-8179498) " I resolved that issue with this change (https://bugs.openjdk.java.net/browse/JDK-8193710) I vaguely recall that I had some issues with userids. The user ids of the host had to match the userids inside the container in order to see Java processes inside of the container.
26-04-2019
I have tried a variety of options, still can not use jcmd. Options I tried: --cap-add=SYS_PTRACE --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --privileged (use with care !!!) In addition to options above, I also tried to expose the port (1099), and tried to map tmpfs to /tmp on the host system: -p 1099:1099 \ --tmpfs /tmp \ Also, tried --net=host to map all host network to the container, did not help. Oh, and just to be sure, I ran container with default user (sudo), and from host system used "sudo jcmd <PID> help". I also used my own user in both cases, matching UID. ================== I keep getting the same error: com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file /proc/<PID>/root/tmp/.java_pidXX: target process <PID> doesn't respond within 10500ms or HotSpot VM not loaded at jdk.attach/sun.tools.attach.VirtualMachineImpl.<init>(VirtualMachineImpl.java:100) at jdk.attach/sun.tools.attach.AttachProviderImpl.attachVirtualMachine(AttachProviderImpl.java:58) at jdk.attach/com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:207) at jdk.jcmd/sun.tools.jcmd.JCmd.executeCommandForPid(JCmd.java:114) at jdk.jcmd/sun.tools.jcmd.JCmd.main(JCmd.java:98)
17-04-2019
The above could happen possibly because the PTRACE capability is not enabled by default. Should try this: --cap-add=SYS_PTRACE
16-04-2019
Attached the initial prototype; work in progress.
05-04-2019