JDK-8339877 : Enhance JCmd by adding a new command to summarize target r/t metadata
  • Type: CSR
  • Component: core-svc
  • Priority: P4
  • Status: Provisional
  • Resolution: Unresolved
  • Fix Versions: 24
  • Submitted: 2024-09-10
  • Updated: 2024-11-19
Related Reports
CSR :  
JDK-8336684 - Enhance JCmd by adding a new command to summarize target r/t metadata
Description
Summary
-------

Add `jcmd <pid> VM.usage_metadata` to print metadata describing the JVM invocation.


Problem
-------

Many monitoring and observability agent(s) (and other such tools) want to obtain metadata from JVMs that describes how they were configured and invoked, e.g:

- JVM version class and/or module path 
- JAVA_HOME
- any JVM flags specified
- any JVM arguments specified
- program arguments 
- JVM start time 
- JVM uptime 
- if executing in the context of a "resource manager" (Docker etc)
- ...

in order to obtain such, these agent(s) must currently invoke multiple `jcmd`'s, if such exists.

This is not optimal for a variety of reasons.

Solution
--------

Add a new command that the `jcmd` command line tool can use to print metadata about JVM usage. 

The output is formatted (either in plain text or as JSON) as a list of key,value tuples.

The command effectively treats the individual metadata "fields" as `System` property names, and where none exist for particular metadata (such as JVM uptime) a "synthetic" property (name) is used in order to uniquely name such metadata.


Here is example (default) output (the output is JSON encoded to enable programmatic parsing).


[Note: fields are shown on a separate line for clarity, the implementation does not reflect this in order to facilitate log append style of collection]

```
135986:
{ 
  "timestamp" : "2024-09-10T12:06:02.953-0700", 
  "jvm.starttime" : "2024-09-10T19:05:41.976+0000", 
  "java.home" : "/home/auser/src/git/jdk/open/build/linux-x86_64-server-release/jdk", 
  "java.version" : "24-internal", 
  "java.vm.version" : "24-internal-adhoc.lpgc.open", 
  "sun.java.launcher" : "SUN_STANDARD", 
  "jvm.args" : [ "-Dapplication.home=/home/auser/src/git/jdk/open/build/linux-x86_64-server-release/jdk", "-Xms8m", "-Djdk.module.main=jdk.jshell" ], 
  "sun.java.command" : "jdk.jshell/jdk.internal.jshell.tool.JShellToolProvider",
  "java.class.path" : "", 
  "jvm.pid" : "135986", 
  "jvm.uptime.ms" : "21123", 
  "os.name" : "Linux",
  "os.version" : "4.18.0-553.16.1.el8_10.x86_64", 
  "os.arch" : "amd64" 
  "os.hostname" : "ahost"
}

```

an example of plain text is:

```
135986:
"timestamp"="2024-09-10T12:26:17.478-0700", 
"jvm.starttime"="2024-09-10T19:05:41.976+0000",
"java.home"="/home/auser/src/git/jdk/open/build/linux-x86_64-server-release/jdk", 
"java.version"="24-internal", 
"java.vm.version"="24-internal-adhoc.auser.open", 
"sun.java.launcher"="SUN_STANDARD", 
"jvm.args"="'-Dapplication.home=/home/auser/src/git/jdk/open/build/linux-x86_64-server-release/jdk', '-Xms8m', '-Djdk.module.main=jdk.jshell'", 
"sun.java.command"="jdk.jshell/jdk.internal.jshell.tool.JShellToolProvider", 
"java.class.path"="", 
"jvm.pid"="135986", 
"jvm.uptime.ms"="1235648", 
"os.name"="Linux", 
"os.version"="4.18.0-553.16.1.el8_10.x86_64", 
"os.arch"="amd64"
"os.hostname"="ahost"
```

Specification
-------------

The man page `jcmd(1)` describes this new `jcmd` as:

`VM.usage_metadata [options]`

Print VM usage metadata as a formatted list of key & (non null) value tuples.

Impact: Low

Note:

The following options must be specified using either key or key=value syntax.

options:

- `-omitIfNull`: (Optional) omit fields if they are null valued (BOOLEAN, false)

- `format`: (Optional) Output format ("plain" or "json") (STRING, plain)

- `filename`:  (Optional) The  file path to the output file into which, append usage metadata (STRING, no default value)

- `fields`: (Optional) a comma separated list of fields (`System` properties, real or synthetic) to print (when non-null) overriding the default (STRING, see below)

Fields are either `System` Property names(c.f: `java.lang.System` for definitions), or certain "synthetic" properties (where there is no pre-existing `System` Property):

- `jvm.args`: any command line arguments specified for the JVM.

- `jvm.container.info`:  (Note: these fields are not affected by the omitIfNull flag, they are only present if the JVM detects the presence of resource mgmt - linux only):
  - `type` (`cgroupv1` or `cgroupv2)`
  -  `memory.limit` or -1 for unrestricted
  - # of `active.cpus` therein.

- `jvm.flags`: the JVM flags.

- `jvm.pid`: the process id of the JVM.

- `jvm.starttime`:  the  time  when the JVM completed its initialization just prior to commencing execution of user code.

- `jvm.uptime.ms`: the uptime (in ms) of the JVM.

- `os.hostname`: the name of the host machine (or container, when the JVM is executing in that context)

- `timestamp`: the current time.

Note: if the value (comma separated list of property names) of the `fields`. parameter is prefixed with a `+`, then the subsequent list of parameters are appended to the `default` listing, if no `+` is present then the property list overrides the default list.

The default set of fields reported are (in order):
`timestamp`
`jvmstarttime`
` java.home`
`java.version`
`java.vm.version`
`sun.java.launcher`
`jvm.flags`
`jvm.args`
`sun.java.cmd`
`java.classpath`
`jdk.module.path`
`jdk.main.module`
`jdk.main.module.class`
`jdk.upgrade.module.path`
`jvm.pid,jvm.uptime.ms`
`jvm.ctr.info.type`
`jvm.ctr.info.memory.limit`
`jvm.ctr.active.cpus`
`os.hostname`
`os.name`
`os.version`
`os.arch`
Comments
[~alanb] given my responses and change(s) are your issues addressed, I would like to move this fwd for 24 note that this jcmd is effectively "Java Usage Tracker lite"++ and is primarily intended to provide a 1-stop cmd to capture JVM usages ...
19-11-2024
I modified `omitIfNull` to be `omitnulls`
19-11-2024
ah sorry by "default" I meant that the set of keys are the "default" set, unless you override with "fields" ... and I agree adding JSON does not require a move to Java, its only that I plan to add JSON support as part of moving it to Java, or more accurately when adding HTTP transport since it would be desirable to that to also support application/json in addition to text/plain.
04-11-2024
> w.r.t JSON, it is not the default format, "plain" is. The reason I asked about this is because the Solution section currently has "Here is example (default) output ..." and it shows the JSON output. The -help output does show the default value for "format" as "plain". Okay, that's the right default as it allows several of the existing commands to add support for JSON without changing their default (it doesn't requires moving code to Java).
04-11-2024
[~alanb] responding to your comments above: - w.r.t "overlap" you are correct however this command acts as a summarizing jcmd permitting agents such as JMS to obtain all this information in a single exchange with the target JVM(*), additionally it adds some significant metadata that is not currently available via other jcmds, e.g container metadata start time etc which is of significant value in order to enable logging of JVM usage. think of this as the equivalent of the "Java Usage Tracking" feature that has been present in the ORCL JDK distro for some time now. - w.r.t JSON, it is not the default format, "plain" is. and yes some, if not all existing commands should support (at least) plain & json, which I will probably implement when I "lift" jcmd out of native and add HTTP/UDS support - w.r.t "object names" obviously the pre-existing system property names are a "well known" interface, some of which are actually defined in the relevant API specs, for the "synthetic" properties, the intent is that those are documented in the jcmd(1) page and as such become a public interface - w.r.t "omitIfNull" happy to remove this if you find it offensive or unnecessary. hope this helps. (*) having a single operation that extracts this metadata from the JVM is very advantageous to agents (such as JMS) that want to provide an audit trail of JVM usage, using multiple jcmds such as those you mention creates a potential race condition where a target JVM may terminate part way thru being interrogated by such agents, an "all or nothing" approach is more desirable.
04-11-2024
Moving back to Provisional until [~alanb]'s concerns are discussed.
04-11-2024
There doesn't seem to be any PR or discussion on the mailing list about this yet. A few comments/questions: There is significant overlap between the output of this new command and existing commands (VM.uptime, VM.info, VM.system_properties, ...). Maybe some of the existing commands should be modified to optionally support printing in JSON format? Are you sure that you want JSON to be the default format? If existing commands are retrofitted to support to print JSON then they will have the opposite default to what is proposed here. Thread.dump_to_file is one example that prints in plain text by default, use -format=json for JSON. Are the object names and the type of the values intended to be a supported interface? JSON is self describing but the question is whether tooling might become dependent on the names and whether any changes will need to be tracked by future CSRs. omitIfNull looks very odd. Are you sure this has to be included and if so, can't it be named consistently with the options supported by other commands.
02-11-2024
[~darcy] [~kevinw] reviewed.
31-10-2024
[~kevinw] s/elide/omit/g s/filepath/filename/g os.hostname is added null fields *are* included by default (with the exception of the container values, which are only written when a container is detected)
29-10-2024
Looks good. Compatibility Kind: behavioral ? If we are not including null fields by default, can the option be "includeNullFields" or similar? (not elideIfNull if skipping null fields is default behaviour). filepath option, could that be called filename? (Most HotSpot DiagnosticCommands use "filename", only ThreadDumpToFileDCmd uses filepath.) Should we include Host name and IP address?
29-10-2024
Moving to Provisional, not Approved. [~lcable], please have another engineer familiar with jcmd review this CSR before it is Finalized for the second phase of review; HTH.
29-10-2024