JDK-8260349 : Cannot programmatically retrieve Metaspace max set via JAVA_TOOL_OPTIONS
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 8-pool,11-pool,15-pool,16
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2021-01-21
  • Updated: 2021-05-06
  • Resolved: 2021-02-01
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 15 JDK 16 JDK 17 Other
11.0.11Fixed 15.0.4Fixed 16.0.1Fixed 17 b08Fixed openjdk8u292Fixed
Related Reports
Relates :  
Relates :  
Relates :  
Description
ADDITIONAL SYSTEM INFORMATION :
macOS 10.14.6

$ java -version
openjdk version "16-ea" 2021-03-16
OpenJDK Runtime Environment (build 16-ea+32-2190)
OpenJDK 64-Bit Server VM (build 16-ea+32-2190, mixed mode, sharing)


A DESCRIPTION OF THE PROBLEM :
When setting the Metaspace max option -XX:MaxMetaspaceSize=256m via the JAVA_TOOL_OPTIONS environment variable, the Metaspace max is not programmatically retrievable from the MemoryPoolMXBean for the Metaspace. However, the MaxMetaspaceSize does appear to be configured according to the output produced by the option -XX:+PrintFlagsFinal which contains:

   size_t MaxMetaspaceSize                         = 268435456                                 {product} {environment}

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
See the provided source code which corresponds to the referenced MetaspaceMaxMain.java file below. The following command (on OSX, at least) reproduces the issue.

JAVA_TOOL_OPTIONS="-XX:MaxMetaspaceSize=256m" java MetaspaceMaxMain.java

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Picked up JAVA_TOOL_OPTIONS: -XX:MaxMetaspaceSize=256m
Metaspace pool max: 268435456
ACTUAL -
Picked up JAVA_TOOL_OPTIONS: -XX:MaxMetaspaceSize=256m
Metaspace pool max: -1

---------- BEGIN SOURCE ----------
import java.lang.management.ManagementFactory;
import java.lang.management.MemoryPoolMXBean;

public class MetaspaceMaxMain {

    public static void main(String[] args) {
        MemoryPoolMXBean metaspaceMemoryPool = ManagementFactory.getPlatformMXBeans(MemoryPoolMXBean.class)
                .stream()
                .filter(pool -> "Metaspace".equals(pool.getName()))
                .findFirst()
                .orElseThrow();
        System.out.println("Metaspace pool max: " + metaspaceMemoryPool.getUsage().getMax());
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Using the JDK_JAVA_OPTIONS environment variable instead of JAVA_TOOL_OPTIONS has the expected behavior. Alternatively, setting the MaxMetaspaceSize on the command line works as expected.


Comments
Fix request (15u): related fixes are in 15 already. Patch applies without adjustments. All-night test runs on win, linux, macos are in progress. Will label after the testing completion.
06-05-2021

Needed for this. Otherwise intermittent test errors. See JDK-8261907.
20-02-2021

Fix Request (16u) Backporting this small low-risk fix prevents this bug from occurring in JDK-16u. The original bug fix patch applied cleanly and included a regression test. After applying the patch to a JDK-16u repo, the fix was regression tested by running Mach5 tiers 1 and 2 on Linux, Solaris, Windows, and Mac OS, and running tiers 3-5 on Linux x64.
19-02-2021

[~stuefe] I presume what is meant is that the new test fails without the functional patch applied, and passes with that patch applied. What kind of regressions are seen on 32-bit? Are these situations where the flag was purportedly set via the env var but was actually ignored because of the bug?
17-02-2021

I don't understand the reasoning behind the 11u downport: "The MaxMetaspaceSizeEnvVarTest.java test passes with the patch and fails otherwise." The test did not exist prior to this patch, so how could it have failed before? We see regressions with this change at least on 32bit platforms, which on which platforms did you test? Which tiers? I am also worried that this patch may introduce subtle compatibility issues if controlling programs parse the output of this mx bean and suddenly see different numbers.
17-02-2021

[~dholmes] Yes, that would make sense. I opened https://bugs.openjdk.java.net/browse/JDK-8261907 to track that bug. Your new test fails. We see this on 32bit Windows, but I have unconfirmed reports of this test error popping up on 64bit Windows too.
17-02-2021

Fix Request (OpenJDK 11u): The patch applies cleanly except for copyright text in memoryPool.cpp. The MaxMetaspaceSizeEnvVarTest.java test passes with the patch and fails otherwise. Webrev: http://cr.openjdk.java.net/~alexsch/sercher/8260349.11u/webrev.00/ The hotspot tests were run, no regressions observed.
05-02-2021

Changeset: b6a73673 Author: David Holmes <dholmes@openjdk.org> Date: 2021-02-01 21:31:25 +0000 URL: https://git.openjdk.java.net/jdk/commit/b6a73673
01-02-2021

Simple fix here: s/FLAG_SET_CMDLINE/!FLAG_IS_DEFAULT/
28-01-2021

./share/runtime/globals_extension.hpp:#define FLAG_IS_CMDLINE(name) (JVMFlag::is_cmdline(FLAG_MEMBER_ENUM(name))) ./share/runtime/flags/jvmFlag.cpp:bool JVMFlag::is_cmdline(JVMFlagsEnum flag) { return flag_from_enum(flag)->is_command_line(); } ./share/runtime/flags/jvmFlag.hpp:bool is_command_line() const { return (_flags & WAS_SET_ON_COMMAND_LINE) != 0; } void set_command_line() { _flags = Flags(_flags | WAS_SET_ON_COMMAND_LINE); } // Note the difference: // f->get_origin() == COMMAND_LINE // f was mostly recently set by the command-line // f->_flags & WAS_SET_ON_COMMAND_LINE // f was specified on the command-line (but may have since been updated by // someone else like FLAG_SET_ERGO) ./share/runtime/flags/jvmFlag.cpp void JVMFlag::setOnCmdLine(JVMFlagsEnum flag) { flag_from_enum(flag)->set_command_line(); } ./share/runtime/globals_extension.hpp:#define FLAG_SET_CMDLINE(name, value) (JVMFlag::setOnCmdLine(FLAG_MEMBER_ENUM(name)) void JVMFlag::set_origin(JVMFlagOrigin new_origin) { int old_flags = _flags; int origin = static_cast<int>(new_origin); assert((origin & VALUE_ORIGIN_MASK) == origin, "sanity"); int was_in_cmdline = (new_origin == JVMFlagOrigin::COMMAND_LINE) ? WAS_SET_ON_COMMAND_LINE : 0; _flags = Flags((_flags & ~VALUE_ORIGIN_MASK) | origin | was_in_cmdline); if ((old_flags & WAS_SET_ON_COMMAND_LINE) != 0) { assert((_flags & WAS_SET_ON_COMMAND_LINE) != 0, "once initialized, should never change"); } } So unless explicitly set via FLAG_SET_CMDLINE(name, value), a flag will be considered "set on cmd line" only if its origin was the command-line. Checking back to Java 7, it seems it has always been this way, so the onus is on the developer to only use FLAG_IS_CMDLINE when they either only want to know if the flag was actually on the command-line, or they know FLAG_SET_CMDLINE has been applied to the flag.
27-01-2021

Thinking more broadly the use of FLAG_IS_CMDLINE seems wrong in general. If it really means "flag was set by user" then its implementation is too specific. I'm going to check the flag code history to see if we went astray somewhere.
27-01-2021

Seems this is day one code for this functionality - JDK-8013590. I couldn't trace back further to the PermGen pools to see if they had a similar option.
27-01-2021

ILW = MMM = P3
27-01-2021

I don't see how this can be platform specific - the code above is shared code. Further investigation is needed. Update: I tested on Linux and the test failed: Picked up JAVA_TOOL_OPTIONS: -XX:MaxMetaspaceSize=256m Metaspace pool max: -1
26-01-2021

ILW = MLM = P4
26-01-2021

OS: Windows 10 JDK16ea32 : Pass JDK 17ea5: Pass Output: Metaspace pool max: 268435456 OS: Oracle Linux 7.6 JDK16ea32:Pass JDK17ea5:Pass Output: Metaspace pool max: 268435456 The program runs fine on windows and linux operating system, the output is as expected, submitter has reported this issue on mac so moving it to Dev team for further analysis.
25-01-2021

The bug is simple and is here: size_t MetaspacePool::calculate_max_size() const { return FLAG_IS_CMDLINE(MaxMetaspaceSize) ? MaxMetaspaceSize : MemoryUsage::undefined_size(); } We only report MaxMetaspaceSize if set on the cmdline. It should also be used if set via the env vars. Note that JDK_JAVA_OPTIONS is processed by the launcher such that the flag appears to be on the command-line for the VM.
25-01-2021