JDK-8226236 : win32: gc/metaspace/TestCapacityUntilGCWrapAround.java fails
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 8u212
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux,windows
  • CPU: x86
  • Submitted: 2019-06-17
  • Updated: 2022-12-02
  • Resolved: 2020-10-15
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 16 Other
11.0.18Fixed 16 b21Fixed openjdk8u372Fixed
Related Reports
Relates :  
JDK-8245707 - Increase Metaspace reserve alignment
Relates :  
JDK-8217432 - MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize
Description
Originally filed at AdoptOpenJDK:
https://github.com/AdoptOpenJDK/openjdk-tests/issues/1162

The test fails on 32bit windows with:
java.lang.IllegalStateException: WB_IncMetaspaceCapacityUntilGC: could not increase capacity until GC due to contention with another thread
        at sun.hotspot.WhiteBox.incMetaspaceCapacityUntilGC(Native Method)
        at TestCapacityUntilGCWrapAround.main(TestCapacityUntilGCWrapAround.java:51)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
        at java.lang.Thread.run(Thread.java:748)
Comments
It is now in 11u, re-added request label for 8u.
25-11-2022
Fix request (11u): Fix to hotspot, fixing gc/metaspace/TestCapacityUntilGCWrapAround.java (hotspot/tier1) on windows x86 Clean backport, Run hotspot/tier1 on win32 build, it fixed that issue. GHA tests passed.
24-11-2022
Sorry, I had been struggling with building jdk11 win32 build(s) to test that backport, but finally succeeded.
24-11-2022
A pull request was submitted for review. URL: https://git.openjdk.org/jdk11u-dev/pull/1549 Date: 2022-11-23 13:03:15 +0000
23-11-2022
[~zzambers] Please make sure to get this into OpenJDK 11 before proposing for 8u. I've removed the approval label meanwhile. Thanks!
23-11-2022
A pull request was submitted for review. URL: https://git.openjdk.org/jdk8u-dev/pull/192 Date: 2022-11-23 00:25:55 +0000
23-11-2022
Fix request (8u): Fix to hotspot, fixing gc/metaspace/TestCapacityUntilGCWrapAround.java (hotspot/tier1) on windows x86 Not a clean backport due to some renames, done tier1 testing including x86 - fixes one test causes no regressions
23-11-2022
Changeset: 038f58d4 Author: Yasumasa Suenaga <ysuenaga@openjdk.org> Date: 2020-10-15 00:21:42 +0000 URL: https://git.openjdk.java.net/jdk/commit/038f58d4
15-10-2020
No, the test is not wrong. No need to change it. This is a real issue. The test wants to test that calling MetaspaceGC::inc_capacity_until_GC() repeatedly will not overflow the gc threshold. Lets see: 1) MetaspaceGC::ergo_initialize() MaxMetaspaceSize, if left unspecified, is supposed to be "infinite". In reality it defaults to max_uintx. But it gets aligned down by Metaspace::reserve_alignment. That one is either one of os::vm_allocation_granularity or four pages, whatever is larger. The four pages thing I recently added with JDK-8245707 to shake loose misuse of this alignment value (exactly cases like this). So on Windows, Metaspace::reserve_alignment has always been os::vm_allocation_granularity (64K). On Linux, it was always 4K, until recently it switched to 16K. This explains the 16K aligned value we see for MaxMetaspaceSize in Alexeys test: 4294950912 (FFFFC000). 2) MetaspaceGC::inc_capacity_until_GC() The increase value causes an overflow - its supposed to do that. Overflow gets handled by: if (new_value < old_capacity_until_GC) { // The addition wrapped around, set new_value to aligned max value. new_value = align_down(max_uintx, Metaspace::commit_alignment()); } which sets the new threshold at max_uintx aligned down by Metaspace::commit_alignment(), which is just os::vm_page size. So the new value is 4294963200, resp. FFFFF000 So the problem is that one value gets aligned down by Metaspace::reserve_alignment() (os::vm_allocation_granularity()), the other by Metaspace::commit_alignment (os::vm_page_size()). Then we compare those values. So the test is okay. It showed us a real issue. I am not 100% sure what the correct behavior would be. Maybe increasing the gc threshold beyond MaxMetaspaceSize should not be an error at all, but we should just cap out at that value? Possible solutions: A) As stated above, going beyond MaxMetaspaceSize could just cap. b) All that alignment business is actually unnecessary and we could remove it for clarity after JEP387 is out of the door. MaxMetaspaceSize does not need to be aligned to anything, neither does the GC threshold. C) as a small workaround, at (2) one could use Metaspace::reserve_alignment, not commit_alignment. That makes no sense semantically but would remove this error. We may want to do both? (A) needs some thinking. (This must have been always a problem on Windows, and only recently on Linux, right?)
14-10-2020
[~shade] I uploaded draft PR to GitHub. Could you test it? https://github.com/openjdk/jdk/pull/628 This change can run on Linux x64, and it works fine on my Linux x64 box.
13-10-2020
Yeah, I can gladly test the fix on x86_32. There seem to be no failures on x86_64.
13-10-2020
I can look into this. It seems to be caused by MaxMetaspaceSize or `inc` in WB_IncMetaspaceCapacityUntilGC() at a glance. `new_value` should not be greater than MaxMetaspaceSize. TestCapacityUntilGCWrapAround passes 4GB - 1 to incMetaspaceCapacityUntilGC(). It seems to be too big. Haven't you seen this issue on 64 bit platforms? If you haven't seen, I will change to pass 2GB to wb.incMetaspaceCapacityUntilGC(), but I do not have environment to evaluate the fix. Could you help?
13-10-2020
Yasumasa, seeing that you did JDK-8217432, can you look at this bug too?
13-10-2020
That new exit was added by JDK-8217432.
24-09-2020
The error message is fake. There are two exits with "false" from WB_IncMetaspaceCapacityUntilGC. Add this printout: diff --git a/src/hotspot/share/memory/metaspace.cpp b/src/hotspot/share/memory/metaspace.cpp index bf8a21f40ca..f8d9cef5f46 100644 --- a/src/hotspot/share/memory/metaspace.cpp +++ b/src/hotspot/share/memory/metaspace.cpp @@ -158,6 +158,7 @@ bool MetaspaceGC::inc_capacity_until_GC(size_t v, size_t* new_cap_until_GC, size if (can_retry != NULL) { *can_retry = false; } + tty->print_cr("new_value > MaxMetaspaceSize: " SIZE_FORMAT " > " SIZE_FORMAT, new_value, MaxMetaspaceSize); return false; } @@ -167,6 +168,7 @@ bool MetaspaceGC::inc_capacity_until_GC(size_t v, size_t* new_cap_until_GC, size size_t prev_value = Atomic::cmpxchg(&_capacity_until_GC, old_capacity_until_GC, new_value); if (old_capacity_until_GC != prev_value) { + tty->print_cr("Contention. old_capacity_until_GC != prev_value: " SIZE_FORMAT " != " SIZE_FORMAT, old_capacity_until_GC, prev_value); return false; } Observe: STDOUT: new_value > MaxMetaspaceSize: 4294963200 > 4294950912 STDERR: java.lang.IllegalStateException: WB_IncMetaspaceCapacityUntilGC: could not increase capacity until GC due to contention with another thread at sun.hotspot.WhiteBox.incMetaspaceCapacityUntilGC(Native Method) at gc.metaspace.TestCapacityUntilGCWrapAround.main(TestCapacityUntilGCWrapAround.java:52) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) at java.base/java.lang.Thread.run(Thread.java:832)
24-09-2020
Also easily reproduces on Linux x86_32: $ CONF=linux-x86-server-fastdebug make images run-test TEST=gc/metaspace/TestCapacityUntilGCWrapAround.java
24-09-2020