JDK-8041506 : The test gc/g1/TestHumongousShrinkHeap.java reports that memory is not de-committed
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 8u40,9
  • Priority: P4
  • Status: Resolved
  • Resolution: Duplicate
  • Submitted: 2014-04-22
  • Updated: 2014-09-08
  • Resolved: 2014-08-26
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 9
9Resolved
Related Reports
Blocks :  
JDK-8038423 - G1: Decommit memory within the heap
Duplicate :  
JDK-8056043 - G1 does not uncommit within the heap after JDK-8038423
Relates :  
JDK-8044132 - Quarantine unstable/broken GC tests
Relates :  
JDK-8041946 - CMM Testing: 8u40 an allocated humongous object at the end of the heap should not prevents shrinking the heap
Description
The test gc/g1/TestHumongousShrinkHeap.java allocates several groups of humongous objects and then frees those which were allocated first.
It expects, that g1 will give some committed memory back to the system. But it doesn't.

The output from the test:
command: main -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=50 -XX:+UseG1GC -XX:G1HeapRegionSize=1M -verbose:gc TestHumongousShrinkHeap
reason: User specified action: run main/othervm -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=50 -XX:+UseG1GC -XX:G1HeapRegionSize=1M -verbose:gc TestHumongousShrinkHeap 
elapsed time (seconds): 1.029
----------System.out:(26/1632)----------
[Full GC (System.gc())  1210K->511K(8192K), 0.0368437 secs]
[init                    ] init: 188.7 MB, used: 524.2 kB, comm: 8.4 MB , freeRatio ~= 93.8%
Will allocate objects of size=943.7 kB
[GC pause (G1 Humongous Allocation) (young) (initial-mark) 2519K->2451K(8192K), 0.0024301 secs]
[GC concurrent-root-region-scan-start]
[GC concurrent-root-region-scan-end, 0.0002982 secs]
[GC concurrent-mark-start]
[GC concurrent-mark-end, 0.0038288 secs]
[GC remark, 0.0010866 secs]
[GC cleanup 7059K->7059K(9216K), 0.0004442 secs]
[GC pause (G1 Evacuation Pause) (young) 10M->10M(28M), 0.0017848 secs]
[GC pause (G1 Humongous Allocation) (young) (initial-mark) 27M->27M(68M), 0.0020361 secs]
[GC concurrent-root-region-scan-start]
[GC concurrent-root-region-scan-end, 0.0004249 secs]
[GC concurrent-mark-start]
[GC concurrent-mark-end, 0.0038802 secs]
[GC remark, 0.0013999 secs]
[GC cleanup 33M->33M(68M), 0.0005887 secs]
[eat #0                  ] init: 188.7 MB, used: 95.0 MB, comm: 108.0 MB, freeRatio ~= 12.1%
[eat #1                  ] init: 188.7 MB, used: 189.3 MB, comm: 212.9 MB, freeRatio ~= 11.0%
[eat #2                  ] init: 188.7 MB, used: 283.7 MB, comm: 317.7 MB, freeRatio ~= 10.7%
[eat #3                  ] init: 188.7 MB, used: 378.1 MB, comm: 422.6 MB, freeRatio ~= 10.5%
[eat #4                  ] init: 188.7 MB, used: 472.5 MB, comm: 527.4 MB, freeRatio ~= 10.4%
[eaten                   ] init: 188.7 MB, used: 472.5 MB, comm: 527.4 MB, freeRatio ~= 10.4%
[Full GC (System.gc())  451M->1438K(503M), 0.0196609 secs]
[free                    ] init: 188.7 MB, used: 1.5 MB , comm: 527.4 MB, freeRatio ~= 99.7%
----------System.err:(21/1357)----------
java.lang.RuntimeException: committed free heap size is not less than committed full heap size, heap hasn't been shrunk?
MinHeapFreeRatio = 10
MaxHeapFreeRatio = 50
	at com.oracle.java.testlibrary.Asserts.error(Asserts.java:450)
	at com.oracle.java.testlibrary.Asserts.assertTrue(Asserts.java:377)
	at com.oracle.java.testlibrary.Asserts.assertLessThan(Asserts.java:84)
	at TestHumongousShrinkHeap.test(TestHumongousShrinkHeap.java:65)
	at TestHumongousShrinkHeap.main(TestHumongousShrinkHeap.java:50)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:484)
	at com.sun.javatest.regtest.MainWrapper$MainThread.run(MainWrapper.java:94)
	at java.lang.Thread.run(Thread.java:744)
Comments
The test should start to pass after JDK-8038423 is fixed. As part of the fix @ignore should be removed from the test description.
18-08-2014
Reevaluated the ILW as it turns out to be a test bug and not a VM bug. I=Bug in test code -> L L=Reproducible - > H W=Don't run that test ->M LHM=P4
30-04-2014
RFE to preserve original test scenario: JDK-8041946
25-04-2014
The test should be updated to allocate smaller objects to be released.
25-04-2014
This is a test bug. Traditionally the heap can only be shrunk from the top down. That means that in order to shrink the heap, there has to be a sufficiently large free area at the end (high addresses) of the heap. In G1 the heap is divided into regions of different kinds. Young regions will be collected (and emptied) at every GC, while old regions will only be included in mixed GCs if they are considered cheap enough. Regions containing large objects (humongous) are never included in young GCs. This means that if we are unlucky and end up with a long lived old or humongous region at the top of the heap we will not be able to shrink even if the rest of the heap is empty. The fix for this implemented in 8u20 is to sort the freelist and deliberately allocate young regions at the top of the heap and old/humongous regions at the bottom of the heap. This will make it far less likely that we end up with a long lived region at the top of the heap. This test creates exactly such a scenario, objects are allocated in such a way to cause a humongous region to be allocated in the end of the heap. Unfortunately, the way it is written makes this region end up in the end of the heap even with sorting. The test allocates ONLY large objects. This means that even though we allocate them from the bottom up, the last one we allocate (which is kept alive at the end of the test) will end up at the top of the heap and stop us from shrinking. To show that sorting works (which was the intention of this test) we should allocate regular objects to fill the heap and only at the end allocate a humongous object. With a non-sorted freelist this will still cause the humongous object to end up at the top of the heap, while with sorting the regular objects will be allocated from the top down, leaving a free region at the bottom for the large object. The test as it looks now can be saved for 8u40. It passes with the changes we have made for decommitting memory within the heap. But for 8u20 this test will not pass as it is written now. FYI, The test is not intended for 7u, the sorting of the freelist has only been backported to 8u.
25-04-2014
Jesper W has been looking at this. It's still not quite clear if this is a VM bug or a test bug. Until we know more we treat it as a VM bug. The test itself is only in 9 at the moment, but the code it tests is in 7u and 8u so it might also affect those. I=Heap doesn't shrink as expected -> M L=Reproducible -> H W=None known -> H MHH=P2
25-04-2014