JDK-8174741 : Allocating direct memory should not eat the heap
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 9
  • Priority: P4
  • Status: Closed
  • Resolution: Cannot Reproduce
  • Submitted: 2017-02-10
  • Updated: 2022-09-20
  • Resolved: 2022-09-20
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 20
20Resolved
Related Reports
Relates :  
JDK-8175183 - Co-located GC test should produce VM GC log.
Relates :  
JDK-8215523 - jstat reports incorrect values for OU for CMS GC
Relates :  
JDK-8069351 - [TESTBUG] gc/memory/Nio_PageAlignDirectMemory fails: Allocating direct memory should not eat the heap!
Description
Test: gc/memory/Nio

[2017-02-10T03:50:03.57] Allocating all the direct memory: 52428800
[2017-02-10T03:50:03.61] ... success
[2017-02-10T03:50:03.63] Used heap before: 35209904
[2017-02-10T03:50:03.63] Used heap after : 81032720
[2017-02-10T03:50:03.63] Try to dump heap to heapDump.hprof file..
[2017-02-10T03:50:03.96] Done
[2017-02-10T03:50:03.97] Exception in thread "main" gc.memory.Nio.Nio$Fault: Allocating direct memory should not eat the heap! Heap dumped to heapDump.hprof file.
[2017-02-10T03:50:03.97] 	at gc.memory.Nio.Nio.checkHeapIsNotAffected(Nio.java:141)
[2017-02-10T03:50:03.97] 	at gc.memory.Nio.Nio.run(Nio.java:55)
[2017-02-10T03:50:03.97] 	at gc.memory.Nio.Nio.main(Nio.java:31)
Comments
Has not actually occurred for a long time, closing as WNF
20-09-2022
Correction, at least the concurrent collectors (G1, CMS) has this problem.
15-11-2017
I could not reproduce the issue after 400 runs. Looking at the hprof file with visualvm, there are a few large int[] arrays on the heap (according to VisualVM), however they are not live, which indicates that they are just dummy objects for heap dumping. Overall ocucpancy is like 800k, with ~80M of dead objects - this corresponds to the roughly 81M of used heap in the log output. One potential cause for this issue can be that the values returned by MemoryUsage are not necessarily precise. I.e. in CMS (that is affected here, but in the related JDK-8069351 it seems to have been parallel) the amount of used memory is calculated by "capacity() - free()". The free() method thas this interesting comment: // "MT-safe, but not MT-precise"(TM), if you will: i.e. // if you do this while the structures are in flux you // may get an approximate answer only; for instance // because there is concurrent allocation either // directly by mutators or for promotion during a GC. // It's "MT-safe", however, in the sense that you are guaranteed // not to crash and burn, for instance, because of walking // pointers that could disappear as you were walking them. // The approximation is because the various components // that are read below are not read atomically (and // further the computation of totalSizeInIndexedFreeLists() // is itself a non-atomic computation. The normal use of // this is during a resize operation at the end of GC // and at that time you are guaranteed to get the // correct actual value. However, for instance, this is // also read completely asynchronously by the "perf-sampler" // that supports jvmstat, and you are apt to see the values // flicker in such cases. MemoryUsage reading by the application is also such a case. So basically what could happen is that due to some single allocation the heap is expanded right before getting memory usage in the test. The values used to calculate free() might not have been updated yet, so the result is very close to the maximum capacity. I do not think this behavior, while unexpected, breaks the MemoryUsage API guarantees. I recommend changing the test to use fixed heap size but otherwise not change the collectors. This would at least fix temporary inconsistencies due to this issue (i.e. capacity() could not increase while the values returned by free() have not been adjusted yet). One other option to find these kind of inconsistencies is to run with -Xlog:gc+heap+exit=info. The heap size printout at the end of the run would show massive temporary misreporting by the MemoryUsage API.
13-02-2017