JDK-8303767 : Identify and address the likely causes of glibc allocator fragmentation
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 11,17,21
  • Priority: P3
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2023-03-07
  • Updated: 2024-11-19
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbdUnresolved
Related Reports
Relates :  
JDK-8193521 - glibc wastes memory with default configuration
Relates :  
JDK-8198756 - Lazy allocation of compiler threads
Relates :  
JDK-8302264 - Improve dynamic compiler threads creation
Description
glibc can sometimes, especially in case of multi threaded applications, keep certain pages in physical memory even though those pages have already been freed. This can be observed as increase in RSS for programs using glibc allocator.

The goal of this issue is to identify if there are any allocation patterns in the JVM that can lead to such memory fragmentation and in turn cause increase in RSS for Java applications.
Comments
We are currently investigating the removal of the Arena ChunkPool. This will likely reduce glibc malloc fragmentation. If a user wants to minimize fragmentation for themselves, then they could either use the HeapTrim functionality on Linux, or they could LD_PRELOAD a different allocator such as Mesh: https://github.com/plasma-umass/Mesh
19-11-2024
Starting a JFR recording with OldObjectSample event enabled will trigger this problem in a very visible way for application starting many threads (in my case the 'akka-uct' from Renaissance benchmark).
14-04-2024
The best mitigation that we have today is the native heap trim functionality that Thomas Stüfe exposed.
06-02-2024
Another possible mitigation might be using a smaller MALLOC_ARENA_MAX (see JDK-8193521).
06-02-2024
Thanks Vladimir! I missed that.
10-03-2023
I assume this is general bug to find all causes of fragmentation and RSS memory increase. Dynamic creation/termination of compiler threads improvement will be addressed by separate RFE JDK-8302264.
09-03-2023
Dynamic creation/termination of compiler threads (added with JDK-8198756) seems to increase the likelihood of fragmentation.
09-03-2023
Tentative ILW = Memory fragmentation leading to increase in RSS, with certain VM allocation patterns, no workaround but disabling of some VM features might mitigate the issue = MMM = P3
09-03-2023