JDK-8320859 : gtest high malloc footprint caused by BufferNodeAllocator stress test
  • Type: Enhancement
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 17,21,22
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2023-11-28
  • Updated: 2025-07-24
  • Resolved: 2023-11-29
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 21 JDK 22
21.0.9Fixed 22 b26Fixed
Related Reports
Duplicate :  
Relates :  
Relates :  
Description
BufferNodeAllocatorTest.stress_free_list_allocator_vm is too expensive. This test accumulates ~1.5 GB of malloc footprint and raises the libc memory retention in my tests from 50m to 800m.

It is quite alone in its hunger. The rest of the tests together accumulate just ~50mb of libc retention.

The buffer does a stress test of the BufferNodeAllocator. Four "mutator threads" race four "gc threads". Mutator threads allocate buffers, GC threads release them. No processing is done on the buffers; this seems to be purely a test of the allocator and its freelist mechanism. The memory footprint of this test depends on the number of retained free buffers in the allocator. The allocator bulk-releases free buffers triggered by a free-count-threshold. On my box, the number of free items is 100k..200k.

To me, its not clear whether the fact that so many freelist items exist indicates a problem with the allocator itself. To me it looks like the free list should be drained if there are more than 10 items in this list.

But each buffer is capacity=1024, *sizeof(void*)+header, so 8KB+x, and with 100..200k of those alive it explains the NMT-reported malloc footprint of ~1.5GB. An easy fix is to reduce the size of this buffer; since they are not processed, their size should not matter. It may be good to keep them above a cache line size though to prevent false sharing during this test.
Comments
[jdk21u-fix-request] Approval Request from Daniel Hu Improves memory footprint of gtest BufferNodeAllocatorTest.stress_free_list_allocator_vm. Applies cleanly from JDK22 and no bugtail. Passes GHA and gtest test suite. Low risk as only test code change.
27-06-2025

A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk21u-dev/pull/1931 Date: 2025-06-27 18:09:48 +0000
27-06-2025

Changeset: abf2e490 Author: Thomas Stuefe <stuefe@openjdk.org> Date: 2023-11-29 13:18:31 +0000 URL: https://git.openjdk.org/jdk/commit/abf2e4909733550c228bd66ccebe33a492269e7f
29-11-2023

A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/16845 Date: 2023-11-28 11:02:24 +0000
28-11-2023

Interesting observations, and possible future test improvements: - Number of free blocks depends on buffer node size. Also in release builds, so this is not buffer zapping. Smaller buffers make for an order of magnitude more free nodes (8K: ~200k, 64byte: >2mio). So, speed of allocator depends on buffer size, which seems strange, since I cannot see explicit buffer initialization. It may be simply an effect of the libc being less efficient when allocating larger buffers. Maybe it mmaps every single one. But this also means that to stress the allocator freelist, smaller buffers make more sense. - Test time influences the number of free blocks. Test runtime goes up, number of free blocks goes up linearily. So the deallocation path in FreeList is more expensive than the allocation path, and the deallocation threads are outraced by the mutators. - The memory allocated for buffer nodes is larger than expected from buffer size: buffer node=cacheline size accumulates ~260MB with ~2mio allocations.
28-11-2023