JDK-8204128 : NMT might report incorrect numbers for Compiler area
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 11,12,14
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2018-05-30
  • Updated: 2024-03-06
  • Resolved: 2019-11-15
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 14
14 b24Fixed
Related Reports
Relates :  
JDK-8187685 - NMT: Tracking compiler memory usage of thread's resource area
Relates :  
JDK-8241086 - Test runtime/NMT/HugeArenaTracking.java is failing on 32bit Windows
Relates :  
JDK-8234270 - [REDO] JDK-8204128 NMT might report incorrect numbers for Compiler area
Description
Here is the example of NMT output produced by stress test:
22898:

Native Memory Tracking:

Total: reserved=34226313KB, committed=14753353KB
-                 Java Heap (reserved=31424512KB, committed=13914112KB)
                            (mmap: reserved=31424512KB, committed=13914112KB) 
 
-                     Class (reserved=1096992KB, committed=52560KB)
                            (classes #5901)
                            (  instance classes #5381, array classes #520)
                            (malloc=1312KB #24829) 
                            (mmap: reserved=1095680KB, committed=51248KB) 
                            (  Metadata:   )
                            (    reserved=47104KB, committed=46256KB)
                            (    used=42088KB)
                            (    free=4168KB)
                            (    waste=0KB =0.00%)
                            (  Class space:)
                            (    reserved=1048576KB, committed=4992KB)
                            (    used=3792KB)
                            (    free=1200KB)
                            (    waste=0KB =0.00%)
 
-                    Thread (reserved=139697KB, committed=7857KB)
                            (thread #135)
                            (stack: reserved=138620KB, committed=6780KB)
                            (malloc=461KB #717) 
                            (arena=616KB #268)
 
-                      Code (reserved=251302KB, committed=114814KB)
                            (malloc=3618KB #9735) 
                            (mmap: reserved=247684KB, committed=111196KB) 
 
-                        GC (reserved=1275090KB, committed=625290KB)
                            (malloc=76178KB #46284) 
                            (mmap: reserved=1198912KB, committed=549112KB) 
 
-                  Compiler (reserved=18014398509481980KB, committed=18014398509481980KB)
                            (malloc=323KB #1073) 
                            (arena=18014398509481657KB #5)
 
-                  Internal (reserved=13309KB, committed=13309KB)
                            (malloc=13269KB #9562) 
                            (mmap: reserved=40KB, committed=40KB) 
 
-                     Other (reserved=170KB, committed=170KB)
                            (malloc=170KB #18) 
 
-                    Symbol (reserved=8044KB, committed=8044KB)
                            (malloc=4968KB #31063) 
                            (arena=3077KB #1)
 
-    Native Memory Tracking (reserved=2749KB, committed=2749KB)
                            (malloc=626KB #8879) 
                            (tracking overhead=2123KB)
 
-               Arena Chunk (reserved=257KB, committed=257KB)
                            (malloc=257KB) 
 
-                   Tracing (reserved=13813KB, committed=13813KB)
                            (malloc=13813KB #258) 
 
-                   Logging (reserved=6KB, committed=6KB)
                            (malloc=6KB #237) 
 
-                 Arguments (reserved=19KB, committed=19KB)
                            (malloc=19KB #483) 
 
-                    Module (reserved=358KB, committed=358KB)
                            (malloc=358KB #2329)
Comments
Thanks [~zgu]
27-04-2020
I assume 11-bp means 11u backport? The right one to backport is JDK-8234270, which I already did.
23-04-2020
URL: https://hg.openjdk.java.net/jdk/jdk/rev/700a2ad8fc19 User: zgu Date: 2019-11-15 20:34:56 +0000
15-11-2019
New reported instance: https://mail.openjdk.java.net/pipermail/hotspot-dev/2019-November/039964.html
14-11-2019
There is not much for me to pursue. Therefore, I am taking my name off this bug. If additional information become available, I can take another look.
25-02-2019
This is rather puzzling. The memory snapshot seems good, otherwise, we should see bogus numbers in "Arena Chunk". No idea how the data can be corrupted during reporting ... May I ask how many times this happened?
12-02-2019
Looks like a NMT bug. 18014398509481657KB overflows size_t. There are a few suspicious long -> int -> size_t castings. I will take a look.
11-02-2019
I had another look at this and I was not able to reproduce it on my machine. Here are some ideas of how this could happen: - Memory corruption of the MallocHeader leads to incorrect values of MallocHeader::_size and screws up memory tracking values - Integer underflow because the size reported MallocMemorySummary::record_malloc differs from what is found by MallocMemorySummary::record_free. A race condition should not be possible, right? - Now that I'm looking at the code around os.cpp, os::realloc looks a bit suspicious, maybe there is an inconsistency (and compile log uses REALLOC_C_HEAP_ARRAY). My guess is that this is a general problem with NMT that only manifests in the compiler area because the compilers allocate and free lots of memory (especially during long running Kitchensink). I'm moving this to runtime, please assign back if it turns out to be a compiler issue.
11-07-2018
We don't know. It only ever showed up with JDK 11.
13-06-2018
Is this a new issue in 11? or can we add 9, 10 to Affects Version/s?
13-06-2018
ILW = NMT reports incorrect numbers for compiler area, with kitchensink and nmt enabled, no workaround = MMH = P3 When fixing this, we should probably also add an assert somewhere to make sure that these numbers are at least in sane bounds.
31-05-2018