JDK-8268722 : ZGC core files are unmanagably big on OSX
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 17
  • Priority: P3
  • Status: Closed
  • Resolution: Won't Fix
  • OS: os_x
  • CPU: x86
  • Submitted: 2021-06-14
  • Updated: 2022-12-13
  • Resolved: 2021-11-12
Related Reports
Relates :  
JDK-8298701 - Cleanup SA entries in ProblemList-zgc.txt.
Relates :  
JDK-8293563 - [macos-aarch64] SA core file tests failing with sun.jvm.hotspot.oops.UnknownOopException
Relates :  
JDK-8268723 - Problem list SA core file tests on OSX when using ZGC
Description
ZGC cores on linux-x64 are noticeably bigger on linux-x64 than with G1, but are still relatively small. SA core file tests that crash on startup will have about a 1gb core with G1 on linux-x64 and 2.5gb on with ZGC. This is with no other heap related settings.

However, on OSX core files sizes are a lot bigger in general than with linux-x64. For example, the same SA core tests produce a 6gb core file with G1 on OSX compared to the 1gb on linux-x64 mentioned above. ZGC greatly magnifies this increase in core file size, and the same SA tests are producing 80gb core files with the OSX + ZGC combination.

Another experiment that was done on core files sizes was to start the JVM with -Xmx128m and then force core dump by using gcore. The G1 core was 3gb and the ZGC core was 18gb.

And one more experiment was done on an MBPro (not a mach5 host). The JVM was launched with -XX:ErrorHandlerTest=1. This created an 8gb core file with G1. With ZGC the core file was 220gb, but the host also crashed due to running out of disk space, so it could actually have been larger than this.
Comments
This doesn't seem to be as much of a problem on aarch64 as it is on x64. I'm seeing core files mostly around 4gb on aarch64 compared to 40gb to 80b on x64. This might be related to JDK-8293563, which discovered that often the java heap, or at least large portions of it, would be missing from the core file. This was worked around by adding -XX:+EnablePreTouch, but it looks like this does not cause bloated core files, probably because the ZGC mapped memory for the heap is actually 16x the size of the heap (See "HeapSpace (Xmx * 16 * 3)" computation mentioned above), and probably only one of these 16 "spaces" gets dumped to the core file. Using -XX:ZMarkStackSpaceLimit=128M -XX:CompressedClassSpaceSize=64M -Xmx128m helps a little on both platforms. On x64 it seems to cut the core files size to less than half of what it was, so still nearly 40gb in some cases. On aarch64 it takes off maybe 20% from the core file size. Since this doesn't seem to be much of an issue for aarch64, and x64 core file testing will likely always be blocked by JDK-8294316, I think we can take the core file tests off of the ZGC problem list. This CR has been closed as WNF, and closed CRs should not appear in the problem list.
13-12-2022
As previously mentioned, we can't really do a lot about this, since we have limited control over how the OS and gcore generates core files. So, I'm closeing this bug. I can mention that the work we're doing on generational ZGC has the side effect of removing heap multi-mapping, which will reduce the amount of address space ZGC uses, so core files should also become a bit smaller.
12-11-2021
There isn't a lot we can do about this other then mitigating the problem using the flags I mentioned. With the tests in question problem listed, I'm leaning towards closing this bug, but for now I'll just move it to 18.
29-06-2021
[~pliden] The gcore example was just one of many showing the large core files. -Xmx128m was used both with the simple gcore example and with SA tests that force a core by causing a crash. Both of them resulted in an 18g core file when using ZGC. I'll try -XX:ZMarkStackSpaceLimit=128M -XX:CompressedClassSpaceSize=64M and see if that helps.
18-06-2021
[~cjplummer] gcore is known to make core files which includes not just committed memory, but also any reserved address space, which makes the core files a lot bigger. On Linux, core files generated by the kernel (through SIGABRT) doesn't have this problem (I'm not sure how macOS works in this regard). In addition to this, ZGC typically reserves more address space than G1. This in combination with use of gcore can explain the increase in size you see. In ZGC, the reserved address space looks roughly like this by default: HeapSpace (Xmx * 16 * 3) + MarkStackSpace (defaults to 8G) + CompressedClassSpace (defaults to 1G) And in G1, is looks roughly like this by default: HeapSpace (Xmx) + CompressedClassSpace (defaults to 1G) To make core files smaller in testing, you can adjust the amount of memory reserved for mark stacks and the compressed class space. For example, most smaller applications will likely work well with -XX:ZMarkStackSpaceLimit=128M -XX:CompressedClassSpaceSize=64M. Of course, keeping -Xmx as small as possible also helps.
18-06-2021