JDK-8208277 : Code cache heap (-XX:ReservedCodeCacheSize) doesn't work with 1GB LargePages
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 8,9,10,11,12
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2018-07-24
  • Updated: 2020-04-28
  • Resolved: 2018-12-06
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 12
11.0.8-oracleFixed 12 b24Fixed
Related Reports
Relates :  
Description
ADDITIONAL SYSTEM INFORMATION :
Linux x64 with default hugepages set as 1GB:

$ cat /proc/meminfo
....
Hugepagesize:    1048576 kB

$ cat /proc/cmdline
...
default_hugepagesz=1G hugepagesz=1GB hugepages=222

A DESCRIPTION OF THE PROBLEM :
When using -XX:+UseLargePages, and having the linux kernel setup for 1GB largepages (instead of the default 2MB), I can't get the Code cache virtual space to be using large page as well.

This happens because code in heap.cpp http://hg.openjdk.java.net/jdk/jdk/file/0ce279d8c9cd/src/hotspot/share/memory/heap.cpp#l110
  if (os::can_execute_large_page_memory()) {
    const size_t min_pages = 8;
    page_size = MIN2(os::page_size_for_region_aligned(committed_size, min_pages),
                     os::page_size_for_region_aligned(rs.size(), min_pages));
  }

So if I set stuff like -XX:ReservedCodeCacheSize=1G -XX:InitialCodeCacheSize=1G -XX:+UseLargePages it doesn't work
because the code above attempts to fit at least 8 pages in 1G. The function page_size_for_region_aligned is dividing 1GB by 8, and then attempts to find a page size which small than that size.
This works if the hugepage is 2MB (< 1GB/8), but doesn't work if hugepage=1GB of course.

The fix is just replacing the number 8 to 1, since I don't see why we should limit ourself to that number.

I tried looking in mercurial history why the number 8 was chosen, and I couldn't figure this out 
it was always like this: http://hg.openjdk.java.net/jdk6/jdk6/hotspot/file/a61af66fc99e/src/share/vm/memory/heap.cpp#l106


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Boot linux machine with 1GB huge pages by setting the correct parameters in kernel (see the runtime information above)
and then run it with:

java -Xmx1g -Xms1g -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+UseLargePages -XX:ReservedCodeCacheSize=1G -XX:InitialCodeCacheSize=1G

it would show in /proc/meminfo that only 1GB page is used (for the heap)

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
expected result is for CodeCache to be on hugepage as well,
and when changing the constant 8 to 1 in the function above (CodeHeap::reserve) I see it happening, and also another hugepage is used.
ACTUAL -
CodeCache isn't using hugepages

CUSTOMER SUBMITTED WORKAROUND :
None, recompile the code with a different constant.

FREQUENCY : always



Comments
Fix request (11u) -- will label after testing completed. I would like to downport this for parity with 11.0.8-oracle. Applies clean.
22-04-2020

http://cr.openjdk.java.net/~thartmann/8208277/webrev.00/
04-12-2018

Similar code is used in CodeCache::page_size() and at other places where page_size_for_region_aligned() is called. ILW = Code cache does not use 1GB large pages, if ReservedCodeCacheSize is set >= 1 GB and OS is configured to use 1 GB pages, no workaround = MMH = P3
09-08-2018

The code in question and the selection of the number of minimum pages specific to CodeCache/CodeHeap which is compiler code. Forwarding to compiler team.
08-08-2018

The intent of this code is to only select large pages if it does not disable savings when minimum size is unequal to maximum size of the requested memory area. Consider the case when somebody uses large pages and sets minimum and maximum size of the requested memory differently, and maximum size is close to large page size - certainly that user expects that actual memory usage is not the maximum from the beginning. E.g. you set a java heap of Xms200M and Xmx1g, use large pages with 1GB size; the user then clearly indicates that he wants to have the VM grow the heap as needed, and not allocate all of the heap upfront, which since 1g large pages are used, is what will happen. If you set that magic constant to "1" then these users will be unhappy :) One potential solution for this problem would be that if the user sets minimum to the same as the maximum size for committing memory, use "1" as divisor.
31-07-2018