JDK-6642862 : Code cache allocation fails with large pages after 6588638
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 7
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: windows_2003
  • CPU: x86
  • Submitted: 2007-12-17
  • Updated: 2011-04-20
  • Resolved: 2010-01-13
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 Other
6u14Fixed hs12Fixed
Related Reports
Relates :  
JDK-6677036 - investigate mapping the code cache with large pages on linux
Description
With JDK7 b23 on Windows Server 2003 64-bit (using the 32-bit JDK):

C:\> z:\7\b23\windows-i586\fastdebug\bin\java -server -Xmn500m -Xms750m -Xmx750m
-Xss96k -XX:+UseParallelOldGC -XX:+AggressiveOpts -XX:+PrintCommandLineFlags -XX
:+UseLargePages -XX:+TracePageSizes -version
VM option '+UseParallelOldGC'
VM option '+AggressiveOpts'
VM option '+PrintCommandLineFlags'
VM option '+UseLargePages'
VM option '+TracePageSizes'
-XX:+AggressiveOpts -XX:MaxHeapSize=786432000 -XX:MaxNewSize=524288000 -XX:NewSi
ze=524288000 -XX:+PrintCommandLineFlags -XX:ThreadStackSize=96 -XX:+TracePageSiz
es -XX:+UseLargePages -XX:+UseParallelGC -XX:+UseParallelOldGC
code heap:  min=2359296 max=50331648 pg_sz=2097152 base=0x02000000 size=50331648

Java HotSpot(TM) Server VM warning: os::commit_memory failed
Error occurred during initialization of VM
Could not reserve enough space for code cache


This is fastdebug; it also fails with product VM.  It does not fail in b21.
Comments
EVALUATION Even though there is no way to request it through the shm* apis, solaris with ISM enables execute permissions on large page memory, at least on solaris 10. Using a JVM where can_execute_large_page_memory() returns true on solaris with ISM and the code cache is mapped at 0xfb800000: $ pmap -s 10827 | egrep FB800000 FB800000 32768K 4M rwxsR [ ism shmid=null ] However, I was not able to test on solaris 8, the only release where ISM should be used, and the execute permissions are not documented. So can_execute_large_page_memory() returns false on solaris with ISM.
25-03-2008
SUGGESTED FIX Add a new method os::can_execute_large_page_memory() that returns true if large page memory can have execute permissions. Returns false on linux and solaris with ISM. Change the code cache creation code to use large pages only if can_execute_large_page_memory() returns true. Initialize the os::_page_sizes array on solaris when ISM is used. Change windows os::reserve_memory_special() to include execute permission. Call the ReservedSpace ctor with parameter large == true when requesting large pages for the code cache and other non-java-heap data structures. --- old/src/share/vm/runtime/os.hpp Mon Mar 24 18:35:15 2008 +++ new/src/share/vm/runtime/os.hpp Mon Mar 24 18:35:14 2008 @@ -236,6 +236,7 @@ static bool large_page_init(); static size_t large_page_size(); static bool can_commit_large_page_memory(); + static bool can_execute_large_page_memory(); // OS interface to polling page static address get_polling_page() { return _polling_page; } --- old/src/os/linux/vm/os_linux.cpp Mon Mar 24 18:35:16 2008 +++ new/src/os/linux/vm/os_linux.cpp Mon Mar 24 18:35:16 2008 @@ -2486,6 +2486,10 @@ return false; } +bool os::can_execute_large_page_memory() { + return false; +} + // Reserve memory at an arbitrary address, only if that area is // available (and not reserved for something else). --- old/src/os/solaris/vm/os_solaris.cpp Mon Mar 24 18:35:19 2008 +++ new/src/os/solaris/vm/os_solaris.cpp Mon Mar 24 18:35:19 2008 @@ -3067,6 +3067,8 @@ if (UseISM) { // ISM disables MPSS to be compatible with old JDK behavior UseMPSS = false; + _page_sizes[0] = _large_page_size; + _page_sizes[1] = vm_page_size(); } UseMPSS = UseMPSS && @@ -3155,6 +3157,10 @@ bool os::can_commit_large_page_memory() { return UseISM ? false : true; } + +bool os::can_execute_large_page_memory() { + return UseISM ? false : true; +} static int os_sleep(jlong millis, bool interruptible) { const jlong limit = INT_MAX; --- old/src/os/windows/vm/os_windows.cpp Mon Mar 24 18:35:21 2008 +++ new/src/os/windows/vm/os_windows.cpp Mon Mar 24 18:35:21 2008 @@ -2523,9 +2523,13 @@ return false; } +bool os::can_execute_large_page_memory() { + return true; +} + char* os::reserve_memory_special(size_t bytes) { DWORD flag = MEM_RESERVE | MEM_COMMIT | MEM_LARGE_PAGES; - char * res = (char *)VirtualAlloc(NULL, bytes, flag, PAGE_READWRITE); + char * res = (char *)VirtualAlloc(NULL, bytes, flag, PAGE_EXECUTE_READWRITE); return res; } --- old/src/share/vm/memory/heap.cpp Mon Mar 24 18:35:24 2008 +++ new/src/share/vm/memory/heap.cpp Mon Mar 24 18:35:23 2008 @@ -102,8 +102,9 @@ _log2_segment_size = exact_log2(segment_size); // Reserve and initialize space for _memory. - const size_t page_size = os::page_size_for_region(committed_size, - reserved_size, 8); + const size_t page_size = os::can_execute_large_page_memory() ? + os::page_size_for_region(committed_size, reserved_size, 8) : + os::vm_page_size(); const size_t granularity = os::vm_allocation_granularity(); const size_t r_align = MAX2(page_size, granularity); const size_t r_size = align_size_up(reserved_size, r_align); @@ -111,7 +112,7 @@ const size_t rs_align = page_size == (size_t) os::vm_page_size() ? 0 : MAX2(page_size, granularity); - ReservedSpace rs(r_size, rs_align, false); + ReservedSpace rs(r_size, rs_align, rs_align > 0); os::trace_page_sizes("code heap", committed_size, reserved_size, page_size, rs.base(), rs.size()); if (!_memory.initialize(rs, c_size)) { --- old/src/share/vm/gc_implementation/parallelScavenge/parMarkBitMap.cpp Mon Mar 24 18:35:25 2008 +++ new/src/share/vm/gc_implementation/parallelScavenge/parMarkBitMap.cpp Mon Mar 24 18:35:25 2008 @@ -41,7 +41,7 @@ const size_t rs_align = page_sz == (size_t) os::vm_page_size() ? 0 : MAX2(page_sz, granularity); - ReservedSpace rs(bytes, rs_align, false); + ReservedSpace rs(bytes, rs_align, rs_align > 0); os::trace_page_sizes("par bitmap", raw_bytes, raw_bytes, page_sz, rs.base(), rs.size()); _virtual_space = new PSVirtualSpace(rs, page_sz); --- old/src/share/vm/gc_implementation/parallelScavenge/psParallelCompact.cpp Mon Mar 24 18:35:27 2008 +++ new/src/share/vm/gc_implementation/parallelScavenge/psParallelCompact.cpp Mon Mar 24 18:35:27 2008 @@ -413,7 +413,7 @@ const size_t rs_align = page_sz == (size_t) os::vm_page_size() ? 0 : MAX2(page_sz, granularity); - ReservedSpace rs(bytes, rs_align, false); + ReservedSpace rs(bytes, rs_align, rs_align > 0); os::trace_page_sizes("par compact", raw_bytes, raw_bytes, page_sz, rs.base(), rs.size()); PSVirtualSpace* vspace = new PSVirtualSpace(rs, page_sz);
20-03-2008
EVALUATION Other structures that may use large pages (the marking bitmap and region table used by the parallel compacting collector) must also set parameter large == true when constructing the ReservedSpace.
20-03-2008
EVALUATION The fix for 6588638 (improve support for large pages) generalized large page support and started mapping the code cache with large pages in addition to the java heap. This did not work on systems where large page memory cannot be committed on demand. Solaris MPSS (the default and preferred method on solaris 9 and later) allows large page memory to be committed on demand and works fine. Linux and solaris with ISM use shared memory segments to get large pages; the memory cannot be committed on demand. Windows uses VirtualAlloc() to get large pages; the memory cannot be committed on demand. There are multiple problems. The first is that the hotspot ReservedSpace ctor takes a boolean parameter, large, which must be set to true to get large page memory on systems that cannot commit large pages on demand. The code cache creation code was not passing large == true to the ReservedSpace ctor when attempting to get large pages. The second is that the code cache memory must have execute permissions. On windows, the code to reserve large page memory was not requesting execute permissions. On linux and solaris + ISM, the shm* apis used to request large page memory do not allow the execute bit to be specified, so the code cache cannot be mapped using large pages on these systems. The final problem, specific to solaris + ISM, is that the JVM's _page_size_table was not being initialized properly. Created RFE 6677036 to track enabling large pages for the code cache on linux, when/if the kernel allows large page memory to have execute permission.
18-03-2008
WORK AROUND Do not use large pages on windows (omit the option -XX:+UseLargePages). This reduces performance on large heaps, so is not acceptable for many users.
18-03-2008