JDK-8012015 : Use PROT_NONE when reserving memory
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Priority: P4
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2013-04-11
  • Updated: 2014-06-26
  • Resolved: 2013-04-30
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8 Other
8Fixed hs25Fixed
Related Reports
Relates :  
Relates :  
Description
Memory is reserved on the *nix platforms using mmap and passing in the MAP_NORESERVE. Before the memory can actually used it needs to be committed, and this is done by calling mmap without the MAP_NORESERVE flag. The commit call also specifies the requested access/protection bits for the address range.

Currently Linux and BSD/OSX the protection used when reserving memory is PROT_READ|PROT_WRITE. This is done in the anon_mmap in the respective os_*.cpp files. This means that the memory range is actually readable and writable, but because the MAP_NORESERVE flag has been specified there is no guarantee that a read/write will succeed - if the system is low on memory and out of swap space for example the read/write may raise a signal.

This is not normally a problem - before the memory is used it is typically committed. However, for subtle bugs where wild pointers are used etc it would be preferable to get a SEGSEGV and catch the bug early rather than have the use of the wild pointer silently succeed.

In the Solaris implementation of anon_mmap there is a comment about exactly this:

  // Map uncommitted pages PROT_NONE so we fail early if we touch an
  // uncommitted page. Otherwise, the read/write might succeed if we
  // have enough swap space to back the physical page.


Also, on both Linux and BSD/OSX the respective pd_uncommit_memory functions both restore the memory to PROT_NONE, so newly reserved memory currently gets PROT_READ|PROT_WRITE, but memory which gets uncommitted gets PROT_NONE which does not appear to be very symmetrical.