JDK-6675312 : Linux glibc stack guard-pages can overlap with hotspot guard pages
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 7
  • Priority: P3
  • Status: Closed
  • Resolution: Won't Fix
  • OS: linux
  • CPU: generic
  • Submitted: 2008-03-14
  • Updated: 2016-11-09
  • Resolved: 2015-06-15
Related Reports
Relates :  
Relates :  
Relates :  
Description
Gary Benson reported on the OpenJDK hotspot-dev mailing list:

 The first thing I discovered is that the current linux code is wrong
 when there are guard pages.  The comment above current_stack_region
 in os_linux_{i486,amd64,x86}.cpp puts the guard page outside the
 region reported by pthread_attr_getstack(), which is not the case.
 It needs to use pthread_attr_getguardsize() and trim that many bytes
 from the bottom of the region reported by pthread_attr_getstack().

For Java threads we don't set any glibc guardpages, but for natively attaching threads we assume that the glibc guardpage is outside the stack area as reported by pthread_attr_getstack and pthread_attr_getstacksize.

However it seems glibc doesn't read the POSIX specification in the same way:

https://bugzilla.redhat.com/show_bug.cgi?id=435337

as a result hotspot's guard pages may overlap the glibc guard pages on attached threads.

Comments
We're not going to address this unless we have a specific instance or reproducer for this issue.
15-06-2015

Based on what we discovered about the OSX stack overflow handling (8009302) this might not be quite so harmless. If the OS detects that the trap occurs in what it considers to be the glibc guard pages then it might process the signal differently. We still have some open issues regarding stackoverflow detection not quite working the way people expect on Linux, and it is possible that this issue could be related. Certainly in light of what was discovered with OSX we should examine the Linux signal handling code in the same way.
04-06-2013

Appears to cause no harm since when attached the vm's signal handler should detect stack overflow and when unattached glibc's should. We should investigate in detail and clean up as part of the POSIX refactoring.
04-06-2013

This kind of spec non-compliance is a problem for us with regard to trying to factor out POSIX support. We need to check what BSD/OSX does here too.
18-03-2013