JDK-7107135 : Stack guard pages are no more protected after loading a shared library with executable stack
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: hs21,hs24,hs25
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: linux
  • CPU: generic
  • Submitted: 2011-11-01
  • Updated: 2016-04-20
  • Resolved: 2013-03-29
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8 Other
7u21Fixed 8Fixed hs24Fixed
Related Reports
Relates :  
JDK-8019583 - [TESTBUG] runtime/7107135 always passes
Relates :  
JDK-8081770 - [TESTBUG] regression Test7107135 needs to remove dependence on locally installed gcc
Relates :  
JDK-6675312 - Linux glibc stack guard-pages can overlap with hotspot guard pages
Description
Goetz Lindenmaier from SAP AG found an issue loading jni libraries where the stack is set to ���execstack���, or where this flag is not set at all.  After loading such a library, the stack guard pages are lost and though stack overflows can no more be detected.

see comments
Comments
Note: http://hg.openjdk.java.net/hsx/hsx24/hotspot/rev/19c1e132e9ee also contains the fixes in JDK-8010389.
08-05-2013
Need to backport from 8009588 -> HSX24: URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/9058789475af User: amurillo Date: 2013-03-15 22:31:21 +0000
18-03-2013
The current plan of action is: + import SAP patch + add a warning message whenever an "old" DLL is loaded + add diagnostic message in error log in case VM crashes due to an indirectly-loaded "old" DLL
07-02-2013
See also: http://mail.openjdk.java.net/pipermail/hotspot-dev/2010-August/003285.html David said the following on 08/29/10 09:03: > Douglas said the following on 08/28/10 23:42: >> I think the long term solution is to modify the >> __make_stacks_executable function in libc/nptl/allocatestack.c such >> that it only adds the PROT_EXEC bit to each thread's stack and leaves >> the current values for the PROT_READ and PROT_WRITE bits as is. > > I find it hard to believe that they would overwrite the permission bits > rather than augment them in the first place. I can't wait to find out > what justification they think they had for doing that. :( <sigh> Looks like it is because the OS gives no way to do otherwise :( mprotect doesn't allow you to add to the existing permissions only set them - and there's no API that I can see to query page permissions. There's some interesting discussion in this LK thread: http://linux.derkeiler.com/Mailing-Lists/Kernel/2005-02/0145.html David
31-01-2013
One possible fix for all but the main thread is for os::Linux::default_guard_size() to return (StackYellowPages+StackRedPages)*page_size() for java threads. However there is a comment saying guard pages are expensive, though it's not clear why letting pthreads do it is more expensive than letting hotspot do it. They both use mprotect. The reason this fix doesn't work for the main thread is because the main thread is created by the launcher so it doesn't call os::create_thread().
15-11-2012
See also: https://blogs.oracle.com/dns/entry/stacks_with_split_personalities
15-11-2012
PUBLIC COMMENTS -------- Original Message -------- Subject: Request for review [new bug](S): Stack guard pages are no more protected after loading a shared library with executable stack. Date: Wed, 26 Oct 2011 14:07:22 +0200 From: Lindenmaier, Goetz <###@###.###> Hi, I am Goetz Lindenmaier from SAP AG, working in the SAP JVM Jit Team. We found an issue loading jni libraries where the stack is set to ���execstack���, or where this flag is not set at all. After loading such a library, the stack guard pages are lost and though stack overflows can no more be detected. We had a lot of libraries in our test systems where the execstack flag was missing. This webrev contains a small test and a possible fix for the problem: http://www.sapjvm.com/gl/webrevs/noexecstack/ As I don���t have a bug ID yet, I just used an arbitrary number to make the test executable with jtreg . Please open a bug for this issue. I���ll fix the test if I have a proper number. This problem exists since 7019808, which adds ���z noexecstack to the linker command on linux. The fix I propose does the following: It reads the elf file to detect whether loading the library can change stack properties. If so, it requests a safepoint and loads the library during the safepoint. After loading the library, it tests the guard pages with SafeFetch. If they are no more protected, it reprotects the guard pages of all Java stacks. There might be cases where reading the elf file does not suffice to know that the stack access rights will change. As I understand, if a properly compiled jni library loads another library which requires execstack, the stack changes access rights, too. This is detected by the SafeFetch, and the stacks are fixed. But in this case the stacks are unprotected for a short time. To improve on this, one would have to request a SafePoint for each library loaded. But anyways, there is no bulletproof solution, as the loaded code could do an mprotect() at some point. Best regards, Goetz.
01-11-2011