United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-7107135 Stack guard pages are no more protected after loading a shared library with executable stack
JDK-7107135 : Stack guard pages are no more protected after loading a shared library with executable stack

Details
Type:
Bug
Submit Date:
2011-11-01
Status:
Closed
Updated Date:
2013-07-28
Project Name:
JDK
Resolved Date:
2013-03-29
Component:
hotspot
OS:
linux
Sub-Component:
runtime
CPU:
generic
Priority:
P3
Resolution:
Fixed
Affected Versions:
hs21,hs24,hs25
Fixed Versions:
hs24 (b55)

Related Reports
Backport:
Backport:
Backport:
Backport:
Backport:
Backport:
Backport:
Backport:
Duplicate:
Duplicate:
Relates:
Relates:
Relates:
Relates:

Sub Tasks

Description
Goetz Lindenmaier from SAP AG found an issue loading jni libraries where the stack is set to ???execstack???, or where this flag is not set at all.  After loading such a library, the stack guard pages are lost and though stack overflows can no more be detected.

see comments

                                    

Comments
PUBLIC COMMENTS

-------- Original Message --------
Subject: 	Request for review [new bug](S): Stack guard pages are no more
protected after loading a shared library with executable stack.
Date: 	Wed, 26 Oct 2011 14:07:22 +0200
From: 	Lindenmaier, Goetz <###@###.###>



Hi,


I am Goetz Lindenmaier from SAP AG, working in the SAP JVM Jit Team.

We found an issue loading jni libraries where the stack is set to
???execstack???, or where this flag is not set at all.  After loading such a library, the
stack guard pages are lost and though stack overflows can no more be detected.

We had a lot of libraries in our test systems where the execstack flag was missing.


This webrev contains a small test and a possible fix for the problem:

http://www.sapjvm.com/gl/webrevs/noexecstack/

As I don???t have a bug ID yet, I just used an arbitrary number to make the test executable with jtreg .  Please open a bug for this issue. 

I???ll fix the test if I have a proper number.

This problem exists since 7019808, which adds ???z noexecstack to the linker command on linux. 



The fix I propose does the following: It reads the elf file to detect whether loading the library can change stack properties.  If so, it requests a safepoint and loads the library during the safepoint.  After loading the library, it tests the guard pages with SafeFetch.  If they are no more protected, it reprotects the guard pages of all Java stacks.

 

There might be cases where reading the elf file does not suffice to know that the stack access rights will change.

As I understand, if a properly compiled jni library loads another library which requires execstack, the stack changes access rights, too.

This is detected by the SafeFetch, and the stacks are fixed.  But in this case the stacks are unprotected for a short time.

To improve on this, one would have to request a SafePoint for each library loaded.  But anyways, there is no bulletproof solution, as the loaded code could do an mprotect() at some point.

Best regards,
  Goetz.
                                     
2011-11-01
See also: https://blogs.oracle.com/dns/entry/stacks_with_split_personalities
                                     
2012-11-15
One possible fix for all but the main thread is for os::Linux::default_guard_size() to return (StackYellowPages+StackRedPages)*page_size() for java threads.  However there is a comment saying guard pages are expensive, though it's not clear why letting pthreads do it is more expensive than letting hotspot do it.  They both use mprotect.

The reason this fix doesn't work for the main thread is because the main thread is created by the launcher so it doesn't call os::create_thread().
                                     
2012-11-15
See also: http://mail.openjdk.java.net/pipermail/hotspot-dev/2010-August/003285.html

David  said the following on 08/29/10 09:03:
> Douglas  said the following on 08/28/10 23:42:
>> I think the long term solution is to modify the
>> __make_stacks_executable function in libc/nptl/allocatestack.c such
>> that it only adds the PROT_EXEC bit to each thread's stack and leaves
>> the current values for the PROT_READ and PROT_WRITE bits as is.
> 
> I find it hard to believe that they would overwrite the permission bits 
> rather than augment them in the first place. I can't wait to find out 
> what justification they think they had for doing that. :(

<sigh> Looks like it is because the OS gives no way to do otherwise :( 
mprotect doesn't allow you to add to the existing permissions only set 
them - and there's no API that I can see to query page permissions.

There's some interesting discussion in this LK thread:

http://linux.derkeiler.com/Mailing-Lists/Kernel/2005-02/0145.html

David

                                     
2013-01-31
The current plan of action is:

+ import SAP patch

+ add a warning message whenever an "old" DLL is loaded

+ add diagnostic message in error log in case VM crashes
  due to an indirectly-loaded "old" DLL

                                     
2013-02-07
Need to backport from 8009588 -> HSX24:

URL: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/9058789475af
User: amurillo
Date: 2013-03-15 22:31:21 +0000
                                     
2013-03-18
URL:   http://hg.openjdk.java.net/hsx/hsx24/hotspot/rev/19c1e132e9ee
User:  zgu
Date:  2013-03-29 16:53:44 +0000

                                     
2013-03-29
Note:  http://hg.openjdk.java.net/hsx/hsx24/hotspot/rev/19c1e132e9ee also contains the fixes in JDK-8010389.
                                     
2013-05-08



Hardware and Software, Engineered to Work Together