-------- Original Message --------
Subject: Request for review [new bug](S): Stack guard pages are no more
protected after loading a shared library with executable stack.
Date: Wed, 26 Oct 2011 14:07:22 +0200
From: Lindenmaier, Goetz <###@###.###>
I am Goetz Lindenmaier from SAP AG, working in the SAP JVM Jit Team.
We found an issue loading jni libraries where the stack is set to
???execstack???, or where this flag is not set at all. After loading such a library, the
stack guard pages are lost and though stack overflows can no more be detected.
We had a lot of libraries in our test systems where the execstack flag was missing.
This webrev contains a small test and a possible fix for the problem:
As I don???t have a bug ID yet, I just used an arbitrary number to make the test executable with jtreg . Please open a bug for this issue.
I???ll fix the test if I have a proper number.
This problem exists since 7019808, which adds ???z noexecstack to the linker command on linux.
The fix I propose does the following: It reads the elf file to detect whether loading the library can change stack properties. If so, it requests a safepoint and loads the library during the safepoint. After loading the library, it tests the guard pages with SafeFetch. If they are no more protected, it reprotects the guard pages of all Java stacks.
There might be cases where reading the elf file does not suffice to know that the stack access rights will change.
As I understand, if a properly compiled jni library loads another library which requires execstack, the stack changes access rights, too.
This is detected by the SafeFetch, and the stacks are fixed. But in this case the stacks are unprotected for a short time.
To improve on this, one would have to request a SafePoint for each library loaded. But anyways, there is no bulletproof solution, as the loaded code could do an mprotect() at some point.