Bug ID: JDK-8169373 Work around linux NPTL stack guard error

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 9
9 b151Fixed

In the Linux NPTL pthread implementation the guard size mechanism is not implemented properly. The posix standard requires to add the size of the guard pages to the stack size, instead Linux takes the space out of 'stacksize'. 

The Posix standard http://pubs.opengroup.org/onlinepubs/9699919799/
says "the implementation allocates extra memory at the overflow end of the stack". The linux man page https://linux.die.net/man/3/pthread_attr_setguardsize says "As at glibc 2.8, the NPTL threading implementation includes the guard area within the stack size allocation, rather than allocating extra space at the end of the stack, as POSIX.1 requires".

I encounter this problem in runtime/Thread/TooSmallStackSize.java on ppc with 64K pages. _compiler_thread_min_stack_allowed is 128K on ppc, and ppc specifies two OS guard pages.  The VM crashes in pthread creation because there is no usable space
in the thread stack after allocating the guard pages.

This fix adapts the requested stack size on Linux by the size of the guard pages to mimick proper behaviour.

I have opened JDK-8288631 to discuss how to handle the workaround code for glibc 2.27+. Let us know what you think.
16-06-2022
[~simonis] I have fixed the comment in JDK-8280059.
17-01-2022
The glibc NPTL bug which subtracted the guard size from the thread stack size instead of adding it on top of it has been fixed in glibc 2.27 (see https://sourceware.org/bugzilla/show_bug.cgi?id=22637). This leads to slightly bigger stack sizes for non-Java threads if running an glibc 2.27+ but I think the effect is negligible and only affects non-Java threads because Java threads don't use glibc guard pages. "JDK-8225035: Thread stack size issue caused by large TLS size" mentioned this in a comment (https://github.com/openjdk/jdk/blob/299022dfacbcb49e3bc5beca8ff9b1fca1101493/src/hotspot/os/linux/os_linux.cpp#L809) but used the wrong glibc version "2.7" instead of "2.27".
28-12-2021
URL: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/548cb3b7b713 User: lana Date: 2017-01-04 20:58:29 +0000
04-01-2017
URL: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/548cb3b7b713 User: dcubed Date: 2016-12-21 04:45:09 +0000
21-12-2016
Dan: until such time as an unchanged repo gets a clean RBT run, chasing failures here seems like a waste of your time. This additional level of testing was never intended to be such a resource drain.
20-12-2016
JDK-6675312 is based on the same bug in linux glibc. But it adresses "natively attaching threads", while I already see problems with threads created by the VM. This also is the issue I want to fix with this change.
10-11-2016
Existing issue JDK-6675312 was closed as will-not-fix.
09-11-2016
This has been discussed in the past and I'm pretty sure we have an open issue related to this ... will try to find it and/or previous discussions.
08-11-2016

Relates :	JDK-8280059 - Incorrect glibc version is used in a comment in os_linux.cpp
Relates :	JDK-8175342 - assert(InstanceKlass::cast(k)->is_initialized()) failed: need to increase java_thread_min_stack_allowed calculation
Relates :	JDK-6675312 - Linux glibc stack guard-pages can overlap with hotspot guard pages
Relates :	JDK-8273373 - Zero: Cannot invoke JVM in primordial threads on Zero
Relates :	JDK-8170655 - [posix] Fix minimum stack size computations
Relates :	JDK-8140520 - segfault on solaris-amd64 with "-XX:VMThreadStackSize=1" option
Relates :	JDK-8288631 - Avoid applying Linux NPTL stack guard workaround for glibc 2.27+