Summary
-------
Add a new (Linux-only) product flag, `AdjustStackSizeForTLS`, to adjust the thread stack size by the amount of on-stack thread-local storage (TLS) that would be taken by glibc, when creating a new thread. This flag is disabled by default.
Problem
-------
The glibc library will allocate some thread-local storage in the stack of a newly created thread, leaving less stack than was requested for the thread to do its work. This is particularly a problem for threads with small stack size. It is an inherited issue from a well-known glibc problem, 'Program with large TLS segments fail' [0]. One concrete example of the TLS usage is the compiler injected `__thread` variables (see gcc `tree-profile.c` [1]). The on-stack static TLS blocks may leave insufficient stack space when the user requested stack size is not large enough. This problem has also been seen in Java applications. In one of the reported JVM failure instances, the issue manifests as a `StackOverflowError` on the process reaper thread [2] [3], which has a small stack size (with JDK 8 the size is 32K, with new versions of JDK it is 128K). The `java.lang.Thread` constructor [5] allows users to specify the stack size for a new thread. The created thread may encounter the TLS problem when the specified size is too small to accommodate the on-stack TLS blocks. This issue may cause symptoms that are difficult to diagnose. As the issue has not been addressed in glibc, we are proposing to handle it in the JVM.
Solution
--------
When creating a new thread within the JVM on Linux, obtain the static on-stack TLS size and add it to the user requested stack size if `AdjustStackSizeForTLS` is enabled. The static TLS size is derived from the pthread min-stack size (result returned from `__pthread_get_minstack()`) by subtracting the non-TLS usage. As a result, the created thread has a larger stack that can accommodate the static TLS blocks [4]. By default, thread stack sizes are not adjusted for TLS.
Specification
-------------
The `AdjustStackSizeForTLS` command-line option may be used to enable thread stack size adjustment for TLS on Linux:
-XX:+AdjustStackSizeForTLS
When creating a new thread, if `AdjustStackSizeForTLS` is true, the static TLS area size is added to the user requested stack size.
References
------------
[0] http://sourceware.org/bugzilla/show_bug.cgi?id=11787
[1] https://github.com/gcc-mirror/gcc/blob/master/gcc/tree-profile.c
[2] https://bugs.openjdk.java.net/browse/JDK-8130425
[3] http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037558.html
[4] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2019-May/034459.html
[5] https://download.java.net/java/early_access/jdk13/docs/api/java.base/java/lang/Thread.html