JDK-6308388 : Linux gets irrecoverable stack error with ulimit set below -Xss<> option.
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 6
  • Priority: P4
  • Status: Closed
  • Resolution: Fixed
  • OS: linux
  • CPU: x86
  • Submitted: 2005-08-09
  • Updated: 2012-10-08
  • Resolved: 2005-09-28
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6
6 b54Fixed
Related Reports
Relates :  
JDK-6316197 - Java launcher should create JVM from non-primordial thread
Description
The regression test case for bug 4318701 says it passes on linux -server but it really fails and leaves an hs_err* file.  It seems Linux doesn't have the check that solaris does that you've specified too large of a stack space relative to your ulimit.

To reproduce:
% ulimit -s 2048
% java -Xss2049k -version
An irrecoverable stack overflow has occurred.
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#
[error occurred during error reporting, step 20, id 0xb]

, pid=6572, tid=3217604736
#
# Java VM: Java HotSpot(TM) Client VM (1.6.0-ea-b46 mixed mode, sharing)
# Problematic frame:
#
[error occurred during error reporting, step 60, id 0xb]

# An error report file with more information is saved as hs_err_pid6572.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

[error occurred during error reporting, step 270, id 0xb]

Aborted (core dumped)

On solaris:
% java -Xss2049k -version
Stack size of 2056 Kb exceeds current limit of 2048 Kb.
(Stack sizes are rounded up to a multiple of the system page size.)
See limit(1) to increase the stack size limit.
Comments
EVALUATION Fixed by reducing primordial stack slightly to give space for ld.so
20-09-2005
EVALUATION On Linux we do check if -Xss/ThreadStackSize is larger than ulimit value. If ThreadStackSize exceeds the system limit, the stack size is quietly reduced to ulimit value. The bug is actually in the OS, in particular, there appears to be a miscalculation of stack size in ld-linux.so.2. For example, if I set stack size to 1024K and stack starts from 0xc0000000, it should end at 0xc0000000 - 1024 * 1024 = 0xbff00000. However, see what actually happens on Redhat AS-3: bash-2.05b$ ulimit -s 1024 bash-2.05b$ cat /proc/self/maps 08048000-0804c000 r-xp 00000000 08:02 196364 /bin/cat 0804c000-0804d000 rw-p 00003000 08:02 196364 /bin/cat 0804d000-0806e000 rw-p 00000000 00:00 0 bfba0000-bfda0000 r--p 00000000 08:02 556061 /usr/lib/locale/locale-archive bfda0000-bfed1000 r-xp 00000000 08:02 147258 /lib/tls/libc-2.3.2.so bfed1000-bfed4000 rw-p 00130000 08:02 147258 /lib/tls/libc-2.3.2.so bfed4000-bfed8000 rw-p 00000000 00:00 0 bfeeb000-bff00000 r-xp 00000000 08:02 556059 /lib/ld-2.3.2.so bff00000-bff01000 rw-p 00015000 08:02 556059 /lib/ld-2.3.2.so bfffb000-c0000000 rwxp fffff000 00:00 0 The last line is main thread stack. Ignore the lower end (0xbfffb000), as main thread stack can grow on demand; it can extend all the way to ulimit limit (0xbff00000) when necessary. Note that ld-2.3.2.so has relocated its own data section to overlap with main thread's stack (0xbff00000-0xbff01000). This bug is hardly noticeable for applications that do not manage thread stacks. But for Java, because we need to throw StackOverflowError, JVM will install guard pages at the lower end of thread stacks. Now because ld.so's data segment sits at the bottom of primordial stack, it means ld.so's data section will be remapped as guard page. Any call into ld.so (e.g. dlopen()) will fault when it tries to access ld.so's data section. The SEGV is captured by JVM, but because it appears to happen within stack red zone, JVM thinks it's an irrecoverable stack overflow error and abort. For whatever reason, this bug only occurs in bash shell. Also if ulimit -s is larger than 2048K, because JVM will limit initial thread's stack size to 2M due to bug 4466587, there will be some buffer between ld.so's data section and JVM's stack guard, then JVM won't crash. Note that the problem will disappear for Java programs once 6316197 is fixed.
15-09-2005
WORK AROUND Several possible workarounds: 1. use tcsh, not bash 2. set ulmit -s > 2048 3. if you have to sset ulimit -s <= 2048, make sure it's larger than Java -Xss value
15-09-2005