Bug ID: JDK-4474311 64 bits VM asserts when stack size set to unlimited

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

Other
1.4.2 mantisFixed

When I try to run tests for 64 bits VM, with stack size set to
unlimited, all tests failed with following assertions. It works fine 
with 32 bits VM or when stack size reset to 8192.( ulimit -s 8192)

# ulimit -s unlimited
# ulimit -a
core file size (blocks)     0
data seg size (kbytes)      unlimited
file size (blocks)          unlimited
open files                  256
pipe size (512 bytes)       10
stack size (kbytes)         unlimited
cpu time (seconds)          unlimited
max user processes          7925
virtual memory (kbytes)     unlimited
# /net/jdk.eng/export/disk6/promoted/jdk/1.4/beta_refresh/b69/binaries/solsparc/bin/java_g -d64 -version
#
# HotSpot Virtual Machine Error, assertion failure
# Please report this error at
# http://java.sun.com/cgi-bin/bugreport.cgi
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.4.0-beta_refresh-b69-debug mixed mode)
#
# assert((address)&st > (address)st.ss_sp-st.ss_size, "Invalid stack size returned")
#
# Error ID: /net/mersparc.eng/usr/re/hotspot/hotspot1.4/ws/src/os/solaris/vm/os_solaris.cpp, 158 [ Unknown ]
#
Dumping core....
root[sh]@hotsparc# /net/jdk.eng/export/disk6/promoted/jdk/1.4/beta_refresh/b69/binaries/solsparc/bin/java -server -version
java version "1.4.0-beta_refresh"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-beta_refresh-b69)
Java HotSpot(TM) Server VM (build 1.4.0-beta_refresh-b69, mixed mode)


Name: rmT116609			Date: 01/30/2002


DESCRIPTION OF THE PROBLEM :

Using the -d64 flag causes the VM to crash on Solaris2.8, Solaris 9 beta.  

$limit stacksize unlimited
$java -d64 -version
#
# HotSpot Virtual Machine Error, Internal Error
# Please report this error at
# http://java.sun.com/cgi-bin/bugreport.cgi
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.4.0-rc-b91 mixed mode)
#
# Error ID: 5448524541442C4F43414C33544F524147450E4350500023 FF
#
# Problematic Thread: 
#
Abort (core dumped)


This bug can be reproduced always.
(Review ID: 138710)
======================================================================

CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: mantis FIXED IN: mantis INTEGRATED IN: mantis VERIFIED IN: mantis-beta
14-06-2004
PUBLIC COMMENTS Verified bug in Mantis build 15
10-06-2004
EVALUATION In the 32-bit libthread, thr_stksegment returns a 'reasonable' value of 0x80000 for the thread stack size if the maximum stack size is set to 'unlimited' by a user. The 64-bit libthread, on the other hand, returns 0xfffffffffffffffd. The assert is triggered because adding the latter value to the stack base wraps the address space. In product mode, we just die a little later because of a product-mode assert. We could fix it by checking for the silly value and using 0x10000 (double the 32-bit maximum) instead. It's a small, easy and safe fix, but not essential for Merlin since there is a workaround. The real fix should be in libthread. paul.hohensee@East 2001-08-02 On solaris-sparc64, if the stack size limit was set to 'unlimited' by the user, thr_stksegment and getrlimit both returned 0fffffffffffffffd, triggering a guarantee in ThreadLocalStorage::get_thread_via_cache_slowly. The corresponding 32-bit value is 0x7fffffff == MAXINT. Fix is to sanity check the result of thr_stksegment and getrlimit and set it to max_intx (== 0x7fffffffffffffff in 64-bit and 0x7fffffff in 32-bit) if the sign bit in the original value is set. ###@###.### 2002-10-18
18-10-2002
WORK AROUND Don't set the stack size to unlimited when running the 64-bit vm. paul.hohensee@East 2001-08-02
02-08-2001

Duplicate :	JDK-4759953 - java dumps core if it has an unlmited stacksize
Relates :	JDK-4812466 - VM crash when stack size set to unlimited on solx86 platform.
Relates :	JDK-6316197 - Java launcher should create JVM from non-primordial thread