JDK-6885308 : The incorrect -XX:StackRedPages, -XX:StackShadowPages, -XX:StackYellowPages could cause VM crash
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: hs17
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: generic
  • CPU: x86
  • Submitted: 2009-09-24
  • Updated: 2011-03-08
  • Resolved: 2011-03-08
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7 Other
6u25Fixed 7Fixed hs20Fixed
Related Reports
Relates :  
Description
Some combinations of -XX:StackRedPages, -XX:StackShadowPages, -XX:StackYellowPages causes VM crash. 
Also VM hangs if StackShadowPages=0.
VM should correctly report about incorrect values. Please note that these options are described in Sun official guide http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/html/crashes.html.
Here is the example (I checked on solaris-i586 only):
-->java -XX:StackShadowPages=70
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (os_solaris.cpp:4865), pid=4199, tid=2
#  Error: guarantee((Solaris::min_stack_allowed >= (StackYellowPages+StackRedPages+StackShadowPages+BytesPerWord +1) * page_size),"need to increase Solaris::min_stack_allowed on this platform")
#
# JRE version: 7.0-b72
# Java VM: Java HotSpot(TM) Server VM (17.0-b01 mixed mode solaris-x86 )
# An error report file with more information is saved as:
# /tmp/xx/hs_err_pid4199.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
Abort (core dumped)

Comments
EVALUATION http://hg.openjdk.java.net/jdk7/hotspot/hotspot/rev/f8c5d1bdaad4
23-08-2010

EVALUATION Added check for shadow/red/yellow pages >0. We should have changed that assert to a vm_exit_during_initialization() while we were doing this.
20-08-2010

EVALUATION http://hg.openjdk.java.net/jdk7/hotspot-rt/hotspot/rev/f8c5d1bdaad4
20-08-2010