JDK-6603312 : Segmentation fault running java -jar SwingSet2.jar in 256 color mode
  • Type: Bug
  • Component: client-libs
  • Sub-Component: java.awt
  • Affected Version: 6u2
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: linux
  • CPU: x86
  • Submitted: 2007-09-11
  • Updated: 2011-05-17
  • Resolved: 2011-05-17
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7
7 b27Fixed
Related Reports
Relates :  
JDK-6603311 - Unable to read java6 rendered UI on SLES10 PPC32 in 256 color mode
Relates :  
JDK-6685121 - (launcher) make ReportErrorMessages accessible by other launcher subsystems
Relates :  
JDK-6684361 - splashscreen.[so|dll] error messages should be L10N'able, and be able to be reported under javaw
Description
OPERATING SYSTEM(S):
--------------------
Linux AMD64

FULL JDK VERSION(S):
-------------------
java version "1.6.0_02"
Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_02-b05, mixed mode)


STEPS TO REPRODUCE
--------------------
- set the display settings of graphics card to 256 color mode
- run java -jar Swingset2.jar and you will see segmentation fault.
Comments
EVALUATION The reason the XAllocColorCells() function fails is the fact that the X server has its available color cells exhausted. Probably we need to force hiding the splash screen if the code cannot allocate any reasonable number of colors on PseudoColor displays.
25-03-2008
SUGGESTED FIX --- a/src/solaris/native/sun/awt/splashscreen/splashscreen_sys.c +++ b/src/solaris/native/sun/awt/splashscreen/splashscreen_sys.c @@ -436,6 +436,7 @@ SplashInitPlatform(Splash * splash) { break; } case PseudoColor: { + int availableColors; int numColors; int numComponents[3]; unsigned long colorIndex[SPLASH_COLOR_MAP_SIZE]; @@ -444,9 +445,20 @@ SplashInitPlatform(Splash * splash) { int depth = XDefaultDepthOfScreen(splash->screen); int scale = 65535 / MAX_COLOR_VALUE; - numColors = GetNumAvailableColors(splash->display, splash->screen, + availableColors = GetNumAvailableColors(splash->display, splash->screen, splash->visual->map_entries); - numColors = quantizeColors(numColors, numComponents); + numColors = quantizeColors(availableColors, numComponents); + if (numColors > availableColors) { + // Could not allocate the color cells. Most probably + // the pool got exhausted. Disable the splash screen. + XCloseDisplay(splash->display); + splash->isVisible = -1; + splash->display = NULL; + splash->screen = NULL; + splash->visual = NULL; + fprintf(stderr, "Warning: unable to initialize the splashscreen. Not enough available color cells.\n"); + return; + } splash->cmap = AllocColors(splash->display, splash->screen, numColors, colorIndex); for (i = 0; i < numColors; i++) {
25-03-2008
EVALUATION The segmentation fault happens at the initColorCube() function (src/share/native/sun/awt/splashscreen/splashscreen_gfx_impl.c) at line pColorMap[colorIndex[n++]] = ... . This happens because the colorIndex[n++] has a big numerical value, and is out of bounds of the pColorMap[] array. The colorIndex[] is the splash->colorIndex field, which gets initialized at SplashInitPlatform() at src/solaris/native/sun/awt/splashscreen/splashscreen_sys.c. Which in turn invokes the AllocColors() function. This function uses the XAllocColorCells() X11 function. And the call returns the status of zero, which indicates a failure according to the documentation. Finally, the splash->colorIndex does not get initialized properly. That may be the reason it contains some undefined data.
24-03-2008
EVALUATION According to the initial investigation it's very likely the problem is in splash screen in 256 color mode. If SwingSet2 is run without splash screen all work fine - without segmentation faults.
15-10-2007
EVALUATION This is unlikely to be a problem with amd64. Experiments show that the problem can easily be reproduced while running on linux-i586. Probably this problem is related to manipulating the X11 visuals when operating on 256-color displays.
15-10-2007