Bug ID: JDK-6636099 toolkit initialization hangs

Type: Bug
Component: client-libs
Sub-Component: java.awt
Affected Version: 6

Priority: P1
Status: Closed
Resolution: Not an Issue
OS: solaris
CPU: generic

Submitted: 2007-11-30
Updated: 2010-05-08
Resolved: 2007-12-07

The java.awt.Desktop.isDesktopSupported method hangs when ssh to "lapwai.sfbay" with DISPLAY set to "lapwai:0.0".  lapwai may not support Desktop API (please check).

Regardless if the system supports Desktop API or not, the isDesktopSupported() should return true/false gracefully instead of hanging.

If DISPLAY is set to a valid system with desktop supported or an invalid value, it works fine.

Attached is the jstack -F -m output and here are the top few frames:

0xff2c12c0      __pollsys + 0x4
0xff262044      pselect + 0x1c8
0xff2623bc      select + 0xa0
0xd6b9d1ec      _XWaitForReadable + 0xdc
0xd6b9d01c      _XRead + 0xec
0xd6b9a598      XOpenDisplay + 0x534
0xfbeb604c      awt_init_Display + 0x290
0xfbeb63dc      Java_sun_awt_X11GraphicsEnvironment_initDisplay + 0x2c
0xfc00e1e8      * sun.awt.X11GraphicsEnvironment.initDisplay(boolean)
bci:940023280 (Interpreted frame)
0xfc00e194      * sun.awt.X11GraphicsEnvironment.initDisplay(boolean) bci:0
(Interpreted frame)
0xfc005a30      * sun.awt.X11GraphicsEnvironment.access$100(boolean) bci:1
line:52 (Interpreted frame)
0xfc005a30      * sun.awt.X11GraphicsEnvironment$1.run() bci:72 line:155
(Interpreted frame)
0xfc00021c      <StubRoutines>

EVALUATION Ok, lets list all situations we have: 1. you provide a valid display to which you are allowed to connect (either you own it or someone has called "xhost +" for you) Application starts and desktop API does work. (Great!) 2. you provide a valid display to which you are NOT allowed to connect. AWT can not initialize toolkit (XOpenDisplay fails) and we throw error, the same way as any other X application. (Still ok, since it is expected behavior for X applications). 3. you do provide an invalid display (which doesn't exist). AWT can not initialize toolkit and throws an error (usual way for X apps). For 2. and 3. you need either provide correct/accessible display, or use AWT in headless mode. In the latter case we Desktop API will throw HeadlessException (see the spec). And finally the case when we have the hang. 4. we start a computer and leave it in login screen. after that :0 display is partially valid :( Xserver has been started and it accepts requests from applications, but it looks like it doesn't respond until after someone log into this computer. In this case AWT hangs on XOpenDiaplay() call. I have tried xlogo and xclock and both demonstrate the same behavior. SO, it looks like it is a common situation for all X-apps. So, for 1-4 cases we have the same behavior as native apps. 1-3 are ok since they provide immediate information what the problem was. 4 is kind of problematic, but I suspect this is a by-design behavior for our X-server (and, most likely, exists for very long time). Thus I'm closing the problem as "not an AWT defect". If someone has good explanation why this is a bug in a way Solaris implements a login screen, feel free to either reopen/reassign this CR or open new one.
07-12-2007
EVALUATION What is the behavior of native applications with the same (not started) X server? Do they gracefully report an error and exit or hang like Java?
07-12-2007
EVALUATION As you and some other expect, it's fairly common for Solaris server to not have the Xserver running, even if it has a graphics card. This situation needs to be handled properly. The isDesktopSupported() should return false in this case instead of hanging nor throw an InternalError. I'm reopening this CR for further investigation.
06-12-2007
EVALUATION I've experimented with local Solaris10 machine and found that the described problem is reproducible when Xserver is not started (e.g. nobody logged in this machine locally) And it looks like XOpenDisplay() waits too long before reporting an error. Thus, I believe we have nothing to do here :( Thus I'm closing the problem as not a defect. If I've missed something feel free to reopen with additional information
03-12-2007
EVALUATION I have logged in lapwai and verified that te problem is with initializing display connection not with Desktop API. I wonder if it is possible to show anything to lapwai:0.0. Unfortunately I can not do this (I'm not in sfbay). Hope submitter can and will provide more information. BTW can you reproduce the same problem on other hosts?
30-11-2007
EVALUATION based on the stack I'd say that initialization of AWT display hangs (not a desktop call) I wonder if submitter can show some toplevel on the display which was used for the test. Another possible minimal test to try is public class minimal_test { public static void main(String[] args) { Toolkit.getDefaultToolkit(); } }
30-11-2007