United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-4684966 : infinite loop in Win32GraphicsEnv.initDisplay() when NetSupport is running

Details
Type:
Bug
Submit Date:
2002-05-14
Status:
Closed
Updated Date:
2003-12-02
Project Name:
JDK
Resolved Date:
2002-11-12
Component:
client-libs
OS:
windows_nt,windows_2000
Sub-Component:
2d
CPU:
x86
Priority:
P4
Resolution:
Fixed
Affected Versions:
1.4.0,1.4.1_02
Fixed Versions:
1.4.1_07 (07)

Related Reports
Backport:
Duplicate:

Sub Tasks

Description
When a machine is running as a connected NetSupport client, it is possible
to hang during any graphical Java app startup in Win32GraphicsEnv.initDisplay().
In particular, this happens when the client is set to 32 bpp display depth.

To reproduce this bug, install a demo of NetSupport manager 
(www.netsupport-inc.com) on two machines: one will be the client (where the
problem will appear) and one will be the controller which connects to the
client.  Using the client configurator, name the client machine.  Using the
controller, connect to that client.  (Be sure to run in 32bpp display
depth; and don't switch display modes while connected in NetSupport; this
crashed my system consistently (nothing to do with Java on this one...)).

Now, on the client (either locally or remotely), run a Java app and note that
it hangs before displaying anything.  If you type ctrl-Break in the console,
you will probably see one of the threads hung in Win32GraphicsEnv.initDisplay().

                                    

Comments
WORK AROUND

This bug should only occur in 32 bpp screen depth.  If users run in any
other depth (8, 15, 16, 24), things should work fine.
###@###.### 2002-05-16
                                     
2002-05-16
EVALUATION

This hang is due to an infinite loop in AwtWin32GraphicsDevice::Initialize().

NetSupport does something very funky when it connects to the client; it fakes
out the OS to tell it that there are two monitors.  When a Java app runs, we
set up color models for each device/monitor on the system.  For this second
NetSupport monitor, we go through the same Initialize() function that we 
always do, only we get an error this time when calling GetDIBits().

We assert on this error in debug mode, but otherwise do nothing about it.
In particular, we assume that the values set in this function are valid and
we continue on about our business in that function.  The problem is that we
spin in loops around the red/green/blue values that were supposed to be set
in that function, waiting for a certain condition to be true (we shift
the values until they are non-null).  But since the function error'd out, the
values are not set, and we could end up spinning in a loop forever
(basically, waiting for a value of "0" to become non-null).

It's not clear why GetDIBits() is failing in this particular situation, although
it seems like a bug in the NetSupport code; we are passing in valid values
to the function and they work on every other platform.  But for some 
reason, the NetSupport code does not like to be called with the biCompression
field in the BITMAPINFOHEADER structure set to BI_BITFIELDS when the display
depth is 32 bpp.  This is a valid value to pass, but it just doesn't work 
in this situation.

There are 2 fixes that should be applied to our code to fix this and
future problems:
	- trap the error and do something intelligent about it.  We could
	and should at least set some default values for the rgb masks and 
	exit the function cleanly without spinning in these loops.
	- Move the 32 bpp case down to the 24 bpp case; let them both use
	BI_RGB instead of 32 bpp using BI_BITFIELDS.  Both of these are
	acceptable values for 32 bpp; the BI_BITFIELDS is a little more 
	general, but the masks for 32 bpp are pre-ordained anyway, so I don't
	know why we need that flexibility in our code.

Of course it would be nice if NetSupport fixed the bug in their app as well,
but that does not fix the whole problem and we cannot count on that
for all of our customers.

###@###.### 2002-05-14

I will leave the code basically the way it is (leaving the 32 bpp case
as a BI_BITFIELDS case), and just handle the error in GetDIBits by
hardcoding the results.

By leaving the code basically as is, we run less risk of breaking something
else.  For example, the SGI NT machines use a bizarre (or at least uncommon)
framebuffer format of RGBX.  If I hardcode 32 bpp bitmasks for colormap
creation, will this cause bad color effects on such framebuffers?  By
simply handling the error instead, I will essentially only change the
functionality on cases which, by definition, do not currently work, thus
leaving existing/working platforms (such as the SGI machine) alone.

###@###.### 2002-05-16

Note to JDC submitters: This bug is fixed in a release (mantis, or 1.4.2)
that is not yet released.  Thus you will not see the fix until you can
download the release that has the fix.  There should be a beta available for
1.4.2 soon; when it is available, try that out and your problems should 
go away...

###@###.### 2003-02-14
                                     
2003-02-14
CONVERTED DATA

BugTraq+ Release Management Values

COMMIT TO FIX:
1.4.1_07
generic
mantis

FIXED IN:
1.4.1_07
mantis

INTEGRATED IN:
1.4.1_07
mantis
mantis-b07

VERIFIED IN:
1.4.1_07


                                     
2004-06-14



Hardware and Software, Engineered to Work Together