JDK-7056092 : Native file dialog fails 1.6u26 win7Pro: Breaks SWT apps, perhaps Eclipse too.
  • Type: Bug
  • Component: client-libs
  • Sub-Component: java.awt
  • Affected Version: 6u26
  • Priority: P1
  • Status: Resolved
  • Resolution: Fixed
  • OS: windows_7
  • CPU: x86
  • Submitted: 2011-06-17
  • Updated: 2012-09-14
  • Resolved: 2011-07-18
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6
6u27 b07Fixed
Related Reports
Duplicate :  
Description
FULL PRODUCT VERSION :
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) Client VM (build 20.1-b02, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Windows 7 Professional ,64-bit.  ver: Microsoft Windows [Version 6.1.7600]

A DESCRIPTION OF THE PROBLEM :
Native file dialogs fail under some circumstances.  see
   https://bugs.eclipse.org/bugs/show_bug.cgi?id=349387

Problem appears only under 1.6u26 (either 32- or 64-bit variants).
Problem appears on Windows 7Pro (3 machines).  Unable to reproduce on Windows XP.


REGRESSION.  Last worked in version 6

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
  To repeat  https://bugs.eclipse.org/bugs/show_bug.cgi?id=349387:

-------------------
  To reproduce, use JNLP to run

http://www.eclipse.org/swt/jws/controlexample.jnlp

after launching, go to the tab "Dialog" and try to create a FileDialog. The
creation fails, and the dialog result is null (shown in the debug pane on the
left). Creation of any other dialog, such as ColorChooser etc, works fine.

-------------------

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
File dialog appears
ACTUAL -
No file dialog appears

ERROR MESSAGES/STACK TRACES THAT OCCUR :
No error.  Result of the call is null as if the user pressed cancel.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
See source code for

  http://www.eclipse.org/swt/jws/controlexample.jnlp
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
For us, we were able to avoid the bug by deferring access to java.awt.Color static field that provoked Color.<clinit> that loaded native library and called initIDs().  It did not help to call the static initializer before initializing the instance.
The test below PASSES using 6u25 b06 (GA), but FAILS using 6u26 b01 - and GA (b03)
It also PASSES using JDK 7.

PASSES means the file dialog appears. FAILS means it does not appear.

Tested on Windows 7 Ultimate 64 bit using 32 bit JREs and the latest
(June 2011) SWT.
-----
import org.eclipse.swt.SWT;
import org.eclipse.swt.widgets.FileDialog;
import org.eclipse.swt.widgets.Shell;

public class FileDialogTest {
	public static void main(String args[]) {
		Runtime.getRuntime().loadLibrary("awt");
		FileDialog fileDialog = new FileDialog(new Shell(), SWT.OPEN);
		fileDialog.setText("Choose a file or this test will fail!");
		fileDialog.open();
	}

}
---

Comments
EVALUATION Before the mentioned change comctl32.dll was loaded from c:\windows\winsxs (by ::LoadLibrary) and now it's loaded from c:\windows\system32 (by JDK_LoadSystemLibrary). I did some experiments to see if the line is the actual vulnerability or we can safely revert this particular line. In my environment comctl32.dll always loaded from winsxs folder. This is just a quick fix (revert this line), I'll experiment with the fix more and will submit a request for review tomorrow.
14-07-2011