JDK-6515708 : IE Browser hangs after opening a file dialog
  • Type: Bug
  • Component: client-libs
  • Sub-Component: java.awt
  • Affected Version: 1.4.2,6,6u1
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: windows_xp,windows_vista
  • CPU: x86
  • Submitted: 2007-01-22
  • Updated: 2011-03-07
  • Resolved: 2011-03-07
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7
6u2Fixed 7 b10Fixed
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.6.0"
Java(TM) SE Runtime Environment (build 1.6.0-b105)
Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows Vista Ultimate, Version 6.0, Build 6000.


EXTRA RELEVANT SYSTEM CONFIGURATION :
Internet connected system.

A DESCRIPTION OF THE PROBLEM :
After closing a file dialogue opened from an applet, the browser showing the applet GUI stops responding. The same behaviour is shown regardless of whether a file is chosen from the dialogue, the "Cancel" button is pressed or the "Ok" button is pressed.

The problem only occurs when the dialogue window opened LAST is closed FIRST. For instance, assume that the same applet is run in two separate browsers B1 and B2. From B1 file dialogue F1 is opened. Subsequently, file dialogue F2 is opened from B2. Now, if F2 is closed before F1, browser B2 hangs. If, on the other hand, F1 is closed before F2 is closed, neither B1 nor B2 hangs.

Additionally, if the Java Console is open during the time when file dialogues are opened and closed, the problem does not occur.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1) Start an applet in a browser. The applet must contain functionality which opens a file dialogue.
2) Make sure that the Java Console is not open.
3) Open a file dialogue from the applet.
4) Terminate the file dialogue by either closing it or choosing a file.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
That the browser/applet should regain focus instead of locking up.
ACTUAL -
The browser hangs and is unresponsive. You cant even resize or close the browser window.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
nothing.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
Not available.
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
If you set focus on an application other than the browser and then select the browser again, it unlocks and you can continue.

If you have the process window active (after pressing ctrl-alt-del and starting it from there ) you wont get the problem
Description of the Problem: After closing a file dialogue opened from an applet, the browser showing the applet GUI stops responding. The same behaviour is shown regardless of whether a file is chosen from the dialogue, the "Cancel" button is pressed or the "Ok" button is pressed.

The problem only occurs when the dialogue window opened LAST is closed FIRST. For instance, assume that the same applet is run in two separate browsers B1 and B2. From B1 file dialogue F1 is opened. Subsequently, file dialogue F2 is opened from B2. Now, if F2 is closed before F1, browser B2 hangs. If, on the other hand, F1 is closed before F2 is closed, neither B1 nor B2 hangs.

Additionally, if the Java Console is open during the time when file dialogues are opened and closed, the problem does not occur.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1) Start an applet in a browser. The applet must contain functionality which opens a file dialogue.
2) Make sure that the Java Console is not open.
3) Open a file dialogue from the applet.
4) Terminate the file dialogue by either closing it or choosing a file.


REPRODUCIBILITY :
This bug can be reproduced always.

CUSTOMER SUBMITTED WORKAROUND :
When the browser hangs, make a mouse click outside the browser. This causes the browser to reassume normal execution.

Source code of Test Case:

import java.applet.Applet;
import java.awt.BorderLayout;
import java.awt.Button;
import java.awt.Container;
import java.awt.FileDialog;
import java.awt.Frame;
import java.awt.Panel;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

/*
 * Created on 11-01-2007
 */

/**
 * This is an applet with a button which opens a file dialogue.
 * @author Lars Elmegaard-Fessel
 */
public class TestApplet extends Applet implements ActionListener {
 
        /**
         * Initialization - set up panel and button.
         */
        public void init(){
                Panel panel = new Panel();
                Button btn = new Button();
                btn.setLabel("Open File Dialogue");
                btn.addActionListener(this);
                panel.add(btn);
                add(panel,BorderLayout.CENTER);
        }

        /* (non-Javadoc)
         * @see 
java.awt.event.ActionListener#actionPerformed(java.awt.event.ActionEvent)
         */
        public void actionPerformed(ActionEvent arg0) {
 
                //Get the frame
                Container c = this;
                while (c != null && !(c instanceof Frame))
                        c = c.getParent();
                //Open a dialogue.
                FileDialog dialog = new FileDialog((Frame) c, 
"Dialog",FileDialog.LOAD);
                dialog.show();
        }
}

Steps to run the test case:

Start the applet.
Press the "Open File Dialogue" button. 
Close the file dialogue window.

Expected behaviour:

The file dialogue window is closed and the browser window containing the 
applet regains focus.

Actual behaviour:

The browser hangs. It does not respond to mouse clicks or keyboard events. 
Additionally, if the file dialogue window was closed while being in front 
of the browser window, some of the file dialogue window remains in the 
browser window.

Comments
EVALUATION I suppose that problem exists somewhere inside file dialog implementation. While looking into AWT code i could observe that ::GetOpenFileName() call hangs on toolkit thread and so user can't interact with applet. After installing symbols on Vista native stack trace becomes available. Here's the information for the toolkit thread: 0701e054 769196ec ntdll!KiFastSystemCallRet 0701e058 7691aa93 USER32!NtUserCallHwndLock+0xc 0701e068 1010931e USER32!SetForegroundWindow+0xf 0701e07c 76913607 awt!AwtDialog::ModalFilterProc+0x8e 0701e098 7691943b USER32!DispatchHookW+0x33 0701e0bc 76ed0e6e USER32!__fnHkINLPCBTACTIVATESTRUCT+0x22 0701e0e8 76918c32 ntdll!KiUserCallbackDispatcher+0x2e 0701e0ec 762c7211 USER32!NtUserDestroyWindow+0xc 0701e160 762b5e4f MSCTF!CheckQueueOnLastWindowDestroyed+0x16a 0701e19c 7691ac92 MSCTF!TF_Notify+0x16f 0701e1b0 7691ac38 USER32!CtfHookProcWorker+0x21 0701e1f0 76917cf5 USER32!CallHookWithSEH+0x21 0701e214 76ed0e6e USER32!__fnHkINDWORD+0x24 0701e240 76918c32 ntdll!KiUserCallbackDispatcher+0x2e 0701e244 76911355 USER32!NtUserDestroyWindow+0xc 0701e278 76911450 USER32!DialogBox2+0x252 0701e2a0 76911492 USER32!InternalDialogBox+0xd0 0701e2c0 769114f5 USER32!DialogBoxIndirectParamAorW+0x37 0701e2e0 7108feec USER32!DialogBoxIndirectParamW+0x1b 0701e300 7573488a IEFRAME!Detour_DialogBoxIndirectParamW+0x2d The line number four tells us that we are inside ModalFilterProc hook procedure, the procedure is the part of modality implementation. As soon as WM_ACTIVATE (or WM_SETFOCUS) event is coming on blocked window into the procedure, the procedure activates blocker window via ::SetForegroundWindow(blocker). In the attached testcase the blocker window is file dialog and blocked window is browser. When i run the spy tool on blocker window (file dialog), i could observe enormous number of WM_ACTIVATE events are coming on file dialog after it's closing. After click on another window the WM_DESTROY message is coming to file dialog and that unfreezes browser. We may not activate blocker window if the window is not visible. This change fixes the issue. And it looks reasonable for me. However, there are questions which are still open. Like, the problem is reproducible only on Windows Vista (it works fine on Window XP), why? Also, the problem appears only for browser and file dialog (it works fine if i replace file dialog or browser on dialog). Moreover, the bug is not reproducible on JDK7 b04. the fix for 6488834 (it specifies owner for file dialog) eliminates the issue. In these cases WM_DESTROY message is coming on file dialog immediately and it doesn't require any actions from user. Anyway, as the problem is critical and we have simple and safe fix it's quite reasonable to use the fix.
01-03-2007

SUGGESTED FIX *** /tmp/geta20867 2007-02-22 12:07:38.000000000 +0300 --- awt_Dialog.cpp 2007-02-22 12:07:19.000000000 +0300 *************** *** 225,231 **** blocker = AwtWindow::GetModalBlocker(blocker); } if (topMostBlocker != NULL) { ! if (topMostBlocker != AwtToolkit::GetInstance().GetHWnd()) { ::SetForegroundWindow(topMostBlocker); } return 1; --- 225,233 ---- blocker = AwtWindow::GetModalBlocker(blocker); } if (topMostBlocker != NULL) { ! if ((topMostBlocker != AwtToolkit::GetInstance().GetHWnd()) && ! ::IsWindowVisible(topMostBlocker)) ! { ::SetForegroundWindow(topMostBlocker); } return 1;
27-02-2007

EVALUATION The bug is reproducible on windows Vista in IE and Mozilla browsers with File Chooser dialog only. The workaround is to click on anything but the browser window to make the browser unfreeze. User can click on browser's taksbar icon, or on desktop, or on any icon in the system tray, or on any other window - the browser will unfreeze and applet continues to function normally. If Java Console is running (even in minimized mode) the bug is not reproducible. If the last active window is anything but the browser window - the bug is not reproducible. If I try to close file chooser dialog programmatically - the bug is not reproducible. Due to all these constraints I need more time to get to the cause of the problem.
31-01-2007