JDK-6764266 : Modal dialog from JavaScript called by JSObject causes IE freeze
  • Type: Bug
  • Component: deploy
  • Sub-Component: plugin
  • Affected Version: 6u10
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: windows
  • CPU: x86
  • Submitted: 2008-10-27
  • Updated: 2011-01-20
  • Resolved: 2009-01-26
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
6u12 b02Fixed
If you show a modal JDialog, then call across the JavaScript bridge and that shows anything that is also modal, then IE hangs (haven't tried firefox). So if your applet happens to be interacting with a complex ajax style web app that pops up modal dialogs itself, and you happen to do that while you're showing a modal JDialog, things freeze.

Turning "enable next generation java plug in" in the control panel works around the problem.

Can anyone reproduce this on their machines?

Code that reproduces the error:


//no it's not in a package
import javax.swing.*;
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
public class FreezeTest extends JApplet {
    public void init() {
        JButton button = new JButton("show dialog");
        button.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent e) {
                Frame frame = (Frame) SwingUtilities.getAncestorOfClass(Frame.class, FreezeTest.this);
                DialogTest test = new DialogTest(frame, true, FreezeTest.this);
    public void destroy() {
    public void start() {
    public void stop() {
    public String getAppletInfo() {
        return "Dummy";


import netscape.javascript.JSObject;
import javax.swing.*;
import java.awt.*;
import java.awt.event.ActionListener;
import java.awt.event.ActionEvent;
import java.applet.Applet;
public class DialogTest extends JDialog {
    private Applet applet;
    public DialogTest(final Frame owner, boolean modal, final Applet applet) {
        super(owner, modal);
        setSize(500, 500);
        this.applet = applet;
        JPanel panel = new JPanel();
        JButton button = new JButton("click me");
        button.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent e) {
                JOptionPane.showMessageDialog(DialogTest.this, "Modal?", "Modal?", JOptionPane.QUESTION_MESSAGE);
                JSObject js = JSObject.getWindow(applet);


      <script type="text/javascript">
          function freezeTest() {
      <object id="FreezeTestApplet" classid="clsid:CAFEEFAC-0016-0000-FFFF-ABCDEFFEDCBA"
            width="300" height="200" align="center" valign="center"
                <param name="archive" 
                <param name="code" value="FreezeTest"/>

Steps to reproduce:
install Java 6 update 10
make sure "Enable the next-generation Java Plug-in (requires browser restart)" is checked in the Java Control Panel
build freezetest.jar
Open freezetest.html with Internet Explorer
click the button
a modal dialog is shown, click ok
another modal dialog (JOptionPane) is shown, click ok
a JavaScript call then shows an alert(), you cannot click ok, the jvm is frozen (java console won't respond)
if you click somewhere in the html page, then you can click ok, but IE is still frozen, and must be "end tasked" [edit: on a colleague's machine, IE shuts down ok, but the frozen dialog that can be "clicked out of", still occurs]

System Config:
Windows XP x64 with all the latest updates (except today's emergency security patch).
Internet Explorer 7.0.5730.11 (32-bit)

Vista (32-bit) + IE 7

SUGGESTED FIX http://sa.sfbay.sun.com/projects/deployment_data/6u12/6764266.1

EVALUATION After discussion with ###@###.###, it sounds likely that the implementation of the modal dialog reactivation hooks in src/plugin/win32/plugin2/common/WindowsHelper.c are the problem. Basically, when we install those hooks, we should pass down the HWND associated with the control window of the applet, and mark all parent windows of it with a particular property that is checked later in the hook -- only if that property is present should the event vetoing mechanism be used. This will prevent the hooks from interfering with the JavaScript alert dialog that is raised and is similar to the mechanism that the AWT uses for in-process modal dialog blocking. Reassigning to the Java Plug-In team.

EVALUATION After investigation, the situation is as follows: - an AWT modal dialog is visible - the application initiates a call down to JavaScript (on the Event Dispatch Thread running the dialog) which raises a modal JavaScript alert dialog In this situation the Java Plug-In's nested message pump on the browser side is not active. The flow of control has been handed back to the web browser. While the JavaScript alert dialog is visible, it is the browser's own message pump which is running on the browser's main thread. Somehow the fact that an AWT modal dialog is visible is interfering with the correct operation of the browser's nested message pump. We have attempted to work around this issue in the Java Plug-In by keeping track of the visible modal AWT dialogs (which was already being done for the purpose of implementing modal dialog blocking of the web browser), and if a Java -> JavaScript call is made while a modal dialog is visible, performing the following calls against that dialog: - setVisible(false); - setModal(false); - setVisible(true); Unfortunately this workaround does not work. Presumably the modal dialog's nested EventQueue can not be popped as long as the dialog is visible, and the modal behavior does not change. Without a way to toggle the modality of the AWT dialogs on the fly there is no workaround we can do in the Java Plug-In for this issue. Reassigning to the AWT team as there does not appear to be any workaround we can do at the Java Plug-In level for this problem.

EVALUATION After trying on several different machines with IE6 and IE7, it's found that the workaround (by clicking on the html page and then clicking the javascript dialog) works with IE7 but not with IE6.

WORK AROUND 1) in the applet code, don't create the DialogTest dialog as a modal dialog by passing false as the second argument as follows: DialogTest test = new DialogTest(frame, false, FreezeTest.this); or 2) When the javascript modal dialog is up and if the browser is frozen, click on the html page and then click back on the dialog. This workaround works with IE7.

EVALUATION I can't reproduce a freeze with this test case and the current Java Plug-In code. I am able to successfully dismiss the JavaScript modal dialog and resume activities. Please confirm the exact environment in which the problem appears to be reproducible.

EVALUATION That is exactly what happened. A modal dialog is raised and a click on the dialog button does java->js call to raise a javascript dialog. the browser side jvm main thread is blocked in native iDispatchInvoke0: "main" prio=6 tid=0x00fe8400 nid=0x454 runnable [0x0013d000..0x0013e7a4] java.lang.Thread.State: RUNNABLE at sun.plugin2.main.server.IExplorerPlugin.iDispatchInvoke0(Native Method) at sun.plugin2.main.server.IExplorerPlugin.iDispatchInvoke(IExplorerPlugin.java:779) at sun.plugin2.main.server.IExplorerPlugin.javaScriptCallImpl(IExplorerPlugin.java:735) at sun.plugin2.main.server.IExplorerPlugin.javaScriptCall(IExplorerPlugin.java:641) at sun.plugin2.main.server.IExplorerPlugin.javaScriptEval(IExplorerPlugin.java:670) at sun.plugin2.main.server.LiveConnectSupport$PerPluginInfo.javaScriptEval(LiveConnectSupport.java:508) at sun.plugin2.main.server.LiveConnectSupport.javaScriptEval(LiveConnectSupport.java:92) at sun.plugin2.main.server.JVMInstance$WorkerThread$3.doWork(JVMInstance.java:642) at sun.plugin2.main.server.JVMInstance$6.run(JVMInstance.java:1118) at sun.plugin2.main.server.WindowsHelper.runMessagePump0(Native Method) at sun.plugin2.main.server.WindowsHelper.runMessagePump(WindowsHelper.java:28) at sun.plugin2.main.server.IExplorerPlugin.runMessagePump(IExplorerPlugin.java:1166) at sun.plugin2.main.server.IExplorerPlugin.runMessagePump(IExplorerPlugin.java:1162) at sun.plugin2.main.server.IExplorerPlugin.waitForSignalWithModalBlocking(IExplorerPlugin.java:1055) at sun.plugin2.main.server.ModalitySupport$1.run(ModalitySupport.java:48) On the client vm side, awt event dispatcher is blocked waiting for the reply of JSObject call.

EVALUATION More detailed investigation is needed, in particular thread dumps on both the browser side and the JVM side at the time the hang occurs. If the JavaScript engine has the flow of control (because Java called down to present the modal JavaScript dialog), then the plug-in is not running a nested message pump on the browser's main thread. Attention was paid to the structure of the nested message pump so that it does not pump input events unless a modal dialog has been raised in the attached JVM associated with the applet when a JavaScript-to-Java call is made.

EVALUATION The problem is probably in the server side plugin main thread message pump. The main thread is doing javascript call while blocking the browser window for the modal dialog (IExplorerPlugin.waitForSignalWithModalBlocking()). Dismissing the windows modal dialog (raised by javascript) may require some message processing on the browser main thread, which is blocked by Plug-in.