IE hangs when you open another IE instance when modal dialog is up and running.
To reproduce,
1. load the attached html "testAppletSimple.htm", this will bring up the
applet
2. close the browser by clicking the "x" of the browser window, this will
trigger the modal dialog window
3. While the modal dialog is up, open a new session of IE
4. Go back to the applet window (not the modal dialog), click on the browser anywhere.
5. You will see that the CPU utilization is up to 100% and applet IE window hangs.
This issue is closely related to bug 6395737.