When running the Java Plug-In out of a development build of the JRE, there is a "classes" directory but no lib/rt.jar. This causes the Java Kernel's preJVMStart() to report the following error upon launch of the first applet in the browser:

Java Error
Could not read file C:\Users\kbr\ws\6uN\j2se\build\WINDOW~1\lib\rt.jar.

The Windows MessageBox call which displays this dialog box runs a subordinate message pump on the current thread. In Firefox, this causes browser-related window messages to be processed out of order, leading to both the old and new Java Plug-In being called in an unexpectedly reentrant fashion. This is causing browser crashes in the new Java Plug-In which are described in the evaluation of 6647070. It is not possible to work around all of these issues because the reentrance to the plug-in's code causes some of the web page's initialization sequence to be performed out of order.

This is a potential problem with any of the error dialogs displayed during preJVMStart(). However, the one above is the most immediately problematic, as it prevents the Java Plug-In developers from effectively using development builds.

The best solution to the problem is as follows. In preJVMStart(), kernel.cpp, if we fail to open rt.jar, check for the existence of the "classes" directory in javaHome. If it is present, assume we're running in a development build and silently return without displaying the error dialog.

There might be other possible and more general solutions such as avoiding the use of the MessageBox API and writing a custom control and subordinate message pump for the error dialog, but this would be a lot of work and it is not even clear that it would solve the problem. It might be the case that even processing WM_PAINT messages in the subordinate message pump might be enough to cause Firefox to get back the flow of control. Note that using MB_TASKMODAL and other flags in the MessageBox call for the error dialog was tried with no success.