SJSAS 9.0 (Sun Application Server) uses NIO for its HTTP Engine. Removing the workaround for 4953599 and running performance test on Solaris literally kill the VM, without any core/log. The problem seems to be that the C heap is corrupted, so lots of threads stack up trying to get native memory from the C heap. Meanwhile, the compiler thread detects the corruption in the C heap and tries to throw a signal, and the signal handler dies -- so either the entire process dies without a core, or the process hangs.
The attachement contains a pstack log showing the signal thread hanging, and the corresponding jstack output for that.
We've tested with 1.5.0_0X and the problem happens in all version.