A fatal error (e.g. assert), if triggered during dynamic initialization, will plain kill the VM without a trace. (A)

A alternative variant of this error: VM will get caught up in an endless recursion, repeat "[Too many errors, abort]\n" endlessly, quickly growing its RSS until the OOM Killer kills it. (B)

These symptoms are all caused by VMError::report() attempting to write to an fdStream object (VMError::out and VMError::log). These object instances are allocated at global file scope:

1199 fdStream VMError::out(defaultStream::output_fd());
1200 fdStream VMError::log; // error log used by VMError::report_and_die()

They are non-trivial (have vtables) and need to be initialized themselves before being used. If the assert happens before they are initialized, the vtables will not yet have been set up, and once we attempt to call out::write() or log::write(), in an context where we only have an outputStream* ptr (e.g. VMError::report()), we crash.

Depending on which one of VMError::log and VMError::out are still uninitialized, we end up in (A) or (B). In both cases, the secondary signal handler (crash_handler()) will catch the signal, re-try error reporting and crash again. This is an endless cycle. We do have recursion detection in place to stop this cycle:

1388       if (recursive_error_count++ > 30) {
1389        out.print_raw_cr("[Too many errors, abort]");
1390        os::die();
1391      }

but this again uses methods of outputStream to write the "Too many errors" text, which will crash again before ever reaching os::die().