In fastdebug / slowdebug builds, should an error happen in a safepoint while JFR is recording, there is an assertion that does not allow the emergency operation to complete successfully. As a side effect, besides not dumping the JFR data, this can give misleading information in the hs_err.log file (due to recursive traps).

For example:

---------------  T H R E A D  ---------------

Current thread (0x00007f4eb03d5800):  VMThread "VM Thread" [stack: 0x00007f4e87422000,0x00007f4e87522000] [id=4219]

Stack: [0x00007f4e87422000,0x00007f4e87522000],  sp=0x00007f4e87520110,  free space=1016k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x185a46f]  VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x25f
V  [libjvm.so+0x185b28f]  VMError::report_and_die(Thread*, void*, char const*, int, char const*, char const*, __va_list_tag*)+0x2f
V  [libjvm.so+0xb50e30]  report_vm_error(char const*, int, char const*, char const*, ...)+0x100
V  [libjvm.so+0xf7ec4f]  JfrTypeManager::write_type_set()+0x19f
V  [libjvm.so+0xf46c79]  JfrRecorderService::finalize_current_chunk_on_vm_error()+0x39
V  [libjvm.so+0xf46dde]  JfrRecorderService::rotate(int)+0x7e
V  [libjvm.so+0xef8ef8]  JfrEmergencyDump::on_vm_shutdown(bool)+0x278
V  [libjvm.so+0x185a72e]  VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x51e
V  [libjvm.so+0x185b28f]  VMError::report_and_die(Thread*, void*, char const*, int, char const*, char const*, __va_list_tag*)+0x2f
V  [libjvm.so+0xb50e30]  report_vm_error(char const*, int, char const*, char const*, ...)+0x100
V  [libjvm.so+0x14b26e0]  os::thread_cpu_time(Thread*, bool)+0x80
V  [libjvm.so+0x178c506]  Thread::print_on(outputStream*, bool) const+0x46
V  [libjvm.so+0x17919de]  JavaThread::print_on(outputStream*, bool) const+0x13e
V  [libjvm.so+0x1795c09]  Threads::print_on(outputStream*, bool, bool, bool, bool)+0x1c9
V  [libjvm.so+0x189f042]  VM_Operation::evaluate()+0x132
V  [libjvm.so+0x189a6ce]  VMThread::evaluate_operation(VM_Operation*) [clone .constprop.51]+0x18e
V  [libjvm.so+0x189af87]  VMThread::loop()+0x4d7
V  [libjvm.so+0x189b593]  VMThread::run()+0xd3
V  [libjvm.so+0x14b3de0]  thread_native_entry(Thread*)+0x100

The TypeSet write routine currently asserts a non-safepointed context, and this is what leads to this issue.

Fix can be quite simple, like:

diff -r 79baec7d831e src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeManager.cpp
--- a/src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeManager.cpp        Tue Jul 03 10:47:50 2018 +0200
+++ b/src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeManager.cpp        Tue Jul 03 12:02:18 2018 +0200
@@ -148,9 +148,8 @@
 }

 void JfrTypeManager::write_type_set() {
-  assert(!SafepointSynchronize::is_at_safepoint(), "invariant");
   // can safepoint here because of Module_lock
-  MutexLockerEx lock(Module_lock);
+  MutexLockerEx lock(SafepointSynchronize::is_at_safepoint() ? NULL : Module_lock);
   JfrCheckpointWriter writer(true, true, Thread::current());
   TypeSet set;
   set.serialize(writer);