HeapDumpTestWithActiveProcess.java is the only test we have that tries doing stack walking with an application that is still in the process of starting up. Because of this it detects a lot of bugs in SA bug due to threads being in bad states. JDK-8244383 is one such example, causing failures with "illegal bci". While addressing JDK-8244383 I added some improved error checking to the test, similar to those in ClhsdbLauncher:

            // This will detect most SA failures, including during the attach.
            output.shouldNotMatch("sun.jvm.hotspot.debugger.DebuggerException:.*$");
            // This will detect unexpected exceptions, like NPEs and asserts, that are caught
            // by sun.jvm.hotspot.CommandProcessor.
            output.shouldNotMatch("^Error: .*$");

In addition I fixed some places in SA that generate the "Error" message, but don't always ensure that it starts on a new line, which it frequently does not:

diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadStackTrace.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadStackTrace.java
--- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadStackTrace.java
+++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ThreadStackTrace.java
@@ -62,7 +62,7 @@
                 }
             }
         } catch (Exception e) {
-            System.out.println("Error occurred during stack walking:");
+            System.err.println("\nError: exception occurred during stack walking:");
             e.printStackTrace();
         }
     }

diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
--- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
+++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/StackTrace.java
@@ -114,7 +114,7 @@
                             vf.printLockInfo(tty, count++);
                         }
                     } catch (Exception e) {
-                        tty.println("Error occurred during stack walking:");
+                        tty.println("\nError: Exception occurred during stack walking:");
                         e.printStackTrace();
                     }
                     tty.println();

With these changes in place, I started to see a lot more failures being detected by the test. One is JDK-8247533, which is being addressed. However, there are others that do not have separate bugs filed for them. I will list them below in separate comments.