Working on JDK-8331295 exposed two issues in the C2 code generator: unnecessary cloning in the matcher and excessive hoisting in the scheduler. Solving either of these problems solves JDK-8331295, and we opted to solve the matcher issue in JDK-8331295.

We should also solve the scheduler issue. The suggested fix below adds a small delta in the primary block frequency comparison in PhaseCFG::is_cheaper_block. The delta is already available in the function and we use it similarly in PhaseCFG::is_cheaper_block for another block frequency comparison. To verify the fix, unwind the changes for JDK-8331295 and run test/hotspot/jtreg/compiler/c2/TestFindNode.java. The run should result in hitting the compilation memory limit. Applying the changes below should fix the issue (double check that there is no compilation bailout for the test method).

diff --git a/src/hotspot/share/opto/gcm.cpp b/src/hotspot/share/opto/gcm.cpp
index c46d69058e9..b13e24deac1 100644
--- a/src/hotspot/share/opto/gcm.cpp
+++ b/src/hotspot/share/opto/gcm.cpp
@@ -1252,13 +1252,15 @@ bool PhaseCFG::is_cheaper_block(Block* LCA, Node* self, uint target_latency,
     return C->randomized_select(cand_cnt);
   }
 
-  // Better Frequency
-  if (LCA->_freq < least_freq) {
+  const double delta = 1 + PROB_UNLIKELY_MAG(4);
+
+  // Better Frequency. Add a small delta to the comparison to not needlessly
+  // hoist because of, e.g., small numerical inaccuracies.
+  if (LCA->_freq * delta < least_freq) {
     return true;
   }
 
   // Otherwise, choose with latency
-  const double delta = 1 + PROB_UNLIKELY_MAG(4);
   if (!in_latency                     &&  // No block containing latency
       LCA->_freq < least_freq * delta &&  // No worse frequency
       target_latency >= end_latency   &&  // within latency range