Given the simple example in Test.java and run with (referred to as normal run):

$  java -XX:CompileCommand=quiet -Xbatch -XX:+PrintCompilation -XX:-TieredCompilation -XX:CompileOnly=Test -XX:+PrintInlining Test

When normally running Test.java, C2 successfully inlines bar():

   2729   25    b        Test::test (5 bytes)
                            @ 0   Test::bar (2 bytes)   inline (hot)

However, when replay compiling test(), it fails to inline bar():

   2734   24    b  4       Test::test (5 bytes)
                              @ 0   Test::bar (2 bytes)   unloaded signature classes

The reason is that replay compilation fails to resolve Integer (which is initially resolved with no protection domain because it belongs to the Java API) with the protection domain of class Test. In the initialization of ciReplay, we only resolve InstanceKlasses without a protection domain:

CompileReplay(const char* filename, TRAPS) {
  ...
  _protection_domain = Handle();
  ...
}

When doing replay compilation of test(), we try to inline bar() and check if all classes in the signature are loaded. This is done by calling ciMethod::has_unloaded_classes_in_signature() by using the protection domain of class Test. But since we have not resolved class Integer with the protection domain of Test, we fail this check and do not perform inlining.

There are different variations of the same problem depending on which Java API class is used in the signature of the inlinee and which methods were already compiled by C2 in the normal run:

1) bar() was already compiled before test():
In this case, C2 has already called Method::load_signature_classes() for bar() and successfully resolved Integer with the protection domain of Test (protection domain Test newly added for String).

2) bar() was not compiled and is public:
Before invoking main(), it is validated by java.base/LauncherHelper::validateMainClass(). It tries to find main() by calling getMethod() which eventually calls the native method getDeclaredMethods0() into the VM to get the list of all public methods inside Test. In this process, all the classes in the method signatures are resolved.

3) bar() was not compiled, is private and we use String instead of Integer:
Same as 2) but this time, bar() is private and therefore ignored by getDeclaredMethods0(). Thus, bar() will not initiate the resolution of String. Instead, the method main() itself does that because it has String[] as parameter.

To fix this, we need to make sure to resolve all classes used in signatures of inlinees with the protection domain of the holder class of the method to be replay compiled in order to inline them.