Since Java 5 the `java.lang.instrument` package provides services that allow Java programming language agents to instrument (i.e. modify the bytecode) of programs running on the Java Virtual Machine. The `java.lang.instrument` functionality is based and implemented on top of the native Java Virtual Machine Tool Interface (JVMTI) also introduced in Java 5. But because the `java.lang.instrument` API is a pure Java API and uses Java classes to instrument Java classes it imposes some usage restrictions which are not very well documented in its API specification.
E.g. the section on ["Bytecode Instrumentation"](https://docs.oracle.com/en/java/javase/21/docs/specs/jvmti.html#bci) in the JVMTI specification explicitly warns that special "*Care must be taken to avoid perturbing dependencies, especially when instrumenting core classes*". The risk of such "perturbing dependencies" is obviously much higher in a Java API like `java.lang.instrument`, but a more detailed explanation and warning is missing from its API documentation.
The most evident class file transformation restriction is that while a class A is being loaded and transformed it is not possible to use this same class directly or transitively from the `ClassFileTransformer::transform()` method. Violating this rule will result in a `ClassCircularityError` (the exact error type is disputable as can be seen in [8164165: JVM throws incorrect exception when ClassFileTransformer.transform() triggers class loading of class already being loaded](https://bugs.openjdk.org/browse/JDK-8164165), but the result would be a `LinkageError in any case).
The risk to run into such a `ClassCircularityError` error increases with the amount of code a transforming agent is transitively using from the `transform()` method. Using popular libraries like ASM, ByteBuddy, etc. for transformation further increases the probability of running into such issues, especially if the agent aims to transform core JDK library classes.
By default, the occurrence of a `ClassCircularityError` in `ClassFileTransformer::transform()` will be handled gracefully with the only consequence that the current transformation target will be loaded unmodified (see `ClassFileTransformer` API spec: "*throwing an exception has the same effect as returning null*"). But unfortunately, it can also have a subtle but at the same time much more far-reaching consequence. If the `ClassCircularityError` occurs during the resolution of a constant pool entry in another, unrelated class, that class will remain in an error state forever due to [ยง5.4.3 Resolution](https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-5.html#jvms-5.4.3) of the Java Virtual Machine Specification which mandates that "*if an attempt by the Java Virtual Machine to resolve a symbolic reference fails because an error is thrown that is an instance of LinkageError (or a subclass), then subsequent attempts to resolve the reference always fail with the same error that was thrown as a result of the initial resolution attempt.*". This means that the `ClassCircularityError` can repeatedly be thrown much later, in user code which is completely unrelated to class file transformation if that code happens to use the same class that failed to resolve a reference during instrumentation. A good example for this scenario are the sporadic `ClassCircularityError` that were seen in user code while using `ConcurrentHashMap`s caused by a change in the popular ByteBuddy library (see [ByteBuddy #1666 for more details](https://github.com/raphw/byte-buddy/issues/1666)).
I'd therefor like to propose to add an @apiNote to j.l.i.ClassFileTransformer which makes users of the API aware of these potential issues:
```
* @apiNote
* If the invocation of {@link #transform transform()} for a class <code>C</code>
* directly or transitively requires loading or resolving the same class <code>C</code>,
* an error is thrown that is an instance of {@link LinkageError} (or a subclass).
* Transforming core JDK classes or using libraries which depend on core JDK classes
* during transformation increases the risk for such errors. If the {@link LinkageError}
* occurs during reference resolution (see section 5.4.3 Resolution of <cite>The
* Java Virtual Machine Specification</cite>) for a class <code>D</code>, the
* corresponding reference resolution in class <code>D</code> will always fail
* with the same error. This means that a {@link LinkageError} triggered during
* transformation of <code>C</code> in a class <code>D</code> not directly related to
* <code>C</code> can repeatedly occur later in arbitrary user code which uses class
* <code>D</code>.
```