# Test Methodology

Tests need to be developed to cover added APIs as part of the 'Foreign Linker API (Incubator) JEP. The tests need to cover:

 - The CLinker API, particularly its `downcallHandle` and `upcallStub` methods, in both positive and negative tests.
 - The `LibraryLookup` api; whether looking up existing symbols works, and whether looking up non-existing symbols fails as specified
 - The `FunctionDescriptor` data-carrier class; whether combinator/transform functions produce the desired results.
 - (As of yet) unsupported platforms. Failure behaviour on these platforms should be deterministic.
 - Bench marking to check the performance of the new implementation against JNI.
 - Native code is tested indirectly through the Java APIs.

# Test Inventory

  Tests have been added under test/jdk/java/foreign to cover the new API. These tests contain a large set of combinatorial tests (TestUpcall and TestDowncall), that are run in several configurations, as well as general API tests for specified API functionality, and spot tests that cover specific problematic cases. Some "whitebox" tests have been used to (abstractly) test the correctness of the ABI support; crucially, these test can detect issues on a given ABI support (e.g. Aarch64) even when running on a different platform (e.g. linux x64).

# Test Configurations

 - The tests should run on all currently supported platforms: Windows-x64, Mac-x64, and Linux-x64, and Linux-AArch64.
 - The JEP does not have any micro-architecture dependencies.
 - There are several tests that depend on being run with the JIT compiler C2 enabled. Namely: test/jdk/java/lang/foreign/TestIntrinsics, */TestUpcallHighArity, and */stackwalk/TestStackWalk.

# Test Success Criteria

Since this is an API that does more or less open-ended code generation, it is not possible to get 100% coverage on all use-cases (since the set of use-cases is infinite). Instead, test cases are crafted based on frequency of use, as well as knowledge of how the target ABI operates to find bugs in different parts of the ABI implementations.

 - At least 80% cyclomatic complexity test coverage.
 - 100% public API coverage.
 - Test pass rate 100%.
 - A coverage of different call shapes as determined by the implementation ABIs, thoroughly covering the most common use-cases.
 - Performance of intrinsified call shapes should be in the same ballpark as JNI, if not faster.