Summary
-------

This CSR refers to the third preview iteration of the Foreign Function & Memory (FFM) API originally targeted for Java 19, with the goal of further refining and consolidating the FFM API.

Problem
-------

Feedback on the previously incubating API revealed the following issues:

 * There are two ways to allocate memory that are equivalent via either the overloads of `MemorySegment::allocateNative` or via the `Arena::allocate` overloads which can be confusing.
 * The duality between `Arena` and `SegmentScope` is a source of confusion: in almost all cases, client code creates arenas, but then has to retrieve the arena scope to interact with other APIs (e.g. `Linker::upcallStub`).
 * The `Linker::nativeLinker` method throws `UnsupportedOperationException` for unsupported platforms.
 * The FFM API allows creation of *some* memory layouts (e.g. padding layouts) whose size is not a multiple of 8 bits, even though such layouts are not well supported.
 * The restricted methods for adapting the size and scope of a (zero-length) memory segment are hard to use, especially to resize existing segments (which is  a fairly common operation when interacting with native libraries).
 * The `VaList` construct is neither used in the JDK nor in third-party libraries.
 * The mechanism for creating unbounded address layouts is overly unrestricted.
 * It is not possible to use linker options for upcalls. This might be useful in the future, e.g. to specify custom policies to handle uncaught exceptions.
 * The handling of `CaptureCallState` (e.g. for "errno") does not take into account unsupported runtime values (e.g. the same capture layout is exposed for both Linux and Windows).
 * The performance of very short-lived native calls (like `clockgettime`) could be improved.
 * Checking sequence and group layout constraints can be made upon layout construction instead of throwing an exception later when a `VarHandle` is constructed.

Solution
-------

Here we describe the main changes brought forward in this CSR:

 *  The static factory `MemorySegment::allocateNative` overloads have been removed. Instead, allocation is made via the `Arena::allocate` overloads. See [this document][2].
 *  `SegmentScope` was dropped in favor of a much smaller construct `MemorySegment.Scope` which only has one predicate `isAlive()` and where the `equals()` method tests if two scopes model the same lifetime (and is alive). See [this document][2].
 * The factories `Arena::(openConfined, openShared)` were renamed to `Arena::(ofConfined, ofShared)`. The `SegmentScope.global()` factory was replaced by `Arena.global()`, while the `SegmentScope.auto()` factory was replaced by `Arena.ofAuto()`. Picking a more neutral name (`of` instead of `open`) was deemed necessary given that automatic arenas cannot be closed.
 * A fall-back linker based on [libffi][3] was added to simplify porting (see https://github.com/openjdk/panama-foreign/pull/770 for details).
 * The memory layout API now uniformly enforces that bit-size and bit-alignment constraints of *all* memory layouts (including padding layouts) must be a multiple of 8.
 * A new restricted instance method, namely `MemorySegment::reinterpret`, was added. This allows to create a new native segment, with new size and segment scope with the same base address as some existing segment. Some overloads are also provided, for usability. The `MemorySegment::ofAddress` method is kept, as a way to create a zero-length native memory segment from a raw long address. As such, this method is no longer restricted.
 * The `VaList` interface was dropped.
 * The `ValueLayout.OfAddress::asUnbounded` method was replaced with the new method `ValueLayout.OfAddress::withTargetLayout`, which allows to associate an address layout with a "target" layout (the layout of the memory region pointed to by a given address). This allows the API to expose a new kind of "dereference" `PathElement`, which can be used to create access var handles for expression such as `*(a.b)[10]`. Also, given the importance of address layouts, `ValueLayout.OfAddress` was moved to a toplevel class, namely `AddressLayout`.
 * Linker options can now be used when creating upcall stubs. (Note: at the time of writing there are no supported linker options for upcall stubs).
 * The `CapturedCallState` interface was removed. A new static method was added to query the layout of the runtime values to be preserved. This layout is now platform specific.
 * A new linker option was added to mark calls to native functions that are extremely short-lived (comparable to a no-op) and do not call back into Java (we call such functions "trivial functions"). This option might be used by the linker to speed up a native call (e.g. by removing thread state transitions).
 * Up-front checks are added in the sequence and group constructors as well as in the associated `withBitAlignment()` methods making sure the [layouts are well formed][4]. 


Specification
-------------

A specdiff and javadoc of the changes as of 2023-04-13 are available below:

 * https://cr.openjdk.org/~pminborg/panama/21/v2/specdiff/overview-summary.html
 * https://cr.openjdk.org/~pminborg/panama/21/v2/javadoc/api/java.base/java/lang/foreign/package-summary.html


References
-------------
 1. https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html
 2. http://cr.openjdk.java.net/~mcimadamore/panama/scoped_arenas.html
 3. https://github.com/libffi/libffi
 4. https://github.com/openjdk/panama-foreign/pull/824

  [1]: https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html
  [2]: http://cr.openjdk.java.net/~mcimadamore/panama/scoped_arenas.html
  [3]: https://github.com/libffi/libffi
  [4]: https://github.com/openjdk/panama-foreign/pull/824