JDK-8234050 : Implementation of Memory Access API (Incubator)
  • Type: CSR
  • Component: core-libs
  • Priority: P3
  • Status: Closed
  • Resolution: Approved
  • Fix Versions: 14
  • Submitted: 2019-11-12
  • Updated: 2020-04-23
  • Resolved: 2019-12-11
Related Reports
CSR :  
JDK-8234049 - Implementation of Memory Access API (Incubator)
Relates :  
JDK-8236769 - Clarify javadoc of memory access API
Relates :  
JDK-8235836 - Cleanup memory access API javadoc
Relates :  
JDK-8243496 - Implementation of Foreign-Memory Access API (Second Incubator)
Relates :  
JDK-8236853 - Memory access API refinements
Description
Summary
-------

Provide an incubator module, jdk.incubator.foreign, which contains an API, referred to as the Memory Access API, that is designed to facilitate safe and structured access to off-heap and on-heap memory.  This API provides the fundamental building blocks to replace JNI.

Problem
-------

To date, there is no optimal solution for accessing off-heap memory. While access to off-heap memory is possible using the ByteBuffer API, such an API has certain limitations (stateful-ness, addressing space bound by the 2G limit, non-deterministic deallocation, structural access) which makes it unsuitable as a general off-heap API, especially when it comes to interoperating with native code. Other alternatives are available, such as Unsafe (efficient, but not supported) or JNI (supported but inefficient),  but ultimately no ideal solution and/or API exists.

Solution
--------

The memory access API addresses the aforementioned problems by providing a memory access API that is *general* (can be used both for off-heap and on-heap access), *safe* (uses of this API cannot cause any hard JVM crash) and *efficient* (this is achieved by making *immutability* and *deterministic-deallocation* two central design choices of the API). Such an API lends itself well to all cases where e.g. the ByteBuffer API is currently used to access off-heap memory; since this new API doesn't incur in the 2G addressing space limit, it is particularly apt to model persistent memory (see https://openjdk.java.net/jeps/352). In addition, since this API separates memory segment descriptions from the way in which such segments are accessed, it also lends well to use cases where the same memory segment needs to be shared across multiple views or *slices* (a common use case in tensor programming, or access to multi-dimensional arrays of values).

Specification
-------------

The implementation of the memory access API exports the following interfaces in the package jdk.incubator.foreign, defined in module jdk.incubator.foreign:

```
MemorySegment                            Models a contiguous region of memory
MemoryAddress                            Models an offset within a memory segment
MemoryLayout                             Models (optional) descriptions of the contents of a memory segment
MemoryLayout.PathElement                 Constructs layout paths which can be helpful to retrieve offset to a specific layout element
```

A `MemorySegment` is a *static* and *immutable* description of a region of memory. A `MemorySegment` is always associated with *spatial bounds* (e.g. the minimum and maximum address within the segment) as well as *temporal bounds* (which define when it is *safe* to access the segment). To support deterministic-deallocaton, memory segments support the `AutoCloseable` interface, so that they can be *closed* when no longer in use (closing a memory segment might trigger deallocation of the memory resources, if any, associated with the segment).

A `MemoryLayout` is a programmatic description of a memory segment contents. The `MemoryLayout` interface provide ways to mechanically derive information from layouts, using so called *layout paths* - that is, given a  toplevel layout, and  a *path* (expressed as a list of `PathElement` instances), it is possible to derive information such as the offset of the selected layout elements within the toplevel layout; or the `VarHandle` accessor required to access the selected layout elements given a `MemoryAddress` instance which points to a memory segment with the toplevel layout.

Additionally, the implementation of the memory access API will export the following classes:

```
GroupLayout 	Models compound layouts (e.g. structs or unions)
SequenceLayout 	Models array layouts.
ValueLayout 	Models value layouts - e.g. sequence of bits

MemoryHandles 	Defines several factory methods for constructing and combining memory access var handles
MemoryLayouts 	Defines useful (and common) layout constants.
```

The first three classes are specific subclasses implementing the `MemoryLayout` interface. Each of those classes provide access to specific properties; for instance a `SequenceLayout` has an (optional) element count, and a sequence element layout. The `MemoryHandles` class defines several factories and combinators for the `VarHandle` instances which can be used to access memory segments. Similarly, the `MemoryLayouts` class contains several layout constants that can be useful to developers.

When the memory access API exits the incubating stage, we plan to make at least the following adjustments:

 * move the functionality from the `jdk.incubator.foreign` module to `java.base`
 * rename the `jdk.incubator.foreign` package to `java.foreign`
 * move the combinators/factories in `MemoryHandles` into `java.lang.invoke.MethodHandles`
 * move some of the constants in `MemoryLayouts`  (such as `JAVA_INT`, `JAVA_FLOAT` and so forth) into the corresponding primitive wrapper class (e.g. `MemoryLayouts::JAVA_INT` will become `Integer::LAYOUT`)

The javadoc for the package with the implementation (updated live) is available at http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc ; a copy (as of December 9, 2019) is also attached here.

More details can be found in the JEP issue - https://bugs.openjdk.java.net/browse/JDK-8227446
Comments
To provide some more detail, suppose there is a memory segment object and acquire is called to create a separate acquired memory segment object. Strong references are kept to both memory segment objects and the thread / execution context of the acquired memory object is killed. If I've interpreted the specs correctly, close cannot be successfully called on either memory segment object. This is a different character of leak than "I lost track of my objects" since there a strong references to the objects still available. If one is creating layouts for, say, the effect of multi-dimensional array access, one way to help coalesce the bounds checks at an API level is to return something that delivers all the values, like a stream.
14-01-2020
That is a good point - losing track of an acquired segment is sneaky because then the parent segment can no longer be closed. I agree the javadoc should say something about this condition. This problem is peculiar to the acquire/close scheme adopted by this API. Plain reference counting, on the other hand, since it allows for explicit manipulation of mutable state, allows to recover from the situation you describe (just call close() the right amount of times). That said, we also believe plain reference counting to be conceptually messier than the acquire/close model offered here. Arguably, the model for shared access is the aspect with most technical uncertainties in the whole API (since it has to reconcile safety, with efficiency in the hot access path and deterministic deallocation), so I expect we will be iterating a bit on that aspect based on feedback from real world use cases. As for the stream-like access, it was a deliberate decision not to provide any access primitive like get/set (or bulk ones, like the one suggested here); while in simple cases it might be possible to add such operation, there is always an ambiguity: what access mode should be used to retrieve elements (plain get, or, say, volatile get?). Then there's the problem with carrier types - offering such stream-oriented accessors will likely require boxing (unless we expose one accessor per primitive). In other words, the space for accessing elements is big: { access modes * carrier types }. Hence the decision to provide low-level support through VarHandle, with the view that clients of this API might construct higher level views on top (e.g. a tensors API).
12-12-2019
Thanks for the comments. I will file a tracker issue to make sure they are properly addressed. To clarify some points: memory leaks are possible (and the javadoc says so) if segments go unreachable and close() is not called. Other similar APIs (e.g. Netty's ByteBuf) feature a 'debug' mode, which catches such leaks, at the cost of loss in peak performance; we might be able to do something better, especially if we refine the allocation story for native segments (rather than just relying on malloc/free). As for stream-based access, the need never arose (as typically layouts are interesting when they are constructed, much less after that - at least from the client perspective). That said, adding e.g. a method to retrieve all the nested layout elements (which possibly return empty list) is not bad, although I guess, longer term I'd prefer to leverage the power of pattern matching for easing the task of extracting info from an already constructed layout.
11-12-2019
Moving to Approved as an incubator API. Various comments from another pass or two over the spec: Typo in MemoryLayout "for a finite sequence layout S whose element layout is E and size is L" "L" is not italicized. typo in MemoryAddress.equals Returns true if and only if the specified object is also *a* address, and it is equal to this address. If MemoryAddress are equal if their offsets and segments are equal, that should be stated explicitly. Consider a rename of "MemoryAddress", perhaps to "MemoryOffset" to more precisely characterize its role. Processors have many addressing modes including base + offset as modeled here. In general, the memory management space has many overloaded terms like "segment", "region", "arena", etc. so it is hard to get terms that are both familiar and don't conflict with some prior usage. There doesn't seem to be a predicate for "can I call close without an exception?" (isAlive && isAccessible) isn't sufficient. Is it the case that there is a memory leak one of the threads that has acquired a memory segment dies without freeing it? MemoryLayout.byteAlignment() should have an @implSpec tag describing its default implementation. For MemoryLayout.equals, what aspects are compared? In particular, would a GroupLayout with and without a name be considered equal? How would stream-based access or some other internal iteration mechanism be used in an API like this?
11-12-2019
I've read through the API and javadoc from Dec 9. It's very good and the javadoc is well written. I don't see any obvious issues and if anything is found then it can resolved or re-worked without compatibility concern because it's an incubating module and so non-final.
11-12-2019
Note - there are two differences between the attached javadoc and the live version: * in the live version, the restriction on overlapping addresses in `MemoryAddress::copy` is dropped (since we have verified with the Hotspot team that `Unsafe::copyMemory` already handles that case correctly) * small update in the javadoc of `MemoryLayout::withName` to address the comment raised before in this CSR Let me know if a new updated attachment is needed.
10-12-2019
Note: given this is an incubating API, and that the schedule is tight, I decided to prioritize critical API issues over stylistic issues (such as naming issues). I'm keeping a list of all the minor, non-semantic changes we have to do, and will file a tracker bug to get at them past integration. Of course, issues in the API which point at serious issues should be addresses ahead of integration (for instance, since this CSR was moved to provisional, two such issues have been found and fixed: MemoryAddress:copy being underspecified, and behavior of memory access var handle also not being fully specified for all the access modes).
09-12-2019
Moving to Provisional. I'll need to take another pass in more detail before approving the request. A few comments on an initial reading: I suppose interactions with fibers vs threads can be considered as both this project and Loom go forward. In GroupLayout.withName, if a new layout is returned, that should also be stated in the main summary sentance (as done for withBitAlignment). It wasn't entirely clear if the offset of a MemoryAddress was constrained to be non-negative, perhaps that is implied by the totality of the specs. I suggest renaming MemoryAddress.offset(long) to something like MemoryAddress.withOffset(long) to put more naming distance between the introspection method offset with no args and a method which returns a new value.
04-12-2019