JDK-8320247 : Some methods make promises about Java array element alignment that are too strong
  • Type: CSR
  • Component: core-libs
  • Priority: P3
  • Status: Closed
  • Resolution: Approved
  • Fix Versions: 23
  • Submitted: 2023-11-16
  • Updated: 2024-02-16
  • Resolved: 2024-02-13
Related Reports
CSR :  
JDK-8318966 - Some methods make promises about Java array element alignment that are too strong
Description
Summary
-------

Change the specification and implementation of `MethodHandles::byteArrayViewVarHandle`, `MethodHandles::byteBufferViewVarHandle`, `ByteBuffer::alignedSlice`, and `ByteBuffer::alignmentOffset` to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation.

Problem
-------

The JVM spec makes no guarantees about the alignment of Java array elements. At most we can assume that the elements are naturally aligned (i.e. aligned according to their size). The named methods are affected by the alignment guarantees for the elements of a `byte[]` which are 1-byte aligned (i.e. not aligned, really). The issue with the named methods is that they guarantee alignment when a `byte[]` is (indirectly) accessed through another primitive type like `int` or `long` (a so-called 'mismatched access'). Given the minimal alignment guarantees of `byte[]` elements, making any such guarantees about the alignment of accesses is incorrect, as they might not hold on all JVM implementations.

The fact that `byte[]` elements can at most be assumed to be 1-byte aligned,  means the start of the array elements can be located at any memory address. This memory address can also change due to a garbage collector moving the array object around (i.e. the address is unstable). Therefore, at whichever offset we access this array, the memory address of the access can also be any memory address, and is also unstable. We can not make any guarantees about the alignment of an arbitrary, unstable memory address. Therefore, we can not make any guarantees that accesses at certain offsets will be aligned (or not). 

Solution
--------

The proposed solution is to relax the guarantees made by the named methods, so that these methods behave consistently on all valid implementations of the JVM.

 - `MethodHandles::byteArrayViewVarHandle`: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an `UnsupportedOperationException` for the unsupported access modes, as mandated by the spec of VarHandle (https://github.com/openjdk/jdk/blob/ffa35d8cf181cfbcb54497e997dbd18a9b62b97e/src/java.base/share/classes/java/lang/invoke/VarHandle.java#L189-L191).

 - `MethodHandles::byteBufferViewVarHandle`: the implementation/spec is incorrect when accessing a heap buffer (wrapping a `byte[]`), for the same reasons as `byteArrayViewVarHandle`. The spec is changed to specify that when accessing a heap buffer, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an `IllegalStateException` when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright `UnsupportedOperationException` for this case, since whether the access modes are supported depends on the byte buffer instance being used.

 - `ByteBuffer::alignedSlice` and `ByteBuffer::alignmentOffset`: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an `UnsupportedOperationException` for all unit sizes greater than 1, when the buffer is non-direct.

These decisions are also informed by similar conclusions reached in the design of the FFM API. See the existing FFM javadoc on alignment: https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#segment-alignment (in particular, see the section starting with "If the segment being accessed is a heap segment, ...") 

Specification
-------------

 Specdiff: https://cr.openjdk.org/~jvernee/alignedOffset_specdiff/v3/overview-summary.html
Comments
Release note is here: https://bugs.openjdk.org/browse/JDK-8325879 This change was also covered in the recent email to quality-discuss: https://mail.openjdk.org/pipermail/quality-discuss/2024-January/001132.html (see '## Heads-up: Java Array Element Alignment - Weakening of Some Methods Guarantees ?')
14-02-2024
Moving to Approved for JDK 23. I recommend a release note and possibly also an explicit entry on a quality-discuss message.
13-02-2024
[~jvernee] has asked me to take a look at this from the point of view of deprecation. While it's true that some functionality is being removed here, it's not like the entire behavior of the method is being removed, for example, like Thread.stop(). Thus, deprecation for removal is unwarranted. We're also not discouraging use of these APIs, so ordinary deprecation is also unwarranted. This is an ordinary change that results in some incompatibilities, which should be judged according to the usual criteria. (As an aside, the changes seem justified to me.)
10-02-2024
I've attached the spec diff in v3.tar.gz file
08-02-2024
Moving to Provisional, not Approved. [~jvernee], before the CSR is Finalized, please include or attach some representation of the specification changes to the CSR (an external link is convenient, but some archival representation is needed too). [~dmlloyd], it is theoretically possible, but very unlikely, that these methods would be modified in update releases via maintenance reviews of the earlier platform specifications.
07-02-2024
[~dmlloyd] > Is there any chance that there could be a new replacement VarHandle-based API on all affected JDK versions, perhaps one that works on long[] and produces an atomics-friendly view handle for shorter types? It could either use long indices, or accept a base offset, or else limit the view in size to the size of the given view array class. The replacement API would be the FFM API, which is also VarHandle-based. It can be used like this: long[] arr = new long[10]; MemorySegment arrSeg = MemorySegment.ofArray(arr); VarHandle vh = ValueLayout.JAVA_INT.varHandle(); // accessing aligned ints vh.setVolatile(arrSeg, 0L, 42); // 0L is offset in bytes vh.getVolatile(arrSeg, 0L); // 42 > Alternatively, is there a way to guarantee that the specification of previous JDK versions won't be retroactively changed (perhaps a warning in the docs instead) so that it would be possible to publish a multi-release JAR that supports both old and new behavior? Specifications are not (typically) changed retroactively. I believe there has been only one exception in the history of Java. I have no plans to 'backport' these spec changes.
18-11-2023
Is there some place where there is a better description of alternatives? The text alludes to being able to use the new FFM API for this purpose by somehow treating a long[] as a byte[], however it's not immediately clear how this could be done, particularly given that the APIs are quite different, and that the FFM API only exists in more recent JDKs. Code that exists today which runs on Java 9 or later will be broken by this change, so it seems like there should be some solution in terms of replacement APIs that are usable on every JDK version that is changing. Is there any chance that there could be a new replacement VarHandle-based API on all affected JDK versions, perhaps one that works on long[] and produces an atomics-friendly view handle for shorter types? It could either use `long` indices, or accept a base offset, or else limit the view in size to the size of the given view array class. Alternatively, is there a way to guarantee that the specification of previous JDK versions won't be retroactively changed (perhaps a warning in the docs instead) so that it would be possible to publish a multi-release JAR that supports both old and new behavior?
17-11-2023