JDK-8307341 : JEP 472: Prepare to Restrict the Use of JNI
  • Type: JEP
  • Component: core-libs
  • Priority: P3
  • Status: Integrated
  • Resolution: Unresolved
  • Fix Versions: 24
  • Submitted: 2023-05-03
  • Updated: 2024-09-23
Related Reports
Blocks :  
JDK-8331671 - Implement JEP 472: Prepare to Restrict the Use of JNI
Relates :  
JDK-8310626 - JEP 454: Foreign Function & Memory API
Relates :  
JDK-8305968 - Integrity by Default
Description
Summary
-------

Issue warnings about uses of the [Java Native Interface (JNI)](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/index.html) and adjust the [Foreign Function & Memory (FFM) API](https://openjdk.org/jeps/454) to issue warnings in a consistent manner. All such warnings aim to prepare developers for a future release that ensures [integrity by default](https://openjdk.org/jeps/8305968) by uniformly restricting JNI and the FFM API. Application developers can avoid both current warnings and future restrictions by selectively enabling these interfaces where essential.


Goals
-----

- Preserve the status of JNI as a standard way to interoperate with native code.

- Prepare the Java ecosystem for a future release that disallows interoperation with native code by default, whether via JNI or the FFM API. As of that release, application developers will have to explicitly enable the use of JNI and the FFM API at startup.

- Align the use of JNI and the FFM API so that library maintainers can migrate from one to the other without requiring application developers to change any command-line options.


## Non-Goals

- It is not a goal to deprecate JNI or to remove JNI from the Java Platform.

- It is not a goal to restrict the behavior of native code called via JNI. For example, all of the native [JNI functions](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/functions.html) will remain usable by native code.


## Motivation

The [Java Native Interface (JNI)](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/index.html) was introduced in JDK 1.1 as the primary means for interoperating between Java code and native code, typically written in C. JNI allows Java code to call native code (a _downcall_) and native code to call Java code (an _upcall_). 

Unfortunately, any interaction at all between Java code and native code is risky because it can compromise the integrity of applications and of the Java Platform itself. According to the policy of [integrity by default](https://openjdk.org/jeps/8305968), all JDK features that are capable of breaking integrity must obtain explicit approval from the application's developer.

Here are four common interactions and their risks:

1. Calling native code can lead to arbitrary [undefined behavior](https://en.wikipedia.org/wiki/Undefined_behavior), including JVM crashes. Such problems cannot be prevented by the Java runtime, nor do they provoke exceptions for Java code to catch.

   For example, this C function takes a `long` value passed from Java code and treats it as an address in memory, storing a value at that address:

   ```
   void Java_pkg_C_setPointerToThree__J(jlong ptr) {
       *(int*)ptr = 3;
   }
   ```

   Calling this C function could corrupt memory used by the JVM, causing the JVM to crash at an unpredictable time, long after the C function returns. Such crashes, and other unexpected behaviors, are difficult to diagnose.

2. Native code and Java code often exchange data through [direct byte buffers](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/functions.html#newdirectbytebuffer), which are regions of memory not managed by the JVM's garbage collector. Native code can produce a byte buffer that is backed by an invalid region of memory; using such a byte buffer in Java code is practically certain to cause undefined behavior.

   For example, this C code constructs a 10-element byte buffer starting at address 0 and returns it to Java code. The JVM will crash when Java code attempts to read or write the byte buffer:

   ```
   return (*env)->NewDirectByteBuffer(env, 0, 10);
   ```

3. Native code can use JNI to [access fields](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/functions.html#accessing-fields-of-objects) and [call methods](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/functions.html#calling-instance-methods) without any access checks by the JVM. Native code can even use JNI to change the values of `final` fields long after they are initialized. Java code that calls native code can therefore violate the integrity of other Java code.

   For example, [`String`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/String.html) objects are specified to be immutable, but this C code mutates a `String` object by writing to an array referenced by a `private` field:

   ```
   jclass clazz = (*env)->FindClass(env, "java/lang/String");
   jfieldID fid = (*env)->GetFieldID(env, clazz , "value", "[B");
   jbyteArray contents = (jbyteArray)(*env)->GetObjectField(env, str, fid);
   jbyte b = 0;
   (*env)->SetByteArrayRegion(env, contents, 0, 1, &b);
   ```

    For another example, arrays are specified to [disallow access beyond their bounds](https://docs.oracle.com/javase/specs/jls/se22/html/jls-15.html#jls-15.10.4), but this C code can write past the end of an array:

   ```
   jbyte *a = (*env)->GetPrimitiveArrayCritical(env, arr, 0);
   a[500] = 3; // may be out of bounds
   (*env)->ReleasePrimitiveArrayCritical(env, arr, a, 0);
   ```

4. Native code which uses certain JNI functions incorrectly, principally [`GetPrimitiveArrayCritical`](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/functions.html#getprimitivearraycritical-releaseprimitivearraycritical) and [`GetStringCritical`](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/functions.html#getstringcritical-releasestringcritical), can cause [undesirable garbage collector behavior](https://shipilev.net/jvm/anatomy-quarks/9-jni-critical-gclocker/) that can manifest at any time during the program's lifetime.

The [Foreign Function & Memory (FFM) API](https://openjdk.org/jeps/454), introduced in JDK 22 as the preferred alternative to JNI, shares the [first and second risks](https://openjdk.org/jeps/454#Safety). In the FFM API we took a proactive approach to mitigating these risks, separating actions that risk integrity from those that do not. Some parts of the FFM API are thus classified as [restricted methods](https://cr.openjdk.org/~iris/se/22/spec/latest/index.html#Restricted-methods), which means the application developer must approve their use and opt in via a `java` launcher command-line option. JNI should follow the FFM API's example toward achieving integrity by default.

Preparing to restrict the use of JNI is part of a long-term coordinated effort to ensure that the Java Platform has [integrity by default](https://openjdk.org/jeps/8305968). Other initiatives include removing the memory-access methods in `sun.misc.Unsafe` ([JEP 471](https://openjdk.org/jeps/471)) and restricting the dynamic loading of agents ([JEP 451](https://openjdk.org/jeps/451)). These efforts will make the Java Platform more secure and more performant. They will also reduce the risk of application developers becoming trapped on older JDK releases due to libraries that break on newer releases when unsupported APIs are changed.


## Description

In JDK 22 and later releases, you can call native code via the Java Native Interface (JNI) or the Foreign Function & Memory (FFM) API. In either case, you must first load a native library and link a Java construct to a function in the library. These loading and linking steps are [restricted](https://openjdk.org/jeps/454#Safety) in the FFM API, which means that they cause a warning to be issued at run time by default. In JDK NN, we will restrict the loading and linking steps in JNI so that they also cause a warning to be issued at run time by default.

We refer to restrictions on loading and linking native libraries as _native access restrictions_. In JDK NN, native access restrictions will apply uniformly whether JNI or the FFM API is used to load and link native libraries. The exact operations that load and link native libraries in JNI, which are now subject to native access restrictions, are described [later](#Warnings-on-loading-native-libraries).

We will strengthen the effect of native access restrictions over time. Rather than issue warnings, a future JDK release will throw exceptions by default when Java code uses JNI or the FFM API to load and link native libraries. The intent is not to discourage use of JNI or the FFM API but, rather, to ensure that applications and the Java Platform have [integrity by default](https://openjdk.org/jeps/8305968).

### Enabling native access

Application developers can avoid warnings (and in the future, exceptions) by enabling native access for selected Java code at startup. Enabling native access acknowledges the application's need to load and link native libraries and lifts the native access restrictions.

Under the policy of [integrity by default](https://openjdk.org/jeps/8305968#Restrictions-on-standard-unsafe-APIs), it is the application developer (or perhaps deployer, on the advice of the application developer) who enables native access, not library developers. Library developers who rely on JNI or the FFM API should inform their users that they will need to enable native access using one of the methods below.

To enable native access for all code on the class path, use the following command-line option:

```
java --enable-native-access=ALL-UNNAMED ...
```

To enable native access for specific modules on the module path, pass a comma-separated list of module names:

```
java --enable-native-access=M1,M2,... ...
```

Code that uses JNI is affected by native access restrictions if

- It calls `System::loadLibrary`, `System::load`, `Runtime::loadLibrary`, or `Runtime::load`, or
- It declares a `native` method.

Code that merely calls a `native` method declared in a different module does not need to have native access enabled.

Most application developers will pass `--enable-native-access` directly to the `java` launcher in a startup script, but other techniques are available:

- You can pass `--enable-native-access` to the launcher indirectly, by setting the environment variable `JDK_JAVA_OPTIONS`.

- You can put `--enable-native-access` in an argument file that is passed to the launcher by a script or an end user, e.g., `java @config`

- You can add `Enable-Native-Access: ALL-UNNAMED` to the manifest of an executable JAR file, i.e., a JAR file that is launched via [`java -jar`](https://docs.oracle.com/en/java/javase/22/docs/specs/man/java.html#synopsis). (The only supported value for the `Enable-Native-Access` manifest entry is `ALL-UNNAMED`; other values cause an exception to be thrown.)

- If you create a custom Java runtime for your application, you can pass the `--enable-native-access` option to `jlink` via the `--add-options` option, so that native access is enabled for the resulting runtime image.

- If your code creates modules dynamically, you can enable native access for them via the [`ModuleLayer.Controller::enableNativeAccess`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/ModuleLayer.Controller.html#enableNativeAccess(java.lang.Module)) method, which is itself a restricted method. Code can dynamically check if its module has native access via the [`Module::isNativeAccessEnabled`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/Module.html#isNativeAccessEnabled()) method.

- The [JNI Invocation API](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/invocation.html) allows a native application to embed a JVM in its own process. A native application which uses the JNI Invocation API can enable native access for modules in the embedded JVM by passing the `--enable-native-access` option when [creating the JVM](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/invocation.html#jni_createjavavm).


### Enabling native access more selectively

The `--enable-native-access=ALL-UNNAMED` option is coarse-grained: It lifts native access restrictions on JNI and the FFM API for all classes on the class path. To limit risk and achieve higher integrity, we recommend moving JAR files that use JNI or the FFM API to the module path. This allows native access to be enabled for those JAR files specifically, not for the class path as a whole. A JAR file can be moved from the class path to the module path without being modularized; the Java runtime will treat it as [automatic module](https://dev.java/learn/modules/automatic-module/) whose name is based on its filename.


### Controlling the effect of native access restrictions

If native access is not enabled for a module then it is illegal for code in that module to perform a restricted operation. What action the Java runtime takes when such an operation is attempted is controlled by a new command-line option, `--illegal-native-access`, which is similar in spirit and form to the `--illegal-access` option introduced by [JEP 261](https://openjdk.org/jeps/261#Relaxed-strong-encapsulation) in JDK 9. It works as follows:

  - `--illegal-native-access=allow` allows the operation to proceed.

  - `--illegal-native-access=warn` allows the operation but issues a warning the first time that illegal native access occurs in a particular module. At most one warning per module is issued.

    This mode is the default in JDK 24. It will be phased out in a future release and, eventually, removed.

  - `--illegal-native-access=deny` throws an `IllegalCallerException` exception for every illegal native access operation.

    This mode will become the default in a future release.

When `deny` becomes the default mode then `allow` will be removed but `warn` will remain supported for at least one release.

To prepare for the future, we recommend that you run existing code with the `deny` mode to identify code that requires native access.


### Aligning the FFM API

Prior to JDK 24, if one or more modules had native access enabled via the `--enable-native-access` option, then attempts to call [restricted FFM methods](https://openjdk.org/jeps/454#Safety) from any other module would cause an `IllegalCallerException` to be thrown.

To align the FFM API with JNI, we will relax this behavior so that illegal native access operations are treated exactly the same by the FFM API as in JNI. This means that, in JDK 24, such operations will result in warnings rather than exceptions.

You can restore the old behavior with this combination of options:

```
java --enable-native-access=M,... --illegal-native-access=deny ...
```


### Warnings on loading native libraries

Native libraries are loaded in JNI via the [`load`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/Runtime.html#load(java.lang.String)) and [`loadLibrary`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/Runtime.html#loadLibrary(java.lang.String)) methods of the `java.lang.Runtime` class.
(The identically named convenience methods [`load`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#load(java.lang.String)) and [`loadLibrary`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#loadLibrary(java.lang.String)) of the `java.lang.System` class merely invoke the corresponding methods of the system-wide `Runtime` instance.)

Loading a native library is risky because it can cause native code to run:

- If a native library defines [initialization functions](https://tldp.org/HOWTO/Program-Library-HOWTO/miscellaneous.html#INIT-AND-CLEANUP) then the operating system runs them when loading the library; these functions contain arbitrary native code.

- If a native library defines a [`JNI_OnLoad`](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/invocation.html#jni_onload) function then the Java runtime invokes it when loading the library; this function also contains arbitrary native code.

Because of the risks, the `load` and `loadLibrary` methods are restricted in JDK 24, just as the [`SymbolLookup::libraryLookup`](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/foreign/SymbolLookup.html#libraryLookup(java.nio.file.Path,java.lang.foreign.Arena)) methods are restricted in the FFM API.

When a restricted method is called from a module for which native access is not enabled, the JVM runs the method but, by default, issues a warning that identifies the caller:

```
WARNING: A restricted method in java.lang.System has been called
WARNING: System::load has been called by com.foo.Server in module com.foo (file:/path/to/com.foo.jar)
WARNING: Use --enable-native-access=com.foo to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled
```

At most one such warning is issued for any particular module, and only if a warning has not yet been issued for that module. The warning is written to the standard error stream.


### Warnings on linking native libraries

When a `native` method is first called, it is automatically linked to a [corresponding function](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/design.html#resolving-native-method-names) in a native library. This linking step, which is called _binding_, is a restricted operation in JDK 24, just as [obtaining a downcall method handle](https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/foreign/Linker.html#downcallHandle(java.lang.foreign.FunctionDescriptor,java.lang.foreign.Linker.Option...)) is a restricted operation in the FFM API.

Upon the first invocation of a `native` method that is declared in a module for which native access is not enabled, the JVM binds the `native` method but, by default, issues a warning that identifies the caller:

```
WARNING: A native method in org.baz.services.Controller has been bound
WARNING: Controller::getData in module org.baz has been called by com.foo.Server in an unnamed module (file:/path/to/foo.jar)
WARNING: Use --enable-native-access=org.baz to avoid a warning for native methods declared in org.baz
WARNING: Native methods will be blocked in a future release unless native access is enabled
```

At most one such warning is issued for any particular module. Specifically:

- The warning is issued only when a `native` method is bound, which happens the first time that the `native` method is called. The warning is not issued every time that the `native` method is called.

- The warning is issued the first time that any `native` method declared in a particular module is bound, unless a warning has already been issued for that module.

The warning is written to the standard error stream.


### Identifying the use of native code

- The JFR events `jdk.NativeLibraryLoad` and `jdk.NativeLibraryUnload` track the loading and unloading of native libraries.

- To help identify libraries that use JNI, a new JDK tool, tentatively named `jnativescan`, statically scans code in a provided module path or class path and reports uses of restricted methods and declarations of `native` methods.


## Future Work

- To promote reliable configuration, allow a module's declaration to assert that the module requires native access, whether via JNI or the FFM API. At startup, the Java runtime would refuse to load any module that requires native access but which does not have native access enabled on the command line.

- To allow the use of the FFM API but not JNI, offer a command-line option that enables the use of the former but not the latter. JNI allows native code to break the encapsulation of Java code, which could interfere with future JVM optimizations in ways that use of the FFM API does not.


## Risks and Assumptions

- JNI has been part of the Java Platform since JDK 1.1, so there is a risk that existing applications will be impacted by restrictions on the use of JNI. An analysis of artifacts on Maven Central found that about 7% of existing artifacts depend on native code. Of these, about 25% use JNI directly; the remainder depend on some other artifact that uses JNI, either directly or indirectly.

- We assume that developers whose applications rely directly or indirectly on native code will be able to configure the Java runtime to enable the use of JNI via `--enable-native-access`, as [described above](#Enabling-native-access). This is similar to how they can already configure the Java runtime to disable strong encapsulation for modules, via `--add-opens`.


## Alternatives

- Rather than restrict the loading of native libraries and the binding of `native` methods, the JVM could apply access control rules when native code uses [JNI functions](https://docs.oracle.com/en/java/javase/22/docs/specs/jni/functions.html) to access Java fields and methods. However, this is insufficient to maintain integrity because any use of native code can lead to undefined behavior, whether or not it uses JNI functions. Portions of the FFM API are restricted for the same reason, even though the FFM API does not offer access to Java objects from native code.
Comments
> We are super early adopters in the field and I am trying to tell you what our problems are with the current design. Yes, and that feedback is appreciated and collected, but we'll need to collect more over the coming months before we make any decisions. > But you should keep in mind the more barriers you erect for users, the more workarounds and hacks will get deployed, harming platform integrity in the end. Because the platform needs absolute confidence in its invariants (e.g. for performance program transformations) no workarounds that violate the platform's integrity will be possible once the process is complete. Integrity will be soundly enforced because that's the only way it can work. Currently, the barriers to attaining integrity -- a prerequisite for portability, security, and performance -- are higher than those for breaking it and cause bigger problems for more people, and so we're focused on shifting the balance of the burden to better reflect the balance of sometimes-contradictory needs that different users have. If we can then reduce the lesser burden on the smaller number of people who need to disable integrity without increasing the burden on those who want the best performance, portability, and security, we will of course do that.
10-09-2024
> As always, to make sure we find the best solution we'll proceed cautiously and learn from the field what the problems are We are super early adopters in the field and I am trying to tell you what our problems are with the current design. We have a very ugly workaround and are not blocked. But you should keep in mind the more barriers you erect for users, the more workarounds and hacks will get deployed, harming platform integrity in the end.
10-09-2024
> The simple fix is to allow unqualified privileges, which would avoid most friction. I believe it will not avoid friction but add friction when the risks it entails materialise, and such risks do materialise sooner or later. We must take into account your use case as well as all others, and look for things that maximise value for the greatest number of people over the next twenty years of the platform, recognising that sometimes different users have conflicting needs. As always, to make sure we find the best solution we'll proceed cautiously and learn from the field what the problems are, how many people they affect and how much they affect them, and consider our way forward after, not before, we learn what we need to learn.
09-09-2024
The simple fix is to allow unqualified privileges, which would avoid most friction. A requirement for specifying all root module names can still be introduced when the static module-based feature is implemented. Having to repeat internal module names like this, unfortunately, is a game-breaker for us as it hinders our ability to change in the future, especially when invalid module names break as well. Those internal modules appear and disappear between configurations and versions, and we don't even want the user to know about them, just like with encapsulated details of a class. Unfortunately, this means we must resort to quite ugly measures and gain native access through SharedSecrets for all other modules than truffle.
09-09-2024
We're aware it will cause some friction but don't think it's all that much friction, especially when considering the potential of this enhancement to Java's capabilities and user experience as part of the larger integrity by default goal, which could only be depended on once all integrity loopholes are closed. I agree that it's possible to reduce that friction a bit -- say, by allowing modules to propagate a capability they've been given to other modules -- but I don't think it's urgent enough to commit *at this time* to a specific version (which doesn't necessarily mean that improvements won't come soon).
09-09-2024
> If you believe that adding the flag enabling native access to Truffle modules is too burdensome on application developers, then you could even write a generic launcher called `truffle` that would enable native access to any Truffle module, but work the same as the `java` launcher in every other way. As a library, we integrate with Java frameworks of all sorts, which all rely/hardcode the Java launcher. We can't realistically choose the launcher for Java embeddings. > I don't think we can currently conclude that adding a flag is a problem with such a high magnitude that it requires an immediate and possibly suboptimal solution. This JEP, as it is written, will cause more breakage/friction than necessary for the users of Truffle and many other Java libraries with native libraries. I hope you reconsider.
09-09-2024
> Libraries seem to be 2nd class citizens on the Java platform. Libraries are always second class on every platform (e.g. compilers cooperate through intrinsics with the standard library in some ways that they cannot with others), and in Java it's not possible for a library to implement, say, the MethodHandle class as efficiently as the standard library does -- so the standard library is always privileged in some ways -- but I don't think they're second class in Java in the way you say they are. > They can't benefit from module integrity like JDK modules, because they might be put on the class-path and run in the unnamed module. What determines the desired level of integrity is always the launcher, which is picked by the application. It is true that the stock `java` launcher in the JDK only does what it can with modules it knows about, but other launchers could do different things. If you believe that adding the flag enabling native access to Truffle modules is too burdensome on application developers, then you could even write a generic launcher called `truffle` that would enable native access to any Truffle module, but work the same as the `java` launcher in every other way. > Additionally, they are not excused from the native access check like JDK modules are. That, too, is the decision of the launcher, and different launchers can make different decisions. > I fear the biggest integrity-buster atm is still the prominent use of the class-path in the community. The use of the classpath has no impact on the integrity of the platform. An application launcher that puts some libraries in the same module just says that it's not interested in whether or not one of those libraries can break the other's invariants, but it doesn't allow them to violate the platform's own invariants (it could do so if it wanted to, though). The integrity of the platform itself has a particularly big impact -- for example, without it, neither programs nor the runtime could trust that Strings are immutable -- but ultimately it's the application that decides which boundaries it wants to respect. > we will have all the mentioned problems with the release of JDK 24 or latest after adopting Panama fully. There are many real problems we'd like to solve, and we take the time solving them. I don't think we can currently conclude that adding a flag is a problem with such a high magnitude that it requires an immediate and possibly suboptimal solution. > I see that risk, but the cost of the current solution is too high unless done using module-descriptors. The warnings will help us evaluate the cost to the ecosystem as a whole. We don't want to make any decisions until we can better judge how many people will be affected and to what degree. > I'd say it's a good compromise because nobody is fully happy. Our goal isn't to find a compromise between users with different needs but to give the greatest value to the greatest number of Java users. We would certainly consider allowing modules to *statically* transfer their native access -- if it's been enabled by the application -- to other modules as they can currently only dynamically, but the urgency of doing so will depend on what we learn from the warning. I don't think we can commit at this time that such a feature will appear in a specific release. Since we don't expect the warning to become an error before JDK 26, I think we still have some time to consider our options.
09-09-2024
> other integrity-busting mechanisms -- including deep reflection and dynamically loaded agents -- that others want to employ. Libraries seem to be 2nd class citizens on the Java platform. They can't benefit from module integrity like JDK modules, because they might be put on the class-path and run in the unnamed module. Additionally, they are not excused from the native access check like JDK modules are. I'd argue the work done by libraries to support unnamed and named module worlds won't give them any payback. I fear the biggest integrity-buster atm is still the prominent use of the class-path in the community. > We usually try not to prioritise shipping a solution before we have a good grasp of the magnitude of the problem it's intended to address. > Over the next few releases, we will monitor the cost of runtime configuration in the field We already prepared for and plan to make big use of Panama, so it will not remain a warning for us much longer. We can't take the time to monitor the situation, we will have all the mentioned problems with the release of JDK 24 or latest after adopting Panama fully. Ironically, as we just learned from our latest prototype, our workaround around the restrictions of JEP 472, to make configuration for our users more reasonable, will involve breaking integrity. > One problem with allowing to break invariants wholesale is that the risks it takes may not materialise for a while, and by the time they do, the cost of fixing their impact may be high. I see that risk, but the cost of the current solution is too high unless done using module-descriptors. You couldn't resolve my runtime configuration concerns either. For now, I'd suggest letting the user choose how far they want to go and support qualified and non-qualified access privileges. It takes very little effort, still gets us one step closer to integrity-by-default, but at the same time causes significantly less friction. I'd say it's a good compromise because nobody is fully happy.
09-09-2024
> Here is my concrete suggestion ... While you may be concerned only with native access, integrity by default covers, in the same way, other integrity-busting mechanisms -- including deep reflection and dynamically loaded agents -- that others want to employ. We usually try not to prioritise shipping a solution before we have a good grasp of the magnitude of the problem it's intended to address. The problem of loss of portability alone was severe -- virtually all the pain migrating from 8 to 9+ was due to it -- and the first component of integrity by default, (deep reflection being disabled by default) was introduced in JDK 16. Over the next few releases, we will monitor the cost of runtime configuration in the field, on the ecosystem as a whole, and only then consider what measures would be appropriate to reduce it, and how urgent they are. Overestimating the severity of a problem can be just as bad as underestimating it, which is why we're proceeding carefully and gradually. In particular, other than issuing a warning to the console, there is no change in behaviour in JDK 23 even if *no* new runtime configuration options are provided, and, as the JEP states, more serious changes to native access will not take place before JDK 26. > other rather new runtimes like Deno that call themselves "secure by default" also have unqualified access privileges. Our standards must be higher because Java applications are bigger and longer lived than JS applications, and the JDK's standard library is much bigger, too, meaning there's a wider surface for portability issues, attacks, and lost optimisations (security is not the only concern that requires integrity). One problem with allowing to break invariants wholesale is that the risks it takes may not materialise for a while, and by the time they do, the cost of fixing their impact may be high. For example, it's certainly possible that the impact on portability, performance, and security is low for a while and then suddenly becomes high (possibly due to an internal JDK change affecting portability or performance, or due to a newly discovered vulnerability in some combination of libraries), at which point the application may have already accumulated quite a few integrity-busting libraries without careful consideration of the risks, making their replacement costly.
07-09-2024
> The "statically checked" additions provide fail-fast. They don't remove the need for flags. Flags are the feature either way. There is only a single command line option required if the module persists the native access privilege in the module-descriptor. All we need to do at runtime is to check that the native-library privilege was supplied to the main module. Everything else we can verify at module load time (and compile time). We wouldn't even need to repeat the main module name. All that is needed is a --enable-native-access. We get module-descriptor granularity and ease of use. Here is my concrete suggestion: 1) Allow usage of --allow-native-access=module1,module2 and --allow-native-access=ALL-UNNAMED as before. 2) Allow no-args usage of --allow-native-access to indicate all modules have native-access in JDK 24. 3) In a future JDK, implement a low-friction module-descriptor based solution with static and dynamic validation for privileges (including e.g. unsafe). Looking around, other rather new runtimes like Deno that call themselves "secure by default" also have unqualified access privileges. > Application developers already have to change the runtime configuration every time they change their dependencies. The majority of configuration changes happen in the class or module-path, generated by a Maven dependency list. Which concrete dependency change do you have in mind that currently requires a manual runtime configuration change?
06-09-2024
> 1) Why do we need to impose breaking changes on module-system users when there are statically checked solutions on the horizon? The "statically checked" additions provide fail-fast. They don't remove the need for flags. Flags are the feature either way. > 2) Why are these restrictions imposed only on external libraries, but not on boot modules like java.desktop? What makes them different? Wouldn't that also violate integrity-by-default? The restrictions apply to *all* modules. As always, the launcher can provide permissions to modules it knows about because the launcher is the program the user ultimately launches. The default JDK launcher offers the permissions to the modules it knows about, but app developers provide their own launchers (either with jlink or with a script). The application user should never configure these flags. The runtime is configured by the launcher. > 3) Do you expect that class-path users will benefit from integrity-by-default? If so, how? For example, module exports are not respected by jars on the class-path. I'm not sure what is meant by "class path users". When launching a Java program, the application configures the runtime as it sees fit. It can put multiple JARs in a single module with the -cp flag and some JARs each in its own module with the -p flag. Integrity by default works as usual either way at a module-granularity. If today you put all your libraries in a single module and have --enable-native-access=ALL-UNNAMED, tomorrow you can put only the library that needs native access on the module path and enable access on a more fine-grained level (in fact, the JEP recommends exactly that). > 4) If class-path users don't get most of the benefits that integrity-by-default provides, why should they care about native access restrictions? Everyone gets the same benefits, and can choose -- even on a library by library basis -- at what granularity they want to control permissions. > The user might still need to opt-in for native access with the main module on the command line, but it would not need to repeat dependent transitive module names. Yes, it's possible that in the future we'll offer a way for a module to inherit another module's permission, requiring fewer module names, but while reducing the listed names from, say, 2-5 to 1-2 isn't some huge gamechanger. It's a nice improvement, but it's not a must-have. > For many developers, this means that they need to change the JVM command line every time they change their dependencies involving a dependency that requires native access. Application developers already have to change the runtime configuration every time they change their dependencies. Also, they there's a good chance they need to change it with every JDK version. An application consists of a tightly-coupled triple of classes, runtime, and runtime configuration. It isn't now and has never been a goal for the same configuration to work across different runtime or application versions. > As my practical examples with Truffle outlined, this is at least twice as complicated as it was before Ok, but for developers who want the best portability, security, and performance it's still at least 10x easier than it was before, so the tradeoff is worth it. > The --opens property can today already be configured with modules. I don't see why native access shouldn't get the same treatment. `opens` gives another module permission to violate your own invariants. Native access means getting permission to violate everyone else's invariants, so it must be done in the application configuration. It's up to the application to decide whether a library can have such a global effect. > I think the current reality is that we don't know how big the impact is. That's why JDK 23 is only emitting a warning. We do things gradually to allow us to reshuffle our future priorities. > Having to add a flag, even if it is just static, is infinitely more complex than what they have to do now, namely nothing. The ability to have portability, security and better performance is a huge benefit. The warning in 23 will help us consider the priorities of reducing the number of modules you need to specify > It also complicates command lines used across multiple JDK versions, where the option might not be available yet. Runtime configurations have never worked well across multiple JDK versions, and are not expected to. A Java application -- and the runtime configuration is part of the application -- is tied to a specific JDK version.
06-09-2024
Thank you for the responses. I have more questions: 1) Why do we need to impose breaking changes on module-system users when there are statically checked solutions on the horizon? Shouldn't we rather delay the change and allow configuration with the module system, just like with opens? 2) Why are these restrictions imposed only on external libraries, but not on boot modules like java.desktop? What makes them different? Wouldn't that also violate integrity-by-default? 3) Do you expect that class-path users will benefit from integrity-by-default? If so, how? For example, module exports are not respected by jars on the class-path. 4) If class-path users don't get most of the benefits that integrity-by-default provides, why should they care about native access restrictions? Some more thoughts: > just as module A can declare a dependency on module B, but module B must still be provided by the application The module graph is a DAG, so I don't think it could give itself native access. The user might still need to opt-in for native access with the main module on the command line, but it would not need to repeat dependent transitive module names. Repeating dependent transitive module names is a configuration duplication and encapsulation breakage, not a minor inconvenience. Even the most cleanly-used module paths would still need to duplicate that part of their configuration on the Java command line. There is going to be a significant impact on the development process as well. Development configurations are not static; they change all the time during experimentation. For many developers, this means that they need to change the JVM command line every time they change their dependencies involving a dependency that requires native access. As my practical examples with Truffle outlined, this is at least twice as complicated as it was before and it keeps on giving with every configuration change. The --opens property can today already be configured with modules. I don't see why native access shouldn't get the same treatment. It would avoid all the additional work we have to do to provide native access to our language implementations in a reasonable way. The integrity by default JEP (https://openjdk.org/jeps/8305968) does not resolve my usability concerns with modules. But if I missed something, please point me to it. > I don't think that adding a flag (which is always the same) really increases their friction a great deal. > So all in all, requiring the flag for the classpath adds a very small bit of burden for some in exchange for reducing it more significantly overall. > For the small percentage of developers who use libraries which require native access I think the current reality is that we don't know how big the impact is. We just know there are 7% of artifacts which is a very significant amount. Obviously, 100% of our users need native access, which makes us care more than others. This will be the first breaking change of that sort in a very long time and it will affect many users. I can't remember a breakage of that sort in the last 20 years. So I don't think we can fully judge what impact it is going to have. Having to add a flag, even if it is just static, is infinitely more complex than what they have to do now, namely nothing. It also complicates command lines used across multiple JDK versions, where the option might not be available yet. > The jnativescan tool can help the user I think it could be a good idea to mention jnativescan in the error message. I don't think Java users will discover it otherwise.
06-09-2024
[~chumer] Let me first dispense with a couple of minor points: 1. The jnativescan tool can help the user know the modules that require native access without running the application even once. 2. We do plan to allow modules to specify their requirement of native access, but that is to allow fail-fast; access would still have to be enabled explicitly by the application -- just as module A can declare a dependency on module B, but module B must still be provided by the application. That's because libraries may *ask* for special privileges, but they must not be able to grant it to themselves, as that would negate some of the benefits of this feature (they can, however, transfer them as you pointed out). I think this answers your first question. Now to the heart of your point. The new capabilities added by this new feature and others like reduce rather than add friction *overall*, but they may add some to some people. It's like collecting less taxes overall, but collecting more from some. It's understandable that those who experience a higher burden may complain, but it can only be done in the context of considering the overall burden on everyone. The people who care about making their application as portable, secure, and performant as possible -- i.e. those who "care about integrity" -- have had pretty much infinite friction so far in achieving the important capabilities this feature adds. Only dynamic analysis can tell if invariants are broken, and that really adds *a lot* of friction -- much more than adding flags. Not only that, it's insufficient for the platform itself for the purpose of performance optimisations it may wish to do based on trusted invariants. SecurityManager could have hypothetically offered the integrity required for portability and security, but configuring it correctly in practice was difficult to the point of impracticality for nearly everyone. Now, there are those who may not care as much about portability, security, and performance, and they will, indeed, experience more friction, but much less than the friction reduced for those who do care more about portability, security, and performance. As for your second question about the classpath: 1. Allowing it to break all invariants would not be practical. It would mean that an application that has *any* class on the classpath -- for whatever reason (say, a library that still hasn't resolved its split packages) -- will not be able to enjoy the portability, security, and performance benefits even though most libraries don't require breaking any invariants. Remember, the goal is to reduce the burden overall. 2. I don't think it's true at all to say that people who don't use the module path don't care as much about portability, security, and performance as those who do. It's just that so far, using the module path couldn't get you those benefits -- because integrity by default isn't yet done -- so there was less reason to use it. But even if it is true that people who use the classpath don't care too much about portability, security, and performance, I don't think that adding a flag (which is always the same) really increases their friction a great deal. So all in all, requiring the flag for the classpath adds a very small bit of burden for some in exchange for reducing it more significantly overall.
05-09-2024
[~chumer] With respect, the idea that "Class-path users, on the other hand, tend to have little interest in their application's integrity." is profoundly wrong. In our experience, application developers are uniformly shocked by the integrity-busting antics of libraries on their class path -- this includes the use of native code to sidestep JVM access control -- and are generally keen to see the Java Platform guarantee integrity by default -- whether for a class path application they work on today, or a modular application they work on tomorrow. In practice, most developers of class path applications will never have to think about --enable-native-access. For the small percentage of developers who use libraries which require native access -- whether via JNI or the FFM API -- we expect the libraries' documentation to explain how to enable native access, so that developers can incorporate --enable-native-access in their own scripts. (The "Integrity by Default" JEP gives some other ways of passing the option, see https://openjdk.org/jeps/8305968#Restrictions-on-standard-unsafe-APIs) For modular applications, we plan to explore how a module's declaration can request native access for itself and then enable native access for other modules. This would be similar to how a module with access to a package can "open" that package to other modules (see https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/Module.html#addOpens(java.lang.String,java.lang.Module)). There is no specific timeline for this exploration at present.
05-09-2024
Before I ask my questions, let me quickly outline what this change means for the Truffle ecosystem: Since Truffle is tightly integrated with the JVM, it requires native access to compile code, as well as native access to load user libraries, such as those required by a Python module when executed with GraalPy. As a result, the GraalPy library requires native access, as do Truffle and its optimizing runtime implementation. As I understand it, if you want to use GraalPy with native libraries, you would now need to set --allow-native-access=org.graalvm.truffle,org.graalvm.truffle.runtime,org.graalvm.graalpy as a Java command line option. However, this is not immediately obvious. First, you will encounter an error due to missing privileges for org.graalvm.truffle, then for org.graalvm.truffle.runtime, and finally for org.graalvm.graalpy. As you can see, what used to work seamlessly now requires at least three failed attempts to identify the problem. Let's assume our user's application now wants to switch to Graal.js from its dependencies. To do this, the user would change the dependency to the Graal.js language implementation coordinates and rerun the application. Previously, this would work because the language was automatically picked up as a service. However, with this change, the existing command line will now fail because org.graalvm.graalpy is no longer on the module path. To fix this, you will need to remove the org.graalvm.graalpy module from the Java command line, possibly in several places. The complexity of making this change has effectively doubled. Fortunately, we have discovered a way to work around this issue. We will introduce our own API in Truffle to load native libraries. This will allow us to set --allow-native-access=org.graalvm.truffle for the org.graalvm.truffle module only, while Truffle can grant access to its language and tool implementations. That being said, I would prefer not to have to do all this work. So here are my questions: 1) Why aren't native-access privileges configured in the module descriptor? For example, a Java module can grant native access to services and required modules. Of course, a module can only do this if it has native access itself. The advantage of this approach is that we receive compiler warnings or errors for integrity violations. This would eliminate the need for multiple failed runs to figure them out. Ultimately, we wouldn't even need to specify anything on the command line, as the main module would already declare its native access requirements. 2) What do we expect to gain from enforcing native-access integrity checks for class-path users? In my experience, module-path users care a lot about the integrity of their applications and are willing to go the extra mile to maintain it. Class-path users, on the other hand, tend to have little interest in their application's integrity. I believe class-path users should not be burdened with setting --allow-native-access=ALL-UNNAMED. By removing this requirement, we would greatly reduce friction for users who don't prioritize integrity, especially with the next JDK upgrade. Thank you.
05-09-2024
[~jbachorik] Not very typical for Java agents to use native code but it may of course come up. As you know -javaagent puts the agent JAR file on the class path so the code is in the unnamed module of the application class loader. It would be very surprising to deploy with a Java agent and grant native access to all code on the class path as a side effect. I think the platform will need support for developing + deploying Java agents as modules at some point. There was an early prototype in JDK 9 but wasn't high priority at the time. I'm not sure what your last statement "makes the bytecode injection very difficult, nigh impossible" means, maybe bring that to serviceability-dev.
23-08-2024
Probably too late for this JEP but still could be considered for the next steps when the access to JNI is really clamped down. --- How would this affect Java agents loaded on startup - via `-javaagent:` VM option? You can ask, 'why Java agent instead of the native agent'? Well, most of the stuff the agents do is more or less related to bytecode injection and that one is infinitely easier in Java agent than in the native agent, thanks to ASM or the new Class-File API. However, the agent might want to delegate some features to a native implementation. In order to do this it needs to load the native library. And for this it needs enabled native access, and more likely than not it will have to enable it for ALL-UNNAMED because the agent would be in an unnamed module. This means the necessity to add an extra parameter, although the user already gave up the integrity by adding `-javaagent:` with an agent that can retransform classes and what is worse, the only way to enable native access for the agent would be granting the same access to all unnamed modules. Would a new manifest entry for the agents, similar to `Can-Retransform`, where the agent author could specify that the agent's need of JNI access make sense? I admit that this does not solve the too-widely granted access but at least would not require `--enable-native-access=ALL-UNNAMED` explicitly, once the integrity is already compromised by `-javaagent`. I know that it is possible to design an agent as a module, but realistically, there are very few such agents, mostly because wrapping the agent into a module makes the bytecode injection very difficult, nigh impossible.
23-08-2024
[~rpressler] thanks for adding the Maven numbers! I agree that the real question is how many applications will be affected, but still interesting to have the numbers for the libraries.
28-05-2024
[~simonis] I've added the following to the JEP: > An analysis of JNI code on Maven Central found that ~7% of the artifacts depend on native code. Of these, 1/4 use JNI directly and the remaining 3/4 depend on some other artifact that uses JNI (either directly, or indirectly). However, note that this change benefits not the *libraries* using JNI but the applications that consume those libraries, as they will now be made aware of the special risk they're exposed to. It is harder to estimate the percentage of applications that will need to acknowledge their use of JNI -- they are the ones directly impacted -- and their percentage may well be very different from 7%. However, 100% of applications will enjoy this enhancement: those that don't use native code will now *know* that they're not exposed to this risk, while those that do depend on native code will know that they are exposed to the risk.
28-05-2024
The Rust Foundation has recently published "Unsafe Rust in the Wild: Notes on the Current State of Unsafe Rust" [1]. It turns out that 19% of all creates with "significant code" make use of the unsafe keyword [2]. It would be interesting to get a similar statistic for all the Java packages at MavenCentral which directly or transitively use JNI. Maybe such a statistic already exists or somebody knows how to generate it? [1] https://foundation.rust-lang.org/news/unsafe-rust-in-the-wild-notes-on-the-current-state-of-unsafe-rust/ [2] https://foundation.rust-lang.org/news/unsafe-rust-in-the-wild-notes-on-the-current-state-of-unsafe-rust/#unsafe-rust-in-the-wild
27-05-2024
[~jvernee] > But, the current implementation throws an error when a value other than `ALL-UNNAMED` is used Fixed. > What is the expected behavior when the JNI RegisterNatives function is used, which can also be used to bind native methods? I think nothing should happen. RegisterNatives can only be called either through the invocation API or code that already has native access enabled.
02-11-2023
Re-reading this after the latest updates. I have a few more comments: - The section on the Enable-Native-Access attribute states: "ALL-UNNAMED is the only value supported in the attribute. Other module names will be ignored." But, the current implementation throws an error when a value other than `ALL-UNNAMED` is used [1]. - With regards to binding of `native` methods, the current text states: "The warning is given only when a native method is bound, which happens the first time that the native method is called. ..." What is the expected behavior when the JNI RegisterNatives function [2] is used, which can also be used to bind native methods? [1]: https://github.com/openjdk/jdk/blob/7a7b1e5a920d71ab717d8993c9258a01f1074a48/src/java.base/share/classes/sun/launcher/LauncherHelper.java#L639 [2]: https://docs.oracle.com/en/java/javase/21/docs/specs/jni/functions.html#registernatives
02-11-2023
Thanks Ron. This text update: > Enabling native access for the unnamed module is coarse-grained because it means granting native access to all classes on the class path. To make the most out of the integrity benefits of this feature, it is recommended to move the JARs that use JNI to the module path so that native access could be granted only to them and not to the unnamed module, thus keeping track of which libraries use JNI (the JARs do not need to be modular to do that; see Automatic Modules). Addresses my concern about bringing the benefit to existing use cases and shows how to avoid this being a cookie banner type problem.
27-10-2023
Thanks Alex. Those changes address my first two concerns.
24-10-2023
> Support the goal of integrity-by-default by issuing warnings when native code is invoked... The first paragraph proposes warnings when native code is invoked. The description indicates the warnings will occur on loading or binding native code rather than on invocation. Should this instead say "Support the goal of integrity-by-default by issuing warnings when native code is loaded or bound"? I know "invoked" is being used colloquially (here and other places) but should we be more exact? > Native code could use the JNI API to access fields and call methods without any access checks by the JVM or even change the values of final fields long after they are initialized because the implementation of JNI does not perform the appropriate checks. This is less an implementation concern with JNI and more of a specification issue. The JNI specification does not require access checks as there may not be sufficient Java-context to support performing the access check in native code. > To prepare developers for these restrictions, JDK NN issues a warning the first time that Java code loads a native library or binds a native method. The warning can be avoided in JDK NN by enabling the use of JNI on the command line; this also prepares the Java code to run without exceptions on a future JDK release. I'm 100% onboard with the goal of requiring a consistent opt-in between JNI and FFM. I have some hesitation regarding the implementation due to there being a single commandline option ("--enable-native-access=ALL-UNNAMED") that enables JNI for the entire classpath. The first legitimate use of JNI throws wide the doors for all other classpath libraries. Are application developers / users going to appreciate having to enable this switch to acknowledge their use of JNI on the classpath given it throws wide the barn door for all classpath libraries? The closest analogy is the "I accept the cookies" prompt on every website - something users reflexively click thru. This solution follows the existing path paved by FFM so from that perspective it's already the accepted solution. Does it actually help move the "integrity by default" needle for the classpath? The approach works great for modules but is of more limited value for the classpath. I don't think anyone would thank us for making them specify JNI-access on a per-package basis either though so this may be the best solution given the overconstrained problem.
20-10-2023