Bug ID: JDK-8197445 Implementation of JEP 181: Nest-Based Access Control

JDK-8197445 : Implementation of JEP 181: Nest-Based Access Control

Type: CSR
Component: hotspot
Sub-Component: runtime

Priority: P3
Status: Closed
Resolution: Approved
Fix Versions: 11

Submitted: 2018-02-08
Updated: 2018-06-21
Resolved: 2018-06-20

Related Reports

CSR :	JDK-8010319 - Implementation of JEP 181: Nest-Based Access Control
Relates :	JDK-8204925 - 6.5: Clarify invokespecial

Description

Summary
-------

JEP 181: Nest-Based Access Control

Problem
-------

Closely-related classes sometimes need to share access `private` members. The Java language supports this, allowing access to all `private` members nested within the same top-level class. The JVM does not, requiring less-secure compiler workarounds and ad hoc solutions like `Unsafe.defineAnonymousClass()`. See [JEP 181](http://openjdk.java.net/jeps/181) for details.

Solution
--------

A _nest_ is a subset of classes in a package. Every class belongs to one nest (by default, a singleton nest containing only itself). Two classes belonging to the same nest are _nestmates_.

Two new JVM attributes are introduced: `NestHost` and `NestMembers`. The first points to a class's _nest host_ class (for example, the top-level class containing it); if there is no `NestHost` attribute, the class is its own nest host. The second, appearing in the nest host, enumerates the classes that are authorized to claim membership in a nest. 

The JVM access control rules are modified to allow access to `private` members from all of a class's nestmates. These checks occur on-demand: the nest host is not loaded (in order to consult its `NestMembers` attribute) until needed to grant access to an otherwise-inaccessible member.

The source language compiler is responsible for deciding which classes and interfaces are nestmates. For example, the javac compiler places a top-level class or interface into a nest with all of its direct, and indirect, nested classes and interfaces. The top-level enclosing class or interface is designated as the nest host. The JVM itself simply defines an access control context based on these attributes with no regard to any language-level scoping or nesting characteristics. In the future, nestmates need not directly correspond to source language constructs.

Traditionally, `invokespecial` is used to invoke `private` members, though `invokevirtual` also has this capability. Rather than perturb the complex rules about supertypes enforced by `invokespecial`, we require invocations of `private` methods in a different class to use `invokevirtual`. The `invokeinterface` instruction is enhanced to support invocation of `private` interface methods. In the process, we unify different method selection algorithms used by the JVM.

The reflection API of <code>java.lang.Class</code> is enhanced to support nest-related queries.

(The `javac` compiler is also updated to make use of nests, but this change does not impact any specifications. Some JLS changes are present to align the JLS description of runtime behavior with JVMS.)

Minor clarifications and adjustments are needed in the core reflection and MethodHandle API's. These are mainly non-normative/descriptive text changes.

<code>MethodHandle.Lookup.findVirtual</code> no longer throws <code>IllegaAccessError</code> if applied to a private interface method.

The specifications for class redefinition and retransformation, in the JVM TI and JDWP specifications, along with the <code>java.lang.instrument</code> and <code>com.sun.jdi</code> API's are enhanced to disallow any modification of the <code>NestHost</code> or <code>NestMembers</code> attributes. For JDWP and JDI we also clarify the notion of restricted redefinitions.

Specification
-------------

Proposed changes to JLS and JVMS are attached. Full updated specifications and specdiffs are available as follows:

 - JVMS:
   http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/JVMS/nestmates.html
 - JLS:
   http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/JLS/nestmates-jls.html
 - java.lang.Class:
   http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/java.lang/overview-summary.html
 -  java.lang.reflect:
   http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/java.lang.reflect/overview-summary.html
 -  java.lang.invoke:
   http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/java.lang.invoke/overview-summary.html
 -  JVM TI:
   http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/JVM-TI/diff.html
 -  java.lang.instrument:
   http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/java.lang.instrument/overview-summary.html
 -  JDWP:
   http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/JDWP/diff.html
 -  com.sun.jdi:
   http://cr.openjdk.java.net/~dholmes/8010319-JEP181/specs/com.sun.jdi/overview-summary.html

Comments

Thanks Joe. A couple of additional minor edits made by Alex. Updated specs attached.
21-06-2018
Moving to Approved.
20-06-2018
After significant discussion Alex Buckley reworked the Class documentation for describing nests and describing the semantics of the new nestmate methods. I've updated the links to the latest spec/spec-diff versions and attached a copy of all the proposed changes.
20-06-2018
From the JEP text "This relaxes the existing constraint that private interface methods must be invoked using invokespecial (JVMS 6.5) and more generally allows invokevirtual to be used for private method invocation, rather than adding to the complex usage rules surrounding invokespecial." Is not immediately apparent to me at least that the JVMS changes for nest mates only loosen existing restrictions and allow additional coding idioms (e.g. "use invokevirtual rather than invokespecial for private method access") while preserving the validity of all existing idioms (e.g. the idioms around invokespecial). I think it would be helpful for the JVMS to have a comment to this effect somewhere. The new text in the JVMS repeated using the phrase "class or interface" (and this phrase is commonly used in existing text as well). It may be helpful to have some positive affirmation that there are no strange interactions between enum-ness or annotation type-ness and being nest host or nest member. If one looks over the method specifications of java.lang.Class, there are many examples like the following: public Class<?> getDeclaringClass() throws SecurityException "If the class or interface represented by this Class object is a member of another class, returns the Class object representing the class in which it was declared. This method returns null if this class or interface is not a member of any other class. ..." public Class<?> getEnclosingClass() throws SecurityException "Returns the immediately enclosing class of the underlying class. If the underlying class is a top level class this method returns null." public String getCanonicalName() "Returns the canonical name of the underlying class as defined by the Java Language Specification. Returns null if the underlying class does not have a canonical name (i.e., if it is a local or anonymous class or an array whose component type does not have a canonical name)." In other words, a common pattern for methods in java.lang.Class is to first define the behavior when the concept in question (declaring class, enclosing class, canonical name, etc.) exists for the class and then to state that null is used as special value when the concept in question is not defined or valid for the class. (This is arguably not the best contemporary API design, but it is locally consistent in Class.) Given this background, one could reasonably expect that after public Class<?> getNestHost() "Returns the nest host of the object represented by this Class." the following sentence would be something like "If the Class does not have a nest host, null is returned." Putting aside the semantics of getNestHost, I view the current form of the specification where the the most common case where a class does not have an explicit nest host defined means this is returned as burrying the lede and not suitable. For getNestMembers, if the current semantics are kept, I recommend either deleting the api note or rephrasing it as "The reference implementation does not delete duplicate nest members." I don't think the "This implementation" phrasing is helpful in this context; I used the wording "In the reference implementation..." in a JDK 11 change (JDK-8202563).
11-06-2018
[~mcimadamore] thank you for the comment on the enum constructors.
11-06-2018
On enums construction; the only possible interaction between this JEP and enums is with respect to private constructors. That is, when you have a class with a private constructor, and the constructor is accessed _outside_ the class itself (e.g. from another member of the nest), javac used to adopt a special code-generation scheme where the private constructor received a package-private accessor trampoline, accepting an extra token class value. This code generation strategy is only there if the private constructor is accessed from outside. Now, in the case of enums, all construction happens in the enum constants themselves, which are desugared as static fields of the enum class. As such, there has never been a need to emit an access trampoline for enum's private constructors - which means nestmates cannot possibly alter what javac has been doing so far. Of course if an enum has a private member and this member is accessed from outside the enum, nestmates will reduce the amount of boilerplate required for such access to take place - but with respect to construction, I don't see anything special here.
11-06-2018
Following code review there were some minor changes to the core reflection API docs as presented here for the EG to consider and approve: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-May/000694.html An updated tarball has been attached.
24-05-2018
Spec changes attached - bundle last updated 2018-05-02
02-05-2018
I've uploaded updates to JVMS and JLS. This addresses the discussion above about JLS 13.1, and improves the presentation throughout, reflecting changes made by Alex as he integrates these specs into the official documents.
02-05-2018
> Before the request is finalized, for archival purpose please attach snapshots of all the specifications being changed in addition to the JVM and JLS. I will attached the bundle of files that can currently be found under: http://cr.openjdk.java.net/~dholmes/8010319/specs/ Note however that the final form of the JLS and JVMS changes, as they will appear in the PDF may be different to what is attached in the html form.
26-03-2018
With regards to the change to JLS 13.1 I first note that any change here is incidental to the Nestmates work. We do not require any change here for Nestmates to work. I find the InnerClasses attribute somewhat confusing. My reading of JVMS 4.7.6 indicates there is an entry for the immediately enclosing class, plus entries for all immediately enclosed classes - but not entries for other enclosing classes up to the top-level class. However the presentation of the InnerClasses attribute by javap does not seem to show this and has other entries in it.
26-03-2018
Clarification as we lost markup here: > "Every nested class and nested interface must have a symbolic reference to each of its immediately enclosing class enclosing classes (8.1.3)." actually reads: "Every nested class and nested interface must have a symbolic reference to each of its enclosing classes (8.1.3)."
26-03-2018
> For example, by the current proposed specification there is no need for a class to list itself as its own NestMember, but my recommendation would be to forbid this state if it is not necessary. What would be the advantage of singling out the "listing itself as its own NestMember" check? It's presence is harmless, the check adds overhead, and it shouldn't be considered special in relation to the other "validity" checks you might propose, if we were to be strict about this. And some other source compiler might decide to include it for its own reasons.
26-03-2018
> API > Class.getNestHost: > "If there is any error accessing the nest host, or the nest host is in any way invalid, then this is returned." > > Allow laxity here seems out of character for the reflection API. Yes but the EG (or specifically John :)) wanted these hidden at this level. I insisted they must be visible somewhere so getNestMembers() performs validity checks. See public discussion thread that starts here: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-October/000386.html but particularly from here: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-October/000394.html
26-03-2018
The the NestMember and NestHost attributes, it would be possible to define well-formedness conditions that would constrain the legal states. For example, by the current proposed specification there is no need for a class to list itself as its own NestMember, but my recommendation would be to forbid this state if it is not necessary.
23-03-2018
> JLS Changes: > > "Every nested class and nested interface must have a symbolic reference to each of its immediately enclosing class enclosing classes (8.1.3)." > > Is this true? The nested classes refer to the top-level nest host, not all the intermediate levels. We had an argument for this, as hinted at in the comment: "It was already true that deeply-nested classes would refer to all of their enclosing classes (per the specification of the InnerClasses attribute)." The idea is that once you refer to your enclosing class, you're required to refer to _its_ enclosing class, etc. But, scrutinizing that argument again, it doesn't seem to be true when there is a local/anonymous class in the chain. A class nested in a local/anonymous class will only reference its enclosing classes up to the local/anonymous class. I'll adjust the spec.
23-03-2018
> JVMS changes: > > Can a class list itself as a NestMember? This doesn't seem to be forbidden by the current spec Sure. NestMembers is for on-demand validation of classes that claim nest membership. Beyond that use, it is free to contain any other garbage data it wishes. However, "Items that do not meet this description are discouraged and will be ignored by access checking." There is no _need_ for a class to list itself in its NestMembers. > No split packages so nest mates must also be in the same module as they are in the same package. Correct? Yep. An ICCE occurs during access checking if the nest host is in a different runtime package (5.4.4). > "This array is consulted during access checking (5.4.4). It should consist of references to other classes and interfaces that belong to the same run-time package and have NestHost attributes referencing this class or interface. Items that do not meet this description are discouraged and will be ignored by access checking." > > Is this too weak of a check? Why? Think if it like the constant pool: you can put whatever garbage you want in the constant pool, but only the entries that get referenced will be resolved. Same with NestMembers: if the named class doesn't have a NestHost attribute pointing to this class, the entry is irrelevant.
23-03-2018
Moving to Provisional. Before the request is finalized, for archival purpose please attach snapshots of all the specifications being changed in addition to the JVM and JLS. Several comments/questions on the changes for consideration: JVMS changes: Can a class list itself as a NestMember? This doesn't seem to be forbidden by the current spec No split packages so nest mates must also be in the same module as they are in the same package. Correct? "This array is consulted during access checking (5.4.4). It should consist of references to other classes and interfaces that belong to the same run-time package and have NestHost attributes referencing this class or interface. Items that do not meet this description are discouraged and will be ignored by access checking." Is this too weak of a check? JLS Changes: "Every nested class and nested interface must have a symbolic reference to each of its immediately enclosing class enclosing classes (8.1.3)." Is this true? The nested classes refer to the top-level nest host, not all the intermediate levels. API Class.getNestHost: "If there is any error accessing the nest host, or the nest host is in any way invalid, then this is returned." Allow laxity here seems out of character for the reflection API. For completeness, I think there should be supporting changes in javax.lang.model.util.Elements. I can help design those under a follow-up issue.
23-03-2018