Bug ID: JDK-8349867 4.10.1.9: Simplify protected check

Type: Enhancement
Component: specification
Sub-Component: vm
Affected Version: 24

Priority: P4
Status: Open
Resolution: Unresolved

Submitted: 2025-02-12
Updated: 2025-05-06

Other
tbdUnresolved

4.10.1.9 contains a lot of discussion and code to accomplish something relatively simple: decide whether a referenced method would resolve to a protected method declared in a superclass, and if so, and ensure the type on the stack is assignable to the current class type.

A few potential simplifications to consider:

- Define a single rule for the relevancy test, and use negation ('\+ foo()') to trivially succeed when all the preconditions aren't met

- Get rid of the 'classesInOtherPkgWithProtectedMember' result list, which never gets used—it's just a search to see if there is at least one match

- Rename the check to something more descriptive, and simplify the inputs—e.g., it looks like 'Target' might be a better input than an entire StackFrame

It seems like the rules mishandle access within the same package. Scenario: - <C,L1> extends <B,L2> extends <Foo,L2> - A separate class, <Foo,L1>, also exists in the same package as C - Both Foo classes declare a protected field, 'x:I' - C does 'getfield Foo.x:I' on an instance of <Foo,L1> - The first 'passesProtectedCheck' fails (Foo is the name of a superclass) - 'classInOtherPackageWithProtectedMember' includes <Foo,L2> in its list (member is protected, L1 != L2) - The second 'passesProtectedCheck' fails (the list is nonempty) - The third 'passesProtectedCheck' fails (member is protected in <Foo,L1>) - The fourth 'passesProtectedCheck' fails (stack type is not <: C) The 'classInOtherPackageWithProtectedMember' rule is supposed to act as an optimization, avoiding unnecessary loading. But the rules want it to also take the place of the needed semantic check, "is this loaded class a superclass?" (I've confirmed that HotSpot successfully passes the protected check when a superclass Foo is in a different package but the referenced Foo, a different class, is not a superclass. And then the resolution-time access check successfully determines whether the referenced Foo is in the same package. So this is only a spec bug.)
06-05-2025
A bug for <init> method invocations: the stack frame passed to the check from invokespecial is 'NextStackFrame', which has no relationship to the "receiver" of the <init> call, which has already been popped. The stack could be empty. The stack could have a random instance of the current class. These arbitrary choices for stack shapes could have an observable impact on the outcome of the rules, as written. (I've confirmed that HotSpot performs the check using the actual type of the class being instantiated, so there's no implementation bug here.)
06-05-2025
More on the <init> method check: generally, attempting to invoke a protected <init> method in another package gets you an IAE at resolution time (5.4.4). The exception is when the method belongs to a superclass. So it is necessary for the verifier to close this hole for protected <init> methods declared in superclasses. The test should be: if this is a protected <init> method declared in a superclass belonging to a different package, fail.
06-05-2025
One confusing thing about this section is its application to <init> method invocations. Per JLS 6.6.2.2, protected constructors can't be invoked via 'new C(...)' outside of their package. The rules handle this case indirectly, but never explain what's going on. Might be easier to enforce it directly under 'invokespecial', or with a separate predicate.
03-05-2025
An inconvenience about the StackFrame parameter is that use sites, like putfield and invokevirtual, have to generate a meaningless PoppedFrame to get at the receiver type, popping other arguments. Somebody has to do this work somewhere though—there's a receiver type embedded in the stack frame, and we have to find it to see how it compares to the type of 'this'.
03-05-2025