JDK-4248990 : Method lookup inconsistent with classic VM behavior
  • Type: Bug
  • Component: specification
  • Sub-Component: vm
  • Affected Version: 1.3.0
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 1999-06-23
  • Updated: 2012-10-09
  • Resolved: 1999-10-14
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
1.3.0 betaFixed
Related Reports
Relates :  
Relates :  
Relates :  
Description
The semantics of method lookup, as determined by our current specs, and 
implemented by Hotspot, differ from the behavior of the classic VM.
This breaks some user programs, and may require non-trivial work arounds
by customers.

We need to decide whether the spec should be changed (and it what way).
Here is  an example, based on real customer code:

package testcase1;

public abstract class Base extends Object {

  abstract void printMessage();

  protected final void callMessage(){
    printMessage();
  }
}
----------------------------------------------
package testcase1;

public abstract class LocalTest extends Base {

  protected abstract void printMessage();

}
----------------------------------------------
package testcase2;
import testcase1.*;

public class RemoteTest extends LocalTest {
 
  protected void printMessage(){
    System.out.println("Hi");
  }

  public static void main(String[] args) {
    RemoteTest me = new RemoteTest();
    me.callMessage();
  }
}
----------------------------------------------

Hotspot throws an AbstractMethodError, while Classic prints out "Hi".
Fortunately, Hotspot actually does the right thing.

Main calls Base.callMessage, which does a lookup for printMessage.
It first finds RemoteTest.printMessage, but the resolved method 
(Base.printMessage) is package private and thus not accessible from 
RemoteTest, so the lookup continues. It then finds LocalTest.printMessage
which is protected and thus visible in RemoteTest, and calls that one,
resulting in the AbstractMethodError (see new JVM spec p. 291).

There are a variety of more or less exotic examples that relate to this issue,
see the coments section.


Background:

The root of the problem is whether package private methods in one package can be
overridden by subclasses declared in another package. 

This problem has a long history. Briefly, the original JLS was flatly
self contradictory on the semantics of dynamic method dispatch.

JLS 6.6.5 partially describes the intent of the JLS authors by means of
an example. It indicates that a package private method may not be overridden
by a class in another package. It is
roughly consistent with the current (revised) spec. Guy and James have
confirmed that chapter 6 reflects their intent.

JLS 15.11.4.4 describes the precise semantics of method lookup. Applying
the algorithm of chapter 15 to the example in chapter 6 does not yield
the results claimed in chapter 6. It allows a package private method to be
overridden by a subclass in another package. This is viewed by some
as a violation of the security guarantees of the Java language. On the other
hand, it is the standard definition of method lookup, as understood by
programmers in C++, Simula, Smalltalk etc. It has been tried and tested for
over 30 years.

The classic VM's behavior has evolved over time. Essentially, a variety of
patches have been applied to make specific cases work a specific way. This
was done because it was unclear what the specification was (chapter 6 did not
give a complete specification). As a result, we don't know how to specify 
a behavior that would be precisely compatible with the Java 2 VM.

In any event, different VM's have different behavior on this issue.  
JDK 1.0 conforms to chapter 15. JDK 1.1 differs (and I'm not sure if 1.2 and 1.1
are the same), and Microsoft (if you care) behaves still differently.
I expect that VM's on small devices using interpreters will implement
chapter 15 semantics as well.

The current spec is based on JLS 8.4.6. 
It prevents the override of package private methods by classes
in another package, yet allows a relatively simple specification. It also
has the advantage that it is still possible to produce a plausible 
implementation based on lookup rather than Vtables. This is important for
small footprint implementations. It's main disadvantage is that it is non-local
and non-transitive. This makes for very non-intuitive behavior in some cases
(like the example in customer code).

Comments
CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: generic FIXED IN: kestrel-beta INTEGRATED IN: kestrel-beta
14-06-2004

PUBLIC COMMENTS Method lookup now conforms to the updated VM and language specs. See the JLS and JVMS maintenance pages at: http://java.sun.com/docs/books/jls/jls-maintenance.html and http://java.sun.com/docs/books/vmspec/2nd-edition/jvms-maintenance.html
10-06-2004

SUGGESTED FIX Here are some options: 1. Leave the spec as it is (if you believe that overriding a package private method is a security issue, tell the customer! We have always stated that we will fix security bugs regardless of compatibility considerations). 2. Revert to standard method lookup (as originally specified in JLS 15.11.4.4 and implemented in JDK 1.0, as well as C++ and all other major single inheritance O-O languages). Most Java programmers probably believe this to be the case, based on C++ experience. This may indicate that fewer real programs will break. On purely technical grounds, I (Gilad) have always advocated (2), but have been in the minority. At this point, I think it's a business decision, not a technical one. Which is more compatible with customer programs and expectations. **************************** Here's another attempt to deal with the problem. Here, I'm assuming we're stuck with requirement that a package private method will not get overridden by a class outside the package *unless it was made public or protected by another subclass within the package*. That, is, I accept, for the sake of argument, the claim that security depends on this. This precludes the use of standard method lookup. I also assume that bug-for-bug compatibility with classic is not acceptable (since we have situations where public methods are not overridden). We then have a political decision as to whether we should invent something new or stand by the existing specs. I will assume, hypothetically, that the decision is "invent something new". The following spec gives an intuitive definition of when a method overrides another, and defines a lookup algorithm (which we need as part of the JVMS for invokevirtual). A method m declared in class C overrides another method m' declared in class C' iff both: 1. C is a subclass of C'. 2. Either a. m' is accessible from C b. m overrides a method m'', m'' != m, m'' != m' that overrides m' We take the definition of subclass to be reflexive. ** The lookup algorithm for invokevirtual is then: Let C be the class of objectref. The actual method to be invoked is selected by the following lookup procedure: If C contains a declaration for an instance method m with the same name and descriptor as the resolved method, and m overrides the resolved method, <- this is the new part then this is the method to be invoked, and the lookup procedure terminates. Otherwise, if C has a superclass, this same lookup procedure is performed recursively using the direct superclass of C ; the method to be invoked is the result of the recursive invocation of this lookup procedure. Otherwise, an AbstractMethodError is raised. ** This spec matches two intuitions ascribed to Java programmers: (1) method override is transitive. (2) package private methods do not get overridden outside the package, unless a subclass within the package has declared the method public or protected. These two assertions are provable, as is: (3) Public and protected methods always get overridden by a matching method in a subclass., and more generally: (4) If m is accessible to class C, then a matching definition of m in C overrides m. (5) The most specialized overriding version of a referenced method is always invoked. Furthermore, there exists a linear time lookup algorithm as well as a vtable construction algorithm, and both can be shown to be equivalent to the spec. gilad.bracha@eng 1999-06-28
28-06-1999

EVALUATION Well, I filed it, of course it's true. gilad.bracha@eng 1999-06-23
23-06-1999