JDK-8220167 : javac should reject class files with bad EnclosingMethod attributes
  • Type: CSR
  • Component: tools
  • Sub-Component: javac
  • Priority: P4
  • Status: Closed
  • Resolution: Approved
  • Fix Versions: 13
  • Submitted: 2019-03-05
  • Updated: 2019-03-08
  • Resolved: 2019-03-08
Related Reports
CSR :  
Description
Summary
-------

javac will reject `class` files with bad `EnclosingMethod` attributes.

Problem
-------

A `class` file may have an `EnclosingMethod` attribute that is syntactically well-formed (according to the JVM's `class` file format checking) but semantically ill-formed (according to the rules of the Java language). This may cause a Java compiler to crash. A notable instance of this problem is when `EnclosingMethod` specifies an enclosing class that could not have enclosed the current class in Java source code. That is, in a `class` file for a local/anonymous class A, `EnclosingMethod` specifies that the enclosing class is C, but the binary name of C is not a prefix of the binary name of A. 

Solution
--------

The `EnclosingMethod` attribute is defined by [JVMS 4.7.7](https://docs.oracle.com/javase/specs/jvms/se11/html/jvms-4.html#jvms-4.7.7) to record the class (and method) that enclosed the source declaration of the current class, which was local/anonymous. The [rules in JLS 13.1](https://docs.oracle.com/javase/specs/jls/se11/html/jls-13.html#jls-13.1-110-A) indicate that a local/anonymous class has a binary name which is an _extension_ of the binary name of its enclosing class. Accordingly, if the `EnclosingMethod` attribute is present in the current `class` file, then its `class_index` item should specify a binary name for the enclosing class which is a _prefix_ of the binary name of the current class. If `class_index` does not follow this pattern, then the attribute is not semantically well-formed, and Java compilers should reject the `class` file.

Specification
-------------

javac will check if the binary name specified by `class_index` in `EnclosingMethod` is a prefix of the binary name of the `class` file that contains `EnclosingMethod`. If not, then javac will refuse to load the `class` file.

Comments
Moving to Approved. I recommend adding a release note for this issue.
08-03-2019