JDK-8287152 : Overhaul doc-comment inheritance
  • Type: CSR
  • Component: tools
  • Sub-Component: javadoc(tool)
  • Priority: P3
  • Status: Closed
  • Resolution: Approved
  • Fix Versions: 21
  • Submitted: 2022-05-23
  • Updated: 2022-12-20
  • Resolved: 2022-11-18
Related Reports
CSR :  
Relates :  
Description
Summary
-------

Improve method documentation inheritance.

Problem
-------

1. The documentation comment specification for the standard doclet poorly
   specifies documentation inheritance.

   The structure of inheritable elements and relationships between them
   are left to reader's intuitions.

2. The `javadoc` tool implements _Method Comments Algorithm_ differently
   from how the documentation comment specification for the standard
   doclet specifies it.

   Although neither the expected nor the actual behavior is ideal, each
   has a good trait that the other does not.

3. Documentation inheritance lacks control.

   Regardless of a particular automatic algorithm, there will always be
   a case where it does a wrong thing.

Solution
--------

1. To improve the specification, clarify the scope and rules of inheritance
   for individual elements.

   Clarify that:

     * `/** {@inheritDoc} */` explicitly inherits **only** the main description,
     * `@throws X {@inheritDoc}` inherits **all** the tags describing the `X`
        exception type, not just some of them,
     * information for a method parameter, formal or type, is inherited by
       the parameter position, not by parameter name, to address possible
       parameter renaming among overrides

2. To reconcile the expected behavior with the actual behavior of _Method
   Comments Algorithm_, re-specify it and re-implement it in the `javadoc`
   tool.

   The proposal is to prefer a superclass to a superinterface, unless the
   superclass is `java.lang.Object`, in which case prefer a superinterface.

   While that proposal generally preserves the current behavior -- to prefer
   a superclass to a superinterface -- it also recognizes that any documentation
   -- even of a superinterface -- is more specific than that of
   `java.lang.Object`.

   To illustrate that change, we'll use diagrams. A diagram depicts an
   inheritance hierarchy. An edge points from a subtype to a supertype.
   Square brackets `[ ]` depict a class, parentheses `( )` depict an
   interface. Asterisk `*` marks the `java.lang.Object` class.

   The search order can be traced by following the sequence of integers,
   starting from `1`. A class or an interface marked with `1` is searched
   first, with `2` is searched second, and so on.

   Let's consider two cases of a method that inherits documentation.

   1. The method is declared in a class.

      The current expected behavior is as follows:

                        (5)
                         ^
               *        /
              [7] (3) (4)
               ^   ^   ^
                \  |  /
                 \ | /
                  [6] (2)
                   ^   ^
                   |  /
                   | /
                  [1]

      The current actual behavior is as follows:

                        (6)
                         ^
               *        /
              [3] (4) (5)
               ^   ^   ^
                \  |  /
                 \ | /
                  [2] (7)
                   ^   ^
                   |  /
                   | /
                  [1]

       This proposal is to change the behavior as follows:

                        (5)
                         ^
               *        /
              [7] (3) (4)
               ^   ^   ^
                \  |  /
                 \ | /
                  [2] (6)
                   ^   ^
                   |  /
                   | /
                  [1]

   2. The method is declared in an interface.

      The current expected behavior is as follows:

              (3) (4)
               ^   ^
                \ /
                (2) (5)
                 ^   ^
                  \ /
                  (1)

      Currently, the specification does not mention methods whose signature is
      override-equivalent to the signature of a public overridable method in
      `java.lang.Object` -- of which there are three: `equals`, `hashCode`,
      and `toString`, so `java.lang.Object` is never visited and, hence,
      is not on the diagram.

      However, the current actual behavior is to treat interfaces as if they
      had `java.lang.Object` as their superclass. Combined with preferring
      a superclass to a superinterface, the search short-circuits on the
      first interface's "superclass", `java.lang.Object`, which means that
      for the three relevant methods superinterfaces are not considered:

              (4) (5)
               ^   ^
                \ /
                (3) (6)
                 ^   ^
                  \ /
                  (1)
                   |
                   v
                  [2]
                   *

      This proposal is to change the behavior as follows:

              (3) (4)
               ^   ^
                \ /
                (2) (5)
                 ^   ^
                  \ /
                  (1)
                   |
                   v
                  [6]
                   *

      This change to the algorithm is **backwards incompatible**, but its scope
      is limited to the three public overridable methods of `java.lang.Object` and any
      unwanted differences in the generated documentation can be corrected by
      using _directed inheritance_, which is also introduced in this CSR below.

3. To control documentation inheritance as required, allow _directed
   inheritance_.

   Directed inheritance allows `@inheritDoc` to provide a supertype:

              {@inheritDoc <class-or-interface>}

   with the semantics that the inheritor gets documentation of the specified
   supertype, which in turn might inherit its documentation.

   While it has always been possible to copy-paste the desired documentation
   explicitly rather than inherit it, unlike this proposal, it's suboptimal
   (Don't Repeat Yourself) and lossy: potentially useful information on
   inheritance source is missing.

Specification
-------------

See the attached `specification-v3.patch` and `api-v3.patch` files. The latter
is no different from `api-v2.patch` and added for clarity, to reflect the
overall proposal revision change.
Comments
Re-approved for JDK 21.
20-12-2022

Moving to Approved.
18-11-2022

I have updated this CSR with the changes spelled out. The change is slightly different from that hinted in this comment because we decided to rip off the band-aid and fix the behavior the right way: https://bugs.openjdk.org/browse/JDK-8287152?focusedCommentId=14536727&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14536727
18-11-2022

[~darcy] I'll try to work this into the body of this CSR: If this CSR is approved, the specification and the reference implementation (the javadoc tool) will become consistent with each other. The main change that this CSR proposes is that to "Method Comments Algorithm". To illustrate that change, I'll use diagrams. A diagram depicts an inheritance hierarchy. An edge points from a subtype to a supertype. Square brackets [ ] depict a class, parentheses ( ) depict an interface. Asterisk * marks the java.lang.Object class. The search order can be traced by following the sequence of integers, starting from 1. A class or an interface marked with "1" is searched first, with "2" is searched second, etc. Method declared in a class ========================== Currently this case is specified as follows: (5) ^ * / [7] (3) (4) ^ ^ ^ \ | / \ | / [6] (2) ^ ^ | / | / [1] But implemented as follows: (6) ^ * / [3] (4) (5) ^ ^ ^ \ | / \ | / [2] (7) ^ ^ | / | / [1] If this CSR is approved, not only will the latter behavior remain the actual implementation, but it will also become the official specification. Method declared in an interface =============================== Currently this case is specified as follows: (3) (4) ^ ^ \ / (2) (5) ^ ^ \ / (1) While this looks the same as this CSR proposes it should look, there's a *special case* that differs. The specification does not mention that case currently, and the javadoc tool implements it differently from how this CSR proposes it should be specified and implemented. It's a case of a method that is override-equivalent to a public method of java.lang.Object (JLS 8.4.2). While such an interface method cannot be default, it can be annotated with @java.lang.Override and is usually declared specifically to provide documentation. So the question is this: in an interface hierarchy, should java.lang.Object be searched and, if so, then when? For example, suppose an interface extends java.util.Collection, which declares java.lang.Object::equals. If that interface then declares `boolean equals(Object)`, but omits its documentation, whose documentation should that method inherit? That of java.lang.Object or that of java.util.Collection? Currently this case is not specified at all. However, the tool implements it *as if* an interface had a superclass, java.lang.Object, which as a superclass is searched before superinterfaces, if any: (4) (5) ^ ^ \ / (3) (6) ^ ^ \ / (1) | v [2] * (Inheritance edges from other interfaces to java.lang.Object, which is searched second, are omitted for clarity.) This behavior is surprising and does not allow for documentation inheritance in interface hierarchies for a method that is override-equivalent to some method in java.lang.Object. This CSR proposes to change that as shown: (3) (4) ^ ^ \ / (2) (5) ^ ^ \ / (1) | v [6] * That is, the interface hierarchy is exhaustively searched first and, if the method is still not found, it's only then that java.lang.Object is searched. This way the premature search in java.lang.Object is avoided. This allows to inherit documentation in method interface hierarchies.
12-11-2022

Moving back to Provisional. [~prappo], please clarify a few items before re-Finalizing: - In the summary, give an overview of the change, in other words "change the spec to match the behavior" - Explicitly state what changes, if any, are needed to the core libs for javadoc to generate the output after this change as it does now.
09-11-2022

That's a new command for me ;-) Where did you get the `-3`
07-11-2022

[~jjg] try to use meta information from Git where available: ``` git apply -3 api.patch ``` That said, I will update the patch so it would apply cleanly.
07-11-2022

As of 3 Nov, `api.patch` does not apply cleanly. ``` $ ( cd open ; patch -p1 ) < play/api.patch patching file src/jdk.compiler/share/classes/com/sun/source/doctree/InheritDocTree.java patching file src/jdk.compiler/share/classes/com/sun/source/util/DocTreeFactory.java Reversed (or previously applied) patch detected! Assume -R? [n] n Apply anyway? [n] n Skipping patch. 2 out of 2 hunks ignored -- saving rejects to file src/jdk.compiler/share/classes/com/sun/source/util/DocTreeFactory.java.rej patching file src/jdk.compiler/share/classes/com/sun/source/util/DocTreeScanner.java Hunk #1 succeeded at 324 (offset 6 lines). Hunk #2 succeeded at 332 (offset 6 lines). ```
03-11-2022

[~darcy] You expectation is correct: that property holds. I'm curious why you asked that. The algorithm described in the attached specification should provide a good mental model of the behavior.
26-05-2022

Moving to Provisional, not approved. For directed inheritDoc, consider a class hierarchy A -> B -> C. If method C.foo inhertiDoc's from B and B does not have its own definition of foo, but A has a definition, I would expect C to get the javadoc of A.foo. In other words "inheritDoc B" used in class C would be equivalent to the meaning of plain "inhertiDoc" in B on that construct. Is that correct? If the above property does not hold, I think it would be brittle to refactor the javadoc up and down type hierarchies.
26-05-2022

Documentation comment inheritance has multiple issues: * The specification is outdated, incomplete, and vague * The reference implementation (the `javadoc` tool) exhibits unspecified behavior * `@see` and single-argument custom tags are inherited * Information provided by `@throws T` is inherited by an overrider that throws `S`, where `Throwable > S :> T` * Only the first of multiple `@throws` for the same exception is inherited * The `javadoc` tool implements the [Method Comments Algorithm][] in exactly the opposite way to how it is specified * It lacks useful a feature such as directed inheritance [Method Comments Algorithm]: https://web.archive.org/web/20220524095558/https://docs.oracle.com/en/java/javase/18/docs/specs/javadoc/doc-comment-spec.html#method-comments-algorithm These issues need to be gradually resolved, and here are the main reasons for that: * Provide documentation authors with a better mental model and tools * Provide implementors with better specification Implementors, such as IntelliJ IDEA 2022.1.1, NetBeans 13, and jshell 18, are known to adhere to the specification when inheriting documentation. This causes discrepancies between documentation produced by those implementors and that of `javadoc`. Those discrepancies confuse authors. While other Documentation comment inheritance are expected to be resolved later, this CSR is concerned with the Methods Comments Algorithm (JDK-6376959) and Directed Inheritance (JDK-6934301). These two are so intertwined, that it makes sense to address them together rather than individually in a separate CSRs.
24-05-2022

[~jjg] I saw you added "File or wire format" to "Interface Kind". Curious: why did you do that?
24-05-2022

In the version of Markdown used here, you should indent the content of a code block, not used the fenced (back-up) syntax.
24-05-2022

Removed inline patches and attached them as files due to rendering issues.
24-05-2022