Summary
-------

Extend the Java class-file format to support a new constant-pool form,
`CONSTANT_Dynamic`.  Loading a `CONSTANT_Dynamic` will delegate creation
to a bootstrap method, just as linking an `invokedynamic` call site
delegates linkage to a bootstrap method.


Goals
-----

We seek to reduce the cost and disruption of creating new forms of
materializable class-file constants, which in turn offers language
designers and compiler implementors broader options for expressivity
and performance.  We do so by creating a single new constant-pool form
that can be parameterized with user-provided behavior, in the form of
a bootstrap method with static arguments.

We will also adjust the link-time handshake between the JVM
and bootstrap methods, so as to adapt the bootstrap API used by
`invokedynamic` to apply also to dynamic constants.  

Based on experience with `invokedynamic` we will make adjustments to
the bootstrapping handshake of both `invokedynamic` and dynamic
constants, loosening certain restrictions on the processing of
argument lists to bootstrap methods.

This work requires some prototyping of JDK library support for a
representative sample of a few kinds of constant types, notably variable
handles [(JEP 193)](193).  In support of such prototyping, this work will
coordinate with other work on basic language support for constant
expressions ([JEP 303](303)).

Non-Goals
---------

This JEP aims to support arbitrary constants in the constant pool.
Although there are proposals for still other uses of bootstrap methods,
such as method recipes, this JEP concentrates on one use.

The success of this JEP does not depend on support from the Java
language or the Java compiler backend, although it is more likely to
succeed if it is used by the compiler backend.

Although large aggregate constants are a weak point in Java's
translation strategy, this JEP cannot solve for aggregates until
there are better ways to encapsulate them in constant forms, such
as frozen arrays or primitive-specialized lists.

Success Metrics
---------------

As a minimal requirement, it should be practical to expose constant-pool
forms to describe primitive class mirrors (`int.class`), `null`, `enum`
constants, and most forms of `VarHandle` in terms of `CONSTANT_Dynamic`.

Dynamic constants must be usable in any context which currently
allows general constant pool constants, such as `CONSTANT_String`
and `CONSTANT_MethodType`.  Thus, they must be valid operands to
the `ldc` instruction and must be allowed as static parameters
to bootstrap methods.

The bootstrap-method handshake should support complex constants
which contain thousands of component arguments, lifting the current
limit of 251 constant arguments.  As a stretch goal, there should
also be a way for the bootstrap method to more accurately control
linkage errors produced by resolving bootstrap method arguments.

At the end of the work we should also have cause to believe that this
mechanism can be made to work for a wide variety of library types,
such as derived method handles, small immutable collections (lists,
maps, sets), numerics, regular expressions, string formatters, or
simple data classes.

Followup work should be identified and documented. See "Possible
extensions" below.

Motivation
----------

Section 4.4 of the Java Virtual Machine Specification describes the
format of the constant pool.  Adding new constant-pool forms, such as
the support for `MethodHandle` and `MethodType` introduced in Java 7,
is a significant endeavor, and sends ripples through the ecosystem, as
it affects all code that parses or interprets class files.  This
presents a very high bar to creating new constant-pool forms.

With `invokedynamic`, the value of storing complex data in the
constant pool is multiplied, since the static argument list for an
`invokedynamic` bootstrap is a sequence of constants.  Designers of
`invokedynamic` protocols (such as the `LambdaMetafactory` added in
Java 8) routinely struggle with the need to encode behavior in terms
of the existing constant set���which in turn necessitates additional
error-prone validation and extraction logic in the bootstrap itself.
Richer, more flexible, more highly-typed constants remove friction
from the development of `invokedynamic` protocols, which in turn
facilitates the movement of complex logic from run time to linkage
time, improving program performance and simplifying compiler logic.

Description
-----------

Just as the linkage of an `invokedynamic` call site involves an upcall
from the JVM to Java-based linkage logic, we can apply this same trick
to the resolution of a constant-pool entry.  A `CONSTANT_Dynamic`
constant-pool entry encodes the bootstrap method to perform the
resolution (a `MethodHandle`), the type of the constant (a `Class`),
and any static bootstrap arguments (an arbitrary sequence of
constants, barring cycles in the constant pool between dynamic
constants.)

We add a new constant-pool form, `CONSTANT_Dynamic` (new constant tag
17), which has two components following its tag byte: the index of a
bootstrap method, in the same format as the index found in a
`CONSTANT_InvokeDynamic`, and a `CONSTANT_NameAndType`, which
encodes the expected type.

Behaviorally, a `CONSTANT_Dynamic` constant is resolved by executing
its bootstrap method on the following parameters: 1. a local `Lookup`
object, 2. the `String` representing the name component of the
constant, 3. the `Class` representing the expected constant type,
and 4. any remaining bootstrap arguments.  As with `invokedynamic`,
multiple threads can race to resolve, but a unique winner will be
chosen and any other contending answers discarded.  Instead of
returning a `CallSite` object, as the `invokedynamic` instruction
requires, the bootstrap method would return a value which would be
immediately converted to the required type.

As with `invokedynamic`, the name component is an additional channel,
besides the type, for passing expression information to the bootstrap
method.  It is expected that just as `invokedynamic` instructions find
uses for the name component (e.g., a method name or some ad-hoc
descriptor) dynamic constants will also find uses for the name (e.g.,
the name of a `enum` constant or the spelling of a symbolic constant).
Putting the `CONSTANT_NameAndType` in both places makes for a more
regular design.  In effect, `CONSTANT_Methodref` and
`CONSTANT_Fieldref` constants are used to refer to named members of
classes, while the analogous `CONSTANT_InvokeDynamic` and
`CONSTANT_Dynamic` constants are used to refer to named entities
with user-programmed bootstraps.

The type component of the constant, with both `invokedynamic` and
`CONSTANT_Dynamic`, determines the effective type of the call site or
constant (respectively).  The bootstrap method does not contribute
or constrain this type information, so that bootstrap methods may
be (and often are) weakly typed whereas the bytecodes themselves
are always strongly typed.

To relax length restrictions on bootstrap specifiers, the language
which defines the invocation of bootstrap methods will be adjusted
(with complete backward compatibility) to allow variable arity
(`ACC_VARARGS`) bootstrap methods to absorb, into their trailing
arguments, all remaining static arguments, even if there are 2^16-1 of
them.  (The class-file format already allows this, although there is no
way to read over-long bootstrap argument lists.)  For consistency, the
`invokeWithArguments` methods of `MethodHandle` will also be expanded
in this way, if the target method has variable arity.  In this way the
invocation of bootstrap methods can be specified in terms of the
weakly typed methods `invokeWithArguments` and `invoke`, just as today
it is specified in terms of `invoke` alone.

Control of bootstrap linkage errors has proven to be a recurring
source of bugs and RFEs from users of `invokedynamic` and the trend is
likely to accelerate as bootstrap methods become more complex (as they
must, with dynamic constants).  If we can find a way to offer fuller
control over exceptions to bootstrap methods, and it can be done
simply, we will consider delivering it as part of this JEP.
Otherwise, it will go on the list of future enhancements.

The draft Java Virtual Machine specification for `CONSTANT_Dynamic` can be
found in [JDK-8189199](https://bugs.openjdk.java.net/browse/JDK-8189199),
the CSR issue associated with the main development issue of this JEP.

### Future work

Possible future extensions include:

  - Support for bulk-scale constants such as arrays or resource tables
  - Further adjustments to the bootstrap-method handshake
  - Other uses of bootstrap methods which may synergize with dynamic constants
  - Attaching dynamic constants to the `ConstantValue` attribute of static fields
  - Surfacing the lazy initialization of constants in the Java language
  - Integrating new constants with special Java language rules for constant expressions

A discussion of design choices can be found in
[JDK-8161256](https://bugs.openjdk.java.net/browse/JDK-8161256), which
deals with a number of related RFEs.  The present JEP was distilled from
this larger list of features.

Alternatives
------------

Many uses of `CONSTANT_Dynamic` can be replaced by equivalent
`invokedynamic` calls. (The call would take zero arguments and be
bound to a method handle that returns the desired constant.)  Such a
workaround does not helps with a key requirement, however, which is
to be able to pass synthetic constants as bootstrap arguments.

Another alternative to `CONSTANT_Dynamic` is using `static final`
fields to name the desired constants, and computing their values in
the static initializer (`<clinit>`).  This approach requires extra
metadata (a throwaway field definition per constant) and is not lazy
enough to avoid bootstrap cycle problems.  Those problems are
routinely solved by building private nested classes with decoupled
static initializers, but that too requires extra metadata.  If
languages evolve to use many such constants, there will be application
bloat from the excessive metadata.

Another approach is spinning static methods which perform the constant
elaboration logic, and then calling them lazily from `invokedynamic`.
Again, such throwaway methods are a metadata overhead which is large
compared to `CONSTANT_Dynamic`.

In practice the metadata overheads for simulating these features are
too large.

Dependencies
------------

This feature is JVM-centric, and so does not depend on higher software
layers.

In order to ensure correct design, it requires at least experimental
adoption by several use cases.  Library prototyping is a must, even if
the prototypes are thrown away.

As with `invokedynamic`, wide adoption requires use by the `javac` back
end, which in turn may require language extensions.  As a basic first
step, translation workarounds which require hidden static methods, such
as translation of `int.class` or switch mapping tables should be examined
and reformulated with new constants, if possible.