JDK-8286021 : Implementation of JEP 430 String Templates (Preview)
  • Type: CSR
  • Component: tools
  • Sub-Component: javac
  • Priority: P3
  • Status: Proposed
  • Resolution: Unresolved
  • Fix Versions: 21
  • Submitted: 2022-05-02
  • Updated: 2022-11-24
Related Reports
CSR :  
JDK-8285932 - Implementation of JEP 430 String Templates (Preview)
Relates :  
JDK-8296302 - JEP 430: String Templates (Preview) Java Language Specification changes
Description
This is the CSR for the implementation of 
[JEP-430 String Templates (Preview)](https://bugs.openjdk.java.net/browse/JDK-8273943)

Summary
-------

Enhance the Java programming language with _string templates_, which are similar to string
literals but contain embedded expressions. A string template is interpreted at run time by
replacing each expression with the result of evaluating that expression, possibly after
further validation and transformation. This is a
[preview language feature and API](http://openjdk.java.net/jeps/12).

Problem
-------

Injecting values into strings using the Java + string operator can often lead to unreadable
code and code that is prone to errors and/or injection attacks.

Solution
--------

[JEP-430 String Templates (Preview)](https://bugs.openjdk.java.net/browse/JDK-8273943)
introduces string templates which will allow users to inject values into a String in situ.
The use of string templates versus string interpolation allows for a richer feature that,
in addition to composition, allows validation and non-string result transformations.

Specification
-------------

The following APIs were modified

 - [java.base/java/lang/invoke/StringConcatFactory.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/java.base/java/lang/invoke/StringConcatFactory.html)
 - [jdk.compiler/com/sun/source/tree/Tree.Kind.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/jdk.compiler/com/sun/source/tree/Tree.Kind.html)
 - [jdk.compiler/com/sun/source/tree/TreeVisitor.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/jdk.compiler/com/sun/source/tree/TreeVisitor.html)
 - [jdk.compiler/com/sun/source/util/SimpleTreeVisitor.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/jdk.compiler/com/sun/source/util/SimpleTreeVisitor.html)
 - [jdk.compiler/com/sun/source/util/TreeScanner.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/jdk.compiler/com/sun/source/util/TreeScanner.html)

The following APIs were added

 - [java.base/java/lang/runtime/TemplateRuntime.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/java.base/java/lang/runtime/TemplateRuntime.html)
 - [java.base/java/lang/template/ProcessorLinkage.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/java.base/java/lang/template/ProcessorLinkage.html)
 - [java.base/java/lang/template/StringProcessor.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/java.base/java/lang/template/StringProcessor.html)
 - [java.base/java/lang/template/StringTemplate.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/java.base/java/lang/template/StringTemplate.html)
 - [java.base/java/lang/template/TemplateProcessor.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/java.base/java/lang/template/TemplateProcessor.html)
 - [java.base/java/lang/template/ValidatingProcessor.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/java.base/java/lang/template/ValidatingProcessor.html)
 - [java.base/java/util/FormatProcessor.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/java.base/java/util/FormatProcessor.html)
 - [jdk.compiler/com/sun/source/tree/StringTemplateTree.html](https://cr.openjdk.java.net/~jlaskey/templates/docs/api/jdk.compiler/com/sun/source/tree/StringTemplateTree.html)
Comments
Attachment <api-4.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
24-11-2022
Attachment <api-2.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
23-11-2022
Attachment <api-1.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
21-11-2022
Attachment <api-6.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
18-11-2022
Attachment <api-6.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
18-11-2022
Attachment <api-5.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
17-11-2022
Attachment <api-3.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
16-11-2022
Attachment <api-2.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
14-11-2022
Added declarations for `process(StringTemplate stringTemplate)` in `TemplateProcessor` and `StringProcessor` to provide specialized javadoc.
14-11-2022
Attachment <api-2.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
14-11-2022
Joe: - the process method appears at the bottom with no text. If method is overridden in this class to narrow the kinds of exceptions thrown. I think this should be explained in the TemplateProcessor.process method and not just in the TemplateProcessor class-level docs. **Got it. Will change.** - StringTemplate is an interface. The STR and RAW fields in StringTemplate are of type TemplateProcessor (or a subtype). Processors are not templates, hence my question on the placement of processor "constants" in a templates interface. **`STR` and `RAW` are where they should be, for reasons similar to why `PI` is in `java.lang.Math` and not `java.lang.Double`. Their type is separate from their purpose and their purpose is to be there for StringTemplates.**
11-11-2022
On the page https://cr.openjdk.java.net/~jlaskey/templates/docs/api/java.base/java/lang/template/TemplateProcessor.html the process method appears at the bottom with no text. If method is overridden in this class to narrow the kinds of exceptions thrown. I think this should be explained in the TemplateProcessor.process method and not just in the TemplateProcessor class-level docs. StringTemplate is an interface. The STR and RAW fields in StringTemplate are of type TemplateProcessor (or a subtype). Processors are *not* templates, hence my question on the placement of processor "constants" in a templates interface.
11-11-2022
Joe: - Should StringTemplateTree say something about "the template needs to start with a string and then strings and expressions are interleaved"? **`StringTemplateTree` is consistent with other elements in ` com.sun.source.tree`.** - In FormatProcessor.linkage, "Construct a MethodHandle that constructs a result based on the bootstrap method information." There is no obvious BSM I see being referred to. **The fuzzyiness of this class's doc is deliberate. This methodology used will change after preview to use TemplateProcessorFactorys. At this point users cannot use this interface (sealed).** - Also, "Construct a Method..." -> "Construct*s* a Method..." as the convention is a short-hand for "[This method being documented ] constructs a Method..." **Will clean up.** - TemplateProcessor.process: the method needs some javadoc **Not sure what you mean. The method is inherited from `ValidatingProcessor`.** - Why are the platform's template processors defined in StringTemplate rather than TemplateProcessor or somewhere else? **I'm assuming you mean `STR` and `RAW`. Most users will go through life never seeing anything other than StringTemplate.** - Should StringTemplate be sealed? **Absolutely not. Users are free to create their own StringTemplates classes. Some examples;** **- A parser could be written to recognize '${...}' in strings to allow for a different kind of template.** **- One of the other JVM languages can adapt their template objects to be usable against Java template processors.** - Why is there a StringTemplate.equals(StringTemplate, StringTemplate) method as opposed to using java.util.Object.equals? **I have since dropped these methods (equals, hashCode).**
10-11-2022
Attachment <api.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
10-11-2022
Attachment <sumdocs.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
10-11-2022
Moving back to Draft. [~jlaskey], please update to address the CSR comments and any changes made in response to code review and re-Propose.
09-11-2022
Should StringTemplateTree say something about "the template needs to start with a string and then strings and expressions are interleaved"? In FormatProcessor.linkage, "Construct a MethodHandle that constructs a result based on the bootstrap method information." There is no obvious BSM I see being referred to. Also, "Construct a Method..." -> "Construct*s* a Method..." as the convention is a short-hand for "[This method being documented ] constructs a Method..." TemplateProcessor.process: the method needs some javadoc Why are the platform's template processors defined in StringTemplate rather than TemplateProcessor or somewhere else? Should StringTemplate be sealed? Why is there a StringTemplate.equals(StringTemplate, StringTemplate) method as opposed to using java.util.Object.equals?
09-11-2022
Just waiting for a few reviews (stabilization). api changes zip added. Will update as we progress.
07-11-2022
Attachment <api.zip> could not be scanned. The system has removed attachment. Please check the file before attempting to upload it again
07-11-2022
[~jlaskey], note that for the archive function of the CSR the proposed changes need to be associated directly with the CSR in some fashion, attaching the patch, a zip of the specdiff, etc. Live links are a handy convenience, but an archival copy is needed too.
04-11-2022
The `jdk.compiler` API changes look good.
01-11-2022
[~jlaskey], the draft JLS changes should also be included in this (or another) CSR for string templates.
01-11-2022
Joe: https://bugs.openjdk.org/browse/JDK-8285944 under review from the architects.
01-11-2022
Brian: "I am unsure of the placement of formatterMethodHandle." It was only being used privately anyway. Removed. Brian: "Consider renaming SimpleProcessor -> SimpleTemplateProcessor" Renamed: - `ValidatingProcessor<R, E>` - `TemplateProcessor<R>` - `StringProcessor` Brian: "Is `StringTemplate::interpolate()` necessary?" Yes. This is the specialized implementation for each StringTemplate. In fact `STR` is implemented as; ``` public static final StringProcessor STR = st -> st.interpolate(); ``` Brian: "Would like to see some spec on what the Object methods toString, equals, and hashCode" Added implementations with implNotes. <br><br> Maurizio: "I think some of the names are too general" See above. Maurizio: "What happens if FOO is broken and uses unchecked casts internally, so that it returns something other than its declared return value? Where does it fail?" Casts are normally picked up at compile time. Check ensures the result of the expression matches. If the processor is recompiled with a different type then it will fail with `CastCheckException` at runtime. Maurizio: "The javadoc of SimpleProcessor contains reference to TemplateRuntime.interpolate which isn't there." Missed that. Fixed. Was suppose to be `StringTemplate.interpolate`. <br><br> Raffaello: "StringConcatFactory.makeConcatWithTemplate() @return has an incorrect statement about the return value." Fixed. Raffaello: "... by returning a customized CallSites from linkage" Fixed. Raffaello: "the mention about the bootstrap method is unclear." Not if this API will be public, so info is deliberately vague. At issue is whether we can't supply the same functionality through another API, maybe a builder. It exists now only to support `FormatProcessor`. Raffaello: "In the JEP itself, Motivation section 3rd bullet" In the context of the JEP this is deliberate, i.e., specifiers based on position are confusing.
21-10-2022
`StringConcatFactory.makeConcatWithTemplate()` `@return` has an incorrect statement about the return value. I second Brian about the home of `Formatter.formatterMethodHandle()`. `ProcessorLinkage`'s documentation is confusing. E.g., "... by returning a customized CallSites from linkage". But the linkage method returns a MH, not a CS. Also, the doc of `linkage()` reads "Construct a MethodHandle that constructs a result based on the bootstrap method information": the mention about the bootstrap method is unclear. In the JEP itself, Motivation section 3rd bullet, the usages of `String.format` and `String.formatted` are at odd with the other mechanisms because of the reverted use of argument indices: 2$ and 1$ rather than 1$ and 2$.
20-10-2022
I think some of the names are too general (e.g. TemplateProcessor) and would benefit from having String prefix (e.g. StringTemplateProcessor). Is the only difference between TemplateProcessor and SimpleProcessor the Throwable? Overall I find SimpleProcessor, StringProcessor visually jarring (they look almost the same). And to me they are all "string processors" but the String in StringProcessor denotes the return. I'll need to look at how the implementation works - but I suspect the compiler trusts the generic types in the TemplateProcessor - e.g. FOO."bar" will have a static type that is the same as the output type parameter in FOO, right? What happens if FOO is broken and uses unchecked casts internally, so that it returns something other than its declared return value? Where does it fail? The javadoc of SimpleProcessor contains reference to TemplateRuntime.interpolate which isn't there.
19-10-2022
Comments on API points (haven't reviewed the specs yet): I am unsure of the placement of `formatterMethodHandle`. In general, classes that "do user stuff" and classes that deal in method handles are disjoint; method handles are too sharp-edged for ordinary users. Since its a static method, I would consider finding another home for it, perhaps somewhere in StringConcat* or TemplateRuntime. Consider renaming SimpleProcessor -> SimpleTemplateProcessor (and friends). These names will generally appear only in `extends` clauses (where the extra verbosity is not painful) and makes the name more evocative and less likely to clash. Arguably `StringTemplate` belongs in `java.lang` because it describes a type that the language knows about. But I realize this will increase the liklihood of collision. Is `StringTemplate::interpolate` necessary? Isn't this the same as `STR.process(st)`? I think I prefer the latter as it highlights the use of interpolation more consistently (both forms can be found in one go with a "find usages" of STR.) It seems that this can be profitably removed. Would like to see some spec on what the Object methods toString, equals, and hashCode mean on `StringTemplate`, especially toString.
19-10-2022