Support Java > 11 language features

[sormuras]

Afaict, GJF runs flawlessly on Java source code units up to Java Language Specification 11. That includes formatting module-info.java files.

• JDK 9 module-info.java handling (Standard) ➡️ JDK 9 module-info.java handling #75

With new official Java syntax being potentially introduced every six months, GJF fails on those files that already make use of new language features. Such as Switch Expressions, Text Blocks, or Records. Here's an excerpt of the JDK 14 feature list which are relevant for GJF's goal "reformatting Java source code":

• Pattern Matching for instanceof (Preview) JEP 305 ➡️ -
• Records (Preview) JEP 359 ➡️ -
• Switch Expressions (Standard) JEP 361 ➡️ Support for switch expressions (JEP 361) #340
• Text Blocks (Second Preview, Standard with Java 15) JEP 368 ➡️ Support Text Blocks (JEP 368) #382

GJF should either support those features and format them or by-pass any "unknown blocks" automagically and format the rest of the source unit as good as possible.

Follow-up from: #387 Support foundation javac
Related thread on twitter: https://twitter.com/sormuras/status/1221515240854229000

[cushon]

(Moving the discussion back here after some related comments in #443.)

We want to support new versions of the language, but (1) I think it's an open question what the best way to do that technically is, and (2) then it still a matter of prioritization and resources.

For (1), I'm not seeing great alternatives to having one branch/release per JDK version, and e.g. the JDK 13 version of g-j-f would require having JDK >= 13. (We could conceivably use mrjars to package multiple of those releases together, at least for Java > 9, but I haven't decided yet if that solves more problems than it creates.)

Historically we've been using a shaded redistribution of javac so the formatter can depend on a consistent version of the internal APIs it uses, and making a best-effort attempt for that redistribution to be compatible with a wider range of JDKs than upstream javac is.

Upstream javac is somewhat backwards compatible for bootstrapping reasons, but with the new release cadence it's still not compatible with a very wide range of JDKs, and 'compatible enough for bootstrapping' doesn't always mean it's as compatible as we'd like. It'd be neat if we could always run the latest-and-greatest javac while maintaining compatibility with the range of JDK versions we need to support, but I'm not sure that's practical anymore.

Another problem the redistribution of javac solved is that javac versions used to see more changes/ fixes within a given major release, e.g. there were changes to javac that affected Error Prone in the first few JDK 8 update releases. (I know we're talking about g-j-f and not Error Prone, but the approach to redistributing javac was informed by Error Prone). With the new release cadence it's a lot less likely that major javac changes will make it into update releases, so e.g. the JDK 13 branch of g-j-f would probably Just Work on all JDK 13 update releases.

[cushon]

The initial implementation of this landed in 4ddb914. It supports everything up to Java 14 (including the preview features), as long as the formatter is built on JDK 14.

I haven't spent a lot of time actually formatting real-world Java 14 code yet, so if you get a chance to try it please report any bugs or surprising output you see.

[sormuras]

The initial implementation of this landed in 4ddb914.

Nice!

It supports everything up to Java 14 (including the preview features), as long as the formatter is built on JDK 14.

...and JDK 14 is used to run the formatter?

Do you have to pass --enable-preview on the CLI when formatting files containing preview language features?

I haven't spent a lot of time actually formatting real-world Java 14 code yet, so if you get a chance to try it please report any bugs or surprising output you see.

Will give it a shot or two tomorrow. Really looking forward to it.

Do you have a twitter handle, Liam?

[cushon]

...and JDK 14 is used to run the formatter?

Yes. The formatter itself is targeting Java 11 language features, so you can run it on JDK 11 and it will still format Java 11 language features, but formatting Java 14 features requires a JDK 14 runtime.

Do you have to pass --enable-preview on the CLI when formatting files containing preview language features?

Nope. That might be worth considering, but it seemed convenient and relatively harmless to always support preview features:

google-java-format/core/src/main/java/com/google/googlejavaformat/java/Formatter.java

Line 121 in 4ddb914

Options.instance(context).put("--enable-preview", "true");

Will give it a shot or two tomorrow. Really looking forward to it.

Great, thanks!

Do you have a twitter handle, Liam?

[ldcushon@, although I don't use it much]

[sormuras]

[...] convenient and relatively harmless to always support preview features: [...]

Fine for me. As long as the normal formatter doesn't choke on it.

Was the 1.8-SNAPSHOT artifact uploaded? Looks like https://oss.sonatype.org/content/repositories/snapshots/com/google/googlejavaformat/google-java-format/1.8-SNAPSHOT/ is lacking some versions...

...that's why I updated my fork and building the formatter locally. Which works like a charm.

[lucasbraune]

Is this an issue? I just cloned and and built this GitHub project using Maven and OpenJDK 13. I was able to run the resulting JAR file (google-java-format-1.8-SNAPSHOT-all-deps.jar) with the java binary that came with OpenJDK 13, but not with the java binary that came with Oracle JDK 14. The input java source on which I tried to run Google Java Format is rather simple, namely the input of issue #435. It has no features that were introduced in Java 14.

Here is the stack trace I got when trying to run the JAR using Java 14:
exp/Experiment.java: error: com.google.googlejavaformat.java.java14.Java14InputAstVisitor java.lang.LinkageError: com.google.googlejavaformat.java.java14.Java14InputAstVisitor at com.google.googlejavaformat.java.Formatter.format(Formatter.java:165) at com.google.googlejavaformat.java.Formatter.getFormatReplacements(Formatter.java:291) at com.google.googlejavaformat.java.Formatter.formatSource(Formatter.java:267) at com.google.googlejavaformat.java.FormatFileCallable.call(FormatFileCallable.java:45) at com.google.googlejavaformat.java.FormatFileCallable.call(FormatFileCallable.java:26) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832)

Caused by: java.lang.ClassNotFoundException: com.google.googlejavaformat.java.java14.Java14InputAstVisitor at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:340) at com.google.googlejavaformat.java.Formatter.format(Formatter.java:160) ... 8 more

[cushon]

The formatter itself is targeting Java 11 language features, so you can run it on JDK 11 and it will still format Java 11 language features, but formatting Java 14 features requires a JDK 14 runtime.

I just cloned and and built this GitHub project using Maven and OpenJDK 13. I was able to run the resulting JAR file (google-java-format-1.8-SNAPSHOT-all-deps.jar) with the java binary that came with OpenJDK 13, but not with the java binary that came with Oracle JDK 14.

It should work if you build and run with JDK 14. Is there a reason to prefer JDK 13 at this point? I didn't have specific plans to support JDK 13 (or 12) now that JDK 14 has been released, since they're both non-LTS releases.

[sormuras]

Extrapolating to JDK 15: support for JDK 14 will be dropped when 15 is released?

Would be fine for me. As long GJF supports 11 and 17 ... i.e. all LTS releases and the current GA'ed one. That is 14 as of today, and 15 in September 2020, and so on.

[JakeWharton]

LTS is a vendor classification and not something intrinsic to the OpenJDK itself. The LTS for 11 and 17 is for Oracle OpenJDK. Azul offers MTS (medium-term support) for their JDK 13 and will offer it for their JDK 15 as well. Any company can come in and offer LTS for any JDK version so it seems unwise to solely target a single vendor's LTS versions.

It should work if you build and run with JDK 14. Is there a reason to prefer JDK 13 at this point?

People may still be migrating since it's only been a few weeks. That being said, unless something is actively preventing supporting 13 (or any other versions) it still would be nice to have support in place.

[navaneeth-spotnana]

#435 is there on Java 11 too.