Skip to content

Skip the StringWrapper work that cannot change the file - #254

Open
abashev wants to merge 2 commits into
21.X.Xfrom
abashev/speed-up-stringwrapper-cheaper-pre-check-parse-r
Open

abashev wants to merge 2 commits into
21.X.Xfrom
abashev/speed-up-stringwrapper-cheaper-pre-check-parse-r

Conversation

@abashev

@abashev abashev commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Closes #250. Ports the part of google/google-java-format#1533 (still open) that keeps our output identical. Based on 21.X.X; no other open pull request touches these files.

What palantir#1533's ideas are worth here differs from upstream. Our wrapOnce formats the file twice: once before the reflow and once after it. Those two passes cost far more than the AST check that palantir#1533 makes lighter. On JavaInputAstVisitor.java (150 KB), one format takes about 60 ms, one parse about 4 ms, and the check, two parses with toString, about 10 ms.

What changes

  • Pre-check: needWrapping and linesNeedWrapping only count a long line when it has a quote on it. A long comment, URL or import holds no literal to reflow, and before, it cost a parse of the whole file.
  • Early return: wrapOnce returns when every replacement is the text it replaces (typically a text block that is indented already) and the first pass changed nothing. The second pass would format the same ranges of the same text, and the AST check would compare the text with itself.
  • No first pass for unchanged files: StringWrapper.wrapFormatted leaves the first pass out. Formatter.formatSourceAndFixImports and the command line call it only when formatting, and fixing imports, returned the file unchanged. Formatting it again then gives the same text, by determinism alone.

The output does not change.

Measurements

Median times in one JVM after warm-up, 21.X.X → this branch:

before after
formatSourceAndFixImports again on a file with a text block (DebugRenderer.java) 8.3 ms 2.9 ms
wrap on a formatter output with a text block and nothing to change 5.8 ms 2.7 ms
wrap on the 26 files of javaformat/java 29 ms 20 ms
formatSourceAndFixImports, first run, JavaInputAstVisitor.java with 10 long strings 223 ms 218 ms

The first run of a file with long strings to wrap gains nothing. Its first pass has to stay, see below.

Not ported

  • Leaving out the first pass for any output of the formatter, as Excavator: Upgrade dependencies palantir/palantir-java-format#1533 does. That made the last row about 1.5x faster, but it changed the output of 5 of the 6170 files below. Formatting our output again is not always a no-op. In java/lang/ProcessBuilder.java, "builder redirectInput()" + " must be PIPE…" + builder.redirectInput() is laid out one way by the first run and another way by the second, and the first pass picks up the second layout. The test suite did not catch it; the corpus did.
  • The lighter AST check: it would save about 10 ms of the 150 ms that wrap takes on the long-string file. Our replacements are not only + chains of literals. They also carry the code and the NON-NLS markers after a chain to the end of its line, re-indented text blocks, and joined literals. The full check is also what checks the second format pass. Saving those milliseconds is not worth weakening it.
  • Reusing the parse in the first pass: it saves one parse of about 4 ms next to a 60 ms format. It would need the tree passed from Source.reflowReplacements through Formatter.getFormatReplacements into format.
  • The allocation-free line scan: needWrapping goes through 131 KB without an early exit in 0.27 ms.

Tests

  • stopsAReflowThatWouldMoveACastToTheFirstPiece and stopsAReflowThatWouldMoveAnInnerClassCreationToTheLastPiece are Excavator: Upgrade dependencies palantir/palantir-java-format#1533's two tests. They pass on 21.X.X too: they pin the AST check, which today makes the formatter fail the whole file on such a literal. A long string literal under a cast, a qualified .new or a method reference fails the whole file #252 proposes to leave such literals alone instead, and would change both.
  • longLinesWithoutAQuoteNeedNoWrapping fails on 21.X.X.
  • wrapsAFileThatFormattingLeavesAsItIsAsItWrapsAnyOther: on a file that formatting leaves as it is, wrapFormatted gives what wrap gives.
  • Corpus: 6170 files, the JDK's java.base/java, jdk.compiler, java.net.http and jdk.httpserver, plus files with text blocks or long string lines from the sources jars in the Gradle cache (189 with text blocks). Each file was formatted twice, the second run on the first's output, with the 21.X.X jar and with this branch's. Both runs are byte-for-byte identical, with no errors.

With the change, JDK 21: ./gradlew check passes.

A long line without a quote on it holds no string literal to reflow,
so needWrapping and linesNeedWrapping no longer parse the file for a
long comment or import.

When every replacement is the text it replaces, as for a text block
that is indented already, and formatting their ranges changed
nothing, wrapOnce returns: the second pass would format the same
ranges of the same text, and the AST check would compare the text
with itself. The first pass is left out as well when formatting
returned the file unchanged, which formatSourceAndFixImports and the
command line know.

The formatter's output alone is not enough to leave out the first
pass, as google-java-format#1533 does: formatting it again lays out
a chain of + another way in 5 of 6170 files of the JDK and of
dependency sources, and the first pass picks that up.

Adds the two tests of google-java-format#1533 for the AST check.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Speed up StringWrapper: cheaper pre-check, parse reuse, lighter AST verification

1 participant