diff --git a/docs/decisions/ADR-001-one-output-per-entry-point.md b/docs/decisions/ADR-001-one-output-per-entry-point.md new file mode 100644 index 000000000..64d97257b --- /dev/null +++ b/docs/decisions/ADR-001-one-output-per-entry-point.md @@ -0,0 +1,77 @@ +# ADR-001: Two entry points, and one output for each on every surface + +## Status +Proposed + +## Date +2026-11-14 + +## Context + +open-java-format runs on many surfaces: the command line and its native binary, the GitHub Action and the pre-commit hook (both run the native binary on whole files), the playground, the Maven plugin, our Gradle Spotless step, Spotless' own `palantirJavaFormat()` step, `formatDiff` (on the JVM or with the native binary), IntelliJ IDEA (Java-based or native) and Eclipse. + +Each of them reaches the formatter through code of its own, and they had drifted apart (#219): + +- `FormatFileCallable.formatFile`, behind the command line, lays out the code, then removes unused imports, sorts them and reflows long strings. `Formatter.formatSourceAndFixImports`, behind the Maven plugin and the Spotless step, fixes the imports first. Where a removed import has a comment, the two write different files: 1 of 3,865 real files. +- Spotless' own step calls `ImportOrderer.reorderImports`, `RemoveUnusedImports.removeUnusedImports` and `Formatter.formatSource`, and never reflows long strings: 94 of the 3,865 files differ. +- `formatDiff` on the JVM goes through `Formatter.getFormatReplacements` and never reflows; with the native binary, and in IntelliJ IDEA, `--output-replacements` reflows, but every long string in the file once one line in the ranges needs it. +- `--lines` and `--offset` printing text format the ranges, then fix the imports and reflow the whole file. +- Tests pinned the pieces, not the surfaces: the 915 cases of `FormatterCasesTest` went through `formatSourceAndFixImports`, and give the same output in both import orders, so none of this showed. + +A file that one surface formats then fails the check of another, and nothing says which output is right. + +## Decision + +There are two entry points, and every surface goes through one of them: + +| Entry | Does | Never does | +|---|---|---| +| **Whole file** | lays out the code, removes unused imports and sorts them, reflows long strings | — | +| **Part of a file** (character or line ranges) | lays out the code the ranges cover, reflows the long strings in that code | touch the imports, or code outside what formatting the ranges rewrites | + +For the same formatter version and input, every surface of an entry gives the same result: the same bytes, or the same diagnostics, for a whole file; the same replacements for a part of one. The command line is the reference for the whole file, since the GitHub Action and the pre-commit hook check against it. + +- IntelliJ IDEA's Reformat is the part-of-a-file entry, also over a whole file, and leaves the imports alone. Optimize imports stays the IDE's own until the plugin's optimizer replaces it ([ADR-003](ADR-003-imports-in-intellij-idea.md), #222). +- `--lines`, `--offset` and `--character-ranges` printing text are the part-of-a-file entry, and do not fix imports. +- Eclipse is a surface of the part-of-a-file entry. + +Accepted exceptions, each documented: + +- **The order of fixing imports** stays as it is until #224 is decided: after the layout on the command line, before it in `formatSourceAndFixImports` and in Spotless' own step ([ADR-004](ADR-004-order-of-fixing-imports.md)). +- **Spotless' own step does not reflow long strings.** `Formatter.formatSource` keeps its meaning; the docs site's Spotless page says so. +- **Syntax of the runtime.** The formatter parses with the javac of the JVM it runs on; the native binary has GraalVM 25's. Code that needs a newer Java than the runtime fails there. The contract holds for code the runtime can parse, and tests that need a newer JDK run in classes of their own, with case files in a folder per JDK ([ADR-002](ADR-002-tests-by-jdk.md)). +- **A file that does not parse** keeps today's behaviour: in-process services throw, out-of-process ones return no replacements. + +## Alternatives Considered + +### Leave each surface as it is +- Pros: no change for anyone. +- Cons: a file formatted on one surface fails another's check; no output is the right one. +- Rejected: the point of a formatter shared by the build, the IDE and CI is that they agree. + +### One entry: IntelliJ IDEA formats a whole file with imports, in one pass +- Pros: Reformat of a whole file would give the command line's bytes with no IDE action in between. +- Cons: Reformat would change imports even when Optimize imports is off. +- Rejected: imports in IntelliJ IDEA go through the IDE's Optimize imports, replaced later by ours ([ADR-003](ADR-003-imports-in-intellij-idea.md)). + +### Make `Formatter.formatSource` reflow long strings, for Spotless' own step +- Pros: the only way from our side to give Spotless' step the command line's long strings. +- Cons: it changes what a public method does, for every caller. +- Rejected: Spotless' step stays an accepted exception. + +### Let the part-of-a-file entry fix imports, as `--lines` does today +- Pros: no change for users of `--lines`. +- Cons: a range would change code outside it. +- Rejected: a range touches only the code it covers. + +### Decide the import order now +- Pros: removes the last exception for whole files. +- Cons: either order changes the output of some surface; google-java-format has just moved to fixing imports before the layout. +- Deferred to #224 ([ADR-004](ADR-004-order-of-fixing-imports.md)). + +## Consequences + +- Each change that brings a surface to its entry is a pull request of its own, with a check through that surface which fails before the change: the in-process service returns what `--output-replacements` prints; the part-of-a-file entry reflows only in the ranges; `--lines` and `--offset` print what the replacements give; Eclipse reflows like the others; the out-of-process `fixImports` passes `--fix-imports-only` (#221). +- Tests that pin what users get go through a surface, `Main` or a `FormatterService`, not through `ImportOrderer`, `RemoveUnusedImports` or `StringWrapper` on their own (#225). +- Users of `--lines`, `--offset` and `--character-ranges` as text see imports left alone and long strings reflowed only in the ranges. +- The exceptions stay until their own decisions: #224 for the import order; Spotless' step for as long as `formatSource` keeps its meaning. diff --git a/docs/decisions/ADR-002-tests-by-jdk.md b/docs/decisions/ADR-002-tests-by-jdk.md new file mode 100644 index 000000000..cd17bced4 --- /dev/null +++ b/docs/decisions/ADR-002-tests-by-jdk.md @@ -0,0 +1,61 @@ +# ADR-002: Tests that need a newer JDK live in classes and folders of their own + +## Status +Proposed + +## Date +2026-11-14 + +## Context + +The formatter parses with the javac of the JVM it runs on, so some of its tests can only pass on a newer JDK than the one the build compiles for (Java 21). CI runs the tests on JDK 21 in its build job and on 25, 26, 27 and 28-ea in its `jdk` jobs. + +Today such tests skip themselves from inside, in three ways: + +- `Assumptions.assumeTrue(Formatter.getRuntimeVersion() >= 23)`: the 7 tests of `ModuleImportTest`, in a `@BeforeAll`, and `moduleImportStaysAmongOrdinaryImports` in `RemoveUnusedImportsTest`. +- `FileBasedTests.VERSIONED_TESTS`, a map from a minimum version to test names, read by `assumeJavaVersionForTest`: the `ModuleImport` pair of `FormatterIntegrationTest`, 4 runs. +- `Assumptions.assumeTrue` on `Modifier.VALUE` existing: `valueClassAndRecord` in `FormatterTest` and `valueClass` in `ModifierOrdererTest`, for the value classes of JEP 401, a preview of JDK 28, whose early-access builds have the `value` modifier. + +On JDK 21, where a local build runs, these 14 tests count as skipped among 1,422, and nothing says a change broke them. That is how the first push of #220 passed locally and failed CI's JDK 25 to 28 builds on `ModuleImportTest`. The case files of `FormatterCasesTest`, in `test-cases/`, have no way to say which JDK they need, so a case of newer syntax cannot be written at all. + +## Decision + +A test that needs a newer JDK says so on its class, and its case files live in a folder of that JDK: + +- **Case files by JDK, side by side.** `test-cases/` becomes `test-cases-java21/`, for any JDK from 21; cases that need JDK 25 go to `test-cases-java25/` beside it, those that need JDK 28 to `test-cases-java28/`, and so on, one level deep. +- **Classes by JDK.** A test class that needs a newer Java carries `@EnabledForJreRange` with that version, as `min = JRE.JAVA_25` (JUnit 6.1.3 has constants up to `JRE.JAVA_28`). On an older JDK it reports as disabled; on that JDK and later it runs with the JDK 21 tests. +- **One class per JDK for the case folders, JDK 21 too.** Each names the folders its JDK parses, one, two or three of them, newest first, as `FormatterCasesJava25Test` runs `test-cases-java25` and `test-cases-java21`; each is enabled from its JDK on, `min = JRE.JAVA_21`, `JAVA_25` or `JAVA_28`, with no upper bound, so on a newer JDK the older folders run again in the newer classes. The code they share takes the folders as arguments. +- **Module imports from JDK 25.** They parse from JDK 23 as a preview, and are final in 25; CI runs no 23 or 24. +- **Preview features by the version that previews them.** Value classes are a preview of JDK 28: their formatting case goes to `test-cases-java28/`, and the `ModifierOrderer` test to a class with `@EnabledForJreRange(min = JRE.JAVA_28)`. CI's JDK 28 job runs early-access builds, which have the `value` modifier. +- **No assumption on the runtime inside a test.** `assumeTrue` on the runtime and `VERSIONED_TESTS` go. + +## Alternatives Considered + +### Keep the assumptions inside the tests +- Pros: no change. +- Cons: three mechanisms; a skipped test inside a class that runs looks like a pass; case files cannot say what they need. +- Rejected: a test that cannot run here should not run here, and say so. + +### Folders inside `test-cases/`, as `test-cases/java25/` +- Pros: one root for the cases. +- Cons: a second level of nesting under folders that already have subfolders (`imports`, `jbang`, `stringwrapper`, `testdata`); the JDK 21 cases would have to be told apart from the JDK subfolders. +- Rejected: one folder per JDK, side by side. + +### Module imports from JDK 23 +- Pros: the tests start where the syntax first parses. +- Cons: a preview there, and no CI job runs 23 or 24. +- Rejected: JDK 25, where they are final and CI runs. + +### A condition on the `value` modifier for value classes +- Pros: the tests would run on any JDK build that has the modifier, whatever its version. +- Cons: a second kind of condition, and a case file cannot carry it; the early-access builds of JDK 28, where the feature is previewed, have the modifier. +- Rejected: preview features go by the version, as the rest. + +## Consequences + +- `FormatterCasesTest` keeps the code that runs the cases; `FormatterCasesJava21Test`, `FormatterCasesJava25Test` and `FormatterCasesJava28Test` run them. A new JDK folder means a new class that names it and the older ones. The comments that name `test-cases/`, in `Level.java` and `RemoveUnusedImportsTest`, change with it. +- The `ModuleImport` pair of `FormatterIntegrationTest` becomes a case in `test-cases-java25/`; `ModuleImportTest` and the module-import test of `RemoveUnusedImportsTest` carry `@EnabledForJreRange(min = JRE.JAVA_25)`. +- The value-class test of `FormatterTest` becomes a case in `test-cases-java28/`; the one of `ModifierOrdererTest` moves to a class with `@EnabledForJreRange(min = JRE.JAVA_28)`. +- On JDK 23 and 24 the module-import tests no longer run. +- Before a push, the tests run on JDK 21 and 25 locally: `./gradlew test -PjavaRuntime=25 -PtestkitGradleVersion=9.3.1`; a change to the JDK 28 tests also on an early-access build of 28, with `-PjavaRuntime=28 -Porg.gradle.java.installations.paths=`. +- Comparing the native binary, which parses JDK 25 code whatever the test JVM, with an in-process surface on `test-cases-java25/` needs a test JVM of 25 or later. diff --git a/docs/decisions/ADR-003-imports-in-intellij-idea.md b/docs/decisions/ADR-003-imports-in-intellij-idea.md new file mode 100644 index 000000000..dcd64312e --- /dev/null +++ b/docs/decisions/ADR-003-imports-in-intellij-idea.md @@ -0,0 +1,51 @@ +# ADR-003: In IntelliJ IDEA, Reformat leaves the imports, and Optimize imports is to become ours + +## Status +Proposed + +## Date +2026-11-14 + +## Context + +With the plugin enabled, IntelliJ IDEA's Reformat runs open-java-format through the part-of-a-file entry ([ADR-001](ADR-001-one-output-per-entry-point.md)): `Main --character-ranges … --output-replacements`, which never touches imports. Imports are left to the IDE's Optimize imports, and that is the IDE's own `JavaImportOptimizer`, with the IDE's import layout and its thresholds for `import a.b.*;`. The command line, the GitHub Action and the build plugins then sort the imports again with `ImportOrderer`. + +In the sources of IntelliJ IDEA 2024.2.6, the platform the plugin is built against: + +- Every path that optimizes imports goes through `OptimizeImportsProcessor.collectOptimizers`: Optimize Imports; Reformat Code and Reformat Files with "Optimize imports"; Actions on Save; "Optimize imports" before a commit; the quick fix for an unused import; "Optimize imports on the fly". It asks `FormattingServiceUtil.findImportsOptimizingService(file)` for a service with `Feature.OPTIMIZE_IMPORTS`, and takes that service's `getImportOptimizers`. +- The plugin's `PalantirJavaFormatFormattingService.getFeatures()` returns `FORMAT_FRAGMENTS` only, and `getImportOptimizers` comes from `AbstractDocumentFormattingService`, which returns none; so `CoreFormattingService` answers with the IDE's optimizer. +- `processFile` of an optimizer runs in a background task under a read action (`OptimizeImportsProcessor.prepareTask`), so the time of one call counts. +- With "Optimize imports" on, Reformat Code on files and Reformat in the editor over the whole file run the formatter first and the optimizer after it (`AbstractLayoutCodeProcessor.getAllProcessors`, `FileInEditorProcessor`); over "only VCS-changed text" the editor runs the optimizer first. + +The plugin had an optimizer of its own for three weeks. google-java-format added one when it moved to the `FormattingService` API (google/google-java-format@84b2c9a2, 2023-03-02), running the formatter inside the IDE, for which users add `--add-exports` to the IDE's VM options. palantir/palantir-java-format#1078 followed it on 2024-04-26, and palantir/palantir-java-format#1089 removed the optimizer on 2024-05-21 as "slow". It called `FormatterService.fixImports`, which out of process runs `Main` without `--fix-imports-only`: a new JVM, and the whole file formatted, on every Optimize imports (#221). With the flag, one call on a small file takes about 20 ms with the native binary and 0.3 s with a new JVM. + +## Decision + +- **Reformat never changes imports**, over a part of a file or a whole one. It is the part-of-a-file entry. +- **Imports change only through Optimize imports.** Until the plugin's optimizer exists, that is the IDE's own. +- **The plugin's optimizer is to replace the IDE's** (#222): `getFeatures()` adds `OPTIMIZE_IMPORTS`, and `getImportOptimizers` returns an optimizer that calls `FormatterService.fixImports`. From then on it runs whenever Optimize imports is on, on every path listed above. +- **`fixImports` out of process passes `--fix-imports-only` first** (#221). + +## Alternatives Considered + +### Reformat of a whole file fixes imports too, in one pass +- Pros: Reformat of a whole file would give the command line's bytes, the imports included, without Optimize imports. +- Cons: Reformat would change imports with Optimize imports off, which the IDE's users do not expect of it. +- Rejected: imports belong to Optimize imports. + +### Keep the IDE's optimizer, and set its import settings to open-java-format's +- Pros: no optimizer of ours; a `CodeStyleSettingsModifier` can set the Java import settings for each file, as EditorConfig does. +- Cons: the IDE's sorting is still the IDE's, not `ImportOrderer`'s. +- Not chosen as the answer; a settings modifier may still help the imports the IDE inserts itself, through auto-import and completion, which no optimizer sees. + +### An optimizer running the formatter inside the IDE, as google-java-format does +- Pros: no process per call. +- Cons: the javac packages have to be reachable from the IDE's JVM. +- Left to how the plugin runs the formatter (#223); the optimizer calls `FormatterService.fixImports` whichever way that is. + +## Consequences + +- Until #222, imports in IntelliJ IDEA follow the IDE's settings, and the command line or CI may sort them again. +- #222 depends on #221. +- The optimizer's time counts inside a read action: about 20 ms a call with the native binary, about 0.3 s with a new JVM, on a small file. "Optimize imports on the fly" runs while typing. +- The plugin's formatting service is asynchronous, and google-java-format's optimizer notes that it "can overwrite the results from the main formatter": the optimizer has to be checked in a running IDE. diff --git a/docs/decisions/ADR-004-order-of-fixing-imports.md b/docs/decisions/ADR-004-order-of-fixing-imports.md new file mode 100644 index 000000000..1133d0246 --- /dev/null +++ b/docs/decisions/ADR-004-order-of-fixing-imports.md @@ -0,0 +1,49 @@ +# ADR-004: The order of fixing imports stays as it is, until #224 decides it + +## Status +Proposed + +## Date +2026-11-14 + +## Context + +The whole-file entry removes unused imports, sorts them and lays out the code ([ADR-001](ADR-001-one-output-per-entry-point.md)), and the code does it in two orders: + +| Order | Where | +|---|---| +| **After the layout:** layout, remove unused, sort, reflow | `FormatFileCallable.formatFile`: the command line, the native binary, the GitHub Action, the pre-commit hook, the playground | +| **Before the layout:** sort, remove unused, layout, reflow | `Formatter.formatSourceAndFixImports`: the Maven plugin and our Gradle Spotless step; Spotless' own `palantirJavaFormat()` calls the same steps in the same order, without the reflow | + +`ImportOrderer` moves a comment along with its import, and `RemoveUnusedImports` leaves the comment of an import it removes behind, so the order decides where that comment ends up: + +- **A comment at the end of a removed import's line:** after the layout it goes with the import above it; before it, it stays where the removed import was sorted to. +- **A comment under a removed import, above the class:** after the layout it stays on the class; before it, a blank line stays between them. +- **An extraneous semicolon between imports**, which javac 21 rejects with `--release 21` and accepts with a warning below it: after the layout the file fails with `extraneous semicolon`; before it, `ImportOrderer` drops the semicolon and the file is formatted. +- **A comment inside a module name**, `import module java./* comment */ base;`: after the layout it stays on one line; before it, the line breaks after the dot. + +Over 3,865 real files, as they are and with an unused import added to each, the orders give different files for 1, `CharsetEncoder` of JDK 21, and each order passes its check on what the other wrote. The 915 cases of `FormatterCasesTest` give the same output in both. + +google-java-format has moved to the other order: google/google-java-format@633ce6be (2026-10-20) removes unused imports, reorders modifiers and sorts imports in one pass before the layout, on its command line too, and remaps the requested character ranges through each of its edits. + +## Decision + +Keep both orders as they are, so that no surface's output changes now. The difference is an accepted exception of [ADR-001](ADR-001-one-output-per-entry-point.md), and #224 decides it later, for instance when google-java-format's `Cleanups` are ported. + +## Alternatives Considered + +### Fix the imports after the layout everywhere, as the command line does +- Pros: the command line, and with it what the GitHub Action and the pre-commit hook check, does not change; ranges stay offsets into the input. +- Cons: the Maven plugin and the Spotless step change in the cases above; an extraneous semicolon between imports fails them as it fails the command line; porting from google-java-format gets harder. +- Not now: it was the first commit of #220, reverted. + +### Fix the imports before the layout everywhere, as google-java-format does +- Pros: follows google-java-format; tolerates an extraneous semicolon; matches Spotless' own step. +- Cons: the command line's output changes in the cases above, and every user of the GitHub Action sees it; ranges have to be remapped through the import edits, and `ImportOrderer` and `RemoveUnusedImports` would return replacements instead of text. +- Not now: the larger change, best made together with google-java-format's `Cleanups`. + +## Consequences + +- The whole-file surfaces of the two groups can write different files where a removed import has a comment, or an import is followed by an extra semicolon. +- Cases that pin one order for these seams wait for #224. +- Whichever order #224 picks, the code of the other group changes, and its users see the files in the cases above change once.