Skip to content

Make the sealed conversation-plan suite pass again, and run it in CI - #968

Merged
lavaman131 merged 4 commits into
mainfrom
conversation-plan-suite
Nov 20, 2026
Merged

lavaman131 merged 4 commits into
mainfrom
conversation-plan-suite

Conversation

@lavaman131

@lavaman131 lavaman131 commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

The conversation-plan suite (e2e/conversation-plan.playwright.config.ts, run sealed by bun scripts/conversation-plan.ts) had not run since its image stopped building in October. #965 made the image build again. That run left 27 tests failing and its last tests cut off by the worker's time limit. This PR makes all 85 pass, gives the run room, and adds a CI job so it can't silently drift again.

Why the tests failed

All of these are the server and client moving on under the suite. None is a server regression.

  • Analysis markers were never drawn. The client draws a message's analysis marker only when built with VITE_MESSAGE_ANALYSIS=on (1c0f506). scripts/e2e.ts and CI build the browser suites' client with it, but the image did not. The image now builds with VITE_MESSAGE_ANALYSIS=on and VITE_CHILD_MOTION_TOGGLE=on, as they do.
  • A person alone in a document is talking to chopin (bc2fcd9). Their room message goes to the Planner and is never analysed, so most single-writer tests waited for a decision that never came.
    • Tests that converse alone now keep a second writer in the document (keepCompany in e2e/jev-wire.ts), as the main suite already does (conversation-initiative.e2e.ts).
    • The scripted Planner scripts background jobs only. It used to fail every ordinary turn with "scripted Planner requires one exact background job header". It now ends an ordinary turn without a word, whether automatic participation's or a person's message to chopin.
  • Routine notices are kept out of Chat (29ece9c). "chopin refined …" and "chopin wrote up …" are recorded but filtered from view. The two tests that expected them visible now read them from the wire and expect none on screen.
  • Projects lists top-level documents only. The child-panel test opens its child from a link in the parent's Chat (seedLinkedChildChannel), as the main suite does.
  • Empty caret lines after a decided card. A decided card may now be followed by empty paragraphs. The two Backspace tests start from the paragraph right after the hidden card, which is the behaviour they test, instead of the document's last paragraph.
  • Shorter option rationales. An option's rationale is at most 160 characters (f94ee94). The long-panel test's suggestion used 239, so the tool refused it. It now uses 159.
  • Screenshot path. The stress test wrote a screenshot into a tracked docs/prototypes/ path, which made the next contained run see a dirty tree. It now goes to the test's own output directory.

Time limit

Run whole, the suite takes 7.3 to 7.4 minutes here. The worker's limit was 10 minutes, and the earlier run hit it. The worker now allows 20 minutes and the supervisor 25.

Product note: no decision cards for a lone writer

With CONVERSATION_PLAN=on, a person alone in a document gets no decision cards from Chat. Their messages go to chopin, as an @chopin message always has, and are not analysed. That follows from the alone rule, which was chosen on purpose, so this PR doesn't change it. It does mean the conversation-plan features only apply once a second writer is present.

CI

A new conversation plan job in ci.yml builds the image and runs the sealed suite. It covers CONVERSATION_PLAN, which is off by default, and takes about as long as the slowest browser shard, so it is not part of either required gate (format, lint, types, tests and e2e). It builds without the GHA layer cache, because the Playwright base would crowd the application image out of the repository's cache. On failure it uploads the run's report. AGENTS.md lists the job.

Verification

  • Contained run: docker build -f e2e/conversation-plan.Dockerfile --build-arg CHOPIN_SOURCE_COMMIT=<head> ., then bun scripts/conversation-plan.ts --image <tag> --postgres-image postgres:17 on 35248fe5:
    • The supervisor's phases all passed: prebuilt images validated, the stopped worker's source and build stamp validated, owned containers started, worker exited.
    • Egress was denied and migrations ran.
    • 85 passed in 7.4 minutes, worker exit 0. Both owned containers were removed and the worktree stayed clean.
  • Rebase: the branch was then rebased onto main (1c366016, an Atomic version bump the scripted harness does not use). The new CI job runs the suite on this head.
  • Outside the container: the same suite also passed, 85 of 85 in 7.3 minutes, with a local PostgreSQL 17.
  • Repo checks: bun run fix (no changes beyond these), bun run ci and bun run types pass.
  • Unit tests: bun test apps/server/src/conversation-plan/prompt-scripted-harness.test.ts scripts/conversation-plan-worker.test.ts e2e/harness/scripted-register.test.ts: 78 pass.
    • prompt-scripted-harness.test.ts pinned the old fixture contract, under which an ordinary message was rejected. That case moved to its own test: the turn ends with no tool call and no error. The test fails on the old fixture.
    • An empty prompt is still refused.
  • CI: the new conversation plan job passed on the first head in 11 minutes 32 seconds.

…the browser suites do

The contained suite opens each message's analysis marker, which the client
draws only when built with VITE_MESSAGE_ANALYSIS=on (1c0f506). scripts/e2e.ts
and CI build the browser suites' client with it and with the child motion
toggle; the image built it without either, so every test that reads a
message's analysis waited on a button that was never drawn.

Co-authored-by: Alex Lavaee <lavaman131@github.com>
The suite had not run since its image stopped building, and the server moved
on under it:

- A person alone in a document is talking to chopin (bc2fcd9), so their room
  message goes to the Planner and is never analysed. Tests that converse alone
  now keep a second writer in the document (keepCompany), as the main suite
  does. The scripted Planner, which scripts background jobs, ends an ordinary
  turn (a person's message to it, or automatic participation's) without a
  word instead of failing it.
- Routine "chopin refined" and "chopin wrote up" notices are recorded but kept
  out of Chat (29ece9c): the tests read them from the wire and expect none in
  view.
- Projects lists top-level documents only, so the child panel test opens its
  child from the parent's Chat link.
- A decided card may be followed by empty caret lines, so the Backspace tests
  start from the paragraph right after the hidden card.
- An option's rationale is at most 160 characters (f94ee94); the long-panel
  test now uses rationales of that length.
- The stress test's screenshot goes to its own output directory instead of a
  tracked docs folder.

Co-authored-by: Alex Lavaee <lavaman131@github.com>
Run whole, the suite takes seven or eight minutes on a quiet machine, close to
the worker's ten-minute limit, which stopped its last tests on a busy one. The
worker now allows twenty minutes and the supervisor twenty-five, past the
worker's own limit with room to start and clean up.

Co-authored-by: Alex Lavaee <lavaman131@github.com>
The suite stopped building and drifted from the server for weeks because
nothing ran it. CI now builds its image and runs it with
scripts/conversation-plan.ts, as a job of its own that starts with the others.
It covers CONVERSATION_PLAN, which is off by default, and takes about as long
as the slowest browser shard, so neither required gate waits on it. The image
is built without the layer cache: its Playwright base would crowd the
application image out of the repository's.

Co-authored-by: Alex Lavaee <lavaman131@github.com>
@lavaman131
lavaman131 force-pushed the conversation-plan-suite branch from f047edf to 1a48e45 Compare November 20, 2026 20:36
@lavaman131
lavaman131 merged commit 7b7f868 into main Nov 20, 2026
25 checks passed
@lavaman131
lavaman131 deleted the conversation-plan-suite branch November 20, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant