Skip to content

feat(state): keep engine state in state/<org>/ - #97

Closed
dhruva-vapi wants to merge 10 commits into
mainfrom
feat/state-folder
Closed

dhruva-vapi wants to merge 10 commits into
mainfrom
feat/state-folder

Conversation

@dhruva-vapi

@dhruva-vapi dhruva-vapi commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Value

V.A.L.U.E. tier: small. One folder, one module that knows the old and new layout, one command that moves, tests for every path, docs.

  • Problem: everything the engine generates and tracks sat under dot-names in three places: .vapi-state.<org>.json, .vapi-state.<org>.snapshots/ and .vapi-state-hash/ at the repository root, and .vapi-org.json inside resources/<org>/ next to the hand-written .vapi-ignore. They read as leftovers. Deleting the state file is silent until the next push, which then creates every resource again as a duplicate.
  • Who it affects: anyone who opens the repository and has to tell which files are theirs to edit and which belong to the engine.

How it works

  1. One folder, state/<org>/. state.json (name → UUID) and org.json (the org pin) are committed. local/baselines/ and local/snapshots/ stay on the machine and are gitignored. state/README.md says the engine writes the folder and what breaks when a file goes missing.
  2. resources/<org>/ is hand-written only once an org has moved. .vapi-ignore stays there, beside the resources it governs. The rule a person can apply at a glance: resources/ is yours, state/ is the engine's.
  3. Upgrading moves nothing. An org is in one layout, decided by where its committed state file is. pull, push, apply and promote read and write the layout they find, on a person's machine and in CI, for as long as the repository stays there. They print one line per command saying the newer layout exists. src/state-paths.ts is the one module that knows both layouts.
  4. Only npm run migrate moves committed state. A promotion or apply run writes the state file and a later step commits it. If that step stages only the old file, moved state drops out of git while every run still passes. So migrate first prints the two paths that step has to stage, then moves every org with one rename per file when every commit step in .github/workflows/ is seen to stage state/. Otherwise it moves nothing and says why.
  5. A step the engine can't read needs a person's word. npm run migrate -- --commit-step-stages-state records it in state/commit-step.json and moves every org. The flag is refused while a workflow is seen to stage only the old file.
  6. The commit-step reader fails closed. A git add counts only as a plain command in a run: step at the repository root that names state or stages everything. After a cd, under a working-directory, after ||, in a loop, a function or a quoted string, with -u, with a variable or an exclusion in its paths: not counted. Such a step may work. The engine can't tell, so migrate asks for the flag.
  7. A CI run stops before any request when its workflow is seen not to commit what it would write. That is state in state/ under a step that stages only the old file or can't be read (and no recorded word), or state at the root under a step that stages only state/. In GitHub Actions the engine reads the workflow that is running it.
  8. A fresh clone keeps a merged change, in both layouts. With no baseline and nothing changed in git, pull used to keep a file only when it was newer than the state file. After a clone that is only the order git wrote the files in. Pull now compares against the moment this machine's engine last saved state, kept beside the baselines. A clone has no such save, so it keeps every file and apply sends the merged change.
  9. Two files that disagree stop the command. If the old and the new state file (or pin) both exist with different content, every writing command stops before any request and names both. With the same content the engine reads state/, keeps the old copy identical, and migrate removes it.
  10. The promotion workflow stages both places, removals included, so a move is committed as one change. It also stops failing on a repository with no root state file: git add exits 128 on a pathspec that matches nothing.
  11. Pull never counts the engine's own files as a person's edits. The keep rule in 8 is off whenever git shows a change under resources/<org>/. In the older layout the engine keeps the org pin there, and between migrate and its commit git shows the pin as deleted. Pull now leaves out every file on the list migrate moves, in both places, so a fresh clone still keeps a merged change.
  12. A first state file at the root says so. In a repository with no state whose commit step can't be read, the first state file goes to the root, as it did before this change. A step that stages only state/ would never commit it, and the engine can't tell. So the command prints one line: where the file goes, why, what has to stage it, and how to start in state/ instead.
  13. The engine's tests can't touch the repository they run in, and nobody else's tests are affected. Every module takes the engine's directory from src/base-dir.ts. Every engine test file imports tests/engine-sandbox.ts first, which points that at a temporary directory and makes every node:fs change inside the repository throw, however the file is started. A test file that doesn't import it, such as one a repository built from this one keeps under tests/, is left alone: the commands it starts and the engine modules it imports work in its repository, and it writes where it likes. The engine never looks at what a test runner leaves in the environment, and the preload npm test adds is what it was before this pull request.
  14. npm test starts no test runner inside a test. The tests that do, such as the suite run inside copies of a repository with every file compared, live in tests/repository/. They run with npm run test:repository and as a CI job that runs only in this repository.

Upgrade notes: docs/guides/commands.md, "Upgrading from an older version".

Verification

Nothing here was run against a live org. Every behaviour is covered by tests against fixtures, a real git origin, or a local stub API.

Path Test
A file is read and written where it is. An org with no state yet goes where the repository's other orgs keep theirs tests/state-layout.test.ts
A command leaves an old repository exactly where it is, on any machine and under any workflow. The older-layout line prints once per command same
pull against a stub API leaves an old repository where it is and keeps every UUID mapping, on a person's machine and in CI same
For an org in state/, this machine's baselines and snapshots follow. A baseline is still read from the old folder same
A move cut short between two files is read where each file is, and migrate finishes it same
Same content in both places: state/ is read and the old copy kept identical. Different content: stop, both files intact, no request sent same
No command moves state by itself, whatever is or isn't in the repository tests/state-layout-unseen-commit.test.ts
What the reader counts as staging state/, where a step runs, and every job in every workflow having to pass same
Every step read as staging state/ commits state/ when it is run for real same
migrate prints the two paths and moves nothing until a step is seen or a person says so. The flag is refused under a workflow that stages only the old file, and writes nothing where it isn't needed same, including the real command
A CI run on the old layout keeps working under a step the engine can't read. A CI run stops when state is in state/ under a step that isn't seen, and when state is at the root under a step that stages only state/ same
Pull in a fresh clone keeps a merged change, wherever the state file is tests/state-move-second-round.test.ts
A git add that leaves state/ out (after cd resources, in another job's folder, inside an echo) is not read as staging it same
With nothing seen to stage state/, a command that is not migrate leaves committed state where it is same
The real commit step, run against a bare origin: an old-layout repository still commits its root state file, and a move is committed whole tests/promotion-workflow-commit.test.ts
The whole suite, run inside a copy of a customer-shaped repository in the older layout, in state/ and with no state, leaves every tracked, untracked and ignored file and .git as it was. Each copy holds a test of the customer's own tests/repository/suite-leaves-the-repository.test.ts
The two test files whose cleanup once removed a repository leave every path of a copy in place, in all three layouts tests/repository/suite-keeps-the-repository.test.ts
An engine command runs the same with NODE_TEST_CONTEXT in its environment. A repository's own test that runs validate on its repository, or writes a report into it, passes as it does on main tests/repository/engine-beside-a-test-runner.test.ts
A repository's own test, run as npm test runs it: the command it starts validates its org, its report lands in the repository, and an engine module it imports works in that repository tests/repository/own-tests-as-before.test.ts
An engine test can't create, change or remove anything in the repository. The engine works in a temporary directory for an engine test and what it starts, also when the test is started by hand without the preload, and in its own repository for everything else, including a copy of src/ a test runs somewhere else tests/engine-sandbox.test.ts
In CI for this repository every test file imports the sandbox first. No test in npm test starts a test runner same
That import check is asked in CI for VapiAI/gitops and nowhere else: not in a clone of it on a person's machine over https or ssh, not on a runner in a fork, in a fork that kept the name or in a repository that kept this origin, not without an origin tests/repository/import-check.test.ts
No module under src/ works out the repository's directory for itself. src/user-agent.ts is the one exception: it reads the engine's own package.json for the version it sends tests/base-dir.test.ts
In a fresh clone, pull keeps a merged change while the org pin is uncommitted, and between migrate and its commit. A person's own uncommitted file there still counts, as before tests/state-records-third-round.test.ts
A first state file at the root gets one line with where, why and what has to stage it. A workflow seen to stage state/ gets none, and once state exists the line is the older-layout one. One line per command, also through a real pull tests/first-state-file.test.ts

Each rule was also checked by breaking it. 33 one-line mutations each fail a test that names the rule: 4 of the fresh-clone rule (a clone no longer keeps its files, the save time ignored, the stamp never written, the old copy not kept identical), 17 of the commit-step reader (a git add counted after cd, after ||, inside an if condition, a loop or a function, with any flag, an exclusion or a variable, in a here-document, under a working-directory, in a checkout sub-folder, under another shell or GIT_DIR, and the job rules), and 12 of the layout and migrate rules (a command moving state when the step is seen, migrate moving with nothing seen or whatever the gate says, a CI run not stopped, a new repository always in state/, a new org ignoring the others, the notice printed every time). Eight more for the rules added after that each fail a named test too: the engine's files counted as edits again, only the state/ paths left out, every file in the folder left out, the first-state line dropped, shortened, printed beside the older-layout line or printed in a repository that has state.

Sixteen for the test sandbox were run on this head, and each fails a named test: the engine ignoring the test directory, or using it from a copy of src/ somewhere else; the write guard letting everything through or missing named imports; the sandbox not naming its repository; the engine refusing to start under a test runner; the preload sandboxing every test file; a test file dropping the import, under tests/ or tests/repository/, or not importing it first; a test in npm test starting a test runner; and the import check asked in every repository, in none, on any runner, in a clone whose origin is this repository, or missing a runner named in lower case.

Outside the suite, a pipeline's own shell steps were run in a fresh clone per run against a bare origin and a recording stub of the API, with a checker comparing the origin's committed state with the platform after every run:

  • A repository that never runs migrate sends the same requests as main, line for line, under upstream's workflow, the pre-change workflow, a helper script, a read-only token, a cd before the git add, and three apply-on-merge pipelines (GitHub Actions, another CI system, and one whose script stages both layouts). No state file moves and nothing is left uncommitted.
  • A change merged without a local apply reaches the platform from a fresh clone, with state at the root and after migrate.
  • After migrate under upstream's workflow the requests are the same as main and the move is committed whole.
  • A workflow reverted to the pre-change one after migrate stops before any request.
  • The steps in "Going back to an older engine" restore every file, ignored ones included, byte for byte.

npm run build passes. npm test passes 645/645 in CI (584 on main). On a person's machine one of those, the import check, is skipped. npm run test:repository passes 9/9.

Things to know

  • --commit-step-stages-state is a person's word and the engine can't check it. If it is passed and the commit step doesn't stage state/, promotion runs pass while state is never committed, and a later run creates resources again. It is refused only when a workflow is plainly seen to stage the old file alone. The harness shows exactly this when the word is wrong.
  • The engine reads .github/workflows/ and nothing else. A pipeline in another CI system, or a script that commits with its own token, is never read. That is why migrate always prints the two paths first.
  • A step that works but can't be read (staging through a script, a commit action, a variable, git add -u) needs the flag once.
  • A repository with no state at all starts in state/ only where a commit step is seen or said to stage it. Anywhere else its first state file goes to the root, as before this change, and the command says so in one line. A script that stages only state/ has to be told about with --commit-step-stages-state before the first pull, or that file is never committed.
  • On a machine with no baselines, the first pull after upgrading can bring a dashboard edit down one pull late. Nothing is lost. The upgrade notes say so.
  • npm test takes about what it took before, for what it runs. On one Mac in one window, back to back: 113 and 116 s on main (584 tests), 159 and 164 s here (645 tests). The tests that run the suite inside copies of a repository are in npm run test:repository and in a CI job of their own, which a fork's CI skips.
  • The check that every test file imports the sandbox is asked only in CI for this repository, so a clone that holds a test of its owner's behaves as it does on main. A contributor who leaves the import out finds out on the pull request. GITHUB_REPOSITORY=VapiAI/gitops npm test asks it on a machine.
  • Reverting to an engine from before state/ after migrate creates duplicates unless the state files are moved back first. The steps are in the upgrade notes.
  • Dashboard backups (<name>.<timestamp>.bkp.md) stay beside the resource they shadow, and tmp/promotion-applied.txt stays in tmp/. .env.<org> is unchanged.
  • After migrate, a change to the org pin no longer triggers the promotion workflow, because the pin left resources/.

dhruva-vapi commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor Author

dhruva-vapi and others added 10 commits November 21, 2026 03:35
Everything the engine generates and tracks sat under dot-names in three
places: .vapi-state.<org>.json, .vapi-state.<org>.snapshots/ and
.vapi-state-hash/ at the repository root, and .vapi-org.json inside
resources/<org>/. It now lives in state/<org>/: state.json and org.json are
committed, local/baselines and local/snapshots stay on the machine.

An older repository keeps working. Every command reads the old places. The
next pull, push or apply on a person's machine moves the org, one rename
per file, and prints one line. The move is held back in a CI run and while
a workflow stages .vapi-state.*.json but not state/, because either would
commit the old file's removal without the new file. The promotion workflow
stages both places.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…to it

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
The guard that holds the move back looked for one known-bad pattern: a
"git add" line naming .vapi-state. with none naming state. A commit step
it couldn't read (a helper script, a commit action with a file pattern, a
pipeline outside GitHub Actions) passed as safe. A command on a person's
machine then moved state, and every later promotion run passed while
state/<org>/state.json was never committed, so resources were created
again as duplicates.

The move now fails closed. State moves by itself only when a workflow is
seen to stage state/, or when nothing in the repository commits state
after a run: no workflow that can write to the repository, and no root
promotion.yml. A CI run stops before any request when it would write
state in a place its commit step isn't seen to stage. In GitHub Actions
it reads the workflow that is running it.

npm run migrate, where nothing is seen to stage state/, prints the two
paths the commit step has to stage and moves nothing. A person who has
updated their step says so with --commit-step-stages-state, which is
recorded in state/commit-step.json. The engine can't check that word.

Docs: what counts as seen, the limit of the flag, what a customer sees
after upgrading, and how to go back to an older engine without creating
duplicates.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
An org is in one layout, decided by where its committed state file is.
Pull, push, apply and promote read and write the layout they find, on a
person's machine and in CI, and print one line saying the newer one
exists. npm run migrate is the only command that moves state, after
saying what the commit step has to stage.

The commit-step reader fails closed: a git add counts only as a plain
command at the repository root that names state or stages everything.
With nothing seen, migrate asks for --commit-step-stages-state.

Pull no longer reads "edited since the last sync" off the state file's
timestamp, which in a fresh clone is the order git wrote the files in.
It compares against the moment this machine's engine last saved state,
and a clone with no such save keeps every file.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
… files out of pull's local edits

Running the test suite in a repository with a state file at its root removed
the repository: two tests cleaned up two folders above the engine's
baselines folder, and in the older layout that is the repository root.

The engine's directory now comes from one place, src/base-dir.ts. The test
preload points it at a temporary directory, makes every node:fs change
inside the repository throw, and the engine refuses a test started without
the preload. A new test runs the whole suite inside a copy of a
customer-shaped repository in the older layout, in state/, and with no
state, and compares every tracked, untracked and ignored file before and
after. That run found one more writer, tests/org-connection.test.ts, which
created tmp/ in the repository.

Pull's keep rule for a file with no baseline is off whenever git shows a
change under resources/<org>/. The engine's own files there (the org pin in
the older layout, also shown as deleted between migrate and its commit) are
no longer counted.

A repository with no state whose commit step can't be read still gets its
first state file at the root, and the command now says where the file goes,
why, and what has to stage it.

The upgrade guide gains one line: on a machine with no baselines, the first
pull after upgrading can bring a dashboard edit down one pull late.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…gnores the test runner

The engine refused to start when NODE_TEST_CONTEXT was in its environment
without a sandbox, and the preload npm test adds sandboxed every test file.
Node's test runner sets that variable for every process it starts, so an
engine command started from a repository's own test exited 1, and such a
test could no longer run the engine on its repository or write a file into
it. Both worked before the sandbox existed.

The sandbox is now a module, tests/engine-sandbox.ts, that every engine
test file imports first: the temporary directory for the engine and the
node:fs guard on the repository. It holds however the file is started, by
hand without the preload included. tests/no-vapi-api.ts is what it was
before the sandbox, and src/base-dir.ts never looks at the test runner's
environment. A test file that doesn't import the module is left alone: the
commands it starts and the engine modules it imports work in its
repository, and it writes where it likes.

In the engine's own repository (a runner whose GITHUB_REPOSITORY is
VapiAI/gitops, or a clone whose origin is) a test file without the import
fails the suite. In any other repository that check is skipped.

tests/base-dir.test.ts fails when a module under src/ works out the
repository's directory for itself.

npm test no longer starts a test runner inside a test. The suite run
inside copies of a customer-shaped repository, and the other tests that
start a runner of their own, move to tests/repository/ and run with
npm run test:repository, one file at a time, and as a CI job in the
engine's own repository. A suite inside a suite tripled the time npm test
took and reached the twenty-second limits some tests give a command
sooner on a slow machine. Each copy now holds a test of the customer's
own and runs the engine's test files only.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…sitory

The check that every test file imports tests/engine-sandbox.ts first was
also asked on a person's machine, wherever origin pointed at VapiAI/gitops.
A repository cloned straight from the engine's keeps that origin, so a test
of its owner's under tests/ made one engine test fail in npm test. Before
the sandbox existed it passed.

The check is now asked only where GITHUB_REPOSITORY is VapiAI/gitops, which
is CI for the engine's own repository. A contributor who leaves the import
out finds out on the pull request. GITHUB_REPOSITORY=VapiAI/gitops npm test
asks it on a machine.

Tests and docs only.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

Replaced by #109 (same commits rebased onto main, opened as the Devin bot so it can be approved).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant