Skip to content

feat(appkit): add appkit db verify - #674

Draft
ditadi wants to merge 36 commits into
stack/database-dev-flow/05-db-initfrom
stack/database-dev-flow/06-db-verify
Draft

ditadi wants to merge 36 commits into
stack/database-dev-flow/05-db-initfrom
stack/database-dev-flow/06-db-verify

Conversation

@ditadi

@ditadi ditadi commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

🥞 Stacked PR (generated by Stack)

Use this link to review incremental changes.


Stacked on #672. The design is #677.

What did you change, and why?

A developer, an agent or CI needs one read-only command that answers "will my app work with its database?". appkit db verify checks the branch .env points at, in one READ ONLY transaction that is rolled back. It reports declared tables and columns the branch lacks, unapplied migrations, migrations the migrator will never run there, and migrations this checkout does not commit. The schema check is findSchemaProblems, split out of the plugin's startup check, so verify reports the problems the app would fail on.

appkit db verify --deployed checks a databricks.yml target's app through the Databricks API only, so it needs no Postgres role on the deploy branch. It checks that the app exists and is running, that its postgres resource names the target's branch and database with CAN_CONNECT_AND_CREATE, that its service principal has a role on the branch, and that its active deployment has every committed migration. It refuses when the target's workspace.host is not the workspace the shell is authenticated against. Both modes exit 1 on any finding, and each finding names its fix.

This revision changes how --deployed proves the migrations:

  • The only evidence is the source snapshot the active deployment runs, deployment_artifacts.source_code_path, read with GET /api/2.0/workspace/export. Each committed migration needs the same journal entry and byte-identical SQL there. A migration in the snapshot that the checkout does not commit fails too.
  • The result is verified, differs with the missing and changed tags, or unavailable with a reason. report.ok is false unless the migrations are verified or none are committed.
  • A failed export, empty content, invalid JSON, a journal that is not drizzle-kit's, or a deployment without a snapshot gives unavailable. A journal 404 counts as no migrations only after GET /api/2.0/workspace/get-status shows the snapshot exists.
  • The fallback to the deployment's create_time and the match by tag name are gone.

This revision also makes these changes:

  • When .databricks/bundle/<target>/variable-overrides.json exists, --deployed resolves the target with it, as a deploy from this checkout and the deploy guard do. A message names that file when a value came from it.
  • writeRefusal and everyBranchRefusal in guard.ts now build the refusal text of introspect, migrate, clean, seed and init. While a databricks.yml reference is unresolved, the text names it and drops advice that leads to appkit db init, because init refuses every branch then too.
  • The reconcile fix for two migrations that share a when moved down to feat(appkit): apply committed database migrations on deploy start #654, so this PR no longer carries a copy.

Review round, 2026-11-09:

  • verify --deployed fails when the app has no service principal or the target's workspace.host cannot be resolved from the checkout, and validates and compares the deployed journal entry by entry.
  • Local verify names declared tables left in public as the first hint, and the out-of-order message says the deployed app refuses that history and how to regenerate the migration.
  • Help text and output say verify --deployed checks the deployed source, binding, running state and role, not that migrations ran on the database.

Fix round for fjakobs's review of #677, 2026-11-15:

  • Local verify reports a fork copy, untouched or with an interrupted rebuild, but only on your own dev or session branch. It gives the same advice as migrate and leaves the copy's tag out of the unknown-migration count.

How do you know it works?

  • verify.test.ts covers the local cases, and runs --deployed against a faked Apps, Workspace and Lakebase API where the developer has no role on the deploy branch. The deployed cases include changed SQL under the same tag, each way the snapshot can be unreadable, a snapshot without a journal, an overrides file, another authenticated workspace, each binding mismatch and each app state. They assert that no database connection and no write happens.
  • guard.test.ts and introspect.test.ts cover the shared refusal text under an unresolved reference.
  • The full vitest suite passed against a local PostgreSQL 14. A clean build, the size gate, the docs build and typecheck passed. The packed tarballs export the same root and ./beta names as main, and ./beta/database-tooling adds only runVerify.
  • An earlier revision checked the apps API's postgres resource format, a missing app's 404, the deployment's source_code_path and getHost live and read-only. This revision was not run against a real Lakebase workspace. Nobody has checked whether format=AUTO returns a file's exact bytes. If it does not, verify reports the migration as changed, which fails closed.

Review round, 2026-11-09: each fix has a failing-then-passing test in its PR, and every layer's appkit and shared tests ran against Postgres 14 after the sync. At the top of the stack, pnpm build, pnpm test (6502 passed, 1 skipped), pnpm -r typecheck, pnpm check, pnpm knip and pnpm docs:build pass.

Fix round for fjakobs's review, 2026-11-15: each fix has a failing-then-passing test in its PR, and every layer's database, plugin and CLI tests ran against Postgres 17 after the sync. A throwaway Lakebase fork confirmed that the developer role can write the service principal's rows but cannot alter its tables. At the top of the stack, pnpm test (6602 passed, 1 skipped), pnpm -r typecheck, pnpm build and pnpm docs:build pass, and oxlint and oxfmt are clean on every changed file.

`appkit db verify` checks, read-only, that the branch .env points at has every
table and column config/database/schema.ts declares and every committed
migration. It also reports the migrations the migrator would never run there:
merged out of order, edited after they ran, or run from a checkout that no
longer commits them.

With --deployed it checks a databricks.yml target's app through the
Databricks API only: the app exists, its postgres resource names the target's
branch and database with CAN_CONNECT_AND_CREATE, its service principal has a
role there, it is running, and its active deployment has every committed
migration. -t picks the target. Verify refuses when the target's
workspace.host is not the workspace the shell is authenticated against. It
reads no database, so it needs no role on the deploy branch.

databricks.yml is read once for the guard, init and verify. The reader follows
include:, resolves ${var.*}, ${bundle.target} and ${bundle.name}, reads the
branch on an app's postgres resource, and lists every target.

Signed-off-by: ditadi <victordperd@gmail.com>
@ditadi
ditadi force-pushed the stack/database-dev-flow/06-db-verify branch from 42df642 to 1b3bde7 Compare October 20, 2026 12:22
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 41109405411 -R databricks/appkit -n appkit-template-0.89.0-pr.909a30d-stack-database-dev-flow-06-db-verify-674 -D appkit-pr-674 \
  && unzip -o "appkit-pr-674/appkit-template-0.89.0-pr.909a30d-stack-database-dev-flow-06-db-verify-674.zip" -d "appkit-pr-674" \
  && databricks apps init --template "appkit-pr-674"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

@databricks/appkit

npm tarball (packed): 1.4 MB (+19 KB) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 1.4 MB (+20 KB) 522 KB (+6.4 KB)
Type declarations 482 KB (+3.5 KB) 176 KB (+1.2 KB)
Source maps 2.9 MB (+41 KB) 970 KB (+12 KB)
Other 11 KB 3.7 KB
Total 4.8 MB (+65 KB) 1.6 MB (+19 KB)
Per-entry composition (own code — deps external (as shipped))
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
. 106 KB 2.5 KB 108 KB external 343 KB
./beta 108 KB (+37 B) 485 B 108 KB (+37 B) external 324 KB (+147 B)
./beta/database-tooling 49 KB (+3.8 KB) 0 B 49 KB (+3.8 KB) external 145 KB (+11 KB)
./testing 45 KB 32 KB 77 KB external 223 KB
./tsdown 520 B 0 B 520 B external 813 B
./type-generator 26 KB 0 B 26 KB external 74 KB

Chunks:

Entry Chunk Load Size (gz)
. index.js initial 101 KB
. utils.js initial 4.6 KB
. remote-tunnel-manager.js lazy 2.5 KB
./beta beta.js initial 88 KB
./beta modular.js initial 6.0 KB
./beta stream-manager.js initial 5.9 KB
./beta service-context.js initial 4.2 KB
./beta databricks.js initial 3.3 KB
./beta client.js initial 647 B
./beta index.js initial 20 B
./beta supervisor-api.js lazy 192 B
./beta databricks.js lazy 178 B
./beta index.js lazy 115 B
./beta/database-tooling index.js initial 49 KB
./testing manifest.js initial 32 KB
./testing index.js initial 10 KB
./testing wide-event-emitter.js initial 2.9 KB
./testing index.js lazy 28 KB
./testing remote-tunnel-manager.js lazy 2.5 KB
./testing utils.js lazy 1.8 KB
./tsdown index.js initial 520 B
./type-generator index.js initial 26 KB

@databricks/appkit-ui

npm tarball (packed): 397 KB (+5 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 429 KB 147 KB
Type declarations 267 KB 98 KB (+2 B)
Source maps 853 KB 286 KB
CSS 16 KB 3.2 KB
Total 1.5 MB 534 KB (+2 B)
Per-entry composition (consumer bundle — deps bundled, peerDeps external)
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
./js 5.3 KB 49 KB 55 KB 208 KB 14 KB
./js/beta 2.1 KB 0 B 2.1 KB 0 B 5.0 KB
./react 433 KB 49 KB 481 KB 1.3 MB 177 KB
./react/beta 4.9 KB 0 B 4.9 KB 0 B 12 KB

Chunks:

Entry Chunk Load Size (gz)
./js index.js initial 5.2 KB
./js chunk initial 120 B
./js apache-arrow lazy 49 KB
./js/beta beta.js initial 2.1 KB
./react index.js initial 431 KB
./react tslib initial 2.1 KB
./react apache-arrow lazy 49 KB
./react/beta beta.js initial 4.9 KB

ditadi and others added 14 commits October 30, 2026 17:20
Conflicts in targets.ts and its test: take the corrected 05 bundle model
(every postgres binding, BundleValue, variable-overrides.json,
readDeployBoundary) and add what verify needs on top: each app's declared and
resolved name, each target's workspace host, the bundle file name, and
defaultBundleTarget, which readDeployTarget now uses. verify --deployed reads
the one postgres binding of the target's app and refuses an app with several,
or a branch or database the files do not resolve. The verify CLI test mocks
the @databricks/appkit/beta/database-tooling path.

Signed-off-by: ditadi <victordperd@gmail.com>
verify --deployed treated an unreadable deployed journal as a reason to
compare the migrations' timestamps with the deployment's create time, and it
matched migrations by tag only, so two checkouts with different SQL under the
same 0001_migration tag passed. Now the only evidence is the source snapshot
the active deployment runs. Each committed migration needs the same journal
entry and byte-identical SQL there. The report says verified, differs (with
the missing and changed tags) or unavailable (with the reason). An export,
parse or snapshot failure is unavailable, and report.ok is false unless the
migrations are verified or none are committed. The deployed journal goes
through the same parseJournal check as the committed one.

Signed-off-by: ditadi <victordperd@gmail.com>
… no migration

verify --deployed returned no migration result before reading the snapshot
when the checkout committed none, so running it from a bundle root whose app
lives in a subfolder passed without looking at the deployment. It now always
reads the snapshot's journal. With nothing committed, a snapshot without
migrations passes and one with migrations is unavailable, naming them. With
migrations committed, ones the snapshot has and the checkout does not commit
fail too, as in local verify. A snapshot without a journal (a 404) has no
migrations, so every committed one is missing and the fix is to deploy.

Signed-off-by: ditadi <victordperd@gmail.com>
The no-snapshot case gave the deployment no source_code_path at all, so a
fallback to that live, mutable folder would have passed. The deployment now
has one, and the test asserts the result is unavailable with no export.

Signed-off-by: ditadi <victordperd@gmail.com>
verify --deployed read the target with databricks.yml's own values only, so a
branch set only in .databricks/bundle/<target>/variable-overrides.json was
refused as unresolved. It now reads the target as a deploy from this checkout
does, and as the deploy guard already reads it: the overrides file's values
when the file exists. The workspace host is resolved with the same values, and
a refusal names the file.

Signed-off-by: ditadi <victordperd@gmail.com>
… lock

With an unresolved reference in databricks.yml, init, clean and seed refuse
every branch, not only migrate. Only migrate, introspect, clean and init take
the history lock; seed, verify and clean --delete-branch take none.

Signed-off-by: ditadi <victordperd@gmail.com>
While databricks.yml has a reference the checkout's files cannot resolve,
migrate applies to no branch and init, introspect, clean and seed refuse
every branch, yet introspect and migrate still pointed at init. One guard
function now builds every refusal: it names the references and where to give
them values, and gives the caller's advice only when some branch can take
writes. The .env branch carries the boundary that classified it.

Signed-off-by: ditadi <victordperd@gmail.com>
A journal 404 counted as "no migrations" even when the snapshot itself was
gone, so a checkout without migrations passed on no evidence. A 404 now
counts only after get-status shows the snapshot exists; otherwise the result
is unavailable. Messages that credit databricks.yml with a branch or database
now name variable-overrides.json when the value came from it, the no-commit
advice no longer sends you to a directory verify cannot run from, and --help
says verify fails on deployed migrations the checkout does not commit and
reads the overrides file.

Signed-off-by: ditadi <victordperd@gmail.com>
…vice principal

An app whose API response has no or an empty service_principal_client_id
skipped the role check and could report ok.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…rvice principal

Without a service_principal_client_id the role check was skipped and the
report could be ok. A missing id is now a problem that names what to
check.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…s do not resolve

An unresolved host read as null, so the check that the shell is on the
target's workspace was skipped.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…o not resolve

The bundle reader now tells a host no file declares (null) from one it
cannot resolve (unresolved, as written). verify refuses the latter before
any API call and says where to give the variable a value, so the check
that the shell is on the target's workspace is never skipped.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
… entries must not verify

Tag membership let [A, B, A], a {} entry and a reordered journal pass as
verified.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…ating it

Each deployed entry must be a migration with a tag, an idx and a when,
and no tag may be listed twice; otherwise the journal is unavailable,
never a match. When both sides list the same tags, each must also sit at
the same place, since the app applies the journal in order.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
ditadi and others added 21 commits November 9, 2026 13:10
… refuses that history

The text said the deployed app skips the migration; deploy fails closed
with MigrationHistoryError, and clean + migrate only fixes the local
branch.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…istory

verify said the deployed app skips a migration merged out of order, and
pointed at clean + migrate, which only rebuilds the local branch. Deploy
fails closed with MigrationHistoryError, so verify now gives that error's
guidance: regenerate the migration after the recorded ones.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…s the cause

The plugin's startup check names that cause and says not to migrate;
verify did not, and recommended migrate, which would create empty app
tables.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…ther hint

The startup check's diagnosis moves into diagnoseSchema, which verify
now shares, so a branch whose declared tables sit in public gets the
schemaName advice as its first hint instead of migrate, which would
create empty app tables.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…ndings, state and role

verify reads no database, so it cannot tell whether the migrations ran
there. The help text, the pass line and the JSDoc now say what it checks
instead: the snapshot's journal and SQL bytes, the postgres binding, the
running state and the service principal's role. The help text also names
the refusal of a workspace.host the files do not resolve.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
verify keeps one postgres binding per resource and reads app names next to
#666's unresolved-structure markers; its test fakes answer #654's journal
check and #655's shared tables-in-public lookup.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
# Conflicts:
#	packages/appkit/src/database/tooling/migrate.ts
#	packages/appkit/src/plugins/database/schema-check.ts
…ld, not an unknown migration

A branch init tagged with appkit:fork-copy reports forkCopy true and ok
false, and the tag row is not counted in unknownMigrations. An untagged
branch reports forkCopy false and nothing else changes. Against the fake
branch and against Postgres.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A branch that `appkit db init` forked keeps the tag row in its copied
journal until the rebuild finishes. Local verify counted that row as a
migration this checkout does not have, which pointed at clean and
migrate, and migrate refuses such a branch. The report now reads the
tag in the same read-only transaction, carries it as forkCopy, leaves
it out of unknownMigrations, and is not ok while it stands. The shared
tooling contract carries the field for the CLI.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…to migrate

With forkCopy set, the verdict line names the copied rows and the init
advice migrate gives when it refuses, and no hint says migrate or clean.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A copy can match schema.ts and the committed migrations and still not
be yours, so its verdict line names the copied rows and the rebuild,
with the same advice migrate gives when it refuses: `appkit db init`,
or `appkit db init --schema migrate` to keep the rows. The other hints
would point at migrate or clean, which refuse or lose the copy.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
# Conflicts:
#	packages/appkit/src/database/tooling/init.ts
A copy the service principal still owns passes verify today: it has
every table and every committed migration, so nothing says its rows
are not yours. Verify must report it the way migrate does when it
refuses, with the init advice for the state the branch is in, and
must leave the fork-copy tag out of the unknown count without
correcting it by hand.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…rate's advice

Local verify read only the tag, so a copy the service principal still
owns passed. It now reads the schema state in the same read-only
transaction and carries migrate's refusal text as the report's
forkCopy, null when the schema is yours, absent or public. The drift
read leaves the tag out through ignoreHashes instead of a hand
correction of the unknown count. The never-writes guard now matches a
statement's verb, as the ownership read names the CREATE privilege.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
The tooling now reports forkCopy as the advice for the state the
branch is in, an untouched copy or an interrupted rebuild. The CLI
must print that line as the verdict, and no hint that says migrate
or clean.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
The tooling builds the line from the state it read, so the CLI prints
it as the verdict instead of its own text, which only knew the
interrupted rebuild.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
The comment audit of the stack killed them. The describe gate, the
setup statements and the stub's regex say what they said.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…ranch

verify reads any .env branch, production included, where another role
owning the app schema is the normal state. The tests now run on a branch
id that a real user name hashes to, with that user in PGUSER, and add a
deploy branch and another user's dev branch that must verify with no
fork-copy report.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…anch

verify reads whichever branch .env names, production included, and
reported any app schema another role owns as the fork's copy, with
advice to rebuild it. Only init forks a copy, onto the developer's own
dev or session branch, so verify now reads the schema state only when
PGUSER's branch id matches the branch, as init wrote both. On any other
branch forkCopy is null and the other checks decide ok.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant