Repository navigation
Conversation
This was referenced Oct 16, 2026
ditadi
force-pushed
the
stack/database-dev-flow/04-db-introspect
branch
from
October 19, 2026 15:02
a2522e5 to
c3e1172
Compare
`appkit db introspect` reads an existing Postgres schema (default: app) from pg_catalog and writes config/database/schema.ts from it, keeping the previous file as schema.ts.bak. It also writes the migration that brings history up to the database, 0000_baseline first and NNNN_introspect after, and records it as applied. That is only true on the branch it read, so it runs only on the developer's dev or session branch, which `appkit db init` creates. The deployed app adopts 0000_baseline on start when its tables already exist. What the schema builder cannot express is skipped and reported, and what it declares differently from the database (a timestamp precision, an index) is listed. Objects it does not read (views, triggers, policies, functions, standalone sequences) are named in a warning, since a database the migrations create will not have them. It refuses when the branch's journal and the checkout's migrations differ, and it refuses a migration that drops something schema.ts left out, since the deployed app runs that migration on start. Recording takes the migration lock and checks the journal again, and a run interrupted before recording is finished by the next one. If anything fails after schema.ts is written, schema.ts, its backup, database.d.ts and the migrations are put back. Signed-off-by: ditadi <victordperd@gmail.com>
ditadi
force-pushed
the
stack/database-dev-flow/04-db-introspect
branch
from
October 20, 2026 12:23
c3e1172 to
a45e7df
Compare
Contributor
🤖 AppKit PR bot🔬 Run evalsStart an eval for this PR from the evals-monitor app: Go to Evals Monitor → 📦 Try this PR's app templateScaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh run download 41109406030 -R databricks/appkit -n appkit-template-0.89.0-pr.8d862d7-stack-database-dev-flow-04-db-introspect-670 -D appkit-pr-670 \
&& unzip -o "appkit-pr-670/appkit-template-0.89.0-pr.8d862d7-stack-database-dev-flow-04-db-introspect-670.zip" -d "appkit-pr-670" \
&& databricks apps init --template "appkit-pr-670"The template pins |
This was referenced Oct 20, 2026
Contributor
📦 Bundle size reportCompared against
|
| dist | raw | gzip |
|---|---|---|
| JS (runtime) | 1.4 MB (+48 KB) | 499 KB (+15 KB) |
| Type declarations | 470 KB (+2.8 KB) | 171 KB (+1.3 KB) |
| Source maps | 2.7 MB (+93 KB) | 930 KB (+27 KB) |
| Other | 11 KB | 3.7 KB |
| Total | 4.6 MB (+143 KB) | 1.6 MB (+43 KB) |
Per-entry composition (own code — deps external (as shipped))
| Entry | Initial (gz) | Lazy (gz) | Total (gz) | node_modules (min) | Own code (min) |
|---|---|---|---|---|---|
. |
106 KB | 2.5 KB | 108 KB | external | 343 KB |
./beta |
108 KB (+40 B) | 485 B | 108 KB (+40 B) | external | 323 KB (+81 B) |
./beta/database-tooling |
37 KB (+9.2 KB) | 0 B | 37 KB (+9.2 KB) | external | 110 KB (+28 KB) |
./testing |
45 KB | 32 KB (+2 B) | 77 KB (+2 B) | external | 223 KB |
./tsdown |
520 B | 0 B | 520 B | external | 813 B |
./type-generator |
26 KB (+1 B) | 0 B | 26 KB (+1 B) | external | 74 KB |
Chunks:
| Entry | Chunk | Load | Size (gz) |
|---|---|---|---|
. |
index.js |
initial | 101 KB |
. |
utils.js |
initial | 4.6 KB |
. |
remote-tunnel-manager.js |
lazy | 2.5 KB |
./beta |
beta.js |
initial | 88 KB |
./beta |
modular.js |
initial | 6.0 KB |
./beta |
stream-manager.js |
initial | 5.9 KB |
./beta |
service-context.js |
initial | 4.2 KB |
./beta |
databricks.js |
initial | 3.3 KB |
./beta |
client.js |
initial | 647 B |
./beta |
index.js |
initial | 20 B |
./beta |
supervisor-api.js |
lazy | 192 B |
./beta |
databricks.js |
lazy | 178 B |
./beta |
index.js |
lazy | 115 B |
./beta/database-tooling |
index.js |
initial | 37 KB |
./testing |
manifest.js |
initial | 32 KB |
./testing |
index.js |
initial | 10 KB |
./testing |
wide-event-emitter.js |
initial | 2.9 KB |
./testing |
index.js |
lazy | 28 KB |
./testing |
remote-tunnel-manager.js |
lazy | 2.5 KB |
./testing |
utils.js |
lazy | 1.8 KB |
./tsdown |
index.js |
initial | 520 B |
./type-generator |
index.js |
initial | 26 KB |
@databricks/appkit-ui
npm tarball (packed): 397 KB (+4 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).
| dist | raw | gzip |
|---|---|---|
| JS (runtime) | 429 KB | 147 KB |
| Type declarations | 267 KB | 98 KB (+1 B) |
| Source maps | 853 KB | 286 KB |
| CSS | 16 KB | 3.2 KB |
| Total | 1.5 MB | 534 KB (+1 B) |
Per-entry composition (consumer bundle — deps bundled, peerDeps external)
| Entry | Initial (gz) | Lazy (gz) | Total (gz) | node_modules (min) | Own code (min) |
|---|---|---|---|---|---|
./js |
5.3 KB | 49 KB | 55 KB | 208 KB | 14 KB |
./js/beta |
2.1 KB | 0 B | 2.1 KB | 0 B | 5.0 KB |
./react |
433 KB | 49 KB | 481 KB | 1.3 MB | 177 KB |
./react/beta |
4.9 KB | 0 B | 4.9 KB | 0 B | 12 KB |
Chunks:
| Entry | Chunk | Load | Size (gz) |
|---|---|---|---|
./js |
index.js |
initial | 5.2 KB |
./js |
chunk |
initial | 120 B |
./js |
apache-arrow |
lazy | 49 KB |
./js/beta |
beta.js |
initial | 2.1 KB |
./react |
index.js |
initial | 431 KB |
./react |
tslib |
initial | 2.1 KB |
./react |
apache-arrow |
lazy | 49 KB |
./react/beta |
beta.js |
initial | 4.9 KB |
Signed-off-by: ditadi <victordperd@gmail.com> # Conflicts: # packages/appkit/src/database/tooling/migrate.ts
Introspect writes and discards migration files too, so it now holds config/database/.appkit-db.lock for its whole run, like migrate. A second command refuses before the catalog is read and changes nothing. The shared CLI test mocks the beta/database-tooling path the CLI loads. Signed-off-by: ditadi <victordperd@gmail.com>
Introspect replaced schema.ts without reading it, so every .private() column came back public and every catalog foreign key became an API include. It now loads the schema.ts it replaces first. A declared column keeps its privacy, and a column the file does not declare is written .private() and named in a warning. A foreign key the file does not declare refuses the run before anything is written, and so does a schema.ts that does not load. Without a schema.ts the first render is unchanged. Signed-off-by: ditadi <victordperd@gmail.com>
With two catalog foreign keys to one table, the one first in constraint name order took the relation, even when schema.ts declared the other. The refusal then asked for a declaration the builder rejects as ambiguous. Foreign keys schema.ts declares now go first. Signed-off-by: ditadi <victordperd@gmail.com>
The refusals told developers to move schema.ts out of config/database, which discards its .private() declarations and makes the next introspect write every column public. A schema.ts that does not load now says to fix it and rerun. An undeclared foreign key says to declare it with fk() in schema.ts, and names the tables schema.ts does not declare yet. Signed-off-by: ditadi <victordperd@gmail.com>
…ct reads A schema.ts of another schema declares nothing in the one introspect reads, so every column renders private and every foreign key is refused. The privacy warning and the refusal now name the schema schema.ts declares, and say that --schema-name with it refreshes the file. Signed-off-by: ditadi <victordperd@gmail.com>
…it replaces The docstring claimed a refresh never widens the API of the replaced schema.ts. What render does is narrower: columns that file does not declare render private, and foreign keys it does not declare are listed for introspect to refuse. Signed-off-by: ditadi <victordperd@gmail.com>
…file changed The drift introspect reads before rendering, and the check markMigrationsApplied repeats under the lock, both leave out migrations the branch recorded but whose committed file changed since. The introspect test mock now records the real file hash, so an edited file shows up the way it does on a branch. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
A migration the branch recorded, whose committed file changed since, ran in a version the snapshot does not describe, so the migration introspect diffs from that snapshot would be recorded against history the branch never had. The drift now carries these migrations: introspect refuses before reading the catalog, and markMigrationsApplied refuses again under the lock when the edit lands between the two reads. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
…keeps Two foreign keys on the same column reach the render loop one after the other, and the second writes its fk() over the first with no skip. With identity keys the first rewrites the column's base type, so the second is skipped for a type reason that names the wrong cause. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
A column with two foreign keys keeps the first one the render loop reaches, and the other is skipped with that as its reason. Declared keys are sorted first, so a refresh keeps the one schema.ts declares. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
Without --schema-name, introspect reads the default app schema even when the schema.ts it refreshes declares another one, so an app on public has to pass the flag on every run. The explicit flag keeps overriding the declaration. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
… absent An app whose schema.ts declares another schema, public say, is refreshed where its tables live instead of in the default app schema. The explicit flag still overrides the declaration, and a schema.ts that does not load is still refused. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
An enum label that is the empty string renders into schema.ts, which the builder then rejects, so the whole run fails instead of skipping the column. A literal default cast to a quoted type name, 'open'::app."order-status" as Postgres prints it, is dropped. The catalog test shows the shape pg_get_expr returns. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
…trospect An enum whose label is the empty string is skipped with that reason, since the builder rejects it and the whole run failed on the generated schema.ts. The literal-default parser takes the cast as pg_get_expr prints it, with quoted names such as app."order-status", so the default is kept. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
…ce options A DEFERRABLE unique constraint renders as a plain .unique() with nothing in the report, unlike a deferrable foreign key. An identity or serial key whose sequence has other options (START WITH 100, say) renders as an exact id()/bigid(), which drizzle-kit writes with the defaults. The catalog test reads both from Postgres. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
…s approximations The catalog reads DEFERRABLE on unique constraints and the options of the sequence a serial or identity column owns, as text so bigint bounds survive JSON. The render lists a deferrable unique key as declared without DEFERRABLE, as it already does for foreign keys, and a key whose sequence differs from the defaults id()/bigid() declare with the clauses on each side. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
introspect writes schema.ts and schema.ts.bak with writeFileSync, which follows a symbolic link at either path and overwrites whatever it points at. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
…ally A new tooling/files.ts holds refuseSymlink and writeFileAtomic: lstat the destination and refuse a link, then write a sibling temporary file and rename it over, so a crash leaves the old file or the new one. introspect refuses a symlinked schema.ts or schema.ts.bak before it reads anything, and writes and restores both through the helper. migrations.ts drops its own copy of the rename and writes the journal through the same helper. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
The introspect test's journal fake now answers the catalog lookup the way #654's journal check reads it (relkind, owner, trust). Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
AutoCRUD is replaced by named operations, so introspect no longer guards the generated HTTP surface. A foreign key the replaced schema.ts does not declare renders with fk() instead of refusing the run. A column it does not declare is written as the catalog has it, not .private(). A .private() the old file declares is kept. Warnings about NOT NULL columns and dropped defaults name app.database.<table>, which still inserts through schema.ts. The round-trip test asserts the schema itself survives a refresh, with no dependency on the crud capabilities. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
…rivate() in introspect Both policies existed for the generated HTTP routes, which named operations replace. A foreign key the replaced schema.ts does not declare is rendered with fk(), like a declared one; the refusal, the newRelations result and the other-schema hint go. A column the old file does not declare is written as the catalog has it. A .private() the old file declares is kept, so a schema.ts that does not load still refuses the run. The NOT NULL and default warnings name app.database.<table>, which still inserts through schema.ts. The render tests ignore the constraint-name warning the fixture's <column>_fkey names raise. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
…it replaces The help text described the private-by-default columns and the foreign key refusal that introspect no longer has. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: ditadi <victordperd@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR (generated by Stack)
Use this link to review incremental changes.
Stacked on #666. The design is #677.
What did you change, and why?
Apps whose tables already exist in Lakebase need a
schema.tsand a migration history that match them.appkit db introspectreads a Postgres schema (appby default, or--schema-name) frompg_catalogand writesconfig/database/schema.ts, keeping the previous file asschema.ts.bak. It also writes the migration that brings history up to the database, and records it as applied. Without history that migration is a first0000_baseline, which the deployed app adopts on start (#654). With history it is the nextNNNN_introspect. Recording is only true on the branch introspect read, so introspect runs only on your dev or session branch.What the builder cannot express is left out with an
// appkit: skipped …comment, and what introspect does not read at all is named in a warning. Introspect refuses when the branch does not record exactly the committed migrations, when its migration would drop somethingschema.tsleft out, and when it would replaceschema.tswith an empty one. If anything fails after the write, it puts backschema.ts, its backup,database.d.tsand the migrations.This revision keeps what the replaced
schema.tsdeclares:schema.tsit replaces first. A declared column keeps its.private()as it was. Every other column, table and foreign key is written as the catalog has it, foreign keys withfk().schema.tsthat exists but does not load refuses the run before anything is read or written.schema.tsdeclares keeps the relation.Review round, 2026-11-09:
schema.tsdeclares when--schema-nameis absent.schema.ts, its.bakand the migration journal are written throughtooling/files.ts(refuse a symlinked destination, temp file + rename).Fix round for fjakobs's review of #677, 2026-11-15:
.private(), and a foreign key the oldschema.tsdoes not declare is written withfk()instead of refusing the run. The insert warnings nameapp.database.<table>instead of the API, and the round-trip test no longer imports the AutoCRUD capabilities.How do you know it works?
render.test.ts,introspect.test.tsandroundtrip.test.tscover declared privacy, new columns and tables written without.private(), kept, dropped and undeclared foreign keys, aschema.tsthat does not load, a run whilemigrateholds the lock, and each drift and drop refusal. The round trip loads both files through the real loader. A refresh keeps what the old file declares, and adds the new column, table and foreign key as the catalog has them.catalog.pg.test.tsandapply.pg.test.tsrun against Postgres. They read a real schema, and record pending migrations without running them..private().dbtests passed against a local PostgreSQL 14, typecheck passed, and a clean build stayed within the size gate. The full vitest suite passed at an earlier revision of this PR and at the top of the stack (feat(appkit): add appkit db check #678). The packed tarballs export the same root and./betanames as main, and./beta/database-toolingadds onlyrunIntrospect. CI runs the full suite on this PR.Review round, 2026-11-09: each fix has a failing-then-passing test in its PR, and every layer's appkit and shared tests ran against Postgres 14 after the sync. At the top of the stack,
pnpm build,pnpm test(6502 passed, 1 skipped),pnpm -r typecheck,pnpm check,pnpm knipandpnpm docs:buildpass.Fix round for fjakobs's review, 2026-11-15: each fix has a failing-then-passing test in its PR, and every layer's database, plugin and CLI tests ran against Postgres 17 after the sync. A throwaway Lakebase fork confirmed that the developer role can write the service principal's rows but cannot alter its tables. At the top of the stack,
pnpm test(6602 passed, 1 skipped),pnpm -r typecheck,pnpm buildandpnpm docs:buildpass, and oxlint and oxfmt are clean on every changed file. After the AutoCRUD removal, the introspect tests failed first and then passed, and at the top of the stackpnpm test(6601 passed, 1 skipped),pnpm -r typecheck,pnpm buildandpnpm docs:buildpass.