Skip to content

feat(appkit): verify database callers and authorize named operations - #689

Merged
ditadi merged 26 commits into
mainfrom
stack/database-stable/03-operation-authorization
Nov 20, 2026
Merged

ditadi merged 26 commits into
mainfrom
stack/database-stable/03-operation-authorization

Conversation

@ditadi

@ditadi ditadi commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

What did you change, and why?

Why: An operation must run only for a verified caller that its policy admits, and nothing it touches may reach telemetry.

Change (unregistered runtime):

  • Adds the request pipeline: verify the caller, refuse cross-site requests, route by name, parse the input, run resource and allow once, run execute once, then validate the answer against its output schema. Only a literal true admits, and a throwing lookup or policy denies.
  • In Databricks Apps the caller comes only from x-forwarded-access-token, checked with the workspace Current User Me API on every request. Local development uses the developer's credentials for same-origin loopback requests. The database connection stays the service principal.
  • A write whose answer fails after execute returned answers 500 with outcome: "unknown". Without an output schema, undefined answers 204, null on GET, PATCH, or PUT answers 404, and any other value answers 500 without being sent. Spans drop operation input from URLs, and everything below the operation span runs with tracing suppressed.
  • An omitted input schema accepts no parameters and gives authorization/execute undefined. Absent input and an empty object are accepted; non-empty objects and other supplied values are rejected before policies run. No ID, create, update, or replacement input is inferred from the method.
  • Explicit input failures include bounded, schema-declared field paths and standard validation messages. Raw Zod/custom messages, submitted values, record keys, and unknown property names never reach the response. Output failures stay opaque.
  • Safe to deploy alone: nothing registers the handler, so no request reaches it.

Interface

The HTTP contract this handler implements. The last PR of this stack mounts it at /api/database/operations/<name>.

GET /api/database/operations/notes.list?input=%7B%22board_id%22%3A7%7D
x-forwarded-access-token: <user token, set by Databricks Apps>

HTTP/1.1 200 OK
Cache-Control: no-store
x-appkit-database-actor: <hex SHA-256 of the workspace host, NUL, and the actor id>

[{ "id": 12, "author": "8412…", "body": "Ship it", "created_at": "…", "note_events": [] }]
PATCH /api/database/operations/notes.update
Content-Type: application/json
x-forwarded-access-token: <user token>

{ "id": 12, "body": "Shipped" }
  • A GET may omit input, or carry one input query parameter holding JSON in at most 8 KiB of query string. A write may omit its body, or carry one JSON body of at most 1 MiB, and accepts no query string. An explicit input schema decides whether an omitted value is valid.
  • execute receives { actor, input, db, appkit, resource }, never the request, its headers, or the token. A policy's resource lookup receives the same context without resource.
Outcome Status Body
Success 200, or 201 for POST; 204 when the operation has no output or answers nothing the answer, or no body
execute answers null on GET, PATCH, or PUT, and no output accepts null 404 { "error": "…" }
execute returns a value but the operation has no output 500 { "error": "…" }
No verified caller (checked before anything else) 401 { "error": "…" }
The workspace identity check cannot answer 503 { "error": "…" }
A cross-site browser request 403 { "error": "…" }
An unknown name, or another method 404 { "error": "…" }
Bad input 400 { "error": "…", "details": [{ "path": ["input"], "message": "This operation accepts no parameters" }] }
A body over 1 MiB, or not JSON 413 or 415 { "error": "…" }
authorize returns false or throws 403, or 404 for a resource policy { "error": "…" }
authorize returns { deny } 403 for "forbidden", 404 for "not-found" { "error": "…" }
A resource policy finds nothing, or its lookup throws 404 { "error": "…" }
A database error in execute its category: 409, 422, 503, or 500 { "error": "…" }
A write whose answer fails after execute returned 500 { "error": "Database write may have completed", "outcome": "unknown" }

How do you know it works?

Testing: All 30 HTTP runtime tests and 5 safe-validation formatter tests pass. They cover omitted inputs for every method, unexpected input rejected before authorization, explicit schema parsing, authentication order, policies, transactions, explicit output projection, replies without an output schema, unknown write outcomes, telemetry redaction, nested field paths, array indices, and redaction of custom issue text and unknown keys. Package typecheck passes. The runtime is exercised through real HTTP with fake workspace and database edges.

Stack

#683 → #687 → #688 → #689 → #690 → #691 → #692

Design: #683. Previous layer: #688. Next layer: #690.

Review feedback: refactor(appkit): answer operations without an output schema with no body replaces the default row projection with the reply rules above.

Review feedback (round 2): fix(appkit): check the origin first and verify each database caller once per token.

  • A cross-origin request answers 403 before any workspace call.
  • Each forwarded token is verified once per 30 s, keyed by its SHA-256 digest. Concurrent requests share one check, and refusals and failures are never cached. A user removed from the workspace can therefore keep access for up to 30 s.
  • The resolver reads the raw SCIM /Me document, so a service principal is refused by applicationId as well as by schema, through one predicate shared with dev-obo.
  • Design 002 states the new order and the reuse.

Restack: @opentelemetry/core is pinned to 2.12.0 to match main's OpenTelemetry upgrade. test(appkit): filter operation spans under the stable HTTP conventions follows the new HTTP span attributes; url.query stays REDACTED. The shared retry wait is fixed separately in #738.


Stack created with GitHub Stacks CLI • Give Feedback 💬

@ditadi
ditadi added this pull request to stack #693 October 23, 2026 15:01
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

@databricks/appkit

npm tarball (packed): 1.3 MB (+3.6 KB) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 1.4 MB (+3.9 KB) 491 KB (+1.7 KB)
Type declarations 481 KB (+525 B) 173 KB (+149 B)
Source maps 2.7 MB (+6.2 KB) 913 KB (+2.3 KB)
Other 11 KB 3.7 KB
Total 4.5 MB (+11 KB) 1.5 MB (+4.1 KB)
Per-entry composition (own code — deps external (as shipped))
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
. 114 KB (+289 B) 2.4 KB 117 KB (+289 B) external 366 KB (+938 B)
./beta 115 KB (+381 B) 484 B 115 KB (+381 B) external 341 KB (+936 B)
./testing 47 KB (+266 B) 34 KB (+32 B) 81 KB (+298 B) external 234 KB (+938 B)
./tsdown 520 B 0 B 520 B external 813 B
./type-generator 27 KB 0 B 27 KB external 79 KB

Chunks:

Entry Chunk Load Size (gz)
. index.js initial 110 KB
. utils.js initial 4.6 KB
. remote-tunnel-manager.js lazy 2.4 KB
./beta beta.js initial 95 KB
./beta modular.js initial 6.1 KB
./beta stream-manager.js initial 5.9 KB
./beta service-context.js initial 4.3 KB
./beta databricks.js initial 3.3 KB
./beta client.js initial 647 B
./beta index.js initial 20 B
./beta supervisor-api.js lazy 191 B
./beta databricks.js lazy 178 B
./beta index.js lazy 115 B
./testing manifest.js initial 33 KB
./testing index.js initial 10 KB
./testing wide-event-emitter.js initial 2.9 KB
./testing index.js lazy 30 KB
./testing remote-tunnel-manager.js lazy 2.4 KB
./testing utils.js lazy 1.8 KB
./tsdown index.js initial 520 B
./type-generator index.js initial 27 KB

@databricks/appkit-ui

npm tarball (packed): 406 KB (+47 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 438 KB (+25 B) 150 KB (+13 B)
Type declarations 271 KB (+20 B) 100 KB (+12 B)
Source maps 865 KB (+64 B) 291 KB (+21 B)
CSS 16 KB 3.2 KB
Total 1.6 MB (+109 B) 545 KB (+46 B)
Per-entry composition (consumer bundle — deps bundled, peerDeps external)
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
./js 5.3 KB 49 KB 55 KB 208 KB 14 KB
./js/beta 2.1 KB (+15 B) 0 B 2.1 KB (+15 B) 0 B 5.0 KB (+22 B)
./react 441 KB 49 KB 489 KB 1.4 MB 179 KB
./react/beta 14 KB (+16 B) 0 B 14 KB (+16 B) 25 KB 15 KB (+22 B)

Chunks:

Entry Chunk Load Size (gz)
./js index.js initial 5.2 KB
./js chunk initial 120 B
./js apache-arrow lazy 49 KB
./js/beta beta.js initial 2.1 KB
./react index.js initial 439 KB
./react tslib initial 2.1 KB
./react apache-arrow lazy 49 KB
./react/beta beta.js initial 14 KB

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 44749778051 -R databricks/appkit -n appkit-template-0.98.0-pr.3b06c2e-stack-database-stable-03-operation-authorization-689 -D appkit-pr-689 \
  && unzip -o "appkit-pr-689/appkit-template-0.98.0-pr.3b06c2e-stack-database-stable-03-operation-authorization-689.zip" -d "appkit-pr-689" \
  && databricks apps init --template "appkit-pr-689"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

@ditadi
ditadi force-pushed the stack/database-stable/03-operation-authorization branch from 41f4918 to 4df6588 Compare October 26, 2026 10:41
@ditadi
ditadi marked this pull request as ready for review October 28, 2026 11:05
@ditadi
ditadi requested a review from a team as a code owner October 28, 2026 11:05
@ditadi
ditadi requested a review from pkosiec October 28, 2026 11:05
@ditadi
ditadi force-pushed the stack/database-stable/03-operation-authorization branch from 4df6588 to 55a3fad Compare November 3, 2026 13:01
@ditadi
ditadi requested review from MarioCadenas and removed request for pkosiec November 3, 2026 13:54
@ditadi
ditadi removed this pull request from stack #693 November 3, 2026 15:18
@ditadi
ditadi force-pushed the stack/database-stable/03-operation-authorization branch from 9f01322 to 9e9a969 Compare November 3, 2026 15:18
@ditadi
ditadi added this pull request to stack #704 November 3, 2026 15:19
@ditadi
ditadi force-pushed the stack/database-stable/03-operation-authorization branch from 9e9a969 to 7cbf040 Compare November 4, 2026 19:15
@ditadi
ditadi force-pushed the stack/database-stable/03-operation-authorization branch from 7cbf040 to 4321d7a Compare November 4, 2026 20:06
@ditadi
ditadi force-pushed the stack/database-stable/03-operation-authorization branch from 4321d7a to f60f1d8 Compare November 6, 2026 10:49
@ditadi
ditadi force-pushed the stack/database-stable/03-operation-authorization branch from f60f1d8 to 08cb4a7 Compare November 16, 2026 12:03
@ditadi
ditadi force-pushed the stack/database-stable/03-operation-authorization branch from 08cb4a7 to 77bcf91 Compare November 16, 2026 15:26
ditadi and others added 26 commits November 20, 2026 14:00
Add the operation request pipeline: verify the caller with the

workspace, refuse cross-site requests, parse the input, run the

policy once and execute once, then validate and project the answer.

The pipeline is not registered yet, so no request reaches it.

Default outputs answer timestamps in RFC 3339, and span redaction

matches the routed path.

Signed-off-by: ditadi <victordperd@gmail.com>
Signed-off-by: ditadi <victordperd@gmail.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…body

The runtime no longer compiles tables or projects a default row. Without
an output schema, undefined answers 204, null on GET, PATCH, and PUT
answers 404, and any other value is a fault, which stays an unknown outcome
after a write. The handler no longer takes the schema.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
Main's instrumentation-http upgrade names the server by server.port and
the request by url.path and url.query instead of http.host and
http.target.

[#tern:skip]

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…nce per token

A cross-origin request now answers 403 before any workspace call. The Apps
boundary verifies a token once per 30 s, shares concurrent checks, and reads
the raw SCIM document so a service principal is refused by application id too.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
An answer over the response limit kept PAYLOAD_TOO_LARGE and answered
413, and a validation error raised by the output schema answered 422.
Design 002 reserves both for the request. Unusable output is the
server's failure: a read answers the opaque 500 and a write keeps its
unknown outcome.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A GET whose output is 2,500 small rows answers 500 today: the output
shares the 10,000-node budget meant for untrusted input, though its
151 KB body is about 3% of the 5 MiB response limit.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…put node limit

An operation's answer counted nodes against the 10,000 limit for untrusted
input, so 2,500 small rows answered 500. Answers now take a node budget
derived from the 5 MiB byte limit: every JSON value takes at least two
bytes with its separator, so any answer that fits the byte limit fits the
budget. Input keeps 10,000, and depth, cycle, and finite-number checks
are unchanged.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
An answer that failed its output schema, was not bounded JSON, exceeded
the response limit, or made its output schema throw answered 500 with no
server log. Each now logs one warning that names the operation and the
reason, never the value or any error message. A throw the classifier
cannot name keeps its single classifier line.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…r fetch metadata

A request with neither Sec-Fetch-Site nor Origin passed the origin check
whatever its method. Every browser sends Origin on a write, so only an
old browser or a non-browser client omits both. Such a write now answers
403 before any workspace call; a GET with neither header still runs. The
local boundary keeps accepting the loopback page's own writes.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
Any throw from an authorization callback was a refusal, so a serialization
failure or a cancelled statement inside a resource lookup answered 404 and
one inside allow answered 403. A transient database failure now fails the
request with the retryable 503 and execute never runs. Every other throw
stays a refusal with its warning.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A 401 or 403 from the workspace identity check answered 401 with no log,
so an app whose user authorization lacks a scope failed silently. A 403
now logs a warning that names the refusal and hints at a missing scope;
a 401, usually an expired token, logs at debug. Neither carries the token
or any header value.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A dropped connection reaches a policy as an INTERNAL database error, so it
still read as 404 or 403 after the transient fix. Fail the request with 500
instead, so a client never confuses an outage with a missing or forbidden
record. Any other throw inside a policy stays a refusal.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A resource policy answers 404 for a missing record before `allow` runs. A
database failure inside `allow` for a found record answers 503 or 500, so
the status tells a caller the record exists. Both must answer the same 404.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A database outage inside `allow` for a found record answered 503 or 500,
while a missing record answered 404 before `allow` ran, so the status
revealed that the record exists. That outage now refuses with the same 404
and the refusal warning. An outage in the resource lookup, or in a policy
with no resource, still fails the request.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
An output schema that throws now raises `UnusableOutput` like every other
answer the server cannot send, so the runtime logs one warning per failure
by checking that error alone, without relying on which throws the database
classifier logs. Its `reason` becomes `logReason`, since it is logged and
never answered.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…e limit

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
The test client now adds the page origin only to writes, through a named
`browserHeaders` helper, so a HEAD without Origin or fetch metadata must
pass the origin check and answer routing's 404.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A HEAD without Origin or fetch metadata was refused as a foreign write.
It reads like a GET, so it now reaches routing, which answers 404 since no
operation declares HEAD. A navigation with `Sec-Fetch-Site: none` still
admits GET alone. The origin check's doc no longer claims that a read with
neither header never comes from a browser page; an old browser sends
neither on a cross-origin no-cors GET.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A database outage inside `allow` for a found record answers 503 for a
transient failure and 500 for an internal one, with the standard bodies,
and execute never runs. A missing record still answers 404 before
`allow` runs. The status can reveal that the record exists during an
outage; the guide documents that trade-off.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A database failure inside `allow` for a found record now answers 503
or 500 like an outage anywhere else in a policy, so execute never runs
and the client sees the failure instead of a refusal. This drops the
refusal path 90a93a5 added; a policy whose database work depends on
whether a record exists can reveal it during an outage, and the guide
documents that.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
An output refinement or transform that throws a not-found
`DatabasePluginError` answers 404 with the standard body on a read and
on a write, never the unknown-outcome 500, and logs no unusable-answer
warning. f456346 wrapped every output schema throw as unusable output
and lost that.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
f456346 wrapped every throw from the output schema's parse as unusable
output, so a refinement or transform that throws a not-found
`DatabasePluginError` answered 500, or the unknown-outcome 500 on a
write. That not-found now passes through unchanged and answers 404, as
it did before. Every other throw is still wrapped and logged once.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…throws not found

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…hrows not found

Output parsing runs after execute returned, so a write's changes may have committed.
A not-found thrown by the output schema now stays a 404 only on a GET. On any other
method it is wrapped as unusable output, logged once, and answers the unknown outcome,
so a caller does not retry a write that may have landed. A null that execute returns
on a nullable method still answers 404.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
@ditadi
ditadi force-pushed the stack/database-stable/03-operation-authorization branch from a92d4d4 to 02fef39 Compare November 20, 2026 14:03
@ditadi
ditadi merged commit 5b2d4ad into main Nov 20, 2026
12 checks passed
@ditadi
ditadi deleted the stack/database-stable/03-operation-authorization branch November 20, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants