Skip to content

feat(workflow-code): workflows and workspace as code - #10129

Open
waleedlatif1 wants to merge 797 commits into
stagingfrom
feat/workspace-code
Open

waleedlatif1 wants to merge 797 commits into
stagingfrom
feat/workspace-code

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Supersedes #10128: this PR now carries workflows as code and workspace as code together, both already reviewed.

Summary

Workflows as code

  • A workflow can be read as typed TypeScript and applied back: sim workflows source get | create | apply with --dry-run, a generated .sim/AGENTS.md and cookbook, and stable exit codes.
  • @sim/workflow-code: compiler (TypeScript → canonical model), printer, materializer (canonical model → stored graph, keeping block ids through a lock file), diagnostics with file:line frames and hints, size and depth limits.
  • @sim/workflow-sdk: the typed SDK generated from the block catalog (agent, condition, table.queryRows, trigger.schedule, …) with workspace resources by stable key (table('tickets'), credential('slack'), skill('tone'), customTool('lookup')).
  • v2 API: GET/POST /api/v2/workflows/{id}/source, create, the source catalog and resources manifest. Every route goes through an authorized application use case; writes are rate limited, refuse workspace API keys, and answer 409 when the draft changed since the read.
  • Code tab: a read-only TypeScript view of the open workflow that refreshes as the canvas or a collaborator changes it.

Workspace as code

  • A directory describes a workspace — tables/, knowledge/<key>/ (settings + docs/), files/, tools/, skills/, workflows/<key>/ — and sim workspaces source get | apply --dir reads it back or applies it in one run, including a workflow and the table, knowledge base, tool or skill it uses.
  • @sim/workspace-code: one codec per kind, a statically parsed declaration reader (never executed), the base file, limits and diagnostics in the same shape. SDK subpath @sim/workflow-sdk/workspace (defineTable, column.*, defineKnowledgeBase, defineTool).
  • Server: GET/POST /api/v2/workspaces/{id}/source and a knowledge document download route; every write goes through the existing domain use case and its authorization.
  • Safety: a three-way rule (local / base / server) writes only what the directory changed and answers 409 when the server changed it too; nothing deleted on the server comes back; deletes need --allow-delete, show what they lose, re-check the resource right before writing, and are refused while a stored workflow still uses it; binary files and documents travel in bounded, resumable rounds with hashes checked before anything is written; a partial apply converges on re-run.

Changes outside the feature flag

  • References inside a JSON string of a JSON code field (e.g. {"summary": "<agent.content>"} in a Response block or table row data) are now JSON-escaped; any value with a newline or quote used to break the JSON.
  • A buffered knowledge document upload now hands its processing to the dispatcher right after the registering commit (the same backend choice the upload-session path makes), in the background; the outbox row stays the durable fallback.
  • The row-locked recheck before a source write compares edges as a set (the edges read has no ORDER BY), removing a spurious 409.

Everything else is behind the global workflow-code flag (off by default). Migrations 0460/0461 add two tables; additive only.

Type of Change

  • New feature
  • Bug fix

Testing

  • Round-trip corpus over every block type at 100% on all properties; property and hostile-input tests for compile, canonicalize, materialize, print and every workspace codec.
  • Real-Postgres integration suites for workflow source (concurrency and staleness, locks, permissions, credential and env visibility, governance) and for every workspace kind plus a cross-kind suite (stale 409s, delete gates, in-use refusals, partial-failure convergence, blob rounds, permissions, limits).
  • HTTP e2e for workflow source read/apply, the workspace source CLI (get → edit → apply → get), and a two-page Code tab UI e2e.
  • Agent dogfooding: agents built the same backends through code vs. today's CLI — the workflow path with the cookbook used ~42% fewer tokens and finished ~5x faster; the workspace path used ~2.75x fewer calls in half the time, caught a column rename before it shipped, and lost no data on a destructive-instruction trap.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Conflicts: apply-workflow-source imports (core moved emptyDiffSummary to
source-diff; workspace keeps PendingResource), requireConsent (core's verb
argument, still exported for workspace apply), SourceTree (core's files/guide
shape, still exported), digest/readWrittenDigests (core's normalized digest and
readSourceJson, still exported). The workspace get reports the guide instead of
the dropped tsconfig flag.
…nd isBuiltin

A dry run naming a skill the same apply creates or adopts resolved through a
wrapper that dropped the core's new resolver members; it now keeps them and
answers a pending skill's name.
…nd knowledge base keys

workspace.d.ts for a workspace directory now lists the skills/, tools/ and
knowledge/ keys the directory declares beside the stored ones, the way its
tables are typed from tables/, so skill('tone') type-checks before an apply
creates the skill and a misspelled key is still an error.
No kind sets KindAdapter.supported, so E_KIND_UNSUPPORTED could never be
raised; every kind is read and planned.
A tool attached to an agent whose type the caller can no longer place had no
catalog declaration, so the printer could not tell its password params apart
and printed a stored key verbatim through raw(). Source reads, applies and the
per-workflow catalog read now include every attached tool type.
… its key

A reference compiles to <variable.key>, and the resolver matches it against
the stored name normalized. A written name that normalized to another key
saved a variable no reference could reach, so compile now reports
E_VARIABLE_NAME_MISMATCH unless the name normalizes to the key or the key is
its legacy ~n collision form.
…apply creates

Inside a workspace apply, a workflow naming a knowledge base, custom tool or
skill the apply creates resolves to a placeholder id, which lint reported as an
unresolved resource. Those references are now left out of the dry run's lint.
cookbook/workspace holds one resource of each kind (table, knowledge base and
document, workspace file, custom tool and its code, skill) and a workflow that
names them by key, embedded as WORKSPACE_COOKBOOK beside the declaration SDK.
…clean and applies

The real-catalog suite type-checks every cookbook declaration and the workflow
against the SDK and compiles the workflow with no error or warning; the
cross-kind suite applies it to an empty workspace (a clean dry run but the
unset env variable), checks the workflow stores every created resource's id,
and re-applies what a get reads as a no-op.
…vergence

.sim/AGENTS.md for a workspace directory now lays out tables, knowledge bases,
files, tools and skills beside workflows, says what --allow-delete deletes and
what it never does, that exit 8 converges by applying again, that a skill
rename reaches a workflow on its next edit, the size cap and skipped list, and
that secrets appear only by name. The get writes the workspace cookbook under
.sim/examples/workspace/.
…ile does

The apply scan matched codeFile( with a regex, so codeFile ('./x.js') and an
aliased import (codeFile as fileCode) sent no file and a create answered
E_CODE_FILE_MISSING. workflow-code now exports codeFileReferences, which parses
with the compiler's Babel options and binds the SDK's named imports by their
local names; the CLI reads the paths from it.
emptyDiffSummary set noop: true, so a dry run the server refused claimed the
source matched the saved draft, which the contract defines noop to mean.
The source routes joined the workflows document without withRequestBodyErrors,
so applyWorkflowSource and createWorkflowFromSource omitted the 415 the v2 JSON
builder answers for a body it cannot read as JSON.
In the record form the key names the field, but an entry's own name was
copied over it, so { subject: { name: 'other' } } stored a field called
other. Compile now refuses a name that differs from the key, and the
record key always stays the stored name.
…ndex

parseStoredPath read an index from any character after a segment, so
<a[0]x3]> parsed as a[0][3] and a code file rewrote it to another path.
Branch extra was spread over the stored row last, so an extra id, title,
value or key replaced the row's own: the stored id no longer matched the
edge handles and the lock, and a value replaced the condition. Compile now
refuses those fields (and editor state, which canonicalize drops) in extra.
…alias

validateSdkNames checked blocks, methods and triggers by hand and missed the
oauthCredential/triggerCredentials alias the generator adds, so --check
passed on a file the generator would change. It now reports everything
extendSdkNames would add.
The recording value bag trapped get, has and ownKeys but not
getOwnPropertyDescriptor, so a thunk that tested a key with Object.hasOwn or
hasOwnProperty.call looked value-independent and was frozen as a constant.
…toggles

A container resize and a handle toggle copy the edge array without changing an
edge, and the draft subscription compared edges by identity, so each read the
source again; the handle-toggle operations were also missing from the
geometry operations. Edges are now compared by element and both toggle
operations count as geometry.
…essages

A declaration of a million bad values produced a million framed findings
(about 700 MB of heap). The reader and field reporter now stop at 100
findings with one E_LIMIT_EXCEEDED, and every message and hint is cut at
1,000 code units, as the workflow reporter does.
…ile rules

A document named CON.txt, ending in a dot, or over 255 bytes was accepted
though no Windows or macOS disk can hold it, and two documents differing
only in letter case or normalization were both written by a get, so one
overwrote the other. Document names now pass filePathProblem, and a local
case or NFC collision is E_KB_DOC_NAME.
The server drops an empty stored description, so a table declared with
description: '' never matched its stored fingerprint and replanned an
update on every apply. The canonical now folds '' to null, as the
knowledge codec does.
…anner

plan.ts already refuses any kind folder over maxResourcesPerKind, so the
files codec's own copy reported the same problem twice. Also drops a test
that restated the files canonical.
…and never refuse a plan for its infos

- Every new workflow folder gets its id minted at plan time (createWorkflowFromSource
  takes it in-process), so new folders that run each other resolve in any folder
  order, cycles included; creates are written callees first where they form no
  cycle, and a write that evaluates again resolves the ones not written yet.
- A workflow's file() names a file by its resource key: a created file plans
  under the key a get would allocate it and claims it once written, and a
  reference made by id is matched to the file's change, so a workflow naming a
  file the same apply uploads waits for its bytes, and deleting a file an
  edited workflow names is E_PLAN_REF_DELETED.
- Workflow test files (tests/<name>.test.js) are never declared, written or
  deleted as plain files.
- The 1000-finding cap keeps errors, then warnings, then infos, and counts the
  rest with an info, so cutting findings never refuses a plan or reclasses a
  refusal as too large.
- Folders sent whole that print exactly as their stored workflows (at the
  base's fingerprint) are no-ops found with one batched read instead of one
  evaluation each: 50 workflows sent whole plan in 91 queries, not 983.
…ld-back changes

Secret adapter: description-only updates through setSecretUseCase, never a value
(visible values the listing returns are dropped). Credential adapter: binds keys
to this workspace's credential of the declared provider (kept, adopted, or
rebound; ambiguous or missing reported), unbinds only with --allow-delete.
A declared reference the workspace cannot satisfy holds back exactly the writes
that use it (PlannedChange.unavailable/held) while the rest apply; workflow
changes now carry their env names and compiled references even when refused.
The directory walk sends the root files as their kinds, workspace.d.ts types the
keys they declare, apply prints and exits 1 for changes held back, and the agent
guide and rules describe the reference kinds.
…fs, per-kind resourceKind

deployments/<workflow key>.ts declares defineDeployment({ live, api, chat }). live is outside the
fingerprint (read on every apply); a chat password is env.NAME or stored(), never a value.
APPLY_ORDER drops the ResourceKind check for WORKSPACE_KIND_SPECS[kind].resourceKind, which also
decides isResourceKeyed.
Deploys, redeploys (live: 'draft'), reports pinned drafts, writes the public API flag and chat
through the deployment and chat use cases, and undeploys only with --allow-delete, counting what
stops. A plan that edits a deployed workflow no deployment file covers warns.
# Conflicts:
#	apps/sim/lib/workflows/application/apply-workflow-source.ts
#	apps/sim/lib/workflows/application/source-evaluate.ts
#	apps/sim/lib/workspace-source/application/types.ts
…ng it

An edited name or description in workflow source now renames or redescribes the
workflow: the graph diff never sees them, the write applies them through the
workflow update use case after the graph commit (whose fingerprint check the
rename would otherwise move), and the plan refuses a name another workflow in
the folder holds as E_NAME_TAKEN at the header (E_APPLY_META_CHANGED is gone).
The fingerprint covers the name, so a server-side rename since the read stays a
stale 409. A workspace apply plans it as an update of the folder with fields
[name], and the diff summary lists the changed meta fields.

Also documents the workspace plan's refusals, key renames, file keys and the
whole-folder comparison in the workspace-code design.
…INDEX.md and namespace docs

Per-operation .d.ts files are merged into their block file: 10,294 files
(15.5 MB) become 513 (12.6 MB), so a get writes in ~110 ms instead of
~2 s, and tsc over every block drops from 0.49 s / 460 MB RSS to
0.05 s / 257 MB. INDEX.md lists each builder with its block, operation,
file and params type (legacy builders apart) plus the block-type to
builder naming rule; each block namespace's doc comment lists its
operations. index.d.ts leaves legacy-only blocks out, and layout.ts says
which files a directory's tsconfig includes. The TypeScript release a
source directory is checked with is pinned in typecheckCommand.
…tools and subflows

The single cookbook becomes cookbook/examples/: cookbook.ts (unchanged),
integrations.ts (webhook trigger, weekday cron, api headers as rows,
table.updateRowById, gmail_v2 sendEmail, onError) and agents.ts (router,
agent tools: customTool, mcp().tool, slack.tool.sendMessage, a child
workflow; runWorkflow with inputMapping; parallel.collection). Embedded
as COOKBOOK_EXAMPLES; the real-catalog test type-checks and compiles
each with zero errors and warnings.
… without an id

- tsconfig.json includes only the blocks the source uses (scanned from
  workflow.ts plus the server's blockTypes) and the core builders' and
  agent/start blocks: a cookbook workflow checks 75 files / 38 MB RSS
  instead of 10,354 / ~460 MB. tsconfig.full.json includes every current
  block (and any legacy-only block in use) for adding a new one; apply
  refreshes both from the cached catalog.
- sim workflows source check --dir compiles locally with the bundled
  compiler and the cached catalog, no request.
- apply reads the workflow id from .sim/base.json when it is omitted.
- AGENTS guides: the pinned bunx -p typescript@7.0.2 tsc command, the
  has-no-exported-member fix, INDEX.md lookup and the naming rule, and
  re-reading workflow.ts after apply rewrites it.
- Writes the three example workflows into .sim/examples/.
… number the field stores

A switch's value() thunk returned "true"/"false" and a slider's returned "0.7": the
BlockConfig type allowed only strings. Canvas and server creation stored the text, and compile
seeded it for an omitted param, so the apply gate refused a block created from source
(file.read: Invalid switch value "true" for folderIncludeSubfolders) and rewrote sliders.

value() now returns string | number | boolean, and every switch and slider default in the
registry (clickhouse, crowdstrike, datadog, file, huggingface, incidentio, okta, perplexity,
posthog, tailscale, tinybird, tts) returns its stored type. A registry-wide test checks every
seeded default validates unchanged against its subBlock type.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1234 files

Confidence score: 4/5

  • In apps/sim/blocks/blocks/tts.ts, style: 0 is treated as absent, so the ElevenLabs request loses the configured value. Check for blank or null explicitly before converting it.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/sim/blocks/blocks/tts.ts">

<violation number="1" location="apps/sim/blocks/blocks/tts.ts:245">
P2: The numeric zero default is now dropped by the ElevenLabs mapper because `params.style` is checked for truthiness. Preserve zero with an explicit blank/null check before converting it.</violation>
</file>

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

max: 1.0,
step: 0.05,
value: () => '0.0',
value: () => 0.0,

@cubic-dev-ai cubic-dev-ai Bot Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The numeric zero default is now dropped by the ElevenLabs mapper because params.style is checked for truthiness. Preserve zero with an explicit blank/null check before converting it.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At apps/sim/blocks/blocks/tts.ts, line 245:

<comment>The numeric zero default is now dropped by the ElevenLabs mapper because `params.style` is checked for truthiness. Preserve zero with an explicit blank/null check before converting it.</comment>

<file context>
@@ -242,7 +242,7 @@ export const TtsBlock: BlockConfig = {
       max: 1.0,
       step: 0.05,
-      value: () => '0.0',
+      value: () => 0.0,
       dependsOn: ['provider'],
       required: false,
</file context>
Fix with cubic

Comment on lines +271 to +274
} else if (planned.action === 'create') {
detail.deploy = true
if (decl.api?.public === true) detail.publicApi = true
if (decl.chat) detail.chat = 'replace'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Private deployments can stay public. This branch only writes the API flag when public is true. Undeploying a workflow keeps its previous isPublicApi value. If a previously public workflow is undeployed, then deployed from a fresh directory with api: { public: false } (or no api field), this branch leaves that value enabled. Once deployment finishes, callers without credentials can run it. Set the declared private default explicitly before making the workflow live.

How this was verified: Undeploy preserves isPublicApi, this create branch skips writing false, and the execution route admits anonymous calls when isPublicApi and isDeployed are both true.

diagnostics.push(refDeletedFinding(change, ref))
}
}
holdUnavailable(changes)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Skipped edits allow unsafe deletes. checkDeletesInUse runs before holdUnavailable and trusts the references from edited workflows. Suppose a workflow currently uses a table, and the directory removes that reference and the table while adding a declared secret that has no value. The workflow edit is held back, but the table delete still runs with --allow-delete. The saved workflow therefore loses a table it still uses. Determine holds first, then check deletes against the saved references of workflows that will not be written.

catalog,
})
}
const renamed = !dryRun && (await renameAsPlanned(principal, context.workflowId, plan))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Renames bypass the guarded commit. A name- or description-only edit can leave sourceMetaChanged false, so this path skips commitSourceMeta and calls updateWorkflow without another freshness check. A teammate's rename after evaluation is silently overwritten instead of returning 409. When the graph also changes, the rename runs after its transaction commits, so a late name conflict can reject an apply whose graph was already saved. Include these fields in the same row-locked, freshness-checked commit, including for metadata-only edits.

Comment on lines +435 to +437
if (serverIdOf(ctx, decl.url) === current.id) {
await remove(current.id)
return written(change.key, await create())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Failed replacements remove MCP servers. Changing only a URL query string keeps the server ID, so this branch deletes the existing server before creating its replacement. Creation still performs DNS and URL checks and can fail. Deletion has already committed and revoked OAuth tokens, leaving the server gone. A retry with the old base treats it as deleted rather than recreating it. Replace the settings without destroying the working record first, or make the replacement roll back on failure.

This branch was successfully deployed

1 active deployment
Preview — 5fc59b1d Deployed Nov 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant