Skip to content

feat(core-sync): share the SDK core-sync scripts and workflows - #3

Draft
pimlock wants to merge 5 commits into
ai-openshell:mainfrom
pimlock:feat/core-sync/pimlock
Draft

pimlock wants to merge 5 commits into
ai-openshell:mainfrom
pimlock:feat/core-sync/pimlock

Conversation

@pimlock

@pimlock pimlock commented Nov 19, 2026 •

Copy link
Copy Markdown

Summary

Draft to get a sense of sharing the SDK core-sync automation. sdk-python and sdk-typescript each carry a copy of the same scripts; this moves them here with reusable workflows, and leaves only the language-specific steps in each SDK as mise tasks.

What's here

  • core-sync/: the scripts both SDKs copied from sdk-python (core tags, bump PRs, drift check and issue, release tagging), with the repository-specific bits taken out:
    • the SDK repository comes from SDK_REPO, GITHUB_REPOSITORY, or the clone;
    • generated paths come from OPENSHELL_GENERATED_PATHS in the SDK's mise.toml;
    • the drift check exports the protos with the SDK's proto:fetch at the pin and at core main and compares the exports with proto:buf, instead of hardcoding proto roots and how to run buf;
    • after regenerating, it runs check before test, so type errors in bindings from core main fail it (the gap found in ci: core sync automation and npm releases with trusted publishing sdk-typescript#4);
    • uncovered RPCs come from an optional core:uncovered-rpcs task.
  • Reusable workflows core-watch.yml, core-drift.yml, and release-tag.yml (the tag job of an SDK's release). They read the App secrets from the caller's sdk-bot environment, so callers pass no secrets.
  • core-sync/README.md: the contract an SDK implements, and how to call it.

One pin

An SDK pins one commit of this repository, in its workflows' uses: lines, and both CI and local runs use that commit's scripts:

  • CI: the reusable workflows load core-sync/ as an action with uses: $/core-sync. Self-repository syntax resolves $/ to this repository at the commit being run, and the action sets CORE_SYNC_DIR. I checked this with a probe called from sdk-typescript: $/probe-action resolved to pimlock/workflows/probe-action@<the pinned sha>, with the rest of the repository at that commit next to it.
  • Locally: the SDK keeps a copy of core-sync/core-sync.sh as scripts/core-sync.sh, and its core:* tasks run through it. It uses CORE_SYNC_DIR when set; otherwise it reads the commit from the uses: line in .github/workflows/core-drift.yml and downloads that commit's core-sync/ once into ~/.cache/openshell-core-sync/<sha>.

Callers: ai-openshell/sdk-typescript#7 and ai-openshell/sdk-python#53 (both point at this fork until it merges).

Security

These scripts and workflows run in SDK repositories with the bot App's secrets (contents, pull request and issue write), so:

  • .github/CODEOWNERS sends core-sync/ and .github/ to @ai-openshell/infra. Without it, main-integrity accepts any one approver here, which is weaker than the SDKs' own rule for these scripts.
  • scripts/core-sync.sh --check-pin, run in SDK CI from the SDK's own copy (not from the pinned commit), fails when the shared workflows are called at different refs or at a commit that isn't on this repository's main. A fork-only commit is reachable through this repository's name, so a pin bump could otherwise point at one. I tested it: a commit on main passes; a commit from the fork addressed as ai-openshell/workflows@<sha> fails with compare status ahead. The same check runs before a local download (CORE_SYNC_ALLOW_UNMERGED=1 skips it, for trying unmerged commits).
  • core-watch checks out with persist-credentials: false, git push authenticates through gh auth setup-git, and open-bump-pr.sh regenerates with GH_TOKEN removed, so the SDK's dependency install scripts (npm postinstall, uv) can't read the token. Before, the token was in .git/config and the environment during npm ci.
  • The workflows that hold the token pin mise (2026.11.6) instead of taking the newest release.

Updates

An SDK gets a change here only when its pin moves. Nothing changes under an SDK on its own, which is intended: this code runs with the bot App's secrets.

  • Dependabot updates reusable-workflow uses: lines, but for a SHA pin it proposes the commit of a newer release tag; it doesn't follow untagged commits on main. This repository has no tags yet, so it would need them (for example v1.2.0 for the whole repository, or per component such as core-sync/v1.2.0; whether Dependabot reads a prefixed tag as a version needs checking).
  • The SDK's Dependabot config needs two changes for this: a group so the three refs move in one PR (ci(core-sync): use the shared core-sync scripts and workflows sdk-python#53 adds it, ci(core-sync): use the shared core-sync scripts and workflows sdk-typescript#7 doesn't yet), and ai-openshell/workflows* in cooldown.exclude, or the 7-day cooldown delays every update.
  • Dependabot doesn't update the SDK's copy of scripts/core-sync.sh. Local runs follow the pin anyway, since the shim reads it from the workflows, but changes to the shim itself (such as --check-pin) have to be copied by hand or by the repo template. The shim should stay small and rarely change.
  • Without tags it's manual: a pin-bump PR in each SDK, three uses: lines to the same main commit. A core-sync:update-pin task could do that edit and run --check-pin.
  • A workflow here could open the pin-bump PRs in every SDK on each merge to main, but it needs an App with write access to all SDK repositories, which is a wider grant than anything here has today.

Alternatives

Approach For Against
Copy into each SDK (today) Each SDK reviews and owns the code it runs; no cross-repository trust Copies drift and fixes land twice (the check gap in ai-openshell/sdk-typescript#4 was in one copy)
Composite actions only; each SDK writes its own workflows No $/, so actionlint works; callers see the triggers and permissions Token handling and hardening (environment, persist-credentials, dropping GH_TOKEN) repeated in every SDK, and that is the part most worth sharing
Versioned CLI (Go binary or gh extension) released from here Real tests and argument parsing instead of bash; no mise needed; one versioned artifact for CI and local A build and release pipeline to maintain; still needs workflows calling it; Dependabot doesn't track a tool version in mise.toml (Renovate does)
File sync from here (repo template, or a workflow here that opens PRs copying the files into each SDK) Each SDK reviews the actual code change in its own PR; no remote code fetched at run time A write-capable App across all SDK repositories; copies can be edited in place and diverge; overlaps with pimlock/openshell-repo-template
Core pushes instead of SDKs polling (repository_dispatch from the core release workflow) Bump PRs right after a release Replaces only the trigger of core-watch; needs a token from NVIDIA/OpenShell into ai-openshell; complements the others rather than replacing them
git submodule of this repository Dependabot's gitsubmodule ecosystem follows a branch, no tags needed Reusable workflows can't come from a submodule, so it covers only the scripts and adds a second pin

Open questions

  • mise. The contract assumes mise: the pin is OPENSHELL_CORE_REF in mise.toml, which the scripts read and rewrite with sed; the SDK's steps are mise tasks (proto:gen, check, test, ...); and the workflows install mise. If we want to drop mise from the SDKs, the contract needs a neutral form, for example the pin in its own file (.openshell-core-ref) and the steps as executables under scripts/. The contract is small (two env values, seven tasks), so the change is mechanical, but every SDK moves at once.
  • Tags and Dependabot (see Updates).
  • actionlint 1.7.12 doesn't parse uses: $/… yet; zizmor does. Only this repository's workflows use it, and this repository doesn't run actionlint.
  • $/ needs runner 2.336.0 or newer, which GitHub-hosted runners have.
  • The repo template (pimlock/openshell-repo-template, change note 0004) describes the same contract and would point here instead of at sdk-python. scripts/core-sync.sh is a good candidate for a template-managed file.

Testing

From both SDK branches, with the scripts downloaded through scripts/core-sync.sh: mise run core:drift passes against core main 90e34ad (buf breaking, regenerate + check + test, uncovered RPCs), core:tags and core:bump --dry-run work, and release:tag --dry-run works. shellcheck and zizmor pass. The reusable workflows haven't run end to end yet; that needs the bot environment, so it will happen when an SDK merges its caller.

Move the core pin automation that sdk-python and sdk-typescript each
carry a copy of into one place: the core-sync scripts (core tags, bump
pull requests, the drift check and issue, release tagging) and reusable
core-watch and core-drift workflows.

SDK repositories pin the scripts as mise remote tasks and provide the
language-specific parts as tasks: proto:fetch, proto:gen, proto:buf,
check, test and optionally core:uncovered-rpcs. The drift check now runs
check before test, so a type error in bindings from core main fails it.

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
An inline v2 config makes the repository root the buf module, so buf
refuses proto/ as the input. A v1 config applies to the directory.

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
The called jobs read the App secrets from the caller's sdk-bot
environment, so secrets: inherit isn't needed.

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
The reusable workflows load core-sync/ as an action with `uses: $/core-sync`,
which resolves to this repository at the commit the caller pinned, and the
action sets CORE_SYNC_DIR. SDKs keep a copy of core-sync.sh, which runs
the scripts from CORE_SYNC_DIR in CI and, locally, downloads the commit
that .github/workflows/core-drift.yml calls. The workflow pin is then the
only one; mise remote tasks with their own ref are no longer needed.

Add release-tag.yml so the release workflow's tag job is shared too.

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
- CODEOWNERS: core-sync/ and .github/ need an infra review, since SDKs
  run them with their bot App's secrets.
- core-sync.sh --check-pin: SDK CI fails when the shared workflows are
  called at different refs or at a commit that isn't on main, which
  catches a pin to a fork-only commit addressed through this repository's
  name. The same check runs before a local download.
- core-watch keeps the App token out of .git/config and drops GH_TOKEN
  while regenerating, so dependency install scripts can't read it; git
  push authenticates through gh.
- Pin mise in the workflows that hold the token.

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant