Skip to content

CI: the rules that decide what runs, as a declarative rules file with a TypeBox schema - #132

Open
agorischek wants to merge 4 commits into
mainfrom
agorischek/ci-rules-file
Open

agorischek wants to merge 4 commits into
mainfrom
agorischek/ci-rules-file

Conversation

@agorischek

Copy link
Copy Markdown
Collaborator

Every CI selection decision now lives in .github/ci-rules.json: a declarative rules file with a TypeBox schema and a description on every rule. This is step (a) of the rules-file plan. It moves today's behaviour into the file, with one intentional fix (below).

The rules file

Section Holds
full Events (push, schedule, workflow_dispatch) and labels (ci:full) that run everything
files How each changed file counts, decided by the first rule that matches: full, docs, owner (the workspace or crate containing it) or readers (only what reads it)
tags Path-based tags, such as release and action
groups Named sets of workspaces and crates: globs, @group, {"kind": "crate"}, {"facts": "coverage"}
conditions Named predicates: rust, engine, platform, release, …
jobs When each ci.yml job runs, keyed by job ID; public-product is a matrix keyed by runner
browser When each suite runs, its shards per Playwright project, macOS shards, and narrow for the renderer harness

A when is one of "full", "docsOnly", "reachesSuite", {"if"}, {"changes"}, {"tag"}, {"generator"}, {"changed"} or {"reaches", "from"?}, or an any, all or not of these.

  • Code: what remains is the graph (ci-facts.ts, unchanged), an evaluator (ci-rules.ts) and file classification (ci-files.ts). ci-plan.ts holds no rule.
  • Schema:
    • The source is tools/cli/src/lib/ci-rules.schema.ts (TypeBox; typebox is added to tools/cli).
    • .github/ci-rules.schema.json is generated from it for editors by the new ci-rules-schema generator. It validates under a standard Draft 2020-12 validator.
    • The planner imports only the schema's types, so the changes job still needs no install.
  • Reference checks: the planner refuses rules that name a group, condition, tag or suite that doesn't exist (ci-rules-check.ts).
  • Summary: the run summary now has a Why column built from the rule descriptions.
  • Workflow: ci.yml reads each job's decision from the changes output of the same ID, and passes --event and --labels to the planner.
  • Tests: the existing cases are now a table, tools/cli/test/ci-plan.cases.json. New tests check that:
    • the file matches the schema;
    • the reference checks catch broken references;
    • ci.yml gates each job on the same-named decision;
    • the generated JSON Schema is current.
  • Docs: CONTRIBUTING.md#what-ci-runs now documents the file instead of restating it.

No behaviour change

The old and new planners give identical plans for every tracked file alone and for 400 random pairs, 4,200 change sets in all: same jobs, matrices, browser and macOS jobs, and renderers. There is one exception:

  • action.yml never reached its rule under the old ordered chain, because the reference-docs generator reads it. It now runs macOS public-product and both Windows Node and integration jobs, which CONTRIBUTING already said it did. A table case covers it.

This PR changes .github/**, tools/** and the lockfile, so it runs everything.

Next

  • (b) Markdown inside packages counts as docs; macOS runs only for Rust, action.yml and packages/cli/scripts/**; the renderer harness runs only for renderer, kit or protocol changes.
  • (c) A lockfile change reaches only the workspaces and crates whose resolved closure changed.
  • (d) Rust cache and cargo nextest, with the macOS check split.

… a TypeBox schema

.github/ci-rules.json now holds every decision the CI plan makes: how each
changed file counts (ordered file rules), tags, named groups of workspaces and
crates, named conditions, when each job of ci.yml runs (by job ID), and when
each browser suite runs, how it's sharded and which narrow to the renderers a
change touches. Every rule has a description; the run summary prints them as
the reasons a job runs. The code keeps only the graph (ci-facts.ts) and an
evaluator for the rules' `when` predicates.

The schema is tools/cli/src/lib/ci-rules.schema.ts (TypeBox), with
.github/ci-rules.schema.json generated from it for editors (registered as the
ci-rules-schema generator). The planner imports only its types, so the
`changes` job still needs no install; it refuses rules naming groups,
conditions, tags or suites that don't exist. Tests: the file against the
schema, the reference checks, ci.yml gating each job on the decision of the
same name, and the existing cases as a table (tools/cli/test/ci-plan.cases.json).

No behaviour change: the old and new planners agree on every tracked file
alone and on 400 random pairs, except action.yml, which the old chain never
tagged (the reference-docs generator reads it, so its rule was never reached):
it now runs macOS and Windows, as CONTRIBUTING.md already said.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The tests allow a stale committed CI rules schema to pass checks, and some running jobs lack an explanation in the Why column.

2 open issues
What changed in this PR

Moves CI selection policy into .github/ci-rules.json, keeping the existing dependency graph and separating rule evaluation from configuration.

Changes:

  • Adds declarative rules, a TypeBox schema, an editor-schema generator, and reference validation.
  • Connects workflow jobs to same-named planner outputs and adds explanations to run summaries.
  • Converts planner tests to table-driven cases and fixes platform-job selection for action.yml.
File Description
tools/​scripts/​ci-rules-schema.mjs Generates and checks the editor schema.
tools/​scripts/​ci-plan.mjs Accepts events and labels; emits decisions and explanations.
tools/​cli/​test/​ci-plan.test.ts Tests rules, references, workflow mappings, and plans.
tools/​cli/​test/​ci-plan.cases.json Defines table-driven selection scenarios.
tools/​cli/​src/​lib/​generated.ts Registers the CI schema generator.
tools/​cli/​src/​lib/​ci-rules.ts Loads rules and evaluates predicates.
tools/​cli/​src/​lib/​ci-rules.schema.ts Defines rule types and validation schema.
tools/​cli/​src/​lib/​ci-rules-check.ts Checks named references and browser suites.
tools/​cli/​src/​lib/​ci-plan.ts Builds CI plans from declarative rules.
tools/​cli/​src/​lib/​ci-files.ts Classifies changed files using ordered rules.
tools/​cli/​src/​lib/​ci-browser.ts Uses configured layouts and suite narrowing.
tools/​cli/​package.json Adds TypeBox as a dependency.
package-lock.json Records the tools package’s TypeBox dependency.
CONTRIBUTING.md Documents rule configuration and maintenance.
.oxfmtrc.json Excludes the generated schema from formatting.
.github/​workflows/​ci.yml Consumes same-named job decisions.
.github/​ci-rules.schema.json Provides the generated editor schema.
.github/​ci-rules.json Defines CI selection and browser-sharding policy.
.considered.yaml Excludes the generated schema from file standards checks.

Review effort: Balanced

Comment thread tools/cli/test/ci-plan.test.ts Resolved
Comment thread tools/cli/src/lib/ci-plan.ts Resolved
agorischek and others added 3 commits November 20, 2026 20:06
…reason gives its own description

Copilot review on #132.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ways" predicate

tools/cli/test/ci-plan.cases.json is now {"$schema", "cases"}, checked against
tools/cli/src/lib/ci-plan-cases.schema.ts, with ci-plan.cases.schema.json generated
beside it; tools/scripts/ci-schemas.mjs (was ci-rules-schema.mjs) writes both
schemas. `always` runs a job whatever changed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants