Skip to content

The test and CI design of 2026-11-20: mtest 0.11 in mcpp, the build's caches and records, the test kit, and CI that verifies a tree once (#986, #976, #927, #802, #293) - #984

Open
Sunrisepeak wants to merge 38 commits into
mainfrom
fix/rc-include-dir-globs
Open

Sunrisepeak wants to merge 38 commits into
mainfrom
fix/rc-include-dir-globs

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

The test and CI design of 2026-11-20 (.agents/docs/2026-11-20-test-architecture-upgrade-design.md), implemented: mcpp's side of mtest 0.11 (SPEC-017 v2.5), the build's caches and records, mcpp's own test kit, and CI that verifies a tree once and releases what it tested. Closes #986, #976, #927, #802 and #293.

Engine

  • A package's instance is a function of its configuration, not of its units (A package's instance directory changes with its set of units, so mcpp test compiles every unit again when it only adds units #986); each cache refusal names its reason.
  • A path enters a cache key by its role; a cacheable package's build program runs into the home's build-cache/v1/out, shared by every project (SPEC-004 §9 item 18).
  • The home's unit cache: a compile whose command, compiler and read files hold an earlier compile's contents is placed, and no compiler starts (a fresh checkout of mcpp: 164 s to 4.1 s).
  • [build] memory_threshold and one budget of slots for the machine (MTEST_BUDGET, SPEC-017 O11).
  • [profile.<name>] dependency_debug (a test program depending on mtest: 43.7 MB to 8.3 MB).
  • The build record (MCPP_BUILD_RECORD, builds.ndjson); a run records its machine class, each case's processor time, steps, surface items and limit; a case keeps its scopes' time.
  • Run fixtures (SPEC-017 O6, P11): served once per run, infra when one is not ready.
  • --changed/--since: a header selects the tests built from the units that read it.
  • mcpp emit surface and MCPP_SURFACE_RECORD.
  • A resource script compiles with its package's include_dirs expanded.

Tests

The kit records steps and surface items, shares one home as a run fixture, gives a case that changes a home its own home whose xlings upstreams are the shared and the user's, declares diagnostic lines, and compares contractual outputs with goldens. New invariants (no compile on an unchanged rebuild; a dev-dependency served from the cache), a waits baseline, and the surface ledger and CPU budget in tests/e2e/mtest.toml.

CI

Tiers reach the jobs; a full run that passes leaves verified-<tree>; a push to main of a verified tree registers that run instead of testing again; release-build.yml holds the four release builds, which the full tier runs and whose Linux program the Linux shards test; release.yml promotes a verified run's archives. Unit shards from main's measured work, main's unit cache, the report's sections and mtest gate, a jitter workflow, and a downstream project-test action. docs/92 and the release skill state the new procedure.

Specs, usage chapters (en and zh) and CHANGELOG are updated with the change.

…ded, not as globs

A relative include_dirs entry is a glob; the compile steps expand it (package_root.cpp), but the resource compiler received it as written. On Windows LLVM's tools expand wildcards in their own command line against files, so llvm-windres's preprocessor saw several inputs and refused with "cannot specify -o when generating multiple output files". compat.protobuf's protoc host tool failed so on windows-2025, which blocked gRPC's code generation there. The resources e2e case 837 gains R4: a member's `*/inc` reaches the resource compiler expanded.
…ws_resources stays under the 400-line limit)
…st's are, keeping a literal directory a build step creates later

resource_include_dirs called expand_dir_glob, which drops a literal entry
that does not exist when the build is planned; before mcpp#984 such an entry
was joined as written. It now calls expand_manifest_include_entry, the
expansion the root manifest's compile steps use, which keeps it (#390).
e2e 837 states R5: gui's generated/inc reaches the rc_object edge.
…its units (#986); each cache refusal names its reason

The instance digest hashed each unit's object path, so mcpp test, which
keeps a unit an inactive feature gates, gave the package a second instance
and compiled every unit again. It now hashes the package's distinct unit
configurations and its imports' digests. A package's include directories
have one expansion, expand_manifest_include_entry, for its compile steps,
its private part and its resource script.

serve_or_fill names why a package is not served (no-identity,
unnamed-sources, unnamed-upstream, outside-store, unaddressable-unit, miss)
under --verbose and in the build context. The token found why
mcpplibs.mtest was never cached: its synthesized entry main had no cache
address, which took the whole package out. An entry unit now compiles with
the image that links it, and the package's other units are cached; a
second project's test shows mcpplibs.mtest as Cached.

e2e: 986 (the issue's reproduction), 1037 B (mtest cached in the second
project).
…mory threshold; an archive run selects case parts; SPEC-017 2.5 (#976, #927)
…tems used and limit; estimates read the runs of the machine's class (SPEC-017 P5.6, P12.4)
…rgest resident set; a test run keeps its builds; a step that runs a minute is named (#927)
…eleton's mtest and the launchers' os with them
…ables under dev ([profile.<name>] dependency_debug; #927)
… home, keyed by its contract; key paths are named by role (#986, #927)
…s contents is placed from the home's unit cache, and no compiler runs (#927)
…a BMI's producer key is stated before it is published, and a relocated attempt's BMI is not moved twice
…26-11-20 §5.16)

The surface is each command and flag of the command line's definition
(mcpplibs.cmdline 0.0.5 exports the tree), each key of the manifest's registry
and each reason token, one '<kind> <name>' per line. With MCPP_SURFACE_RECORD,
an invocation appends the command and flags it was given, the registry keys
its manifests state and the reasons it records (mcpp.surface).
A program that declares a fixture (a component event with fixture) serves it
under MTEST_SERVE before the first program whose cases use it; each user gets
its endpoint as MTEST_FIXTURE_<NAME>; the last user's end closes its input.
fixtures.ndjson states each fixture's preparation; programs.ndjson names the
fixtures a program used; a fixture that is not ready leaves its users not run,
'infra' in the store (P11.4).
… (design 2026-11-20 §5.11)

The build directory's records name the files each unit read: ninja's deps log
(format 4, read directly) and the scans' depfiles. When every unit of the
project has a record, a changed header selects the tests built from a unit
that read it, and the tests that run a program of the package when such a
unit is not a test's own; otherwise every test is affected, as before.
… the runner's new clauses

Item 17 states the instance digest as a function of the package's
configuration; 18 the roles of paths in keys and the home OUT_DIR of a
named-source package's build program; 19 [build] memory_threshold; 20
[profile.<name>] dependency_debug. SPEC-017 states mcpp's side of O6 and the
surface mcpp emits and records. self env announces test-steps, test-budget,
test-jitter and test-fixtures, which SPEC-017 names. An invalid memory
threshold names the source that holds instead.
…that read the user's as upstreams, invariants, waits, diagnostic lines, golden outputs (design 2026-11-20 §5.10, §5.16, §5.18, §5.20, §5.21, §6)

- Each mcpp command a case runs records the surface items it used
  (MCPP_SURFACE_RECORD) as the case's exercise event.
- The shared home is the run fixture 'home', filled once per run; every group
  uses it. A case whose assertion holds only while nothing else writes its home
  runs mcpp in a home of its own (own_home) instead of holding the machine.
- Inherited homes, in the kit and in _inherit_toolchain.sh, name the user's
  homes as upstreams instead of linking each payload, and copy no subos
  (mcpp#293); 914 checks that no file the upstream's entries reach is written.
- Invariant cases with bounds: an unchanged rebuild compiles nothing, and a
  dev-dependency is served from the cache to a second project.
- Waits for a time only fall (check_waits.sh, waits.baseline); five that only
  moved a modification time touch the file instead.
- The kit declares mcpp's warning and error lines diagnostic, under report.
- The machine protocol and the surface are compared whole with golden files.
…et, run fixtures, header selection, the home's caches, the build record, emit surface and the records an invocation appends; the e2e rules of the kit; the spread of functions measured in e2e-coverage
…n tested (design 2026-11-20 §5.22, §7.2)

- ci.yml: the run's tree; the tiers reach the jobs (bootstrap, cross, the target matrix, the
  upgrade, openkal, the iOS and MSVC-xlings jobs, the second macOS image and Linux's other
  toolchains run in `full`); a push to main whose tree a run verified registers that run and tests
  nothing; `verdict` judges every job and leaves `verified-<tree>` after a `full` pass;
  `merge_group` is accepted.
- release-build.yml: the four release builds, moved from release.yml. ci.yml's `full` tier runs
  them, and its Linux shards test the program of the Linux archive (use-built-mcpp `form`).
- release.yml: `find` reads the tag's tree; `promote` takes the verifying run's archives, checked
  against their checksums and version; without such a run the four builds run, and the summary
  says which.
- The unit shards come from the measured work of main's last run; build.yml restores main's unit
  cache and, on main, keeps the entries its build used.
- The report's sections and `mtest gate`; jitter.yml (weekly, by hand, and on a flip in main's
  run); the downstream action project-test; main_run.sh resolves main's registered run.
- docs/92 and the mcpp-release skill: the release pull request passes `ci:full` before it is
  merged, main's run is not cancelled, and the release promotes.
…es, measurements, and where it differs from the design
@Sunrisepeak Sunrisepeak changed the title fix: a resource script compiles with its package's include_dirs expanded, not as globs The test and CI design of 2026-11-20: mtest 0.11 in mcpp, the build's caches and records, the test kit, and CI that verifies a tree once (#986, #976, #927, #802, #293) Nov 20, 2026
…ache served (staged); 1122 bounds the dependency's compiles at its entry unit
…s (a restored sandbox keeps an index older than xim-pkgindex c0023dd)
…ft (a run inside another runner's budget ran out its time)
…he BMIs it imports (-fmodule-file-deps), and the files are read after the compile wrote them
… cases that read the old home's payloads, outputs in the project, a BMI count and one entry of gc read the new ones; the cap test runs outside the run's budget

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment