Repository navigation
feat: the cache collects the work directories of plans that describe - #977
Open
NopeCharon wants to merge 2 commits into
Open
NopeCharon wants to merge 2 commits into
NopeCharon wants to merge 2 commits into
Conversation
`mcpp emit build-database`, `mcpp pack --list-formats` and the pin check of `mcpp publish` plan a project without writing into it, each in a directory of the home: `cache/build-database/<key>`, `cache/pack-formats/<key>` and `cache/publish-plan/<key>`, one per project. Nothing removed one. `cache gc`, `prune` and `clean` read only the build cache, so the directory of a project that no longer exists stayed for ever: one home held 626 of them, 3.4 GB, and another 1.9 GB under pack-formats (mcpp#969). The three commands now take their directory from one function, mcpp::bmi_cache::plan_work_dir. It creates the directory and records each use on its `.mcpp-plan` record: the record is written with the project root when the directory is new, and otherwise its time is set by mcpp::home::mark_used, the same rule the build cache's entries follow. The cache's maintenance walks these directories as entries of kind `work`: - `mcpp cache list` shows each one, named by the command and the project its record names. - The collection after a build, `cache prune --older-than`, `cache gc --older-than` and `gc --max-size` collect a directory by its record's time. A directory written before the record is dated by its own time. - `mcpp cache clean --work` empties the three areas, and `--all` now includes them. A bare `cache clean` still drops package entries only. Removing a directory costs the next plan of that project only writing it again (SPEC-005 R2.1 already calls the directory a cache). e2e 1119 seeds the three areas in a fresh home and checks the listing, the collection by age and by size, and the clean flags. e2e 921 checks that `emit build-database` records the project in its work directory. docs/09 and docs/04 describe the store and the flag, in English and Chinese. Closes mcpp-community#969.
…e return The product writes and reads .mcpp-plan in binary; the e2e case seeded it in text mode, so on Windows the project label ended in a carriage return.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
mcpp emit build-database,mcpp pack --list-formatsand the pin check ofmcpp publishplan a project without writing into it. Each plans in a directory of the home, one per project:cache/build-database/<key>,cache/pack-formats/<key>andcache/publish-plan/<key>. Nothing removed these directories.mcpp cache gc,pruneandcleanread only the build cache, so the directory of a project that no longer exists stayed for ever. The issue measured 626 of them, 3.4 GB, in one home, and another 1.9 GB underpack-formats.The three commands now get their directory from one function,
mcpp::bmi_cache::plan_work_dir. It creates the directory and records each use on a.mcpp-planrecord. A new directory gets the record with the project root in it. For an existing one,mcpp::home::mark_usedsets the record's time, the same rule the build cache's entries follow.The cache maintenance walks these directories as entries of kind
work:mcpp cache listshows each one, named by the command and the project its record names, e.g.emit build-database (work directory of /home/me/demo).[cache] unused_days),cache prune --older-than,cache gc --older-thanandgc --max-sizecollect a directory by its record's time. A directory written by an earlier mcpp, without a record, is dated by its own time.mcpp cache clean --workempties the three areas, and--allnow includes them. A barecache cleanstill drops package entries only.Removing a directory costs the next plan of that project only writing it again. SPEC-005 R2.1 already calls the directory a cache that can be deleted at any time.
docs/09(store table and a paragraph) anddocs/04(thecache cleansynopsis) are updated in English and Chinese.Closes #969.
Criteria
e2e
cache::entries/1119 the work directories of plans are cache entriesis new and needs no toolchain. It seeds the three areas in a fresh home: one directory used 3 days ago, one used now, and one from before the record, 3 days old. It checks:workrows ofcache list --jsonand their labels;prune --older-than 1dandgc --older-than 1dremove the old directories and keep the fresh one;gc --max-size 1Bremoves the fresh one;cleankeeps all three, andclean --workandclean --allremove them.On a Linux aarch64 host I built mcpp with this branch (released 2026.11.19.1 as the bootstrap) and ran the case with the fresh binary as
MCPP_E2E_MCPP: it passes. With the released 2026.11.19.1 as the binary under test it fails at its first check (cache list does not list the three work directories,"entries": []).e2e
cdb::sets/921now also checks thatemit build-databasewrites the.mcpp-planrecord naming the project. It passes on the same host. I did not run it against the released binary. That binary writes no record, so the check cannot pass there.On the same host,
cache::entries/172,174,184,959and1029andcdb::sets/840and913pass with the fresh binary.cache::entries/212fails there with and without this change: it expects GCC'sstd.gcm, and this host's default toolchain producesstd.pcm. That case is left to CI's GCC hosts.I checked by hand with the fresh binary in a new project.
emit build-databaseandpack --list-formatseach created their directory with the record.cache listshowed both asworkrows.cache clean --workremoved them (Cleaned work directories of plans (142.8 KiB)).The Windows and macOS runs of 1119 are left to CI. The case uses only file-system calls that
1029already uses on those hosts.Intersections
cdb::sets/921: its host lock under the work directory is not rewritten by a repeated emit, and the record names the projectcache listentriescache list --jsonkeeps its{root, entries}shape and each entry's keyscache::entries/1119reads theworkrows through the same keys174assertsgc --max-sizecounts work directoriescache::entries/1029(gc --max-size 0keeps the std module)cache clean --work,--allincludes itcache cleandrops package entries onlycache::entries/1119(barecleankeeps the work directories),174Compatibility
cache listshows newworkrows. Directories that existing homes already hold are listed and, by their own time, collected by the first build that runs the daily collection after 30 days unused, or by an explicitgc/prune. A removed directory is recreated by the next plan, which then re-runs that project's build programs once, as on a first run. The paths and keys of the work directories are unchanged.cache clean --allnow also removes them. The gc summary still names its figure "package and tool entries", and that figure now includes the work directories, as it already included patched source trees.Checks before merging
bash .github/tools/check_docs_style.sh,check_docs_structure.shandcheck_version_pins.shpass.python3 .github/tools/check_workflow_assertions.pypasses (a step asserts what its name says).git log origin/main..HEAD -i --grep='Co-Authored-By'prints nothing.compose one from the branch's commits. Suggested subject:
feat: the cache collects the work directories of plans that describe.