Repository navigation
test(ci): measure the slow tier with #806's CI changes alone - #812
Closed
earakely-scale wants to merge 7 commits into
Closed
earakely-scale wants to merge 7 commits into
earakely-scale wants to merge 7 commits into
Conversation
…ipe's partition, with no base disk made A computer's disk keeps only its base disk's EFI system partition and size. With no published base disk, as on every run after the base's sources change until a release publishes them, the composer made a whole base disk first: the base's root filesystem written into a 16 GB disk and exported, then converted back to raw so the composed disk could cut out its 64 MiB partition. That took 4 to over 10 minutes on CI's runners, enough to time out the slow tier's composing fixtures. The composed disk now takes the recipe's own partition (its uki stage) at the base's size, which is what the made base disk held; the disk's key is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… does, its containers alone Each computer fixture composed every form of its base: the container and the disk of each arch, the native ones probed by gate 3, though only the launcher's tests boot a fixture's disk image. Building and booting a disk added about 2 minutes to each of the slow tier's fixtures. They now compose as `agent-env run` does for local Docker (Composing.for_run), so a test that deploys the disk form composes it first, as a run's deploy would; the launcher's tests (and the Modal disk tests) take desktop_with_disk, which composes the disk too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ach, and refresh the durations table Each slow-tier shard ran a file's tests together, so the three longest files, chromium_app_test (about 15 minutes on main), browser_multi_env_test (14) and browser_computer_test (14), each held a shard to at least that long whatever the rest weighed. The durations table can now name a file in split_in, which cuts it into that many runs of neighbouring tests, each charged the setup of the file's module fixtures that its runner does anew, and these three are cut in two. With main's latest green run's times, rewritten into the table by slow_test_durations.py from all of its JUnit reports, the slowest shard of twelve goes from 14.8 to 11.9 minutes of tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… boots it alone A computer app env that names a computer and a driver, as agentenv's chromium does, was composed alone whenever a bundle wrote it, though most runs deploy it only on a desktop, which composes it into its own image. A run's compose now leaves it to the first deploy that boots it alone (compose_unstarted), as it leaves a form no deploy boots; a register, with or without a --sandbox, still composes it, since others deploy what it writes. In CI, composing chromium alone took 2 to 10 minutes in each shard whose fixtures used it only as a desktop's app. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pytest-xdist's loadfile hands a worker the next file as it frees, in collected order, so a shard whose longest file was collected last ran it last: shard 10 of a run held 20 minutes of tests, about 10 for each of its two workers, and took 13.4, its 7-minute run_bundle_claude_code_test starting late. The shard hook now orders a shard's files by the seconds the durations table gives them, heaviest first, each file's tests together in their collected order, which packs the two workers as evenly as the files allow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale
added a commit
that referenced
this pull request
Nov 19, 2026
…uns of the new fixtures The durations table was main's, whose fixtures composed disks and every app alone, and where a file that reused a computer another file on its shard had composed looked short: browser_clock_test weighed 2 minutes and took 8 on a shard where it composed chromium alone first. The table is now each unit's longer time over two runs of these changes (#812's 44504100677 and 44506657938), each written by slow_test_durations.py from all of a run's JUnit reports, so a file is weighed as it costs where it composes its own computers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uns of the new fixtures The durations table was main's, whose fixtures composed disks and every app alone, and where a file that reused a computer another file on its shard had composed looked short: browser_clock_test weighed 2 minutes and took 8 on a shard where it composed chromium alone first. The table is now each unit's longer time over two runs of these changes (#812's 44504100677 and 44506657938), each written by slow_test_durations.py from all of a run's JUnit reports, so a file is weighed as it costs where it composes its own computers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A computer app that names a computer and a driver, as agentenv's chromium does, was composed alone by every fixture whose bundle put it on a desktop, though those fixtures deploy it only on the desktop, which composes it into its own image: 2 to 10 minutes of each such shard. While a fixture writes its bundle, composer.compose now composes no computer app alone but the fixture's own (env_id, as the browser computer fixture's is chromium), or every one with disks; a test that deploys another alone has its deploy compose it first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measurement only: not for review or merge. I'll close it once CI has run.
This is
mainplus #806's three CI commits and nothing else:agent-env rundoes for local Docker;split_incuts the three longest slow-tier files in two, and the durations table is refreshed frommain's last green run.#806 also changes the ubuntu-24.04 base, so its CI builds the base cold in every shard. This branch leaves the base unchanged, so its slow tier runs as
main's would with these changes. It measures them.🤖 Generated with Claude Code