Skip to content

scripts: build rbgo every run instead of reusing a /tmp binary - #968

Merged
tannevaled merged 1 commit into
mainfrom
rbgo-built-every-run
Nov 21, 2026
Merged

tannevaled merged 1 commit into
mainfrom
rbgo-built-every-run

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Summary

Closes #964.

When RBGO was unset, five harnesses used a fixed /tmp binary and reused whatever was already there.

script old default
scripts/conformance/rubyspec/run.sh /tmp/rbgo-rubyspec, built only if absent
scripts/conformance/rspec/run.sh /tmp/rbgo, built only if absent
scripts/conformance/apps/run.sh any existing /tmp/rbgo, else /tmp/rbgo-apps
scripts/oracle.sh any existing /tmp/rbgo, else /tmp/rbgo-oracle
bench/modules/run.sh /tmp/rbgo, never built

On a machine with several checkouts, a sweep or a benchmark could run another commit's interpreter. The output looked the same as after a fresh build.

Each script now builds ./cmd/rbgo from its own checkout into a mktemp -d directory and removes that directory on exit. An explicit RBGO is still used as given. rubyspec/run.sh and rspec/run.sh also print the binary's vcs.revision and vcs.modified, read with go version -m.

A cleanup leak in rubyspec/run.sh

rubyspec/run.sh left files behind on every run, for two reasons:

  • Its second trap replaced the first and did not name the rbgo directory.
  • The final exec "$RATCHET" replaced the shell, so no EXIT trap ran.

The judge now runs as a child process followed by exit $?. Its exit codes reach the caller unchanged: 0 ok, 1 regression, 2 unreadable input. A single trap now removes the results file, the ratchet directory and the rbgo directory.

Docs

The cmd/phantom recipe in scripts/conformance/rubyspec/README.md pointed -rbgo at /tmp/rbgo-rubyspec, which no longer exists after a run. The recipe now builds its own binary first.

Test plan

  • gotest -count=1 . -run "TestASpecCannotDictateItsOwnRecordedPassCount|TestAHandRunStillGetsItsTallyOnStdout|TestShellCodeHasSeesCodeAndNotComments|TestTheRatchetReadsTheTallyFromItsOwnFile": all four ran and passed (27 s).
  • bash -n / sh -n on all five scripts: clean.
  • scripts/oracle.sh -e 'p [1, 2, 3].sum' with RBGO unset: it built its own binary, and rbgo, MRI, JRuby and TruffleRuby all printed 6 (agree).
  • The full ratchet runs in CI (rubyspec lane).

🤖 Generated with Claude Code

Five harnesses defaulted RBGO to a fixed /tmp path and reused whatever
was already there: rubyspec/run.sh (/tmp/rbgo-rubyspec), rspec/run.sh,
apps/run.sh and oracle.sh (/tmp/rbgo, which rspec/run.sh writes from any
checkout), and bench/modules/run.sh (/tmp/rbgo, never built at all). On a
machine with several checkouts, a sweep could measure another commit's
interpreter and print the same line as a fresh build.

With RBGO unset, each now builds ./cmd/rbgo from its own checkout into a
mktemp -d directory and removes it on EXIT. An explicit RBGO is still
honoured. rubyspec/run.sh and rspec/run.sh print the binary's
vcs.revision/vcs.modified, read from the binary itself.

rubyspec/run.sh also leaked files on every run. Its second trap replaced
the first without naming the rbgo directory, and the final `exec` of the
judge skipped EXIT traps altogether. The judge now runs as a child
process followed by `exit $?`, and one trap removes RESULTS, the ratchet
directory and the rbgo directory.

The README's cmd/phantom recipe pointed -rbgo at /tmp/rbgo-rubyspec,
which no longer exists after a run. It now builds its own binary.

Closes #964

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

run.sh measures whatever binary is already at /tmp/rbgo-rubyspec: it builds rbgo only when none is there

1 participant