From b3cdbf9e466f49ec2a2ae1030d72d964e15d8ee8 Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Sat, 10 Oct 2026 05:58:24 +0000 Subject: [PATCH] ci(rbe): auto-PR a worker-env pin sync when gascity main re-pins (mc-gv9fk) (#7455) rbe-west's pool workers advertise gastownhall/gascity main's worker-env pin; when gascity re-pins (a Go/dolt/OS bump, e.g. #7387) beads' copy falls out of step and every remote-execution run stops in bazel.yml's preflight until a human notices and copies the fix by hand (#7419 took ~100 minutes). This adds a scheduled workflow (every 15 minutes, plus workflow_dispatch) that runs tools/rbe/worker-env-sync against gascity main, and on a real mismatch (exit 1) copies the manifest and pin with the new tools/rbe/worker-env-sync-copy, opens/updates a PR on a fixed branch with status/needs-review-auto, and arms auto-merge. GitHub never runs checks on a PR a plain GITHUB_TOKEN opens, so this needs a GitHub App token -- but gastownhall-autofix (contents:write on this repo) cannot open or label a PR. Two Apps split the two write roles instead: gastownhall-autofix mints push-token (contents:write only) and does the git push of bot/worker-env-sync via plain git; gascity-autofix (pull-requests:write only) mints pr-token and does everything PR-shaped -- create/update, the label, and arming auto-merge -- via gh. peter-evans/create-pull-request takes one token for the whole flow, so it can't honor this split; plain git and gh replace it. Every ${{ }} is resolved into a step's env: first, never written inside a run: script. Both Apps' id/private-key secrets live in the same autofix environment, gated by its deployment-branch policy to main; schedule and workflow_dispatch are never fork events, so neither key is ever readable by a fork. Needs admin setup: the gascity-autofix App (id 5259233) installed on gastownhall/beads (and gascity) with pull-requests:write and metadata read, and its GASCITY_AUTOFIX_APP_ID / GASCITY_AUTOFIX_APP_PRIVATE_KEY secrets added to the autofix environment alongside the existing AUTOFIX_APP_ID / AUTOFIX_APP_PRIVATE_KEY. Co-authored-by: Claude Sonnet 5 --- .github/workflows/worker-env-autosync.yml | 196 ++++++++++++++++ scripts/BUILD.bazel | 1 + scripts/worker_env_autosync_test.go | 259 ++++++++++++++++++++++ tools/rbe/worker-env-sync-copy | 41 ++++ 4 files changed, 497 insertions(+) create mode 100644 .github/workflows/worker-env-autosync.yml create mode 100644 scripts/worker_env_autosync_test.go create mode 100755 tools/rbe/worker-env-sync-copy diff --git a/.github/workflows/worker-env-autosync.yml b/.github/workflows/worker-env-autosync.yml new file mode 100644 index 000000000..2978a5617 --- /dev/null +++ b/.github/workflows/worker-env-autosync.yml @@ -0,0 +1,196 @@ +# Keep beads' rbe worker-env pin in step with gastownhall/gascity main +# without the ~100-minute wait #7419 fixed by hand. +# +# rbe-west's OSS pool is gascity's: its workers advertise the sha256 of the +# toolchain manifest gascity's tools/rbe/worker-env measures, and the +# schedulers match //platforms:rbe_worker's worker-env exec property +# exactly. beads' tools/rbe/worker-env.txt and the platforms/BUILD.bazel pin +# are a byte-for-byte copy of gascity main's; when gascity moves the pin (a +# Go, dolt, or OS/library bump -- #7387 was a Go security release), every +# beads remote-execution run stops in bazel.yml's rbe-job preflight +# (tools/rbe/worker-env-sync) until the copy catches up. nightly.yml's +# rbe-worker-env-sync job already opens a ci/infra issue for a human to fix; +# this instead opens the fix itself, every 15 minutes, so the gap is +# minutes, not until someone reads the issue. +# +# tools/rbe/worker-env-sync only detects the mismatch (exit 1) or that it +# could not compare with gascity (exit 2, e.g. GitHub is briefly +# unreachable; the next run retries). The write half, tools/rbe/worker-env- +# sync-copy, runs only after that exit-1 detection, on a checkout already in +# hand -- so a transient gascity outage can turn "in step" into "unknown", +# never into an unwanted write. +# +# TWO TOKENS, ONE JOB EACH: GitHub never runs checks on a PR a plain +# GITHUB_TOKEN opens (its loop-prevention rule), so this needs a GitHub App +# token to open the PR. The gastownhall-autofix App (contents:write only on +# this repo) mints push-token and does the git push of the fixed branch +# (bot/worker-env-sync); it cannot open, label or merge a PR. The separate +# gascity-autofix App (pull-requests:write only, no contents) mints pr-token +# and does everything PR-shaped: create/update, the status/needs-review-auto +# label, and arming auto-merge. Splitting them means a bug in the PR-shaped +# half can never push a commit, and a bug in the push half can never touch a +# PR; neither token can do the other's job even by accident. Both Apps' +# credentials are secrets of the same `autofix` environment. +# +# peter-evans/create-pull-request takes one token for the whole push+PR +# flow, so it cannot honor this split; this uses plain git for the push and +# gh for the PR instead. Every `${{ }}` is resolved into a step's `env:`, +# never written inside a `run:` script, so a value that happened to look +# like shell (a title or body containing `` ` `` or `$`) is data to the +# shell, not something GitHub's expression syntax or the shell re-parses. +# +# SECURITY: schedule and workflow_dispatch are never fork events (no +# pull_request trigger here at all), so neither App's private key -- secrets +# of the `autofix` environment, whose deployment-branch policy allows only +# main -- is ever readable by a fork. The guard on github.repository is +# defense in depth for a fork of this repo, where the secrets would not +# exist anyway. Both mint steps run only once worker-env-sync has already +# found a real mismatch, so a run that is in step or can't compare asks +# neither App for a token. gh pr merge --auto only arms GitHub's native +# auto-merge (it still waits for every required check and the branch +# protection rule); it does not merge anything itself. +name: rbe worker-env autosync + +on: + schedule: + - cron: "*/15 * * * *" + workflow_dispatch: + +permissions: + contents: read + +jobs: + autosync: + name: Sync worker-env pin with gascity main + if: github.repository == 'gastownhall/beads' + runs-on: ubuntu-latest + timeout-minutes: 10 + # Grants the job read access to this environment's secrets (both Apps' + # id and private key); its deployment-branch policy restricts that to + # jobs running on main, which schedule and workflow_dispatch on this + # repo always are. + environment: autofix + permissions: + contents: read + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6 + with: + persist-credentials: false + + # Never fails the step itself: rc is the signal every later step + # gates on (1, a real mismatch; 2, gascity unreachable -- the next + # scheduled run retries; 0, in step). A run every 15 minutes failing + # red for however long a mismatch takes to review and merge would be + # noise, not signal. + - name: Compare the worker-env manifest and pin with gascity's main + id: sync + run: | + rc=0 + tools/rbe/worker-env-sync main || rc=$? + echo "rc=$rc" >> "$GITHUB_OUTPUT" + + - name: Copy gascity main's worker-env manifest and pin + if: steps.sync.outputs.rc == '1' + run: tools/rbe/worker-env-sync-copy main + + - name: Confirm only the worker-env files moved + if: steps.sync.outputs.rc == '1' + run: | + changed="$(git diff --name-only)" + untracked="$(git ls-files --others --exclude-standard)" + want=$'platforms/BUILD.bazel\ntools/rbe/worker-env.txt' + if [ -n "$untracked" ] || [ "$(printf '%s\n' "$changed" | sort)" != "$(printf '%s\n' "$want" | sort)" ]; then + echo "::error::worker-env-sync-copy touched more than platforms/BUILD.bazel and tools/rbe/worker-env.txt:" + printf '%s\n' "$changed" "$untracked" + exit 1 + fi + + # contents:write only, on this repo alone: all it can do is push + # bot/worker-env-sync. Minted only once a real mismatch needs a push. + - name: Mint push token (gastownhall-autofix) + id: push-token + if: steps.sync.outputs.rc == '1' + uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3 + with: + app-id: ${{ secrets.AUTOFIX_APP_ID }} + private-key: ${{ secrets.AUTOFIX_APP_PRIVATE_KEY }} + owner: gastownhall + repositories: beads + permission-contents: write + + # pull-requests:write only, on this repo alone: all it can do is + # open, label and arm auto-merge on a PR -- never push a commit. + - name: Mint PR token (gascity-autofix) + id: pr-token + if: steps.sync.outputs.rc == '1' + uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3 + with: + app-id: ${{ secrets.GASCITY_AUTOFIX_APP_ID }} + private-key: ${{ secrets.GASCITY_AUTOFIX_APP_PRIVATE_KEY }} + owner: gastownhall + repositories: beads + permission-pull-requests: write + + # push-token alone; it cannot see a PR exists, let alone touch one. + # Leased to the branch: a force-with-lease scoped to refs/heads/ + # bot/worker-env-sync only, so a push can never move any other ref. + - name: Commit and push the sync branch + if: steps.sync.outputs.rc == '1' + env: + PUSH_TOKEN: ${{ steps.push-token.outputs.token }} + REPO: ${{ github.repository }} + BRANCH: bot/worker-env-sync + COMMIT_SUBJECT: "build(rbe): sync worker-env pin with gascity main" + GIT_AUTHOR_NAME: gastownhall-autofix[bot] + GIT_AUTHOR_EMAIL: gastownhall-autofix[bot]@users.noreply.github.com + GIT_COMMITTER_NAME: gastownhall-autofix[bot] + GIT_COMMITTER_EMAIL: gastownhall-autofix[bot]@users.noreply.github.com + run: | + git checkout -B "$BRANCH" + git add platforms/BUILD.bazel tools/rbe/worker-env.txt + git commit -m "$COMMIT_SUBJECT" + git push "https://x-access-token:${PUSH_TOKEN}@github.com/${REPO}.git" \ + "HEAD:refs/heads/$BRANCH" "--force-with-lease=refs/heads/$BRANCH" + + # pr-token alone; it cannot push a commit, only act on the PR the + # step above already pushed. + - name: Open or update the sync PR + id: pr + if: steps.sync.outputs.rc == '1' + env: + GH_TOKEN: ${{ steps.pr-token.outputs.token }} + REPO: ${{ github.repository }} + BRANCH: bot/worker-env-sync + TITLE: "build(rbe): sync worker-env pin with gascity main" + LABEL: status/needs-review-auto + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + body="$(cat <> "$GITHUB_OUTPUT" + + - name: Enable auto-merge + if: steps.sync.outputs.rc == '1' && steps.pr.outputs.number + env: + GH_TOKEN: ${{ steps.pr-token.outputs.token }} + REPO: ${{ github.repository }} + PR_NUMBER: ${{ steps.pr.outputs.number }} + run: gh pr merge --auto --squash --repo "$REPO" "$PR_NUMBER" diff --git a/scripts/BUILD.bazel b/scripts/BUILD.bazel index 08cf03e48..b3d3294a1 100644 --- a/scripts/BUILD.bazel +++ b/scripts/BUILD.bazel @@ -61,6 +61,7 @@ go_test( "pr_risk_bazel_coverage_test.go", "rbe_worker_env_test.go", "repo_helpers_test.go", + "worker_env_autosync_test.go", ], # keep data = [ "//:repo_other_files", diff --git a/scripts/worker_env_autosync_test.go b/scripts/worker_env_autosync_test.go new file mode 100644 index 000000000..f3a4871f7 --- /dev/null +++ b/scripts/worker_env_autosync_test.go @@ -0,0 +1,259 @@ +package scripts_test + +import ( + "os" + "path/filepath" + "reflect" + "strings" + "testing" + + "gopkg.in/yaml.v3" +) + +// worker-env-autosync.yml closes the #7419 gap: nightly.yml's rbe-worker- +// env-sync job only opens a ci/infra issue for a human to fix a gascity +// re-pin (#7387's ~100-minute stall); this instead runs tools/rbe/worker- +// env-sync every 15 minutes and, on a real mismatch (exit 1), copies +// gascity main's manifest and pin (tools/rbe/worker-env-sync-copy) and +// opens or updates a fixed-branch PR with auto-merge armed. +const ( + workerEnvAutosyncWorkflow = "worker-env-autosync.yml" + workerEnvAutosyncJob = "autosync" + workerEnvAutosyncBranch = "bot/worker-env-sync" + workerEnvSyncCopy = "tools/rbe/worker-env-sync-copy" +) + +// TestWorkerEnvAutosyncWorkflowTriggersOnScheduleAndDispatchOnly: the App +// private key this job mints is a secret of the autofix environment +// (deployment-branch policy: main only), and the only events that can ever +// run on main without a fork in the loop are schedule and workflow_dispatch +// -- pull_request (or pull_request_target) here would hand a fork a path to +// it. +func TestWorkerEnvAutosyncWorkflowTriggersOnScheduleAndDispatchOnly(t *testing.T) { + var parsed struct { + On map[string]any `yaml:"on"` + } + text := readPolicyFile(t, sourceRepoRoot(t), ".github/workflows/"+workerEnvAutosyncWorkflow) + if err := yaml.Unmarshal([]byte(text), &parsed); err != nil { + t.Fatal(err) + } + if got := len(parsed.On); got != 2 { + t.Fatalf("%s on: has %d triggers (%v), want exactly schedule and workflow_dispatch", workerEnvAutosyncWorkflow, got, parsed.On) + } + if _, ok := parsed.On["workflow_dispatch"]; !ok { + t.Errorf("%s has no workflow_dispatch trigger", workerEnvAutosyncWorkflow) + } + schedule, ok := parsed.On["schedule"].([]any) + if !ok || len(schedule) != 1 { + t.Fatalf("%s schedule = %+v, want exactly one cron entry", workerEnvAutosyncWorkflow, parsed.On["schedule"]) + } + cron, _ := schedule[0].(map[string]any) + if cron["cron"] != "*/15 * * * *" { + t.Errorf("%s schedule cron = %v, want */15 * * * * (every 15 minutes)", workerEnvAutosyncWorkflow, cron["cron"]) + } + for _, forbidden := range []string{"pull_request", "pull_request_target", "push", "merge_group", "workflow_run"} { + if _, ok := parsed.On[forbidden]; ok { + t.Errorf("%s must never run on %s (forks or an unprivileged push could reach the autofix App token)", workerEnvAutosyncWorkflow, forbidden) + } + } +} + +// TestWorkerEnvAutosyncWorkflowPermissionsAreMinimal: least privilege. +// Top-level and the job's own grant both stay read-only: every write this +// job makes (the push, the PR, the label, auto-merge) goes through one of +// the two minted App tokens, never the job's own GITHUB_TOKEN permissions, +// so there is nothing here for either App token's role to leak into. +func TestWorkerEnvAutosyncWorkflowPermissionsAreMinimal(t *testing.T) { + var top struct { + Permissions map[string]string `yaml:"permissions"` + } + text := readPolicyFile(t, sourceRepoRoot(t), ".github/workflows/"+workerEnvAutosyncWorkflow) + if err := yaml.Unmarshal([]byte(text), &top); err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(top.Permissions, map[string]string{"contents": "read"}) { + t.Errorf("%s top-level permissions = %v, want contents: read", workerEnvAutosyncWorkflow, top.Permissions) + } + + job := readCIWorkflow(t, workerEnvAutosyncWorkflow).job(t, workerEnvAutosyncJob) + wantJobPerms := map[string]any{"contents": "read"} + if !reflect.DeepEqual(job.Permissions, wantJobPerms) { + t.Errorf("%s job %s permissions = %v, want %v (contents:read alone; both write roles come from the minted App tokens)", workerEnvAutosyncWorkflow, workerEnvAutosyncJob, job.Permissions, wantJobPerms) + } + if job.RunsOn != "ubuntu-latest" { + t.Errorf("%s job %s runs-on = %q, want ubuntu-latest", workerEnvAutosyncWorkflow, workerEnvAutosyncJob, job.RunsOn) + } + if job.If != "github.repository == 'gastownhall/beads'" { + t.Errorf("%s job %s if = %q, want the fork guard github.repository == 'gastownhall/beads'", workerEnvAutosyncWorkflow, workerEnvAutosyncJob, job.If) + } + if job.Environment.Name != "autofix" { + t.Errorf("%s job %s environment = %q, want autofix (both Apps' id/private-key secrets live there, gated to main)", workerEnvAutosyncWorkflow, workerEnvAutosyncJob, job.Environment.Name) + } +} + +// TestWorkerEnvAutosyncWorkflowShape: the job compares with gascity main, +// gates every write step on a real mismatch (rc == 1, never rc == 2's +// "unknown" or rc == 0's "in step"), mints two App tokens only once a +// mismatch needs a write, pushes the fixed branch with push-token alone, +// opens/labels the PR and arms auto-merge with pr-token alone (so it is +// not a GITHUB_TOKEN PR - those never trigger CI), and arms auto-merge +// rather than merging directly. +func TestWorkerEnvAutosyncWorkflowShape(t *testing.T) { + job := readCIWorkflow(t, workerEnvAutosyncWorkflow).job(t, workerEnvAutosyncJob) + + sync := job.step(t, "Compare the worker-env manifest and pin with gascity's main") + if sync.ID != "sync" || !strings.Contains(sync.Run, "tools/rbe/worker-env-sync main") { + t.Errorf("sync step: id %q, run %q; want id sync running tools/rbe/worker-env-sync main", sync.ID, sync.Run) + } + + const mismatchIf = "steps.sync.outputs.rc == '1'" + for _, name := range []string{ + "Copy gascity main's worker-env manifest and pin", + "Confirm only the worker-env files moved", + "Mint push token (gastownhall-autofix)", + "Mint PR token (gascity-autofix)", + "Commit and push the sync branch", + "Open or update the sync PR", + } { + step := job.step(t, name) + if step.If != mismatchIf { + t.Errorf("step %q if = %q, want %q (a write step must run only on a real mismatch)", name, step.If, mismatchIf) + } + } + + copyStep := job.step(t, "Copy gascity main's worker-env manifest and pin") + if !strings.Contains(copyStep.Run, workerEnvSyncCopy+" main") { + t.Errorf("copy step run %q, want it to call %s main", copyStep.Run, workerEnvSyncCopy) + } + + pushMint := job.step(t, "Mint push token (gastownhall-autofix)") + if pushMint.ID != "push-token" || actionFamily(pushMint.Uses) != "actions/create-github-app-token" || + pushMint.With["app-id"] != "${{ secrets.AUTOFIX_APP_ID }}" || pushMint.With["private-key"] != "${{ secrets.AUTOFIX_APP_PRIVATE_KEY }}" || + pushMint.With["permission-contents"] != "write" || pushMint.With["permission-pull-requests"] != "" { + t.Errorf("push-token mint = %+v, want the gastownhall-autofix mint (AUTOFIX_APP_ID/AUTOFIX_APP_PRIVATE_KEY, contents:write only)", pushMint) + } + + prMint := job.step(t, "Mint PR token (gascity-autofix)") + if prMint.ID != "pr-token" || actionFamily(prMint.Uses) != "actions/create-github-app-token" || + prMint.With["app-id"] != "${{ secrets.GASCITY_AUTOFIX_APP_ID }}" || prMint.With["private-key"] != "${{ secrets.GASCITY_AUTOFIX_APP_PRIVATE_KEY }}" || + prMint.With["permission-pull-requests"] != "write" || prMint.With["permission-contents"] != "" { + t.Errorf("pr-token mint = %+v, want the gascity-autofix mint (GASCITY_AUTOFIX_APP_ID/GASCITY_AUTOFIX_APP_PRIVATE_KEY, pull-requests:write only)", prMint) + } + if pushMint.Uses != prMint.Uses { + t.Errorf("push-token and pr-token mint different action pins: %q vs %q, want the same SHA-pinned actions/create-github-app-token", pushMint.Uses, prMint.Uses) + } + + push := job.step(t, "Commit and push the sync branch") + if push.Uses != "" || !strings.Contains(push.Run, "git push") { + t.Errorf("push step = %+v, want a plain git push (no action; push-token and pr-token cannot be split through one action's single token input)", push) + } + if push.Env["PUSH_TOKEN"] != "${{ steps.push-token.outputs.token }}" { + t.Errorf("push step env PUSH_TOKEN = %q, want the minted push-token", push.Env["PUSH_TOKEN"]) + } + if _, leaked := push.Env["GH_TOKEN"]; leaked { + t.Errorf("push step sets GH_TOKEN; the push step must never see pr-token") + } + if !strings.Contains(push.Run, workerEnvAutosyncBranch) && push.Env["BRANCH"] != workerEnvAutosyncBranch { + t.Errorf("push step does not reference the fixed branch %q", workerEnvAutosyncBranch) + } + if !strings.Contains(push.Run, "force-with-lease=refs/heads/$BRANCH") { + t.Errorf("push step run %q, want a --force-with-lease scoped to the fixed branch's ref alone", push.Run) + } + + pr := job.step(t, "Open or update the sync PR") + if pr.Uses != "" { + t.Fatalf("PR step uses %q, want plain gh (no action; the token split rules out peter-evans/create-pull-request, which takes one token)", pr.Uses) + } + if pr.Env["GH_TOKEN"] != "${{ steps.pr-token.outputs.token }}" { + t.Errorf("PR step env GH_TOKEN = %q, want the minted pr-token", pr.Env["GH_TOKEN"]) + } + if _, leaked := pr.Env["PUSH_TOKEN"]; leaked { + t.Errorf("PR step sets PUSH_TOKEN; the PR step must never see push-token") + } + if pr.Env["BRANCH"] != workerEnvAutosyncBranch { + t.Errorf("PR step env BRANCH = %q, want the fixed branch %q (so a later run updates the same PR instead of opening a new one)", pr.Env["BRANCH"], workerEnvAutosyncBranch) + } + if !strings.Contains(pr.Run, "gh pr create") || !strings.Contains(pr.Run, "gh pr edit") || !strings.Contains(pr.Run, `--add-label "$LABEL"`) { + t.Errorf("PR step run %q, want it to create-or-update the PR and add the label via gh", pr.Run) + } + if pr.Env["LABEL"] != "status/needs-review-auto" { + t.Errorf("PR step env LABEL = %q, want status/needs-review-auto (routes it through the automated review workflow)", pr.Env["LABEL"]) + } + + merge := job.step(t, "Enable auto-merge") + if merge.If != mismatchIf+" && steps.pr.outputs.number" { + t.Errorf("merge step if = %q, want gated on a real mismatch and a PR having been opened", merge.If) + } + if merge.Uses != "" || !strings.Contains(merge.Run, "gh pr merge --auto") { + t.Errorf("merge step = %+v, want a gh pr merge --auto call (arms auto-merge; never merges outright)", merge) + } + if merge.Env["GH_TOKEN"] != "${{ steps.pr-token.outputs.token }}" { + t.Errorf("merge step env GH_TOKEN = %q, want pr-token (enabling auto-merge is PR-shaped, not a push)", merge.Env["GH_TOKEN"]) + } +} + +// TestWorkerEnvAutosyncWorkflowNeverInterpolatesExpressionsInRun: every +// `${{ }}` GitHub would otherwise substitute directly into a shell script +// is instead assigned to a step's `env:` first, so a value that happens to +// contain shell metacharacters (a PR title, a run URL) is data the shell +// sees through a variable, never text GitHub splices into the script +// before bash ever runs it. +func TestWorkerEnvAutosyncWorkflowNeverInterpolatesExpressionsInRun(t *testing.T) { + job := readCIWorkflow(t, workerEnvAutosyncWorkflow).job(t, workerEnvAutosyncJob) + for _, step := range job.Steps { + if strings.Contains(step.Run, "${{") { + t.Errorf("step %q run contains a ${{ }} expression; move it to env: and reference the variable instead:\n%s", step.Name, step.Run) + } + } +} + +// TestWorkerEnvSyncCopyAgainstFixture runs the real tools/rbe/worker-env- +// sync-copy against a scratch checkout and a local gascity (reusing +// rbe_worker_env_test.go's fixture): it overwrites the manifest and the pin +// byte for byte, leaves everything else untouched, and fails loudly rather +// than writing anything when gascity's platforms/BUILD.bazel carries no +// pin to copy. +func TestWorkerEnvSyncCopyAgainstFixture(t *testing.T) { + f := newRBEWorkerEnvFixture(t) + stalePin := rbeTestPin(rbeTestStaleManifest) + beads := f.beads(t, rbeTestStaleManifest, stalePin) + rbeTestWrite(t, filepath.Join(beads, workerEnvSyncCopy), readPolicyFile(t, bazelPolicyRoot(t), workerEnvSyncCopy), 0o755) + // A file the copy must never touch. + rbeTestWrite(t, filepath.Join(beads, "tools/rbe/untouched.txt"), "keep\n", 0o644) + + code, out := f.run(t, beads, workerEnvSyncCopy+" main\n") + if code != 0 { + t.Fatalf("worker-env-sync-copy: exit %d\n%s", code, out) + } + if !strings.Contains(out, "copied gascity main's worker-env ("+f.pin+")") { + t.Errorf("worker-env-sync-copy output %q, want it to report the copied pin %s", out, f.pin) + } + gotManifest, err := os.ReadFile(filepath.Join(beads, rbeWorkerEnvManifest)) + if err != nil || string(gotManifest) != f.manifest { + t.Errorf("worker-env.txt after copy = %q, %v; want gascity's manifest %q", gotManifest, err, f.manifest) + } + gotBuild, err := os.ReadFile(filepath.Join(beads, rbeWorkerPlatformBuild)) + if err != nil || !strings.Contains(string(gotBuild), "\"worker-env\": \""+f.pin+"\",") { + t.Errorf("platforms/BUILD.bazel after copy = %q, %v; want the pin %s", gotBuild, err, f.pin) + } + untouched, err := os.ReadFile(filepath.Join(beads, "tools/rbe/untouched.txt")) + if err != nil || string(untouched) != "keep\n" { + t.Errorf("worker-env-sync-copy touched an unrelated file: %q, %v", untouched, err) + } + + // gascity's platforms/BUILD.bazel with no pin at all: nothing copied. + noPinGascity := t.TempDir() + rbeTestWrite(t, filepath.Join(noPinGascity, "main/tools/rbe/worker-env.txt"), "arch x86_64\n", 0o644) + rbeTestWrite(t, filepath.Join(noPinGascity, "main/platforms/BUILD.bazel"), "platform(\n name = \"rbe_worker\",\n)\n", 0o644) + beadsNoPin := f.beads(t, rbeTestStaleManifest, stalePin) + rbeTestWrite(t, filepath.Join(beadsNoPin, workerEnvSyncCopy), readPolicyFile(t, bazelPolicyRoot(t), workerEnvSyncCopy), 0o755) + cmd := "WORKER_ENV_SYNC_URL=file://" + noPinGascity + " " + workerEnvSyncCopy + " main\n" + code, out = f.run(t, beadsNoPin, cmd) + if code == 0 || strings.Contains(out, "copied gascity") { + t.Errorf("worker-env-sync-copy with no gascity pin: exit %d\n%s, want a failure and nothing copied", code, out) + } + stillStale, err := os.ReadFile(filepath.Join(beadsNoPin, rbeWorkerEnvManifest)) + if err != nil || string(stillStale) != rbeTestStaleManifest { + t.Errorf("worker-env-sync-copy wrote despite no gascity pin: %q, %v", stillStale, err) + } +} diff --git a/tools/rbe/worker-env-sync-copy b/tools/rbe/worker-env-sync-copy new file mode 100755 index 000000000..4f0b30be2 --- /dev/null +++ b/tools/rbe/worker-env-sync-copy @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +# worker-env-sync-copy: copy gascity's worker-env manifest and pin here. +# +# tools/rbe/worker-env-sync-copy [REF] +# +# tools/rbe/worker-env-sync only detects an out-of-step pin (exit 1) or that +# it could not compare (exit 2); it never writes. This is the write half, +# run only after that exit-1 detection: it fetches gascity's +# tools/rbe/worker-env.txt and platforms/BUILD.bazel at REF (default main) +# and overwrites beads' copies, byte for byte, the same edit #7419 made by +# hand. It does not touch git (no add, commit, or push) -- the caller +# (worker-env-autosync.yml) stages and opens the PR. +# +# Env: WORKER_ENV_SYNC_URL, gascity's raw-content base (default +# https://raw.githubusercontent.com/gastownhall/gascity). Tests only. +# Run from the repository root. +set -Eeuo pipefail + +ref=${1:-main} +base=${WORKER_ENV_SYNC_URL:-https://raw.githubusercontent.com/gastownhall/gascity}/$ref +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT + +curl -fsSL --connect-timeout 5 --max-time 15 --retry 2 -o "$tmp/worker-env.txt" "$base/tools/rbe/worker-env.txt" +curl -fsSL --connect-timeout 5 --max-time 15 --retry 2 -o "$tmp/BUILD.bazel" "$base/platforms/BUILD.bazel" + +pin=$(sed -n 's/^ *"worker-env": "\(sha256:[0-9a-f]\{64\}\)",$/\1/p' "$tmp/BUILD.bazel") +if [ -z "$pin" ]; then + echo "worker-env-sync-copy: gascity $ref's platforms/BUILD.bazel has no worker-env pin; nothing copied" >&2 + exit 1 +fi + +cp "$tmp/worker-env.txt" tools/rbe/worker-env.txt +sed -i -E "s#\"worker-env\": \"sha256:[0-9a-f]{64}\",#\"worker-env\": \"$pin\",#" platforms/BUILD.bazel + +ours=$(sed -n 's/^ *"worker-env": "\(sha256:[0-9a-f]\{64\}\)",$/\1/p' platforms/BUILD.bazel) +if [ "$ours" != "$pin" ]; then + echo "worker-env-sync-copy: platforms/BUILD.bazel still pins $ours after the edit, want $pin" >&2 + exit 1 +fi +echo "worker-env-sync-copy: copied gascity $ref's worker-env ($pin)"