ci(rbe): auto-PR a worker-env pin sync when gascity main re-pins (mc-gv9fk) (#7455)
rbe-west's pool workers advertise gastownhall/gascity main's worker-env pin; when gascity re-pins (a Go/dolt/OS bump, e.g. #7387) beads' copy falls out of step and every remote-execution run stops in bazel.yml's preflight until a human notices and copies the fix by hand (#7419 took ~100 minutes). This adds a scheduled workflow (every 15 minutes, plus workflow_dispatch) that runs tools/rbe/worker-env-sync against gascity main, and on a real mismatch (exit 1) copies the manifest and pin with the new tools/rbe/worker-env-sync-copy, opens/updates a PR on a fixed branch with status/needs-review-auto, and arms auto-merge. GitHub never runs checks on a PR a plain GITHUB_TOKEN opens, so this needs a GitHub App token -- but gastownhall-autofix (contents:write on this repo) cannot open or label a PR. Two Apps split the two write roles instead: gastownhall-autofix mints push-token (contents:write only) and does the git push of bot/worker-env-sync via plain git; gascity-autofix (pull-requests:write only) mints pr-token and does everything PR-shaped -- create/update, the label, and arming auto-merge -- via gh. peter-evans/create-pull-request takes one token for the whole flow, so it can't honor this split; plain git and gh replace it. Every ${{ }} is resolved into a step's env: first, never written inside a run: script. Both Apps' id/private-key secrets live in the same autofix environment, gated by its deployment-branch policy to main; schedule and workflow_dispatch are never fork events, so neither key is ever readable by a fork. Needs admin setup: the gascity-autofix App (id 5259233) installed on gastownhall/beads (and gascity) with pull-requests:write and metadata read, and its GASCITY_AUTOFIX_APP_ID / GASCITY_AUTOFIX_APP_PRIVATE_KEY secrets added to the autofix environment alongside the existing AUTOFIX_APP_ID / AUTOFIX_APP_PRIVATE_KEY. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
11b5d3271e
commit
b3cdbf9e46
4 changed files with 497 additions and 0 deletions
196
.github/workflows/worker-env-autosync.yml
vendored
Normal file
196
.github/workflows/worker-env-autosync.yml
vendored
Normal file
|
|
@ -0,0 +1,196 @@
|
|||
# Keep beads' rbe worker-env pin in step with gastownhall/gascity main
|
||||
# without the ~100-minute wait #7419 fixed by hand.
|
||||
#
|
||||
# rbe-west's OSS pool is gascity's: its workers advertise the sha256 of the
|
||||
# toolchain manifest gascity's tools/rbe/worker-env measures, and the
|
||||
# schedulers match //platforms:rbe_worker's worker-env exec property
|
||||
# exactly. beads' tools/rbe/worker-env.txt and the platforms/BUILD.bazel pin
|
||||
# are a byte-for-byte copy of gascity main's; when gascity moves the pin (a
|
||||
# Go, dolt, or OS/library bump -- #7387 was a Go security release), every
|
||||
# beads remote-execution run stops in bazel.yml's rbe-job preflight
|
||||
# (tools/rbe/worker-env-sync) until the copy catches up. nightly.yml's
|
||||
# rbe-worker-env-sync job already opens a ci/infra issue for a human to fix;
|
||||
# this instead opens the fix itself, every 15 minutes, so the gap is
|
||||
# minutes, not until someone reads the issue.
|
||||
#
|
||||
# tools/rbe/worker-env-sync only detects the mismatch (exit 1) or that it
|
||||
# could not compare with gascity (exit 2, e.g. GitHub is briefly
|
||||
# unreachable; the next run retries). The write half, tools/rbe/worker-env-
|
||||
# sync-copy, runs only after that exit-1 detection, on a checkout already in
|
||||
# hand -- so a transient gascity outage can turn "in step" into "unknown",
|
||||
# never into an unwanted write.
|
||||
#
|
||||
# TWO TOKENS, ONE JOB EACH: GitHub never runs checks on a PR a plain
|
||||
# GITHUB_TOKEN opens (its loop-prevention rule), so this needs a GitHub App
|
||||
# token to open the PR. The gastownhall-autofix App (contents:write only on
|
||||
# this repo) mints push-token and does the git push of the fixed branch
|
||||
# (bot/worker-env-sync); it cannot open, label or merge a PR. The separate
|
||||
# gascity-autofix App (pull-requests:write only, no contents) mints pr-token
|
||||
# and does everything PR-shaped: create/update, the status/needs-review-auto
|
||||
# label, and arming auto-merge. Splitting them means a bug in the PR-shaped
|
||||
# half can never push a commit, and a bug in the push half can never touch a
|
||||
# PR; neither token can do the other's job even by accident. Both Apps'
|
||||
# credentials are secrets of the same `autofix` environment.
|
||||
#
|
||||
# peter-evans/create-pull-request takes one token for the whole push+PR
|
||||
# flow, so it cannot honor this split; this uses plain git for the push and
|
||||
# gh for the PR instead. Every `${{ }}` is resolved into a step's `env:`,
|
||||
# never written inside a `run:` script, so a value that happened to look
|
||||
# like shell (a title or body containing `` ` `` or `$`) is data to the
|
||||
# shell, not something GitHub's expression syntax or the shell re-parses.
|
||||
#
|
||||
# SECURITY: schedule and workflow_dispatch are never fork events (no
|
||||
# pull_request trigger here at all), so neither App's private key -- secrets
|
||||
# of the `autofix` environment, whose deployment-branch policy allows only
|
||||
# main -- is ever readable by a fork. The guard on github.repository is
|
||||
# defense in depth for a fork of this repo, where the secrets would not
|
||||
# exist anyway. Both mint steps run only once worker-env-sync has already
|
||||
# found a real mismatch, so a run that is in step or can't compare asks
|
||||
# neither App for a token. gh pr merge --auto only arms GitHub's native
|
||||
# auto-merge (it still waits for every required check and the branch
|
||||
# protection rule); it does not merge anything itself.
|
||||
name: rbe worker-env autosync
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "*/15 * * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
autosync:
|
||||
name: Sync worker-env pin with gascity main
|
||||
if: github.repository == 'gastownhall/beads'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
# Grants the job read access to this environment's secrets (both Apps'
|
||||
# id and private key); its deployment-branch policy restricts that to
|
||||
# jobs running on main, which schedule and workflow_dispatch on this
|
||||
# repo always are.
|
||||
environment: autofix
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
# Never fails the step itself: rc is the signal every later step
|
||||
# gates on (1, a real mismatch; 2, gascity unreachable -- the next
|
||||
# scheduled run retries; 0, in step). A run every 15 minutes failing
|
||||
# red for however long a mismatch takes to review and merge would be
|
||||
# noise, not signal.
|
||||
- name: Compare the worker-env manifest and pin with gascity's main
|
||||
id: sync
|
||||
run: |
|
||||
rc=0
|
||||
tools/rbe/worker-env-sync main || rc=$?
|
||||
echo "rc=$rc" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Copy gascity main's worker-env manifest and pin
|
||||
if: steps.sync.outputs.rc == '1'
|
||||
run: tools/rbe/worker-env-sync-copy main
|
||||
|
||||
- name: Confirm only the worker-env files moved
|
||||
if: steps.sync.outputs.rc == '1'
|
||||
run: |
|
||||
changed="$(git diff --name-only)"
|
||||
untracked="$(git ls-files --others --exclude-standard)"
|
||||
want=$'platforms/BUILD.bazel\ntools/rbe/worker-env.txt'
|
||||
if [ -n "$untracked" ] || [ "$(printf '%s\n' "$changed" | sort)" != "$(printf '%s\n' "$want" | sort)" ]; then
|
||||
echo "::error::worker-env-sync-copy touched more than platforms/BUILD.bazel and tools/rbe/worker-env.txt:"
|
||||
printf '%s\n' "$changed" "$untracked"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# contents:write only, on this repo alone: all it can do is push
|
||||
# bot/worker-env-sync. Minted only once a real mismatch needs a push.
|
||||
- name: Mint push token (gastownhall-autofix)
|
||||
id: push-token
|
||||
if: steps.sync.outputs.rc == '1'
|
||||
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3
|
||||
with:
|
||||
app-id: ${{ secrets.AUTOFIX_APP_ID }}
|
||||
private-key: ${{ secrets.AUTOFIX_APP_PRIVATE_KEY }}
|
||||
owner: gastownhall
|
||||
repositories: beads
|
||||
permission-contents: write
|
||||
|
||||
# pull-requests:write only, on this repo alone: all it can do is
|
||||
# open, label and arm auto-merge on a PR -- never push a commit.
|
||||
- name: Mint PR token (gascity-autofix)
|
||||
id: pr-token
|
||||
if: steps.sync.outputs.rc == '1'
|
||||
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3
|
||||
with:
|
||||
app-id: ${{ secrets.GASCITY_AUTOFIX_APP_ID }}
|
||||
private-key: ${{ secrets.GASCITY_AUTOFIX_APP_PRIVATE_KEY }}
|
||||
owner: gastownhall
|
||||
repositories: beads
|
||||
permission-pull-requests: write
|
||||
|
||||
# push-token alone; it cannot see a PR exists, let alone touch one.
|
||||
# Leased to the branch: a force-with-lease scoped to refs/heads/
|
||||
# bot/worker-env-sync only, so a push can never move any other ref.
|
||||
- name: Commit and push the sync branch
|
||||
if: steps.sync.outputs.rc == '1'
|
||||
env:
|
||||
PUSH_TOKEN: ${{ steps.push-token.outputs.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
BRANCH: bot/worker-env-sync
|
||||
COMMIT_SUBJECT: "build(rbe): sync worker-env pin with gascity main"
|
||||
GIT_AUTHOR_NAME: gastownhall-autofix[bot]
|
||||
GIT_AUTHOR_EMAIL: gastownhall-autofix[bot]@users.noreply.github.com
|
||||
GIT_COMMITTER_NAME: gastownhall-autofix[bot]
|
||||
GIT_COMMITTER_EMAIL: gastownhall-autofix[bot]@users.noreply.github.com
|
||||
run: |
|
||||
git checkout -B "$BRANCH"
|
||||
git add platforms/BUILD.bazel tools/rbe/worker-env.txt
|
||||
git commit -m "$COMMIT_SUBJECT"
|
||||
git push "https://x-access-token:${PUSH_TOKEN}@github.com/${REPO}.git" \
|
||||
"HEAD:refs/heads/$BRANCH" "--force-with-lease=refs/heads/$BRANCH"
|
||||
|
||||
# pr-token alone; it cannot push a commit, only act on the PR the
|
||||
# step above already pushed.
|
||||
- name: Open or update the sync PR
|
||||
id: pr
|
||||
if: steps.sync.outputs.rc == '1'
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.pr-token.outputs.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
BRANCH: bot/worker-env-sync
|
||||
TITLE: "build(rbe): sync worker-env pin with gascity main"
|
||||
LABEL: status/needs-review-auto
|
||||
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
run: |
|
||||
body="$(cat <<BODY
|
||||
rbe-west's pool workers advertise gastownhall/gascity main's worker-env
|
||||
pin; beads' copy (\`tools/rbe/worker-env.txt\` and the
|
||||
\`platforms/BUILD.bazel\` pin) had fallen out of step, so beads' remote
|
||||
actions were queueing on a pin no worker serves
|
||||
(\`tools/rbe/worker-env-sync main\`, run $RUN_URL).
|
||||
|
||||
This copies gascity main's manifest and pin byte for byte, same as
|
||||
#7419. Auto-merge is armed: it lands once the automated review and the
|
||||
rest of the required checks are green.
|
||||
BODY
|
||||
)"
|
||||
number="$(gh pr list --repo "$REPO" --head "$BRANCH" --base main --state open --json number --jq '.[0].number')"
|
||||
if [ -n "$number" ]; then
|
||||
gh pr edit "$number" --repo "$REPO" --title "$TITLE" --body "$body"
|
||||
else
|
||||
gh pr create --repo "$REPO" --base main --head "$BRANCH" --title "$TITLE" --body "$body"
|
||||
number="$(gh pr list --repo "$REPO" --head "$BRANCH" --base main --state open --json number --jq '.[0].number')"
|
||||
fi
|
||||
gh pr edit "$number" --repo "$REPO" --add-label "$LABEL"
|
||||
echo "number=$number" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Enable auto-merge
|
||||
if: steps.sync.outputs.rc == '1' && steps.pr.outputs.number
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.pr-token.outputs.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ steps.pr.outputs.number }}
|
||||
run: gh pr merge --auto --squash --repo "$REPO" "$PR_NUMBER"
|
||||
|
|
@ -61,6 +61,7 @@ go_test(
|
|||
"pr_risk_bazel_coverage_test.go",
|
||||
"rbe_worker_env_test.go",
|
||||
"repo_helpers_test.go",
|
||||
"worker_env_autosync_test.go",
|
||||
], # keep
|
||||
data = [
|
||||
"//:repo_other_files",
|
||||
|
|
|
|||
259
scripts/worker_env_autosync_test.go
Normal file
259
scripts/worker_env_autosync_test.go
Normal file
|
|
@ -0,0 +1,259 @@
|
|||
package scripts_test
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// worker-env-autosync.yml closes the #7419 gap: nightly.yml's rbe-worker-
|
||||
// env-sync job only opens a ci/infra issue for a human to fix a gascity
|
||||
// re-pin (#7387's ~100-minute stall); this instead runs tools/rbe/worker-
|
||||
// env-sync every 15 minutes and, on a real mismatch (exit 1), copies
|
||||
// gascity main's manifest and pin (tools/rbe/worker-env-sync-copy) and
|
||||
// opens or updates a fixed-branch PR with auto-merge armed.
|
||||
const (
|
||||
workerEnvAutosyncWorkflow = "worker-env-autosync.yml"
|
||||
workerEnvAutosyncJob = "autosync"
|
||||
workerEnvAutosyncBranch = "bot/worker-env-sync"
|
||||
workerEnvSyncCopy = "tools/rbe/worker-env-sync-copy"
|
||||
)
|
||||
|
||||
// TestWorkerEnvAutosyncWorkflowTriggersOnScheduleAndDispatchOnly: the App
|
||||
// private key this job mints is a secret of the autofix environment
|
||||
// (deployment-branch policy: main only), and the only events that can ever
|
||||
// run on main without a fork in the loop are schedule and workflow_dispatch
|
||||
// -- pull_request (or pull_request_target) here would hand a fork a path to
|
||||
// it.
|
||||
func TestWorkerEnvAutosyncWorkflowTriggersOnScheduleAndDispatchOnly(t *testing.T) {
|
||||
var parsed struct {
|
||||
On map[string]any `yaml:"on"`
|
||||
}
|
||||
text := readPolicyFile(t, sourceRepoRoot(t), ".github/workflows/"+workerEnvAutosyncWorkflow)
|
||||
if err := yaml.Unmarshal([]byte(text), &parsed); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := len(parsed.On); got != 2 {
|
||||
t.Fatalf("%s on: has %d triggers (%v), want exactly schedule and workflow_dispatch", workerEnvAutosyncWorkflow, got, parsed.On)
|
||||
}
|
||||
if _, ok := parsed.On["workflow_dispatch"]; !ok {
|
||||
t.Errorf("%s has no workflow_dispatch trigger", workerEnvAutosyncWorkflow)
|
||||
}
|
||||
schedule, ok := parsed.On["schedule"].([]any)
|
||||
if !ok || len(schedule) != 1 {
|
||||
t.Fatalf("%s schedule = %+v, want exactly one cron entry", workerEnvAutosyncWorkflow, parsed.On["schedule"])
|
||||
}
|
||||
cron, _ := schedule[0].(map[string]any)
|
||||
if cron["cron"] != "*/15 * * * *" {
|
||||
t.Errorf("%s schedule cron = %v, want */15 * * * * (every 15 minutes)", workerEnvAutosyncWorkflow, cron["cron"])
|
||||
}
|
||||
for _, forbidden := range []string{"pull_request", "pull_request_target", "push", "merge_group", "workflow_run"} {
|
||||
if _, ok := parsed.On[forbidden]; ok {
|
||||
t.Errorf("%s must never run on %s (forks or an unprivileged push could reach the autofix App token)", workerEnvAutosyncWorkflow, forbidden)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestWorkerEnvAutosyncWorkflowPermissionsAreMinimal: least privilege.
|
||||
// Top-level and the job's own grant both stay read-only: every write this
|
||||
// job makes (the push, the PR, the label, auto-merge) goes through one of
|
||||
// the two minted App tokens, never the job's own GITHUB_TOKEN permissions,
|
||||
// so there is nothing here for either App token's role to leak into.
|
||||
func TestWorkerEnvAutosyncWorkflowPermissionsAreMinimal(t *testing.T) {
|
||||
var top struct {
|
||||
Permissions map[string]string `yaml:"permissions"`
|
||||
}
|
||||
text := readPolicyFile(t, sourceRepoRoot(t), ".github/workflows/"+workerEnvAutosyncWorkflow)
|
||||
if err := yaml.Unmarshal([]byte(text), &top); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(top.Permissions, map[string]string{"contents": "read"}) {
|
||||
t.Errorf("%s top-level permissions = %v, want contents: read", workerEnvAutosyncWorkflow, top.Permissions)
|
||||
}
|
||||
|
||||
job := readCIWorkflow(t, workerEnvAutosyncWorkflow).job(t, workerEnvAutosyncJob)
|
||||
wantJobPerms := map[string]any{"contents": "read"}
|
||||
if !reflect.DeepEqual(job.Permissions, wantJobPerms) {
|
||||
t.Errorf("%s job %s permissions = %v, want %v (contents:read alone; both write roles come from the minted App tokens)", workerEnvAutosyncWorkflow, workerEnvAutosyncJob, job.Permissions, wantJobPerms)
|
||||
}
|
||||
if job.RunsOn != "ubuntu-latest" {
|
||||
t.Errorf("%s job %s runs-on = %q, want ubuntu-latest", workerEnvAutosyncWorkflow, workerEnvAutosyncJob, job.RunsOn)
|
||||
}
|
||||
if job.If != "github.repository == 'gastownhall/beads'" {
|
||||
t.Errorf("%s job %s if = %q, want the fork guard github.repository == 'gastownhall/beads'", workerEnvAutosyncWorkflow, workerEnvAutosyncJob, job.If)
|
||||
}
|
||||
if job.Environment.Name != "autofix" {
|
||||
t.Errorf("%s job %s environment = %q, want autofix (both Apps' id/private-key secrets live there, gated to main)", workerEnvAutosyncWorkflow, workerEnvAutosyncJob, job.Environment.Name)
|
||||
}
|
||||
}
|
||||
|
||||
// TestWorkerEnvAutosyncWorkflowShape: the job compares with gascity main,
|
||||
// gates every write step on a real mismatch (rc == 1, never rc == 2's
|
||||
// "unknown" or rc == 0's "in step"), mints two App tokens only once a
|
||||
// mismatch needs a write, pushes the fixed branch with push-token alone,
|
||||
// opens/labels the PR and arms auto-merge with pr-token alone (so it is
|
||||
// not a GITHUB_TOKEN PR - those never trigger CI), and arms auto-merge
|
||||
// rather than merging directly.
|
||||
func TestWorkerEnvAutosyncWorkflowShape(t *testing.T) {
|
||||
job := readCIWorkflow(t, workerEnvAutosyncWorkflow).job(t, workerEnvAutosyncJob)
|
||||
|
||||
sync := job.step(t, "Compare the worker-env manifest and pin with gascity's main")
|
||||
if sync.ID != "sync" || !strings.Contains(sync.Run, "tools/rbe/worker-env-sync main") {
|
||||
t.Errorf("sync step: id %q, run %q; want id sync running tools/rbe/worker-env-sync main", sync.ID, sync.Run)
|
||||
}
|
||||
|
||||
const mismatchIf = "steps.sync.outputs.rc == '1'"
|
||||
for _, name := range []string{
|
||||
"Copy gascity main's worker-env manifest and pin",
|
||||
"Confirm only the worker-env files moved",
|
||||
"Mint push token (gastownhall-autofix)",
|
||||
"Mint PR token (gascity-autofix)",
|
||||
"Commit and push the sync branch",
|
||||
"Open or update the sync PR",
|
||||
} {
|
||||
step := job.step(t, name)
|
||||
if step.If != mismatchIf {
|
||||
t.Errorf("step %q if = %q, want %q (a write step must run only on a real mismatch)", name, step.If, mismatchIf)
|
||||
}
|
||||
}
|
||||
|
||||
copyStep := job.step(t, "Copy gascity main's worker-env manifest and pin")
|
||||
if !strings.Contains(copyStep.Run, workerEnvSyncCopy+" main") {
|
||||
t.Errorf("copy step run %q, want it to call %s main", copyStep.Run, workerEnvSyncCopy)
|
||||
}
|
||||
|
||||
pushMint := job.step(t, "Mint push token (gastownhall-autofix)")
|
||||
if pushMint.ID != "push-token" || actionFamily(pushMint.Uses) != "actions/create-github-app-token" ||
|
||||
pushMint.With["app-id"] != "${{ secrets.AUTOFIX_APP_ID }}" || pushMint.With["private-key"] != "${{ secrets.AUTOFIX_APP_PRIVATE_KEY }}" ||
|
||||
pushMint.With["permission-contents"] != "write" || pushMint.With["permission-pull-requests"] != "" {
|
||||
t.Errorf("push-token mint = %+v, want the gastownhall-autofix mint (AUTOFIX_APP_ID/AUTOFIX_APP_PRIVATE_KEY, contents:write only)", pushMint)
|
||||
}
|
||||
|
||||
prMint := job.step(t, "Mint PR token (gascity-autofix)")
|
||||
if prMint.ID != "pr-token" || actionFamily(prMint.Uses) != "actions/create-github-app-token" ||
|
||||
prMint.With["app-id"] != "${{ secrets.GASCITY_AUTOFIX_APP_ID }}" || prMint.With["private-key"] != "${{ secrets.GASCITY_AUTOFIX_APP_PRIVATE_KEY }}" ||
|
||||
prMint.With["permission-pull-requests"] != "write" || prMint.With["permission-contents"] != "" {
|
||||
t.Errorf("pr-token mint = %+v, want the gascity-autofix mint (GASCITY_AUTOFIX_APP_ID/GASCITY_AUTOFIX_APP_PRIVATE_KEY, pull-requests:write only)", prMint)
|
||||
}
|
||||
if pushMint.Uses != prMint.Uses {
|
||||
t.Errorf("push-token and pr-token mint different action pins: %q vs %q, want the same SHA-pinned actions/create-github-app-token", pushMint.Uses, prMint.Uses)
|
||||
}
|
||||
|
||||
push := job.step(t, "Commit and push the sync branch")
|
||||
if push.Uses != "" || !strings.Contains(push.Run, "git push") {
|
||||
t.Errorf("push step = %+v, want a plain git push (no action; push-token and pr-token cannot be split through one action's single token input)", push)
|
||||
}
|
||||
if push.Env["PUSH_TOKEN"] != "${{ steps.push-token.outputs.token }}" {
|
||||
t.Errorf("push step env PUSH_TOKEN = %q, want the minted push-token", push.Env["PUSH_TOKEN"])
|
||||
}
|
||||
if _, leaked := push.Env["GH_TOKEN"]; leaked {
|
||||
t.Errorf("push step sets GH_TOKEN; the push step must never see pr-token")
|
||||
}
|
||||
if !strings.Contains(push.Run, workerEnvAutosyncBranch) && push.Env["BRANCH"] != workerEnvAutosyncBranch {
|
||||
t.Errorf("push step does not reference the fixed branch %q", workerEnvAutosyncBranch)
|
||||
}
|
||||
if !strings.Contains(push.Run, "force-with-lease=refs/heads/$BRANCH") {
|
||||
t.Errorf("push step run %q, want a --force-with-lease scoped to the fixed branch's ref alone", push.Run)
|
||||
}
|
||||
|
||||
pr := job.step(t, "Open or update the sync PR")
|
||||
if pr.Uses != "" {
|
||||
t.Fatalf("PR step uses %q, want plain gh (no action; the token split rules out peter-evans/create-pull-request, which takes one token)", pr.Uses)
|
||||
}
|
||||
if pr.Env["GH_TOKEN"] != "${{ steps.pr-token.outputs.token }}" {
|
||||
t.Errorf("PR step env GH_TOKEN = %q, want the minted pr-token", pr.Env["GH_TOKEN"])
|
||||
}
|
||||
if _, leaked := pr.Env["PUSH_TOKEN"]; leaked {
|
||||
t.Errorf("PR step sets PUSH_TOKEN; the PR step must never see push-token")
|
||||
}
|
||||
if pr.Env["BRANCH"] != workerEnvAutosyncBranch {
|
||||
t.Errorf("PR step env BRANCH = %q, want the fixed branch %q (so a later run updates the same PR instead of opening a new one)", pr.Env["BRANCH"], workerEnvAutosyncBranch)
|
||||
}
|
||||
if !strings.Contains(pr.Run, "gh pr create") || !strings.Contains(pr.Run, "gh pr edit") || !strings.Contains(pr.Run, `--add-label "$LABEL"`) {
|
||||
t.Errorf("PR step run %q, want it to create-or-update the PR and add the label via gh", pr.Run)
|
||||
}
|
||||
if pr.Env["LABEL"] != "status/needs-review-auto" {
|
||||
t.Errorf("PR step env LABEL = %q, want status/needs-review-auto (routes it through the automated review workflow)", pr.Env["LABEL"])
|
||||
}
|
||||
|
||||
merge := job.step(t, "Enable auto-merge")
|
||||
if merge.If != mismatchIf+" && steps.pr.outputs.number" {
|
||||
t.Errorf("merge step if = %q, want gated on a real mismatch and a PR having been opened", merge.If)
|
||||
}
|
||||
if merge.Uses != "" || !strings.Contains(merge.Run, "gh pr merge --auto") {
|
||||
t.Errorf("merge step = %+v, want a gh pr merge --auto call (arms auto-merge; never merges outright)", merge)
|
||||
}
|
||||
if merge.Env["GH_TOKEN"] != "${{ steps.pr-token.outputs.token }}" {
|
||||
t.Errorf("merge step env GH_TOKEN = %q, want pr-token (enabling auto-merge is PR-shaped, not a push)", merge.Env["GH_TOKEN"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestWorkerEnvAutosyncWorkflowNeverInterpolatesExpressionsInRun: every
|
||||
// `${{ }}` GitHub would otherwise substitute directly into a shell script
|
||||
// is instead assigned to a step's `env:` first, so a value that happens to
|
||||
// contain shell metacharacters (a PR title, a run URL) is data the shell
|
||||
// sees through a variable, never text GitHub splices into the script
|
||||
// before bash ever runs it.
|
||||
func TestWorkerEnvAutosyncWorkflowNeverInterpolatesExpressionsInRun(t *testing.T) {
|
||||
job := readCIWorkflow(t, workerEnvAutosyncWorkflow).job(t, workerEnvAutosyncJob)
|
||||
for _, step := range job.Steps {
|
||||
if strings.Contains(step.Run, "${{") {
|
||||
t.Errorf("step %q run contains a ${{ }} expression; move it to env: and reference the variable instead:\n%s", step.Name, step.Run)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestWorkerEnvSyncCopyAgainstFixture runs the real tools/rbe/worker-env-
|
||||
// sync-copy against a scratch checkout and a local gascity (reusing
|
||||
// rbe_worker_env_test.go's fixture): it overwrites the manifest and the pin
|
||||
// byte for byte, leaves everything else untouched, and fails loudly rather
|
||||
// than writing anything when gascity's platforms/BUILD.bazel carries no
|
||||
// pin to copy.
|
||||
func TestWorkerEnvSyncCopyAgainstFixture(t *testing.T) {
|
||||
f := newRBEWorkerEnvFixture(t)
|
||||
stalePin := rbeTestPin(rbeTestStaleManifest)
|
||||
beads := f.beads(t, rbeTestStaleManifest, stalePin)
|
||||
rbeTestWrite(t, filepath.Join(beads, workerEnvSyncCopy), readPolicyFile(t, bazelPolicyRoot(t), workerEnvSyncCopy), 0o755)
|
||||
// A file the copy must never touch.
|
||||
rbeTestWrite(t, filepath.Join(beads, "tools/rbe/untouched.txt"), "keep\n", 0o644)
|
||||
|
||||
code, out := f.run(t, beads, workerEnvSyncCopy+" main\n")
|
||||
if code != 0 {
|
||||
t.Fatalf("worker-env-sync-copy: exit %d\n%s", code, out)
|
||||
}
|
||||
if !strings.Contains(out, "copied gascity main's worker-env ("+f.pin+")") {
|
||||
t.Errorf("worker-env-sync-copy output %q, want it to report the copied pin %s", out, f.pin)
|
||||
}
|
||||
gotManifest, err := os.ReadFile(filepath.Join(beads, rbeWorkerEnvManifest))
|
||||
if err != nil || string(gotManifest) != f.manifest {
|
||||
t.Errorf("worker-env.txt after copy = %q, %v; want gascity's manifest %q", gotManifest, err, f.manifest)
|
||||
}
|
||||
gotBuild, err := os.ReadFile(filepath.Join(beads, rbeWorkerPlatformBuild))
|
||||
if err != nil || !strings.Contains(string(gotBuild), "\"worker-env\": \""+f.pin+"\",") {
|
||||
t.Errorf("platforms/BUILD.bazel after copy = %q, %v; want the pin %s", gotBuild, err, f.pin)
|
||||
}
|
||||
untouched, err := os.ReadFile(filepath.Join(beads, "tools/rbe/untouched.txt"))
|
||||
if err != nil || string(untouched) != "keep\n" {
|
||||
t.Errorf("worker-env-sync-copy touched an unrelated file: %q, %v", untouched, err)
|
||||
}
|
||||
|
||||
// gascity's platforms/BUILD.bazel with no pin at all: nothing copied.
|
||||
noPinGascity := t.TempDir()
|
||||
rbeTestWrite(t, filepath.Join(noPinGascity, "main/tools/rbe/worker-env.txt"), "arch x86_64\n", 0o644)
|
||||
rbeTestWrite(t, filepath.Join(noPinGascity, "main/platforms/BUILD.bazel"), "platform(\n name = \"rbe_worker\",\n)\n", 0o644)
|
||||
beadsNoPin := f.beads(t, rbeTestStaleManifest, stalePin)
|
||||
rbeTestWrite(t, filepath.Join(beadsNoPin, workerEnvSyncCopy), readPolicyFile(t, bazelPolicyRoot(t), workerEnvSyncCopy), 0o755)
|
||||
cmd := "WORKER_ENV_SYNC_URL=file://" + noPinGascity + " " + workerEnvSyncCopy + " main\n"
|
||||
code, out = f.run(t, beadsNoPin, cmd)
|
||||
if code == 0 || strings.Contains(out, "copied gascity") {
|
||||
t.Errorf("worker-env-sync-copy with no gascity pin: exit %d\n%s, want a failure and nothing copied", code, out)
|
||||
}
|
||||
stillStale, err := os.ReadFile(filepath.Join(beadsNoPin, rbeWorkerEnvManifest))
|
||||
if err != nil || string(stillStale) != rbeTestStaleManifest {
|
||||
t.Errorf("worker-env-sync-copy wrote despite no gascity pin: %q, %v", stillStale, err)
|
||||
}
|
||||
}
|
||||
41
tools/rbe/worker-env-sync-copy
Executable file
41
tools/rbe/worker-env-sync-copy
Executable file
|
|
@ -0,0 +1,41 @@
|
|||
#!/usr/bin/env bash
|
||||
# worker-env-sync-copy: copy gascity's worker-env manifest and pin here.
|
||||
#
|
||||
# tools/rbe/worker-env-sync-copy [REF]
|
||||
#
|
||||
# tools/rbe/worker-env-sync only detects an out-of-step pin (exit 1) or that
|
||||
# it could not compare (exit 2); it never writes. This is the write half,
|
||||
# run only after that exit-1 detection: it fetches gascity's
|
||||
# tools/rbe/worker-env.txt and platforms/BUILD.bazel at REF (default main)
|
||||
# and overwrites beads' copies, byte for byte, the same edit #7419 made by
|
||||
# hand. It does not touch git (no add, commit, or push) -- the caller
|
||||
# (worker-env-autosync.yml) stages and opens the PR.
|
||||
#
|
||||
# Env: WORKER_ENV_SYNC_URL, gascity's raw-content base (default
|
||||
# https://raw.githubusercontent.com/gastownhall/gascity). Tests only.
|
||||
# Run from the repository root.
|
||||
set -Eeuo pipefail
|
||||
|
||||
ref=${1:-main}
|
||||
base=${WORKER_ENV_SYNC_URL:-https://raw.githubusercontent.com/gastownhall/gascity}/$ref
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
|
||||
curl -fsSL --connect-timeout 5 --max-time 15 --retry 2 -o "$tmp/worker-env.txt" "$base/tools/rbe/worker-env.txt"
|
||||
curl -fsSL --connect-timeout 5 --max-time 15 --retry 2 -o "$tmp/BUILD.bazel" "$base/platforms/BUILD.bazel"
|
||||
|
||||
pin=$(sed -n 's/^ *"worker-env": "\(sha256:[0-9a-f]\{64\}\)",$/\1/p' "$tmp/BUILD.bazel")
|
||||
if [ -z "$pin" ]; then
|
||||
echo "worker-env-sync-copy: gascity $ref's platforms/BUILD.bazel has no worker-env pin; nothing copied" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cp "$tmp/worker-env.txt" tools/rbe/worker-env.txt
|
||||
sed -i -E "s#\"worker-env\": \"sha256:[0-9a-f]{64}\",#\"worker-env\": \"$pin\",#" platforms/BUILD.bazel
|
||||
|
||||
ours=$(sed -n 's/^ *"worker-env": "\(sha256:[0-9a-f]\{64\}\)",$/\1/p' platforms/BUILD.bazel)
|
||||
if [ "$ours" != "$pin" ]; then
|
||||
echo "worker-env-sync-copy: platforms/BUILD.bazel still pins $ours after the edit, want $pin" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "worker-env-sync-copy: copied gascity $ref's worker-env ($pin)"
|
||||
Loading…
Add table
Add a link
Reference in a new issue