fix(scripts): update-nix-vendorhash no longer reports a build failure as success (#7504)
* fix(scripts): update-nix-vendorhash no longer reports a build failure as success scripts/update-nix-vendorhash.sh treated the absence of a "got:" hash line as proof the vendorHash was already correct, so a build failure unrelated to a hash mismatch (e.g. a Go toolchain too old for a dependency) was misreported as success and the workflow went green on a broken build (#7496). Fixed by capturing the intentional-failure build's real exit status instead of discarding it with `|| true`, and using that exit status (not a "hash mismatch" string grep) to tell a genuine no-op apart from an unrelated build break. The step-5 verification build had the same shape of bug: piping through `tee` without pipefail meant the if-condition tested tee's exit status, not the build's, so a failing verification build still printed "Build successful!". Scoped `set -o pipefail` to just that one line rather than the whole script, since the earlier hash-extraction step legitimately relies on `grep -o` returning non-zero on no match. Fixes gastownhall/beads#7498. Added an independent `nix build .#default` step to update-vendor-hash.yml so the workflow no longer trusts the script's exit status alone, even if a future bug in the script reintroduces a false-success path. Fixes gastownhall/beads#7499. Added TestPinnedDoltCLIAtLeastRecommendedMin, asserting the pinned Dolt CLI version (scripts/ci/install-dolt.sh) is at least doltversion.RecommendedMin, so the two policies can't silently drift apart — RecommendedMin's own doc comment already claims this alignment but nothing enforced it. Fixes gastownhall/beads#7500. Verified by reproducing both update-nix-vendorhash.sh failure modes with a stub `nix` binary (a non-hash build failure, and a step-5 verification build failure) and confirming the intended successful hash-update path still works end to end. go build/vet clean, golangci-lint 0 issues, go test ./scripts/... passes except a pre-existing unrelated failure (TestControlQueryScriptPreservesReadyProbeFailure, tracked separately as beads#7293 and untouched by this change). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr * chore: empty commit to retrigger CI -- retrying after a 3-hour quota recovery window * chore: empty commit to retrigger CI -- isolated single retry after real-bug fixes landed on sibling PRs * fix(scripts): add missing BUILD.bazel dep, sharpen vendorhash diagnostics scripts_test now imports internal/doltversion (for TestPinnedDoltCLIAtLeastRecommendedMin) but scripts/BUILD.bazel had no matching dep row, which would fail the target's compile under rules_go once Bazel actually runs against this branch (the Bazel / jobs failing so far died in bazel.fork-cert, unrelated to this). Also, per review: scope set -o pipefail to just the step-5 verify block (turn it back off after, rather than leaving it on for the rest of the script with a comment claiming it's scoped when it isn't), and keep the old "hash mismatch" string check as a sub-case in the ACTUAL_HASH-empty branch, so a hash mismatch whose got: line fails to parse (e.g. a future Nix output-format change) is diagnosed differently from a build that failed for an unrelated reason. Re-verified all four failure/success paths with a stub nix binary (non-hash failure, hash-mismatch-with-unparseable-got-line, step-5-verify failure, and the genuine happy path). go build/vet clean, golangci-lint 0 issues. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr * fix(scripts): update-nix-vendorhash preserves operator backups, restores on any handled failure The script backed up default.nix to a fixed name in the repo root, default.nix.backup, before editing it -- but that is also the conventional name an operator gives their own manual backup. Running the script unconditionally overwrote (then later deleted) any such file. A failed step-5 verification build also left the candidate hash in default.nix with no backup at all to restore from, since the backup had already been deleted right after step 4 applied the real hash. Moved the backup into a private per-run scratch directory (mktemp -d, cleaned on every exit via an EXIT trap) instead of a fixed repo-root name, so it can never collide with an operator's own default.nix.backup. Copy with cp -p so a restore brings back the exact original bytes AND mode, not just content. Kept the backup around through step 5 instead of deleting it right after step 4, and added a restore on the step-5 verification-failure path, which previously left the unverified candidate hash in place. Also: the "no hash mismatch, already correct" exit-0 path previously inferred correctness purely from the fake-hash probe build's unexpected success, without ever building the real, restored file. It now runs a second build against the restored file and only exits 0 if that succeeds too, surfacing a diagnostic log if it doesn't. Fixes gastownhall/beads#7529. Verified with a stub nix binary across five scenarios: a pre-existing operator default.nix.backup (distinct content + mode 600) survives untouched in every one; a non-hash build failure restores the original (content + mode 640); the already-correct path now runs two builds and only succeeds if both do; the core bug repro (hash extracted, step-5 verify fails) now restores the original instead of leaving the unverified candidate hash with no backup; the genuine happy path still updates the hash correctly. No leftover scratch directories in any case. go build unaffected (shell-only change). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr * fix(scripts): update-nix-vendorhash restores the original on an interrupted run A SIGINT/SIGTERM during the probe build (the long step) reaches this script and its nix child as one process group; the child dies, this script exits through the EXIT trap with the placeholder hash still in default.nix. The plain `rm -rf "$SCRATCH_DIR"` trap deleted the only backup at that point, leaving no way back except `git checkout default.nix` -- which also discards any uncommitted operator edits. Worse than every handled failure case this file already restores from. The placeholder is never a valid end state, so the EXIT trap now checks for it and restores first, before dropping the scratch directory. Every handled exit path has already restored or verified default.nix by the time it runs, so this is a no-op there -- it only fires on an interrupted run. Found by review on PR #7504 (beads#7529's own follow-up): reproduced and confirmed with a stub nix that signals its own parent process group, matching a real Ctrl-C mid-build. Confirmed the bug first (plain trap: placeholder left in place, backup gone, matching the report exactly), then confirmed the fix (original hash and mode restored, operator backup untouched, no leftover scratch dir). Also re-ran the full existing scenario matrix (non-hash failure, hash mismatch with step-5 verify failure, genuine happy path) to confirm no regression. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr * fix(scripts): restore default.nix if vendorhash verification is interrupted The EXIT trap restored default.nix only while it still held the placeholder hash. A SIGINT/SIGTERM during the step-5 verification build therefore left the unverified candidate hash in place and deleted the only backup with the scratch dir. Track VERIFIED instead: until a build has verified default.nix as it stands, any exit restores the original. On handled paths this is a no-op, since they have already restored or verified the file. Write the build logs under ${TMPDIR:-/tmp} with per-run names. The verification log had a fixed /tmp name shared by every run and every user. Add the regression tests the #7498/#7499/#7529 fixes lacked: - TestUpdateNixVendorHashReportsTheRealBuildResult runs the script with a stub nix over a non-hash build failure, an unparseable mismatch, a failed verification, a half-parsed got: hash, a real update, and a probe that accepts the placeholder (restored file builds, or not). Every failure exits 1 with default.nix restored byte for byte and mode for mode, the operator's own default.nix.backup untouched and the named logs kept. Against main's script it reproduces each false success. - TestUpdateNixVendorHashInterruptRestoresOriginal signals the script's process group during the probe build and during verification. - TestUpdateVendorHashWorkflowBuildsIndependently pins the workflow's unconditional `nix build .#default` between the script and the match-report and push steps. The scenario matrix, operator-backup and file-mode fixture, and workflow ordering check are adapted from BlueboxC's correction of this PR in their fork (62e7718e), as described on #7498 and #7529. Refs #7504, #7496, #7498, #7499, #7500, #7529 Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen Co-authored-by: BlueboxC <david@blueboxcomputers.es> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com> Co-authored-by: BlueboxC <david@blueboxcomputers.es>
This commit is contained in:
parent
9c4b3d8bab
commit
7838c4b449
6 changed files with 635 additions and 16 deletions
8
.github/workflows/update-vendor-hash.yml
vendored
8
.github/workflows/update-vendor-hash.yml
vendored
|
|
@ -82,6 +82,14 @@ jobs:
|
|||
- name: Update vendorHash
|
||||
run: ./scripts/update-nix-vendorhash.sh
|
||||
|
||||
- name: Verify the build independently of the script's exit status
|
||||
# update-nix-vendorhash.sh's own verification build (its step 5) is
|
||||
# internal to the script; this step re-derives the same signal from
|
||||
# a source the workflow doesn't have to trust: a plain `nix build`
|
||||
# that this job step fails on if it fails, regardless of what the
|
||||
# script itself reported.
|
||||
run: nix build .#default
|
||||
|
||||
- name: Detect whether hash changed
|
||||
id: changed
|
||||
run: |
|
||||
|
|
|
|||
|
|
@ -83,6 +83,7 @@ go_test(
|
|||
# re-runs the rest of this suite too.
|
||||
env = {"BEADS_TEST_GO": "$(rlocationpath @go_sdk//:bin/go)"},
|
||||
deps = [
|
||||
"//internal/doltversion",
|
||||
"//internal/formula",
|
||||
"//internal/testutil/bazeltest",
|
||||
"//scripts/internal/testbash",
|
||||
|
|
@ -142,6 +143,8 @@ go_test(
|
|||
"repo_helpers_test.go",
|
||||
"required_suite_test.go",
|
||||
"test_script_test.go",
|
||||
"update_nix_vendorhash_test.go",
|
||||
"update_nix_vendorhash_unix_test.go",
|
||||
"upgrade_smoke_loop_test.go",
|
||||
],
|
||||
data = [
|
||||
|
|
|
|||
|
|
@ -16,6 +16,8 @@ import (
|
|||
"testing"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
|
||||
"github.com/steveyegge/beads/internal/doltversion"
|
||||
)
|
||||
|
||||
func TestCIWorkflowArtifactOwnership(t *testing.T) {
|
||||
|
|
@ -1533,6 +1535,57 @@ func TestPinnedDoltCLIMatchesContainerImage(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestPinnedDoltCLIAtLeastRecommendedMin guards doltversion.RecommendedMin
|
||||
// against the pinned Dolt CLI version (scripts/ci/install-dolt.sh) drifting
|
||||
// below it (#7500). TestPinnedDoltCLIMatchesContainerImage holds the container
|
||||
// image and tools/bazel/dolt.bzl to that same pin, so this covers every CI
|
||||
// Dolt. A pin lowered below RecommendedMin would mean every CI run exercises a
|
||||
// Dolt CLI the package's own policy already warns is below the matrix-tested
|
||||
// floor.
|
||||
func TestPinnedDoltCLIAtLeastRecommendedMin(t *testing.T) {
|
||||
root := sourceRepoRoot(t)
|
||||
|
||||
installer, err := os.ReadFile(filepath.Join(root, "scripts", "ci", "install-dolt.sh"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cliVersion := captureOne(t, `(?m)^readonly version="([0-9]+\.[0-9]+\.[0-9]+)"$`, string(installer), "scripts/ci/install-dolt.sh")
|
||||
|
||||
pinned := doltversion.MustParse(cliVersion)
|
||||
if !pinned.AtLeast(doltversion.RecommendedMin) {
|
||||
t.Errorf("pinned Dolt CLI %s is below doltversion.RecommendedMin %s", cliVersion, doltversion.RecommendedMin)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUpdateVendorHashWorkflowBuildsIndependently guards #7499: the dependabot
|
||||
// vendorHash job must not take scripts/update-nix-vendorhash.sh's exit status
|
||||
// on trust. A script that wrongly exits 0 on a broken build (as it did on
|
||||
// #7441) is caught only if an unconditional `nix build .#default` runs after
|
||||
// it and before the job reports a match or pushes the hash.
|
||||
func TestUpdateVendorHashWorkflowBuildsIndependently(t *testing.T) {
|
||||
const verify = "Verify the build independently of the script's exit status"
|
||||
job := readCIWorkflow(t, "update-vendor-hash.yml").job(t, "update-vendor-hash")
|
||||
if job.ContinueOnError {
|
||||
t.Error("update-vendor-hash job has continue-on-error; a failed build would not fail it")
|
||||
}
|
||||
step := job.step(t, verify)
|
||||
if strings.TrimSpace(step.Run) != "nix build .#default" || step.If != "" || step.ContinueOnError != nil {
|
||||
t.Errorf("%q: run %q, if %q, continue-on-error %v; want an unconditional `nix build .#default`",
|
||||
verify, step.Run, step.If, step.ContinueOnError)
|
||||
}
|
||||
update := job.stepIndex(t, "Update vendorHash")
|
||||
if run := strings.TrimSpace(job.Steps[update].Run); run != "./scripts/update-nix-vendorhash.sh" {
|
||||
t.Errorf("\"Update vendorHash\" runs %q, want ./scripts/update-nix-vendorhash.sh", run)
|
||||
}
|
||||
at := job.stepIndex(t, verify)
|
||||
for _, later := range []string{"Detect whether hash changed", "Commit and push hash update"} {
|
||||
if i := job.stepIndex(t, later); !(update < at && at < i) {
|
||||
t.Errorf("%q is step %d, %q step %d and %q step %d; the build must run after the script and before %q",
|
||||
verify, at, "Update vendorHash", update, later, i, later)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func captureOne(t *testing.T, pattern, body, source string) string {
|
||||
t.Helper()
|
||||
|
||||
|
|
|
|||
|
|
@ -117,8 +117,43 @@ log_info "Current vendorHash: $CURRENT_HASH"
|
|||
log_info "Setting temporary bad hash to trigger error..."
|
||||
FAKE_HASH="sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
|
||||
|
||||
# Backup the file
|
||||
cp default.nix default.nix.backup
|
||||
# Back up to a private per-run scratch directory, never to a fixed name in
|
||||
# the repo root: `default.nix.backup` there is also the conventional name an
|
||||
# operator gives their own manual backup, and overwriting (then later
|
||||
# deleting) that file on every run can destroy one on sight. The scratch
|
||||
# dir is removed on every exit path, handled or not; a handled failure
|
||||
# restores from it first via restore_original.
|
||||
SCRATCH_DIR=$(mktemp -d "${TMPDIR:-/tmp}/nix-vendorhash.XXXXXX")
|
||||
BACKUP="$SCRATCH_DIR/default.nix.orig"
|
||||
# -p preserves the exact original bytes and mode; a handled failure must
|
||||
# restore both, not just the content.
|
||||
cp -p default.nix "$BACKUP"
|
||||
|
||||
restore_original() {
|
||||
cp -p "$BACKUP" default.nix
|
||||
}
|
||||
|
||||
# A SIGINT/SIGTERM during the long steps (the nix builds) reaches this process
|
||||
# and its nix child as one group; the child dies, and this script exits
|
||||
# through the EXIT trap with default.nix still holding the placeholder hash
|
||||
# (probe build) or an unverified candidate hash (step 5). Plain
|
||||
# `rm -rf "$SCRATCH_DIR"` there would delete the only backup while leaving
|
||||
# either in place -- worse than every handled failure, since recovery becomes
|
||||
# `git checkout default.nix`, which also discards any uncommitted operator
|
||||
# edits. Neither is a valid end state: VERIFIED is set only once a build has
|
||||
# verified default.nix as it stands, so restore the original whenever the run
|
||||
# ends before that. Every handled exit below has already restored or verified
|
||||
# default.nix by the time this runs, so it is a no-op on every path except an
|
||||
# interrupted one.
|
||||
VERIFIED=false
|
||||
cleanup() {
|
||||
if [ "$VERIFIED" != true ] && [ -f "$BACKUP" ] && ! cmp -s "$BACKUP" default.nix; then
|
||||
restore_original
|
||||
log_info "Restored original default.nix (run interrupted)"
|
||||
fi
|
||||
rm -rf "$SCRATCH_DIR"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# Detect sed version (GNU vs BSD). BSD sed requires an explicit backup-suffix
|
||||
# argument to -i; GNU sed must NOT be given one.
|
||||
|
|
@ -143,8 +178,21 @@ sed_inplace "s|vendorHash = \".*\";|vendorHash = \"$FAKE_HASH\";|" default.nix
|
|||
|
||||
# Step 2: Run nix build and capture the error
|
||||
log_info "Building to get actual hash (this will fail intentionally)..."
|
||||
BUILD_LOG="/tmp/nix-vendorhash-build-$$.log"
|
||||
run_nix build > "$BUILD_LOG" 2>&1 || true
|
||||
# Per-run log names under ${TMPDIR:-/tmp}, like the scratch dir: a fixed name
|
||||
# in a shared /tmp collides with another run's (or another user's) log. They
|
||||
# live outside the scratch dir because a failure keeps the logs it names.
|
||||
BUILD_LOG="${TMPDIR:-/tmp}/nix-vendorhash-build-$$.log"
|
||||
VERIFY_LOG="${TMPDIR:-/tmp}/nix-vendorhash-verify-$$.log"
|
||||
# This build is expected to fail (the fake hash above is designed to trigger
|
||||
# a hash-mismatch error), so capture its real exit status explicitly instead
|
||||
# of `|| true`, which would discard it. The exit status (not a "hash
|
||||
# mismatch" string grep, which can't tell a build that failed for an
|
||||
# unrelated reason from one that actually succeeded) is what step 3 below
|
||||
# uses to tell "no update needed" apart from "the build is broken".
|
||||
set +e
|
||||
run_nix build > "$BUILD_LOG" 2>&1
|
||||
BUILD_EXIT=$?
|
||||
set -e
|
||||
|
||||
# Step 3: Extract the actual hash from the error message
|
||||
# Error format: "error: hash mismatch ... got: sha256-ACTUAL_HASH" (note: multiple spaces)
|
||||
|
|
@ -163,18 +211,45 @@ if [ -z "$ACTUAL_HASH" ]; then
|
|||
log_info "Full build log saved to: $BUILD_LOG"
|
||||
|
||||
# Restore backup
|
||||
mv default.nix.backup default.nix
|
||||
restore_original
|
||||
log_info "Restored original default.nix"
|
||||
|
||||
# Check if hash is actually the same
|
||||
if grep -q "hash mismatch" "$BUILD_LOG"; then
|
||||
log_error "Hash mismatch detected but couldn't extract hash"
|
||||
# Tell "the build genuinely succeeded with the fake hash rejected, so
|
||||
# the real hash is already correct" apart from "the build failed for an
|
||||
# unrelated reason and never got far enough to report a hash mismatch"
|
||||
# by the build's own exit status, not by whether "hash mismatch" happens
|
||||
# to appear in the log — a toolchain error or any other failure mode
|
||||
# has no such string and was previously misreported as success.
|
||||
if [ "$BUILD_EXIT" -eq 0 ]; then
|
||||
# The probe build's success only means the FAKE hash didn't trigger
|
||||
# a mismatch — it was never a build of the real, restored file. A
|
||||
# success exit must actually build that file, not just infer it
|
||||
# from the probe.
|
||||
log_info "No hash mismatch in the probe build - verifying the restored file builds as-is..."
|
||||
RESTORE_BUILD_LOG="${TMPDIR:-/tmp}/nix-vendorhash-restore-build-$$.log"
|
||||
set +e
|
||||
run_nix build > "$RESTORE_BUILD_LOG" 2>&1
|
||||
RESTORE_BUILD_EXIT=$?
|
||||
set -e
|
||||
if [ "$RESTORE_BUILD_EXIT" -eq 0 ]; then
|
||||
VERIFIED=true
|
||||
log_success "No hash mismatch - vendorHash is already correct!"
|
||||
rm -f "$BUILD_LOG" "$RESTORE_BUILD_LOG"
|
||||
exit 0
|
||||
else
|
||||
log_error "The probe build reported no hash mismatch, but the restored file doesn't build either (exit $RESTORE_BUILD_EXIT) — investigate before retrying"
|
||||
log_error "Check the restored-file build log: $RESTORE_BUILD_LOG"
|
||||
log_error "Probe build log: $BUILD_LOG"
|
||||
exit 1
|
||||
fi
|
||||
elif grep -q "hash mismatch" "$BUILD_LOG"; then
|
||||
log_error "Build reported a hash mismatch (exit $BUILD_EXIT) but the got: line didn't match the extraction regex — Nix's output format may have changed"
|
||||
log_error "Check the full log: $BUILD_LOG"
|
||||
exit 1
|
||||
else
|
||||
log_success "No hash mismatch - vendorHash is already correct!"
|
||||
rm -f "$BUILD_LOG"
|
||||
exit 0
|
||||
log_error "Build failed for a reason other than a hash mismatch (exit $BUILD_EXIT)"
|
||||
log_error "Check the full log: $BUILD_LOG"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
|
|
@ -184,12 +259,22 @@ log_success "Extracted actual hash: $ACTUAL_HASH"
|
|||
log_info "Updating default.nix with correct hash..."
|
||||
sed_inplace "s|vendorHash = \".*\";|vendorHash = \"$ACTUAL_HASH\";|" default.nix
|
||||
|
||||
# Remove backup
|
||||
rm default.nix.backup
|
||||
# $BACKUP is kept until step 5 verifies: restore_original below needs it if
|
||||
# the verification build fails, and the EXIT trap if that build is
|
||||
# interrupted. The trap removes the scratch dir either way.
|
||||
|
||||
# Step 5: Verify the update by rebuilding
|
||||
log_info "Verifying update with a clean build..."
|
||||
if run_nix build 2>&1 | tee /tmp/nix-build-verify.log; then
|
||||
# Without pipefail, this if-condition's status is tee's (which basically
|
||||
# always succeeds), not run_nix build's, so a real build failure here was
|
||||
# reported as "Build successful!". Scoped to just this if (turned back off
|
||||
# right after) rather than set for the rest of the script: the ACTUAL_HASH
|
||||
# extraction above relies on `grep -o` legitimately returning non-zero on no
|
||||
# match, which pipefail would turn into a script abort under `set -e`.
|
||||
set -o pipefail
|
||||
if run_nix build 2>&1 | tee "$VERIFY_LOG"; then
|
||||
set +o pipefail
|
||||
VERIFIED=true
|
||||
log_success "Build successful! vendorHash updated correctly."
|
||||
echo ""
|
||||
log_info "Summary:"
|
||||
|
|
@ -199,10 +284,13 @@ if run_nix build 2>&1 | tee /tmp/nix-build-verify.log; then
|
|||
log_success "default.nix has been updated"
|
||||
|
||||
# Clean up build logs
|
||||
rm -f "$BUILD_LOG" /tmp/nix-build-verify.log
|
||||
rm -f "$BUILD_LOG" "$VERIFY_LOG"
|
||||
else
|
||||
set +o pipefail
|
||||
log_error "Build failed after updating hash"
|
||||
log_error "See /tmp/nix-build-verify.log for details"
|
||||
log_error "See $VERIFY_LOG for details"
|
||||
log_error "Initial build log: $BUILD_LOG"
|
||||
restore_original
|
||||
log_info "Restored original default.nix (candidate hash did not verify)"
|
||||
exit 1
|
||||
fi
|
||||
|
|
|
|||
340
scripts/update_nix_vendorhash_test.go
Normal file
340
scripts/update_nix_vendorhash_test.go
Normal file
|
|
@ -0,0 +1,340 @@
|
|||
package scripts_test
|
||||
|
||||
import (
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"runtime"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The fixture's vendorHash values carry '+' and '/' on purpose: the script's
|
||||
// got: regex and its sed replacement must both pass them through intact.
|
||||
const (
|
||||
nixFixtureOriginalHash = "sha256-9A9lzocuEii/t2AHC1ca5uQsB965NqhQW3wpmwklZeg="
|
||||
nixFixtureUpdatedHash = "sha256-IL80Qs9c7ZG17R/hRF6D+F6WVmGB0I+k60+nFuUpIJA="
|
||||
// nixFixturePlaceholder is the known-bad hash the script probes with.
|
||||
nixFixturePlaceholder = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
|
||||
nixFixtureOperatorBak = "operator backup\n"
|
||||
)
|
||||
|
||||
// nixStub stands in for `nix build`. Each call records the vendorHash it was
|
||||
// asked to build, then acts out the next word of NIX_STUB_PLAN (the last word
|
||||
// repeats):
|
||||
//
|
||||
// toolchain fails before any hash check (#7441's Go-floor failure)
|
||||
// mismatch reports a hash mismatch whose got: line is NIX_STUB_GOT
|
||||
// unparseable reports a hash mismatch in a got: format the script can't read
|
||||
// garbled reports a got: line its regex only half matches ("sha256-")
|
||||
// ok builds
|
||||
// fail fails after the hash was accepted
|
||||
// block records blocked-<call>, then sleeps until signalled
|
||||
const nixStub = `#!/bin/sh
|
||||
set -eu
|
||||
[ "${1:-}" = build ] || { echo "nix stub: unexpected invocation: $*" >&2; exit 64; }
|
||||
call=0
|
||||
if [ -s "$NIX_STUB_STATE/count" ]; then
|
||||
IFS= read -r call <"$NIX_STUB_STATE/count"
|
||||
fi
|
||||
call=$((call + 1))
|
||||
printf '%s\n' "$call" >"$NIX_STUB_STATE/count"
|
||||
grep 'vendorHash = ' default.nix >>"$NIX_STUB_STATE/builds"
|
||||
n=$call
|
||||
set -- $NIX_STUB_PLAN
|
||||
while [ "$n" -gt 1 ] && [ "$#" -gt 1 ]; do
|
||||
shift
|
||||
n=$((n - 1))
|
||||
done
|
||||
mismatch_header="error: hash mismatch in fixed-output derivation '/nix/store/x-beads-go-modules.drv':"
|
||||
case "$1" in
|
||||
toolchain)
|
||||
echo 'go: github.com/dolthub/driver/v2@v2.4.1 requires go >= 1.26.5 (running go 1.26.4; GOTOOLCHAIN=local)' >&2
|
||||
exit 1 ;;
|
||||
mismatch)
|
||||
printf '%s\n specified: %s\n got: %s\n' "$mismatch_header" "$NIX_STUB_PLACEHOLDER" "$NIX_STUB_GOT" >&2
|
||||
exit 1 ;;
|
||||
unparseable)
|
||||
printf '%s\n got: sha256:0c4fd1m3dpbg2l2kq3zcmjb5x8qgvs1zl5z0a3k1yhb5rwbnk6zz\n' "$mismatch_header" >&2
|
||||
exit 1 ;;
|
||||
garbled)
|
||||
printf '%s\n got: sha256-!\n' "$mismatch_header" >&2
|
||||
exit 1 ;;
|
||||
ok)
|
||||
exit 0 ;;
|
||||
fail)
|
||||
echo "error: builder for '/nix/store/x-beads.drv' failed with exit code 2" >&2
|
||||
exit 1 ;;
|
||||
block)
|
||||
: >"$NIX_STUB_STATE/blocked-$call"
|
||||
exec sleep 60 ;;
|
||||
*)
|
||||
echo "nix stub: unknown plan step $1" >&2
|
||||
exit 64 ;;
|
||||
esac
|
||||
`
|
||||
|
||||
var nixFixtureVendorHashRe = regexp.MustCompile(`vendorHash = "([^"]*)";`)
|
||||
|
||||
// nixVendorHashFixture is a checkout holding only default.nix (and an
|
||||
// operator's own default.nix.backup), with the nix stub first on PATH and
|
||||
// TMPDIR fenced into the test, so it sees every file the script leaves behind.
|
||||
type nixVendorHashFixture struct {
|
||||
bash, script, dir, state, tmp string
|
||||
env []string
|
||||
}
|
||||
|
||||
func newNixVendorHashFixture(t *testing.T, plan string) nixVendorHashFixture {
|
||||
t.Helper()
|
||||
|
||||
bash := requireHostTool(t, "bash")
|
||||
bin := t.TempDir()
|
||||
fx := nixVendorHashFixture{bash: bash, dir: t.TempDir(), state: t.TempDir(), tmp: t.TempDir()}
|
||||
writeExecutable(t, filepath.Join(bin, "nix"), nixStub)
|
||||
writeNixFixtureFile(t, filepath.Join(fx.dir, "default.nix"), nixFixtureDefaultNix(nixFixtureOriginalHash), 0o640)
|
||||
writeNixFixtureFile(t, filepath.Join(fx.dir, "default.nix.backup"), nixFixtureOperatorBak, 0o600)
|
||||
|
||||
pathEnv := shellPathEnv()
|
||||
binPath := shellPathUnderEnv(t, bash, bin, pathEnv)
|
||||
commandPath := binPath + ":" + os.Getenv("PATH") + ":/usr/bin:/bin"
|
||||
if runtime.GOOS == "windows" {
|
||||
commandPath = binPath + ":/usr/bin:/bin"
|
||||
}
|
||||
fx.env = []string{
|
||||
"PATH=" + os.Getenv("PATH"),
|
||||
"BEADS_TEST_COMMAND_PATH=" + commandPath,
|
||||
"LC_ALL=C",
|
||||
"LANG=C",
|
||||
"BASH_ENV=",
|
||||
"ENV=",
|
||||
"TMPDIR=" + shellPathUnderEnv(t, bash, fx.tmp, pathEnv),
|
||||
"NIX_STUB_STATE=" + shellPathUnderEnv(t, bash, fx.state, pathEnv),
|
||||
"NIX_STUB_PLAN=" + plan,
|
||||
"NIX_STUB_GOT=" + nixFixtureUpdatedHash,
|
||||
"NIX_STUB_PLACEHOLDER=" + nixFixturePlaceholder,
|
||||
}
|
||||
requireShellCommandPath(t, bash, fx.dir, fx.env, "nix", binPath+"/nix")
|
||||
fx.script = shellPathUnderEnv(t, bash, filepath.Join(sourceRepoRoot(t), "scripts", "update-nix-vendorhash.sh"), pathEnv)
|
||||
return fx
|
||||
}
|
||||
|
||||
func nixFixtureDefaultNix(hash string) string {
|
||||
return "# operator comment the script must keep\n{ buildGoModule }:\nbuildGoModule {\n vendorHash = \"" + hash + "\";\n}\n"
|
||||
}
|
||||
|
||||
func writeNixFixtureFile(t *testing.T, path, body string, mode os.FileMode) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(path, []byte(body), mode); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// WriteFile's mode is filtered by the umask; the mode assertions need it exact.
|
||||
if err := os.Chmod(path, mode); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// command runs the repository's script from the fixture checkout, the way the
|
||||
// workflow and CONTRIBUTING.md run it from the repository root.
|
||||
func (fx nixVendorHashFixture) command() *exec.Cmd {
|
||||
cmd := bashScriptCommand(fx.bash, fx.script)
|
||||
cmd.Dir = fx.dir
|
||||
cmd.Env = fx.env
|
||||
return cmd
|
||||
}
|
||||
|
||||
// builds returns the vendorHash each stubbed `nix build` saw, in call order.
|
||||
func (fx nixVendorHashFixture) builds(t *testing.T) []string {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(filepath.Join(fx.state, "builds"))
|
||||
if os.IsNotExist(err) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var hashes []string
|
||||
for _, m := range nixFixtureVendorHashRe.FindAllStringSubmatch(string(data), -1) {
|
||||
hashes = append(hashes, m[1])
|
||||
}
|
||||
return hashes
|
||||
}
|
||||
|
||||
// assertCheckout checks the files the script may touch: default.nix holds
|
||||
// wantHash with its original bytes and mode otherwise, the operator's backup is
|
||||
// untouched, and nothing else was left beside them.
|
||||
func (fx nixVendorHashFixture) assertCheckout(t *testing.T, wantHash string) {
|
||||
t.Helper()
|
||||
|
||||
path := filepath.Join(fx.dir, "default.nix")
|
||||
got, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := nixFixtureDefaultNix(wantHash); string(got) != want {
|
||||
t.Errorf("default.nix =\n%s\nwant\n%s", got, want)
|
||||
}
|
||||
backup, err := os.ReadFile(filepath.Join(fx.dir, "default.nix.backup"))
|
||||
if err != nil || string(backup) != nixFixtureOperatorBak {
|
||||
t.Errorf("operator's default.nix.backup = %q (%v), want it untouched (%q)", backup, err, nixFixtureOperatorBak)
|
||||
}
|
||||
if runtime.GOOS != "windows" {
|
||||
for name, want := range map[string]os.FileMode{"default.nix": 0o640, "default.nix.backup": 0o600} {
|
||||
info, err := os.Stat(filepath.Join(fx.dir, name))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if info.Mode().Perm() != want {
|
||||
t.Errorf("%s mode = %o, want %o", name, info.Mode().Perm(), want)
|
||||
}
|
||||
}
|
||||
}
|
||||
entries, err := os.ReadDir(fx.dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var names []string
|
||||
for _, entry := range entries {
|
||||
names = append(names, entry.Name())
|
||||
}
|
||||
if !slices.Equal(names, []string{"default.nix", "default.nix.backup"}) {
|
||||
t.Errorf("checkout holds %q, want only default.nix and the operator's default.nix.backup", names)
|
||||
}
|
||||
}
|
||||
|
||||
// tmpEntries lists what the script left in its TMPDIR.
|
||||
func (fx nixVendorHashFixture) tmpEntries(t *testing.T) []string {
|
||||
t.Helper()
|
||||
entries, err := os.ReadDir(fx.tmp)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var names []string
|
||||
for _, entry := range entries {
|
||||
names = append(names, entry.Name())
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
// TestUpdateNixVendorHashReportsTheRealBuildResult is the regression net for
|
||||
// #7498 and #7529. The script used to exit 0 ("vendorHash is already
|
||||
// correct!") when the probe build failed for a reason other than the hash,
|
||||
// and to print "Build successful!" when its verification build failed, which
|
||||
// let update-vendor-hash.yml go green on #7441's broken Nix build (#7499). A
|
||||
// failure must exit non-zero, restore default.nix byte for byte and mode for
|
||||
// mode, leave an operator's own default.nix.backup alone, and keep the build
|
||||
// log it names; a success must have built the file it leaves behind.
|
||||
func TestUpdateNixVendorHashReportsTheRealBuildResult(t *testing.T) {
|
||||
const (
|
||||
original = nixFixtureOriginalHash
|
||||
updated = nixFixtureUpdatedHash
|
||||
placeholder = nixFixturePlaceholder
|
||||
)
|
||||
tests := []struct {
|
||||
name string
|
||||
plan string
|
||||
wantExit int
|
||||
wantHash string
|
||||
wantBuilds []string
|
||||
wantOutput string
|
||||
}{
|
||||
{
|
||||
name: "build fails for a reason other than the hash",
|
||||
plan: "toolchain",
|
||||
wantExit: 1,
|
||||
wantHash: original,
|
||||
wantBuilds: []string{placeholder},
|
||||
wantOutput: "Build failed for a reason other than a hash mismatch",
|
||||
},
|
||||
{
|
||||
name: "hash mismatch with a got line the regex cannot read",
|
||||
plan: "unparseable",
|
||||
wantExit: 1,
|
||||
wantHash: original,
|
||||
wantBuilds: []string{placeholder},
|
||||
wantOutput: "didn't match the extraction regex",
|
||||
},
|
||||
{
|
||||
name: "verification build fails",
|
||||
plan: "mismatch fail",
|
||||
wantExit: 1,
|
||||
wantHash: original,
|
||||
wantBuilds: []string{placeholder, updated},
|
||||
wantOutput: "Build failed after updating hash",
|
||||
},
|
||||
{
|
||||
name: "half-parsed got hash fails verification",
|
||||
plan: "garbled fail",
|
||||
wantExit: 1,
|
||||
wantHash: original,
|
||||
wantBuilds: []string{placeholder, "sha256-"},
|
||||
wantOutput: "Build failed after updating hash",
|
||||
},
|
||||
{
|
||||
name: "hash updated and verified",
|
||||
plan: "mismatch ok",
|
||||
wantExit: 0,
|
||||
wantHash: updated,
|
||||
wantBuilds: []string{placeholder, updated},
|
||||
wantOutput: "default.nix has been updated",
|
||||
},
|
||||
{
|
||||
name: "probe accepts the placeholder and the restored file builds",
|
||||
plan: "ok",
|
||||
wantExit: 0,
|
||||
wantHash: original,
|
||||
wantBuilds: []string{placeholder, original},
|
||||
wantOutput: "vendorHash is already correct",
|
||||
},
|
||||
{
|
||||
name: "probe accepts the placeholder but the restored file does not build",
|
||||
plan: "ok fail",
|
||||
wantExit: 1,
|
||||
wantHash: original,
|
||||
wantBuilds: []string{placeholder, original},
|
||||
wantOutput: "the restored file doesn't build either",
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
fx := newNixVendorHashFixture(t, test.plan)
|
||||
output, err := fx.command().CombinedOutput()
|
||||
if got := exitCode(err); got != test.wantExit {
|
||||
t.Fatalf("exit = %d, want %d\n%s", got, test.wantExit, output)
|
||||
}
|
||||
if !strings.Contains(string(output), test.wantOutput) {
|
||||
t.Errorf("output lacks %q:\n%s", test.wantOutput, output)
|
||||
}
|
||||
if got := fx.builds(t); !slices.Equal(got, test.wantBuilds) {
|
||||
t.Errorf("nix built vendorHash %q, want %q\n%s", got, test.wantBuilds, output)
|
||||
}
|
||||
fx.assertCheckout(t, test.wantHash)
|
||||
|
||||
// The private backup directory never outlives the run. A success
|
||||
// removes its logs too; a failure keeps the build logs it names.
|
||||
var logs []string
|
||||
for _, name := range fx.tmpEntries(t) {
|
||||
if !strings.HasSuffix(name, ".log") {
|
||||
t.Errorf("TMPDIR still holds %s after the run", name)
|
||||
continue
|
||||
}
|
||||
logs = append(logs, name)
|
||||
}
|
||||
if test.wantExit == 0 && len(logs) != 0 {
|
||||
t.Errorf("successful run left logs %q in TMPDIR", logs)
|
||||
}
|
||||
if test.wantExit != 0 {
|
||||
if len(logs) == 0 {
|
||||
t.Errorf("failed run kept no build log in TMPDIR\n%s", output)
|
||||
}
|
||||
for _, name := range logs {
|
||||
if !strings.Contains(string(output), name) {
|
||||
t.Errorf("failed run kept %s but never named it\n%s", name, output)
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
127
scripts/update_nix_vendorhash_unix_test.go
Normal file
127
scripts/update_nix_vendorhash_unix_test.go
Normal file
|
|
@ -0,0 +1,127 @@
|
|||
//go:build !windows
|
||||
|
||||
package scripts_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strconv"
|
||||
"syscall"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TestUpdateNixVendorHashInterruptRestoresOriginal signals the script's whole
|
||||
// process group mid-build, as a terminal Ctrl-C or a cancelled job does. Until
|
||||
// a build has verified the file it leaves behind, an interrupted run must put
|
||||
// the original default.nix back: during the probe build default.nix holds the
|
||||
// placeholder hash, and during the verification build an unverified candidate;
|
||||
// neither may outlive the run once its private backup is gone. SIGTERM rather
|
||||
// than SIGINT because a process started with SIGINT ignored (any background
|
||||
// job of a non-interactive shell) passes that on to the script, and the EXIT
|
||||
// trap under test runs the same way for both.
|
||||
func TestUpdateNixVendorHashInterruptRestoresOriginal(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
plan string
|
||||
blockedBuild int
|
||||
wantBuilds []string
|
||||
}{
|
||||
{
|
||||
name: "during the probe build",
|
||||
plan: "block",
|
||||
blockedBuild: 1,
|
||||
wantBuilds: []string{nixFixturePlaceholder},
|
||||
},
|
||||
{
|
||||
name: "during the verification build",
|
||||
plan: "mismatch block",
|
||||
blockedBuild: 2,
|
||||
wantBuilds: []string{nixFixturePlaceholder, nixFixtureUpdatedHash},
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
fx := newNixVendorHashFixture(t, test.plan)
|
||||
cmd := fx.command()
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
|
||||
cmd.WaitDelay = 10 * time.Second
|
||||
var output bytes.Buffer
|
||||
cmd.Stdout = &output
|
||||
cmd.Stderr = &output
|
||||
if err := cmd.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
group := -cmd.Process.Pid
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- cmd.Wait() }()
|
||||
// stop kills the group unless the script has already been reaped
|
||||
// (its group id may be reused after that), then returns the output,
|
||||
// which is only safe to read once Wait has returned.
|
||||
reaped := false
|
||||
stop := func() string {
|
||||
if !reaped {
|
||||
_ = syscall.Kill(group, syscall.SIGKILL)
|
||||
<-done
|
||||
reaped = true
|
||||
}
|
||||
return output.String()
|
||||
}
|
||||
t.Cleanup(func() { stop() })
|
||||
|
||||
marker := filepath.Join(fx.state, "blocked-"+strconv.Itoa(test.blockedBuild))
|
||||
deadline := time.Now().Add(30 * time.Second)
|
||||
for {
|
||||
if _, err := os.Stat(marker); err == nil {
|
||||
break
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
reaped = true
|
||||
t.Fatalf("script exited (%v) before nix build %d started\n%s", err, test.blockedBuild, stop())
|
||||
default:
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatalf("nix build %d never started\n%s", test.blockedBuild, stop())
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
if err := syscall.Kill(group, syscall.SIGTERM); err != nil {
|
||||
t.Fatalf("signal the script's process group: %v\n%s", err, stop())
|
||||
}
|
||||
|
||||
var err error
|
||||
select {
|
||||
case err = <-done:
|
||||
reaped = true
|
||||
case <-time.After(30 * time.Second):
|
||||
t.Fatalf("script still running 30s after SIGTERM\n%s", stop())
|
||||
}
|
||||
// bash runs its EXIT trap, then dies of the signal it caught (or,
|
||||
// with a TERM trap of its own, exits 128+15).
|
||||
var exitErr *exec.ExitError
|
||||
if !errors.As(err, &exitErr) {
|
||||
t.Fatalf("script exited %v after SIGTERM, want it to fail\n%s", err, output.String())
|
||||
}
|
||||
status, ok := exitErr.Sys().(syscall.WaitStatus)
|
||||
killed := ok && status.Signaled() && status.Signal() == syscall.SIGTERM
|
||||
if !killed && (!ok || !status.Exited() || status.ExitStatus() != 128+int(syscall.SIGTERM)) {
|
||||
t.Errorf("script ended with %v after SIGTERM, want termination by SIGTERM\n%s", exitErr, output.String())
|
||||
}
|
||||
if got := fx.builds(t); !slices.Equal(got, test.wantBuilds) {
|
||||
t.Errorf("nix built vendorHash %q, want %q\n%s", got, test.wantBuilds, output.String())
|
||||
}
|
||||
fx.assertCheckout(t, nixFixtureOriginalHash)
|
||||
for _, name := range fx.tmpEntries(t) {
|
||||
if info, err := os.Stat(filepath.Join(fx.tmp, name)); err == nil && info.IsDir() {
|
||||
t.Errorf("interrupted run left its private directory %s behind", name)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue