beads/scripts/update-nix-vendorhash.sh
Rongjun GENG 7838c4b449
fix(scripts): update-nix-vendorhash no longer reports a build failure as success (#7504)
* fix(scripts): update-nix-vendorhash no longer reports a build failure as success

scripts/update-nix-vendorhash.sh treated the absence of a "got:" hash
line as proof the vendorHash was already correct, so a build failure
unrelated to a hash mismatch (e.g. a Go toolchain too old for a
dependency) was misreported as success and the workflow went green on
a broken build (#7496). Fixed by capturing the intentional-failure
build's real exit status instead of discarding it with `|| true`, and
using that exit status (not a "hash mismatch" string grep) to tell a
genuine no-op apart from an unrelated build break.

The step-5 verification build had the same shape of bug: piping
through `tee` without pipefail meant the if-condition tested tee's
exit status, not the build's, so a failing verification build still
printed "Build successful!". Scoped `set -o pipefail` to just that one
line rather than the whole script, since the earlier hash-extraction
step legitimately relies on `grep -o` returning non-zero on no match.

Fixes gastownhall/beads#7498.

Added an independent `nix build .#default` step to
update-vendor-hash.yml so the workflow no longer trusts the script's
exit status alone, even if a future bug in the script reintroduces a
false-success path.

Fixes gastownhall/beads#7499.

Added TestPinnedDoltCLIAtLeastRecommendedMin, asserting the pinned
Dolt CLI version (scripts/ci/install-dolt.sh) is at least
doltversion.RecommendedMin, so the two policies can't silently drift
apart — RecommendedMin's own doc comment already claims this alignment
but nothing enforced it.

Fixes gastownhall/beads#7500.

Verified by reproducing both update-nix-vendorhash.sh failure modes
with a stub `nix` binary (a non-hash build failure, and a step-5
verification build failure) and confirming the intended successful
hash-update path still works end to end. go build/vet clean,
golangci-lint 0 issues, go test ./scripts/... passes except a
pre-existing unrelated failure (TestControlQueryScriptPreservesReadyProbeFailure,
tracked separately as beads#7293 and untouched by this change).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr

* chore: empty commit to retrigger CI -- retrying after a 3-hour quota recovery window

* chore: empty commit to retrigger CI -- isolated single retry after real-bug fixes landed on sibling PRs

* fix(scripts): add missing BUILD.bazel dep, sharpen vendorhash diagnostics

scripts_test now imports internal/doltversion (for
TestPinnedDoltCLIAtLeastRecommendedMin) but scripts/BUILD.bazel had no
matching dep row, which would fail the target's compile under
rules_go once Bazel actually runs against this branch (the Bazel /
jobs failing so far died in bazel.fork-cert, unrelated to this).

Also, per review: scope set -o pipefail to just the step-5 verify
block (turn it back off after, rather than leaving it on for the rest
of the script with a comment claiming it's scoped when it isn't), and
keep the old "hash mismatch" string check as a sub-case in the
ACTUAL_HASH-empty branch, so a hash mismatch whose got: line fails to
parse (e.g. a future Nix output-format change) is diagnosed
differently from a build that failed for an unrelated reason.

Re-verified all four failure/success paths with a stub nix binary
(non-hash failure, hash-mismatch-with-unparseable-got-line,
step-5-verify failure, and the genuine happy path). go build/vet
clean, golangci-lint 0 issues.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr

* fix(scripts): update-nix-vendorhash preserves operator backups, restores on any handled failure

The script backed up default.nix to a fixed name in the repo root,
default.nix.backup, before editing it -- but that is also the
conventional name an operator gives their own manual backup. Running
the script unconditionally overwrote (then later deleted) any such
file. A failed step-5 verification build also left the candidate hash
in default.nix with no backup at all to restore from, since the
backup had already been deleted right after step 4 applied the real
hash.

Moved the backup into a private per-run scratch directory
(mktemp -d, cleaned on every exit via an EXIT trap) instead of a
fixed repo-root name, so it can never collide with an operator's own
default.nix.backup. Copy with cp -p so a restore brings back the
exact original bytes AND mode, not just content. Kept the backup
around through step 5 instead of deleting it right after step 4, and
added a restore on the step-5 verification-failure path, which
previously left the unverified candidate hash in place.

Also: the "no hash mismatch, already correct" exit-0 path previously
inferred correctness purely from the fake-hash probe build's
unexpected success, without ever building the real, restored file.
It now runs a second build against the restored file and only exits 0
if that succeeds too, surfacing a diagnostic log if it doesn't.

Fixes gastownhall/beads#7529.

Verified with a stub nix binary across five scenarios: a pre-existing
operator default.nix.backup (distinct content + mode 600) survives
untouched in every one; a non-hash build failure restores the
original (content + mode 640); the already-correct path now runs two
builds and only succeeds if both do; the core bug repro (hash
extracted, step-5 verify fails) now restores the original instead of
leaving the unverified candidate hash with no backup; the genuine
happy path still updates the hash correctly. No leftover scratch
directories in any case. go build unaffected (shell-only change).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr

* fix(scripts): update-nix-vendorhash restores the original on an interrupted run

A SIGINT/SIGTERM during the probe build (the long step) reaches this
script and its nix child as one process group; the child dies, this
script exits through the EXIT trap with the placeholder hash still in
default.nix. The plain `rm -rf "$SCRATCH_DIR"` trap deleted the only
backup at that point, leaving no way back except `git checkout
default.nix` -- which also discards any uncommitted operator edits.
Worse than every handled failure case this file already restores
from.

The placeholder is never a valid end state, so the EXIT trap now
checks for it and restores first, before dropping the scratch
directory. Every handled exit path has already restored or verified
default.nix by the time it runs, so this is a no-op there -- it only
fires on an interrupted run.

Found by review on PR #7504 (beads#7529's own follow-up): reproduced
and confirmed with a stub nix that signals its own parent process
group, matching a real Ctrl-C mid-build. Confirmed the bug first
(plain trap: placeholder left in place, backup gone, matching the
report exactly), then confirmed the fix (original hash and mode
restored, operator backup untouched, no leftover scratch dir). Also
re-ran the full existing scenario matrix (non-hash failure, hash
mismatch with step-5 verify failure, genuine happy path) to confirm no
regression.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr

* fix(scripts): restore default.nix if vendorhash verification is interrupted

The EXIT trap restored default.nix only while it still held the placeholder
hash. A SIGINT/SIGTERM during the step-5 verification build therefore left
the unverified candidate hash in place and deleted the only backup with the
scratch dir. Track VERIFIED instead: until a build has verified default.nix
as it stands, any exit restores the original. On handled paths this is a
no-op, since they have already restored or verified the file.

Write the build logs under ${TMPDIR:-/tmp} with per-run names. The
verification log had a fixed /tmp name shared by every run and every user.

Add the regression tests the #7498/#7499/#7529 fixes lacked:
- TestUpdateNixVendorHashReportsTheRealBuildResult runs the script with a
  stub nix over a non-hash build failure, an unparseable mismatch, a failed
  verification, a half-parsed got: hash, a real update, and a probe that
  accepts the placeholder (restored file builds, or not). Every failure
  exits 1 with default.nix restored byte for byte and mode for mode, the
  operator's own default.nix.backup untouched and the named logs kept.
  Against main's script it reproduces each false success.
- TestUpdateNixVendorHashInterruptRestoresOriginal signals the script's
  process group during the probe build and during verification.
- TestUpdateVendorHashWorkflowBuildsIndependently pins the workflow's
  unconditional `nix build .#default` between the script and the
  match-report and push steps.

The scenario matrix, operator-backup and file-mode fixture, and workflow
ordering check are adapted from BlueboxC's correction of this PR in their
fork (62e7718e), as described on #7498 and #7529.

Refs #7504, #7496, #7498, #7499, #7500, #7529

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-authored-by: BlueboxC <david@blueboxcomputers.es>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: BlueboxC <david@blueboxcomputers.es>
2026-10-14 22:18:15 +00:00

296 lines
12 KiB
Bash
Executable file

#!/bin/bash
set -e
# =============================================================================
# UPDATE NIX VENDORHASH SCRIPT
# =============================================================================
#
# Automatically updates the vendorHash in default.nix after go.mod changes.
# This script:
# 1. Sets vendorHash to a known-bad hash to trigger an error
# 2. Runs nix build to get the actual hash from the error message
# 3. Extracts the correct hash from the error
# 4. Updates default.nix with the correct hash
# 5. Verifies the update by rebuilding
#
# Requirements:
# - Either nix installed locally, OR
# - Docker (will automatically use nixos/nix Docker image)
#
# Usage:
# ./scripts/update-nix-vendorhash.sh
#
# =============================================================================
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
log_info() {
echo -e "${BLUE}==>${NC} $1"
}
log_success() {
echo -e "${GREEN}✓${NC} $1"
}
log_warning() {
echo -e "${YELLOW}⚠${NC} $1"
}
log_error() {
echo -e "${RED}✗${NC} $1" >&2
}
# Check if we're in the repo root
if [ ! -f "default.nix" ]; then
log_error "Must run from repository root (default.nix not found)"
exit 1
fi
# Determine if we should use Docker for Nix
USE_DOCKER=false
NIX_CMD="nix"
if ! command -v nix &> /dev/null; then
log_warning "nix command not found locally"
# Check if Docker is available
if command -v docker &> /dev/null; then
log_info "Docker detected - will use nixos/nix Docker image"
USE_DOCKER=true
# Pull the Nix Docker image if not present
if ! docker image inspect nixos/nix:latest &> /dev/null; then
log_info "Pulling nixos/nix:latest Docker image..."
docker pull nixos/nix:latest
fi
else
log_error "Neither nix nor docker found."
log_error "Please install one of:"
log_error " - Nix: https://nixos.org/download.html"
log_error " - Docker: https://docs.docker.com/get-docker/"
exit 1
fi
fi
# Function to run nix commands (either locally or via Docker)
run_nix() {
if [ "$USE_DOCKER" = true ]; then
# Get absolute path of current directory
ABS_PWD=$(cd "$(pwd)" && pwd)
# Run nix in Docker container with current directory mounted
# Mount at both /workspace and the original path (for git worktree compatibility)
DOCKER_VOLUMES="-v $ABS_PWD:/workspace -v $ABS_PWD:$ABS_PWD"
# If this is a git worktree, mount the main git directory too
if [ -f .git ] && grep -q "^gitdir: " .git; then
GIT_COMMON_DIR=$(git rev-parse --git-common-dir 2>/dev/null || echo "")
if [ -n "$GIT_COMMON_DIR" ] && [ -d "$GIT_COMMON_DIR" ]; then
# Mount the main git directory at the same path
DOCKER_VOLUMES="$DOCKER_VOLUMES -v $GIT_COMMON_DIR:$GIT_COMMON_DIR"
fi
fi
docker run --rm \
$DOCKER_VOLUMES \
-w /workspace \
nixos/nix:latest \
nix "$@" --extra-experimental-features "nix-command flakes"
else
nix "$@"
fi
}
log_info "Updating vendorHash in default.nix..."
echo ""
# Get current vendorHash
CURRENT_HASH=$(grep 'vendorHash = ' default.nix | sed 's/.*"\(.*\)".*/\1/')
log_info "Current vendorHash: $CURRENT_HASH"
# Step 1: Set to a known-bad hash to trigger an error
log_info "Setting temporary bad hash to trigger error..."
FAKE_HASH="sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
# Back up to a private per-run scratch directory, never to a fixed name in
# the repo root: `default.nix.backup` there is also the conventional name an
# operator gives their own manual backup, and overwriting (then later
# deleting) that file on every run can destroy one on sight. The scratch
# dir is removed on every exit path, handled or not; a handled failure
# restores from it first via restore_original.
SCRATCH_DIR=$(mktemp -d "${TMPDIR:-/tmp}/nix-vendorhash.XXXXXX")
BACKUP="$SCRATCH_DIR/default.nix.orig"
# -p preserves the exact original bytes and mode; a handled failure must
# restore both, not just the content.
cp -p default.nix "$BACKUP"
restore_original() {
cp -p "$BACKUP" default.nix
}
# A SIGINT/SIGTERM during the long steps (the nix builds) reaches this process
# and its nix child as one group; the child dies, and this script exits
# through the EXIT trap with default.nix still holding the placeholder hash
# (probe build) or an unverified candidate hash (step 5). Plain
# `rm -rf "$SCRATCH_DIR"` there would delete the only backup while leaving
# either in place -- worse than every handled failure, since recovery becomes
# `git checkout default.nix`, which also discards any uncommitted operator
# edits. Neither is a valid end state: VERIFIED is set only once a build has
# verified default.nix as it stands, so restore the original whenever the run
# ends before that. Every handled exit below has already restored or verified
# default.nix by the time this runs, so it is a no-op on every path except an
# interrupted one.
VERIFIED=false
cleanup() {
if [ "$VERIFIED" != true ] && [ -f "$BACKUP" ] && ! cmp -s "$BACKUP" default.nix; then
restore_original
log_info "Restored original default.nix (run interrupted)"
fi
rm -rf "$SCRATCH_DIR"
}
trap cleanup EXIT
# Detect sed version (GNU vs BSD). BSD sed requires an explicit backup-suffix
# argument to -i; GNU sed must NOT be given one.
#
# This is a function rather than a SED_INPLACE string because a string cannot
# carry an EMPTY argument. After `SED_INPLACE="sed -i ''"` is word-split the two
# quote characters survive as a literal argument, so BSD sed reads '' as the
# backup SUFFIX and writes `default.nix''` beside the real file on every macOS
# run. Nothing ignores that stray, and it holds the placeholder
# sha256-AAAA... hash, so a `git add -A` can commit a vendorHash that never
# validates.
sed_inplace() {
if sed --version 2>/dev/null | grep -q "GNU sed"; then
sed -i "$@"
else
sed -i '' "$@"
fi
}
# Update to fake hash
sed_inplace "s|vendorHash = \".*\";|vendorHash = \"$FAKE_HASH\";|" default.nix
# Step 2: Run nix build and capture the error
log_info "Building to get actual hash (this will fail intentionally)..."
# Per-run log names under ${TMPDIR:-/tmp}, like the scratch dir: a fixed name
# in a shared /tmp collides with another run's (or another user's) log. They
# live outside the scratch dir because a failure keeps the logs it names.
BUILD_LOG="${TMPDIR:-/tmp}/nix-vendorhash-build-$$.log"
VERIFY_LOG="${TMPDIR:-/tmp}/nix-vendorhash-verify-$$.log"
# This build is expected to fail (the fake hash above is designed to trigger
# a hash-mismatch error), so capture its real exit status explicitly instead
# of `|| true`, which would discard it. The exit status (not a "hash
# mismatch" string grep, which can't tell a build that failed for an
# unrelated reason from one that actually succeeded) is what step 3 below
# uses to tell "no update needed" apart from "the build is broken".
set +e
run_nix build > "$BUILD_LOG" 2>&1
BUILD_EXIT=$?
set -e
# Step 3: Extract the actual hash from the error message
# Error format: "error: hash mismatch ... got: sha256-ACTUAL_HASH" (note: multiple spaces)
# Use portable grep + sed instead of grep -P (which requires GNU grep)
ACTUAL_HASH=$(grep -o 'got:[[:space:]]*sha256-[A-Za-z0-9+/=]*' "$BUILD_LOG" | head -1 | sed 's/got:[[:space:]]*//')
if [ -z "$ACTUAL_HASH" ]; then
log_error "Failed to extract hash from nix build output"
log_error "This might mean:"
log_error " - Go dependencies haven't changed (no update needed)"
log_error " - There's a different build error"
echo ""
log_info "Build output (last 30 lines):"
tail -30 "$BUILD_LOG"
echo ""
log_info "Full build log saved to: $BUILD_LOG"
# Restore backup
restore_original
log_info "Restored original default.nix"
# Tell "the build genuinely succeeded with the fake hash rejected, so
# the real hash is already correct" apart from "the build failed for an
# unrelated reason and never got far enough to report a hash mismatch"
# by the build's own exit status, not by whether "hash mismatch" happens
# to appear in the log — a toolchain error or any other failure mode
# has no such string and was previously misreported as success.
if [ "$BUILD_EXIT" -eq 0 ]; then
# The probe build's success only means the FAKE hash didn't trigger
# a mismatch — it was never a build of the real, restored file. A
# success exit must actually build that file, not just infer it
# from the probe.
log_info "No hash mismatch in the probe build - verifying the restored file builds as-is..."
RESTORE_BUILD_LOG="${TMPDIR:-/tmp}/nix-vendorhash-restore-build-$$.log"
set +e
run_nix build > "$RESTORE_BUILD_LOG" 2>&1
RESTORE_BUILD_EXIT=$?
set -e
if [ "$RESTORE_BUILD_EXIT" -eq 0 ]; then
VERIFIED=true
log_success "No hash mismatch - vendorHash is already correct!"
rm -f "$BUILD_LOG" "$RESTORE_BUILD_LOG"
exit 0
else
log_error "The probe build reported no hash mismatch, but the restored file doesn't build either (exit $RESTORE_BUILD_EXIT) — investigate before retrying"
log_error "Check the restored-file build log: $RESTORE_BUILD_LOG"
log_error "Probe build log: $BUILD_LOG"
exit 1
fi
elif grep -q "hash mismatch" "$BUILD_LOG"; then
log_error "Build reported a hash mismatch (exit $BUILD_EXIT) but the got: line didn't match the extraction regex — Nix's output format may have changed"
log_error "Check the full log: $BUILD_LOG"
exit 1
else
log_error "Build failed for a reason other than a hash mismatch (exit $BUILD_EXIT)"
log_error "Check the full log: $BUILD_LOG"
exit 1
fi
fi
log_success "Extracted actual hash: $ACTUAL_HASH"
# Step 4: Update default.nix with the correct hash
log_info "Updating default.nix with correct hash..."
sed_inplace "s|vendorHash = \".*\";|vendorHash = \"$ACTUAL_HASH\";|" default.nix
# $BACKUP is kept until step 5 verifies: restore_original below needs it if
# the verification build fails, and the EXIT trap if that build is
# interrupted. The trap removes the scratch dir either way.
# Step 5: Verify the update by rebuilding
log_info "Verifying update with a clean build..."
# Without pipefail, this if-condition's status is tee's (which basically
# always succeeds), not run_nix build's, so a real build failure here was
# reported as "Build successful!". Scoped to just this if (turned back off
# right after) rather than set for the rest of the script: the ACTUAL_HASH
# extraction above relies on `grep -o` legitimately returning non-zero on no
# match, which pipefail would turn into a script abort under `set -e`.
set -o pipefail
if run_nix build 2>&1 | tee "$VERIFY_LOG"; then
set +o pipefail
VERIFIED=true
log_success "Build successful! vendorHash updated correctly."
echo ""
log_info "Summary:"
echo " Old hash: $CURRENT_HASH"
echo " New hash: $ACTUAL_HASH"
echo ""
log_success "default.nix has been updated"
# Clean up build logs
rm -f "$BUILD_LOG" "$VERIFY_LOG"
else
set +o pipefail
log_error "Build failed after updating hash"
log_error "See $VERIFY_LOG for details"
log_error "Initial build log: $BUILD_LOG"
restore_original
log_info "Restored original default.nix (candidate hash did not verify)"
exit 1
fi