beads/scripts/test_script_test.go
Julian Knutsen c1fc671fcf
ci: cross-build Windows test binaries on Linux (advisory side-by-side), drop PR macOS full suite (F4) (#7135)
* ci(F4): advisory Linux-cross-built Windows test binaries, macOS cache restore

Side-by-side rollout slice of the PR CI -> 5 min plan (F4):

- Add pr.yml `windows-test-binaries`: a Linux job that cross-compiles
  bd.exe and two go-test-c binaries (cgo and nocgo) for windows/amd64
  via mingw (CC/CXX=x86_64-w64-mingw32-*, matching release.yml's
  goreleaser setup), driven by a committed manifest
  (scripts/ci/windows-test-binaries.txt) and build script
  (scripts/ci/build-windows-test-binaries.sh). Artifacts are uploaded
  and consumed by two new advisory "-prebuilt" twins
  (test-windows-liveness-prebuilt, worktree-remove-windows-prebuilt)
  that run the identical test selection/flags/env (including
  BEADS_TEST_REPO_ROOT) against the prebuilt binaries via
  scripts/ci/run-go-test-binary.sh and scripts/test.sh's new
  BEADS_TEST_PREBUILT_TEST_BINARY short-circuit.
- The original native Windows jobs are untouched and remain required;
  the new jobs are advisory only (not in CI_GATE_REQUIRED) for this
  rollout phase. Flipping to the cross-built path later is a one-line
  CI_GATE_REQUIRED change.
- Add main.yml `windows-test-binaries-cache`: seeds the Go build cache
  for the cross-compile toolchain/packages on a schedule, mirroring
  the existing module/build cache warmers; reorder build-artifacts'
  warm-cache step to precede the module-cache save for a cleaner
  build -> warm -> save grouping.
- Restore macOS's native build-cache save step in main.yml (lost
  during an earlier job consolidation).
- Remove the non-required "Test (macos-latest)" full-suite job from
  PRs; the native macOS gate jobs still run, so required coverage is
  unaffected. This does drop a non-required pre-merge macOS race
  signal; see docs/notes for the accepted tradeoff.
- Add scripts/windows_test_binaries_manifest_test.go: policy tests
  pinning the manifest's three advisory-phase entries, pinning the
  mingw toolchain to release.yml's, checking consumer jobs only
  reference manifest-listed binaries, checking upload/download
  artifact names and `needs` match, checking the prebuilt jobs select
  exactly the same tests (-run literals) as their native counterparts,
  and checking none of the new jobs reference secrets (fork safety).
  All mutation-tested.
- Add two scripts/ci_workflow_apt_get_timeout_test.go budget entries
  for the new "Install mingw cross-compiler" steps.
- Fix two latent Bazel-only scripts_test failures surfaced while
  wiring the new test file into scripts/BUILD.bazel: materialize
  cmd/bd's sources via the existing //cmd/bd:go_srcs filegroup so
  sourceRepoRoot(t)-based package-directory checks work under Bazel's
  runfiles root, and strip Bazel's own TEST_TIMEOUT env var before
  exercising scripts/test.sh's hardcoded default in
  test_script_test.go (name collision with test.sh's own override
  knob).

Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint; shellcheck; make ci-pr-policy;
make bazel-sync-check; a from-scratch /tmp clone cross-build using
the exact job commands (cold 130.2s, warm/cache-hit 30.3s, PE32+
x86-64 validity via file/objdump, byte-identical SHA-256 across
cold/warm/traced runs); wine -test.list confirms
TestPrimeBinaryPortfolio/TestIsServerProbablyRunning* present in the
cgo binary and the full TestWorktreeRemove* family (including
Windows-specific variants) present in the nocgo binary.

Scoping note: only 2 of the spec's 4 consumer jobs (liveness,
worktree-remove) get prebuilt twins, and only 3 of 8 spec-listed
manifest binaries are built this slice; preflight, doc-freshness and
credential-fixture consumers are deferred as documented follow-up.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci): address F4 review findings (SF-1..SF-6 + nits)

Addresses the F4 review (~/beads-bazel-plan/f4/review.md): no blockers, six
substantive findings plus several nits.

- SF-1: main.yml's windows-test-binaries-cache job now seeds the Windows
  cross-compile cache from both runner selections pr.yml's consumer job can
  land on (Blacksmith for same-repo PRs/merge_group, ubuntu-latest for
  forks/Dependabot) via a 2-leg matrix with identical keys/paths, instead of
  a single ubuntu-latest leg Blacksmith runners can never see.
- SF-2: removed the two consumer-less "Warm non-race Go build cache
  (test-only packages)" steps from main.yml (build-artifacts, test macOS
  leg); they had no real consumer and cost ~270s on main's macOS leg for
  nothing.
- SF-3: scripts/ci/run-go-test-binary.sh now injects the same
  -test.timeout=10m default (when -timeout is omitted) and unconditional
  -test.paniconexit0 flag `go test` itself injects; added
  timeout-minutes to test-windows-liveness-prebuilt; added a direct
  regression test (TestRunGoTestBinaryDefaultsTimeoutWhenOmitted) for the
  omitted-timeout branch, which no path through scripts/test.sh can reach
  (test.sh always supplies its own -timeout default first).
- SF-4: scripts/test.sh's BEADS_TEST_PREBUILT_TEST_BINARY branch now runs
  run-go-test-binary.sh as a child process (capturing and re-exiting its
  status) instead of `exec`-ing it, so the EXIT trap (beads_test_env_cleanup)
  still runs and the per-run BEADS_TEST_ENV_ROOT temp dir (and, with
  BEADS_TEST_SHARED_SERVER=1, the shared dolt sql-server) is cleaned up
  instead of leaked/orphaned.
- SF-5: closed the five surviving mutations (cgo/nocgo swap between the
  Windows prebuilt twins, dropped -count=1, an unguarded twin in
  ci-gate.needs, a goreleaser compiler change, a PKGDIR change) with
  dedicated policy tests, and replaced the three-edits-per-job required/
  advisory toggle with a single committed WINDOWS_PREBUILT_REQUIRED flag
  (pr.yml top-level env) in the style of the existing BAZEL_RETIRES_LEGACY_*
  flags: both Windows pairs are now unconditionally in ci-gate.needs and
  always evaluated, and only which pair lands in CI_GATE_REQUIRED vs.
  CI_GATE_SKIPPED_OK is decided by the flag, in one place.
- SF-6: dropped the //cmd/bd:go_srcs data dep from scripts_test; the
  prebuilt-binary contract test now exercises ./scripts/ci (already a data
  dep of this target) instead of ./cmd/bd, since it never builds or runs the
  real package.
- Nits: removed dangling ~/beads-bazel-plan references and a nonexistent
  test name from committed comments; log the resolved mingw-w64 version
  when cross-compiling; removed a no-op shellcheck disable comment;
  documented why run-go-test-binary.sh's BEADS_TEST_REPO_ROOT re-export
  doesn't race beads_test_env_enter's unset; rewrote the pr.yml tombstone
  comment for the removed macOS PR job to state the actual, narrower
  tradeoff instead of an overclaiming "main.yml covers this" statement.

Verified: bazel --nohome_rc test //scripts:scripts_test passes; actionlint
clean on pr.yml/main.yml; shellcheck clean on all edited scripts (only
pre-existing, unrelated warnings remain elsewhere); make bazel-sync-check
and make ci-pr-policy both pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(scripts): isolate prebuilt-binary contract test from ambient hermetic env

TestTestScriptPrebuiltTestBinaryContract's "cleans up BEADS_TEST_ENV_ROOT
after the binary exits" subtest (added for F4 review finding SF-4) passed
locally but failed in CI's scripts-go-checks/scripts-test leg.

scripts/ci/scripts-go-test.sh (the CI driver for this package's `go test
./scripts/...` run) calls beads_test_env_enter in its own shell before
invoking go test, so BEADS_TEST_ENV_ACTIVE=1 and an outer
BEADS_TEST_ENV_ROOT are already present in this test binary's own
os.Environ(). runTestScriptPrebuiltBinary built the scripts/test.sh
subprocess's env from os.Environ() (minus TEST_TIMEOUT), so that ambient
state leaked through. Inside the subprocess, scripts/test.sh's own
beads_test_env_enter call (test.sh:17) saw the already-active guard and
no-opped: no new root, no new EXIT trap armed in that process - so the
fake binary observed the outer, still-alive root, and the subtest's
"directory is gone after test.sh exits" assertion failed, even though the
SF-4 fix (run the prebuilt binary as a child instead of exec'ing it) is
correct.

Every real production caller of scripts/test.sh's prebuilt path (pr.yml's
Windows liveness/worktree-remove prebuilt steps) runs it as the first
command of a fresh GitHub Actions `run:` step, never nested under an
already-entered hermetic env, so this nesting is purely an artifact of
this test binary's own ambient environment under scripts-go-test.sh, not
a real CI behavior gap.

Strip BEADS_TEST_ENV_ACTIVE/_ROOT/_DISABLE/_KEEP (alongside the existing
TEST_TIMEOUT filter) from the child env in runTestScriptPrebuiltBinary so
the subtest deterministically exercises scripts/test.sh's real top-level,
non-nested invocation regardless of what ambient hermetic-env state this
test binary itself happens to be running under.

Verified: reproduced the CI failure locally by exporting
BEADS_TEST_ENV_ACTIVE=1/BEADS_TEST_ENV_ROOT before `go test ./scripts/...`;
confirmed the fix resolves it; re-ran bazel test //scripts:scripts_test
and scripts/ci/scripts-go-test.sh directly, both pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-03 11:49:56 -07:00

514 lines
18 KiB
Go

package scripts_test
import (
"io"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"testing"
)
const (
testScriptFakeGoLogEnv = "BEADS_TEST_SCRIPT_FAKE_GO_LOG"
testScriptExpectedBinaryEnv = "BEADS_TEST_SCRIPT_EXPECTED_BINARY"
testScriptExpectedBaseEnv = "BEADS_TEST_SCRIPT_EXPECTED_BASENAME"
testScriptDriverEnv = "BEADS_TEST_SCRIPT_DRIVER"
testScriptNativeSuffixEnv = "BEADS_TEST_SCRIPT_NATIVE_SUFFIX"
testScriptLaunchProbeEnv = "BEADS_TEST_SCRIPT_LAUNCH_PROBE"
)
const testScriptFakeGo = `#!/usr/bin/env bash
set -euo pipefail
record() {
printf '%s\n' "$1" >>"$BEADS_TEST_SCRIPT_FAKE_GO_LOG"
}
case "${1:-}" in
env)
record env
if [[ $# -ne 2 || "$2" != "GOEXE" ]]; then
printf 'fake go: unsupported env arguments: %s\n' "$*" >&2
exit 90
fi
printf '%s\n' "$BEADS_TEST_SCRIPT_NATIVE_SUFFIX"
;;
build)
record build
shift
output=""
while [[ $# -gt 0 ]]; do
if [[ "$1" == "-o" ]]; then
if [[ $# -lt 2 ]]; then
printf 'fake go: -o is missing its output\n' >&2
exit 90
fi
output="$2"
shift 2
else
shift
fi
done
if [[ -z "$output" || "$output" != "$BEADS_TEST_SCRIPT_EXPECTED_BINARY" ]]; then
printf 'fake go: build output %q, want %q\n' "$output" "$BEADS_TEST_SCRIPT_EXPECTED_BINARY" >&2
exit 90
fi
cp -f -- "$BEADS_TEST_SCRIPT_DRIVER" "$output"
chmod +x "$output"
;;
test)
record test
"$BEADS_TEST_SCRIPT_DRIVER" \
-test.run '^TestTestScriptPrebuiltBinaryLaunchProbe$' \
-test.count=1
;;
*)
printf 'fake go: unsupported command: %s\n' "$*" >&2
exit 90
;;
esac
`
func TestTestScriptPrebuiltBinaryContract(t *testing.T) {
t.Run("generated path uses the native executable suffix and launches", func(t *testing.T) {
commands := runTestScriptWithFakeGo(t, "")
assertFakeGoCommands(t, commands, "env", "build", "test")
})
t.Run("caller supplied binary wins without a build", func(t *testing.T) {
fixtureRoot := filepath.Join(t.TempDir(), "caller override with spaces")
if err := os.MkdirAll(fixtureRoot, 0o755); err != nil {
t.Fatalf("create caller fixture root: %v", err)
}
callerBinary := filepath.Join(fixtureRoot, "caller supplied bd"+nativeExecutableSuffix())
copyCurrentTestExecutable(t, callerBinary)
commands := runTestScriptWithFakeGo(t, callerBinary)
assertFakeGoCommands(t, commands, "test")
})
}
// TestTestScriptPrebuiltBinaryLaunchProbe is selected only by the fake go test
// process above. Keeping the os/exec probe in a normal test avoids claiming the
// package-wide TestMain authority needed by other script-selection contracts.
func TestTestScriptPrebuiltBinaryLaunchProbe(t *testing.T) {
if os.Getenv(testScriptLaunchProbeEnv) != "1" {
t.Skip("re-exec probe runs only under the test.sh fake-go driver")
}
prebuilt := os.Getenv("BEADS_TEST_BD_BINARY")
expected := os.Getenv(testScriptExpectedBinaryEnv)
if prebuilt == "" || expected == "" || !sameTestScriptFile(prebuilt, expected) {
t.Fatalf("exported prebuilt binary %q is not expected file %q", prebuilt, expected)
}
if want := os.Getenv(testScriptExpectedBaseEnv); filepath.Base(prebuilt) != want {
t.Fatalf("exported prebuilt basename = %q, want %q", filepath.Base(prebuilt), want)
}
command := exec.Command(prebuilt, "-test.run=^$")
output, err := command.CombinedOutput()
if err != nil {
t.Fatalf("launch exported prebuilt binary through os/exec: %v\n%s", err, output)
}
}
func runTestScriptWithFakeGo(t *testing.T, callerBinary string) []string {
t.Helper()
root := filepath.Join(t.TempDir(), "test script root with spaces")
fakeBin := filepath.Join(root, "fake go bin")
testEnvRoot := filepath.Join(root, "isolated test environment")
tempRoot := filepath.Join(root, "temporary files")
for _, path := range []string{fakeBin, testEnvRoot, tempRoot} {
if err := os.MkdirAll(path, 0o755); err != nil {
t.Fatalf("create fixture directory %s: %v", path, err)
}
}
fakeGo := filepath.Join(fakeBin, "go")
if err := os.WriteFile(fakeGo, []byte(testScriptFakeGo), 0o755); err != nil {
t.Fatalf("write fake go: %v", err)
}
callLog := filepath.Join(root, "fake go calls")
if err := os.WriteFile(callLog, nil, 0o600); err != nil {
t.Fatalf("initialize fake-go call log: %v", err)
}
expected := callerBinary
if expected == "" {
expected = filepath.Join(testEnvRoot, "prebuilt-bd", "bd"+nativeExecutableSuffix())
}
bash, err := exec.LookPath("bash")
if err != nil {
t.Fatalf("bash is required to exercise scripts/test.sh: %v", err)
}
repoRoot := sourceRepoRoot(t)
env := testScriptEnvironment(testEnvRoot, tempRoot, expected, callerBinary)
fakeBinShellPath := shellPathUnderEnv(t, bash, fakeBin, env)
fakeGoShellPath := shellPathUnderEnv(t, bash, fakeGo, env)
driverShellPath := shellPathUnderEnv(t, bash, currentTestExecutable(t), env)
callLogShellPath := shellPathUnderEnv(t, bash, callLog, env)
env = append(env,
"BEADS_TEST_COMMAND_PATH="+fakeBinShellPath+":/usr/bin:/bin",
testScriptDriverEnv+"="+driverShellPath,
testScriptFakeGoLogEnv+"="+callLogShellPath,
)
cmd := exec.Command(
bash,
"--noprofile",
"--norc",
"-c",
`PATH="$BEADS_TEST_COMMAND_PATH"; export PATH; exec "$BASH" --noprofile --norc "$1" "$2"`,
"test-script",
shellPathUnderEnv(t, bash, filepath.Join(repoRoot, "scripts", "test.sh"), env),
"./cmd/bd",
)
cmd.Dir = repoRoot
cmd.Env = env
requireShellCommandPath(t, bash, repoRoot, env, "go", fakeGoShellPath)
output, runErr := cmd.CombinedOutput()
if runErr != nil {
t.Fatalf("scripts/test.sh failed: %v\n%s", runErr, output)
}
content, err := os.ReadFile(callLog)
if err != nil {
t.Fatalf("read fake-go call log: %v", err)
}
return strings.Fields(string(content))
}
func testScriptEnvironment(testEnvRoot string, tempRoot string, expected string, callerBinary string) []string {
home := filepath.Join(testEnvRoot, "home")
env := []string{
"PATH=/usr/bin:/bin",
"HOME=" + portableTestScriptPath(home),
"USERPROFILE=" + portableTestScriptPath(home),
"TMPDIR=" + portableTestScriptPath(tempRoot),
"TEMP=" + portableTestScriptPath(tempRoot),
"TMP=" + portableTestScriptPath(tempRoot),
"LC_ALL=C",
"LANG=C",
"BASH_ENV=",
"ENV=",
"CGO_ENABLED=1",
"GOFLAGS=",
"BEADS_TEST_ENV_ACTIVE=1",
"BEADS_TEST_ENV_ROOT=" + portableTestScriptPath(testEnvRoot),
testScriptExpectedBinaryEnv + "=" + portableTestScriptPath(expected),
testScriptExpectedBaseEnv + "=" + filepath.Base(expected),
testScriptNativeSuffixEnv + "=" + nativeExecutableSuffix(),
testScriptLaunchProbeEnv + "=1",
}
if callerBinary != "" {
env = append(env, "BEADS_TEST_BD_BINARY="+portableTestScriptPath(callerBinary))
}
for _, key := range []string{"SYSTEMROOT", "WINDIR", "COMSPEC", "PATHEXT"} {
if value, ok := os.LookupEnv(key); ok {
env = append(env, key+"="+value)
}
}
return env
}
func assertFakeGoCommands(t *testing.T, commands []string, want ...string) {
t.Helper()
if strings.Join(commands, " ") != strings.Join(want, " ") {
t.Fatalf("fake-go commands = %q, want %q", commands, want)
}
}
func copyCurrentTestExecutable(t *testing.T, destination string) {
t.Helper()
input, err := os.Open(currentTestExecutable(t))
if err != nil {
t.Fatalf("open current test executable: %v", err)
}
defer input.Close()
output, err := os.OpenFile(destination, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o755)
if err != nil {
t.Fatalf("create native test executable: %v", err)
}
if _, err := io.Copy(output, input); err != nil {
_ = output.Close()
t.Fatalf("copy native test executable: %v", err)
}
if err := output.Close(); err != nil {
t.Fatalf("close native test executable: %v", err)
}
}
func currentTestExecutable(t *testing.T) string {
t.Helper()
path, err := os.Executable()
if err != nil {
t.Fatalf("resolve current test executable: %v", err)
}
return path
}
func sameTestScriptFile(first string, second string) bool {
firstInfo, firstErr := os.Stat(first)
secondInfo, secondErr := os.Stat(second)
return firstErr == nil && secondErr == nil && os.SameFile(firstInfo, secondInfo)
}
func nativeExecutableSuffix() string {
if runtime.GOOS == "windows" {
return ".exe"
}
return ""
}
func portableTestScriptPath(path string) string {
return filepath.ToSlash(filepath.Clean(path))
}
// fakePrebuiltTestBinary is a stand-in for a cross-compiled `go test -c`
// binary: it records its own cwd, its BEADS_TEST_REPO_ROOT, and every
// argument it was launched with, then exits 0. The flag-mapping contract it
// exercises (scripts/test.sh -> scripts/ci/run-go-test-binary.sh -> this
// binary) is host-independent, so a bash stand-in is representative even
// though the real artifact is a Windows PE .exe: that artifact's own
// Windows-launch behavior is exercised by the CI job itself, not by this
// policy test.
const fakePrebuiltTestBinary = `#!/usr/bin/env bash
set -euo pipefail
{
printf 'cwd=%s\n' "$PWD"
printf 'repo_root=%s\n' "${BEADS_TEST_REPO_ROOT:-}"
printf 'env_root=%s\n' "${BEADS_TEST_ENV_ROOT:-}"
for a in "$@"; do
printf 'arg=%s\n' "$a"
done
} >>"$BEADS_TEST_SCRIPT_PREBUILT_LOG"
`
const testScriptPrebuiltLogEnv = "BEADS_TEST_SCRIPT_PREBUILT_LOG"
type prebuiltTestBinaryRun struct {
cwd string
repoRoot string
envRoot string
args []string
}
// TestTestScriptPrebuiltTestBinaryContract pins scripts/test.sh's
// BEADS_TEST_PREBUILT_TEST_BINARY mode (F4.4's run-go-test-binary.sh path):
// it must map go-test-style flags to -test.* flags, launch the binary from
// the package directory (the same convention `go test` itself uses), export
// BEADS_TEST_REPO_ROOT, and refuse a request naming more than one package
// (a prebuilt binary is compiled from exactly one package, so there is no
// single binary that could run a multi-package request).
func TestTestScriptPrebuiltTestBinaryContract(t *testing.T) {
if runtime.GOOS == "windows" {
// The bash stand-in above is a Linux/macOS-only harness; see its doc
// comment. The real cross-built .exe is launched by the Windows job,
// not this test.
t.Skip("prebuilt-binary contract is exercised with a bash stand-in on Linux/macOS")
}
t.Run("maps go-test flags to -test.* flags, cwd, and BEADS_TEST_REPO_ROOT", func(t *testing.T) {
repoRoot := sourceRepoRoot(t)
run := runTestScriptWithPrebuiltBinary(t, repoRoot, []string{
"-v", "-timeout", "5m", "-skip", "TestBar", "-run", "TestFoo", "-count", "1", "./scripts/ci",
})
wantArgs := []string{
"-test.timeout", "5m",
"-test.parallel", "4",
"-test.v",
"-test.skip", "TestBar",
"-test.run", "TestFoo",
"-test.count", "1",
"-test.paniconexit0",
}
if strings.Join(run.args, " ") != strings.Join(wantArgs, " ") {
t.Fatalf("args = %v, want %v", run.args, wantArgs)
}
wantCwd := filepath.Join(repoRoot, "scripts", "ci")
if !sameTestScriptDir(t, run.cwd, wantCwd) {
t.Fatalf("cwd = %q, want %q", run.cwd, wantCwd)
}
if !sameTestScriptDir(t, run.repoRoot, repoRoot) {
t.Fatalf("BEADS_TEST_REPO_ROOT = %q, want %q", run.repoRoot, repoRoot)
}
})
t.Run("supports -count=N form alongside -count N", func(t *testing.T) {
repoRoot := sourceRepoRoot(t)
run := runTestScriptWithPrebuiltBinary(t, repoRoot, []string{"-count=1", "./scripts/ci"})
wantArgs := []string{"-test.timeout", "25m", "-test.parallel", "4", "-test.count", "1", "-test.paniconexit0"}
if strings.Join(run.args, " ") != strings.Join(wantArgs, " ") {
t.Fatalf("args = %v, want %v", run.args, wantArgs)
}
})
t.Run("refuses more than one package", func(t *testing.T) {
repoRoot := sourceRepoRoot(t)
output, err := runTestScriptWithPrebuiltBinaryExpectFailure(t, repoRoot, []string{"./scripts/ci", "./scripts"})
if err == nil {
t.Fatalf("expected scripts/test.sh to fail for two packages, output:\n%s", output)
}
if !strings.Contains(string(output), "requires exactly one package") {
t.Fatalf("expected a single-package error, got:\n%s", output)
}
})
t.Run("cleans up BEADS_TEST_ENV_ROOT after the binary exits (no exec short-circuit)", func(t *testing.T) {
// Regression test for F4 review SF-4: scripts/test.sh's prebuilt-binary
// branch used to `exec` run-go-test-binary.sh, which replaces the
// shell process image before the EXIT trap armed by
// beads_test_env_enter (beads_test_env_cleanup) ever runs - leaking
// the per-run BEADS_TEST_ENV_ROOT temp directory (and, with
// BEADS_TEST_SHARED_SERVER=1, potentially orphaning a shared dolt
// sql-server process). Running the binary as a child and exiting with
// its captured status instead lets the trap fire normally.
repoRoot := sourceRepoRoot(t)
run := runTestScriptWithPrebuiltBinary(t, repoRoot, []string{"-count=1", "./scripts/ci"})
if run.envRoot == "" {
t.Fatal("fake prebuilt binary never observed BEADS_TEST_ENV_ROOT; cannot assert cleanup")
}
if _, err := os.Stat(run.envRoot); !os.IsNotExist(err) {
t.Fatalf("BEADS_TEST_ENV_ROOT %q still exists after scripts/test.sh exited (cleanup trap did not run); stat err = %v", run.envRoot, err)
}
})
}
func runTestScriptWithPrebuiltBinary(t *testing.T, repoRoot string, args []string) prebuiltTestBinaryRun {
t.Helper()
output, logContent, err := runTestScriptPrebuiltBinary(t, repoRoot, args)
if err != nil {
t.Fatalf("scripts/test.sh failed: %v\n%s", err, output)
}
run := prebuiltTestBinaryRun{}
for _, line := range strings.Split(strings.TrimRight(logContent, "\n"), "\n") {
switch {
case strings.HasPrefix(line, "cwd="):
run.cwd = strings.TrimPrefix(line, "cwd=")
case strings.HasPrefix(line, "repo_root="):
run.repoRoot = strings.TrimPrefix(line, "repo_root=")
case strings.HasPrefix(line, "env_root="):
run.envRoot = strings.TrimPrefix(line, "env_root=")
case strings.HasPrefix(line, "arg="):
run.args = append(run.args, strings.TrimPrefix(line, "arg="))
}
}
if run.cwd == "" {
t.Fatalf("fake prebuilt binary never ran; scripts/test.sh output:\n%s\nlog:\n%s", output, logContent)
}
return run
}
func runTestScriptWithPrebuiltBinaryExpectFailure(t *testing.T, repoRoot string, args []string) ([]byte, error) {
t.Helper()
output, _, err := runTestScriptPrebuiltBinary(t, repoRoot, args)
return output, err
}
// filterEnv returns environ with any "key=..." entry for the given keys
// removed, so a caller can force a subprocess to fall back to its own
// hardcoded default instead of inheriting the host/sandbox's value.
func filterEnv(environ []string, keys ...string) []string {
drop := make(map[string]bool, len(keys))
for _, k := range keys {
drop[k] = true
}
filtered := make([]string, 0, len(environ))
for _, kv := range environ {
name, _, ok := strings.Cut(kv, "=")
if ok && drop[name] {
continue
}
filtered = append(filtered, kv)
}
return filtered
}
func runTestScriptPrebuiltBinary(t *testing.T, repoRoot string, args []string) (output []byte, logContent string, err error) {
t.Helper()
root := t.TempDir()
fakeBin := filepath.Join(root, "fake-prebuilt-bd-cgo.test")
if err := os.WriteFile(fakeBin, []byte(fakePrebuiltTestBinary), 0o755); err != nil {
t.Fatalf("write fake prebuilt test binary: %v", err)
}
logPath := filepath.Join(root, "fake-prebuilt.log")
if err := os.WriteFile(logPath, nil, 0o600); err != nil {
t.Fatalf("initialize fake prebuilt binary log: %v", err)
}
bash, lookErr := exec.LookPath("bash")
if lookErr != nil {
t.Fatalf("bash is required to exercise scripts/test.sh: %v", lookErr)
}
// Bazel's test runner sets its own TEST_TIMEOUT in the sandbox env
// (seconds, e.g. "300" for the default moderate size) - the same name
// scripts/test.sh reads to override its "25m" default. Strip it so the
// "-count=N form" subtest's assertion on that hardcoded default is not
// at the mercy of whatever test size this target happens to run under;
// go test does not set this variable, so only Bazel needs the filter.
//
// scripts/ci/scripts-go-test.sh (the CI driver for this very package's
// `go test ./scripts/...` run) calls beads_test_env_enter in its own
// shell before invoking go test, so this process's own os.Environ() can
// already carry an inherited BEADS_TEST_ENV_ACTIVE=1 / BEADS_TEST_ENV_ROOT
// / BEADS_TEST_ENV_DISABLE / BEADS_TEST_ENV_KEEP from that outer,
// longer-lived shell. If left in the child env below, scripts/test.sh's
// own beads_test_env_enter call (scripts/test.sh:17) sees the
// already-active guard and no-ops: no new root, no new EXIT trap - so the
// directory observed by the fake binary is the outer root, which is
// still alive (and rightly so: it's owned by the outer process) when
// scripts/test.sh exits. That makes the "cleans up BEADS_TEST_ENV_ROOT"
// subtest below fail only when this test binary happens to run nested
// under scripts-go-test.sh, even though the SF-4 fix it is guarding is
// correct. Every real production caller of scripts/test.sh's prebuilt
// path (pr.yml's Windows liveness/worktree-remove prebuilt steps) runs it
// as the first command of a fresh GitHub Actions `run:` step, never
// nested under an already-entered hermetic env - so strip the hermetic
// env markers here to deterministically exercise that same top-level,
// non-nested invocation regardless of the ambient env this test binary
// itself happens to be running under.
env := append(filterEnv(os.Environ(),
"TEST_TIMEOUT",
"BEADS_TEST_ENV_ACTIVE",
"BEADS_TEST_ENV_ROOT",
"BEADS_TEST_ENV_DISABLE",
"BEADS_TEST_ENV_KEEP",
),
"BEADS_TEST_BD_BINARY=/nonexistent-bd-not-needed-in-prebuilt-test-binary-mode",
"BEADS_TEST_PREBUILT_TEST_BINARY="+fakeBin,
"GITHUB_WORKSPACE="+repoRoot,
testScriptPrebuiltLogEnv+"="+logPath,
)
cmdArgs := append([]string{filepath.Join(repoRoot, "scripts", "test.sh")}, args...)
cmd := exec.Command(bash, cmdArgs...)
cmd.Dir = repoRoot
cmd.Env = env
output, err = cmd.CombinedOutput()
logBytes, readErr := os.ReadFile(logPath)
if readErr != nil {
t.Fatalf("read fake prebuilt binary log: %v", readErr)
}
return output, string(logBytes), err
}
func sameTestScriptDir(t *testing.T, first, second string) bool {
t.Helper()
firstInfo, firstErr := os.Stat(first)
secondInfo, secondErr := os.Stat(second)
if firstErr != nil || secondErr != nil {
return false
}
return os.SameFile(firstInfo, secondInfo)
}