* ci(F4): advisory Linux-cross-built Windows test binaries, macOS cache restore Side-by-side rollout slice of the PR CI -> 5 min plan (F4): - Add pr.yml `windows-test-binaries`: a Linux job that cross-compiles bd.exe and two go-test-c binaries (cgo and nocgo) for windows/amd64 via mingw (CC/CXX=x86_64-w64-mingw32-*, matching release.yml's goreleaser setup), driven by a committed manifest (scripts/ci/windows-test-binaries.txt) and build script (scripts/ci/build-windows-test-binaries.sh). Artifacts are uploaded and consumed by two new advisory "-prebuilt" twins (test-windows-liveness-prebuilt, worktree-remove-windows-prebuilt) that run the identical test selection/flags/env (including BEADS_TEST_REPO_ROOT) against the prebuilt binaries via scripts/ci/run-go-test-binary.sh and scripts/test.sh's new BEADS_TEST_PREBUILT_TEST_BINARY short-circuit. - The original native Windows jobs are untouched and remain required; the new jobs are advisory only (not in CI_GATE_REQUIRED) for this rollout phase. Flipping to the cross-built path later is a one-line CI_GATE_REQUIRED change. - Add main.yml `windows-test-binaries-cache`: seeds the Go build cache for the cross-compile toolchain/packages on a schedule, mirroring the existing module/build cache warmers; reorder build-artifacts' warm-cache step to precede the module-cache save for a cleaner build -> warm -> save grouping. - Restore macOS's native build-cache save step in main.yml (lost during an earlier job consolidation). - Remove the non-required "Test (macos-latest)" full-suite job from PRs; the native macOS gate jobs still run, so required coverage is unaffected. This does drop a non-required pre-merge macOS race signal; see docs/notes for the accepted tradeoff. - Add scripts/windows_test_binaries_manifest_test.go: policy tests pinning the manifest's three advisory-phase entries, pinning the mingw toolchain to release.yml's, checking consumer jobs only reference manifest-listed binaries, checking upload/download artifact names and `needs` match, checking the prebuilt jobs select exactly the same tests (-run literals) as their native counterparts, and checking none of the new jobs reference secrets (fork safety). All mutation-tested. - Add two scripts/ci_workflow_apt_get_timeout_test.go budget entries for the new "Install mingw cross-compiler" steps. - Fix two latent Bazel-only scripts_test failures surfaced while wiring the new test file into scripts/BUILD.bazel: materialize cmd/bd's sources via the existing //cmd/bd:go_srcs filegroup so sourceRepoRoot(t)-based package-directory checks work under Bazel's runfiles root, and strip Bazel's own TEST_TIMEOUT env var before exercising scripts/test.sh's hardcoded default in test_script_test.go (name collision with test.sh's own override knob). Verified: go test ./scripts/...; bazel --nohome_rc test //scripts:scripts_test; actionlint; shellcheck; make ci-pr-policy; make bazel-sync-check; a from-scratch /tmp clone cross-build using the exact job commands (cold 130.2s, warm/cache-hit 30.3s, PE32+ x86-64 validity via file/objdump, byte-identical SHA-256 across cold/warm/traced runs); wine -test.list confirms TestPrimeBinaryPortfolio/TestIsServerProbablyRunning* present in the cgo binary and the full TestWorktreeRemove* family (including Windows-specific variants) present in the nocgo binary. Scoping note: only 2 of the spec's 4 consumer jobs (liveness, worktree-remove) get prebuilt twins, and only 3 of 8 spec-listed manifest binaries are built this slice; preflight, doc-freshness and credential-fixture consumers are deferred as documented follow-up. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(ci): address F4 review findings (SF-1..SF-6 + nits) Addresses the F4 review (~/beads-bazel-plan/f4/review.md): no blockers, six substantive findings plus several nits. - SF-1: main.yml's windows-test-binaries-cache job now seeds the Windows cross-compile cache from both runner selections pr.yml's consumer job can land on (Blacksmith for same-repo PRs/merge_group, ubuntu-latest for forks/Dependabot) via a 2-leg matrix with identical keys/paths, instead of a single ubuntu-latest leg Blacksmith runners can never see. - SF-2: removed the two consumer-less "Warm non-race Go build cache (test-only packages)" steps from main.yml (build-artifacts, test macOS leg); they had no real consumer and cost ~270s on main's macOS leg for nothing. - SF-3: scripts/ci/run-go-test-binary.sh now injects the same -test.timeout=10m default (when -timeout is omitted) and unconditional -test.paniconexit0 flag `go test` itself injects; added timeout-minutes to test-windows-liveness-prebuilt; added a direct regression test (TestRunGoTestBinaryDefaultsTimeoutWhenOmitted) for the omitted-timeout branch, which no path through scripts/test.sh can reach (test.sh always supplies its own -timeout default first). - SF-4: scripts/test.sh's BEADS_TEST_PREBUILT_TEST_BINARY branch now runs run-go-test-binary.sh as a child process (capturing and re-exiting its status) instead of `exec`-ing it, so the EXIT trap (beads_test_env_cleanup) still runs and the per-run BEADS_TEST_ENV_ROOT temp dir (and, with BEADS_TEST_SHARED_SERVER=1, the shared dolt sql-server) is cleaned up instead of leaked/orphaned. - SF-5: closed the five surviving mutations (cgo/nocgo swap between the Windows prebuilt twins, dropped -count=1, an unguarded twin in ci-gate.needs, a goreleaser compiler change, a PKGDIR change) with dedicated policy tests, and replaced the three-edits-per-job required/ advisory toggle with a single committed WINDOWS_PREBUILT_REQUIRED flag (pr.yml top-level env) in the style of the existing BAZEL_RETIRES_LEGACY_* flags: both Windows pairs are now unconditionally in ci-gate.needs and always evaluated, and only which pair lands in CI_GATE_REQUIRED vs. CI_GATE_SKIPPED_OK is decided by the flag, in one place. - SF-6: dropped the //cmd/bd:go_srcs data dep from scripts_test; the prebuilt-binary contract test now exercises ./scripts/ci (already a data dep of this target) instead of ./cmd/bd, since it never builds or runs the real package. - Nits: removed dangling ~/beads-bazel-plan references and a nonexistent test name from committed comments; log the resolved mingw-w64 version when cross-compiling; removed a no-op shellcheck disable comment; documented why run-go-test-binary.sh's BEADS_TEST_REPO_ROOT re-export doesn't race beads_test_env_enter's unset; rewrote the pr.yml tombstone comment for the removed macOS PR job to state the actual, narrower tradeoff instead of an overclaiming "main.yml covers this" statement. Verified: bazel --nohome_rc test //scripts:scripts_test passes; actionlint clean on pr.yml/main.yml; shellcheck clean on all edited scripts (only pre-existing, unrelated warnings remain elsewhere); make bazel-sync-check and make ci-pr-policy both pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(scripts): isolate prebuilt-binary contract test from ambient hermetic env TestTestScriptPrebuiltTestBinaryContract's "cleans up BEADS_TEST_ENV_ROOT after the binary exits" subtest (added for F4 review finding SF-4) passed locally but failed in CI's scripts-go-checks/scripts-test leg. scripts/ci/scripts-go-test.sh (the CI driver for this package's `go test ./scripts/...` run) calls beads_test_env_enter in its own shell before invoking go test, so BEADS_TEST_ENV_ACTIVE=1 and an outer BEADS_TEST_ENV_ROOT are already present in this test binary's own os.Environ(). runTestScriptPrebuiltBinary built the scripts/test.sh subprocess's env from os.Environ() (minus TEST_TIMEOUT), so that ambient state leaked through. Inside the subprocess, scripts/test.sh's own beads_test_env_enter call (test.sh:17) saw the already-active guard and no-opped: no new root, no new EXIT trap armed in that process - so the fake binary observed the outer, still-alive root, and the subtest's "directory is gone after test.sh exits" assertion failed, even though the SF-4 fix (run the prebuilt binary as a child instead of exec'ing it) is correct. Every real production caller of scripts/test.sh's prebuilt path (pr.yml's Windows liveness/worktree-remove prebuilt steps) runs it as the first command of a fresh GitHub Actions `run:` step, never nested under an already-entered hermetic env, so this nesting is purely an artifact of this test binary's own ambient environment under scripts-go-test.sh, not a real CI behavior gap. Strip BEADS_TEST_ENV_ACTIVE/_ROOT/_DISABLE/_KEEP (alongside the existing TEST_TIMEOUT filter) from the child env in runTestScriptPrebuiltBinary so the subtest deterministically exercises scripts/test.sh's real top-level, non-nested invocation regardless of what ambient hermetic-env state this test binary itself happens to be running under. Verified: reproduced the CI failure locally by exporting BEADS_TEST_ENV_ACTIVE=1/BEADS_TEST_ENV_ROOT before `go test ./scripts/...`; confirmed the fix resolves it; re-ran bazel test //scripts:scripts_test and scripts/ci/scripts-go-test.sh directly, both pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
514 lines
18 KiB
Go
514 lines
18 KiB
Go
package scripts_test
|
|
|
|
import (
|
|
"io"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"runtime"
|
|
"strings"
|
|
"testing"
|
|
)
|
|
|
|
const (
|
|
testScriptFakeGoLogEnv = "BEADS_TEST_SCRIPT_FAKE_GO_LOG"
|
|
testScriptExpectedBinaryEnv = "BEADS_TEST_SCRIPT_EXPECTED_BINARY"
|
|
testScriptExpectedBaseEnv = "BEADS_TEST_SCRIPT_EXPECTED_BASENAME"
|
|
testScriptDriverEnv = "BEADS_TEST_SCRIPT_DRIVER"
|
|
testScriptNativeSuffixEnv = "BEADS_TEST_SCRIPT_NATIVE_SUFFIX"
|
|
testScriptLaunchProbeEnv = "BEADS_TEST_SCRIPT_LAUNCH_PROBE"
|
|
)
|
|
|
|
const testScriptFakeGo = `#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
record() {
|
|
printf '%s\n' "$1" >>"$BEADS_TEST_SCRIPT_FAKE_GO_LOG"
|
|
}
|
|
|
|
case "${1:-}" in
|
|
env)
|
|
record env
|
|
if [[ $# -ne 2 || "$2" != "GOEXE" ]]; then
|
|
printf 'fake go: unsupported env arguments: %s\n' "$*" >&2
|
|
exit 90
|
|
fi
|
|
printf '%s\n' "$BEADS_TEST_SCRIPT_NATIVE_SUFFIX"
|
|
;;
|
|
build)
|
|
record build
|
|
shift
|
|
output=""
|
|
while [[ $# -gt 0 ]]; do
|
|
if [[ "$1" == "-o" ]]; then
|
|
if [[ $# -lt 2 ]]; then
|
|
printf 'fake go: -o is missing its output\n' >&2
|
|
exit 90
|
|
fi
|
|
output="$2"
|
|
shift 2
|
|
else
|
|
shift
|
|
fi
|
|
done
|
|
if [[ -z "$output" || "$output" != "$BEADS_TEST_SCRIPT_EXPECTED_BINARY" ]]; then
|
|
printf 'fake go: build output %q, want %q\n' "$output" "$BEADS_TEST_SCRIPT_EXPECTED_BINARY" >&2
|
|
exit 90
|
|
fi
|
|
cp -f -- "$BEADS_TEST_SCRIPT_DRIVER" "$output"
|
|
chmod +x "$output"
|
|
;;
|
|
test)
|
|
record test
|
|
"$BEADS_TEST_SCRIPT_DRIVER" \
|
|
-test.run '^TestTestScriptPrebuiltBinaryLaunchProbe$' \
|
|
-test.count=1
|
|
;;
|
|
*)
|
|
printf 'fake go: unsupported command: %s\n' "$*" >&2
|
|
exit 90
|
|
;;
|
|
esac
|
|
`
|
|
|
|
func TestTestScriptPrebuiltBinaryContract(t *testing.T) {
|
|
t.Run("generated path uses the native executable suffix and launches", func(t *testing.T) {
|
|
commands := runTestScriptWithFakeGo(t, "")
|
|
assertFakeGoCommands(t, commands, "env", "build", "test")
|
|
})
|
|
|
|
t.Run("caller supplied binary wins without a build", func(t *testing.T) {
|
|
fixtureRoot := filepath.Join(t.TempDir(), "caller override with spaces")
|
|
if err := os.MkdirAll(fixtureRoot, 0o755); err != nil {
|
|
t.Fatalf("create caller fixture root: %v", err)
|
|
}
|
|
callerBinary := filepath.Join(fixtureRoot, "caller supplied bd"+nativeExecutableSuffix())
|
|
copyCurrentTestExecutable(t, callerBinary)
|
|
|
|
commands := runTestScriptWithFakeGo(t, callerBinary)
|
|
assertFakeGoCommands(t, commands, "test")
|
|
})
|
|
}
|
|
|
|
// TestTestScriptPrebuiltBinaryLaunchProbe is selected only by the fake go test
|
|
// process above. Keeping the os/exec probe in a normal test avoids claiming the
|
|
// package-wide TestMain authority needed by other script-selection contracts.
|
|
func TestTestScriptPrebuiltBinaryLaunchProbe(t *testing.T) {
|
|
if os.Getenv(testScriptLaunchProbeEnv) != "1" {
|
|
t.Skip("re-exec probe runs only under the test.sh fake-go driver")
|
|
}
|
|
|
|
prebuilt := os.Getenv("BEADS_TEST_BD_BINARY")
|
|
expected := os.Getenv(testScriptExpectedBinaryEnv)
|
|
if prebuilt == "" || expected == "" || !sameTestScriptFile(prebuilt, expected) {
|
|
t.Fatalf("exported prebuilt binary %q is not expected file %q", prebuilt, expected)
|
|
}
|
|
if want := os.Getenv(testScriptExpectedBaseEnv); filepath.Base(prebuilt) != want {
|
|
t.Fatalf("exported prebuilt basename = %q, want %q", filepath.Base(prebuilt), want)
|
|
}
|
|
|
|
command := exec.Command(prebuilt, "-test.run=^$")
|
|
output, err := command.CombinedOutput()
|
|
if err != nil {
|
|
t.Fatalf("launch exported prebuilt binary through os/exec: %v\n%s", err, output)
|
|
}
|
|
}
|
|
|
|
func runTestScriptWithFakeGo(t *testing.T, callerBinary string) []string {
|
|
t.Helper()
|
|
|
|
root := filepath.Join(t.TempDir(), "test script root with spaces")
|
|
fakeBin := filepath.Join(root, "fake go bin")
|
|
testEnvRoot := filepath.Join(root, "isolated test environment")
|
|
tempRoot := filepath.Join(root, "temporary files")
|
|
for _, path := range []string{fakeBin, testEnvRoot, tempRoot} {
|
|
if err := os.MkdirAll(path, 0o755); err != nil {
|
|
t.Fatalf("create fixture directory %s: %v", path, err)
|
|
}
|
|
}
|
|
|
|
fakeGo := filepath.Join(fakeBin, "go")
|
|
if err := os.WriteFile(fakeGo, []byte(testScriptFakeGo), 0o755); err != nil {
|
|
t.Fatalf("write fake go: %v", err)
|
|
}
|
|
callLog := filepath.Join(root, "fake go calls")
|
|
if err := os.WriteFile(callLog, nil, 0o600); err != nil {
|
|
t.Fatalf("initialize fake-go call log: %v", err)
|
|
}
|
|
|
|
expected := callerBinary
|
|
if expected == "" {
|
|
expected = filepath.Join(testEnvRoot, "prebuilt-bd", "bd"+nativeExecutableSuffix())
|
|
}
|
|
|
|
bash, err := exec.LookPath("bash")
|
|
if err != nil {
|
|
t.Fatalf("bash is required to exercise scripts/test.sh: %v", err)
|
|
}
|
|
repoRoot := sourceRepoRoot(t)
|
|
env := testScriptEnvironment(testEnvRoot, tempRoot, expected, callerBinary)
|
|
fakeBinShellPath := shellPathUnderEnv(t, bash, fakeBin, env)
|
|
fakeGoShellPath := shellPathUnderEnv(t, bash, fakeGo, env)
|
|
driverShellPath := shellPathUnderEnv(t, bash, currentTestExecutable(t), env)
|
|
callLogShellPath := shellPathUnderEnv(t, bash, callLog, env)
|
|
env = append(env,
|
|
"BEADS_TEST_COMMAND_PATH="+fakeBinShellPath+":/usr/bin:/bin",
|
|
testScriptDriverEnv+"="+driverShellPath,
|
|
testScriptFakeGoLogEnv+"="+callLogShellPath,
|
|
)
|
|
|
|
cmd := exec.Command(
|
|
bash,
|
|
"--noprofile",
|
|
"--norc",
|
|
"-c",
|
|
`PATH="$BEADS_TEST_COMMAND_PATH"; export PATH; exec "$BASH" --noprofile --norc "$1" "$2"`,
|
|
"test-script",
|
|
shellPathUnderEnv(t, bash, filepath.Join(repoRoot, "scripts", "test.sh"), env),
|
|
"./cmd/bd",
|
|
)
|
|
cmd.Dir = repoRoot
|
|
cmd.Env = env
|
|
requireShellCommandPath(t, bash, repoRoot, env, "go", fakeGoShellPath)
|
|
output, runErr := cmd.CombinedOutput()
|
|
if runErr != nil {
|
|
t.Fatalf("scripts/test.sh failed: %v\n%s", runErr, output)
|
|
}
|
|
|
|
content, err := os.ReadFile(callLog)
|
|
if err != nil {
|
|
t.Fatalf("read fake-go call log: %v", err)
|
|
}
|
|
return strings.Fields(string(content))
|
|
}
|
|
|
|
func testScriptEnvironment(testEnvRoot string, tempRoot string, expected string, callerBinary string) []string {
|
|
home := filepath.Join(testEnvRoot, "home")
|
|
env := []string{
|
|
"PATH=/usr/bin:/bin",
|
|
"HOME=" + portableTestScriptPath(home),
|
|
"USERPROFILE=" + portableTestScriptPath(home),
|
|
"TMPDIR=" + portableTestScriptPath(tempRoot),
|
|
"TEMP=" + portableTestScriptPath(tempRoot),
|
|
"TMP=" + portableTestScriptPath(tempRoot),
|
|
"LC_ALL=C",
|
|
"LANG=C",
|
|
"BASH_ENV=",
|
|
"ENV=",
|
|
"CGO_ENABLED=1",
|
|
"GOFLAGS=",
|
|
"BEADS_TEST_ENV_ACTIVE=1",
|
|
"BEADS_TEST_ENV_ROOT=" + portableTestScriptPath(testEnvRoot),
|
|
testScriptExpectedBinaryEnv + "=" + portableTestScriptPath(expected),
|
|
testScriptExpectedBaseEnv + "=" + filepath.Base(expected),
|
|
testScriptNativeSuffixEnv + "=" + nativeExecutableSuffix(),
|
|
testScriptLaunchProbeEnv + "=1",
|
|
}
|
|
if callerBinary != "" {
|
|
env = append(env, "BEADS_TEST_BD_BINARY="+portableTestScriptPath(callerBinary))
|
|
}
|
|
for _, key := range []string{"SYSTEMROOT", "WINDIR", "COMSPEC", "PATHEXT"} {
|
|
if value, ok := os.LookupEnv(key); ok {
|
|
env = append(env, key+"="+value)
|
|
}
|
|
}
|
|
return env
|
|
}
|
|
|
|
func assertFakeGoCommands(t *testing.T, commands []string, want ...string) {
|
|
t.Helper()
|
|
if strings.Join(commands, " ") != strings.Join(want, " ") {
|
|
t.Fatalf("fake-go commands = %q, want %q", commands, want)
|
|
}
|
|
}
|
|
|
|
func copyCurrentTestExecutable(t *testing.T, destination string) {
|
|
t.Helper()
|
|
input, err := os.Open(currentTestExecutable(t))
|
|
if err != nil {
|
|
t.Fatalf("open current test executable: %v", err)
|
|
}
|
|
defer input.Close()
|
|
|
|
output, err := os.OpenFile(destination, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o755)
|
|
if err != nil {
|
|
t.Fatalf("create native test executable: %v", err)
|
|
}
|
|
if _, err := io.Copy(output, input); err != nil {
|
|
_ = output.Close()
|
|
t.Fatalf("copy native test executable: %v", err)
|
|
}
|
|
if err := output.Close(); err != nil {
|
|
t.Fatalf("close native test executable: %v", err)
|
|
}
|
|
}
|
|
|
|
func currentTestExecutable(t *testing.T) string {
|
|
t.Helper()
|
|
path, err := os.Executable()
|
|
if err != nil {
|
|
t.Fatalf("resolve current test executable: %v", err)
|
|
}
|
|
return path
|
|
}
|
|
|
|
func sameTestScriptFile(first string, second string) bool {
|
|
firstInfo, firstErr := os.Stat(first)
|
|
secondInfo, secondErr := os.Stat(second)
|
|
return firstErr == nil && secondErr == nil && os.SameFile(firstInfo, secondInfo)
|
|
}
|
|
|
|
func nativeExecutableSuffix() string {
|
|
if runtime.GOOS == "windows" {
|
|
return ".exe"
|
|
}
|
|
return ""
|
|
}
|
|
|
|
func portableTestScriptPath(path string) string {
|
|
return filepath.ToSlash(filepath.Clean(path))
|
|
}
|
|
|
|
// fakePrebuiltTestBinary is a stand-in for a cross-compiled `go test -c`
|
|
// binary: it records its own cwd, its BEADS_TEST_REPO_ROOT, and every
|
|
// argument it was launched with, then exits 0. The flag-mapping contract it
|
|
// exercises (scripts/test.sh -> scripts/ci/run-go-test-binary.sh -> this
|
|
// binary) is host-independent, so a bash stand-in is representative even
|
|
// though the real artifact is a Windows PE .exe: that artifact's own
|
|
// Windows-launch behavior is exercised by the CI job itself, not by this
|
|
// policy test.
|
|
const fakePrebuiltTestBinary = `#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
{
|
|
printf 'cwd=%s\n' "$PWD"
|
|
printf 'repo_root=%s\n' "${BEADS_TEST_REPO_ROOT:-}"
|
|
printf 'env_root=%s\n' "${BEADS_TEST_ENV_ROOT:-}"
|
|
for a in "$@"; do
|
|
printf 'arg=%s\n' "$a"
|
|
done
|
|
} >>"$BEADS_TEST_SCRIPT_PREBUILT_LOG"
|
|
`
|
|
|
|
const testScriptPrebuiltLogEnv = "BEADS_TEST_SCRIPT_PREBUILT_LOG"
|
|
|
|
type prebuiltTestBinaryRun struct {
|
|
cwd string
|
|
repoRoot string
|
|
envRoot string
|
|
args []string
|
|
}
|
|
|
|
// TestTestScriptPrebuiltTestBinaryContract pins scripts/test.sh's
|
|
// BEADS_TEST_PREBUILT_TEST_BINARY mode (F4.4's run-go-test-binary.sh path):
|
|
// it must map go-test-style flags to -test.* flags, launch the binary from
|
|
// the package directory (the same convention `go test` itself uses), export
|
|
// BEADS_TEST_REPO_ROOT, and refuse a request naming more than one package
|
|
// (a prebuilt binary is compiled from exactly one package, so there is no
|
|
// single binary that could run a multi-package request).
|
|
func TestTestScriptPrebuiltTestBinaryContract(t *testing.T) {
|
|
if runtime.GOOS == "windows" {
|
|
// The bash stand-in above is a Linux/macOS-only harness; see its doc
|
|
// comment. The real cross-built .exe is launched by the Windows job,
|
|
// not this test.
|
|
t.Skip("prebuilt-binary contract is exercised with a bash stand-in on Linux/macOS")
|
|
}
|
|
|
|
t.Run("maps go-test flags to -test.* flags, cwd, and BEADS_TEST_REPO_ROOT", func(t *testing.T) {
|
|
repoRoot := sourceRepoRoot(t)
|
|
run := runTestScriptWithPrebuiltBinary(t, repoRoot, []string{
|
|
"-v", "-timeout", "5m", "-skip", "TestBar", "-run", "TestFoo", "-count", "1", "./scripts/ci",
|
|
})
|
|
|
|
wantArgs := []string{
|
|
"-test.timeout", "5m",
|
|
"-test.parallel", "4",
|
|
"-test.v",
|
|
"-test.skip", "TestBar",
|
|
"-test.run", "TestFoo",
|
|
"-test.count", "1",
|
|
"-test.paniconexit0",
|
|
}
|
|
if strings.Join(run.args, " ") != strings.Join(wantArgs, " ") {
|
|
t.Fatalf("args = %v, want %v", run.args, wantArgs)
|
|
}
|
|
|
|
wantCwd := filepath.Join(repoRoot, "scripts", "ci")
|
|
if !sameTestScriptDir(t, run.cwd, wantCwd) {
|
|
t.Fatalf("cwd = %q, want %q", run.cwd, wantCwd)
|
|
}
|
|
if !sameTestScriptDir(t, run.repoRoot, repoRoot) {
|
|
t.Fatalf("BEADS_TEST_REPO_ROOT = %q, want %q", run.repoRoot, repoRoot)
|
|
}
|
|
})
|
|
|
|
t.Run("supports -count=N form alongside -count N", func(t *testing.T) {
|
|
repoRoot := sourceRepoRoot(t)
|
|
run := runTestScriptWithPrebuiltBinary(t, repoRoot, []string{"-count=1", "./scripts/ci"})
|
|
wantArgs := []string{"-test.timeout", "25m", "-test.parallel", "4", "-test.count", "1", "-test.paniconexit0"}
|
|
if strings.Join(run.args, " ") != strings.Join(wantArgs, " ") {
|
|
t.Fatalf("args = %v, want %v", run.args, wantArgs)
|
|
}
|
|
})
|
|
|
|
t.Run("refuses more than one package", func(t *testing.T) {
|
|
repoRoot := sourceRepoRoot(t)
|
|
output, err := runTestScriptWithPrebuiltBinaryExpectFailure(t, repoRoot, []string{"./scripts/ci", "./scripts"})
|
|
if err == nil {
|
|
t.Fatalf("expected scripts/test.sh to fail for two packages, output:\n%s", output)
|
|
}
|
|
if !strings.Contains(string(output), "requires exactly one package") {
|
|
t.Fatalf("expected a single-package error, got:\n%s", output)
|
|
}
|
|
})
|
|
|
|
t.Run("cleans up BEADS_TEST_ENV_ROOT after the binary exits (no exec short-circuit)", func(t *testing.T) {
|
|
// Regression test for F4 review SF-4: scripts/test.sh's prebuilt-binary
|
|
// branch used to `exec` run-go-test-binary.sh, which replaces the
|
|
// shell process image before the EXIT trap armed by
|
|
// beads_test_env_enter (beads_test_env_cleanup) ever runs - leaking
|
|
// the per-run BEADS_TEST_ENV_ROOT temp directory (and, with
|
|
// BEADS_TEST_SHARED_SERVER=1, potentially orphaning a shared dolt
|
|
// sql-server process). Running the binary as a child and exiting with
|
|
// its captured status instead lets the trap fire normally.
|
|
repoRoot := sourceRepoRoot(t)
|
|
run := runTestScriptWithPrebuiltBinary(t, repoRoot, []string{"-count=1", "./scripts/ci"})
|
|
if run.envRoot == "" {
|
|
t.Fatal("fake prebuilt binary never observed BEADS_TEST_ENV_ROOT; cannot assert cleanup")
|
|
}
|
|
if _, err := os.Stat(run.envRoot); !os.IsNotExist(err) {
|
|
t.Fatalf("BEADS_TEST_ENV_ROOT %q still exists after scripts/test.sh exited (cleanup trap did not run); stat err = %v", run.envRoot, err)
|
|
}
|
|
})
|
|
}
|
|
|
|
func runTestScriptWithPrebuiltBinary(t *testing.T, repoRoot string, args []string) prebuiltTestBinaryRun {
|
|
t.Helper()
|
|
output, logContent, err := runTestScriptPrebuiltBinary(t, repoRoot, args)
|
|
if err != nil {
|
|
t.Fatalf("scripts/test.sh failed: %v\n%s", err, output)
|
|
}
|
|
|
|
run := prebuiltTestBinaryRun{}
|
|
for _, line := range strings.Split(strings.TrimRight(logContent, "\n"), "\n") {
|
|
switch {
|
|
case strings.HasPrefix(line, "cwd="):
|
|
run.cwd = strings.TrimPrefix(line, "cwd=")
|
|
case strings.HasPrefix(line, "repo_root="):
|
|
run.repoRoot = strings.TrimPrefix(line, "repo_root=")
|
|
case strings.HasPrefix(line, "env_root="):
|
|
run.envRoot = strings.TrimPrefix(line, "env_root=")
|
|
case strings.HasPrefix(line, "arg="):
|
|
run.args = append(run.args, strings.TrimPrefix(line, "arg="))
|
|
}
|
|
}
|
|
if run.cwd == "" {
|
|
t.Fatalf("fake prebuilt binary never ran; scripts/test.sh output:\n%s\nlog:\n%s", output, logContent)
|
|
}
|
|
return run
|
|
}
|
|
|
|
func runTestScriptWithPrebuiltBinaryExpectFailure(t *testing.T, repoRoot string, args []string) ([]byte, error) {
|
|
t.Helper()
|
|
output, _, err := runTestScriptPrebuiltBinary(t, repoRoot, args)
|
|
return output, err
|
|
}
|
|
|
|
// filterEnv returns environ with any "key=..." entry for the given keys
|
|
// removed, so a caller can force a subprocess to fall back to its own
|
|
// hardcoded default instead of inheriting the host/sandbox's value.
|
|
func filterEnv(environ []string, keys ...string) []string {
|
|
drop := make(map[string]bool, len(keys))
|
|
for _, k := range keys {
|
|
drop[k] = true
|
|
}
|
|
filtered := make([]string, 0, len(environ))
|
|
for _, kv := range environ {
|
|
name, _, ok := strings.Cut(kv, "=")
|
|
if ok && drop[name] {
|
|
continue
|
|
}
|
|
filtered = append(filtered, kv)
|
|
}
|
|
return filtered
|
|
}
|
|
|
|
func runTestScriptPrebuiltBinary(t *testing.T, repoRoot string, args []string) (output []byte, logContent string, err error) {
|
|
t.Helper()
|
|
|
|
root := t.TempDir()
|
|
fakeBin := filepath.Join(root, "fake-prebuilt-bd-cgo.test")
|
|
if err := os.WriteFile(fakeBin, []byte(fakePrebuiltTestBinary), 0o755); err != nil {
|
|
t.Fatalf("write fake prebuilt test binary: %v", err)
|
|
}
|
|
logPath := filepath.Join(root, "fake-prebuilt.log")
|
|
if err := os.WriteFile(logPath, nil, 0o600); err != nil {
|
|
t.Fatalf("initialize fake prebuilt binary log: %v", err)
|
|
}
|
|
|
|
bash, lookErr := exec.LookPath("bash")
|
|
if lookErr != nil {
|
|
t.Fatalf("bash is required to exercise scripts/test.sh: %v", lookErr)
|
|
}
|
|
|
|
// Bazel's test runner sets its own TEST_TIMEOUT in the sandbox env
|
|
// (seconds, e.g. "300" for the default moderate size) - the same name
|
|
// scripts/test.sh reads to override its "25m" default. Strip it so the
|
|
// "-count=N form" subtest's assertion on that hardcoded default is not
|
|
// at the mercy of whatever test size this target happens to run under;
|
|
// go test does not set this variable, so only Bazel needs the filter.
|
|
//
|
|
// scripts/ci/scripts-go-test.sh (the CI driver for this very package's
|
|
// `go test ./scripts/...` run) calls beads_test_env_enter in its own
|
|
// shell before invoking go test, so this process's own os.Environ() can
|
|
// already carry an inherited BEADS_TEST_ENV_ACTIVE=1 / BEADS_TEST_ENV_ROOT
|
|
// / BEADS_TEST_ENV_DISABLE / BEADS_TEST_ENV_KEEP from that outer,
|
|
// longer-lived shell. If left in the child env below, scripts/test.sh's
|
|
// own beads_test_env_enter call (scripts/test.sh:17) sees the
|
|
// already-active guard and no-ops: no new root, no new EXIT trap - so the
|
|
// directory observed by the fake binary is the outer root, which is
|
|
// still alive (and rightly so: it's owned by the outer process) when
|
|
// scripts/test.sh exits. That makes the "cleans up BEADS_TEST_ENV_ROOT"
|
|
// subtest below fail only when this test binary happens to run nested
|
|
// under scripts-go-test.sh, even though the SF-4 fix it is guarding is
|
|
// correct. Every real production caller of scripts/test.sh's prebuilt
|
|
// path (pr.yml's Windows liveness/worktree-remove prebuilt steps) runs it
|
|
// as the first command of a fresh GitHub Actions `run:` step, never
|
|
// nested under an already-entered hermetic env - so strip the hermetic
|
|
// env markers here to deterministically exercise that same top-level,
|
|
// non-nested invocation regardless of the ambient env this test binary
|
|
// itself happens to be running under.
|
|
env := append(filterEnv(os.Environ(),
|
|
"TEST_TIMEOUT",
|
|
"BEADS_TEST_ENV_ACTIVE",
|
|
"BEADS_TEST_ENV_ROOT",
|
|
"BEADS_TEST_ENV_DISABLE",
|
|
"BEADS_TEST_ENV_KEEP",
|
|
),
|
|
"BEADS_TEST_BD_BINARY=/nonexistent-bd-not-needed-in-prebuilt-test-binary-mode",
|
|
"BEADS_TEST_PREBUILT_TEST_BINARY="+fakeBin,
|
|
"GITHUB_WORKSPACE="+repoRoot,
|
|
testScriptPrebuiltLogEnv+"="+logPath,
|
|
)
|
|
|
|
cmdArgs := append([]string{filepath.Join(repoRoot, "scripts", "test.sh")}, args...)
|
|
cmd := exec.Command(bash, cmdArgs...)
|
|
cmd.Dir = repoRoot
|
|
cmd.Env = env
|
|
output, err = cmd.CombinedOutput()
|
|
|
|
logBytes, readErr := os.ReadFile(logPath)
|
|
if readErr != nil {
|
|
t.Fatalf("read fake prebuilt binary log: %v", readErr)
|
|
}
|
|
return output, string(logBytes), err
|
|
}
|
|
|
|
func sameTestScriptDir(t *testing.T, first, second string) bool {
|
|
t.Helper()
|
|
firstInfo, firstErr := os.Stat(first)
|
|
secondInfo, secondErr := os.Stat(second)
|
|
if firstErr != nil || secondErr != nil {
|
|
return false
|
|
}
|
|
return os.SameFile(firstInfo, secondInfo)
|
|
}
|