test: ban ungated host go/gofmt execs under Bazel; pin compact_gc_fixture pure/norace (#7485)
* test: ban ungated host go/gofmt execs under Bazel; pin compact_gc_fixture pure/norace Follow-up to #7478. Adds two policy tests on //scripts:go_sources_test: - host_tool_exec_policy_test.go: an AST scan of every *_test.go file and internal/testutil's non-test .go files for exec.Command/CommandContext/ LookPath("go"/"gofmt") that isn't gated by bazeltest.IsBazel()/TEST_SRCDIR in its enclosing function, with an explicit, justified allowlist for the handful of sites gated by caller dataflow instead (bazeltest.PrebuiltBD() checked before calling a separate helper). "dolt" is out of scope: the hermetic copy resolves first on Bazel's PATH, unlike go/gofmt. - host_tool_shell_policy_test.go: the same policy for the shell scripts under scripts/ that Bazel sh_test/go_test targets actually execute (fmt-check.sh, the repochecks guards, gofmt-bin.sh, the migration-test harness), matching the gating pattern #7478 added to fmt-check.sh and scripts/ci/lib/test-env.sh. Both are mutation-verified (an added ungated exec.Command("go","version")/ `go build` is detected, then reverted cleanly). Also sets pure = "on" and race = "off" on cmd/bd/BUILD.bazel's compact_gc_fixture go_binary, matching :bd_for_tests/:cmd_norace: it is a testonly, stdlib-only fixture, so neither cgo nor the race detector buys it anything, under any --config. Verified TestRunCompactDoltTargetsOnlyAuthorizedActiveDatabase still passes via `bazel --nohome_rc test` under both --config=prcore and --config=doltserver-cmd. Known pre-existing issue, out of scope here: gosec's GoVersion() call still execs a host `go` from third-party code this policy doesn't reach. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(scripts): address review of host-tool exec/shell policy (#7485) Fixes all 6 maintainer-city-c3 review items against the ban-ungated-go/ gofmt-exec policy, each with a mutation test proving the fix catches the described regression: 1. Shell scan now gates per-branch/per-call-site (shellLineGating), not by any TEST_SRCDIR mention anywhere in the whole file. 2. Shell scanner adds gofmt and `go tool` detection, fixes the heredoc-start regex (no longer swallows the rest of the script on an arithmetic `<<` shift or a `<<<` here-string), and replaces the blanket printf/echo/git-grep line skip with a quoted-span/command- substitution-aware check that still catches a real exec joined onto a message line with `&&`. 3. shellPolicyScope gains pr-lint.sh, check-versions.sh, upgrade-smoke-test.sh and the 4 .github/scripts shard fallbacks, each cited against its real BUILD.bazel/.bzl wiring; corrects two factually wrong "not executed" doc-comment claims (pr-lint.sh and upgrade-smoke-test.sh ARE Bazel-executed); and adds TestShellPolicyScopeCoversBazelShellData, a mechanical backstop that fails on a Bazel-referenced .sh under scripts/ or .github/scripts/ missing from scope. 4. hostToolExecAllowlist no longer exempts goBuildBDCommand's 9 call sites by helper name; helperIsGatedByAllCallers checks every real call site is itself gated. 5. Both allowlists are keyed by file + enclosing function + exact snippet, not line number. 6. gated()/shellLineGating understand condition polarity, so an inverted `if !IsBazel() { t.Skip() }` (or its shell equivalent) is correctly NOT counted as a gate. Also extends the Go scanner's condPolarity to recognize findPrebuiltBDBinary()/bazeltest.PrebuiltBD() result variables as a TEST_SRCDIR-equivalent gate (proven safe by reading their implementation), and the shell scanner's analogous shellBazelSetVars for CANDIDATE_BIN/SOURCE_TAG_SQLITE_BIN/BEADS_TEST_*_BINARY, including through a script's own default-value variable alias (`VAR="${BAZEL_VAR:-default}"`). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(scripts): give go_sources_test runfiles access to the new shell scope The shell-policy rewrite (host_tool_shell_policy_test.go) added shellPolicyScope entries for scripts/ci/pr-lint.sh, scripts/check-versions.sh, scripts/upgrade-smoke-test.sh, and the four .github/scripts/*-test-shard.sh files, plus a scope-coverage guard (TestShellPolicyScopeCoversBazelShellData) that walks every tracked BUILD.bazel file. None of those were runfiles of //scripts:go_sources_test, so bazel test passed locally under plain `go test` but failed in CI's hermetic Bazel sandbox with "no such file or directory" for the new scope entries and "found only 0 BUILD.bazel files" for the guard's walk. Add //:repo_other_files (every tracked non-Go, non-Markdown file, including every BUILD/.bzl file) to go_sources_test's data, mirroring the same pattern already used by //scripts:scripts_test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(scripts): rename AST-fixture function so it isn't counted as a real test tools/bazel/equivalence.py compares `go test`'s discovered top-level tests against what Bazel actually ran, using a raw-text regex (`^func\s+(Test\w*)\s*\(\s*\w+\s+\*testing\.T\s*\)`) rather than compiling the file. The "inverted IsBazel skip" fixture in host_tool_exec_policy_test.go embeds a column-0 `func TestSomething(t *testing.T) {` inside a backtick string used to build an AST for findHostToolExecCalls, which the regex scan counted as a real top-level test that Bazel's go_sources_test then "failed to run" (CI's Bazel / test leg: EQUIVALENCE: FAIL on "scripts TestSomething"). Rename the fixture's inner function to bazelOnlyFixture; the test's behavior is unchanged since the name was never semantically significant to the scan being exercised. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(scripts): close host-go-policy gaps from PR #7485 re-review Four fixes from the final review round: 1. test.sh/conformance.sh/release.sh ARE in scope (they run under Bazel via scripts/** data globs); drop the false "not in scope" claims and add justified fn-keyed allowlist entries for their 8 flagged execs. 2. Go AST: an exec inside the Bazel-true branch, before its own abort, is no longer wrongly counted as gated (M17); a gate inside a sibling closure no longer covers execs outside it. 3. Shell scanner: abort/skip gate state now resets at function-scope end (a top-level `go build` after build_candidate in upgrade-smoke-test.sh is correctly flagged), and `bash -c 'go build'` is detected again. 4. Split host_tool_shell_policy_test.go into its own go_test (shell_host_tool_policy_test) with narrow, curated data instead of riding go_sources_test's //:repo_other_files; restored go_sources_test's data to its pre-PR scope. Dropped TestShellPolicyScopeCoversBazelShellData: its regex backstop could not see Starlark labels/.bzl/globs and still missed real in-scope scripts, so it wasn't earning its cache-rerun cost. The new target carries no tags, so it still runs in the required "Bazel / test" lane. Each of 1-3 ships with a mutation-proof fixture that fails on the prior code and passes on the fix. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(scripts): rename sibling-closure fixture to avoid false test-equivalence hit tools/bazel/equivalence.py line-scans *_test.go files for a column-0 `func Test\w*(t *testing.T)` to build its "what go test runs" list; it reads raw text, not compiled Go, so the fixture's embedded raw-string Go source (used to feed findHostToolExecCalls a parseable AST) was mistaken for a real top-level test the package defines, and the equivalence check failed because no go_test target "ran" it. Rename the fixture function from TestX to siblingClosureFixture, matching the existing convention a few lines down (bazelOnlyFixture) that documents the same pitfall. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
c18db4ebbd
commit
b7dba4d4e4
5 changed files with 2176 additions and 1 deletions
|
|
@ -212,10 +212,17 @@ go_test(
|
|||
deps = ["//internal/testutil/bazeltest"],
|
||||
)
|
||||
|
||||
# check-testing-short.sh over every Go file, test and non-test.
|
||||
# check-testing-short.sh over every Go file, test and non-test, plus the
|
||||
# Go-exec half of the host-go-under-Bazel policy (host_tool_exec_policy_test.go
|
||||
# scans //:repo_go_srcs and //:repo_go_test_srcs for a literal go/gofmt exec).
|
||||
# The shell-script half (host_tool_shell_policy_test.go) is its own
|
||||
# :shell_host_tool_policy_test below: it reads a fixed, narrow list of shell
|
||||
# scripts, not every Go source file, so it would otherwise pay this target's
|
||||
# full re-run cost for files it never looks at.
|
||||
go_test(
|
||||
name = "go_sources_test",
|
||||
srcs = [
|
||||
"host_tool_exec_policy_test.go",
|
||||
"repo_helpers_test.go",
|
||||
"testing_short_tree_test.go",
|
||||
],
|
||||
|
|
@ -227,6 +234,52 @@ go_test(
|
|||
deps = ["//internal/testutil/bazeltest"],
|
||||
)
|
||||
|
||||
# The shell-script half of the host-go-under-Bazel policy
|
||||
# (host_tool_shell_policy_test.go: see its package comment for shellPolicyScope
|
||||
# and the detection tradeoffs). Split out of go_sources_test (which otherwise
|
||||
# re-ran on every Go-source change to serve only this file) with data limited
|
||||
# to exactly the shell scripts shellPolicyScope names -- no BUILD.bazel/.bzl
|
||||
# walk, and so no //:repo_other_files: TestShellPolicyScopeCoversBazelShellData
|
||||
# (dropped; see that test's old doc comment in version control) could not
|
||||
# parse a `//pkg:file.sh` label or a glob() either, so paying 758 extra input
|
||||
# files and ~4 points of cache-rerun rate across the whole repo for a walk
|
||||
# that demonstrably still missed real in-scope scripts was not worth it.
|
||||
go_test(
|
||||
name = "shell_host_tool_policy_test",
|
||||
srcs = [
|
||||
"host_tool_shell_policy_test.go",
|
||||
"repo_helpers_test.go",
|
||||
],
|
||||
data = [
|
||||
"check-build-tags.sh",
|
||||
"check-doc-freshness.sh",
|
||||
"check-go-install-guidance.sh",
|
||||
"check-migration-hygiene.sh",
|
||||
"check-testing-short.sh",
|
||||
"check-versions.sh",
|
||||
"check-winget-portable-alias.sh",
|
||||
"check-workapi-frontend-boundary.sh",
|
||||
"ci/fmt-check.sh",
|
||||
"ci/gofmt-bin.sh",
|
||||
"ci/pr-lint.sh",
|
||||
"conformance.sh",
|
||||
"migrate-legacy-to-current.sh",
|
||||
"migration-test/historical-dolt-upgrade-test.sh",
|
||||
"migration-test/legacy-bridge-test.sh",
|
||||
"migration-test/lib/binary.sh",
|
||||
"release.sh",
|
||||
"test.sh",
|
||||
"upgrade-smoke-test.sh",
|
||||
"//:.github/scripts/embedded-storage-test-shard.sh",
|
||||
"//:.github/scripts/embedded-test-shard.sh",
|
||||
"//:.github/scripts/proxied-test-shard.sh",
|
||||
"//:.github/scripts/server-storage-test-shard.sh",
|
||||
"//scripts/repochecks:run_check.sh",
|
||||
"//scripts/repochecks:types_gen_drift_test.sh",
|
||||
],
|
||||
deps = ["//internal/testutil/bazeltest"],
|
||||
)
|
||||
|
||||
# Scans of every tracked file (//:repo_files): any change re-runs them, so
|
||||
# keep them few and fast.
|
||||
go_test(
|
||||
|
|
|
|||
1001
scripts/host_tool_exec_policy_test.go
Normal file
1001
scripts/host_tool_exec_policy_test.go
Normal file
File diff suppressed because it is too large
Load diff
1103
scripts/host_tool_shell_policy_test.go
Normal file
1103
scripts/host_tool_shell_policy_test.go
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -355,6 +355,17 @@ sh_test(
|
|||
},
|
||||
)
|
||||
|
||||
# Readable by //scripts:go_sources_test's host_tool_shell_policy_test.go,
|
||||
# which scans every sh_test PROGRAM/srcs this package runs under Bazel for
|
||||
# ungated bare host-`go` execs (f4-spec.md §9).
|
||||
exports_files(
|
||||
[
|
||||
"run_check.sh",
|
||||
"types_gen_drift_test.sh",
|
||||
],
|
||||
visibility = ["//scripts:__pkg__"],
|
||||
)
|
||||
|
||||
# --- begin repo_files (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---
|
||||
|
||||
filegroup(
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue