* fix(bazel): remove host-go dep from compact GC test; drop frozen shard blocks Replace the ungated `go build` in compact_gc_target_test.go's dolt-stand-in fixture with a Bazel-built go_binary (cmd/bd:compact_gc_fixture) wired via rlocationpath env + data, mirroring the existing preflight_go_fixture pattern. Falls back to `go build` only when not running under Bazel (bazeltest.IsBazel()). Also gate test_repo_beads_guard_test.go's `go env GOCACHE`/`GOMODCACHE` probes on !bazeltest.IsBazel(), since they previously ran unconditionally in every cmd/bd TestMain. Separately, delete the legacy 20-shard embedded-cmd and 15-shard proxied-cmd "frozen" manifest blocks in .github/scripts/*-test-shards.txt: their only consumers were pr-risk.yml fork/push jobs that have since migrated to bazel.yml. Update gen_embedded_cmd_shard_manifest.py and gen_proxied_shard_manifest.py to drop the frozen-block rendering/guard logic and default to the live Bazel-only shard counts (100 and 34), and refresh stale comments/docs (cmd/bd/BUILD.bazel, scripts/shard_manifest_test.go, engdocs/TESTING.md, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md) that still described the deleted blocks. Verified no other BUILD/.github/scripts references read them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * build(bazel): auto-sync BUILD files Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/38069936930 (`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy). Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see scripts/bazel-autofix-push.sh. * fix(bazel): gate remaining host-go calls; fix shard-manifest --weights footgun Address PR #7478 review findings (bee-ghosttrack): - scripts/ci/lib/test-env.sh: skip the GOCACHE/GOMODCACHE `go env` probes when TEST_SRCDIR is set, so scripts/test.sh's beads_test_env_enter never shells out to a bare `go` when reached from a Bazel-run go_test (e.g. //scripts:shell_scripts_test's TestTestScriptPrebuiltTestBinaryContract), where a farm masks host go/dolt to exit 127. Nothing on that path needs a cache directory. - scripts/ci/fmt-check.sh: describe_gofmt() no longer calls a PATH `go` under Bazel. It now resolves BEADS_TEST_GO (the hermetic SDK go already wired into scripts/prlintmake:prlintmake_test's env) via TEST_SRCDIR, and skips the version-reporting arm entirely when that isn't set (e.g. scripts/repochecks:fmt_test), rather than ever touching a possibly-masked PATH `go`. - Verified with a PATH stub (go/dolt scripts that log and exit 127) via `bazel --nohome_rc test --config=prcore //scripts/... //cmd/bd:bd_test --test_env=PATH=<stubdir>:/usr/bin:/bin:/usr/local/bin`: 26/26 targets pass, zero stub invocations logged. - gen_proxied_shard_manifest.py / gen_embedded_cmd_shard_manifest.py: default --weights to duration now that the manifests' only remaining blocks are duration-packed (the legacy inits-packed blocks were deleted alongside the PR Risk fork/push jobs that read them). Previously the bare `--write` default (inits) silently clobbered the live block's duration header and repacked with the wrong cost model; --check couldn't see it. Added TestProxiedShardManifestBareWriteIsIdempotent and TestEmbeddedCmdShardManifestBareWriteIsIdempotent to pin a bare --write as a no-op against the committed files. - cmd/bd/compact_gc_target_test.go: resolve the non-Bazel fallback fixture path from this file's own directory (runtime.Caller) instead of the test binary's working directory. - Refresh stale comments in .github/workflows/bazel.yml, scripts/shard_manifest_test.go, scripts/pr_risk_bazel_coverage_test.go, and scripts/embedded_shard_count_test.go that still described the deleted legacy 20/15-shard blocks and their retired fork/push jobs in the present tense. - Confirmed cmd/bd/BUILD.bazel's compact_gc_fixture go_binary (implicit pure/race defaults) still passes TestRunCompactDoltTargetsOnlyAuthorizedActiveDatabase under `bazel test //cmd/bd:bd_test`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
159 lines
6.4 KiB
Go
159 lines
6.4 KiB
Go
package scripts_test
|
|
|
|
// The Bazel shard counts the CI policy tests (//scripts:scripts_test) and the
|
|
// shard-manifest tests (//scripts:go_test_sources_test) check against, read
|
|
// from each sharded target's own BUILD.bazel rule. Helpers only: both targets
|
|
// compile this file, so a test here would run twice.
|
|
|
|
import (
|
|
"fmt"
|
|
"regexp"
|
|
"strconv"
|
|
"testing"
|
|
)
|
|
|
|
// shardCountPattern matches a rule block's ` shard_count = N,` line, the
|
|
// same pattern bazelProxiedShardCount (below) inlines
|
|
// for its own single use; shared here since both embedded accessors below
|
|
// need it.
|
|
var shardCountPattern = regexp.MustCompile(`(?m)^ shard_count = (\d+),$`)
|
|
|
|
// embeddedCmdTargets are the cmd/bd rules that split the bazel-embedded
|
|
// lane's cmd/bd manifest block between them (more shards than Bazel's
|
|
// 50-per-rule cap), in shard order.
|
|
var embeddedCmdTargets = []string{"bd_embedded_test", "bd_embedded_part2_test"}
|
|
|
|
// embeddedCmdPart is one of embeddedCmdTargets: its Bazel shard_count and the
|
|
// range of the manifest block it runs (go_test_manifest_shard.sh's
|
|
// --shard-offset/--shard-total args).
|
|
type embeddedCmdPart struct {
|
|
label string
|
|
shards, offset, total int
|
|
}
|
|
|
|
// spec is the part's check_shard_coverage.py --suite SHARDS argument.
|
|
func (p embeddedCmdPart) spec() string {
|
|
return fmt.Sprintf("%d@%d/%d", p.shards, p.offset, p.total)
|
|
}
|
|
|
|
var (
|
|
shardOffsetArg = regexp.MustCompile(`(?m)^ "--shard-offset=(\d+)",$`)
|
|
shardTotalArg = regexp.MustCompile(`(?m)^ "--shard-total=(\d+)",$`)
|
|
)
|
|
|
|
// bazelEmbeddedCmdParts reads embeddedCmdTargets' shard_count and shard
|
|
// range from cmd/bd/BUILD.bazel and fails unless the ranges tile one block
|
|
// exactly once, in order.
|
|
func bazelEmbeddedCmdParts(t *testing.T) []embeddedCmdPart {
|
|
t.Helper()
|
|
build := readPolicyFile(t, sourceRepoRoot(t), "cmd/bd/BUILD.bazel")
|
|
var parts []embeddedCmdPart
|
|
next := 0
|
|
for _, name := range embeddedCmdTargets {
|
|
rule := bazelRuleBlock(build, name)
|
|
p := embeddedCmdPart{label: "//cmd/bd:" + name}
|
|
for _, f := range []struct {
|
|
re *regexp.Regexp
|
|
dst *int
|
|
what string
|
|
}{{shardCountPattern, &p.shards, "shard_count = N,"}, {shardOffsetArg, &p.offset, `"--shard-offset=N",`}, {shardTotalArg, &p.total, `"--shard-total=N",`}} {
|
|
m := f.re.FindStringSubmatch(rule)
|
|
if m == nil {
|
|
t.Fatalf("cmd/bd:%s has no %s in cmd/bd/BUILD.bazel:\n%s", name, f.what, rule)
|
|
}
|
|
n, err := strconv.Atoi(m[1])
|
|
if err != nil {
|
|
t.Fatalf("cmd/bd:%s %s: %v", name, f.what, err)
|
|
}
|
|
*f.dst = n
|
|
}
|
|
if p.offset != next || (len(parts) > 0 && p.total != parts[0].total) {
|
|
t.Fatalf("cmd/bd:%s runs shards %d..%d of %d; want it to start at shard %d of the same block as %v", name, p.offset+1, p.offset+p.shards, p.total, next+1, parts)
|
|
}
|
|
next += p.shards
|
|
parts = append(parts, p)
|
|
}
|
|
if next != parts[0].total {
|
|
t.Fatalf("%v run %d shards of a %d-shard block; their shard_counts must sum to --shard-total", parts, next, parts[0].total)
|
|
}
|
|
return parts
|
|
}
|
|
|
|
// bazelEmbeddedCmdShardCount returns the shard total of the bazel-embedded
|
|
// lane's cmd/bd manifest block, which embeddedCmdTargets split between them
|
|
// (bazelEmbeddedCmdParts): the single source of truth for the Bazel-only
|
|
// lane's cmd/bd shard split (slice F1). It no longer has to equal PR
|
|
// Risk's/main.yml's legacy "Test (Embedded Dolt Cmd N/20)" fork/push jobs'
|
|
// matrix size — those jobs, and the frozen 20-shard block they read, have
|
|
// since been deleted (see engdocs/TESTING.md) — mirrors bazelProxiedShardCount
|
|
// (F2; see that function's doc comment below for the shared rationale, not
|
|
// repeated here).
|
|
func bazelEmbeddedCmdShardCount(t *testing.T) int {
|
|
t.Helper()
|
|
return bazelEmbeddedCmdParts(t)[0].total
|
|
}
|
|
|
|
// bazelEmbeddedStorageShardCount returns
|
|
// embeddeddolt:embeddeddolt_embedded_test's own shard_count from
|
|
// internal/storage/embeddeddolt/BUILD.bazel: the single source of truth for
|
|
// the Bazel-only bazel-embedded lane's storage shard split (slice F1), which
|
|
// no longer has to equal PR Risk's/main.yml's legacy "Test (Embedded Dolt
|
|
// Storage N/5)" fork/push jobs' matrix size. Mirrors
|
|
// bazelEmbeddedCmdShardCount above; see its doc comment.
|
|
func bazelEmbeddedStorageShardCount(t *testing.T) int {
|
|
t.Helper()
|
|
root := sourceRepoRoot(t)
|
|
rule := bazelRuleBlock(readPolicyFile(t, root, "internal/storage/embeddeddolt/BUILD.bazel"), "embeddeddolt_embedded_test")
|
|
m := shardCountPattern.FindStringSubmatch(rule)
|
|
if m == nil {
|
|
t.Fatalf("embeddeddolt:embeddeddolt_embedded_test has no `shard_count = N,` in internal/storage/embeddeddolt/BUILD.bazel:\n%s", rule)
|
|
}
|
|
n, err := strconv.Atoi(m[1])
|
|
if err != nil {
|
|
t.Fatalf("embeddeddolt:embeddeddolt_embedded_test shard_count: %v", err)
|
|
}
|
|
return n
|
|
}
|
|
|
|
// bazelServerFullShardCount returns dolt:dolt_server_full_test's own
|
|
// shard_count from internal/storage/dolt/BUILD.bazel: the single source of
|
|
// truth for the bazel-server-storage lane's full-suite shard split, now that
|
|
// PR Risk's legacy "Test (Server Dolt Full Suite N/16)" job, whose matrix it
|
|
// used to mirror, is retired (ga-96smfk.22). Mirrors
|
|
// bazelEmbeddedCmdShardCount above.
|
|
func bazelServerFullShardCount(t *testing.T) int {
|
|
t.Helper()
|
|
root := sourceRepoRoot(t)
|
|
rule := bazelRuleBlock(readPolicyFile(t, root, "internal/storage/dolt/BUILD.bazel"), "dolt_server_full_test")
|
|
m := shardCountPattern.FindStringSubmatch(rule)
|
|
if m == nil {
|
|
t.Fatalf("dolt:dolt_server_full_test has no `shard_count = N,` in internal/storage/dolt/BUILD.bazel:\n%s", rule)
|
|
}
|
|
n, err := strconv.Atoi(m[1])
|
|
if err != nil {
|
|
t.Fatalf("dolt:dolt_server_full_test shard_count: %v", err)
|
|
}
|
|
return n
|
|
}
|
|
|
|
// The default shard manifest of a PR Risk shard script.
|
|
var shardManifestDefault = regexp.MustCompile(`\$\{BEADS_TEST_SHARD_MANIFEST:-([^}]+)\}`)
|
|
|
|
// bazelProxiedShardCount returns cmd/bd:bd_proxied_test's own shard_count
|
|
// from cmd/bd/BUILD.bazel: the single source of truth for the Bazel-only
|
|
// bazel-proxied lane's shard split, which no longer has to equal PR
|
|
// Risk's/main.yml's legacy test-proxied-cmd jobs' matrix size (F2).
|
|
func bazelProxiedShardCount(t *testing.T) int {
|
|
t.Helper()
|
|
root := sourceRepoRoot(t)
|
|
rule := bazelRuleBlock(readPolicyFile(t, root, "cmd/bd/BUILD.bazel"), "bd_proxied_test")
|
|
m := regexp.MustCompile(`(?m)^ shard_count = (\d+),$`).FindStringSubmatch(rule)
|
|
if m == nil {
|
|
t.Fatalf("cmd/bd:bd_proxied_test has no `shard_count = N,` in cmd/bd/BUILD.bazel:\n%s", rule)
|
|
}
|
|
n, err := strconv.Atoi(m[1])
|
|
if err != nil {
|
|
t.Fatalf("cmd/bd:bd_proxied_test shard_count: %v", err)
|
|
}
|
|
return n
|
|
}
|