beads/cmd/bd/compact_gc_target_test.go
Julian Knutsen 3e7705a2cd
fix(bazel): remove host-go dep from compact GC test; drop frozen shard blocks (#7478)
* fix(bazel): remove host-go dep from compact GC test; drop frozen shard blocks

Replace the ungated `go build` in compact_gc_target_test.go's dolt-stand-in
fixture with a Bazel-built go_binary (cmd/bd:compact_gc_fixture) wired via
rlocationpath env + data, mirroring the existing preflight_go_fixture
pattern. Falls back to `go build` only when not running under Bazel
(bazeltest.IsBazel()). Also gate test_repo_beads_guard_test.go's `go env
GOCACHE`/`GOMODCACHE` probes on !bazeltest.IsBazel(), since they previously
ran unconditionally in every cmd/bd TestMain.

Separately, delete the legacy 20-shard embedded-cmd and 15-shard
proxied-cmd "frozen" manifest blocks in .github/scripts/*-test-shards.txt:
their only consumers were pr-risk.yml fork/push jobs that have since
migrated to bazel.yml. Update gen_embedded_cmd_shard_manifest.py and
gen_proxied_shard_manifest.py to drop the frozen-block rendering/guard
logic and default to the live Bazel-only shard counts (100 and 34),
and refresh stale comments/docs (cmd/bd/BUILD.bazel,
scripts/shard_manifest_test.go, engdocs/TESTING.md,
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md) that still described the deleted
blocks. Verified no other BUILD/.github/scripts references read them.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): auto-sync BUILD files

Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/38069936930
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.

* fix(bazel): gate remaining host-go calls; fix shard-manifest --weights footgun

Address PR #7478 review findings (bee-ghosttrack):

- scripts/ci/lib/test-env.sh: skip the GOCACHE/GOMODCACHE `go env` probes
  when TEST_SRCDIR is set, so scripts/test.sh's beads_test_env_enter never
  shells out to a bare `go` when reached from a Bazel-run go_test (e.g.
  //scripts:shell_scripts_test's TestTestScriptPrebuiltTestBinaryContract),
  where a farm masks host go/dolt to exit 127. Nothing on that path needs a
  cache directory.
- scripts/ci/fmt-check.sh: describe_gofmt() no longer calls a PATH `go`
  under Bazel. It now resolves BEADS_TEST_GO (the hermetic SDK go already
  wired into scripts/prlintmake:prlintmake_test's env) via TEST_SRCDIR, and
  skips the version-reporting arm entirely when that isn't set (e.g.
  scripts/repochecks:fmt_test), rather than ever touching a possibly-masked
  PATH `go`.
- Verified with a PATH stub (go/dolt scripts that log and exit 127) via
  `bazel --nohome_rc test --config=prcore //scripts/... //cmd/bd:bd_test
  --test_env=PATH=<stubdir>:/usr/bin:/bin:/usr/local/bin`: 26/26 targets
  pass, zero stub invocations logged.
- gen_proxied_shard_manifest.py / gen_embedded_cmd_shard_manifest.py:
  default --weights to duration now that the manifests' only remaining
  blocks are duration-packed (the legacy inits-packed blocks were deleted
  alongside the PR Risk fork/push jobs that read them). Previously the bare
  `--write` default (inits) silently clobbered the live block's duration
  header and repacked with the wrong cost model; --check couldn't see it.
  Added TestProxiedShardManifestBareWriteIsIdempotent and
  TestEmbeddedCmdShardManifestBareWriteIsIdempotent to pin a bare --write
  as a no-op against the committed files.
- cmd/bd/compact_gc_target_test.go: resolve the non-Bazel fallback fixture
  path from this file's own directory (runtime.Caller) instead of the test
  binary's working directory.
- Refresh stale comments in .github/workflows/bazel.yml,
  scripts/shard_manifest_test.go, scripts/pr_risk_bazel_coverage_test.go,
  and scripts/embedded_shard_count_test.go that still described the deleted
  legacy 20/15-shard blocks and their retired fork/push jobs in the present
  tense.
- Confirmed cmd/bd/BUILD.bazel's compact_gc_fixture go_binary (implicit
  pure/race defaults) still passes TestRunCompactDoltTargetsOnlyAuthorizedActiveDatabase
  under `bazel test //cmd/bd:bd_test`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-10 19:24:21 +00:00

354 lines
14 KiB
Go

package main
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"reflect"
"runtime"
"strings"
"testing"
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/hooks"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/testutil/bazeltest"
)
type compactGCStoreStub struct {
storage.DoltStorage
directory string
err error
}
func (s *compactGCStoreStub) ExternalGCPath(context.Context) (string, error) {
return s.directory, s.err
}
func (s *compactGCStoreStub) ActiveDatabaseSize(ctx context.Context) (int64, error) {
if sizer, ok := s.DoltStorage.(storage.ActiveDatabaseSizer); ok {
return sizer.ActiveDatabaseSize(ctx)
}
return 0, &storage.ErrUnsupported{Op: "ActiveDatabaseSize", Backend: "fixture"}
}
func TestRunCompactDoltTargetsOnlyAuthorizedActiveDatabase(t *testing.T) {
clearTelemetryEnv(t)
t.Setenv("BD_JSON_ENVELOPE", "0")
toolDir := buildCompactGCFixture(t)
// wantBackend pins which ErrUnsupported.Backend the refusal names, which is
// the only observable that distinguishes "the active store refused" (the
// store's own "external" sentinel, reached through storage.UnwrapStore) from
// "the interface assertion missed" ("active store"). Both render the same
// top-level message, so the *through decorators* rows are vacuous for the
// decorator claim without it.
//
// Known residual: "unsupported dry run through decorators" still cannot
// discriminate. Its benign dry-run preview is byte-identical for both error
// origins and carries no backend, so no CLI-level assertion distinguishes
// them; the non-dry-run decorator row is what pins that path.
for _, tc := range []struct {
name, mode, pathKind string
shared, dry, unsupported bool
decorated bool
plain, envelope bool
calls int
wantErr bool
wantMessage string
wantBackend string
wantHint string
}{
{name: "owned active", calls: 1},
{name: "shared active despite stale project root", shared: true, calls: 1},
{name: "owned active through decorators", decorated: true, calls: 1},
{name: "shared active through decorators", decorated: true, shared: true, calls: 1},
{name: "older CLI fallback", mode: "fallback", calls: 2},
{name: "real failure is not retried", mode: "failure", calls: 1, wantErr: true},
{name: "dry run", dry: true},
{name: "unsupported with plausible local path", unsupported: true, wantErr: true, wantBackend: "external"},
{name: "unsupported text output", unsupported: true, plain: true, wantErr: true, wantBackend: "external"},
{name: "unsupported enveloped output", unsupported: true, envelope: true, wantErr: true, wantBackend: "external"},
{name: "unsupported dry run", unsupported: true, dry: true},
{name: "unsupported through decorators", decorated: true, unsupported: true, wantErr: true, wantBackend: "external"},
{name: "unsupported dry run through decorators", decorated: true, unsupported: true, dry: true},
{name: "cancelled before the active database is resolved", pathKind: "cancelled", wantErr: true,
wantMessage: "external Dolt garbage collection was interrupted", wantHint: "Re-run 'bd admin compact --dolt'"},
{name: "cancelled in dry run", pathKind: "cancelled", dry: true, wantErr: true,
wantMessage: "external Dolt garbage collection was interrupted", wantHint: "Re-run 'bd admin compact --dolt'"},
{name: "deadline exceeded before the active database is resolved", pathKind: "deadline", wantErr: true,
wantMessage: "external Dolt garbage collection was interrupted", wantHint: "Re-run 'bd admin compact --dolt'"},
{name: "size capability is insufficient", pathKind: "size-only", wantErr: true, wantBackend: "active store"},
{name: "general locator is insufficient", pathKind: "locator-only", wantErr: true, wantBackend: "active store"},
{name: "missing declared path", pathKind: "missing", wantErr: true, wantMessage: "active Dolt database directory is unavailable"},
{name: "missing declared path in dry run", pathKind: "missing", dry: true, wantErr: true, wantMessage: "active Dolt database directory is unavailable"},
{name: "file is not a database directory", pathKind: "file", wantErr: true, wantMessage: "is not a directory"},
{name: "empty declared path", pathKind: "empty", wantErr: true, wantMessage: "requires an absolute active database directory"},
{name: "relative declared path", pathKind: "relative", wantErr: true, wantMessage: "requires an absolute active database directory"},
} {
t.Run(tc.name, func(t *testing.T) {
oldStore, oldDry, oldJSON := store, compactDryRun, jsonOutput
beads.ResetCaches()
t.Cleanup(func() { store, compactDryRun, jsonOutput = oldStore, oldDry, oldJSON; beads.ResetCaches() })
project := t.TempDir()
beadsDir := filepath.Join(project, ".beads")
stale := filepath.Join(beadsDir, "dolt")
root := stale
if tc.shared {
root = filepath.Join(t.TempDir(), "central", "dolt")
}
active := filepath.Join(root, "active")
sibling := filepath.Join(root, "sibling")
for _, dir := range []string{stale, active, sibling} {
if err := os.MkdirAll(dir, 0o700); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "keep"), []byte("unchanged"), 0o600); err != nil {
t.Fatal(err)
}
}
t.Setenv("BEADS_DIR", beadsDir)
t.Setenv("PATH", toolDir+string(os.PathListSeparator)+os.Getenv("PATH"))
logPath := filepath.Join(t.TempDir(), "calls.jsonl")
t.Setenv("BEADS_COMPACT_GC_FIXTURE_LOG", logPath)
t.Setenv("BEADS_COMPACT_GC_FIXTURE_MODE", tc.mode)
candidate := &compactGCStoreStub{DoltStorage: &gcSizeStoreStub{size: 42}, directory: active}
if tc.unsupported {
candidate.err = &storage.ErrUnsupported{Op: "ExternalGCPath", Backend: "external"}
}
store = candidate
switch tc.pathKind {
case "size-only":
store = &gcSizeStoreStub{size: 42}
case "locator-only":
store = &gcLocatorOnlyStoreStub{path: active}
case "missing":
candidate.directory = filepath.Join(root, "missing")
case "file":
candidate.directory = filepath.Join(active, "keep")
case "empty":
candidate.directory = ""
case "relative":
candidate.directory = "relative-database"
case "cancelled":
// What the real DoltStore.ExternalGCPath returns for a cancelled
// ctx: the bare context error, not an ErrUnsupported.
candidate.err = context.Canceled
case "deadline":
candidate.err = context.DeadlineExceeded
}
if tc.decorated {
t.Setenv("BD_OTEL_STDOUT", "true")
store = wireStorageDecorators(store, hooks.NewRunner(filepath.Join(project, "missing-hooks")), false)
if storage.UnwrapStore(store) != candidate {
t.Fatal("production decorator chain lost the active store")
}
}
compactDryRun, jsonOutput = tc.dry, !tc.plain
if tc.envelope {
t.Setenv("BD_JSON_ENVELOPE", "1")
}
var runErr error
var output string
if tc.wantErr {
output = captureStderr(t, func() { runErr = runCompactDolt(t.Context()) })
} else {
output = captureStdout(t, func() error { runErr = runCompactDolt(t.Context()); return nil })
}
if (runErr != nil) != tc.wantErr {
t.Fatalf("runCompactDolt error = %v, wantErr %v", runErr, tc.wantErr)
}
if tc.wantErr {
if code, ok := exitCodeFromError(runErr); !ok || code != 1 {
t.Fatalf("refusal exit code = %d, recognized %v; error %v", code, ok, runErr)
}
}
if tc.wantErr && tc.mode != "failure" {
message, hint := output, output
if tc.plain {
if !strings.HasPrefix(output, "Error: ") || !strings.Contains(output, "\nHint: ") {
t.Fatalf("missing text error/hint: %q", output)
}
} else {
var result map[string]interface{}
if err := json.Unmarshal([]byte(output), &result); err != nil {
t.Fatalf("error stderr is not JSON: %v: %s", err, output)
}
if result["schema_version"] != float64(JSONSchemaVersion) {
t.Fatalf("missing error schema version: %v", result)
}
if tc.envelope {
var ok bool
result, ok = result["data"].(map[string]interface{})
if !ok {
t.Fatalf("missing error envelope data: %s", output)
}
}
message, _ = result["error"].(string)
hint, _ = result["hint"].(string)
}
wantMessage := tc.wantMessage
if wantMessage == "" {
wantMessage = "cannot select a local database for external Dolt garbage collection"
}
wantHint := tc.wantHint
if wantHint == "" {
wantHint = "bd doctor"
}
if !strings.Contains(message, wantMessage) || !strings.Contains(hint, wantHint) {
t.Fatalf("refusal lost its cause or actionable hint: %s", output)
}
// A cancellation must not be dressed as a permanent authority
// refusal: neither the refusal message nor its four-cause hint may
// appear, in dry-run or otherwise.
if tc.pathKind == "cancelled" || tc.pathKind == "deadline" {
for _, forbidden := range []string{"cannot select a local database", "BEADS_DOLT_SERVER_PORT", "server administrator"} {
if strings.Contains(message, forbidden) || strings.Contains(hint, forbidden) {
t.Errorf("cancellation reported as an authority refusal (%q present): %s", forbidden, output)
}
}
}
if tc.wantBackend != "" {
wantBackend := "not supported by the " + tc.wantBackend + " backend"
if !strings.Contains(message, wantBackend) {
t.Errorf("refusal names the wrong origin: want %q in %q", wantBackend, message)
}
}
if tc.unsupported {
for _, condition := range []string{"BEADS_DOLT_SERVER_PORT", "BEADS_DOLT_PORT", "auto-start", "external", "gateway", "proxied", "socket", "TLS", "server administrator"} {
if !strings.Contains(hint, condition) {
t.Errorf("unsupported hint omits %q: %s", condition, hint)
}
}
}
}
data, err := os.ReadFile(logPath)
if err != nil && !os.IsNotExist(err) {
t.Fatal(err)
}
var calls []struct {
Dir string
Args []string
}
for _, line := range strings.Split(strings.TrimSpace(string(data)), "\n") {
if line == "" {
continue
}
var call struct {
Dir string
Args []string
}
if err := json.Unmarshal([]byte(line), &call); err != nil {
t.Fatal(err)
}
calls = append(calls, call)
}
if len(calls) != tc.calls {
t.Fatalf("external GC calls = %v, want %d", calls, tc.calls)
}
for i, call := range calls {
wantArgs := []string{"gc", "--archive-level", "0"}
if i == 1 {
wantArgs = []string{"gc"}
}
gotInfo, err := os.Stat(call.Dir)
if err != nil {
t.Fatal(err)
}
wantInfo, err := os.Stat(active)
if err != nil {
t.Fatal(err)
}
if !os.SameFile(gotInfo, wantInfo) || !reflect.DeepEqual(call.Args, wantArgs) {
t.Fatalf("GC call = %+v, want directory %q, args %v", call, active, wantArgs)
}
}
for _, dir := range []string{stale, active, sibling} {
data, err := os.ReadFile(filepath.Join(dir, "keep"))
if err != nil || string(data) != "unchanged" {
t.Fatalf("changed sentinel in %q: %q, %v", dir, data, err)
}
_, err = os.Stat(filepath.Join(dir, "gc-ran"))
wantMarker := dir == active && tc.calls > 0 && !tc.wantErr
if (err == nil) != wantMarker {
t.Fatalf("GC marker in %q: %v, want marker %v", dir, err, wantMarker)
}
}
if !tc.wantErr {
var result map[string]interface{}
if err := json.Unmarshal([]byte(output), &result); err != nil {
t.Fatalf("JSON: %v: %s", err, output)
}
if tc.unsupported {
_, hasDoltPath := result["dolt_path"]
if result["available"] != false || hasDoltPath {
t.Fatalf("unsupported target was guessed: %v", result)
}
} else if result["dolt_path"] != active || result["size_before"] != float64(42) {
t.Fatalf("reported scope = %v, want active %q and size42", result, active)
}
}
})
}
}
// compactGCFixtureEnv names the environment variable a go_test sets from its
// BUILD file to the prebuilt compact-gc-fixture.go binary (built once by
// Bazel as a testonly go_binary), the same pattern preflight_test.go uses for
// BEADS_TEST_PREFLIGHT_GO:
//
// data = [":compact_gc_fixture"],
// env = {"BEADS_TEST_COMPACT_GC_FIXTURE": "$(rlocationpath :compact_gc_fixture)"},
//
// so this test never needs a host Go toolchain under Bazel.
const compactGCFixtureEnv = "BEADS_TEST_COMPACT_GC_FIXTURE"
// buildCompactGCFixture produces a directory holding a `dolt` executable that
// stands in for the external Dolt CLI. Under Bazel it copies the prebuilt
// fixture from runfiles (bazeltest.RunfileEnv); outside Bazel (plain
// `go test`) it falls back to compiling testdata/compact-gc-fixture.go with
// the host Go toolchain, as before.
func buildCompactGCFixture(t *testing.T) string {
t.Helper()
dir := t.TempDir()
name := "dolt"
if runtime.GOOS == "windows" {
name += ".exe"
}
target := filepath.Join(dir, name)
if bazeltest.IsBazel() {
fixture, err := bazeltest.RunfileEnv(compactGCFixtureEnv)
if err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(fixture)
if err != nil {
t.Fatalf("read prebuilt GC fixture: %v", err)
}
if err := os.WriteFile(target, data, 0o755); err != nil {
t.Fatalf("install prebuilt GC fixture: %v", err)
}
return dir
}
realGo, err := exec.LookPath("go")
if err != nil {
t.Fatalf("locate Go toolchain for subprocess fixture: %v", err)
}
// Absolute, resolved from this file's own directory rather than the test
// binary's working directory: `go test` happens to run from the package
// directory, but nothing here should depend on that (and t.Chdir, a
// -run subset launched from elsewhere, etc. would otherwise break this
// silently with a confusing "no such file" from `go build`).
_, thisFile, _, ok := runtime.Caller(0)
if !ok {
t.Fatal("runtime.Caller failed")
}
path := filepath.Join(filepath.Dir(thisFile), "testdata", "compact-gc-fixture.go")
cmd := exec.Command(realGo, "build", "-o", target, path)
cmd.Env = append(os.Environ(), "CGO_ENABLED=0", "GOWORK=off")
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("build owned GC fixture: %v\n%s", err, output)
}
return dir
}