* fix(bazel): remove host-go dep from compact GC test; drop frozen shard blocks Replace the ungated `go build` in compact_gc_target_test.go's dolt-stand-in fixture with a Bazel-built go_binary (cmd/bd:compact_gc_fixture) wired via rlocationpath env + data, mirroring the existing preflight_go_fixture pattern. Falls back to `go build` only when not running under Bazel (bazeltest.IsBazel()). Also gate test_repo_beads_guard_test.go's `go env GOCACHE`/`GOMODCACHE` probes on !bazeltest.IsBazel(), since they previously ran unconditionally in every cmd/bd TestMain. Separately, delete the legacy 20-shard embedded-cmd and 15-shard proxied-cmd "frozen" manifest blocks in .github/scripts/*-test-shards.txt: their only consumers were pr-risk.yml fork/push jobs that have since migrated to bazel.yml. Update gen_embedded_cmd_shard_manifest.py and gen_proxied_shard_manifest.py to drop the frozen-block rendering/guard logic and default to the live Bazel-only shard counts (100 and 34), and refresh stale comments/docs (cmd/bd/BUILD.bazel, scripts/shard_manifest_test.go, engdocs/TESTING.md, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md) that still described the deleted blocks. Verified no other BUILD/.github/scripts references read them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * build(bazel): auto-sync BUILD files Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/38069936930 (`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy). Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see scripts/bazel-autofix-push.sh. * fix(bazel): gate remaining host-go calls; fix shard-manifest --weights footgun Address PR #7478 review findings (bee-ghosttrack): - scripts/ci/lib/test-env.sh: skip the GOCACHE/GOMODCACHE `go env` probes when TEST_SRCDIR is set, so scripts/test.sh's beads_test_env_enter never shells out to a bare `go` when reached from a Bazel-run go_test (e.g. //scripts:shell_scripts_test's TestTestScriptPrebuiltTestBinaryContract), where a farm masks host go/dolt to exit 127. Nothing on that path needs a cache directory. - scripts/ci/fmt-check.sh: describe_gofmt() no longer calls a PATH `go` under Bazel. It now resolves BEADS_TEST_GO (the hermetic SDK go already wired into scripts/prlintmake:prlintmake_test's env) via TEST_SRCDIR, and skips the version-reporting arm entirely when that isn't set (e.g. scripts/repochecks:fmt_test), rather than ever touching a possibly-masked PATH `go`. - Verified with a PATH stub (go/dolt scripts that log and exit 127) via `bazel --nohome_rc test --config=prcore //scripts/... //cmd/bd:bd_test --test_env=PATH=<stubdir>:/usr/bin:/bin:/usr/local/bin`: 26/26 targets pass, zero stub invocations logged. - gen_proxied_shard_manifest.py / gen_embedded_cmd_shard_manifest.py: default --weights to duration now that the manifests' only remaining blocks are duration-packed (the legacy inits-packed blocks were deleted alongside the PR Risk fork/push jobs that read them). Previously the bare `--write` default (inits) silently clobbered the live block's duration header and repacked with the wrong cost model; --check couldn't see it. Added TestProxiedShardManifestBareWriteIsIdempotent and TestEmbeddedCmdShardManifestBareWriteIsIdempotent to pin a bare --write as a no-op against the committed files. - cmd/bd/compact_gc_target_test.go: resolve the non-Bazel fallback fixture path from this file's own directory (runtime.Caller) instead of the test binary's working directory. - Refresh stale comments in .github/workflows/bazel.yml, scripts/shard_manifest_test.go, scripts/pr_risk_bazel_coverage_test.go, and scripts/embedded_shard_count_test.go that still described the deleted legacy 20/15-shard blocks and their retired fork/push jobs in the present tense. - Confirmed cmd/bd/BUILD.bazel's compact_gc_fixture go_binary (implicit pure/race defaults) still passes TestRunCompactDoltTargetsOnlyAuthorizedActiveDatabase under `bazel test //cmd/bd:bd_test`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
354 lines
14 KiB
Go
354 lines
14 KiB
Go
package main
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"reflect"
|
|
"runtime"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/steveyegge/beads/internal/beads"
|
|
"github.com/steveyegge/beads/internal/hooks"
|
|
"github.com/steveyegge/beads/internal/storage"
|
|
"github.com/steveyegge/beads/internal/testutil/bazeltest"
|
|
)
|
|
|
|
type compactGCStoreStub struct {
|
|
storage.DoltStorage
|
|
directory string
|
|
err error
|
|
}
|
|
|
|
func (s *compactGCStoreStub) ExternalGCPath(context.Context) (string, error) {
|
|
return s.directory, s.err
|
|
}
|
|
|
|
func (s *compactGCStoreStub) ActiveDatabaseSize(ctx context.Context) (int64, error) {
|
|
if sizer, ok := s.DoltStorage.(storage.ActiveDatabaseSizer); ok {
|
|
return sizer.ActiveDatabaseSize(ctx)
|
|
}
|
|
return 0, &storage.ErrUnsupported{Op: "ActiveDatabaseSize", Backend: "fixture"}
|
|
}
|
|
|
|
func TestRunCompactDoltTargetsOnlyAuthorizedActiveDatabase(t *testing.T) {
|
|
clearTelemetryEnv(t)
|
|
t.Setenv("BD_JSON_ENVELOPE", "0")
|
|
toolDir := buildCompactGCFixture(t)
|
|
// wantBackend pins which ErrUnsupported.Backend the refusal names, which is
|
|
// the only observable that distinguishes "the active store refused" (the
|
|
// store's own "external" sentinel, reached through storage.UnwrapStore) from
|
|
// "the interface assertion missed" ("active store"). Both render the same
|
|
// top-level message, so the *through decorators* rows are vacuous for the
|
|
// decorator claim without it.
|
|
//
|
|
// Known residual: "unsupported dry run through decorators" still cannot
|
|
// discriminate. Its benign dry-run preview is byte-identical for both error
|
|
// origins and carries no backend, so no CLI-level assertion distinguishes
|
|
// them; the non-dry-run decorator row is what pins that path.
|
|
for _, tc := range []struct {
|
|
name, mode, pathKind string
|
|
shared, dry, unsupported bool
|
|
decorated bool
|
|
plain, envelope bool
|
|
calls int
|
|
wantErr bool
|
|
wantMessage string
|
|
wantBackend string
|
|
wantHint string
|
|
}{
|
|
{name: "owned active", calls: 1},
|
|
{name: "shared active despite stale project root", shared: true, calls: 1},
|
|
{name: "owned active through decorators", decorated: true, calls: 1},
|
|
{name: "shared active through decorators", decorated: true, shared: true, calls: 1},
|
|
{name: "older CLI fallback", mode: "fallback", calls: 2},
|
|
{name: "real failure is not retried", mode: "failure", calls: 1, wantErr: true},
|
|
{name: "dry run", dry: true},
|
|
{name: "unsupported with plausible local path", unsupported: true, wantErr: true, wantBackend: "external"},
|
|
{name: "unsupported text output", unsupported: true, plain: true, wantErr: true, wantBackend: "external"},
|
|
{name: "unsupported enveloped output", unsupported: true, envelope: true, wantErr: true, wantBackend: "external"},
|
|
{name: "unsupported dry run", unsupported: true, dry: true},
|
|
{name: "unsupported through decorators", decorated: true, unsupported: true, wantErr: true, wantBackend: "external"},
|
|
{name: "unsupported dry run through decorators", decorated: true, unsupported: true, dry: true},
|
|
{name: "cancelled before the active database is resolved", pathKind: "cancelled", wantErr: true,
|
|
wantMessage: "external Dolt garbage collection was interrupted", wantHint: "Re-run 'bd admin compact --dolt'"},
|
|
{name: "cancelled in dry run", pathKind: "cancelled", dry: true, wantErr: true,
|
|
wantMessage: "external Dolt garbage collection was interrupted", wantHint: "Re-run 'bd admin compact --dolt'"},
|
|
{name: "deadline exceeded before the active database is resolved", pathKind: "deadline", wantErr: true,
|
|
wantMessage: "external Dolt garbage collection was interrupted", wantHint: "Re-run 'bd admin compact --dolt'"},
|
|
{name: "size capability is insufficient", pathKind: "size-only", wantErr: true, wantBackend: "active store"},
|
|
{name: "general locator is insufficient", pathKind: "locator-only", wantErr: true, wantBackend: "active store"},
|
|
{name: "missing declared path", pathKind: "missing", wantErr: true, wantMessage: "active Dolt database directory is unavailable"},
|
|
{name: "missing declared path in dry run", pathKind: "missing", dry: true, wantErr: true, wantMessage: "active Dolt database directory is unavailable"},
|
|
{name: "file is not a database directory", pathKind: "file", wantErr: true, wantMessage: "is not a directory"},
|
|
{name: "empty declared path", pathKind: "empty", wantErr: true, wantMessage: "requires an absolute active database directory"},
|
|
{name: "relative declared path", pathKind: "relative", wantErr: true, wantMessage: "requires an absolute active database directory"},
|
|
} {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
oldStore, oldDry, oldJSON := store, compactDryRun, jsonOutput
|
|
beads.ResetCaches()
|
|
t.Cleanup(func() { store, compactDryRun, jsonOutput = oldStore, oldDry, oldJSON; beads.ResetCaches() })
|
|
project := t.TempDir()
|
|
beadsDir := filepath.Join(project, ".beads")
|
|
stale := filepath.Join(beadsDir, "dolt")
|
|
root := stale
|
|
if tc.shared {
|
|
root = filepath.Join(t.TempDir(), "central", "dolt")
|
|
}
|
|
active := filepath.Join(root, "active")
|
|
sibling := filepath.Join(root, "sibling")
|
|
for _, dir := range []string{stale, active, sibling} {
|
|
if err := os.MkdirAll(dir, 0o700); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err := os.WriteFile(filepath.Join(dir, "keep"), []byte("unchanged"), 0o600); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
t.Setenv("BEADS_DIR", beadsDir)
|
|
t.Setenv("PATH", toolDir+string(os.PathListSeparator)+os.Getenv("PATH"))
|
|
logPath := filepath.Join(t.TempDir(), "calls.jsonl")
|
|
t.Setenv("BEADS_COMPACT_GC_FIXTURE_LOG", logPath)
|
|
t.Setenv("BEADS_COMPACT_GC_FIXTURE_MODE", tc.mode)
|
|
candidate := &compactGCStoreStub{DoltStorage: &gcSizeStoreStub{size: 42}, directory: active}
|
|
if tc.unsupported {
|
|
candidate.err = &storage.ErrUnsupported{Op: "ExternalGCPath", Backend: "external"}
|
|
}
|
|
store = candidate
|
|
switch tc.pathKind {
|
|
case "size-only":
|
|
store = &gcSizeStoreStub{size: 42}
|
|
case "locator-only":
|
|
store = &gcLocatorOnlyStoreStub{path: active}
|
|
case "missing":
|
|
candidate.directory = filepath.Join(root, "missing")
|
|
case "file":
|
|
candidate.directory = filepath.Join(active, "keep")
|
|
case "empty":
|
|
candidate.directory = ""
|
|
case "relative":
|
|
candidate.directory = "relative-database"
|
|
case "cancelled":
|
|
// What the real DoltStore.ExternalGCPath returns for a cancelled
|
|
// ctx: the bare context error, not an ErrUnsupported.
|
|
candidate.err = context.Canceled
|
|
case "deadline":
|
|
candidate.err = context.DeadlineExceeded
|
|
}
|
|
if tc.decorated {
|
|
t.Setenv("BD_OTEL_STDOUT", "true")
|
|
store = wireStorageDecorators(store, hooks.NewRunner(filepath.Join(project, "missing-hooks")), false)
|
|
if storage.UnwrapStore(store) != candidate {
|
|
t.Fatal("production decorator chain lost the active store")
|
|
}
|
|
}
|
|
compactDryRun, jsonOutput = tc.dry, !tc.plain
|
|
if tc.envelope {
|
|
t.Setenv("BD_JSON_ENVELOPE", "1")
|
|
}
|
|
var runErr error
|
|
var output string
|
|
if tc.wantErr {
|
|
output = captureStderr(t, func() { runErr = runCompactDolt(t.Context()) })
|
|
} else {
|
|
output = captureStdout(t, func() error { runErr = runCompactDolt(t.Context()); return nil })
|
|
}
|
|
if (runErr != nil) != tc.wantErr {
|
|
t.Fatalf("runCompactDolt error = %v, wantErr %v", runErr, tc.wantErr)
|
|
}
|
|
if tc.wantErr {
|
|
if code, ok := exitCodeFromError(runErr); !ok || code != 1 {
|
|
t.Fatalf("refusal exit code = %d, recognized %v; error %v", code, ok, runErr)
|
|
}
|
|
}
|
|
if tc.wantErr && tc.mode != "failure" {
|
|
message, hint := output, output
|
|
if tc.plain {
|
|
if !strings.HasPrefix(output, "Error: ") || !strings.Contains(output, "\nHint: ") {
|
|
t.Fatalf("missing text error/hint: %q", output)
|
|
}
|
|
} else {
|
|
var result map[string]interface{}
|
|
if err := json.Unmarshal([]byte(output), &result); err != nil {
|
|
t.Fatalf("error stderr is not JSON: %v: %s", err, output)
|
|
}
|
|
if result["schema_version"] != float64(JSONSchemaVersion) {
|
|
t.Fatalf("missing error schema version: %v", result)
|
|
}
|
|
if tc.envelope {
|
|
var ok bool
|
|
result, ok = result["data"].(map[string]interface{})
|
|
if !ok {
|
|
t.Fatalf("missing error envelope data: %s", output)
|
|
}
|
|
}
|
|
message, _ = result["error"].(string)
|
|
hint, _ = result["hint"].(string)
|
|
}
|
|
wantMessage := tc.wantMessage
|
|
if wantMessage == "" {
|
|
wantMessage = "cannot select a local database for external Dolt garbage collection"
|
|
}
|
|
wantHint := tc.wantHint
|
|
if wantHint == "" {
|
|
wantHint = "bd doctor"
|
|
}
|
|
if !strings.Contains(message, wantMessage) || !strings.Contains(hint, wantHint) {
|
|
t.Fatalf("refusal lost its cause or actionable hint: %s", output)
|
|
}
|
|
// A cancellation must not be dressed as a permanent authority
|
|
// refusal: neither the refusal message nor its four-cause hint may
|
|
// appear, in dry-run or otherwise.
|
|
if tc.pathKind == "cancelled" || tc.pathKind == "deadline" {
|
|
for _, forbidden := range []string{"cannot select a local database", "BEADS_DOLT_SERVER_PORT", "server administrator"} {
|
|
if strings.Contains(message, forbidden) || strings.Contains(hint, forbidden) {
|
|
t.Errorf("cancellation reported as an authority refusal (%q present): %s", forbidden, output)
|
|
}
|
|
}
|
|
}
|
|
if tc.wantBackend != "" {
|
|
wantBackend := "not supported by the " + tc.wantBackend + " backend"
|
|
if !strings.Contains(message, wantBackend) {
|
|
t.Errorf("refusal names the wrong origin: want %q in %q", wantBackend, message)
|
|
}
|
|
}
|
|
if tc.unsupported {
|
|
for _, condition := range []string{"BEADS_DOLT_SERVER_PORT", "BEADS_DOLT_PORT", "auto-start", "external", "gateway", "proxied", "socket", "TLS", "server administrator"} {
|
|
if !strings.Contains(hint, condition) {
|
|
t.Errorf("unsupported hint omits %q: %s", condition, hint)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
data, err := os.ReadFile(logPath)
|
|
if err != nil && !os.IsNotExist(err) {
|
|
t.Fatal(err)
|
|
}
|
|
var calls []struct {
|
|
Dir string
|
|
Args []string
|
|
}
|
|
for _, line := range strings.Split(strings.TrimSpace(string(data)), "\n") {
|
|
if line == "" {
|
|
continue
|
|
}
|
|
var call struct {
|
|
Dir string
|
|
Args []string
|
|
}
|
|
if err := json.Unmarshal([]byte(line), &call); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
calls = append(calls, call)
|
|
}
|
|
if len(calls) != tc.calls {
|
|
t.Fatalf("external GC calls = %v, want %d", calls, tc.calls)
|
|
}
|
|
for i, call := range calls {
|
|
wantArgs := []string{"gc", "--archive-level", "0"}
|
|
if i == 1 {
|
|
wantArgs = []string{"gc"}
|
|
}
|
|
gotInfo, err := os.Stat(call.Dir)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
wantInfo, err := os.Stat(active)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if !os.SameFile(gotInfo, wantInfo) || !reflect.DeepEqual(call.Args, wantArgs) {
|
|
t.Fatalf("GC call = %+v, want directory %q, args %v", call, active, wantArgs)
|
|
}
|
|
}
|
|
for _, dir := range []string{stale, active, sibling} {
|
|
data, err := os.ReadFile(filepath.Join(dir, "keep"))
|
|
if err != nil || string(data) != "unchanged" {
|
|
t.Fatalf("changed sentinel in %q: %q, %v", dir, data, err)
|
|
}
|
|
_, err = os.Stat(filepath.Join(dir, "gc-ran"))
|
|
wantMarker := dir == active && tc.calls > 0 && !tc.wantErr
|
|
if (err == nil) != wantMarker {
|
|
t.Fatalf("GC marker in %q: %v, want marker %v", dir, err, wantMarker)
|
|
}
|
|
}
|
|
if !tc.wantErr {
|
|
var result map[string]interface{}
|
|
if err := json.Unmarshal([]byte(output), &result); err != nil {
|
|
t.Fatalf("JSON: %v: %s", err, output)
|
|
}
|
|
if tc.unsupported {
|
|
_, hasDoltPath := result["dolt_path"]
|
|
if result["available"] != false || hasDoltPath {
|
|
t.Fatalf("unsupported target was guessed: %v", result)
|
|
}
|
|
} else if result["dolt_path"] != active || result["size_before"] != float64(42) {
|
|
t.Fatalf("reported scope = %v, want active %q and size42", result, active)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// compactGCFixtureEnv names the environment variable a go_test sets from its
|
|
// BUILD file to the prebuilt compact-gc-fixture.go binary (built once by
|
|
// Bazel as a testonly go_binary), the same pattern preflight_test.go uses for
|
|
// BEADS_TEST_PREFLIGHT_GO:
|
|
//
|
|
// data = [":compact_gc_fixture"],
|
|
// env = {"BEADS_TEST_COMPACT_GC_FIXTURE": "$(rlocationpath :compact_gc_fixture)"},
|
|
//
|
|
// so this test never needs a host Go toolchain under Bazel.
|
|
const compactGCFixtureEnv = "BEADS_TEST_COMPACT_GC_FIXTURE"
|
|
|
|
// buildCompactGCFixture produces a directory holding a `dolt` executable that
|
|
// stands in for the external Dolt CLI. Under Bazel it copies the prebuilt
|
|
// fixture from runfiles (bazeltest.RunfileEnv); outside Bazel (plain
|
|
// `go test`) it falls back to compiling testdata/compact-gc-fixture.go with
|
|
// the host Go toolchain, as before.
|
|
func buildCompactGCFixture(t *testing.T) string {
|
|
t.Helper()
|
|
dir := t.TempDir()
|
|
name := "dolt"
|
|
if runtime.GOOS == "windows" {
|
|
name += ".exe"
|
|
}
|
|
target := filepath.Join(dir, name)
|
|
if bazeltest.IsBazel() {
|
|
fixture, err := bazeltest.RunfileEnv(compactGCFixtureEnv)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
data, err := os.ReadFile(fixture)
|
|
if err != nil {
|
|
t.Fatalf("read prebuilt GC fixture: %v", err)
|
|
}
|
|
if err := os.WriteFile(target, data, 0o755); err != nil {
|
|
t.Fatalf("install prebuilt GC fixture: %v", err)
|
|
}
|
|
return dir
|
|
}
|
|
realGo, err := exec.LookPath("go")
|
|
if err != nil {
|
|
t.Fatalf("locate Go toolchain for subprocess fixture: %v", err)
|
|
}
|
|
// Absolute, resolved from this file's own directory rather than the test
|
|
// binary's working directory: `go test` happens to run from the package
|
|
// directory, but nothing here should depend on that (and t.Chdir, a
|
|
// -run subset launched from elsewhere, etc. would otherwise break this
|
|
// silently with a confusing "no such file" from `go build`).
|
|
_, thisFile, _, ok := runtime.Caller(0)
|
|
if !ok {
|
|
t.Fatal("runtime.Caller failed")
|
|
}
|
|
path := filepath.Join(filepath.Dir(thisFile), "testdata", "compact-gc-fixture.go")
|
|
cmd := exec.Command(realGo, "build", "-o", target, path)
|
|
cmd.Env = append(os.Environ(), "CGO_ENABLED=0", "GOWORK=off")
|
|
if output, err := cmd.CombinedOutput(); err != nil {
|
|
t.Fatalf("build owned GC fixture: %v\n%s", err, output)
|
|
}
|
|
return dir
|
|
}
|