fix(compact): target external GC at the active database (#6327)
* fix(compact): report active database size (#5619) Measure legacy admin compact --dolt reporting through ActiveDatabaseSizer so sibling databases cannot distort reclaimed-space numbers. Omit unavailable advisory measurements without blocking GC. Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com> Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell * docs(compact): explain the legacy size field names Keep the existing compact JSON schema distinct from the newer GC helper. The active external-GC target is addressed by the separate issue6313 leaf. * fix(compact): select the authoritative local active GC directory Ask the active storage instance for its optional external-GC directory capability. Reuse the authority captured at store open, and refuse stale, unsupported or invalid local targets rather than guessing from a root, general CLI path or advisory size. Preserve archive-level fallback behavior. Add native owned-process target and no-launch regressions plus instance authority tests; retain the parent active-size and legacy JSON contracts. Fixes #6313 * fix(compact): resolve GC capability through storage decorators Use the existing storage unwrapping boundary so external GC sees the captured active database behind the root command's dependency, telemetry and hook decorators. Cover owned, shared and unsupported stores through the real chain, and describe the active database in compact help. Refs gastownhall/beads#6313 Agent: unknown-model/unknown-reasoning * fix(compact): preserve external GC refusal diagnostics (#6313) Route all four new target refusals through the standard error/hint formatter and assert their stderr schema, message, hint and exit status. Preserve the relative-path refusal and unavailable dry-run target omission. Update the requested compact and flatten help mirrors without regenerating other docs. Addresses the diagnostics and documentation review on PR #6327. Existing PR #5668 ancestry and mandatory parent-first sequencing remain unchanged. Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell * docs(compact): align flag help with active GC target (saa-cgh.94) Authored by /root/cgh86_review; closes the independent root review help finding. Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell * docs: retain release-pinned CLI references (saa-cgh.94) Preserve corrected live compact/flatten help while generated references continue to match docs/cli-docs.pin v1.2.2. Fix the exact six-line hosted docs-policy failure without changing pin authority. Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell * fix(compact): classify cancellation before the GC authority refusal Maintainer review fixes for #6327, review iteration 1. Non-gating findings only: the synthesis and the quality scorecard both returned approve with no blocker and no major finding. Taken: - Error Handling & Resilience (minor, both lanes). runCompactDolt formatted a cancelled or expired context as the permanent "cannot select a local database" refusal plus its four-cause hint, and in --dry-run turned it into a hard exit-1. Classify context.Canceled/DeadlineExceeded ahead of the ErrUnsupported arm and report the interruption instead. The ctx probe stays in ExternalGCPath: it is the only cancellation checkpoint before the uninterruptible external `dolt gc`, so the finding's other arm (drop the probe) would trade a misleading message for launching a multi-minute GC on an interrupted command. - Change Impact / Blast Radius (minor). Document why resolveLocalActiveDatabaseDir keeps the cfg.AutoStart conjunct, and make the CLI hint name the remedy (`bd config set dolt.auto-start true`) instead of only the condition. The offered code arm is declined: AutoStart rides resolveAutoStart's own exclusions (forced false for ServerModeExternal and under BEADS_TEST_MODE=1), so dropping it widens GC authority - a behavior change, not a cleanup. - Release Safety (minor). Add the three new test files to the gazelle explicit go_test srcs lists so this PR's assertions are visible under `bazel test`. Verified with `bazel run //:gazelle -- --mode=diff`: gazelle requests none of the three, so the rows match its own output. The two rows it still wants are pre-existing base drift in unrelated files and are left alone. - Test Evidence Quality (nit). The two "unsupported ... through decorators" rows were measured vacuous for the decorator claim. Pin the refusal's origin through ErrUnsupported.Backend ("external" = the active store refused, "active store" = the interface assertion missed) and pin the opposite polarity on the size-only/locator-only rows. Residual recorded on the test: the dry-run decorator row still cannot discriminate, because its benign preview is identical for both origins. Declined, and carried to the human-approval summary: - Debuggability & Operability (minor, gate impact none). Reusing the `bd gc` companions adds production output in an iteration no further review pass grades, and the alternative arm would assert the refs are left in place "deliberately", which nothing in the change establishes. - Resource Lifecycle & Cleanup (nit, Required fix: none). exec.CommandContext would kill a mid-flight `dolt gc`; that is a data-safety question for the GC path rather than a cleanup. Validation: `go build ./...` and `go vet` on cmd/bd + internal/storage{,/dolt} clean; scripts/ci/pr-lint.sh against the pinned base green (gofmt, golangci-lint 0 issues, windows cross-lint 0 issues), proved non-vacuous by a misspell planted on an added line; 66 PASS / 0 FAIL / 0 SKIP over the covering cmd/bd selectors. Both new pins are mutation-proved: removing the classification arm reds exactly the three new cancellation rows and nothing else, and dropping storage.UnwrapStore now reds three decorator rows where it previously reddened two. internal/storage/dolt/store.go is comment-only. --------- Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local> Co-authored-by: Gas City Merge Queue <merge-queue@gascity.local>
This commit is contained in:
parent
89348699b0
commit
8269a88cbf
9 changed files with 579 additions and 57 deletions
|
|
@ -548,6 +548,8 @@ go_test(
|
|||
"comments_test.go",
|
||||
"compact_embedded_test.go",
|
||||
"compact_gc_flag_test.go",
|
||||
"compact_gc_size_test.go",
|
||||
"compact_gc_target_test.go",
|
||||
"compact_plan_test.go",
|
||||
"compact_test.go",
|
||||
"compact_tier2_embedded_test.go",
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ package main
|
|||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
|
|
@ -60,7 +61,7 @@ Dolt Garbage Collection:
|
|||
With auto-commit per mutation, Dolt commit history grows over time. Use
|
||||
--dolt to run Dolt garbage collection and reclaim disk space.
|
||||
|
||||
--dolt: Run Dolt GC on .beads/dolt directory to free disk space.
|
||||
--dolt: Run Dolt GC on the active database to free disk space.
|
||||
This removes unreachable commits and compacts storage.
|
||||
|
||||
Examples:
|
||||
|
|
@ -111,7 +112,7 @@ Examples:
|
|||
|
||||
// Handle dolt GC mode
|
||||
if compactDolt {
|
||||
return runCompactDolt()
|
||||
return runCompactDolt(ctx)
|
||||
}
|
||||
|
||||
// Count active modes
|
||||
|
|
@ -743,8 +744,8 @@ func runCompactApply(ctx context.Context, store storage.DoltStorage) error {
|
|||
return nil
|
||||
}
|
||||
|
||||
// runCompactDolt runs Dolt garbage collection on the .beads/dolt directory
|
||||
func runCompactDolt() error {
|
||||
// runCompactDolt runs external Dolt GC in the locally owned active database.
|
||||
func runCompactDolt(ctx context.Context) error {
|
||||
start := time.Now()
|
||||
|
||||
// Find beads directory
|
||||
|
|
@ -753,14 +754,34 @@ func runCompactDolt() error {
|
|||
return HandleErrorWithHint(activeWorkspaceNotFoundError(), diagHint())
|
||||
}
|
||||
|
||||
// Check for dolt directory
|
||||
doltPath := filepath.Join(beadsDir, "dolt")
|
||||
if _, err := os.Stat(doltPath); os.IsNotExist(err) {
|
||||
if compactDryRun {
|
||||
// Resolve operation authority from the opened store, never a stale project
|
||||
// directory, general CLI path, or advisory size measurement.
|
||||
var doltPath string
|
||||
var pathErr error
|
||||
if locator, ok := storage.UnwrapStore(store).(storage.ExternalGCLocator); ok {
|
||||
doltPath, pathErr = locator.ExternalGCPath(ctx)
|
||||
} else {
|
||||
pathErr = &storage.ErrUnsupported{Op: "ExternalGCPath", Backend: "active store"}
|
||||
}
|
||||
if pathErr != nil {
|
||||
// Classify cancellation before formatting. ExternalGCPath probes ctx, so an
|
||||
// interrupt or deadline surfaces here as a bare context error, and it is not
|
||||
// an authority refusal: without this arm a SIGINT renders as the permanent
|
||||
// "cannot select a local database" message plus the four-cause hint about
|
||||
// port overrides and proxied servers, none of which applies. The ctx probe
|
||||
// stays in ExternalGCPath deliberately — it is the only cancellation
|
||||
// checkpoint before the uninterruptible external `dolt gc` below.
|
||||
if errors.Is(pathErr, context.Canceled) || errors.Is(pathErr, context.DeadlineExceeded) {
|
||||
return HandleErrorWithHint(
|
||||
fmt.Sprintf("external Dolt garbage collection was interrupted: %v", pathErr),
|
||||
"the command stopped before the active database was resolved, so no garbage collection ran. "+
|
||||
"Re-run 'bd admin compact --dolt' to collect.")
|
||||
}
|
||||
var unsupported *storage.ErrUnsupported
|
||||
if compactDryRun && errors.As(pathErr, &unsupported) {
|
||||
if jsonOutput {
|
||||
output := map[string]interface{}{
|
||||
"dry_run": true,
|
||||
"dolt_path": doltPath,
|
||||
"available": false,
|
||||
}
|
||||
if err := outputJSON(output); err != nil {
|
||||
|
|
@ -769,27 +790,46 @@ func runCompactDolt() error {
|
|||
return nil
|
||||
}
|
||||
fmt.Printf("DRY RUN - Dolt garbage collection\n\n")
|
||||
fmt.Printf("Dolt directory: %s\n", doltPath)
|
||||
fmt.Printf("No local Dolt directory found; nothing to collect.\n")
|
||||
fmt.Printf("The active database is not available for local external garbage collection.\n")
|
||||
return nil
|
||||
}
|
||||
return HandleErrorWithHint(fmt.Sprintf("Dolt directory not found at %s", doltPath), "--dolt flag is only for repositories using the Dolt backend")
|
||||
return HandleErrorWithHint(
|
||||
fmt.Sprintf("cannot select a local database for external Dolt garbage collection: %v", pathErr),
|
||||
"--dolt requires a locally managed owned/shared Dolt database. Port overrides (BEADS_DOLT_SERVER_PORT/BEADS_DOLT_PORT), "+
|
||||
"external/gateway/proxied servers, and socket/TLS connections do not authorize local GC; owned mode also requires "+
|
||||
"auto-start, so a hand-started owned server needs 'bd config set dolt.auto-start true'. "+
|
||||
"Run 'bd doctor' to inspect the connection or ask the server administrator to run GC.")
|
||||
}
|
||||
if !filepath.IsAbs(doltPath) {
|
||||
return HandleErrorWithHint(
|
||||
fmt.Sprintf("external Dolt garbage collection requires an absolute active database directory, got %q", doltPath),
|
||||
"run 'bd doctor' and correct the active database configuration to use an absolute directory")
|
||||
}
|
||||
info, err := os.Stat(doltPath)
|
||||
if err != nil {
|
||||
return HandleErrorWithHint(
|
||||
fmt.Sprintf("active Dolt database directory is unavailable: %v", err),
|
||||
"check that the active database directory exists and is accessible; run 'bd doctor' to inspect the database configuration")
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return HandleErrorWithHint(
|
||||
fmt.Sprintf("active Dolt database path %q is not a directory", doltPath),
|
||||
"run 'bd doctor' and correct the active database configuration to point to a directory, not a file")
|
||||
}
|
||||
|
||||
// Get size before GC
|
||||
sizeBefore, err := getDirSize(doltPath)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Warning: could not calculate directory size: %v\n", err)
|
||||
sizeBefore = 0
|
||||
}
|
||||
// Measure only the active database. The shared .beads/dolt root may contain
|
||||
// sibling databases unrelated to this GC operation.
|
||||
sizeBefore := storeSizeBytes(ctx)
|
||||
|
||||
if compactDryRun {
|
||||
if jsonOutput {
|
||||
output := map[string]interface{}{
|
||||
"dry_run": true,
|
||||
"dolt_path": doltPath,
|
||||
"size_before": sizeBefore,
|
||||
"size_display": formatBytes(sizeBefore),
|
||||
"dry_run": true,
|
||||
"dolt_path": doltPath,
|
||||
}
|
||||
if sizeBefore >= 0 {
|
||||
output["size_before"] = sizeBefore
|
||||
output["size_display"] = formatBytes(sizeBefore)
|
||||
}
|
||||
if err := outputJSON(output); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
||||
|
|
@ -798,7 +838,9 @@ func runCompactDolt() error {
|
|||
}
|
||||
fmt.Printf("DRY RUN - Dolt garbage collection\n\n")
|
||||
fmt.Printf("Dolt directory: %s\n", doltPath)
|
||||
fmt.Printf("Current size: %s\n", formatBytes(sizeBefore))
|
||||
if sizeBefore >= 0 {
|
||||
fmt.Printf("Current size: %s\n", formatBytes(sizeBefore))
|
||||
}
|
||||
fmt.Printf("\nRun without --dry-run to perform garbage collection.\n")
|
||||
return nil
|
||||
}
|
||||
|
|
@ -839,28 +881,32 @@ func runCompactDolt() error {
|
|||
return SilentExit()
|
||||
}
|
||||
|
||||
// Get size after GC
|
||||
sizeAfter, err := getDirSize(doltPath)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Warning: could not calculate directory size after GC: %v\n", err)
|
||||
sizeAfter = 0
|
||||
}
|
||||
// Measure the same active-database scope after GC.
|
||||
sizeAfter := storeSizeBytes(ctx)
|
||||
|
||||
elapsed := time.Since(start)
|
||||
freed := sizeBefore - sizeAfter
|
||||
if freed < 0 {
|
||||
freed = 0 // GC may not always reduce size
|
||||
}
|
||||
|
||||
if jsonOutput {
|
||||
// Preserve compact's legacy size_before/size_after field names;
|
||||
// addGCSizeJSON uses a different schema and is not interchangeable.
|
||||
result := map[string]interface{}{
|
||||
"success": true,
|
||||
"dolt_path": doltPath,
|
||||
"size_before": sizeBefore,
|
||||
"size_after": sizeAfter,
|
||||
"freed_bytes": freed,
|
||||
"freed_display": formatBytes(freed),
|
||||
"elapsed_ms": elapsed.Milliseconds(),
|
||||
"success": true,
|
||||
"dolt_path": doltPath,
|
||||
"elapsed_ms": elapsed.Milliseconds(),
|
||||
}
|
||||
if sizeBefore >= 0 {
|
||||
result["size_before"] = sizeBefore
|
||||
}
|
||||
if sizeAfter >= 0 {
|
||||
result["size_after"] = sizeAfter
|
||||
}
|
||||
if sizeBefore >= 0 && sizeAfter >= 0 {
|
||||
freed := sizeBefore - sizeAfter
|
||||
if freed < 0 {
|
||||
freed = 0 // GC may not always reduce size
|
||||
}
|
||||
result["freed_bytes"] = freed
|
||||
result["freed_display"] = formatBytes(freed)
|
||||
}
|
||||
if err := outputJSON(result); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
||||
|
|
@ -869,7 +915,9 @@ func runCompactDolt() error {
|
|||
}
|
||||
|
||||
fmt.Printf("✓ Dolt garbage collection complete\n")
|
||||
fmt.Printf(" %s → %s (freed %s)\n", formatBytes(sizeBefore), formatBytes(sizeAfter), formatBytes(freed))
|
||||
if line := gcSizeLine(sizeBefore, sizeAfter); line != "" {
|
||||
fmt.Printf(" %s\n", line)
|
||||
}
|
||||
fmt.Printf(" Time: %v\n", elapsed)
|
||||
return nil
|
||||
}
|
||||
|
|
@ -897,21 +945,6 @@ func isUnknownArchiveLevelFlagError(output string) bool {
|
|||
strings.Contains(lower, "flag provided but not defined")
|
||||
}
|
||||
|
||||
// getDirSize calculates the total size of a directory recursively
|
||||
func getDirSize(path string) (int64, error) {
|
||||
var size int64
|
||||
err := filepath.Walk(path, func(_ string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !info.IsDir() {
|
||||
size += info.Size()
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return size, err
|
||||
}
|
||||
|
||||
// formatBytes formats a byte count as a human-readable string
|
||||
func formatBytes(b int64) string {
|
||||
const unit = 1024
|
||||
|
|
@ -962,7 +995,7 @@ func init() {
|
|||
compactCmd.Flags().StringVar(&compactSummary, "summary", "", "Path to summary file (use '-' for stdin)")
|
||||
compactCmd.Flags().StringVar(&compactActor, "actor", "agent", "Actor name for audit trail")
|
||||
compactCmd.Flags().IntVar(&compactLimit, "limit", 0, "Limit number of candidates (0 = no limit)")
|
||||
compactCmd.Flags().BoolVar(&compactDolt, "dolt", false, "Dolt mode: run Dolt garbage collection on .beads/dolt")
|
||||
compactCmd.Flags().BoolVar(&compactDolt, "dolt", false, "Dolt mode: run Dolt garbage collection on the active database")
|
||||
|
||||
// Note: compactCmd is added to adminCmd in admin.go
|
||||
}
|
||||
|
|
|
|||
88
cmd/bd/compact_gc_size_test.go
Normal file
88
cmd/bd/compact_gc_size_test.go
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
)
|
||||
|
||||
func TestRunCompactDoltDryRunUsesActiveDatabaseSizer(t *testing.T) {
|
||||
output := runCompactDoltDryRunForTest(t, &gcSizeStoreStub{size: 42}, true)
|
||||
|
||||
var result map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(output), &result); err != nil {
|
||||
t.Fatalf("decode dry-run JSON: %v\noutput: %s", err, output)
|
||||
}
|
||||
if got := result["size_before"]; got != float64(42) {
|
||||
t.Fatalf("size_before = %#v, want 42 from ActiveDatabaseSizer", got)
|
||||
}
|
||||
if got := result["size_display"]; got != "42 B" {
|
||||
t.Fatalf("size_display = %#v, want %q", got, "42 B")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunCompactDoltDryRunOmitsUnsupportedSize(t *testing.T) {
|
||||
unsupported := &gcSizeStoreStub{err: &storage.ErrUnsupported{
|
||||
Op: "ActiveDatabaseSize",
|
||||
Backend: "external",
|
||||
}}
|
||||
|
||||
t.Run("JSON", func(t *testing.T) {
|
||||
output := runCompactDoltDryRunForTest(t, unsupported, true)
|
||||
var result map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(output), &result); err != nil {
|
||||
t.Fatalf("decode dry-run JSON: %v\noutput: %s", err, output)
|
||||
}
|
||||
for _, key := range []string{"size_before", "size_display"} {
|
||||
if got, ok := result[key]; ok {
|
||||
t.Fatalf("unsupported measurement emitted %s=%#v", key, got)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("text", func(t *testing.T) {
|
||||
output := runCompactDoltDryRunForTest(t, unsupported, false)
|
||||
if strings.Contains(output, "Current size:") {
|
||||
t.Fatalf("unsupported measurement emitted a text size line:\n%s", output)
|
||||
}
|
||||
if !strings.Contains(output, "Run without --dry-run") {
|
||||
t.Fatalf("dry-run completion guidance missing:\n%s", output)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func runCompactDoltDryRunForTest(t *testing.T, candidate storage.DoltStorage, asJSON bool) string {
|
||||
t.Helper()
|
||||
|
||||
oldStore, oldDryRun, oldJSON := store, compactDryRun, jsonOutput
|
||||
t.Cleanup(func() {
|
||||
store, compactDryRun, jsonOutput = oldStore, oldDryRun, oldJSON
|
||||
})
|
||||
|
||||
beadsDir := filepath.Join(t.TempDir(), ".beads")
|
||||
doltDir := filepath.Join(beadsDir, "dolt")
|
||||
if err := os.MkdirAll(doltDir, 0o755); err != nil {
|
||||
t.Fatalf("create shared Dolt root: %v", err)
|
||||
}
|
||||
// This sibling-root content makes the old recursive-walk behavior return a
|
||||
// value distinct from the fake active database measurement.
|
||||
if err := os.WriteFile(filepath.Join(doltDir, "sibling-data"), make([]byte, 128), 0o644); err != nil {
|
||||
t.Fatalf("seed sibling data: %v", err)
|
||||
}
|
||||
t.Setenv("BEADS_DIR", beadsDir)
|
||||
|
||||
active := filepath.Join(doltDir, "active")
|
||||
if err := os.Mkdir(active, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
store = &compactGCStoreStub{DoltStorage: candidate, directory: active}
|
||||
compactDryRun = true
|
||||
jsonOutput = asJSON
|
||||
return captureStdout(t, func() error {
|
||||
return runCompactDolt(t.Context())
|
||||
})
|
||||
}
|
||||
325
cmd/bd/compact_gc_target_test.go
Normal file
325
cmd/bd/compact_gc_target_test.go
Normal file
|
|
@ -0,0 +1,325 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/beads"
|
||||
"github.com/steveyegge/beads/internal/hooks"
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
)
|
||||
|
||||
type compactGCStoreStub struct {
|
||||
storage.DoltStorage
|
||||
directory string
|
||||
err error
|
||||
}
|
||||
|
||||
func (s *compactGCStoreStub) ExternalGCPath(context.Context) (string, error) {
|
||||
return s.directory, s.err
|
||||
}
|
||||
|
||||
func (s *compactGCStoreStub) ActiveDatabaseSize(ctx context.Context) (int64, error) {
|
||||
if sizer, ok := s.DoltStorage.(storage.ActiveDatabaseSizer); ok {
|
||||
return sizer.ActiveDatabaseSize(ctx)
|
||||
}
|
||||
return 0, &storage.ErrUnsupported{Op: "ActiveDatabaseSize", Backend: "fixture"}
|
||||
}
|
||||
|
||||
func TestRunCompactDoltTargetsOnlyAuthorizedActiveDatabase(t *testing.T) {
|
||||
clearTelemetryEnv(t)
|
||||
t.Setenv("BD_JSON_ENVELOPE", "0")
|
||||
toolDir := buildCompactGCFixture(t)
|
||||
// wantBackend pins which ErrUnsupported.Backend the refusal names, which is
|
||||
// the only observable that distinguishes "the active store refused" (the
|
||||
// store's own "external" sentinel, reached through storage.UnwrapStore) from
|
||||
// "the interface assertion missed" ("active store"). Both render the same
|
||||
// top-level message, so the *through decorators* rows are vacuous for the
|
||||
// decorator claim without it.
|
||||
//
|
||||
// Known residual: "unsupported dry run through decorators" still cannot
|
||||
// discriminate. Its benign dry-run preview is byte-identical for both error
|
||||
// origins and carries no backend, so no CLI-level assertion distinguishes
|
||||
// them; the non-dry-run decorator row is what pins that path.
|
||||
for _, tc := range []struct {
|
||||
name, mode, pathKind string
|
||||
shared, dry, unsupported bool
|
||||
decorated bool
|
||||
plain, envelope bool
|
||||
calls int
|
||||
wantErr bool
|
||||
wantMessage string
|
||||
wantBackend string
|
||||
wantHint string
|
||||
}{
|
||||
{name: "owned active", calls: 1},
|
||||
{name: "shared active despite stale project root", shared: true, calls: 1},
|
||||
{name: "owned active through decorators", decorated: true, calls: 1},
|
||||
{name: "shared active through decorators", decorated: true, shared: true, calls: 1},
|
||||
{name: "older CLI fallback", mode: "fallback", calls: 2},
|
||||
{name: "real failure is not retried", mode: "failure", calls: 1, wantErr: true},
|
||||
{name: "dry run", dry: true},
|
||||
{name: "unsupported with plausible local path", unsupported: true, wantErr: true, wantBackend: "external"},
|
||||
{name: "unsupported text output", unsupported: true, plain: true, wantErr: true, wantBackend: "external"},
|
||||
{name: "unsupported enveloped output", unsupported: true, envelope: true, wantErr: true, wantBackend: "external"},
|
||||
{name: "unsupported dry run", unsupported: true, dry: true},
|
||||
{name: "unsupported through decorators", decorated: true, unsupported: true, wantErr: true, wantBackend: "external"},
|
||||
{name: "unsupported dry run through decorators", decorated: true, unsupported: true, dry: true},
|
||||
{name: "cancelled before the active database is resolved", pathKind: "cancelled", wantErr: true,
|
||||
wantMessage: "external Dolt garbage collection was interrupted", wantHint: "Re-run 'bd admin compact --dolt'"},
|
||||
{name: "cancelled in dry run", pathKind: "cancelled", dry: true, wantErr: true,
|
||||
wantMessage: "external Dolt garbage collection was interrupted", wantHint: "Re-run 'bd admin compact --dolt'"},
|
||||
{name: "deadline exceeded before the active database is resolved", pathKind: "deadline", wantErr: true,
|
||||
wantMessage: "external Dolt garbage collection was interrupted", wantHint: "Re-run 'bd admin compact --dolt'"},
|
||||
{name: "size capability is insufficient", pathKind: "size-only", wantErr: true, wantBackend: "active store"},
|
||||
{name: "general locator is insufficient", pathKind: "locator-only", wantErr: true, wantBackend: "active store"},
|
||||
{name: "missing declared path", pathKind: "missing", wantErr: true, wantMessage: "active Dolt database directory is unavailable"},
|
||||
{name: "missing declared path in dry run", pathKind: "missing", dry: true, wantErr: true, wantMessage: "active Dolt database directory is unavailable"},
|
||||
{name: "file is not a database directory", pathKind: "file", wantErr: true, wantMessage: "is not a directory"},
|
||||
{name: "empty declared path", pathKind: "empty", wantErr: true, wantMessage: "requires an absolute active database directory"},
|
||||
{name: "relative declared path", pathKind: "relative", wantErr: true, wantMessage: "requires an absolute active database directory"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
oldStore, oldDry, oldJSON := store, compactDryRun, jsonOutput
|
||||
beads.ResetCaches()
|
||||
t.Cleanup(func() { store, compactDryRun, jsonOutput = oldStore, oldDry, oldJSON; beads.ResetCaches() })
|
||||
project := t.TempDir()
|
||||
beadsDir := filepath.Join(project, ".beads")
|
||||
stale := filepath.Join(beadsDir, "dolt")
|
||||
root := stale
|
||||
if tc.shared {
|
||||
root = filepath.Join(t.TempDir(), "central", "dolt")
|
||||
}
|
||||
active := filepath.Join(root, "active")
|
||||
sibling := filepath.Join(root, "sibling")
|
||||
for _, dir := range []string{stale, active, sibling} {
|
||||
if err := os.MkdirAll(dir, 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "keep"), []byte("unchanged"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
t.Setenv("BEADS_DIR", beadsDir)
|
||||
t.Setenv("PATH", toolDir+string(os.PathListSeparator)+os.Getenv("PATH"))
|
||||
logPath := filepath.Join(t.TempDir(), "calls.jsonl")
|
||||
t.Setenv("BEADS_COMPACT_GC_FIXTURE_LOG", logPath)
|
||||
t.Setenv("BEADS_COMPACT_GC_FIXTURE_MODE", tc.mode)
|
||||
candidate := &compactGCStoreStub{DoltStorage: &gcSizeStoreStub{size: 42}, directory: active}
|
||||
if tc.unsupported {
|
||||
candidate.err = &storage.ErrUnsupported{Op: "ExternalGCPath", Backend: "external"}
|
||||
}
|
||||
store = candidate
|
||||
switch tc.pathKind {
|
||||
case "size-only":
|
||||
store = &gcSizeStoreStub{size: 42}
|
||||
case "locator-only":
|
||||
store = &gcLocatorOnlyStoreStub{path: active}
|
||||
case "missing":
|
||||
candidate.directory = filepath.Join(root, "missing")
|
||||
case "file":
|
||||
candidate.directory = filepath.Join(active, "keep")
|
||||
case "empty":
|
||||
candidate.directory = ""
|
||||
case "relative":
|
||||
candidate.directory = "relative-database"
|
||||
case "cancelled":
|
||||
// What the real DoltStore.ExternalGCPath returns for a cancelled
|
||||
// ctx: the bare context error, not an ErrUnsupported.
|
||||
candidate.err = context.Canceled
|
||||
case "deadline":
|
||||
candidate.err = context.DeadlineExceeded
|
||||
}
|
||||
if tc.decorated {
|
||||
t.Setenv("BD_OTEL_STDOUT", "true")
|
||||
store = wireStorageDecorators(store, hooks.NewRunner(filepath.Join(project, "missing-hooks")), false)
|
||||
if storage.UnwrapStore(store) != candidate {
|
||||
t.Fatal("production decorator chain lost the active store")
|
||||
}
|
||||
}
|
||||
compactDryRun, jsonOutput = tc.dry, !tc.plain
|
||||
if tc.envelope {
|
||||
t.Setenv("BD_JSON_ENVELOPE", "1")
|
||||
}
|
||||
var runErr error
|
||||
var output string
|
||||
if tc.wantErr {
|
||||
output = captureStderr(t, func() { runErr = runCompactDolt(t.Context()) })
|
||||
} else {
|
||||
output = captureStdout(t, func() error { runErr = runCompactDolt(t.Context()); return nil })
|
||||
}
|
||||
if (runErr != nil) != tc.wantErr {
|
||||
t.Fatalf("runCompactDolt error = %v, wantErr %v", runErr, tc.wantErr)
|
||||
}
|
||||
if tc.wantErr {
|
||||
if code, ok := exitCodeFromError(runErr); !ok || code != 1 {
|
||||
t.Fatalf("refusal exit code = %d, recognized %v; error %v", code, ok, runErr)
|
||||
}
|
||||
}
|
||||
if tc.wantErr && tc.mode != "failure" {
|
||||
message, hint := output, output
|
||||
if tc.plain {
|
||||
if !strings.HasPrefix(output, "Error: ") || !strings.Contains(output, "\nHint: ") {
|
||||
t.Fatalf("missing text error/hint: %q", output)
|
||||
}
|
||||
} else {
|
||||
var result map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(output), &result); err != nil {
|
||||
t.Fatalf("error stderr is not JSON: %v: %s", err, output)
|
||||
}
|
||||
if result["schema_version"] != float64(JSONSchemaVersion) {
|
||||
t.Fatalf("missing error schema version: %v", result)
|
||||
}
|
||||
if tc.envelope {
|
||||
var ok bool
|
||||
result, ok = result["data"].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("missing error envelope data: %s", output)
|
||||
}
|
||||
}
|
||||
message, _ = result["error"].(string)
|
||||
hint, _ = result["hint"].(string)
|
||||
}
|
||||
wantMessage := tc.wantMessage
|
||||
if wantMessage == "" {
|
||||
wantMessage = "cannot select a local database for external Dolt garbage collection"
|
||||
}
|
||||
wantHint := tc.wantHint
|
||||
if wantHint == "" {
|
||||
wantHint = "bd doctor"
|
||||
}
|
||||
if !strings.Contains(message, wantMessage) || !strings.Contains(hint, wantHint) {
|
||||
t.Fatalf("refusal lost its cause or actionable hint: %s", output)
|
||||
}
|
||||
// A cancellation must not be dressed as a permanent authority
|
||||
// refusal: neither the refusal message nor its four-cause hint may
|
||||
// appear, in dry-run or otherwise.
|
||||
if tc.pathKind == "cancelled" || tc.pathKind == "deadline" {
|
||||
for _, forbidden := range []string{"cannot select a local database", "BEADS_DOLT_SERVER_PORT", "server administrator"} {
|
||||
if strings.Contains(message, forbidden) || strings.Contains(hint, forbidden) {
|
||||
t.Errorf("cancellation reported as an authority refusal (%q present): %s", forbidden, output)
|
||||
}
|
||||
}
|
||||
}
|
||||
if tc.wantBackend != "" {
|
||||
wantBackend := "not supported by the " + tc.wantBackend + " backend"
|
||||
if !strings.Contains(message, wantBackend) {
|
||||
t.Errorf("refusal names the wrong origin: want %q in %q", wantBackend, message)
|
||||
}
|
||||
}
|
||||
if tc.unsupported {
|
||||
for _, condition := range []string{"BEADS_DOLT_SERVER_PORT", "BEADS_DOLT_PORT", "auto-start", "external", "gateway", "proxied", "socket", "TLS", "server administrator"} {
|
||||
if !strings.Contains(hint, condition) {
|
||||
t.Errorf("unsupported hint omits %q: %s", condition, hint)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
data, err := os.ReadFile(logPath)
|
||||
if err != nil && !os.IsNotExist(err) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var calls []struct {
|
||||
Dir string
|
||||
Args []string
|
||||
}
|
||||
for _, line := range strings.Split(strings.TrimSpace(string(data)), "\n") {
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
var call struct {
|
||||
Dir string
|
||||
Args []string
|
||||
}
|
||||
if err := json.Unmarshal([]byte(line), &call); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
calls = append(calls, call)
|
||||
}
|
||||
if len(calls) != tc.calls {
|
||||
t.Fatalf("external GC calls = %v, want %d", calls, tc.calls)
|
||||
}
|
||||
for i, call := range calls {
|
||||
wantArgs := []string{"gc", "--archive-level", "0"}
|
||||
if i == 1 {
|
||||
wantArgs = []string{"gc"}
|
||||
}
|
||||
gotInfo, err := os.Stat(call.Dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wantInfo, err := os.Stat(active)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !os.SameFile(gotInfo, wantInfo) || !reflect.DeepEqual(call.Args, wantArgs) {
|
||||
t.Fatalf("GC call = %+v, want directory %q, args %v", call, active, wantArgs)
|
||||
}
|
||||
}
|
||||
for _, dir := range []string{stale, active, sibling} {
|
||||
data, err := os.ReadFile(filepath.Join(dir, "keep"))
|
||||
if err != nil || string(data) != "unchanged" {
|
||||
t.Fatalf("changed sentinel in %q: %q, %v", dir, data, err)
|
||||
}
|
||||
_, err = os.Stat(filepath.Join(dir, "gc-ran"))
|
||||
wantMarker := dir == active && tc.calls > 0 && !tc.wantErr
|
||||
if (err == nil) != wantMarker {
|
||||
t.Fatalf("GC marker in %q: %v, want marker %v", dir, err, wantMarker)
|
||||
}
|
||||
}
|
||||
if !tc.wantErr {
|
||||
var result map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(output), &result); err != nil {
|
||||
t.Fatalf("JSON: %v: %s", err, output)
|
||||
}
|
||||
if tc.unsupported {
|
||||
_, hasDoltPath := result["dolt_path"]
|
||||
if result["available"] != false || hasDoltPath {
|
||||
t.Fatalf("unsupported target was guessed: %v", result)
|
||||
}
|
||||
} else if result["dolt_path"] != active || result["size_before"] != float64(42) {
|
||||
t.Fatalf("reported scope = %v, want active %q and size42", result, active)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func buildCompactGCFixture(t *testing.T) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
const source = `package main
|
||||
import ("encoding/json"; "fmt"; "os")
|
||||
func main() {
|
||||
dir, err := os.Getwd(); if err != nil { panic(err) }
|
||||
log, err := os.OpenFile(os.Getenv("BEADS_COMPACT_GC_FIXTURE_LOG"), os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0600); if err != nil { panic(err) }
|
||||
if err := json.NewEncoder(log).Encode(struct { Dir string; Args []string }{dir, os.Args[1:]}); err != nil { panic(err) }; if err := log.Close(); err != nil { panic(err) }
|
||||
mode := os.Getenv("BEADS_COMPACT_GC_FIXTURE_MODE")
|
||||
if mode == "fallback" && len(os.Args) == 4 { fmt.Fprintln(os.Stderr, "unknown flag: --archive-level"); os.Exit(23) }
|
||||
if mode == "failure" { fmt.Fprintln(os.Stderr, "genuine GC failure"); os.Exit(23) }
|
||||
if err := os.WriteFile("gc-ran", []byte("collected"), 0600); err != nil { panic(err) }
|
||||
}
|
||||
`
|
||||
path := filepath.Join(dir, "fixture.go")
|
||||
if err := os.WriteFile(path, []byte(source), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
name := "dolt"
|
||||
if runtime.GOOS == "windows" {
|
||||
name += ".exe"
|
||||
}
|
||||
cmd := exec.Command("go", "build", "-o", filepath.Join(dir, name), path)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "CGO_ENABLED=0", "GOWORK=off")
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("build owned GC fixture: %v\n%s", err, output)
|
||||
}
|
||||
return dir
|
||||
}
|
||||
|
|
@ -39,7 +39,7 @@ This is irreversible — all commit history is lost. The resulting database
|
|||
has exactly one commit containing all current data.
|
||||
|
||||
Use this when:
|
||||
- Your .beads/dolt directory has grown very large
|
||||
- Your active database has grown very large
|
||||
- You don't need commit-level history (time travel)
|
||||
- You want to start fresh with minimal storage
|
||||
|
||||
|
|
|
|||
|
|
@ -212,6 +212,7 @@ go_test(
|
|||
"events_since_test.go",
|
||||
"events_test.go",
|
||||
"expected_revision_contract_test.go",
|
||||
"external_gc_path_test.go",
|
||||
"federation_pull_settle_test.go",
|
||||
"federation_test.go",
|
||||
"field_length_test.go",
|
||||
|
|
|
|||
42
internal/storage/dolt/external_gc_path_test.go
Normal file
42
internal/storage/dolt/external_gc_path_test.go
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
package dolt
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
)
|
||||
|
||||
func TestDoltStoreExternalGCPathKeepsInstanceAuthority(t *testing.T) {
|
||||
clearSizingModeEnv(t)
|
||||
beadsDir := t.TempDir()
|
||||
cfg := &Config{Path: filepath.Join(beadsDir, "dolt"), BeadsDir: beadsDir, Database: "active", ServerHost: "localhost", AutoStart: true}
|
||||
owned := &DoltStore{localActiveDatabaseDir: resolveLocalActiveDatabaseDir(cfg)}
|
||||
wantOwned := filepath.Join(cfg.Path, "active")
|
||||
t.Setenv("BEADS_DOLT_SERVER_PORT", "44001")
|
||||
external := &DoltStore{dbPath: cfg.Path, database: cfg.Database, localActiveDatabaseDir: resolveLocalActiveDatabaseDir(cfg)}
|
||||
t.Setenv("BEADS_DOLT_SERVER_PORT", "")
|
||||
sharedRoot := t.TempDir()
|
||||
t.Setenv("BEADS_DOLT_SHARED_SERVER", "1")
|
||||
t.Setenv("BEADS_SHARED_SERVER_DIR", sharedRoot)
|
||||
shared := &DoltStore{localActiveDatabaseDir: resolveLocalActiveDatabaseDir(cfg)}
|
||||
wantShared := filepath.Join(sharedRoot, "dolt", "active")
|
||||
for _, mode := range []string{"", "1"} {
|
||||
t.Setenv("BEADS_DOLT_SHARED_SERVER", mode)
|
||||
t.Setenv("BEADS_SHARED_SERVER_DIR", t.TempDir())
|
||||
for _, tc := range []struct {
|
||||
store *DoltStore
|
||||
want string
|
||||
}{{owned, wantOwned}, {shared, wantShared}} {
|
||||
if got, err := tc.store.ExternalGCPath(t.Context()); err != nil || got != tc.want {
|
||||
t.Fatalf("ExternalGCPath = %q, %v; want captured %q", got, err, tc.want)
|
||||
}
|
||||
}
|
||||
got, err := external.ExternalGCPath(t.Context())
|
||||
var unsupported *storage.ErrUnsupported
|
||||
if got != "" || !errors.As(err, &unsupported) {
|
||||
t.Fatalf("external instance gained local GC authority: %q, %v", got, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1762,6 +1762,17 @@ func resolveLocalActiveDatabaseDir(cfg *Config) string {
|
|||
|
||||
// Owned mode plus effective auto-start authority is the affirmative proof
|
||||
// that the configured data root belongs to this local beads instance.
|
||||
//
|
||||
// The AutoStart conjunct is deliberate and is deliberately stricter than
|
||||
// ownership alone: ResolveServerMode already decides ownership, so this
|
||||
// clause is a fail-closed backstop that rides resolveAutoStart's own
|
||||
// exclusions (it is forced false for ServerModeExternal and under
|
||||
// BEADS_TEST_MODE=1). The accepted cost is that an owned workspace which
|
||||
// sets "dolt.auto-start: false" and starts its server by hand is refused
|
||||
// local external GC; the CLI hint names that remedy explicitly. Relaxing
|
||||
// this to ResolveServerMode alone would widen GC authority and is a
|
||||
// behavior change, not a cleanup — do not drop it without re-reviewing the
|
||||
// test-mode and external-server paths above.
|
||||
if !cfg.AutoStart || doltserver.ResolveServerMode(cfg.BeadsDir) != doltserver.ServerModeOwned {
|
||||
return ""
|
||||
}
|
||||
|
|
@ -3346,6 +3357,18 @@ func (s *DoltStore) ActiveDatabaseSize(ctx context.Context) (int64, error) {
|
|||
return size, nil
|
||||
}
|
||||
|
||||
// ExternalGCPath uses the local authority captured when this store was opened.
|
||||
// Later environment changes or stale client-local paths cannot grant it.
|
||||
func (s *DoltStore) ExternalGCPath(ctx context.Context) (string, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if s.localActiveDatabaseDir == "" {
|
||||
return "", &storage.ErrUnsupported{Op: "ExternalGCPath", Backend: "dolt-server"}
|
||||
}
|
||||
return s.localActiveDatabaseDir, nil
|
||||
}
|
||||
|
||||
// DoltGC runs Dolt's default, generational garbage collection to reclaim disk
|
||||
// space. Pins a single connection to avoid session state loss on pooled *sql.DB.
|
||||
func (s *DoltStore) DoltGC(ctx context.Context) error {
|
||||
|
|
|
|||
|
|
@ -691,6 +691,14 @@ type ActiveDatabaseSizer interface {
|
|||
ActiveDatabaseSize(ctx context.Context) (int64, error)
|
||||
}
|
||||
|
||||
// ExternalGCLocator supplies the authoritative local working directory for
|
||||
// the active database's external garbage-collection tool. Implementations
|
||||
// return *ErrUnsupported when this instance cannot authorize that operation.
|
||||
// Neither a readable size nor a general store path grants this capability.
|
||||
type ExternalGCLocator interface {
|
||||
ExternalGCPath(ctx context.Context) (string, error)
|
||||
}
|
||||
|
||||
// GarbageCollector provides Dolt garbage collection capability.
|
||||
// Callers that need to reclaim disk space should type-assert to this interface.
|
||||
type GarbageCollector interface {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue