* fix(compact): report active database size (#5619) Measure legacy admin compact --dolt reporting through ActiveDatabaseSizer so sibling databases cannot distort reclaimed-space numbers. Omit unavailable advisory measurements without blocking GC. Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com> Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell * docs(compact): explain the legacy size field names Keep the existing compact JSON schema distinct from the newer GC helper. The active external-GC target is addressed by the separate issue6313 leaf. * fix(compact): select the authoritative local active GC directory Ask the active storage instance for its optional external-GC directory capability. Reuse the authority captured at store open, and refuse stale, unsupported or invalid local targets rather than guessing from a root, general CLI path or advisory size. Preserve archive-level fallback behavior. Add native owned-process target and no-launch regressions plus instance authority tests; retain the parent active-size and legacy JSON contracts. Fixes #6313 * fix(compact): resolve GC capability through storage decorators Use the existing storage unwrapping boundary so external GC sees the captured active database behind the root command's dependency, telemetry and hook decorators. Cover owned, shared and unsupported stores through the real chain, and describe the active database in compact help. Refs gastownhall/beads#6313 Agent: unknown-model/unknown-reasoning * fix(compact): preserve external GC refusal diagnostics (#6313) Route all four new target refusals through the standard error/hint formatter and assert their stderr schema, message, hint and exit status. Preserve the relative-path refusal and unavailable dry-run target omission. Update the requested compact and flatten help mirrors without regenerating other docs. Addresses the diagnostics and documentation review on PR #6327. Existing PR #5668 ancestry and mandatory parent-first sequencing remain unchanged. Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell * docs(compact): align flag help with active GC target (saa-cgh.94) Authored by /root/cgh86_review; closes the independent root review help finding. Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell * docs: retain release-pinned CLI references (saa-cgh.94) Preserve corrected live compact/flatten help while generated references continue to match docs/cli-docs.pin v1.2.2. Fix the exact six-line hosted docs-policy failure without changing pin authority. Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell * fix(compact): classify cancellation before the GC authority refusal Maintainer review fixes for #6327, review iteration 1. Non-gating findings only: the synthesis and the quality scorecard both returned approve with no blocker and no major finding. Taken: - Error Handling & Resilience (minor, both lanes). runCompactDolt formatted a cancelled or expired context as the permanent "cannot select a local database" refusal plus its four-cause hint, and in --dry-run turned it into a hard exit-1. Classify context.Canceled/DeadlineExceeded ahead of the ErrUnsupported arm and report the interruption instead. The ctx probe stays in ExternalGCPath: it is the only cancellation checkpoint before the uninterruptible external `dolt gc`, so the finding's other arm (drop the probe) would trade a misleading message for launching a multi-minute GC on an interrupted command. - Change Impact / Blast Radius (minor). Document why resolveLocalActiveDatabaseDir keeps the cfg.AutoStart conjunct, and make the CLI hint name the remedy (`bd config set dolt.auto-start true`) instead of only the condition. The offered code arm is declined: AutoStart rides resolveAutoStart's own exclusions (forced false for ServerModeExternal and under BEADS_TEST_MODE=1), so dropping it widens GC authority - a behavior change, not a cleanup. - Release Safety (minor). Add the three new test files to the gazelle explicit go_test srcs lists so this PR's assertions are visible under `bazel test`. Verified with `bazel run //:gazelle -- --mode=diff`: gazelle requests none of the three, so the rows match its own output. The two rows it still wants are pre-existing base drift in unrelated files and are left alone. - Test Evidence Quality (nit). The two "unsupported ... through decorators" rows were measured vacuous for the decorator claim. Pin the refusal's origin through ErrUnsupported.Backend ("external" = the active store refused, "active store" = the interface assertion missed) and pin the opposite polarity on the size-only/locator-only rows. Residual recorded on the test: the dry-run decorator row still cannot discriminate, because its benign preview is identical for both origins. Declined, and carried to the human-approval summary: - Debuggability & Operability (minor, gate impact none). Reusing the `bd gc` companions adds production output in an iteration no further review pass grades, and the alternative arm would assert the refs are left in place "deliberately", which nothing in the change establishes. - Resource Lifecycle & Cleanup (nit, Required fix: none). exec.CommandContext would kill a mid-flight `dolt gc`; that is a data-safety question for the GC path rather than a cleanup. Validation: `go build ./...` and `go vet` on cmd/bd + internal/storage{,/dolt} clean; scripts/ci/pr-lint.sh against the pinned base green (gofmt, golangci-lint 0 issues, windows cross-lint 0 issues), proved non-vacuous by a misspell planted on an added line; 66 PASS / 0 FAIL / 0 SKIP over the covering cmd/bd selectors. Both new pins are mutation-proved: removing the classification arm reds exactly the three new cancellation rows and nothing else, and dropping storage.UnwrapStore now reds three decorator rows where it previously reddened two. internal/storage/dolt/store.go is comment-only. --------- Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local> Co-authored-by: Gas City Merge Queue <merge-queue@gascity.local>
1001 lines
31 KiB
Go
1001 lines
31 KiB
Go
package main
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/spf13/cobra"
|
|
"github.com/steveyegge/beads/internal/beads"
|
|
"github.com/steveyegge/beads/internal/compact"
|
|
"github.com/steveyegge/beads/internal/config"
|
|
"github.com/steveyegge/beads/internal/storage"
|
|
"github.com/steveyegge/beads/internal/types"
|
|
)
|
|
|
|
var (
|
|
compactDryRun bool
|
|
compactTier int
|
|
compactAll bool
|
|
compactID string
|
|
compactForce bool
|
|
compactBatch int
|
|
compactWorkers int
|
|
compactStats bool
|
|
compactAnalyze bool
|
|
compactApply bool
|
|
compactAuto bool
|
|
compactSummary string
|
|
compactActor string
|
|
compactLimit int
|
|
compactDolt bool
|
|
)
|
|
|
|
var compactCmd = &cobra.Command{
|
|
Use: "compact",
|
|
Short: "Compact old closed issues to save space",
|
|
SilenceUsage: true,
|
|
SilenceErrors: true,
|
|
Long: `Compact old closed issues using semantic summarization.
|
|
|
|
Compaction reduces database size by summarizing closed issues that are no longer
|
|
actively referenced. This is permanent graceful decay - original content is discarded.
|
|
|
|
Modes:
|
|
- Analyze: Export candidates for agent review (no API key needed)
|
|
- Apply: Accept agent-provided summary (no API key needed)
|
|
- Auto: AI-powered compaction (requires ANTHROPIC_API_KEY, MINIMAX_API_KEY, or ai.api_key)
|
|
- Dolt: Run Dolt garbage collection (for Dolt-backend repositories)
|
|
|
|
Tiers:
|
|
- Tier 1: Semantic compression (30 days closed, 70% reduction)
|
|
- Tier 2: Ultra compression (90 days closed) - planned, not yet implemented
|
|
|
|
Dolt Garbage Collection:
|
|
With auto-commit per mutation, Dolt commit history grows over time. Use
|
|
--dolt to run Dolt garbage collection and reclaim disk space.
|
|
|
|
--dolt: Run Dolt GC on the active database to free disk space.
|
|
This removes unreachable commits and compacts storage.
|
|
|
|
Examples:
|
|
# Dolt garbage collection
|
|
bd compact --dolt # Run Dolt GC
|
|
bd compact --dolt --dry-run # Preview without running GC
|
|
|
|
# Agent-driven workflow (recommended)
|
|
bd compact --analyze --json # Get candidates with full content
|
|
bd compact --apply --id bd-42 --summary summary.txt
|
|
bd compact --apply --id bd-42 --summary - < summary.txt
|
|
|
|
# AI-powered workflow
|
|
bd compact --auto --dry-run # Preview candidates
|
|
bd compact --auto --all # Compact all eligible issues
|
|
bd compact --auto --id bd-42 # Compact specific issue
|
|
MINIMAX_API_KEY=... bd compact --auto --all
|
|
|
|
# Statistics
|
|
bd compact --stats # Show statistics
|
|
`,
|
|
RunE: func(_ *cobra.Command, _ []string) error {
|
|
if usesProxiedServer() {
|
|
if compactDolt {
|
|
return runCompactDoltProxiedServer(rootCtx)
|
|
}
|
|
// Same string the "admin compact" refusal row carries; the
|
|
// pre-provider gate short-circuits this path, so the two are kept
|
|
// identical rather than left to drift.
|
|
return HandleErrorRespectJSON("only 'bd admin compact --dolt' is supported in proxied-server mode")
|
|
}
|
|
// Block mutating operations in embedded mode; allow --stats, --analyze, --dry-run read-only paths.
|
|
if !compactStats && !compactAnalyze && !compactDryRun {
|
|
if err := requireServerMode("compact"); err != nil {
|
|
return HandleError("%v", err)
|
|
}
|
|
}
|
|
// Compact modifies data unless --stats or --analyze or --dry-run or --dolt with --dry-run
|
|
if !compactStats && !compactAnalyze && !compactDryRun && !(compactDolt && compactDryRun) {
|
|
CheckReadonly("compact")
|
|
}
|
|
ctx := rootCtx
|
|
|
|
// Handle compact stats first
|
|
if compactStats {
|
|
return runCompactStats(ctx, store)
|
|
}
|
|
|
|
// Handle dolt GC mode
|
|
if compactDolt {
|
|
return runCompactDolt(ctx)
|
|
}
|
|
|
|
// Count active modes
|
|
activeModes := 0
|
|
if compactAnalyze {
|
|
activeModes++
|
|
}
|
|
if compactApply {
|
|
activeModes++
|
|
}
|
|
if compactAuto {
|
|
activeModes++
|
|
}
|
|
|
|
// Check for exactly one mode
|
|
if activeModes == 0 {
|
|
return HandleError("must specify one mode: --analyze, --apply, or --auto")
|
|
}
|
|
if activeModes > 1 {
|
|
return HandleError("cannot use multiple modes together (--analyze, --apply, --auto are mutually exclusive)")
|
|
}
|
|
|
|
// Only Tier 1 compaction is implemented. Reject other tiers up front with
|
|
// a clear message rather than failing deep inside a mode.
|
|
if compactTier != 1 {
|
|
return HandleError("Tier %d compaction is not yet implemented; only --tier 1 is available", compactTier)
|
|
}
|
|
|
|
// Handle analyze mode (requires direct database access)
|
|
if compactAnalyze {
|
|
if err := ensureDirectMode("compact --analyze requires direct database access"); err != nil {
|
|
return HandleErrorWithHint(err.Error(), diagHint())
|
|
}
|
|
return runCompactAnalyze(ctx, store)
|
|
}
|
|
|
|
// Handle apply mode (requires direct database access)
|
|
if compactApply {
|
|
if err := ensureDirectMode("compact --apply requires direct database access"); err != nil {
|
|
return HandleErrorWithHint(err.Error(), diagHint())
|
|
}
|
|
if compactID == "" {
|
|
return HandleError("--apply requires --id")
|
|
}
|
|
if compactSummary == "" {
|
|
return HandleError("--apply requires --summary")
|
|
}
|
|
return runCompactApply(ctx, store)
|
|
}
|
|
|
|
// Handle auto mode (legacy)
|
|
if compactAuto {
|
|
// Validation checks
|
|
if compactID != "" && compactAll {
|
|
return HandleError("cannot use --id and --all together")
|
|
}
|
|
if compactForce && compactID == "" {
|
|
return HandleError("--force requires --id")
|
|
}
|
|
if compactID == "" && !compactAll && !compactDryRun {
|
|
return HandleError("must specify --all, --id, or --dry-run")
|
|
}
|
|
|
|
// Direct mode
|
|
apiKey, _ := config.ResolveAIAPIKey("")
|
|
if apiKey == "" && !compactDryRun {
|
|
return HandleError("--auto mode requires ANTHROPIC_API_KEY, MINIMAX_API_KEY, or ai.api_key in config")
|
|
}
|
|
|
|
compactCfg := &compact.Config{
|
|
APIKey: apiKey,
|
|
Concurrency: compactWorkers,
|
|
DryRun: compactDryRun,
|
|
}
|
|
|
|
compactor, err := compact.New(store, apiKey, compactCfg)
|
|
if err != nil {
|
|
return HandleError("failed to create compactor: %v", err)
|
|
}
|
|
|
|
if compactID != "" {
|
|
return runCompactSingle(ctx, compactor, store, compactID)
|
|
}
|
|
|
|
return runCompactAll(ctx, compactor, store)
|
|
}
|
|
return nil
|
|
},
|
|
}
|
|
|
|
func runCompactSingle(ctx context.Context, compactor *compact.Compactor, store storage.DoltStorage, issueID string) error {
|
|
start := time.Now()
|
|
|
|
if !compactForce {
|
|
eligible, reason, err := store.CheckEligibility(ctx, issueID, compactTier)
|
|
if err != nil {
|
|
return HandleError("failed to check eligibility: %v", err)
|
|
}
|
|
if !eligible {
|
|
return HandleError("%s is not eligible for Tier %d compaction: %s", issueID, compactTier, reason)
|
|
}
|
|
}
|
|
|
|
issue, err := store.GetIssue(ctx, issueID)
|
|
if err != nil {
|
|
return HandleError("failed to get issue: %v", err)
|
|
}
|
|
|
|
originalSize := len(issue.Description) + len(issue.Design) + len(issue.Notes) + len(issue.AcceptanceCriteria)
|
|
|
|
if compactDryRun {
|
|
ageDays := 0
|
|
var closedAtStr string
|
|
if issue.ClosedAt != nil {
|
|
ageDays = int(time.Since(*issue.ClosedAt).Hours() / 24)
|
|
closedAtStr = issue.ClosedAt.Format(time.RFC3339)
|
|
}
|
|
|
|
candidate := map[string]interface{}{
|
|
"id": issueID,
|
|
"title": issue.Title,
|
|
"closed_at": closedAtStr,
|
|
"age_days": ageDays,
|
|
"content_size": originalSize,
|
|
}
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"dry_run": true,
|
|
"tier": compactTier,
|
|
"candidates": []interface{}{candidate},
|
|
"summary": map[string]interface{}{
|
|
"total_candidates": 1,
|
|
"total_content_bytes": originalSize,
|
|
},
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("DRY RUN - Tier %d compaction\n\n", compactTier)
|
|
fmt.Printf(" %-12s %-40s %8s %10s\n", "ID", "TITLE", "AGE", "SIZE")
|
|
title := issue.Title
|
|
if len(title) > 40 {
|
|
title = title[:37] + "..."
|
|
}
|
|
fmt.Printf(" %-12s %-40s %5dd %10d B\n", issueID, title, ageDays, originalSize)
|
|
fmt.Printf("\nSummary: 1 candidate, %d bytes total content\n", originalSize)
|
|
return nil
|
|
}
|
|
|
|
var compactErr error
|
|
if compactTier == 1 {
|
|
compactErr = compactor.CompactTier1(ctx, issueID)
|
|
} else {
|
|
return HandleError("Tier 2 compaction not yet implemented")
|
|
}
|
|
|
|
if compactErr != nil {
|
|
return HandleError("%v", compactErr)
|
|
}
|
|
|
|
issue, err = store.GetIssue(ctx, issueID)
|
|
if err != nil {
|
|
return HandleError("failed to get updated issue: %v", err)
|
|
}
|
|
|
|
compactedSize := len(issue.Description)
|
|
savingBytes := originalSize - compactedSize
|
|
elapsed := time.Since(start)
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"success": true,
|
|
"tier": compactTier,
|
|
"issue_id": issueID,
|
|
"original_size": originalSize,
|
|
"compacted_size": compactedSize,
|
|
"saved_bytes": savingBytes,
|
|
"reduction_pct": float64(savingBytes) / float64(originalSize) * 100,
|
|
"elapsed_ms": elapsed.Milliseconds(),
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("✓ Compacted %s (Tier %d)\n", issueID, compactTier)
|
|
fmt.Printf(" %d → %d bytes (saved %d, %.1f%%)\n",
|
|
originalSize, compactedSize, savingBytes,
|
|
float64(savingBytes)/float64(originalSize)*100)
|
|
fmt.Printf(" Time: %v\n", elapsed)
|
|
return nil
|
|
}
|
|
|
|
func runCompactAll(ctx context.Context, compactor *compact.Compactor, store storage.DoltStorage) error {
|
|
start := time.Now()
|
|
|
|
var candidates []string
|
|
if compactTier == 1 {
|
|
tier1, err := store.GetTier1Candidates(ctx)
|
|
if err != nil {
|
|
return HandleError("failed to get candidates: %v", err)
|
|
}
|
|
for _, c := range tier1 {
|
|
candidates = append(candidates, c.IssueID)
|
|
}
|
|
} else {
|
|
tier2, err := store.GetTier2Candidates(ctx)
|
|
if err != nil {
|
|
return HandleError("failed to get candidates: %v", err)
|
|
}
|
|
for _, c := range tier2 {
|
|
candidates = append(candidates, c.IssueID)
|
|
}
|
|
}
|
|
|
|
if len(candidates) == 0 {
|
|
if jsonOutput {
|
|
if err := outputJSON(map[string]interface{}{
|
|
"success": true,
|
|
"count": 0,
|
|
"message": "No eligible candidates",
|
|
}); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
fmt.Println("No eligible candidates for compaction")
|
|
return nil
|
|
}
|
|
|
|
if compactDryRun {
|
|
type dryRunCandidate struct {
|
|
ID string `json:"id"`
|
|
Title string `json:"title"`
|
|
ClosedAt string `json:"closed_at"`
|
|
AgeDays int `json:"age_days"`
|
|
ContentSize int `json:"content_size"`
|
|
}
|
|
|
|
var dryCandidates []dryRunCandidate
|
|
totalSize := 0
|
|
for _, id := range candidates {
|
|
issue, err := store.GetIssue(ctx, id)
|
|
if err != nil {
|
|
continue
|
|
}
|
|
contentSize := len(issue.Description) + len(issue.Design) + len(issue.Notes) + len(issue.AcceptanceCriteria)
|
|
totalSize += contentSize
|
|
|
|
ageDays := 0
|
|
var closedAtStr string
|
|
if issue.ClosedAt != nil {
|
|
ageDays = int(time.Since(*issue.ClosedAt).Hours() / 24)
|
|
closedAtStr = issue.ClosedAt.Format(time.RFC3339)
|
|
}
|
|
|
|
dryCandidates = append(dryCandidates, dryRunCandidate{
|
|
ID: issue.ID,
|
|
Title: issue.Title,
|
|
ClosedAt: closedAtStr,
|
|
AgeDays: ageDays,
|
|
ContentSize: contentSize,
|
|
})
|
|
}
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"dry_run": true,
|
|
"tier": compactTier,
|
|
"candidates": dryCandidates,
|
|
"summary": map[string]interface{}{
|
|
"total_candidates": len(dryCandidates),
|
|
"total_content_bytes": totalSize,
|
|
},
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("DRY RUN - Tier %d compaction\n\n", compactTier)
|
|
fmt.Printf(" %-12s %-40s %8s %10s\n", "ID", "TITLE", "AGE", "SIZE")
|
|
for _, c := range dryCandidates {
|
|
title := c.Title
|
|
if len(title) > 40 {
|
|
title = title[:37] + "..."
|
|
}
|
|
fmt.Printf(" %-12s %-40s %5dd %10d B\n", c.ID, title, c.AgeDays, c.ContentSize)
|
|
}
|
|
fmt.Printf("\nSummary: %d candidates, %d bytes total content\n", len(dryCandidates), totalSize)
|
|
return nil
|
|
}
|
|
|
|
if !jsonOutput {
|
|
fmt.Printf("Compacting %d issues (Tier %d)...\n\n", len(candidates), compactTier)
|
|
}
|
|
|
|
results, err := compactor.CompactTier1Batch(ctx, candidates)
|
|
if err != nil {
|
|
return HandleError("batch compaction failed: %v", err)
|
|
}
|
|
|
|
successCount := 0
|
|
failCount := 0
|
|
totalSaved := 0
|
|
totalOriginal := 0
|
|
|
|
for i, result := range results {
|
|
if !jsonOutput {
|
|
fmt.Printf("[%s] %d/%d\r", progressBar(i+1, len(results)), i+1, len(results))
|
|
}
|
|
|
|
if result.Err != nil {
|
|
failCount++
|
|
} else {
|
|
successCount++
|
|
totalOriginal += result.OriginalSize
|
|
totalSaved += (result.OriginalSize - result.CompactedSize)
|
|
}
|
|
}
|
|
|
|
elapsed := time.Since(start)
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"success": true,
|
|
"tier": compactTier,
|
|
"total": len(results),
|
|
"succeeded": successCount,
|
|
"failed": failCount,
|
|
"saved_bytes": totalSaved,
|
|
"original_size": totalOriginal,
|
|
"elapsed_ms": elapsed.Milliseconds(),
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("\n\nCompleted in %v\n\n", elapsed)
|
|
fmt.Printf("Summary:\n")
|
|
fmt.Printf(" Succeeded: %d\n", successCount)
|
|
fmt.Printf(" Failed: %d\n", failCount)
|
|
if totalOriginal > 0 {
|
|
fmt.Printf(" Saved: %d bytes (%.1f%%)\n", totalSaved, float64(totalSaved)/float64(totalOriginal)*100)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func runCompactStats(ctx context.Context, store storage.DoltStorage) error {
|
|
tier1, err := store.GetTier1Candidates(ctx)
|
|
if err != nil {
|
|
return HandleError("failed to get Tier 1 candidates: %v", err)
|
|
}
|
|
|
|
tier2, err := store.GetTier2Candidates(ctx)
|
|
if err != nil {
|
|
return HandleError("failed to get Tier 2 candidates: %v", err)
|
|
}
|
|
|
|
tier1Size := 0
|
|
for _, c := range tier1 {
|
|
tier1Size += c.OriginalSize
|
|
}
|
|
|
|
tier2Size := 0
|
|
for _, c := range tier2 {
|
|
tier2Size += c.OriginalSize
|
|
}
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"tier1": map[string]interface{}{
|
|
"candidates": len(tier1),
|
|
"total_size": tier1Size,
|
|
},
|
|
"tier2": map[string]interface{}{
|
|
"candidates": len(tier2),
|
|
"total_size": tier2Size,
|
|
"implemented": false,
|
|
},
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Println("Compaction Statistics")
|
|
fmt.Printf("Tier 1 (30+ days closed):\n")
|
|
fmt.Printf(" Candidates: %d\n", len(tier1))
|
|
fmt.Printf(" Total size: %d bytes\n", tier1Size)
|
|
if tier1Size > 0 {
|
|
fmt.Printf(" Estimated savings: %d bytes (70%%)\n\n", tier1Size*7/10)
|
|
}
|
|
|
|
fmt.Printf("Tier 2 (90+ days closed, Tier 1 compacted): not yet implemented\n")
|
|
fmt.Printf(" Candidates: %d\n", len(tier2))
|
|
fmt.Printf(" Total size: %d bytes\n", tier2Size)
|
|
return nil
|
|
}
|
|
|
|
func runCompactAnalyze(ctx context.Context, store storage.DoltStorage) error {
|
|
type Candidate struct {
|
|
ID string `json:"id"`
|
|
Title string `json:"title"`
|
|
Description string `json:"description"`
|
|
Design string `json:"design"`
|
|
Notes string `json:"notes"`
|
|
AcceptanceCriteria string `json:"acceptance_criteria"`
|
|
SizeBytes int `json:"size_bytes"`
|
|
AgeDays int `json:"age_days"`
|
|
Tier int `json:"tier"`
|
|
Compacted bool `json:"compacted"`
|
|
}
|
|
|
|
var candidates []Candidate
|
|
|
|
// Single issue mode
|
|
if compactID != "" {
|
|
issue, err := store.GetIssue(ctx, compactID)
|
|
if err != nil {
|
|
return HandleError("failed to get issue: %v", err)
|
|
}
|
|
|
|
sizeBytes := len(issue.Description) + len(issue.Design) + len(issue.Notes) + len(issue.AcceptanceCriteria)
|
|
ageDays := 0
|
|
if issue.ClosedAt != nil {
|
|
ageDays = int(time.Since(*issue.ClosedAt).Hours() / 24)
|
|
}
|
|
|
|
candidates = append(candidates, Candidate{
|
|
ID: issue.ID,
|
|
Title: issue.Title,
|
|
Description: issue.Description,
|
|
Design: issue.Design,
|
|
Notes: issue.Notes,
|
|
AcceptanceCriteria: issue.AcceptanceCriteria,
|
|
SizeBytes: sizeBytes,
|
|
AgeDays: ageDays,
|
|
Tier: compactTier,
|
|
Compacted: issue.CompactionLevel > 0,
|
|
})
|
|
} else {
|
|
// Get tier candidates
|
|
var tierCandidates []*types.CompactionCandidate
|
|
var err error
|
|
if compactTier == 1 {
|
|
tierCandidates, err = store.GetTier1Candidates(ctx)
|
|
} else {
|
|
tierCandidates, err = store.GetTier2Candidates(ctx)
|
|
}
|
|
if err != nil {
|
|
return HandleError("failed to get candidates: %v", err)
|
|
}
|
|
|
|
// Apply limit if specified
|
|
if compactLimit > 0 && len(tierCandidates) > compactLimit {
|
|
tierCandidates = tierCandidates[:compactLimit]
|
|
}
|
|
|
|
// Fetch full details for each candidate
|
|
for _, c := range tierCandidates {
|
|
issue, err := store.GetIssue(ctx, c.IssueID)
|
|
if err != nil {
|
|
continue // Skip issues we can't fetch
|
|
}
|
|
|
|
ageDays := int(time.Since(c.ClosedAt).Hours() / 24)
|
|
|
|
candidates = append(candidates, Candidate{
|
|
ID: issue.ID,
|
|
Title: issue.Title,
|
|
Description: issue.Description,
|
|
Design: issue.Design,
|
|
Notes: issue.Notes,
|
|
AcceptanceCriteria: issue.AcceptanceCriteria,
|
|
SizeBytes: c.OriginalSize,
|
|
AgeDays: ageDays,
|
|
Tier: compactTier,
|
|
Compacted: issue.CompactionLevel > 0,
|
|
})
|
|
}
|
|
}
|
|
|
|
if jsonOutput {
|
|
totalSize := 0
|
|
for _, c := range candidates {
|
|
totalSize += c.SizeBytes
|
|
}
|
|
output := map[string]interface{}{
|
|
"candidates": candidates,
|
|
"summary": map[string]interface{}{
|
|
"total_candidates": len(candidates),
|
|
"total_content_bytes": totalSize,
|
|
},
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// Human-readable output
|
|
fmt.Printf("Compaction Candidates (Tier %d)\n\n", compactTier)
|
|
fmt.Printf(" %-12s %-40s %8s %10s\n", "ID", "TITLE", "AGE", "SIZE")
|
|
totalSize := 0
|
|
for _, c := range candidates {
|
|
compactStatus := ""
|
|
if c.Compacted {
|
|
compactStatus = " *"
|
|
}
|
|
title := c.Title
|
|
if len(title) > 40 {
|
|
title = title[:37] + "..."
|
|
}
|
|
fmt.Printf(" %-12s %-40s %5dd %10d B%s\n", c.ID, title, c.AgeDays, c.SizeBytes, compactStatus)
|
|
totalSize += c.SizeBytes
|
|
}
|
|
fmt.Printf("\nSummary: %d candidates, %d bytes total content\n", len(candidates), totalSize)
|
|
return nil
|
|
}
|
|
|
|
func runCompactApply(ctx context.Context, store storage.DoltStorage) error {
|
|
start := time.Now()
|
|
|
|
// Read summary
|
|
var summaryBytes []byte
|
|
var err error
|
|
if compactSummary == "-" {
|
|
// Read from stdin
|
|
summaryBytes, err = io.ReadAll(os.Stdin)
|
|
if err != nil {
|
|
return HandleError("failed to read summary from stdin: %v", err)
|
|
}
|
|
} else {
|
|
// #nosec G304 -- summary file path provided explicitly by operator
|
|
summaryBytes, err = os.ReadFile(compactSummary)
|
|
if err != nil {
|
|
return HandleError("failed to read summary file: %v", err)
|
|
}
|
|
}
|
|
summary := string(summaryBytes)
|
|
|
|
// Get issue
|
|
issue, err := store.GetIssue(ctx, compactID)
|
|
if err != nil {
|
|
return HandleError("failed to get issue: %v", err)
|
|
}
|
|
|
|
// Calculate sizes
|
|
originalSize := len(issue.Description) + len(issue.Design) + len(issue.Notes) + len(issue.AcceptanceCriteria)
|
|
compactedSize := len(summary)
|
|
|
|
// Check eligibility unless --force
|
|
if !compactForce {
|
|
eligible, reason, err := store.CheckEligibility(ctx, compactID, compactTier)
|
|
if err != nil {
|
|
return HandleError("failed to check eligibility: %v", err)
|
|
}
|
|
if !eligible {
|
|
return HandleErrorWithHint(fmt.Sprintf("%s is not eligible for Tier %d compaction: %s", compactID, compactTier, reason), "use --force to bypass eligibility checks")
|
|
}
|
|
|
|
// Enforce size reduction unless --force
|
|
if compactedSize >= originalSize {
|
|
return HandleErrorWithHint(fmt.Sprintf("summary (%d bytes) is not shorter than original (%d bytes)", compactedSize, originalSize), "use --force to bypass size validation")
|
|
}
|
|
}
|
|
|
|
// Apply compaction
|
|
actor := compactActor
|
|
if actor == "" {
|
|
actor = "agent"
|
|
}
|
|
|
|
updates := map[string]interface{}{
|
|
"description": summary,
|
|
"design": "",
|
|
"notes": "",
|
|
"acceptance_criteria": "",
|
|
}
|
|
|
|
if err := store.UpdateIssue(ctx, compactID, updates, actor); err != nil {
|
|
return HandleError("failed to update issue: %v", err)
|
|
}
|
|
|
|
commitHash := compact.GetCurrentCommitHash()
|
|
if err := store.ApplyCompaction(ctx, compactID, compactTier, originalSize, compactedSize, commitHash); err != nil {
|
|
return HandleError("failed to apply compaction: %v", err)
|
|
}
|
|
|
|
savingBytes := originalSize - compactedSize
|
|
reductionPct := float64(savingBytes) / float64(originalSize) * 100
|
|
eventData := fmt.Sprintf("Tier %d compaction: %d → %d bytes (saved %d, %.1f%%)", compactTier, originalSize, compactedSize, savingBytes, reductionPct)
|
|
if err := store.AddComment(ctx, compactID, actor, eventData); err != nil {
|
|
return HandleError("failed to record event: %v", err)
|
|
}
|
|
|
|
elapsed := time.Since(start)
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"success": true,
|
|
"issue_id": compactID,
|
|
"tier": compactTier,
|
|
"original_size": originalSize,
|
|
"compacted_size": compactedSize,
|
|
"saved_bytes": savingBytes,
|
|
"reduction_pct": reductionPct,
|
|
"elapsed_ms": elapsed.Milliseconds(),
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("✓ Compacted %s (Tier %d)\n", compactID, compactTier)
|
|
fmt.Printf(" %d → %d bytes (saved %d, %.1f%%)\n", originalSize, compactedSize, savingBytes, reductionPct)
|
|
fmt.Printf(" Time: %v\n", elapsed)
|
|
return nil
|
|
}
|
|
|
|
// runCompactDolt runs external Dolt GC in the locally owned active database.
|
|
func runCompactDolt(ctx context.Context) error {
|
|
start := time.Now()
|
|
|
|
// Find beads directory
|
|
beadsDir := beads.FindBeadsDir()
|
|
if beadsDir == "" {
|
|
return HandleErrorWithHint(activeWorkspaceNotFoundError(), diagHint())
|
|
}
|
|
|
|
// Resolve operation authority from the opened store, never a stale project
|
|
// directory, general CLI path, or advisory size measurement.
|
|
var doltPath string
|
|
var pathErr error
|
|
if locator, ok := storage.UnwrapStore(store).(storage.ExternalGCLocator); ok {
|
|
doltPath, pathErr = locator.ExternalGCPath(ctx)
|
|
} else {
|
|
pathErr = &storage.ErrUnsupported{Op: "ExternalGCPath", Backend: "active store"}
|
|
}
|
|
if pathErr != nil {
|
|
// Classify cancellation before formatting. ExternalGCPath probes ctx, so an
|
|
// interrupt or deadline surfaces here as a bare context error, and it is not
|
|
// an authority refusal: without this arm a SIGINT renders as the permanent
|
|
// "cannot select a local database" message plus the four-cause hint about
|
|
// port overrides and proxied servers, none of which applies. The ctx probe
|
|
// stays in ExternalGCPath deliberately — it is the only cancellation
|
|
// checkpoint before the uninterruptible external `dolt gc` below.
|
|
if errors.Is(pathErr, context.Canceled) || errors.Is(pathErr, context.DeadlineExceeded) {
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("external Dolt garbage collection was interrupted: %v", pathErr),
|
|
"the command stopped before the active database was resolved, so no garbage collection ran. "+
|
|
"Re-run 'bd admin compact --dolt' to collect.")
|
|
}
|
|
var unsupported *storage.ErrUnsupported
|
|
if compactDryRun && errors.As(pathErr, &unsupported) {
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"dry_run": true,
|
|
"available": false,
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
fmt.Printf("DRY RUN - Dolt garbage collection\n\n")
|
|
fmt.Printf("The active database is not available for local external garbage collection.\n")
|
|
return nil
|
|
}
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("cannot select a local database for external Dolt garbage collection: %v", pathErr),
|
|
"--dolt requires a locally managed owned/shared Dolt database. Port overrides (BEADS_DOLT_SERVER_PORT/BEADS_DOLT_PORT), "+
|
|
"external/gateway/proxied servers, and socket/TLS connections do not authorize local GC; owned mode also requires "+
|
|
"auto-start, so a hand-started owned server needs 'bd config set dolt.auto-start true'. "+
|
|
"Run 'bd doctor' to inspect the connection or ask the server administrator to run GC.")
|
|
}
|
|
if !filepath.IsAbs(doltPath) {
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("external Dolt garbage collection requires an absolute active database directory, got %q", doltPath),
|
|
"run 'bd doctor' and correct the active database configuration to use an absolute directory")
|
|
}
|
|
info, err := os.Stat(doltPath)
|
|
if err != nil {
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("active Dolt database directory is unavailable: %v", err),
|
|
"check that the active database directory exists and is accessible; run 'bd doctor' to inspect the database configuration")
|
|
}
|
|
if !info.IsDir() {
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("active Dolt database path %q is not a directory", doltPath),
|
|
"run 'bd doctor' and correct the active database configuration to point to a directory, not a file")
|
|
}
|
|
|
|
// Measure only the active database. The shared .beads/dolt root may contain
|
|
// sibling databases unrelated to this GC operation.
|
|
sizeBefore := storeSizeBytes(ctx)
|
|
|
|
if compactDryRun {
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"dry_run": true,
|
|
"dolt_path": doltPath,
|
|
}
|
|
if sizeBefore >= 0 {
|
|
output["size_before"] = sizeBefore
|
|
output["size_display"] = formatBytes(sizeBefore)
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
fmt.Printf("DRY RUN - Dolt garbage collection\n\n")
|
|
fmt.Printf("Dolt directory: %s\n", doltPath)
|
|
if sizeBefore >= 0 {
|
|
fmt.Printf("Current size: %s\n", formatBytes(sizeBefore))
|
|
}
|
|
fmt.Printf("\nRun without --dry-run to perform garbage collection.\n")
|
|
return nil
|
|
}
|
|
|
|
// Check if dolt command is available
|
|
if _, err := exec.LookPath("dolt"); err != nil {
|
|
return HandleErrorWithHint("dolt command not found in PATH", "install Dolt from https://github.com/dolthub/dolt")
|
|
}
|
|
|
|
if !jsonOutput {
|
|
fmt.Printf("Running Dolt garbage collection...\n")
|
|
}
|
|
|
|
// Run dolt gc without archive compression. Level 0 writes classic Snappy
|
|
// table files instead of zstd archives, matching the in-process GC paths.
|
|
// The external `dolt` on PATH has no version guarantee (unlike the
|
|
// in-process paths, which are pinned by go.mod), so an older dolt that
|
|
// predates --archive-level would otherwise abort compact where plain
|
|
// `dolt gc` used to work. Detect that specific unknown-flag rejection
|
|
// and retry with plain `dolt gc` rather than fail outright; any other
|
|
// error (a genuine GC failure) is not swallowed.
|
|
cmd := exec.Command("dolt", "gc", "--archive-level", "0") // #nosec G204 -- fixed command, no user input
|
|
cmd.Dir = doltPath
|
|
output, err := cmd.CombinedOutput()
|
|
if err != nil && isUnknownArchiveLevelFlagError(string(output)) {
|
|
if !jsonOutput {
|
|
fmt.Fprintf(os.Stderr, "Notice: external dolt does not support --archive-level; falling back to plain 'dolt gc' (new table files may still use zstd archives)\n")
|
|
}
|
|
fallbackCmd := exec.Command("dolt", "gc") // #nosec G204 -- fixed command, no user input
|
|
fallbackCmd.Dir = doltPath
|
|
output, err = fallbackCmd.CombinedOutput()
|
|
}
|
|
if err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: dolt gc failed: %v\n", err)
|
|
if len(output) > 0 {
|
|
fmt.Fprintf(os.Stderr, "Output: %s\n", string(output))
|
|
}
|
|
return SilentExit()
|
|
}
|
|
|
|
// Measure the same active-database scope after GC.
|
|
sizeAfter := storeSizeBytes(ctx)
|
|
|
|
elapsed := time.Since(start)
|
|
|
|
if jsonOutput {
|
|
// Preserve compact's legacy size_before/size_after field names;
|
|
// addGCSizeJSON uses a different schema and is not interchangeable.
|
|
result := map[string]interface{}{
|
|
"success": true,
|
|
"dolt_path": doltPath,
|
|
"elapsed_ms": elapsed.Milliseconds(),
|
|
}
|
|
if sizeBefore >= 0 {
|
|
result["size_before"] = sizeBefore
|
|
}
|
|
if sizeAfter >= 0 {
|
|
result["size_after"] = sizeAfter
|
|
}
|
|
if sizeBefore >= 0 && sizeAfter >= 0 {
|
|
freed := sizeBefore - sizeAfter
|
|
if freed < 0 {
|
|
freed = 0 // GC may not always reduce size
|
|
}
|
|
result["freed_bytes"] = freed
|
|
result["freed_display"] = formatBytes(freed)
|
|
}
|
|
if err := outputJSON(result); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("✓ Dolt garbage collection complete\n")
|
|
if line := gcSizeLine(sizeBefore, sizeAfter); line != "" {
|
|
fmt.Printf(" %s\n", line)
|
|
}
|
|
fmt.Printf(" Time: %v\n", elapsed)
|
|
return nil
|
|
}
|
|
|
|
// isUnknownArchiveLevelFlagError reports whether output (the combined
|
|
// stdout+stderr of `dolt gc --archive-level 0`) indicates the external dolt
|
|
// binary rejected --archive-level as an unrecognized flag, rather than a
|
|
// genuine GC failure. Older Dolt releases that predate the flag report this
|
|
// via the pinned dolt module's argparser, e.g.:
|
|
//
|
|
// error: unknown option `archive-level'
|
|
//
|
|
// (see libraries/utils/argparser/errors.go in the pinned dolthub/dolt/go
|
|
// module). The check requires both an "unknown flag" phrasing AND the flag
|
|
// name to appear in the output, so a real GC failure that happens to
|
|
// contain the word "unknown" elsewhere is never misclassified as a missing
|
|
// flag and silently swallowed — genuine failures continue to fail.
|
|
func isUnknownArchiveLevelFlagError(output string) bool {
|
|
lower := strings.ToLower(output)
|
|
if !strings.Contains(lower, "archive-level") && !strings.Contains(lower, "archive_level") {
|
|
return false
|
|
}
|
|
return strings.Contains(lower, "unknown option") ||
|
|
strings.Contains(lower, "unknown flag") ||
|
|
strings.Contains(lower, "flag provided but not defined")
|
|
}
|
|
|
|
// formatBytes formats a byte count as a human-readable string
|
|
func formatBytes(b int64) string {
|
|
const unit = 1024
|
|
if b < unit {
|
|
return fmt.Sprintf("%d B", b)
|
|
}
|
|
div, exp := int64(unit), 0
|
|
for n := b / unit; n >= unit; n /= unit {
|
|
div *= unit
|
|
exp++
|
|
}
|
|
return fmt.Sprintf("%.1f %cB", float64(b)/float64(div), "KMGTPE"[exp])
|
|
}
|
|
|
|
// progressBar renders a text-based progress bar.
|
|
func progressBar(current, total int) string {
|
|
const width = 40
|
|
if total == 0 {
|
|
return "[" + string(make([]byte, width)) + "]"
|
|
}
|
|
filled := (current * width) / total
|
|
bar := ""
|
|
for i := 0; i < width; i++ {
|
|
if i < filled {
|
|
bar += "█"
|
|
} else {
|
|
bar += " "
|
|
}
|
|
}
|
|
return "[" + bar + "]"
|
|
}
|
|
|
|
func init() {
|
|
compactCmd.Flags().BoolVar(&compactDryRun, "dry-run", false, "Preview without compacting")
|
|
compactCmd.Flags().IntVar(&compactTier, "tier", 1, "Compaction tier (only tier 1 is implemented)")
|
|
compactCmd.Flags().BoolVar(&compactAll, "all", false, "Process all candidates")
|
|
compactCmd.Flags().StringVar(&compactID, "id", "", "Compact specific issue")
|
|
compactCmd.Flags().BoolVar(&compactForce, "force", false, "Force compact (bypass checks, requires --id)")
|
|
compactCmd.Flags().IntVar(&compactBatch, "batch-size", 10, "Issues per batch")
|
|
compactCmd.Flags().IntVar(&compactWorkers, "workers", 5, "Parallel workers")
|
|
compactCmd.Flags().BoolVar(&compactStats, "stats", false, "Show compaction statistics")
|
|
compactCmd.Flags().BoolVar(&jsonOutput, "json", false, "Output JSON format")
|
|
|
|
// New mode flags
|
|
compactCmd.Flags().BoolVar(&compactAnalyze, "analyze", false, "Analyze mode: export candidates for agent review")
|
|
compactCmd.Flags().BoolVar(&compactApply, "apply", false, "Apply mode: accept agent-provided summary")
|
|
compactCmd.Flags().BoolVar(&compactAuto, "auto", false, "Auto mode: AI-powered compaction")
|
|
compactCmd.Flags().StringVar(&compactSummary, "summary", "", "Path to summary file (use '-' for stdin)")
|
|
compactCmd.Flags().StringVar(&compactActor, "actor", "agent", "Actor name for audit trail")
|
|
compactCmd.Flags().IntVar(&compactLimit, "limit", 0, "Limit number of candidates (0 = no limit)")
|
|
compactCmd.Flags().BoolVar(&compactDolt, "dolt", false, "Dolt mode: run Dolt garbage collection on the active database")
|
|
|
|
// Note: compactCmd is added to adminCmd in admin.go
|
|
}
|