* fix(cli): stop eleven subcommands shadowing the root --json flag
--json is a persistent flag on rootCmd bound to jsonOutput. Ten
subcommands (restore, admin compact, repo add/remove/list/sync,
migrate, migrate sync/hooks/schema) also registered a local --json bound
to the same variable. pflag's AddFlagSet keeps the local flag and drops
the inherited one, so rootCmd.PersistentFlags().Changed("json") stayed
false for those commands; PersistentPreRunE and
refreshBoundCommandConfig then treated the flag as unset and reset
jsonOutput from config. `bd restore <id> --json` and friends printed text
unless config said json: true. restore_embedded_test.go recorded the
symptom as a "known issue" and skipped its JSON assertion.
bd preflight registered an unbound local Bool("json") instead; it read
its own flag and was not affected by the reset, but it shadowed the root
flag the same way and ignored a configured json default. It now inherits
the root flag like status, update and the proxied close/reopen routes.
Remove the eleven local registrations. Add
TestNoSubcommandShadowsRootJSONFlag, which walks the command tree and
fails on any command that registers its own --json (local or
persistent). Restore the restore_json assertion (parse the output, check
ID and description). Flip the compact and migrate schema registration
tests to assert the flag is inherited, matching stale_test.go.
Visible change: --json moves from each command's Flags section to Global
Flags in --help, with the root description. docs/cli-reference is pinned
to the release tag and regenerates at release time.
* fix(cli): make the --json shadow guard side-effect free
TestNoSubcommandShadowsRootJSONFlag walked every command through
LocalNonPersistentFlags(), and the --json assertions in compact_test.go and
migrate_test.go used InheritedFlags(). Both call cobra's
mergePersistentFlags(), which copies every root persistent flag into the
command's Flags() for the rest of the test process. TestServeFlags pins the
exact flag set of `bd serve`, so `go test ./cmd/bd` failed whenever it ran
after the guard; CI only stayed green because the two tests land in
different name-hash shards.
Compare the command's Flags().Lookup("json") against the root flag pointer
instead. A local registration is a different *pflag.Flag; an inherited one
is the same pointer, and a command that has not merged yet returns nil.
Verified: the four tests pass together, and re-adding a local --json to
`bd restore` still fails the guard.
bd preflight also drops its `jsonOutput, _ := cmd.Flags().GetBool("json")`
shadow of the package-level variable. The root flag is bound to that
variable and PersistentPreRun already resolves the flag/config precedence,
so the local copy was redundant and swallowed the lookup error.
* review: re-key the preflight --json test to the root flag
Base #6293 added TestPreflightJSONFlagBindsGlobal, which opens by requiring
preflightCmd to own a local --json. This PR's guard forbids exactly that, so
the two tests were mutually exclusive and the adopted head was red. The base
test arrived with a base advance after this PR was written, so nothing in the
contributed work is wrong.
Adapt the test rather than restore the local flag: every property it pins
survives re-keyed to the root persistent flag, which is BoolVar'd to the same
package global. The opening assertion becomes the shadow-aware compare this PR
already introduced in compact_test.go, the Changed==true branch is driven
through rootCmd.PersistentFlags().Set (flag.Value.Set never sets Changed), and
the config half is unchanged. Renamed to TestPreflightJSONInheritsRootFlag so
the name stops asserting the very thing the PR removes.
Also correct three CHANGELOG clauses that were true against the contributor's
older base and false at this one: #6293's commandJSONFlagChanged already
consults a subcommand's own Changed bit before the root's, preflight's local
flag was bound rather than unbound, and #6293 already announced that preflight
honors the json config default. The bullet now claims what this PR delivers
here -- deleting the shadowing registrations instead of compensating for them.
Compile the new guard into the Bazel test target. cmd/bd/json_flag_shadow_test.go
was absent from the explicit srcs of go_test(name = "bd_test"), so the invariant
this PR exists to create never ran in the BAZEL* lanes -- and those lanes are
required for merge: CI_GATE_REQUIRED on origin/main lists BAZEL, BAZEL_TEST,
BAZEL_PURE, BAZEL_EMBEDDED, BAZEL_DOLTSERVER, BAZEL_PROXIED and
BAZEL_SERVER_STORAGE. Added in sorted position between jira_test.go and
kv_embedded_test.go; the file is untagged package main, so bd_test is the right
target under the root gazelle:build_tags gms_pure_go. make bazel-sync-check
cannot catch this -- it maintains only the go_srcs filegroup globs, not go_test
srcs.
Finally, reconcile the neighbouring #6293 bullet, which this PR invalidates. It
still described preflight's --json as "bound to the same global every sibling
command binds" and closed on "because an unbound flag reports as unset" -- both
describing the local registration deleted here, fourteen lines from the bullet
announcing the deletion and shipping in the same [Unreleased] section. Trimmed
to the user-visible outcome #6293 still owns; the new bullet carries the
registration story.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1000 lines
31 KiB
Go
1000 lines
31 KiB
Go
package main
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/spf13/cobra"
|
|
"github.com/steveyegge/beads/internal/beads"
|
|
"github.com/steveyegge/beads/internal/compact"
|
|
"github.com/steveyegge/beads/internal/config"
|
|
"github.com/steveyegge/beads/internal/storage"
|
|
"github.com/steveyegge/beads/internal/types"
|
|
)
|
|
|
|
var (
|
|
compactDryRun bool
|
|
compactTier int
|
|
compactAll bool
|
|
compactID string
|
|
compactForce bool
|
|
compactBatch int
|
|
compactWorkers int
|
|
compactStats bool
|
|
compactAnalyze bool
|
|
compactApply bool
|
|
compactAuto bool
|
|
compactSummary string
|
|
compactActor string
|
|
compactLimit int
|
|
compactDolt bool
|
|
)
|
|
|
|
var compactCmd = &cobra.Command{
|
|
Use: "compact",
|
|
Short: "Compact old closed issues to save space",
|
|
SilenceUsage: true,
|
|
SilenceErrors: true,
|
|
Long: `Compact old closed issues using semantic summarization.
|
|
|
|
Compaction reduces database size by summarizing closed issues that are no longer
|
|
actively referenced. This is permanent graceful decay - original content is discarded.
|
|
|
|
Modes:
|
|
- Analyze: Export candidates for agent review (no API key needed)
|
|
- Apply: Accept agent-provided summary (no API key needed)
|
|
- Auto: AI-powered compaction (requires ANTHROPIC_API_KEY, MINIMAX_API_KEY, or ai.api_key)
|
|
- Dolt: Run Dolt garbage collection (for Dolt-backend repositories)
|
|
|
|
Tiers:
|
|
- Tier 1: Semantic compression (30 days closed, 70% reduction)
|
|
- Tier 2: Ultra compression (90 days closed) - planned, not yet implemented
|
|
|
|
Dolt Garbage Collection:
|
|
With auto-commit per mutation, Dolt commit history grows over time. Use
|
|
--dolt to run Dolt garbage collection and reclaim disk space.
|
|
|
|
--dolt: Run Dolt GC on the active database to free disk space.
|
|
This removes unreachable commits and compacts storage.
|
|
|
|
Examples:
|
|
# Dolt garbage collection
|
|
bd compact --dolt # Run Dolt GC
|
|
bd compact --dolt --dry-run # Preview without running GC
|
|
|
|
# Agent-driven workflow (recommended)
|
|
bd compact --analyze --json # Get candidates with full content
|
|
bd compact --apply --id bd-42 --summary summary.txt
|
|
bd compact --apply --id bd-42 --summary - < summary.txt
|
|
|
|
# AI-powered workflow
|
|
bd compact --auto --dry-run # Preview candidates
|
|
bd compact --auto --all # Compact all eligible issues
|
|
bd compact --auto --id bd-42 # Compact specific issue
|
|
MINIMAX_API_KEY=... bd compact --auto --all
|
|
|
|
# Statistics
|
|
bd compact --stats # Show statistics
|
|
`,
|
|
RunE: func(_ *cobra.Command, _ []string) error {
|
|
if usesProxiedServer() {
|
|
if compactDolt {
|
|
return runCompactDoltProxiedServer(rootCtx)
|
|
}
|
|
// Same string the "admin compact" refusal row carries; the
|
|
// pre-provider gate short-circuits this path, so the two are kept
|
|
// identical rather than left to drift.
|
|
return HandleErrorRespectJSON("only 'bd admin compact --dolt' is supported in proxied-server mode")
|
|
}
|
|
// Block mutating operations in embedded mode; allow --stats, --analyze, --dry-run read-only paths.
|
|
if !compactStats && !compactAnalyze && !compactDryRun {
|
|
if err := requireServerMode("compact"); err != nil {
|
|
return HandleError("%v", err)
|
|
}
|
|
}
|
|
// Compact modifies data unless --stats or --analyze or --dry-run or --dolt with --dry-run
|
|
if !compactStats && !compactAnalyze && !compactDryRun && !(compactDolt && compactDryRun) {
|
|
CheckReadonly("compact")
|
|
}
|
|
ctx := rootCtx
|
|
|
|
// Handle compact stats first
|
|
if compactStats {
|
|
return runCompactStats(ctx, store)
|
|
}
|
|
|
|
// Handle dolt GC mode
|
|
if compactDolt {
|
|
return runCompactDolt(ctx)
|
|
}
|
|
|
|
// Count active modes
|
|
activeModes := 0
|
|
if compactAnalyze {
|
|
activeModes++
|
|
}
|
|
if compactApply {
|
|
activeModes++
|
|
}
|
|
if compactAuto {
|
|
activeModes++
|
|
}
|
|
|
|
// Check for exactly one mode
|
|
if activeModes == 0 {
|
|
return HandleError("must specify one mode: --analyze, --apply, or --auto")
|
|
}
|
|
if activeModes > 1 {
|
|
return HandleError("cannot use multiple modes together (--analyze, --apply, --auto are mutually exclusive)")
|
|
}
|
|
|
|
// Only Tier 1 compaction is implemented. Reject other tiers up front with
|
|
// a clear message rather than failing deep inside a mode.
|
|
if compactTier != 1 {
|
|
return HandleError("Tier %d compaction is not yet implemented; only --tier 1 is available", compactTier)
|
|
}
|
|
|
|
// Handle analyze mode (requires direct database access)
|
|
if compactAnalyze {
|
|
if err := ensureDirectMode("compact --analyze requires direct database access"); err != nil {
|
|
return HandleErrorWithHint(err.Error(), diagHint())
|
|
}
|
|
return runCompactAnalyze(ctx, store)
|
|
}
|
|
|
|
// Handle apply mode (requires direct database access)
|
|
if compactApply {
|
|
if err := ensureDirectMode("compact --apply requires direct database access"); err != nil {
|
|
return HandleErrorWithHint(err.Error(), diagHint())
|
|
}
|
|
if compactID == "" {
|
|
return HandleError("--apply requires --id")
|
|
}
|
|
if compactSummary == "" {
|
|
return HandleError("--apply requires --summary")
|
|
}
|
|
return runCompactApply(ctx, store)
|
|
}
|
|
|
|
// Handle auto mode (legacy)
|
|
if compactAuto {
|
|
// Validation checks
|
|
if compactID != "" && compactAll {
|
|
return HandleError("cannot use --id and --all together")
|
|
}
|
|
if compactForce && compactID == "" {
|
|
return HandleError("--force requires --id")
|
|
}
|
|
if compactID == "" && !compactAll && !compactDryRun {
|
|
return HandleError("must specify --all, --id, or --dry-run")
|
|
}
|
|
|
|
// Direct mode
|
|
apiKey, _ := config.ResolveAIAPIKey("")
|
|
if apiKey == "" && !compactDryRun {
|
|
return HandleError("--auto mode requires ANTHROPIC_API_KEY, MINIMAX_API_KEY, or ai.api_key in config")
|
|
}
|
|
|
|
compactCfg := &compact.Config{
|
|
APIKey: apiKey,
|
|
Concurrency: compactWorkers,
|
|
DryRun: compactDryRun,
|
|
}
|
|
|
|
compactor, err := compact.New(store, apiKey, compactCfg)
|
|
if err != nil {
|
|
return HandleError("failed to create compactor: %v", err)
|
|
}
|
|
|
|
if compactID != "" {
|
|
return runCompactSingle(ctx, compactor, store, compactID)
|
|
}
|
|
|
|
return runCompactAll(ctx, compactor, store)
|
|
}
|
|
return nil
|
|
},
|
|
}
|
|
|
|
func runCompactSingle(ctx context.Context, compactor *compact.Compactor, store storage.DoltStorage, issueID string) error {
|
|
start := time.Now()
|
|
|
|
if !compactForce {
|
|
eligible, reason, err := store.CheckEligibility(ctx, issueID, compactTier)
|
|
if err != nil {
|
|
return HandleError("failed to check eligibility: %v", err)
|
|
}
|
|
if !eligible {
|
|
return HandleError("%s is not eligible for Tier %d compaction: %s", issueID, compactTier, reason)
|
|
}
|
|
}
|
|
|
|
issue, err := store.GetIssue(ctx, issueID)
|
|
if err != nil {
|
|
return HandleError("failed to get issue: %v", err)
|
|
}
|
|
|
|
originalSize := len(issue.Description) + len(issue.Design) + len(issue.Notes) + len(issue.AcceptanceCriteria)
|
|
|
|
if compactDryRun {
|
|
ageDays := 0
|
|
var closedAtStr string
|
|
if issue.ClosedAt != nil {
|
|
ageDays = int(time.Since(*issue.ClosedAt).Hours() / 24)
|
|
closedAtStr = issue.ClosedAt.Format(time.RFC3339)
|
|
}
|
|
|
|
candidate := map[string]interface{}{
|
|
"id": issueID,
|
|
"title": issue.Title,
|
|
"closed_at": closedAtStr,
|
|
"age_days": ageDays,
|
|
"content_size": originalSize,
|
|
}
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"dry_run": true,
|
|
"tier": compactTier,
|
|
"candidates": []interface{}{candidate},
|
|
"summary": map[string]interface{}{
|
|
"total_candidates": 1,
|
|
"total_content_bytes": originalSize,
|
|
},
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("DRY RUN - Tier %d compaction\n\n", compactTier)
|
|
fmt.Printf(" %-12s %-40s %8s %10s\n", "ID", "TITLE", "AGE", "SIZE")
|
|
title := issue.Title
|
|
if len(title) > 40 {
|
|
title = title[:37] + "..."
|
|
}
|
|
fmt.Printf(" %-12s %-40s %5dd %10d B\n", issueID, title, ageDays, originalSize)
|
|
fmt.Printf("\nSummary: 1 candidate, %d bytes total content\n", originalSize)
|
|
return nil
|
|
}
|
|
|
|
var compactErr error
|
|
if compactTier == 1 {
|
|
compactErr = compactor.CompactTier1(ctx, issueID)
|
|
} else {
|
|
return HandleError("Tier 2 compaction not yet implemented")
|
|
}
|
|
|
|
if compactErr != nil {
|
|
return HandleError("%v", compactErr)
|
|
}
|
|
|
|
issue, err = store.GetIssue(ctx, issueID)
|
|
if err != nil {
|
|
return HandleError("failed to get updated issue: %v", err)
|
|
}
|
|
|
|
compactedSize := len(issue.Description)
|
|
savingBytes := originalSize - compactedSize
|
|
elapsed := time.Since(start)
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"success": true,
|
|
"tier": compactTier,
|
|
"issue_id": issueID,
|
|
"original_size": originalSize,
|
|
"compacted_size": compactedSize,
|
|
"saved_bytes": savingBytes,
|
|
"reduction_pct": float64(savingBytes) / float64(originalSize) * 100,
|
|
"elapsed_ms": elapsed.Milliseconds(),
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("✓ Compacted %s (Tier %d)\n", issueID, compactTier)
|
|
fmt.Printf(" %d → %d bytes (saved %d, %.1f%%)\n",
|
|
originalSize, compactedSize, savingBytes,
|
|
float64(savingBytes)/float64(originalSize)*100)
|
|
fmt.Printf(" Time: %v\n", elapsed)
|
|
return nil
|
|
}
|
|
|
|
func runCompactAll(ctx context.Context, compactor *compact.Compactor, store storage.DoltStorage) error {
|
|
start := time.Now()
|
|
|
|
var candidates []string
|
|
if compactTier == 1 {
|
|
tier1, err := store.GetTier1Candidates(ctx)
|
|
if err != nil {
|
|
return HandleError("failed to get candidates: %v", err)
|
|
}
|
|
for _, c := range tier1 {
|
|
candidates = append(candidates, c.IssueID)
|
|
}
|
|
} else {
|
|
tier2, err := store.GetTier2Candidates(ctx)
|
|
if err != nil {
|
|
return HandleError("failed to get candidates: %v", err)
|
|
}
|
|
for _, c := range tier2 {
|
|
candidates = append(candidates, c.IssueID)
|
|
}
|
|
}
|
|
|
|
if len(candidates) == 0 {
|
|
if jsonOutput {
|
|
if err := outputJSON(map[string]interface{}{
|
|
"success": true,
|
|
"count": 0,
|
|
"message": "No eligible candidates",
|
|
}); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
fmt.Println("No eligible candidates for compaction")
|
|
return nil
|
|
}
|
|
|
|
if compactDryRun {
|
|
type dryRunCandidate struct {
|
|
ID string `json:"id"`
|
|
Title string `json:"title"`
|
|
ClosedAt string `json:"closed_at"`
|
|
AgeDays int `json:"age_days"`
|
|
ContentSize int `json:"content_size"`
|
|
}
|
|
|
|
var dryCandidates []dryRunCandidate
|
|
totalSize := 0
|
|
for _, id := range candidates {
|
|
issue, err := store.GetIssue(ctx, id)
|
|
if err != nil {
|
|
continue
|
|
}
|
|
contentSize := len(issue.Description) + len(issue.Design) + len(issue.Notes) + len(issue.AcceptanceCriteria)
|
|
totalSize += contentSize
|
|
|
|
ageDays := 0
|
|
var closedAtStr string
|
|
if issue.ClosedAt != nil {
|
|
ageDays = int(time.Since(*issue.ClosedAt).Hours() / 24)
|
|
closedAtStr = issue.ClosedAt.Format(time.RFC3339)
|
|
}
|
|
|
|
dryCandidates = append(dryCandidates, dryRunCandidate{
|
|
ID: issue.ID,
|
|
Title: issue.Title,
|
|
ClosedAt: closedAtStr,
|
|
AgeDays: ageDays,
|
|
ContentSize: contentSize,
|
|
})
|
|
}
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"dry_run": true,
|
|
"tier": compactTier,
|
|
"candidates": dryCandidates,
|
|
"summary": map[string]interface{}{
|
|
"total_candidates": len(dryCandidates),
|
|
"total_content_bytes": totalSize,
|
|
},
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("DRY RUN - Tier %d compaction\n\n", compactTier)
|
|
fmt.Printf(" %-12s %-40s %8s %10s\n", "ID", "TITLE", "AGE", "SIZE")
|
|
for _, c := range dryCandidates {
|
|
title := c.Title
|
|
if len(title) > 40 {
|
|
title = title[:37] + "..."
|
|
}
|
|
fmt.Printf(" %-12s %-40s %5dd %10d B\n", c.ID, title, c.AgeDays, c.ContentSize)
|
|
}
|
|
fmt.Printf("\nSummary: %d candidates, %d bytes total content\n", len(dryCandidates), totalSize)
|
|
return nil
|
|
}
|
|
|
|
if !jsonOutput {
|
|
fmt.Printf("Compacting %d issues (Tier %d)...\n\n", len(candidates), compactTier)
|
|
}
|
|
|
|
results, err := compactor.CompactTier1Batch(ctx, candidates)
|
|
if err != nil {
|
|
return HandleError("batch compaction failed: %v", err)
|
|
}
|
|
|
|
successCount := 0
|
|
failCount := 0
|
|
totalSaved := 0
|
|
totalOriginal := 0
|
|
|
|
for i, result := range results {
|
|
if !jsonOutput {
|
|
fmt.Printf("[%s] %d/%d\r", progressBar(i+1, len(results)), i+1, len(results))
|
|
}
|
|
|
|
if result.Err != nil {
|
|
failCount++
|
|
} else {
|
|
successCount++
|
|
totalOriginal += result.OriginalSize
|
|
totalSaved += (result.OriginalSize - result.CompactedSize)
|
|
}
|
|
}
|
|
|
|
elapsed := time.Since(start)
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"success": true,
|
|
"tier": compactTier,
|
|
"total": len(results),
|
|
"succeeded": successCount,
|
|
"failed": failCount,
|
|
"saved_bytes": totalSaved,
|
|
"original_size": totalOriginal,
|
|
"elapsed_ms": elapsed.Milliseconds(),
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("\n\nCompleted in %v\n\n", elapsed)
|
|
fmt.Printf("Summary:\n")
|
|
fmt.Printf(" Succeeded: %d\n", successCount)
|
|
fmt.Printf(" Failed: %d\n", failCount)
|
|
if totalOriginal > 0 {
|
|
fmt.Printf(" Saved: %d bytes (%.1f%%)\n", totalSaved, float64(totalSaved)/float64(totalOriginal)*100)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func runCompactStats(ctx context.Context, store storage.DoltStorage) error {
|
|
tier1, err := store.GetTier1Candidates(ctx)
|
|
if err != nil {
|
|
return HandleError("failed to get Tier 1 candidates: %v", err)
|
|
}
|
|
|
|
tier2, err := store.GetTier2Candidates(ctx)
|
|
if err != nil {
|
|
return HandleError("failed to get Tier 2 candidates: %v", err)
|
|
}
|
|
|
|
tier1Size := 0
|
|
for _, c := range tier1 {
|
|
tier1Size += c.OriginalSize
|
|
}
|
|
|
|
tier2Size := 0
|
|
for _, c := range tier2 {
|
|
tier2Size += c.OriginalSize
|
|
}
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"tier1": map[string]interface{}{
|
|
"candidates": len(tier1),
|
|
"total_size": tier1Size,
|
|
},
|
|
"tier2": map[string]interface{}{
|
|
"candidates": len(tier2),
|
|
"total_size": tier2Size,
|
|
"implemented": false,
|
|
},
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Println("Compaction Statistics")
|
|
fmt.Printf("Tier 1 (30+ days closed):\n")
|
|
fmt.Printf(" Candidates: %d\n", len(tier1))
|
|
fmt.Printf(" Total size: %d bytes\n", tier1Size)
|
|
if tier1Size > 0 {
|
|
fmt.Printf(" Estimated savings: %d bytes (70%%)\n\n", tier1Size*7/10)
|
|
}
|
|
|
|
fmt.Printf("Tier 2 (90+ days closed, Tier 1 compacted): not yet implemented\n")
|
|
fmt.Printf(" Candidates: %d\n", len(tier2))
|
|
fmt.Printf(" Total size: %d bytes\n", tier2Size)
|
|
return nil
|
|
}
|
|
|
|
func runCompactAnalyze(ctx context.Context, store storage.DoltStorage) error {
|
|
type Candidate struct {
|
|
ID string `json:"id"`
|
|
Title string `json:"title"`
|
|
Description string `json:"description"`
|
|
Design string `json:"design"`
|
|
Notes string `json:"notes"`
|
|
AcceptanceCriteria string `json:"acceptance_criteria"`
|
|
SizeBytes int `json:"size_bytes"`
|
|
AgeDays int `json:"age_days"`
|
|
Tier int `json:"tier"`
|
|
Compacted bool `json:"compacted"`
|
|
}
|
|
|
|
var candidates []Candidate
|
|
|
|
// Single issue mode
|
|
if compactID != "" {
|
|
issue, err := store.GetIssue(ctx, compactID)
|
|
if err != nil {
|
|
return HandleError("failed to get issue: %v", err)
|
|
}
|
|
|
|
sizeBytes := len(issue.Description) + len(issue.Design) + len(issue.Notes) + len(issue.AcceptanceCriteria)
|
|
ageDays := 0
|
|
if issue.ClosedAt != nil {
|
|
ageDays = int(time.Since(*issue.ClosedAt).Hours() / 24)
|
|
}
|
|
|
|
candidates = append(candidates, Candidate{
|
|
ID: issue.ID,
|
|
Title: issue.Title,
|
|
Description: issue.Description,
|
|
Design: issue.Design,
|
|
Notes: issue.Notes,
|
|
AcceptanceCriteria: issue.AcceptanceCriteria,
|
|
SizeBytes: sizeBytes,
|
|
AgeDays: ageDays,
|
|
Tier: compactTier,
|
|
Compacted: issue.CompactionLevel > 0,
|
|
})
|
|
} else {
|
|
// Get tier candidates
|
|
var tierCandidates []*types.CompactionCandidate
|
|
var err error
|
|
if compactTier == 1 {
|
|
tierCandidates, err = store.GetTier1Candidates(ctx)
|
|
} else {
|
|
tierCandidates, err = store.GetTier2Candidates(ctx)
|
|
}
|
|
if err != nil {
|
|
return HandleError("failed to get candidates: %v", err)
|
|
}
|
|
|
|
// Apply limit if specified
|
|
if compactLimit > 0 && len(tierCandidates) > compactLimit {
|
|
tierCandidates = tierCandidates[:compactLimit]
|
|
}
|
|
|
|
// Fetch full details for each candidate
|
|
for _, c := range tierCandidates {
|
|
issue, err := store.GetIssue(ctx, c.IssueID)
|
|
if err != nil {
|
|
continue // Skip issues we can't fetch
|
|
}
|
|
|
|
ageDays := int(time.Since(c.ClosedAt).Hours() / 24)
|
|
|
|
candidates = append(candidates, Candidate{
|
|
ID: issue.ID,
|
|
Title: issue.Title,
|
|
Description: issue.Description,
|
|
Design: issue.Design,
|
|
Notes: issue.Notes,
|
|
AcceptanceCriteria: issue.AcceptanceCriteria,
|
|
SizeBytes: c.OriginalSize,
|
|
AgeDays: ageDays,
|
|
Tier: compactTier,
|
|
Compacted: issue.CompactionLevel > 0,
|
|
})
|
|
}
|
|
}
|
|
|
|
if jsonOutput {
|
|
totalSize := 0
|
|
for _, c := range candidates {
|
|
totalSize += c.SizeBytes
|
|
}
|
|
output := map[string]interface{}{
|
|
"candidates": candidates,
|
|
"summary": map[string]interface{}{
|
|
"total_candidates": len(candidates),
|
|
"total_content_bytes": totalSize,
|
|
},
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// Human-readable output
|
|
fmt.Printf("Compaction Candidates (Tier %d)\n\n", compactTier)
|
|
fmt.Printf(" %-12s %-40s %8s %10s\n", "ID", "TITLE", "AGE", "SIZE")
|
|
totalSize := 0
|
|
for _, c := range candidates {
|
|
compactStatus := ""
|
|
if c.Compacted {
|
|
compactStatus = " *"
|
|
}
|
|
title := c.Title
|
|
if len(title) > 40 {
|
|
title = title[:37] + "..."
|
|
}
|
|
fmt.Printf(" %-12s %-40s %5dd %10d B%s\n", c.ID, title, c.AgeDays, c.SizeBytes, compactStatus)
|
|
totalSize += c.SizeBytes
|
|
}
|
|
fmt.Printf("\nSummary: %d candidates, %d bytes total content\n", len(candidates), totalSize)
|
|
return nil
|
|
}
|
|
|
|
func runCompactApply(ctx context.Context, store storage.DoltStorage) error {
|
|
start := time.Now()
|
|
|
|
// Read summary
|
|
var summaryBytes []byte
|
|
var err error
|
|
if compactSummary == "-" {
|
|
// Read from stdin
|
|
summaryBytes, err = io.ReadAll(os.Stdin)
|
|
if err != nil {
|
|
return HandleError("failed to read summary from stdin: %v", err)
|
|
}
|
|
} else {
|
|
// #nosec G304 -- summary file path provided explicitly by operator
|
|
summaryBytes, err = os.ReadFile(compactSummary)
|
|
if err != nil {
|
|
return HandleError("failed to read summary file: %v", err)
|
|
}
|
|
}
|
|
summary := string(summaryBytes)
|
|
|
|
// Get issue
|
|
issue, err := store.GetIssue(ctx, compactID)
|
|
if err != nil {
|
|
return HandleError("failed to get issue: %v", err)
|
|
}
|
|
|
|
// Calculate sizes
|
|
originalSize := len(issue.Description) + len(issue.Design) + len(issue.Notes) + len(issue.AcceptanceCriteria)
|
|
compactedSize := len(summary)
|
|
|
|
// Check eligibility unless --force
|
|
if !compactForce {
|
|
eligible, reason, err := store.CheckEligibility(ctx, compactID, compactTier)
|
|
if err != nil {
|
|
return HandleError("failed to check eligibility: %v", err)
|
|
}
|
|
if !eligible {
|
|
return HandleErrorWithHint(fmt.Sprintf("%s is not eligible for Tier %d compaction: %s", compactID, compactTier, reason), "use --force to bypass eligibility checks")
|
|
}
|
|
|
|
// Enforce size reduction unless --force
|
|
if compactedSize >= originalSize {
|
|
return HandleErrorWithHint(fmt.Sprintf("summary (%d bytes) is not shorter than original (%d bytes)", compactedSize, originalSize), "use --force to bypass size validation")
|
|
}
|
|
}
|
|
|
|
// Apply compaction
|
|
actor := compactActor
|
|
if actor == "" {
|
|
actor = "agent"
|
|
}
|
|
|
|
updates := map[string]interface{}{
|
|
"description": summary,
|
|
"design": "",
|
|
"notes": "",
|
|
"acceptance_criteria": "",
|
|
}
|
|
|
|
if err := store.UpdateIssue(ctx, compactID, updates, actor); err != nil {
|
|
return HandleError("failed to update issue: %v", err)
|
|
}
|
|
|
|
commitHash := compact.GetCurrentCommitHash()
|
|
if err := store.ApplyCompaction(ctx, compactID, compactTier, originalSize, compactedSize, commitHash); err != nil {
|
|
return HandleError("failed to apply compaction: %v", err)
|
|
}
|
|
|
|
savingBytes := originalSize - compactedSize
|
|
reductionPct := float64(savingBytes) / float64(originalSize) * 100
|
|
eventData := fmt.Sprintf("Tier %d compaction: %d → %d bytes (saved %d, %.1f%%)", compactTier, originalSize, compactedSize, savingBytes, reductionPct)
|
|
if err := store.AddComment(ctx, compactID, actor, eventData); err != nil {
|
|
return HandleError("failed to record event: %v", err)
|
|
}
|
|
|
|
elapsed := time.Since(start)
|
|
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"success": true,
|
|
"issue_id": compactID,
|
|
"tier": compactTier,
|
|
"original_size": originalSize,
|
|
"compacted_size": compactedSize,
|
|
"saved_bytes": savingBytes,
|
|
"reduction_pct": reductionPct,
|
|
"elapsed_ms": elapsed.Milliseconds(),
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("✓ Compacted %s (Tier %d)\n", compactID, compactTier)
|
|
fmt.Printf(" %d → %d bytes (saved %d, %.1f%%)\n", originalSize, compactedSize, savingBytes, reductionPct)
|
|
fmt.Printf(" Time: %v\n", elapsed)
|
|
return nil
|
|
}
|
|
|
|
// runCompactDolt runs external Dolt GC in the locally owned active database.
|
|
func runCompactDolt(ctx context.Context) error {
|
|
start := time.Now()
|
|
|
|
// Find beads directory
|
|
beadsDir := beads.FindBeadsDir()
|
|
if beadsDir == "" {
|
|
return HandleErrorWithHint(activeWorkspaceNotFoundError(), diagHint())
|
|
}
|
|
|
|
// Resolve operation authority from the opened store, never a stale project
|
|
// directory, general CLI path, or advisory size measurement.
|
|
var doltPath string
|
|
var pathErr error
|
|
if locator, ok := storage.UnwrapStore(store).(storage.ExternalGCLocator); ok {
|
|
doltPath, pathErr = locator.ExternalGCPath(ctx)
|
|
} else {
|
|
pathErr = &storage.ErrUnsupported{Op: "ExternalGCPath", Backend: "active store"}
|
|
}
|
|
if pathErr != nil {
|
|
// Classify cancellation before formatting. ExternalGCPath probes ctx, so an
|
|
// interrupt or deadline surfaces here as a bare context error, and it is not
|
|
// an authority refusal: without this arm a SIGINT renders as the permanent
|
|
// "cannot select a local database" message plus the four-cause hint about
|
|
// port overrides and proxied servers, none of which applies. The ctx probe
|
|
// stays in ExternalGCPath deliberately — it is the only cancellation
|
|
// checkpoint before the uninterruptible external `dolt gc` below.
|
|
if errors.Is(pathErr, context.Canceled) || errors.Is(pathErr, context.DeadlineExceeded) {
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("external Dolt garbage collection was interrupted: %v", pathErr),
|
|
"the command stopped before the active database was resolved, so no garbage collection ran. "+
|
|
"Re-run 'bd admin compact --dolt' to collect.")
|
|
}
|
|
var unsupported *storage.ErrUnsupported
|
|
if compactDryRun && errors.As(pathErr, &unsupported) {
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"dry_run": true,
|
|
"available": false,
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
fmt.Printf("DRY RUN - Dolt garbage collection\n\n")
|
|
fmt.Printf("The active database is not available for local external garbage collection.\n")
|
|
return nil
|
|
}
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("cannot select a local database for external Dolt garbage collection: %v", pathErr),
|
|
"--dolt requires a locally managed owned/shared Dolt database. Port overrides (BEADS_DOLT_SERVER_PORT/BEADS_DOLT_PORT), "+
|
|
"external/gateway/proxied servers, and socket/TLS connections do not authorize local GC; owned mode also requires "+
|
|
"auto-start, so a hand-started owned server needs 'bd config set dolt.auto-start true'. "+
|
|
"Run 'bd doctor' to inspect the connection or ask the server administrator to run GC.")
|
|
}
|
|
if !filepath.IsAbs(doltPath) {
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("external Dolt garbage collection requires an absolute active database directory, got %q", doltPath),
|
|
"run 'bd doctor' and correct the active database configuration to use an absolute directory")
|
|
}
|
|
info, err := os.Stat(doltPath)
|
|
if err != nil {
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("active Dolt database directory is unavailable: %v", err),
|
|
"check that the active database directory exists and is accessible; run 'bd doctor' to inspect the database configuration")
|
|
}
|
|
if !info.IsDir() {
|
|
return HandleErrorWithHint(
|
|
fmt.Sprintf("active Dolt database path %q is not a directory", doltPath),
|
|
"run 'bd doctor' and correct the active database configuration to point to a directory, not a file")
|
|
}
|
|
|
|
// Measure only the active database. The shared .beads/dolt root may contain
|
|
// sibling databases unrelated to this GC operation.
|
|
sizeBefore := storeSizeBytes(ctx)
|
|
|
|
if compactDryRun {
|
|
if jsonOutput {
|
|
output := map[string]interface{}{
|
|
"dry_run": true,
|
|
"dolt_path": doltPath,
|
|
}
|
|
if sizeBefore >= 0 {
|
|
output["size_before"] = sizeBefore
|
|
output["size_display"] = formatBytes(sizeBefore)
|
|
}
|
|
if err := outputJSON(output); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
fmt.Printf("DRY RUN - Dolt garbage collection\n\n")
|
|
fmt.Printf("Dolt directory: %s\n", doltPath)
|
|
if sizeBefore >= 0 {
|
|
fmt.Printf("Current size: %s\n", formatBytes(sizeBefore))
|
|
}
|
|
fmt.Printf("\nRun without --dry-run to perform garbage collection.\n")
|
|
return nil
|
|
}
|
|
|
|
// Check if dolt command is available
|
|
if _, err := exec.LookPath("dolt"); err != nil {
|
|
return HandleErrorWithHint("dolt command not found in PATH", "install Dolt from https://github.com/dolthub/dolt")
|
|
}
|
|
|
|
if !jsonOutput {
|
|
fmt.Printf("Running Dolt garbage collection...\n")
|
|
}
|
|
|
|
// Run dolt gc without archive compression. Level 0 writes classic Snappy
|
|
// table files instead of zstd archives, matching the in-process GC paths.
|
|
// The external `dolt` on PATH has no version guarantee (unlike the
|
|
// in-process paths, which are pinned by go.mod), so an older dolt that
|
|
// predates --archive-level would otherwise abort compact where plain
|
|
// `dolt gc` used to work. Detect that specific unknown-flag rejection
|
|
// and retry with plain `dolt gc` rather than fail outright; any other
|
|
// error (a genuine GC failure) is not swallowed.
|
|
cmd := exec.Command("dolt", "gc", "--archive-level", "0") // #nosec G204 -- fixed command, no user input
|
|
cmd.Dir = doltPath
|
|
output, err := cmd.CombinedOutput()
|
|
if err != nil && isUnknownArchiveLevelFlagError(string(output)) {
|
|
if !jsonOutput {
|
|
fmt.Fprintf(os.Stderr, "Notice: external dolt does not support --archive-level; falling back to plain 'dolt gc' (new table files may still use zstd archives)\n")
|
|
}
|
|
fallbackCmd := exec.Command("dolt", "gc") // #nosec G204 -- fixed command, no user input
|
|
fallbackCmd.Dir = doltPath
|
|
output, err = fallbackCmd.CombinedOutput()
|
|
}
|
|
if err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: dolt gc failed: %v\n", err)
|
|
if len(output) > 0 {
|
|
fmt.Fprintf(os.Stderr, "Output: %s\n", string(output))
|
|
}
|
|
return SilentExit()
|
|
}
|
|
|
|
// Measure the same active-database scope after GC.
|
|
sizeAfter := storeSizeBytes(ctx)
|
|
|
|
elapsed := time.Since(start)
|
|
|
|
if jsonOutput {
|
|
// Preserve compact's legacy size_before/size_after field names;
|
|
// addGCSizeJSON uses a different schema and is not interchangeable.
|
|
result := map[string]interface{}{
|
|
"success": true,
|
|
"dolt_path": doltPath,
|
|
"elapsed_ms": elapsed.Milliseconds(),
|
|
}
|
|
if sizeBefore >= 0 {
|
|
result["size_before"] = sizeBefore
|
|
}
|
|
if sizeAfter >= 0 {
|
|
result["size_after"] = sizeAfter
|
|
}
|
|
if sizeBefore >= 0 && sizeAfter >= 0 {
|
|
freed := sizeBefore - sizeAfter
|
|
if freed < 0 {
|
|
freed = 0 // GC may not always reduce size
|
|
}
|
|
result["freed_bytes"] = freed
|
|
result["freed_display"] = formatBytes(freed)
|
|
}
|
|
if err := outputJSON(result); err != nil {
|
|
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
fmt.Printf("✓ Dolt garbage collection complete\n")
|
|
if line := gcSizeLine(sizeBefore, sizeAfter); line != "" {
|
|
fmt.Printf(" %s\n", line)
|
|
}
|
|
fmt.Printf(" Time: %v\n", elapsed)
|
|
return nil
|
|
}
|
|
|
|
// isUnknownArchiveLevelFlagError reports whether output (the combined
|
|
// stdout+stderr of `dolt gc --archive-level 0`) indicates the external dolt
|
|
// binary rejected --archive-level as an unrecognized flag, rather than a
|
|
// genuine GC failure. Older Dolt releases that predate the flag report this
|
|
// via the pinned dolt module's argparser, e.g.:
|
|
//
|
|
// error: unknown option `archive-level'
|
|
//
|
|
// (see libraries/utils/argparser/errors.go in the pinned dolthub/dolt/go
|
|
// module). The check requires both an "unknown flag" phrasing AND the flag
|
|
// name to appear in the output, so a real GC failure that happens to
|
|
// contain the word "unknown" elsewhere is never misclassified as a missing
|
|
// flag and silently swallowed — genuine failures continue to fail.
|
|
func isUnknownArchiveLevelFlagError(output string) bool {
|
|
lower := strings.ToLower(output)
|
|
if !strings.Contains(lower, "archive-level") && !strings.Contains(lower, "archive_level") {
|
|
return false
|
|
}
|
|
return strings.Contains(lower, "unknown option") ||
|
|
strings.Contains(lower, "unknown flag") ||
|
|
strings.Contains(lower, "flag provided but not defined")
|
|
}
|
|
|
|
// formatBytes formats a byte count as a human-readable string
|
|
func formatBytes(b int64) string {
|
|
const unit = 1024
|
|
if b < unit {
|
|
return fmt.Sprintf("%d B", b)
|
|
}
|
|
div, exp := int64(unit), 0
|
|
for n := b / unit; n >= unit; n /= unit {
|
|
div *= unit
|
|
exp++
|
|
}
|
|
return fmt.Sprintf("%.1f %cB", float64(b)/float64(div), "KMGTPE"[exp])
|
|
}
|
|
|
|
// progressBar renders a text-based progress bar.
|
|
func progressBar(current, total int) string {
|
|
const width = 40
|
|
if total == 0 {
|
|
return "[" + string(make([]byte, width)) + "]"
|
|
}
|
|
filled := (current * width) / total
|
|
bar := ""
|
|
for i := 0; i < width; i++ {
|
|
if i < filled {
|
|
bar += "█"
|
|
} else {
|
|
bar += " "
|
|
}
|
|
}
|
|
return "[" + bar + "]"
|
|
}
|
|
|
|
func init() {
|
|
compactCmd.Flags().BoolVar(&compactDryRun, "dry-run", false, "Preview without compacting")
|
|
compactCmd.Flags().IntVar(&compactTier, "tier", 1, "Compaction tier (only tier 1 is implemented)")
|
|
compactCmd.Flags().BoolVar(&compactAll, "all", false, "Process all candidates")
|
|
compactCmd.Flags().StringVar(&compactID, "id", "", "Compact specific issue")
|
|
compactCmd.Flags().BoolVar(&compactForce, "force", false, "Force compact (bypass checks, requires --id)")
|
|
compactCmd.Flags().IntVar(&compactBatch, "batch-size", 10, "Issues per batch")
|
|
compactCmd.Flags().IntVar(&compactWorkers, "workers", 5, "Parallel workers")
|
|
compactCmd.Flags().BoolVar(&compactStats, "stats", false, "Show compaction statistics")
|
|
|
|
// New mode flags
|
|
compactCmd.Flags().BoolVar(&compactAnalyze, "analyze", false, "Analyze mode: export candidates for agent review")
|
|
compactCmd.Flags().BoolVar(&compactApply, "apply", false, "Apply mode: accept agent-provided summary")
|
|
compactCmd.Flags().BoolVar(&compactAuto, "auto", false, "Auto mode: AI-powered compaction")
|
|
compactCmd.Flags().StringVar(&compactSummary, "summary", "", "Path to summary file (use '-' for stdin)")
|
|
compactCmd.Flags().StringVar(&compactActor, "actor", "agent", "Actor name for audit trail")
|
|
compactCmd.Flags().IntVar(&compactLimit, "limit", 0, "Limit number of candidates (0 = no limit)")
|
|
compactCmd.Flags().BoolVar(&compactDolt, "dolt", false, "Dolt mode: run Dolt garbage collection on the active database")
|
|
|
|
// Note: compactCmd is added to adminCmd in admin.go
|
|
}
|