bd flatten and bd compact squash history then run DOLT_GC, but never
deleted remote-tracking refs first — and any workspace that has ever
pushed or fetched has them. GC treats those refs as reachable roots, so
on exactly the workspaces these commands target (big, long-lived,
synced) the GC reclaimed ~nothing and the operator got no cue. Observed
in prod: the wyvern store stayed ~10G through a compression attempt
(2026-07-17).
- versioncontrolops: new ListRemoteRefs / PruneRemoteRefs / ListTags
(dolt_remote_branches + DOLT_BRANCH('-D','-r'); dolt_tags list-only).
- storage.RemoteRefPruner optional interface, implemented on DoltStore
and EmbeddedDoltStore.
- bd flatten / bd compact: prune between squash and GC, report store
size before/after so a no-op reclaim is visible, warn on tags (which
also anchor history but are user-created and never deleted), and
surface refs/tags in --dry-run and JSON output.
- bd gc audit: it never squashes (its Compact phase is informational),
so it keeps its remote refs — they double as the smart migrate gate's
cached remote tip (AS OF 'remotes/origin/main', #4516) — and instead
reports size before/after plus a note when refs/tags anchor history.
- Regression test (real Dolt): the old chain stays reachable through
remotes/origin/main after Flatten, prune removes the anchor, tags
survive, GC runs clean, prune is idempotent.
The next push or fetch re-creates pruned refs at the new tip; nothing
is deleted on the remote itself.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
83 lines
3.1 KiB
Go
83 lines
3.1 KiB
Go
package versioncontrolops
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
)
|
|
|
|
// Compact squashes old Dolt commits into a single base commit while preserving
|
|
// recent commits via cherry-pick. The recipe:
|
|
// 1. Create temp branch at the boundary commit (last old commit)
|
|
// 2. Checkout temp branch
|
|
// 3. Soft-reset to initial commit (collapses old history into working set)
|
|
// 4. Commit as single squashed base
|
|
// 5. Cherry-pick each recent commit on top
|
|
// 6. Checkout main, hard-reset to temp branch
|
|
// 7. Delete temp branch
|
|
//
|
|
// Callers should run PruneRemoteRefs and then DoltGC afterward to reclaim disk
|
|
// space — remote-tracking refs still anchor the pre-compact chain, and GC
|
|
// alone reclaims nothing while they exist (bd-agctw).
|
|
//
|
|
// conn must be a single database connection (not a pooled *sql.DB) since the
|
|
// stored procedures rely on session-scoped state (current branch, working set).
|
|
func Compact(ctx context.Context, conn DBConn, initialHash, boundaryHash string, oldCommits int, recentHashes []string) (retErr error) {
|
|
branchCreated := false
|
|
|
|
// Best-effort cleanup: if any step fails after creating the temp branch,
|
|
// try to return to main and delete the temp branch so future compactions
|
|
// aren't blocked by a leftover branch.
|
|
defer func() {
|
|
if retErr != nil && branchCreated {
|
|
_, _ = conn.ExecContext(ctx, "CALL DOLT_CHECKOUT('main')")
|
|
_, _ = conn.ExecContext(ctx, "CALL DOLT_BRANCH('-D', 'compact-tmp')")
|
|
}
|
|
}()
|
|
|
|
execSQL := func(name, query string, args ...interface{}) error {
|
|
if _, err := conn.ExecContext(ctx, query, args...); err != nil {
|
|
return fmt.Errorf("compact step %q: %w", name, err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
if err := execSQL("create temp branch", "CALL DOLT_BRANCH('compact-tmp', ?)", boundaryHash); err != nil {
|
|
return err
|
|
}
|
|
branchCreated = true
|
|
|
|
if err := execSQL("checkout temp", "CALL DOLT_CHECKOUT('compact-tmp')"); err != nil {
|
|
return err
|
|
}
|
|
if err := execSQL("soft reset to initial", "CALL DOLT_RESET('--soft', ?)", initialHash); err != nil {
|
|
return err
|
|
}
|
|
msg := fmt.Sprintf("compact: squash %d commits into base snapshot", oldCommits)
|
|
if err := execSQL("commit squashed base", "CALL DOLT_COMMIT('-Am', ?)", msg); err != nil {
|
|
return err
|
|
}
|
|
|
|
// --allow-empty: the preserved window can contain empty commits (a Dolt
|
|
// auto-commit with no table change, or a bd create double-commit whose
|
|
// leading member has 0 changed tables). Without this flag DOLT_CHERRY_PICK
|
|
// aborts the entire replay at the first empty commit with Error 1105
|
|
// ("The previous cherry-pick commit is empty. Use --allow-empty ..."),
|
|
// leaving compaction permanently blocked on active databases. See #3815.
|
|
for _, hash := range recentHashes {
|
|
if err := execSQL(fmt.Sprintf("cherry-pick %s", hash[:min(8, len(hash))]), "CALL DOLT_CHERRY_PICK('--allow-empty', ?)", hash); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
|
|
if err := execSQL("checkout main", "CALL DOLT_CHECKOUT('main')"); err != nil {
|
|
return err
|
|
}
|
|
if err := execSQL("reset main to compacted", "CALL DOLT_RESET('--hard', 'compact-tmp')"); err != nil {
|
|
return err
|
|
}
|
|
if err := execSQL("delete temp branch", "CALL DOLT_BRANCH('-D', 'compact-tmp')"); err != nil {
|
|
return err
|
|
}
|
|
|
|
return nil
|
|
}
|