From 8bb0d36be08e16fa7324cb079520a2b168741ec0 Mon Sep 17 00:00:00 2001 From: Steve Yegge Date: Thu, 30 Jul 2026 18:05:31 -0700 Subject: [PATCH] docs(recovery): fold squash-window lessons into the history-squash runbook (bd-gpgdy) (#5176) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three operational lessons from the July squash windows, in generic terms: - Git-backed remotes never shrink on force-push: the remote manifest keeps every historical table file listed as current, so the store must be replaced in place — delete refs/dolt/data and the __dolt_remote_info__ branch on the git remote, then force-push a fresh store (verified on two production remotes: 587M->57.8M and 274M->2.6M reachable). - Sync jobs and writers are separate switches: a peer's sync job stays off until that machine has verifiably re-cloned; unfencing writers must not implicitly re-arm peer sync (the re-pollution hole hit twice). - First push from a re-cloned machine is far larger than a routine tick; timeout-capped sync jobs orphan partial uploads — one manual uncapped push before handing back to the schedule. Gates: go test ./test/docsync PASS, check-doc-freshness.sh PASS. Co-authored-by: Claude Fable 5 --- docs/recovery/history-squash.md | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/docs/recovery/history-squash.md b/docs/recovery/history-squash.md index 77b9b01f3..ff37ed241 100644 --- a/docs/recovery/history-squash.md +++ b/docs/recovery/history-squash.md @@ -112,6 +112,19 @@ then `bd dolt push --force`. Every other clone must re-clone after a squash regardless, so replacing the remote costs nothing extra. +On a *git-backed* remote (issue data riding your code remote under +`refs/dolt/data`) there is no fresh path to pick — the storage is the git +repository itself, and its manifest lists every historical table file as +current, so the full old store stays reachable even after the force-push. +Replace the data plane in place instead: delete the Dolt data refs on the +git remote, then force-push to rebuild a fresh store holding only live +chunks. Code branches are untouched. + +```bash +git push origin :refs/dolt/data :refs/heads/__dolt_remote_info__ +bd dolt push --force +``` + **Step 5:** Verify, then re-clone everywhere else. On this machine: ```bash @@ -122,9 +135,19 @@ bd list -n 5 Every other clone of this database must be re-created from the squashed remote. Old clones must not pull: the two chains still share the root, so a pull can "succeed" as a cross-merge that re-anchors the entire old history — -and a later push resurrects the bloat on the squashed remote. Re-enable each -machine's sync jobs only after that machine has re-cloned; unfence your -writers last. +and a later push resurrects the bloat on the squashed remote. + +Treat sync jobs and writers as separate switches. Re-enable each machine's +sync job only after *that machine* has re-cloned — verified, not assumed — +and unfence writers last; unfencing writers must never implicitly restart a +peer's sync job. One peer syncing from its old clone re-uploads the entire +pre-squash store to the fresh remote. + +The first push from a re-cloned machine can be far larger than a routine +sync. If that machine's scheduled sync job enforces a timeout, the capped +push can die mid-upload on every tick, orphaning partial uploads on the +remote — run one manual `bd dolt push` (no timeout) right after re-cloning +and hand back to the schedule once it completes. ## Prevention