fix(export): heal the delete wedge — prove deletions via store history instead of refusing forever (#5896) (#6059)

* feat(storage): add HistoryPresence capability backed by dolt_history_issues

Auto-export's orphan guard can only prove a deletion when it has a valid
diff anchor AND the store implements storage.DiffStore. Embedded mode — the
default open path — implements neither, so every `bd delete` there leaves a
JSONL-only id the guard can never prove and wedges auto-export forever
(#5896).

HistoryPresence answers the anchor-free question instead: does this id appear
anywhere in HEAD's committed history of the issues table? dolt_history_issues
walks HEAD's ancestry, so a rewound-away or never-committed id stays absent —
that asymmetry is what makes "in history, gone now" safe to read as a real
deletion rather than #4988's torn store.

One shared SQL helper (issueops.HistoricalIssueIDsInTx), two stores, and a
cross-store contract test that pins both halves, including the DOLT_RESET
rewind case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(export): heal the delete wedge instead of refusing forever

The orphan guard refused, so the export skipped, so state was never saved,
so the anchor never refreshed — and the same refusal fired on every command
after that, forever. The documented recovery (`bd init --from-jsonl`)
resurrects the deleted issue, i.e. it undoes what the user asked for.

Both pre-export guards now read ONE reconciliation of the JSONL against the
store, and a JSONL-only id is refused only if no proof source can account for
it. Three sources, cheapest first: the in-process delete handoff from
deleteBatch, the existing #5806 anchor diff, and the store's own committed
history (capped at 100 candidates — a wider divergence is #4988 territory,
not a delete pattern). Anything still unproven refuses with the unchanged
message.

Three things that are each a bug on their own if missed:

- proven ids are threaded into tryIncrementalExport as forced removals. A
  deletion the diff cannot see is a deletion the patch would otherwise carry
  forward forever, silently.
- the empty-overwrite guard subtracts them too. Deleting the LAST exported
  issue trips that guard first, so the literal one-issue repro wedged there
  and never reached the orphan guard at all.
- the heal prints a stderr notice naming what it reconciled.

Phase 3 rides along: export-state.json now carries a format version, and an
unknown future version is discarded to empty state. That was unsafe to add
before this change — discarding drops the anchor, and an anchorless state
plus a pending delete WAS the permanent wedge.

Fixes #5896

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(export): retarget comments and a test name at reconcileAutoExportJSONL

Follow-up to the guard restructure: the function the comments cite no longer
exists under that name.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Julian Knutsen 2026-08-29 21:24:29 -07:00 • committed by GitHub
commit 34a2f239d8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 1560 additions and 103 deletions

View file

@ -364,6 +364,14 @@ func deleteBatch(_ *cobra.Command, issueIDs []string, force bool, dryRun bool, c
commandDidWrite.Store(true)
// Tell this command's own auto-export that these ids are supposed to be
// gone. Without it the export's orphan guard sees them as JSONL-only
// records absent from the store, refuses to overwrite, and never saves
// state — which is the permanent wedge of GH#5896. Recorded for the
// requested ids; anything cascade removed alongside them is proven later
// by the store's own history instead.
commandDeletedIssueIDs.add(resolvedIDs...)
// NO COMMIT COMPENSATION HERE, for the reason the single-id path gives:
// the role versions the deletion itself, on the store the rows were
// deleted from, and defers it in batch and off modes.

View file

@ -11,6 +11,7 @@ import (
"path/filepath"
"sort"
"strings"
"sync"
"time"
"github.com/steveyegge/beads/internal/atomicfile"
@ -39,8 +40,27 @@ const doltWorkingRef = "WORKING"
// exceeds this prints a one-line stderr tip pointing at the fix levers.
const slowExportWarnThreshold = 3 * time.Second
// exportAutoStateVersion is the format version stamped into
// .beads/export-state.json. BUMP IT whenever the MEANING of a persisted
// field changes — LastDiffAnchor's semantics above all — so a newer binary's
// state file is discarded by an older one instead of being reinterpreted.
// Adding a purely additive optional field does not need a bump.
//
// Discarding is only safe because of the deletion proofs in
// proveDeletedIssueIDs: dropping the state drops the diff anchor, and before
// GH#5896 an absent anchor plus a pending delete was itself a permanent
// wedge, which made "invalidate the state file" a cure worse than the
// disease. Now an anchorless cycle just proves what it can from history and
// re-anchors on the way out.
const exportAutoStateVersion = 1
// exportAutoState tracks auto-export state to avoid redundant work.
type exportAutoState struct {
// FormatVersion is exportAutoStateVersion at write time. Absent (0) means
// a file written before versioning existed; those are field-for-field
// identical to v1 and load as-is.
FormatVersion int `json:"v,omitempty"`
LastDoltCommit string `json:"last_dolt_commit"`
Timestamp time.Time `json:"timestamp"`
Issues int `json:"issues"`
@ -128,21 +148,31 @@ func maybeAutoExport(ctx context.Context, allowEmptyOverwrite bool) error {
return nil
}
// Reconcile the existing JSONL against the store ONCE, and let BOTH
// fail-safe guards below read the same answer. A deletion this store can
// prove is not divergence for either of them: the empty-overwrite guard
// fires first, so without a shared verdict `bd delete <last issue>`
// wedges there before the orphan guard ever gets to prove anything
// (GH#5896).
var provenDeleted []string
if !allowEmptyOverwrite {
if skip, existingCount, err := shouldSkipEmptyAutoExport(ctx, fullPath); err != nil {
rec, err := reconcileAutoExportJSONL(ctx, fullPath, state.LastDiffAnchor)
if err != nil {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: failed to compare existing JSONL against local store: %v\n", err)
return nil
}
provenDeleted = rec.provenDeleted
if skip, existingCount, err := shouldSkipEmptyAutoExport(ctx, rec); err != nil {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: failed to check existing JSONL: %v\n", err)
return nil
} else if skip {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: current database would export 0 issues, but %s already contains %d issue(s); refusing to overwrite. Run `bd init --from-jsonl` to import the JSONL file, or move it aside and retry.\n", fullPath, existingCount)
return nil
}
}
if !allowEmptyOverwrite {
if missingIDs, err := missingJSONLIssueIDsInStore(ctx, fullPath, state.LastDiffAnchor); err != nil {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: failed to compare existing JSONL against local store: %v\n", err)
return nil
} else if len(missingIDs) > 0 {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: %s contains %d JSONL-only issue record(s) absent from the local Dolt store (%s); refusing to overwrite. Run `bd init --from-jsonl` to import the JSONL file, or move it aside and retry.\n", fullPath, len(missingIDs), strings.Join(sampleStrings(missingIDs, 5), ", "))
if len(rec.unproven) > 0 {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: %s contains %d JSONL-only issue record(s) absent from the local Dolt store (%s); refusing to overwrite. Run `bd init --from-jsonl` to import the JSONL file, or move it aside and retry.\n", fullPath, len(rec.unproven), strings.Join(sampleStrings(rec.unproven, 5), ", "))
return nil
}
}
@ -167,8 +197,14 @@ func maybeAutoExport(ctx context.Context, allowEmptyOverwrite bool) error {
debug.Logf("auto-export: failed to resolve diff anchor (%v); preserving previous anchor\n", anchorErr)
anchorCommit = state.LastDiffAnchor
}
// provenDeleted rides along as FORCED removals: an id proved by the
// in-process handoff or by store history is invisible to
// dolt_diff(anchor, WORKING), so an incremental patch would faithfully
// preserve its stale JSONL line forever even though the guard just
// cleared the export to run. A full export needs nothing — it rewrites
// from the store.
issueCount, memoryCount, dirtyIDs, didIncremental, err := tryIncrementalExport(
ctx, fullPath, state.LastDiffAnchor, doltWorkingRef, state.LastDirtyIDs,
ctx, fullPath, state.LastDiffAnchor, doltWorkingRef, state.LastDirtyIDs, provenDeleted,
)
if err != nil {
debug.Logf("auto-export: incremental failed (%v); falling back to full\n", err)
@ -182,6 +218,13 @@ func maybeAutoExport(ctx context.Context, allowEmptyOverwrite bool) error {
dirtyIDs = nil
}
// Never heal silently. The user asked for a delete and is getting a line
// removed from a git-tracked file; say so, and name what went.
if len(provenDeleted) > 0 {
fmt.Fprintf(os.Stderr, "auto-export: reconciled %d deleted issue(s) out of %s (deleted from store: %s)\n",
len(provenDeleted), fullPath, strings.Join(sampleStrings(provenDeleted, 5), ", "))
}
if dur := time.Since(exportStart); dur > slowExportWarnThreshold && !didIncremental {
// Only warn on full exports — the whole point of incremental is to
// get us under the threshold, so a slow incremental is noise.
@ -265,12 +308,15 @@ func shouldExport(state *exportAutoState, interval time.Duration) bool {
return time.Since(state.Timestamp) >= interval
}
func shouldSkipEmptyAutoExport(ctx context.Context, path string) (bool, int, error) {
existingCount, err := countIssueRecordsInJSONL(path)
if err != nil {
return false, 0, err
}
if existingCount == 0 {
// shouldSkipEmptyAutoExport is the "store would write 0 issues over a
// populated file" fail-safe. It reads the shared reconciliation rather than
// re-parsing the JSONL so that ids already proven deleted don't count as
// records worth protecting — deleting the last exported issue is the shape
// that used to wedge here, one guard earlier than the orphan guard everyone
// blamed (GH#5896).
func shouldSkipEmptyAutoExport(ctx context.Context, rec *autoExportReconciliation) (bool, int, error) {
existingCount := rec.jsonlIssueCount - len(rec.provenDeleted)
if existingCount <= 0 {
return false, 0, nil
}
@ -282,15 +328,37 @@ func shouldSkipEmptyAutoExport(ctx context.Context, path string) (bool, int, err
return len(issues) == 0, existingCount, nil
}
func countIssueRecordsInJSONL(path string) (int, error) {
ids, err := issueIDsInJSONL(path)
if err != nil {
return 0, err
}
return len(ids), nil
// historyProofMaxCandidates caps how many JSONL-only ids the store-history
// proof will ask about in one export. A divergence wider than this is not a
// delete pattern — it is GH#4988 territory (a torn, stale, or foreign JSONL),
// where the conservative refusal is the right answer and a 100-way history
// scan is wasted work on the way to it.
const historyProofMaxCandidates = 100
// autoExportReconciliation is the single comparison of the existing JSONL
// against the live store that both pre-export guards read.
type autoExportReconciliation struct {
// jsonlIssueCount is every issue record in the file, in auto-export
// scope or not — the number the empty-overwrite guard reports.
jsonlIssueCount int
// provenDeleted are in-scope JSONL ids absent from the store that some
// proof source accounted for as real deletions. The export must drop
// their lines; neither guard may refuse over them.
provenDeleted []string
// unproven are in-scope JSONL ids absent from the store that nothing
// could account for. The orphan guard refuses over these, unchanged.
unproven []string
}
func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) ([]string, error) {
// reconcileAutoExportJSONL compares the on-disk export against the store and
// splits the JSONL-only issue ids into proven deletions and unexplained
// divergence.
//
// fromCommit is the previous cycle's diff anchor, used by one of the three
// proof sources; the other two need no anchor, which is what lets a store
// wedged by an earlier binary (no anchor, or an anchor that no longer
// resolves) recover on its first run under this code.
func reconcileAutoExportJSONL(ctx context.Context, path, fromCommit string) (*autoExportReconciliation, error) {
// GH#4988: only refuse when *in-scope* JSONL issue records are absent from
// the store. Ephemeral wisps, templates, and infra types are outside
// auto-export scope (buildAutoExportFilter / GH#3649). Compaction can
@ -300,8 +368,9 @@ func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) (
if err != nil {
return nil, err
}
rec := &autoExportReconciliation{jsonlIssueCount: len(existing)}
if len(existing) == 0 {
return nil, nil
return rec, nil
}
// Store-side query stays unfiltered and MaxRows: 0 (opts out of
@ -321,24 +390,49 @@ func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) (
}
candidates := make([]string, 0)
for _, rec := range existing {
if rec.Ephemeral || rec.IsTemplate || infraTypeSet[string(rec.IssueType)] {
for _, record := range existing {
if record.Ephemeral || record.IsTemplate || infraTypeSet[string(record.IssueType)] {
continue
}
if _, ok := localIDs[rec.ID]; !ok {
candidates = append(candidates, rec.ID)
if _, ok := localIDs[record.ID]; !ok {
candidates = append(candidates, record.ID)
}
}
if len(candidates) == 0 {
return nil, nil
return rec, nil
}
// Before treating a JSONL-only id as an orphan/corruption, ask dolt_diff
// whether the deletion is real and already recorded — a normal `bd
// delete` since the last export's diff anchor, not data loss. Without an
// anchor (cold start / no successful export yet) there is nothing to
// diff against, so every candidate falls through to the conservative
// "missing" verdict below, same as before this check existed.
rec.provenDeleted, rec.unproven = proveDeletedIssueIDs(ctx, candidates, fromCommit)
return rec, nil
}
// proveDeletedIssueIDs splits JSONL-only candidate ids into the ones a proof
// source accounts for as real deletions and the ones nothing can explain.
//
// Sources run cheapest-first, and each one only sees what the previous could
// not prove:
//
// 1. the in-process delete handoff — this very command deleted the id a few
// milliseconds ago. Free, and the only source that can see a create and a
// delete that both sit uncommitted in the working set.
// 2. dolt_diff against the previous export's anchor (GH#5806) — a DoltStore
// fast path that also covers deletions that arrived from a peer.
// 3. the store's own committed history (GH#5896) — anchor-free, works on
// both stores, and the only source that can heal a store some earlier
// binary already wedged.
//
// Whatever survives all three is unexplained divergence, and the caller keeps
// refusing to overwrite: that is GH#4988's fail-safe, unchanged.
func proveDeletedIssueIDs(ctx context.Context, candidates []string, fromCommit string) (proven, unproven []string) {
unproven = candidates
// P1 — same-process delete handoff.
proven, unproven = splitProvenDeletions(proven, unproven, commandDeletedIssueIDs.has)
if len(unproven) == 0 {
return proven, unproven
}
// P2 — dolt_diff against the last export's anchor.
//
// The anchor must still be on the CURRENT HEAD's history for that proof
// to mean anything. dolt_diff(anchor, WORKING) reports a row as "removed"
@ -349,14 +443,11 @@ func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) (
// there would silently drop a live record from issues.jsonl, which is
// exactly #4988's corruption class this guard exists to prevent. So the
// deletion proof is accepted only when the anchor is reachable from HEAD
// (CommitExists queries dolt_log, i.e. HEAD's forward history); otherwise
// we keep refusing, which is the conservative pre-existing behavior.
// (CommitExists queries dolt_log, i.e. HEAD's forward history).
//
// Residual, deliberately not fixed here (tracked by #5896): an anchor
// that has become unresolvable — after a history squash or GC — combined
// with a pending delete is a permanent wedge. The guard refuses, so the
// export skips, so state is never saved, so the anchor never refreshes.
// Recovery is `bd init --from-jsonl` or moving issues.jsonl aside.
// A missing anchor, an off-history anchor, a diff error, or a store with
// no DiffStore at all (embedded mode — the default) all just fall through
// to P3 now, instead of ending the search.
if fromCommit != "" && diffAnchorOnCurrentHistory(ctx, fromCommit) {
if ds, ok := storage.UnwrapStore(store).(storage.DiffStore); ok {
if changed, diffErr := ds.ChangedIssueIDs(ctx, fromCommit, doltWorkingRef); diffErr == nil {
@ -364,25 +455,101 @@ func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) (
for _, id := range changed.Removed {
removed[id] = struct{}{}
}
proven := candidates[:0]
for _, id := range candidates {
if _, ok := removed[id]; !ok {
proven = append(proven, id)
}
}
candidates = proven
proven, unproven = splitProvenDeletions(proven, unproven, func(id string) bool {
_, ok := removed[id]
return ok
})
} else {
debug.Logf("auto-export: anchor diff %s..%s failed (%v); falling through to the history proof\n",
fromCommit, doltWorkingRef, diffErr)
}
// A diff error (e.g. anchor unreachable after history rewrite)
// leaves candidates untouched — we can't prove the deletion, so
// fall back to refusing, exactly as before this check existed.
}
}
if len(unproven) == 0 {
return proven, unproven
}
return candidates, nil
// P3 — the store's own committed history. "Was here, is gone" is a
// deletion this store recorded; "was never here" is data this store never
// had, which stays unproven. See storage.HistoryPresence for why that
// asymmetry is what keeps #4988's guarantee intact.
if len(unproven) > historyProofMaxCandidates {
debug.Logf("auto-export: %d unproven JSONL-only ids exceeds the history-proof cap %d; refusing without asking\n",
len(unproven), historyProofMaxCandidates)
return proven, unproven
}
hp, ok := storage.UnwrapStore(store).(storage.HistoryPresence)
if !ok {
debug.Logf("auto-export: store does not implement HistoryPresence; JSONL-only ids stay unproven\n")
return proven, unproven
}
historical, err := hp.HistoricalIssueIDs(ctx, unproven)
if err != nil {
debug.Logf("auto-export: history proof failed (%v); treating JSONL-only ids as unproven\n", err)
return proven, unproven
}
proven, unproven = splitProvenDeletions(proven, unproven, func(id string) bool {
_, ok := historical[id]
return ok
})
return proven, unproven
}
// splitProvenDeletions moves the ids isProven accepts from unproven onto
// proven, preserving order on both sides.
func splitProvenDeletions(proven, unproven []string, isProven func(string) bool) ([]string, []string) {
remaining := make([]string, 0, len(unproven))
for _, id := range unproven {
if isProven(id) {
proven = append(proven, id)
continue
}
remaining = append(remaining, id)
}
return proven, remaining
}
// deletedIssueIDSet is the in-process handoff from the delete commands to
// auto-export: "this process just hard-deleted these ids, they are supposed to
// be gone". Deliberately NOT persisted — a cross-process ledger would mean the
// delete path read-modify-writes export-state.json underneath a concurrent
// exporter, and a lost entry there silently re-wedges. When the handoff misses
// (throttled export, separate invocation, peer's delete), the history proof
// picks the id up as soon as the deletion is committed.
type deletedIssueIDSet struct {
mu sync.Mutex
ids map[string]struct{}
}
func (s *deletedIssueIDSet) add(ids ...string) {
if len(ids) == 0 {
return
}
s.mu.Lock()
defer s.mu.Unlock()
if s.ids == nil {
s.ids = make(map[string]struct{}, len(ids))
}
for _, id := range ids {
s.ids[id] = struct{}{}
}
}
func (s *deletedIssueIDSet) reset() {
s.mu.Lock()
defer s.mu.Unlock()
s.ids = nil
}
func (s *deletedIssueIDSet) has(id string) bool {
s.mu.Lock()
defer s.mu.Unlock()
_, ok := s.ids[id]
return ok
}
// commitHistoryChecker is the narrow slice of storage.VersionControl that
// missingJSONLIssueIDsInStore needs. It is declared locally and asserted
// reconcileAutoExportJSONL needs. It is declared locally and asserted
// separately from storage.DiffStore so a store implementing one but not the
// other still degrades to the conservative verdict instead of panicking.
type commitHistoryChecker interface {
@ -392,7 +559,7 @@ type commitHistoryChecker interface {
// diffAnchorOnCurrentHistory reports whether anchor is still reachable from
// the current HEAD. It is the ancestry precondition on trusting a dolt_diff
// "removed" verdict as proof of a real `bd delete` rather than a history
// rewind — see missingJSONLIssueIDsInStore. Fails closed: a store that
// rewind — see reconcileAutoExportJSONL. Fails closed: a store that
// cannot answer, or an error while asking, yields false so the caller keeps
// refusing to overwrite.
func diffAnchorOnCurrentHistory(ctx context.Context, anchor string) bool {
@ -495,18 +662,6 @@ func issueRecordsInJSONL(path string) ([]jsonlIssueRecord, error) {
return records, nil
}
func issueIDsInJSONL(path string) ([]string, error) {
records, err := issueRecordsInJSONL(path)
if err != nil {
return nil, err
}
ids := make([]string, len(records))
for i, rec := range records {
ids[i] = rec.ID
}
return ids, nil
}
func sampleStrings(values []string, limit int) []string {
if len(values) <= limit {
return values
@ -693,7 +848,7 @@ func exportToFile(ctx context.Context, path string, includeMemories bool) (issue
// safe to drop when its id is no longer in the store (a TTL-compacted wisp,
// GH#4988); if the store still has it, dropping it repeats #4069's data
// loss. Deliberately unfiltered + MaxRows: 0, for the same reason as
// missingJSONLIssueIDsInStore's store-side query: this guard's failure mode
// reconcileAutoExportJSONL's store-side query: this guard's failure mode
// is a permanent wedge, so the query must stay maximally permissive.
func storeKnownIssueIDs(ctx context.Context) (map[string]struct{}, error) {
issues, err := store.SearchIssues(ctx, "", types.IssueFilter{Limit: 0, MaxRows: 0})
@ -809,11 +964,19 @@ func loadExportAutoState(beadsDir string) *exportAutoState {
path, len(data), err, string(data[:min(len(data), 200)]))
return &exportAutoState{}
}
if state.FormatVersion > exportAutoStateVersion {
debug.Logf("auto-export: state file %s is format v%d, this binary understands v%d; starting from empty state\n",
path, state.FormatVersion, exportAutoStateVersion)
return &exportAutoState{}
}
return &state
}
func saveExportAutoState(beadsDir string, state *exportAutoState) {
path := filepath.Join(beadsDir, exportAutoStateFile)
// Stamped here rather than at the call sites so there is exactly one
// place that decides what version a written file claims to be.
state.FormatVersion = exportAutoStateVersion
data, err := json.Marshal(state)
if err != nil {
fmt.Fprintf(os.Stderr, "Warning: auto-export: failed to marshal state: %v\n", err)
@ -1034,12 +1197,19 @@ func pathInsideDir(path, dir string) bool {
// didIncremental=false when any precondition fails so the caller can fall
// back to a full export.
//
// forcedRemovals are ids the caller has already proven deleted by some means
// the diff cannot see (the in-process delete handoff, or the store's committed
// history — see proveDeletedIssueIDs). They are unioned into the diff's own
// removals, because a proven-deleted id is by definition absent at BOTH diff
// endpoints in the cases that matter, so the patch would otherwise carry its
// stale line forward forever.
//
// dirtyIDs returns the RAW diff's upserted ids (never carryIDs) for the
// caller to persist as next cycle's carryIDs. Carrying only the
// freshly-diffed ids — not the ones merely carried-in — makes a stable
// no-op cycle self-heal: a carried id that produces no new diff simply
// drops out instead of accumulating forever.
func tryIncrementalExport(ctx context.Context, fullPath, fromCommit, toCommit string, carryIDs []string) (issueCount, memoryCount int, dirtyIDs []string, didIncremental bool, err error) {
func tryIncrementalExport(ctx context.Context, fullPath, fromCommit, toCommit string, carryIDs, forcedRemovals []string) (issueCount, memoryCount int, dirtyIDs []string, didIncremental bool, err error) {
if fromCommit == "" {
debug.Logf("auto-export: incremental skipped — no prior commit hash recorded\n")
return 0, 0, nil, false, nil
@ -1069,8 +1239,16 @@ func tryIncrementalExport(ctx context.Context, fullPath, fromCommit, toCommit st
// from last cycle (e.g. a working-set value that reverted between two
// diff anchors nets to "no change" against the anchor, but the live row
// still needs re-fetching to correct a stale patch from last cycle).
removed := make(map[string]bool, len(changed.Removed)+len(forcedRemovals))
for _, id := range changed.Removed {
removed[id] = true
}
for _, id := range forcedRemovals {
removed[id] = true
}
upsertIDs := unionStrings(changed.Upserted, carryIDs)
total := len(upsertIDs) + len(changed.Removed)
total := len(upsertIDs) + len(removed)
if total == 0 {
// Nothing changed and nothing carried over. Still a valid
// "incremental" outcome: no issues to rewrite, just refresh the
@ -1135,10 +1313,6 @@ func tryIncrementalExport(ctx context.Context, fullPath, fromCommit, toCommit st
// changed.Removed from the issues-table diff; rely on that.
}
removed := make(map[string]bool, len(changed.Removed)+len(droppedByFilter))
for _, id := range changed.Removed {
removed[id] = true
}
for id := range droppedByFilter {
removed[id] = true
}

View file

@ -13,21 +13,22 @@ import (
"github.com/steveyegge/beads/internal/types"
)
// TestMissingJSONLIssueIDsInStore_IgnoresCompactedWisp pins the actual
// GH#4988 behaviour at the layer that wedged auto-export permanently:
// missingJSONLIssueIDsInStore itself. maphew's review noted that
// TestReconcileAutoExportJSONL_IgnoresCompactedWisp pins the actual GH#4988
// behaviour at the layer that wedged auto-export permanently: the
// JSONL-vs-store reconciliation itself (formerly missingJSONLIssueIDsInStore,
// now reconcileAutoExportJSONL). maphew's review noted that
// TestMissingJSONLIssueIDsInStore is named in the original guard commit's
// validation list but doesn't exist in the tree; this closes that gap.
//
// Seeds one persistent issue (present in the store) and a JSONL file that
// additionally lists an ephemeral wisp id that is NOT in the store — the
// TTL-compaction scenario, where a wisp was written to issues.jsonl and
// later deleted from Dolt by compaction. Asserts `missing` comes back
// empty: an out-of-scope (ephemeral) row's absence from the store must not
// be treated as a hard orphan, or auto-export wedges forever.
// later deleted from Dolt by compaction. Asserts the reconciliation reports
// nothing at all: an out-of-scope (ephemeral) row's absence from the store
// must not be treated as a hard orphan, or auto-export wedges forever.
//
// Requires a live Dolt test server/container (cgo build); skips otherwise.
func TestMissingJSONLIssueIDsInStore_IgnoresCompactedWisp(t *testing.T) {
func TestReconcileAutoExportJSONL_IgnoresCompactedWisp(t *testing.T) {
if testDoltServerPort == 0 {
t.Skip("Dolt test server not available")
}
@ -85,11 +86,16 @@ func TestMissingJSONLIssueIDsInStore_IgnoresCompactedWisp(t *testing.T) {
t.Fatal(err)
}
missing, err := missingJSONLIssueIDsInStore(ctx, jsonlPath, "")
rec, err := reconcileAutoExportJSONL(ctx, jsonlPath, "")
if err != nil {
t.Fatalf("missingJSONLIssueIDsInStore: %v", err)
t.Fatalf("reconcileAutoExportJSONL: %v", err)
}
if len(missing) != 0 {
t.Fatalf("missing = %v, want empty (compacted wisp is out-of-scope and must not be a hard orphan)", missing)
if len(rec.unproven) != 0 {
t.Fatalf("unproven = %v, want empty (compacted wisp is out-of-scope and must not be a hard orphan)", rec.unproven)
}
// And it must be out of scope, not merely proven: an ephemeral row never
// reaches a proof source, so nothing about it is "reconciled" either.
if len(rec.provenDeleted) != 0 {
t.Fatalf("provenDeleted = %v, want empty (an out-of-scope wisp is never a candidate in the first place)", rec.provenDeleted)
}
}

View file

@ -314,6 +314,18 @@ func (s *spyDiffStore) GetStateHash(ctx context.Context) (string, error) {
return sh.GetStateHash(ctx)
}
// Forwarded for the same reason as the two above: the spy hides the concrete
// store from UnwrapStore, so any capability it does not re-declare simply
// vanishes from the guard's point of view. Without this the orphan guard's
// history proof silently degrades to "unproven" under the spy (GH#5896).
func (s *spyDiffStore) HistoricalIssueIDs(ctx context.Context, ids []string) (map[string]struct{}, error) {
hp, ok := storage.UnwrapStore(s.DoltStorage).(storage.HistoryPresence)
if !ok {
return nil, fmt.Errorf("spyDiffStore: wrapped store does not implement HistoryPresence")
}
return hp.HistoricalIssueIDs(ctx, ids)
}
func autoExportTestDir(t *testing.T) string {
t.Helper()
dir := t.TempDir()
@ -1222,7 +1234,11 @@ func TestShouldExport(t *testing.T) {
}
}
func TestCountIssueRecordsInJSONL(t *testing.T) {
// TestIssueRecordsInJSONL_CountsAndOrdersInScopeIDs pins the parse the
// pre-export guards read: exactly one record per id, memories and tombstones
// and id-less lines skipped, sorted by id. Both guards now take their counts
// and their candidate list from this one call.
func TestIssueRecordsInJSONL_CountsAndOrdersInScopeIDs(t *testing.T) {
tmpDir := t.TempDir()
path := filepath.Join(tmpDir, "issues.jsonl")
data := strings.Join([]string{
@ -1238,20 +1254,19 @@ func TestCountIssueRecordsInJSONL(t *testing.T) {
t.Fatal(err)
}
got, err := countIssueRecordsInJSONL(path)
records, err := issueRecordsInJSONL(path)
if err != nil {
t.Fatalf("countIssueRecordsInJSONL: %v", err)
t.Fatalf("issueRecordsInJSONL: %v", err)
}
if got != 2 {
t.Fatalf("countIssueRecordsInJSONL = %d, want 2", got)
if len(records) != 2 {
t.Fatalf("issueRecordsInJSONL returned %d records, want 2: %v", len(records), records)
}
ids, err := issueIDsInJSONL(path)
if err != nil {
t.Fatalf("issueIDsInJSONL: %v", err)
ids := make([]string, len(records))
for i, rec := range records {
ids[i] = rec.ID
}
if got := strings.Join(ids, ","); got != "bd-1,bd-2" {
t.Fatalf("issueIDsInJSONL = %q, want bd-1,bd-2", got)
t.Fatalf("issueRecordsInJSONL ids = %q, want bd-1,bd-2", got)
}
}
@ -1686,7 +1701,7 @@ func TestTryIncrementalExport_PatchesChangedIssuesAndDropsRemoved(t *testing.T)
}
c2 := h.mustCommit(t, ctx, "mutate")
issueCount, memoryCount, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
issueCount, memoryCount, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport returned error: %v", err)
}
@ -1742,7 +1757,7 @@ func TestTryIncrementalExport_DropsIssueWhenFlippedToTemplate(t *testing.T) {
}
c2 := h.mustCommit(t, ctx, "promote to template")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}
@ -1770,7 +1785,7 @@ func TestTryIncrementalExport_FallsBackWhenFileMissing(t *testing.T) {
// No existing file → must return didIncremental=false and leave the
// disk untouched so the caller falls back to the full-export path.
exportPath := filepath.Join(h.beadsDir, "issues.jsonl")
issueCount, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
issueCount, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@ -1810,7 +1825,7 @@ func TestTryIncrementalExport_ThresholdExceededFallsBack(t *testing.T) {
h.mustCreateBatch(t, ctx, incrementalExportThreshold+1, "thr-")
c2 := h.mustCommit(t, ctx, "flood")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@ -1875,7 +1890,7 @@ func TestMaybeAutoExport_SecondRunTakesIncrementalPath_ServerMode(t *testing.T)
}
// Both ChangedIssueIDs call sites must have fired: the orphan guard's
// proof-of-deletion probe (missingJSONLIssueIDsInStore, which runs
// proof-of-deletion probe (reconcileAutoExportJSONL, which runs
// because deleting e2e-b leaves it JSONL-only) and tryIncrementalExport's
// own diff. A bare "called at least once" check is NOT sufficient and was
// the PR #5806 round-5 review finding: the guard's own call satisfies it
@ -2055,7 +2070,7 @@ func TestTryIncrementalExport_NeverLeaksMemoriesIntoAutoExport(t *testing.T) {
}
c2 := h.mustCommit(t, ctx, "mutate")
_, memCount, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, memCount, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}
@ -2105,7 +2120,7 @@ func TestTryIncrementalExport_PreservesPreExistingMemoryAcrossPatch(t *testing.T
}
c2 := h.mustCommit(t, ctx, "mutate")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}
@ -2206,7 +2221,7 @@ func TestTryIncrementalExport_ExcludesConfiguredOwnerFromPatchedIssues(t *testin
}
c2 := h.mustCommit(t, ctx, "mutate")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}
@ -2239,7 +2254,7 @@ func TestTryIncrementalExport_PatchedLinesIncludeTypeField(t *testing.T) {
}
c2 := h.mustCommit(t, ctx, "mutate")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}

View file

@ -0,0 +1,793 @@
//go:build cgo
package main
import (
"context"
"encoding/json"
"os"
"path/filepath"
"reflect"
"sort"
"strings"
"testing"
"time"
"github.com/steveyegge/beads/internal/config"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/types"
)
// These tests cover GH#5896: `bd delete` used to leave a JSONL-only issue id
// that no pre-export guard could account for, so auto-export refused, never
// saved state, and refused again on every later command — forever, with the
// documented recovery (`bd init --from-jsonl`) undoing the delete.
//
// They run against a fake store that models EMBEDDED mode's capability set —
// the default open path, and the one the in-window #5806 fix could not reach:
// no StateHasher, no DiffStore, but CommitExists and (new) HistoryPresence.
// That is the same deliberate split TestMaybeAutoExport_EmbeddedModeFallsBack
// CleanlyToFullExport makes: what these assert is the GUARD's wiring — which
// proof sources it consults, what it does with the answers, what reaches the
// file — while the capability's own semantics against a real Dolt engine are
// pinned by the cross-store contract in internal/storage/storagecontract.
// fakeEmbeddedStore models EmbeddedDoltStore's capability surface for
// auto-export: HEAD-only change detection, no dolt_diff, but able to answer
// "was this id ever in my history".
type fakeEmbeddedStore struct {
storage.DoltStorage
issues []*types.Issue
historical map[string]bool
// knownCommits is what CommitExists says yes to. An anchor that isn't
// here models a rewind, a squash, or a state file from another clone.
knownCommits map[string]bool
headCommit string
// historyAsked records every id handed to HistoricalIssueIDs, so a test
// can assert what the guard did NOT ask about.
historyAsked []string
historyCalls int
historyFailed bool
}
func (f *fakeEmbeddedStore) GetCurrentCommit(_ context.Context) (string, error) {
if f.headCommit == "" {
return "head-1", nil
}
return f.headCommit, nil
}
func (f *fakeEmbeddedStore) GetInfraTypes(_ context.Context) map[string]bool { return nil }
func (f *fakeEmbeddedStore) GetConfig(_ context.Context, _ string) (string, error) {
return "", nil
}
func (f *fakeEmbeddedStore) SearchIssues(_ context.Context, _ string, _ types.IssueFilter) ([]*types.Issue, error) {
return f.issues, nil
}
func (f *fakeEmbeddedStore) GetLabelsForIssues(_ context.Context, _ []string) (map[string][]string, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) GetDependencyRecordsForIssues(_ context.Context, _ []string) (map[string][]*types.Dependency, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) GetCommentsForIssues(_ context.Context, _ []string) (map[string][]*types.Comment, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) GetCommentCounts(_ context.Context, _ []string) (map[string]int, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) GetDependencyCounts(_ context.Context, _ []string) (map[string]*types.DependencyCounts, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) CommitExists(_ context.Context, hash string) (bool, error) {
return f.knownCommits[hash], nil
}
func (f *fakeEmbeddedStore) HistoricalIssueIDs(_ context.Context, ids []string) (map[string]struct{}, error) {
f.historyCalls++
f.historyAsked = append(f.historyAsked, ids...)
if f.historyFailed {
return nil, context.DeadlineExceeded
}
present := make(map[string]struct{})
for _, id := range ids {
if f.historical[id] {
present[id] = struct{}{}
}
}
return present, nil
}
// deleteFromStore removes an id from the store's live rows the way a hard
// delete does, leaving its history intact.
func (f *fakeEmbeddedStore) deleteFromStore(id string) {
kept := f.issues[:0]
for _, iss := range f.issues {
if iss.ID != id {
kept = append(kept, iss)
}
}
f.issues = kept
}
// fakeHistorylessStore is fakeEmbeddedStore without HistoryPresence: the
// degradation case (an older or third-party store), and the fixture the P1
// handoff test needs so nothing but the handoff can prove anything.
type fakeHistorylessStore struct {
*fakeEmbeddedStore
}
func (f *fakeHistorylessStore) HistoricalIssueIDs(context.Context, []string) (map[string]struct{}, error) {
panic("HistoryPresence must not be reached on a store that does not implement it")
}
func newWedgeHealTestStore(t *testing.T, issues ...*types.Issue) *fakeEmbeddedStore {
t.Helper()
historical := make(map[string]bool, len(issues))
for _, iss := range issues {
historical[iss.ID] = true
}
return &fakeEmbeddedStore{
issues: issues,
historical: historical,
knownCommits: map[string]bool{},
headCommit: "head-1",
}
}
func wedgeHealIssue(id, title string) *types.Issue {
return &types.Issue{
ID: id,
Title: title,
Status: types.StatusOpen,
Priority: 2,
IssueType: types.TypeTask,
}
}
// setupWedgeHealTest wires config, globals and a .beads dir, and returns the
// beads dir plus the export path.
func setupWedgeHealTest(t *testing.T, store2 storage.DoltStorage) (beadsDir, exportPath string) {
t.Helper()
initConfigForTest(t)
config.Set("export.auto", true)
// The throttle is not what these tests are about, and a 1ms interval is
// long enough to swallow a second export that lands in the same
// millisecond as the first.
config.Set("export.interval", "1ns")
saveAndRestoreGlobals(t)
dir := autoExportTestDir(t)
store = store2
storeMutex.Lock()
storeActive = true
storeMutex.Unlock()
t.Cleanup(func() {
storeMutex.Lock()
storeActive = false
storeMutex.Unlock()
})
beadsDir = filepath.Join(dir, ".beads")
return beadsDir, filepath.Join(beadsDir, "issues.jsonl")
}
func jsonlIDs(t *testing.T, path string) []string {
t.Helper()
records, err := issueRecordsInJSONL(path)
if err != nil {
t.Fatalf("issueRecordsInJSONL(%s): %v", path, err)
}
ids := make([]string, len(records))
for i, rec := range records {
ids[i] = rec.ID
}
sort.Strings(ids)
return ids
}
// writeWedgedExportState writes the on-disk state a pre-fix binary leaves
// behind: a last-seen commit, an open throttle window, and whatever anchor
// the caller wants to model.
func writeWedgedExportState(t *testing.T, beadsDir, lastCommit, anchor string) {
t.Helper()
saveExportAutoState(beadsDir, &exportAutoState{
LastDoltCommit: lastCommit,
LastDiffAnchor: anchor,
Timestamp: time.Time{},
Issues: 2,
})
}
// TestMaybeAutoExport_DeleteHealsInsteadOfWedging_Embedded is the issue's own
// repro on the default backend: create → export → delete → export must
// SUCCEED and the export must reflect the deletion.
func TestMaybeAutoExport_DeleteHealsInsteadOfWedging_Embedded(t *testing.T) {
fake := newWedgeHealTestStore(t,
wedgeHealIssue("wh-a", "Alpha"),
wedgeHealIssue("wh-b", "Beta"),
)
beadsDir, exportPath := setupWedgeHealTest(t, fake)
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
if got, want := jsonlIDs(t, exportPath), []string{"wh-a", "wh-b"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Fatalf("setup: exported ids = %v, want %v", got, want)
}
// Delete wh-b the way a pre-existing store would have it — gone from the
// live rows, still in history — and touch wh-a so the test can also tell
// a healed export from a skipped one by its CONTENT.
fake.deleteFromStore("wh-b")
fake.issues[0].Title = "Alpha renamed"
fake.headCommit = "head-2"
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
})
if got, want := jsonlIDs(t, exportPath), []string{"wh-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("after delete, exported ids = %v, want %v — the deletion was not reflected", got, want)
}
if titles := loadIssueTitles(t, exportPath); titles["wh-a"] != "Alpha renamed" {
t.Errorf("wh-a title = %q, want %q — the export was skipped, not healed", titles["wh-a"], "Alpha renamed")
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "head-2" {
t.Errorf("LastDoltCommit = %q, want %q — state must advance or the next command wedges on the same comparison", state.LastDoltCommit, "head-2")
}
if !strings.Contains(stderr, "wh-b") || !strings.Contains(stderr, "reconciled") {
t.Errorf("stderr = %q, want a heal notice naming wh-b", stderr)
}
if strings.Contains(stderr, "refusing to overwrite") {
t.Errorf("stderr = %q, want no refusal", stderr)
}
}
// TestMaybeAutoExport_DeleteLastIssueHealsEmptyGuard covers the trap the
// single-issue repro actually hits: with nothing left in the store, the
// EMPTY-OVERWRITE guard fires before the orphan guard, and refuses on "0
// issues vs 1 record" unless it too knows the record is a proven deletion.
func TestMaybeAutoExport_DeleteLastIssueHealsEmptyGuard(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-only", "The only one"))
beadsDir, exportPath := setupWedgeHealTest(t, fake)
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
if got := jsonlIDs(t, exportPath); len(got) != 1 {
t.Fatalf("setup: exported ids = %v, want exactly wh-only", got)
}
fake.deleteFromStore("wh-only")
fake.headCommit = "head-2"
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "would export 0 issues") {
t.Fatalf("empty-overwrite guard refused a fully-proven deletion: %q", stderr)
}
if _, err := os.Stat(exportPath); !os.IsNotExist(err) {
t.Errorf("issues.jsonl still exists after the last issue was deleted (stat err = %v); the empty export removes the file", err)
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "head-2" {
t.Errorf("LastDoltCommit = %q, want %q", state.LastDoltCommit, "head-2")
}
}
// TestMaybeAutoExport_PreWedgedStoreRecovers is the upgrade path: the delete
// already happened under a binary that had no proof for it, so there is no
// in-process handoff to lean on and the anchor is useless. The first export
// under this code must heal anyway, from history alone.
func TestMaybeAutoExport_PreWedgedStoreRecovers(t *testing.T) {
for _, tc := range []struct {
name string
anchor string
}{
// R4: the state file was lost, or no export ever succeeded.
{name: "AnchorLost", anchor: ""},
// R2 stand-in: a well-formed anchor that no longer resolves, as
// after a history squash, a GC, or a branch checkout.
{name: "AnchorNoLongerResolves", anchor: "abcdefghijklmnopqrstuvwxyz012345"},
} {
t.Run(tc.name, func(t *testing.T) {
// The store never knew wh-b as a live row in this process; only
// its history remembers it. That IS the pre-fix on-disk shape.
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
fake.historical["wh-b"] = true
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-b", "title": "Beta", "issue_type": "task"},
)
writeWedgedExportState(t, beadsDir, "head-0", tc.anchor)
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "refusing to overwrite") {
t.Fatalf("still wedged: %q", stderr)
}
if got, want := jsonlIDs(t, exportPath), []string{"wh-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("exported ids = %v, want %v", got, want)
}
if !strings.Contains(stderr, "wh-b") {
t.Errorf("stderr = %q, want a heal notice naming wh-b", stderr)
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "head-1" {
t.Errorf("LastDoltCommit = %q, want head-1 (state must be re-saved or the wedge survives the upgrade)", state.LastDoltCommit)
}
})
}
}
// TestMaybeAutoExport_UnprovableJSONLIDStillRefuses is the fail-safe half.
// An id this store has no history of is indistinguishable from GH#4988's torn
// or replaced store, and must keep refusing with the unchanged message —
// including after a rewind, where the id's commits are no longer reachable
// from HEAD and so are no longer in dolt_history_issues.
func TestMaybeAutoExport_UnprovableJSONLIDStillRefuses(t *testing.T) {
for _, tc := range []struct {
name string
anchor string
}{
{name: "NeverExisted", anchor: ""},
{name: "RewoundAwayWithStaleAnchor", anchor: "abcdefghijklmnopqrstuvwxyz012345"},
} {
t.Run(tc.name, func(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-ghost", "title": "Ghost", "issue_type": "task"},
)
writeWedgedExportState(t, beadsDir, "head-0", tc.anchor)
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if !strings.Contains(stderr, "refusing to overwrite") {
t.Errorf("stderr = %q, want the unchanged orphan refusal", stderr)
}
if !strings.Contains(stderr, "wh-ghost") {
t.Errorf("stderr = %q, want the refusal to name wh-ghost", stderr)
}
if got := jsonlIDs(t, exportPath); len(got) != 2 {
t.Errorf("exported ids = %v, want both records preserved — an unprovable divergence must never lose a line", got)
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "head-0" {
t.Errorf("LastDoltCommit = %q, want head-0 (a refusal must not advance state)", state.LastDoltCommit)
}
})
}
}
// TestMaybeAutoExport_HistoryProofFailureRefuses: a store that cannot answer
// the history question is not a store that answered "no". The guard must fall
// back to refusing, not to overwriting.
func TestMaybeAutoExport_HistoryProofFailureRefuses(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
fake.historical["wh-b"] = true
fake.historyFailed = true
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-b", "title": "Beta", "issue_type": "task"},
)
writeWedgedExportState(t, beadsDir, "head-0", "")
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if !strings.Contains(stderr, "refusing to overwrite") {
t.Errorf("stderr = %q, want a refusal when the history proof itself failed", stderr)
}
if got := jsonlIDs(t, exportPath); len(got) != 2 {
t.Errorf("exported ids = %v, want both records preserved", got)
}
}
// TestMaybeAutoExport_DeleteHandoffProvesWhatHistoryCannot covers the one
// shape no store-side proof can ever reach: an issue created AND deleted
// without either write reaching a commit. The in-process handoff from
// deleteBatch is the only witness, and the store here has no HistoryPresence
// at all, so nothing else can cover for it.
func TestMaybeAutoExport_DeleteHandoffProvesWhatHistoryCannot(t *testing.T) {
inner := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
fake := &fakeHistorylessStore{fakeEmbeddedStore: inner}
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-uncommitted", "title": "Never committed", "issue_type": "task"},
)
writeWedgedExportState(t, beadsDir, "head-0", "")
commandDeletedIssueIDs.reset()
t.Cleanup(commandDeletedIssueIDs.reset)
commandDeletedIssueIDs.add("wh-uncommitted")
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "refusing to overwrite") {
t.Fatalf("the delete handoff did not prove the deletion: %q", stderr)
}
if got, want := jsonlIDs(t, exportPath), []string{"wh-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("exported ids = %v, want %v", got, want)
}
}
// TestMaybeAutoExport_ScopeSkipsNeverReachAProofSource: out-of-scope JSONL
// rows (ephemeral wisps, templates, infra types) are not candidates in the
// first place — GH#4988's own fix — so they must not cost a history query
// either.
func TestMaybeAutoExport_ScopeSkipsNeverReachAProofSource(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
_, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-wisp", "title": "Compacted wisp", "issue_type": "task", "ephemeral": true},
map[string]any{"_type": "issue", "id": "wh-tpl", "title": "Template", "issue_type": "task", "is_template": true},
)
rec, err := reconcileAutoExportJSONL(context.Background(), exportPath, "")
if err != nil {
t.Fatalf("reconcileAutoExportJSONL: %v", err)
}
if len(rec.unproven) != 0 || len(rec.provenDeleted) != 0 {
t.Errorf("unproven = %v, provenDeleted = %v, want both empty", rec.unproven, rec.provenDeleted)
}
if fake.historyCalls != 0 {
t.Errorf("HistoricalIssueIDs called %d times with %v; out-of-scope rows must never reach a proof source", fake.historyCalls, fake.historyAsked)
}
if rec.jsonlIssueCount != 3 {
t.Errorf("jsonlIssueCount = %d, want 3 (the empty-overwrite guard counts every record, in scope or not)", rec.jsonlIssueCount)
}
}
// TestProveDeletedIssueIDs_HistoryProofIsCapped: past the cap, a divergence is
// not a delete pattern, and the guard refuses without paying for the scan.
func TestProveDeletedIssueIDs_HistoryProofIsCapped(t *testing.T) {
fake := newWedgeHealTestStore(t)
setupWedgeHealTest(t, fake)
candidates := make([]string, historyProofMaxCandidates+1)
for i := range candidates {
candidates[i] = "wh-" + string(rune('a'+i%26)) + string(rune('0'+i/26))
fake.historical[candidates[i]] = true
}
proven, unproven := proveDeletedIssueIDs(context.Background(), candidates, "")
if len(proven) != 0 {
t.Errorf("proven = %d ids, want 0 past the cap", len(proven))
}
if len(unproven) != len(candidates) {
t.Errorf("unproven = %d ids, want all %d", len(unproven), len(candidates))
}
if fake.historyCalls != 0 {
t.Errorf("HistoricalIssueIDs called %d times; past the cap the guard must not ask", fake.historyCalls)
}
}
// TestExportAutoState_VersionRoundTripAndFutureDiscard is the GH#5896 R5
// acceptance: today's unversioned files load unchanged, a file from a future
// binary is discarded rather than reinterpreted, and — the part that makes
// discarding safe at all — discarding does not wedge, because the deletion
// proofs no longer need the anchor it threw away.
func TestExportAutoState_VersionRoundTripAndFutureDiscard(t *testing.T) {
t.Run("StampsTheCurrentVersion", func(t *testing.T) {
dir := t.TempDir()
saveExportAutoState(dir, &exportAutoState{LastDoltCommit: "c1", LastDiffAnchor: "a1"})
data, err := os.ReadFile(filepath.Join(dir, exportAutoStateFile))
if err != nil {
t.Fatalf("read state: %v", err)
}
var raw map[string]any
if err := json.Unmarshal(data, &raw); err != nil {
t.Fatalf("unmarshal state: %v", err)
}
if got := raw["v"]; got != float64(exportAutoStateVersion) {
t.Errorf("state file v = %v, want %d", got, exportAutoStateVersion)
}
state := loadExportAutoState(dir)
if state.LastDoltCommit != "c1" || state.LastDiffAnchor != "a1" {
t.Errorf("round trip lost fields: %+v", state)
}
})
t.Run("LoadsAnUnversionedFileAsIs", func(t *testing.T) {
dir := t.TempDir()
legacy := `{"last_dolt_commit":"c1","last_diff_anchor":"a1","issues":7}`
if err := os.WriteFile(filepath.Join(dir, exportAutoStateFile), []byte(legacy), 0o600); err != nil {
t.Fatal(err)
}
state := loadExportAutoState(dir)
if state.LastDoltCommit != "c1" || state.LastDiffAnchor != "a1" || state.Issues != 7 {
t.Errorf("pre-versioning state file did not load as-is: %+v", state)
}
})
t.Run("DiscardsAFutureVersionWithoutWedging", func(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
fake.historical["wh-b"] = true
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-b", "title": "Beta", "issue_type": "task"},
)
future := `{"v":999,"last_dolt_commit":"head-1","last_diff_anchor":"future-anchor"}`
if err := os.WriteFile(filepath.Join(beadsDir, exportAutoStateFile), []byte(future), 0o600); err != nil {
t.Fatal(err)
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "" || state.LastDiffAnchor != "" {
t.Fatalf("future-version state was not discarded: %+v", state)
}
// Discarding drops the anchor AND the last-seen commit, so this run
// looks like a cold start with a stale JSONL — pre-#5896 the exact
// R4 permanent wedge. It must export and re-anchor instead.
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "refusing to overwrite") {
t.Fatalf("discarding a future-version state file wedged the export: %q", stderr)
}
if got, want := jsonlIDs(t, exportPath), []string{"wh-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("exported ids = %v, want %v", got, want)
}
state := loadExportAutoState(beadsDir)
if state.FormatVersion != exportAutoStateVersion || state.LastDoltCommit != "head-1" {
t.Errorf("state after re-anchor = %+v, want v%d at head-1", state, exportAutoStateVersion)
}
})
}
// The tests below run against a REAL DoltStore over the test sql-server —
// server mode, where the store does implement DiffStore and the incremental
// path is live. What the fake-store tests cannot reach is exactly what these
// are for: dolt_history_issues answering for itself, and proven-deleted ids
// having to survive a dolt_diff-driven incremental PATCH rather than a full
// rewrite from the store.
// TestMaybeAutoExport_DeleteHealsInsteadOfWedging_ServerMode is the issue's
// repro on the server-backed store, with the anchor deliberately blanked so
// the #5806 anchor-diff proof cannot fire and the history proof has to carry
// it — the R2/R4 residue this fix is for.
func TestMaybeAutoExport_DeleteHealsInsteadOfWedging_ServerMode(t *testing.T) {
h, ctx := setupIncrementalExportTest(t)
initConfigForTest(t)
config.Set("export.auto", true)
config.Set("export.interval", "1ns")
h.mustCreate(t, ctx, "sw-a", "Alpha")
h.mustCreate(t, ctx, "sw-b", "Beta")
h.mustCommit(t, ctx, "baseline")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
exportPath := filepath.Join(h.beadsDir, "issues.jsonl")
if got := countIssueLines(t, exportPath); got != 2 {
t.Fatalf("setup: %d issue lines, want 2", got)
}
// Delete through the store directly — no in-process handoff — and blank
// the anchor, which is what a squashed history or a lost state file
// leaves behind. Only dolt_history_issues can answer now.
if err := h.store.DeleteIssue(ctx, "sw-b"); err != nil {
t.Fatalf("DeleteIssue: %v", err)
}
h.mustCommit(t, ctx, "delete sw-b")
state := loadExportAutoState(h.beadsDir)
state.LastDiffAnchor = ""
state.Timestamp = time.Time{}
saveExportAutoState(h.beadsDir, state)
stderr := captureStderr(t, func() {
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "refusing to overwrite") {
t.Fatalf("still wedged on the server-backed store: %q", stderr)
}
if got, want := jsonlIDs(t, exportPath), []string{"sw-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("exported ids = %v, want %v", got, want)
}
if !strings.Contains(stderr, "sw-b") {
t.Errorf("stderr = %q, want a heal notice naming sw-b", stderr)
}
if loadExportAutoState(h.beadsDir).LastDiffAnchor == "" {
t.Error("the export re-anchored to nothing; the next cycle would take the same anchorless path forever")
}
}
// TestMaybeAutoExport_ProvenDeletionReachesTheIncrementalPatch is plumbing
// consequence #1, and the one that fails SILENTLY if missed: the guard can
// clear the export while the incremental patch still carries the deleted
// record forward, because a row absent at BOTH diff endpoints produces no
// diff row at all.
//
// The setup makes that shape exactly: sx-x is created and deleted entirely
// AFTER the anchor, so dolt_diff(anchor, WORKING) never mentions it, while
// sx-y is modified so the run genuinely stays on the incremental path.
func TestMaybeAutoExport_ProvenDeletionReachesTheIncrementalPatch(t *testing.T) {
h, ctx := setupIncrementalExportTest(t)
initConfigForTest(t)
config.Set("export.auto", true)
config.Set("export.interval", "1ns")
spy := &spyDiffStore{DoltStorage: h.store}
store = spy
h.mustCreate(t, ctx, "sx-y", "Yankee")
h.mustCommit(t, ctx, "baseline")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
exportPath := filepath.Join(h.beadsDir, "issues.jsonl")
anchor := loadExportAutoState(h.beadsDir).LastDiffAnchor
if anchor == "" {
t.Fatal("setup: no diff anchor recorded, the incremental path can't be under test")
}
// sx-x exists only between two commits that both come after the anchor,
// and its JSONL line is written by an export that ran while it was live.
h.mustCreate(t, ctx, "sx-x", "X-ray")
h.mustCommit(t, ctx, "add sx-x")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
if _, ok := loadIssueTitles(t, exportPath)["sx-x"]; !ok {
t.Fatalf("setup: sx-x must be in the export before it is deleted")
}
// Pin the anchor back to before sx-x ever existed. Now the diff's two
// endpoints both lack it, exactly like a create+delete inside one
// throttle window.
state := loadExportAutoState(h.beadsDir)
state.LastDiffAnchor = anchor
state.Timestamp = time.Time{}
saveExportAutoState(h.beadsDir, state)
if err := h.store.DeleteIssue(ctx, "sx-x"); err != nil {
t.Fatalf("DeleteIssue: %v", err)
}
h.mustCommit(t, ctx, "delete sx-x")
if err := h.store.UpdateIssue(ctx, "sx-y", map[string]interface{}{"title": "Yankee renamed"}, "tester"); err != nil {
t.Fatalf("UpdateIssue: %v", err)
}
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("third maybeAutoExport: %v", err)
}
// LastDirtyIDs reaches state only from a successful incremental patch —
// the full-export fallback nils it — and sx-y is the only upserted id, so
// exactly {sx-y} is what says "this run was incremental AND it ran".
if got, want := loadExportAutoState(h.beadsDir).LastDirtyIDs, []string{"sx-y"}; !reflect.DeepEqual(got, want) {
t.Fatalf("LastDirtyIDs = %v, want %v — the run was skipped or fell back to a full export, and this test only means something on the incremental path", got, want)
}
titles := loadIssueTitles(t, exportPath)
if _, stillThere := titles["sx-x"]; stillThere {
t.Error("sx-x survived the incremental patch: a deletion proven outside dolt_diff never reached the file as a forced removal")
}
if titles["sx-y"] != "Yankee renamed" {
t.Errorf("sx-y title = %q, want %q", titles["sx-y"], "Yankee renamed")
}
// Content-equivalence control: an incrementally-patched file must describe
// the same issue set, field for field, as a fresh full export of the same
// state — including in what it OMITS.
controlPath := filepath.Join(t.TempDir(), "control.jsonl")
if _, _, err := exportToFile(ctx, controlPath, false); err != nil {
t.Fatalf("control exportToFile: %v", err)
}
got := jsonlRecordsByID(t, exportPath)
want := jsonlRecordsByID(t, controlPath)
if len(got) != len(want) {
t.Fatalf("incremental export has %d records, control full export has %d", len(got), len(want))
}
for id, wantRec := range want {
gotRec, ok := got[id]
if !ok {
t.Errorf("record %s present in control export, missing from incremental export", id)
continue
}
if !reflect.DeepEqual(gotRec, wantRec) {
t.Errorf("record %s differs between incremental and full export:\n incremental: %v\n full: %v", id, gotRec, wantRec)
}
}
}
// TestMaybeAutoExport_HistoryProofDoesNotSurviveARewind is the safety sibling
// of TestMaybeAutoExport_HistoryRewindDoesNotProveDeletion, aimed at the new
// proof source rather than the anchor one. After a hard reset past an id's
// creation, dolt_history_issues no longer reaches those commits, so the
// history proof must come up empty and the guard must keep refusing — with
// the line still in the file.
func TestMaybeAutoExport_HistoryProofDoesNotSurviveARewind(t *testing.T) {
h, ctx := setupIncrementalExportTest(t)
initConfigForTest(t)
config.Set("export.auto", true)
config.Set("export.interval", "1ns")
h.mustCreate(t, ctx, "sr-a", "Alpha")
c1 := h.mustCommit(t, ctx, "baseline")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
exportPath := filepath.Join(h.beadsDir, "issues.jsonl")
h.mustCreate(t, ctx, "sr-x", "Rewound")
h.mustCommit(t, ctx, "add sr-x")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
if titles := loadIssueTitles(t, exportPath); titles["sr-x"] != "Rewound" {
t.Fatalf("setup: sr-x must be exported before the rewind, got %v", titles)
}
raw, ok := storage.UnwrapStore(h.store).(storage.RawDBAccessor)
if !ok {
t.Skip("store does not expose raw DB access")
}
if _, err := raw.DB().ExecContext(ctx, "CALL DOLT_RESET('--hard', ?)", c1); err != nil {
t.Fatalf("CALL DOLT_RESET('--hard', %s): %v", c1, err)
}
// Blank the anchor so the anchor-diff proof cannot even be consulted:
// whatever happens next is the history proof's answer alone.
state := loadExportAutoState(h.beadsDir)
state.LastDiffAnchor = ""
state.Timestamp = time.Time{}
saveExportAutoState(h.beadsDir, state)
stderr := captureStderr(t, func() {
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("third maybeAutoExport: %v", err)
}
})
if !strings.Contains(stderr, "refusing to overwrite") {
t.Errorf("stderr = %q, want the unchanged refusal: a rewound-away id is data this store LOST, not a deletion it recorded", stderr)
}
if _, stillThere := loadIssueTitles(t, exportPath)["sr-x"]; !stillThere {
t.Error("sr-x was silently dropped from issues.jsonl after a history rewind: the history proof accepted a rewound-away row as a proven deletion")
}
}

View file

@ -110,6 +110,13 @@ var (
// an intentional empty JSONL artifact instead of treating it as ambiguous.
commandMayEmptyJSONLExport atomic.Bool
// commandDeletedIssueIDs is the in-process handoff from the delete
// commands to post-run auto-export: ids this command hard-deleted are
// PROVEN gone, so the export's orphan guard skips them instead of
// mistaking a deliberate delete for a torn store and refusing forever
// (GH#5896). See deletedIssueIDSet for why it is not persisted.
commandDeletedIssueIDs deletedIssueIDSet
// commandDidExplicitDoltCommit is set when a command already created a Dolt commit
// explicitly (e.g., bd sync in dolt-native mode, hook flows, bd vc commit).
// This prevents a redundant auto-commit attempt in PersistentPostRun.
@ -892,6 +899,7 @@ var rootCmd = &cobra.Command{
// Reset per-command write tracking (used by Dolt auto-commit).
commandDidWrite.Store(false)
commandMayEmptyJSONLExport.Store(false)
commandDeletedIssueIDs.reset()
commandDidExplicitDoltCommit = false
commandDidWriteTipMetadata = false
commandTipIDsShown = make(map[string]struct{})

View file

@ -0,0 +1,43 @@
package dolt
import (
"context"
"testing"
"github.com/steveyegge/beads/internal/storage/storagecontract"
"github.com/steveyegge/beads/internal/types"
)
// TestHistoryPresenceContract runs the HistoryPresence contract against the
// server-backed store. It shares a body with the embedded leg
// (issueops.HistoricalIssueIDsInTx) and differs only in how the read
// connection is opened, which is what this wiring checks.
func TestHistoryPresenceContract(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
fixture := storagecontract.HistoryPresenceFixture{
Prefix: "hp",
Presence: store,
CreateIssue: func(ctx context.Context, id, title string) error {
return store.CreateIssue(ctx, &types.Issue{
ID: id,
Title: title,
Status: types.StatusOpen,
Priority: 2,
IssueType: types.TypeTask,
}, "tester")
},
DeleteIssue: store.DeleteIssue,
Commit: store.Commit,
CurrentCommit: store.GetCurrentCommit,
ResetHard: func(ctx context.Context, commit string) error {
_, err := store.db.ExecContext(ctx, "CALL DOLT_RESET('--hard', ?)", commit)
return err
},
}
storagecontract.RunHistoryPresenceContract(t, ctx, fixture)
}

View file

@ -33,6 +33,30 @@ func (s *DoltStore) History(ctx context.Context, issueID string) ([]*storage.His
return result, err
}
// HistoricalIssueIDs reports which of ids appear in HEAD's committed history
// of the issues table. Implements storage.HistoryPresence.
//
// Uses withReadTxLongTimeout for the same reason History does (ga-ahnxx): the
// dolt_history_issues scan underneath is proportional to commits × revisions
// and can outrun the shared pool's 10s ReadTimeout on a mature repo, where it
// would surface as an intermittent i/o timeout rather than an answer.
func (s *DoltStore) HistoricalIssueIDs(ctx context.Context, ids []string) (map[string]struct{}, error) {
var present map[string]struct{}
err := s.withReadTxLongTimeout(ctx, func(tx *sql.Tx) error {
var err error
present, err = issueops.HistoricalIssueIDsInTx(ctx, tx, ids)
if err != nil {
return wrapQueryError("check issue history presence", err)
}
return nil
})
return present, err
}
// The auto-export guard reaches this through storage.UnwrapStore, so the
// assertion must keep holding on the concrete store.
var _ storage.HistoryPresence = (*DoltStore)(nil)
// AsOf returns the state of an issue at a specific commit hash or branch ref.
// Implements storage.VersionedStorage.
func (s *DoltStore) AsOf(ctx context.Context, issueID string, ref string) (*types.Issue, error) {

View file

@ -0,0 +1,44 @@
//go:build cgo
package embeddeddolt_test
import (
"context"
"testing"
"github.com/steveyegge/beads/internal/storage/storagecontract"
"github.com/steveyegge/beads/internal/types"
)
// TestHistoryPresenceContract runs the HistoryPresence contract against the
// embedded store — the DEFAULT open path, and the one with no DiffStore, so
// this capability is auto-export's only way to prove a deletion there
// (GH#5896).
func TestHistoryPresenceContract(t *testing.T) {
skipUnlessEmbeddedDolt(t)
te := newTestEnv(t, "hp")
ctx := t.Context()
fixture := storagecontract.HistoryPresenceFixture{
Prefix: "hp",
Presence: te.store,
CreateIssue: func(ctx context.Context, id, title string) error {
return te.store.CreateIssue(ctx, &types.Issue{
ID: id,
Title: title,
Status: types.StatusOpen,
Priority: 2,
IssueType: types.TypeTask,
}, "tester")
},
DeleteIssue: te.store.DeleteIssue,
Commit: te.store.Commit,
CurrentCommit: te.store.GetCurrentCommit,
ResetHard: func(ctx context.Context, commit string) error {
te.exec(t, ctx, "CALL DOLT_RESET('--hard', ?)", commit)
return nil
},
}
storagecontract.RunHistoryPresenceContract(t, ctx, fixture)
}

View file

@ -274,6 +274,26 @@ func (s *EmbeddedDoltStore) CommitExists(ctx context.Context, commitHash string)
return exists, err
}
// HistoricalIssueIDs reports which of ids appear in HEAD's committed history
// of the issues table. Implements storage.HistoryPresence.
//
// This is the capability that lets embedded mode — the default open path, and
// the one with no DiffStore — prove a deletion for auto-export's orphan guard
// instead of wedging on it forever (GH#5896). Same body as the server-backed
// store; only the connection differs.
func (s *EmbeddedDoltStore) HistoricalIssueIDs(ctx context.Context, ids []string) (map[string]struct{}, error) {
var present map[string]struct{}
err := s.withConn(ctx, false, func(tx *sql.Tx) error {
var err error
present, err = issueops.HistoricalIssueIDsInTx(ctx, tx, ids)
return err
})
return present, err
}
// The auto-export guard reaches this through storage.UnwrapStore.
var _ storage.HistoryPresence = (*EmbeddedDoltStore)(nil)
func (s *EmbeddedDoltStore) Status(ctx context.Context) (*storage.Status, error) {
var status *storage.Status
err := s.withDBConn(ctx, func(db versioncontrolops.DBConn) error {

View file

@ -0,0 +1,26 @@
package storage
import "context"
// HistoryPresence reports which of the given issue ids appear anywhere in the
// CURRENT branch's committed history of the issues table — that is, in some
// commit reachable from HEAD. A rewound-away past does not count, and neither
// does the uncommitted working set.
//
// That asymmetry is the whole point. "In this store's history but not in this
// store now" is safe to read as "a deletion this store recorded", while "not in
// this store's history and not in this store now" is indistinguishable from a
// torn, replaced, or rewound store — the corruption class GH#4988's auto-export
// orphan guard exists to catch. Auto-export uses the first as proof that a
// JSONL-only record was really deleted, and keeps refusing to overwrite on the
// second (GH#5896).
//
// It is a capability interface: callers should type-assert after UnwrapStore
// and degrade to the conservative path when a store does not implement it.
type HistoryPresence interface {
// HistoricalIssueIDs returns the subset of ids that appear in HEAD's
// committed history. Ids the store has never seen are simply absent from
// the result; an empty or nil ids slice returns an empty set and issues
// no query.
HistoricalIssueIDs(ctx context.Context, ids []string) (map[string]struct{}, error)
}

View file

@ -0,0 +1,86 @@
package issueops
import (
"context"
"fmt"
"strings"
)
// historicalIssueIDsChunk caps how many ids go into one IN-list. Dolt's
// history tables fan a row out per commit per id, and very wide IN-lists are
// where its planner starts choosing badly; the callers' own candidate cap is
// far below this, so chunking is a floor, not a hot path.
const historicalIssueIDsChunk = 200
// HistoricalIssueIDsInTx returns the subset of ids that appear anywhere in the
// current branch's committed history of the issues table.
//
// dolt_history_issues walks HEAD's ancestry, so an id whose only commits were
// rewound away (DOLT_RESET --hard, a checkout, a force-pushed rewrite) is NOT
// reported, and neither is an id that exists only in the uncommitted working
// set. Both omissions are load-bearing: see storage.HistoryPresence.
//
// The subquery wrapper is the same workaround HistoryInTx uses — Dolt's
// planner assumes a predicate on the PK of a dolt_history_* table matches at
// most one row, which is false there (one row per commit per id).
func HistoricalIssueIDsInTx(ctx context.Context, tx DBTX, ids []string) (map[string]struct{}, error) {
present := make(map[string]struct{})
if len(ids) == 0 {
return present, nil
}
// Dedupe up front so a caller's repeated id can't inflate a chunk.
unique := make([]string, 0, len(ids))
seen := make(map[string]struct{}, len(ids))
for _, id := range ids {
if id == "" {
continue
}
if _, dup := seen[id]; dup {
continue
}
seen[id] = struct{}{}
unique = append(unique, id)
}
for start := 0; start < len(unique); start += historicalIssueIDsChunk {
end := start + historicalIssueIDsChunk
if end > len(unique) {
end = len(unique)
}
if err := historicalIssueIDChunkInTx(ctx, tx, unique[start:end], present); err != nil {
return nil, err
}
}
return present, nil
}
func historicalIssueIDChunkInTx(ctx context.Context, tx DBTX, chunk []string, present map[string]struct{}) error {
args := make([]any, len(chunk))
for i, id := range chunk {
args[i] = id
}
query := `
SELECT DISTINCT h.id
FROM (
SELECT id FROM dolt_history_issues
) h
WHERE h.id IN (` + strings.TrimSuffix(strings.Repeat("?,", len(chunk)), ",") + `)`
rows, err := tx.QueryContext(ctx, query, args...)
if err != nil {
return fmt.Errorf("query issue history presence: %w", err)
}
defer rows.Close()
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return fmt.Errorf("scan issue history presence: %w", err)
}
present[id] = struct{}{}
}
if err := rows.Err(); err != nil {
return fmt.Errorf("iterate issue history presence: %w", err)
}
return nil
}

View file

@ -0,0 +1,210 @@
// Package storagecontract holds cross-store contracts for storage CAPABILITY
// interfaces — the narrow optional interfaces a store may implement and that
// callers reach by type-asserting after storage.UnwrapStore.
//
// It is deliberately not backend/conformance. That package is the ROLE tier:
// its Run*(t, ctx, <Name>Fixture) entrypoints are enumerated by a drift gate
// that demands a RoleContractBundle field and a wiring on every registered leg
// (internal/storage/contract_leg_registry_test.go). A capability that only two
// of those legs can implement — HistoryPresence needs Dolt history, which the
// unit-of-work leg has none of — would have to be waived there rather than
// described. Here the wiring list IS the claim.
package storagecontract
import (
"context"
"testing"
"github.com/steveyegge/beads/internal/storage"
)
// HistoryPresenceFixture supplies the store-specific hooks the contract needs.
// Every hook must act on the SAME store as Presence.
type HistoryPresenceFixture struct {
// Prefix namespaces the ids each case seeds so several suites can share
// one database.
Prefix string
// Presence is the capability under test.
Presence storage.HistoryPresence
// CreateIssue seeds one durable issue.
CreateIssue func(ctx context.Context, id, title string) error
// DeleteIssue hard-deletes one issue.
DeleteIssue func(ctx context.Context, id string) error
// Commit makes a Dolt commit of whatever is staged.
Commit func(ctx context.Context, msg string) error
// CurrentCommit returns HEAD.
CurrentCommit func(ctx context.Context) (string, error)
// ResetHard rewinds the branch to commit, discarding the working set. A
// nil hook means the leg cannot rewind its own history, and the rewind
// case SKIPS rather than passing quietly.
ResetHard func(ctx context.Context, commit string) error
}
// RunHistoryPresenceContract runs every case in order against one fixture.
//
// The cases share a store and run sequentially on purpose: the last one
// rewinds the branch, which would pull the ground out from under any case
// still running. Each case names the exact ids it seeded.
func RunHistoryPresenceContract(t *testing.T, ctx context.Context, fx HistoryPresenceFixture) {
t.Helper()
t.Run("ReportsALiveCommittedID", func(t *testing.T) {
RunHistoryPresenceReportsALiveCommittedID(t, ctx, fx)
})
t.Run("ReportsADeletedID", func(t *testing.T) {
RunHistoryPresenceReportsADeletedID(t, ctx, fx)
})
t.Run("OmitsAnIDThatNeverExisted", func(t *testing.T) {
RunHistoryPresenceOmitsAnIDThatNeverExisted(t, ctx, fx)
})
t.Run("OmitsAnUncommittedCreateAndDelete", func(t *testing.T) {
RunHistoryPresenceOmitsAnUncommittedCreateAndDelete(t, ctx, fx)
})
t.Run("AnswersAnEmptyRequest", func(t *testing.T) {
RunHistoryPresenceAnswersAnEmptyRequest(t, ctx, fx)
})
t.Run("OmitsARewoundAwayID", func(t *testing.T) {
RunHistoryPresenceOmitsARewoundAwayID(t, ctx, fx)
})
}
// RunHistoryPresenceReportsALiveCommittedID: an id that is committed and still
// in the store is in its history. The auto-export guard never asks about one —
// a live id is not a candidate — but a capability that answered "absent" here
// would be reporting deletions, not history.
func RunHistoryPresenceReportsALiveCommittedID(t *testing.T, ctx context.Context, fx HistoryPresenceFixture) {
t.Helper()
id := fx.Prefix + "-live"
mustCreate(t, ctx, fx, id, "Live")
mustCommit(t, ctx, fx, "seed live")
assertPresence(t, ctx, fx, []string{id}, map[string]bool{id: true})
}
// RunHistoryPresenceReportsADeletedID is the proof auto-export depends on: an
// id whose creation was committed and which has since been deleted is in
// history and absent from the store — the signature of a real deletion.
func RunHistoryPresenceReportsADeletedID(t *testing.T, ctx context.Context, fx HistoryPresenceFixture) {
t.Helper()
id := fx.Prefix + "-gone"
mustCreate(t, ctx, fx, id, "Doomed")
mustCommit(t, ctx, fx, "seed doomed")
if err := fx.DeleteIssue(ctx, id); err != nil {
t.Fatalf("DeleteIssue(%s): %v", id, err)
}
mustCommit(t, ctx, fx, "delete doomed")
assertPresence(t, ctx, fx, []string{id}, map[string]bool{id: true})
}
// RunHistoryPresenceOmitsAnIDThatNeverExisted is the fail-safe half: a JSONL
// line naming an id this store has never held is exactly GH#4988's torn-store
// signal, and must stay unproven.
func RunHistoryPresenceOmitsAnIDThatNeverExisted(t *testing.T, ctx context.Context, fx HistoryPresenceFixture) {
t.Helper()
id := fx.Prefix + "-neverexisted"
assertPresence(t, ctx, fx, []string{id}, map[string]bool{id: false})
}
// RunHistoryPresenceOmitsAnUncommittedCreateAndDelete pins that history
// presence follows COMMITS, not the working set. This shape — created and
// deleted without either write reaching a commit — is the one no store-side
// proof can ever cover, which is why auto-export also carries an in-process
// delete handoff.
func RunHistoryPresenceOmitsAnUncommittedCreateAndDelete(t *testing.T, ctx context.Context, fx HistoryPresenceFixture) {
t.Helper()
id := fx.Prefix + "-working"
before, err := fx.CurrentCommit(ctx)
if err != nil {
t.Fatalf("CurrentCommit before: %v", err)
}
mustCreate(t, ctx, fx, id, "Working set only")
if err := fx.DeleteIssue(ctx, id); err != nil {
t.Fatalf("DeleteIssue(%s): %v", id, err)
}
after, err := fx.CurrentCommit(ctx)
if err != nil {
t.Fatalf("CurrentCommit after: %v", err)
}
if before != after {
t.Skipf("this store commits its own writes (HEAD moved %s → %s), so it has no uncommitted-create shape to assert about", before, after)
}
assertPresence(t, ctx, fx, []string{id}, map[string]bool{id: false})
}
// RunHistoryPresenceAnswersAnEmptyRequest: no ids in, no ids out, no error.
// The guard calls this only when it has candidates, but a capability that
// errored on an empty slice would push that precondition onto every caller.
func RunHistoryPresenceAnswersAnEmptyRequest(t *testing.T, ctx context.Context, fx HistoryPresenceFixture) {
t.Helper()
present, err := fx.Presence.HistoricalIssueIDs(ctx, nil)
if err != nil {
t.Fatalf("HistoricalIssueIDs(nil): %v", err)
}
if len(present) != 0 {
t.Errorf("HistoricalIssueIDs(nil) = %v, want an empty set", present)
}
}
// RunHistoryPresenceOmitsARewoundAwayID is the asymmetry that makes the whole
// capability safe to read as proof of deletion. dolt_history_issues walks
// HEAD's ancestry, so a hard reset to before an id's creation makes its
// commits unreachable and the id unproven — the store looks like it lost data,
// which is precisely what it did, and auto-export must keep refusing.
//
// Runs last: it rewinds the branch the earlier cases seeded.
func RunHistoryPresenceOmitsARewoundAwayID(t *testing.T, ctx context.Context, fx HistoryPresenceFixture) {
t.Helper()
if fx.ResetHard == nil {
t.Skip("fixture cannot rewind history")
}
id := fx.Prefix + "-rewound"
anchor, err := fx.CurrentCommit(ctx)
if err != nil {
t.Fatalf("CurrentCommit: %v", err)
}
mustCreate(t, ctx, fx, id, "Rewound")
mustCommit(t, ctx, fx, "seed rewound")
assertPresence(t, ctx, fx, []string{id}, map[string]bool{id: true})
if err := fx.ResetHard(ctx, anchor); err != nil {
t.Fatalf("ResetHard(%s): %v", anchor, err)
}
assertPresence(t, ctx, fx, []string{id}, map[string]bool{id: false})
}
func assertPresence(t *testing.T, ctx context.Context, fx HistoryPresenceFixture, ask []string, want map[string]bool) {
t.Helper()
present, err := fx.Presence.HistoricalIssueIDs(ctx, ask)
if err != nil {
t.Fatalf("HistoricalIssueIDs(%v): %v", ask, err)
}
for id, wantPresent := range want {
_, got := present[id]
if got != wantPresent {
t.Errorf("HistoricalIssueIDs(%v): %s present=%v, want %v", ask, id, got, wantPresent)
}
}
for id := range present {
if _, asked := want[id]; !asked {
t.Errorf("HistoricalIssueIDs(%v) returned %s, which was not asked about", ask, id)
}
}
}
func mustCreate(t *testing.T, ctx context.Context, fx HistoryPresenceFixture, id, title string) {
t.Helper()
if err := fx.CreateIssue(ctx, id, title); err != nil {
t.Fatalf("CreateIssue(%s): %v", id, err)
}
}
func mustCommit(t *testing.T, ctx context.Context, fx HistoryPresenceFixture, msg string) {
t.Helper()
if err := fx.Commit(ctx, msg); err != nil {
t.Fatalf("Commit(%q): %v", msg, err)
}
}