From 387959695d2e112e57b5d21398b409cc0517b018 Mon Sep 17 00:00:00 2001 From: Steve Yegge Date: Fri, 19 Jun 2026 21:31:33 -0700 Subject: [PATCH] fix(dolt): pre-pull commit must include config (kv.memory.* pull-wedge) (#4412) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(dolt): pre-pull commit must include config; auto-resolve convergent kv.memory.* config conflicts bd remember stores persistent memories as kv.memory.* rows in the synced config table. Commit() deliberately excludes config (GH#2455, to avoid sweeping a concurrent writer's stale issue_prefix into an unrelated commit), so the GH#2474 pre-pull auto-commit at pullFromRemote (store.go) and PullFrom (federation.go) leaves config permanently dirty once any memory exists. The "clean the working set before merging" step then leaves config dirty and DOLT_MERGE refuses to start ("cannot merge with uncommitted changes"). Two correct PRs colliding over config. Diagnosed from a live server-mode repro (Wyvern wy-71t); see PROPOSAL-pull-config-wedge.md. Fix (server mode): add commitBeforePull, which stages config explicitly via DOLT_ADD rather than relying on CommitWithConfig's DOLT_COMMIT('-Am'). The proposal author observed -Am not staging config reliably under the server-mode stored-procedure path, and the new embedded test TestEmbeddedConfigCommitStagesConfig confirms -Am DOES stage config in the embedded engine — so the unreliability is server-mode-specific and explicit staging is the robust choice. Commit() and commitBeforePull now share commitWorkingSet(includeConfig bool). Hardening (both modes): making config a committed/synced table means two clones editing the same memory key collide on a config merge conflict. Extend TryAutoResolveMergeConflicts to auto-resolve config conflicts with --theirs ONLY when every conflicted row is a kv.memory.* key (machine-convergent, same policy as metadata); any other config key (issue_prefix above all) is left for the operator. Without this the primary fix would trade a silent pull wedge for an operator-visible conflict wedge on the exact bd remember workload. Embedded pre-pull (CommitPending -> -Am) already stages config and is unchanged; it gains the resolver hardening. Tests: server-mode pull_config_wedge_test.go (commit-includes-config + resolver positive/negative, run under Docker in CI) and embedded pull_config_wedge_test.go (in-process: -Am-stages-config probe + resolver positive/negative, all green). Co-Authored-By: Claude Opus 4.8 (1M context) * docs: add pull-config-wedge diagnosis/proposal (PR #4412) The diagnosis-and-patch write-up behind this PR: how GH#2455 (Commit excludes config) and GH#2474 (pre-pull auto-commit) collide over the synced config table, leaving kv.memory.* rows permanently uncommitted so DOLT_MERGE refuses to start. Includes the live-DB validation (Wyvern wy-71t) and the rejected alternatives. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(dolt): adopt-pr maintainer fixes for pull/sync config wedge Maintainer review fixups for PR #4412, squashed into one commit on top of the contributor's branch. Pre-merge config wedge (sync): bd federation sync committed with the config-excluding Commit before its merge while this PR switched the pull paths to commitBeforePull, so any DB that ran `bd remember` carried dirty kv.memory.* config rows and hit the "cannot merge with uncommitted changes" wedge. Sync now uses commitBeforePull too, with a regression test. Pre-pull auto-commit safety (GH#2455): commitBeforePull no longer stages the whole config table. It auto-commits only convergent memory rows (kv.memory.*); if any other config key is dirty (issue_prefix above all) it refuses with operator guidance instead of sweeping a stale value into the merge basis, restoring the GH#2455 protection the broad pre-pull staging had reopened. Sync --strategy config commit: a config-only conflict resolution committed through the config-excluding Commit, so it could report success without creating the merge-resolution commit and re-wedge the next sync. Sync now commits resolutions through a config-inclusive path (commitMergeResolution). Embedded sync parity: EmbeddedDoltStore.Sync fetched and merged without the pre-merge CommitPending guard already used by embedded Pull/PullRemote/PullFrom and the new server Sync, so the same dirty kv.memory.* working set could still wedge `bd federation sync` in embedded mode. Sync now commits pending changes before fetch/merge (embedded Commit is DOLT_COMMIT('-Am'), which stages config), with an embedded regression test that drives Sync with a dirty kv.memory.* row. KV/memory namespace collision: `bd kv set memory.*` stored to kv.memory.*, indistinguishable from a `bd remember` memory the merge resolver auto-resolves with --theirs. validateKVKey now reserves the memory. prefix so a generic KV value can't be silently overridden by the remote on pull. Shared prefix + tests: centralize the kv.memory. prefix in internal/storage/kvkeys (used by cmd/bd and the merge resolver) with a contract test; route the remaining cmd/bd kv. prefix literals (the doctor KV check, the validateKVKey nested-prefix guard, and the memory listing) through the kvkeys constants; and strengthen the server-mode non-memory config conflict test to prove a conflict materialized before the resolver declines it. Maintainer review fixup for PR #4412. Co-Authored-By: Claude Opus 4.8 --------- Co-authored-by: Claude Opus 4.8 (1M context) Co-authored-by: Eddie the Engineer --- PROPOSAL-pull-config-wedge.md | 173 +++++ cmd/bd/doctor/kv.go | 3 +- cmd/bd/export_test.go | 7 +- cmd/bd/kv.go | 15 +- cmd/bd/kv_test.go | 9 + cmd/bd/memory.go | 6 +- cmd/bd/vc.go | 9 +- internal/storage/dolt/federation.go | 19 +- .../storage/dolt/pull_config_wedge_test.go | 646 ++++++++++++++++++ internal/storage/dolt/store.go | 157 ++++- internal/storage/embeddeddolt/federation.go | 11 + .../embeddeddolt/pull_config_wedge_test.go | 231 +++++++ .../storage/embeddeddolt/version_control.go | 9 + internal/storage/kvkeys/kvkeys.go | 29 + internal/storage/kvkeys/kvkeys_test.go | 19 + internal/storage/version_control.go | 7 + .../storage/versioncontrolops/mergesettle.go | 120 +++- 17 files changed, 1438 insertions(+), 32 deletions(-) create mode 100644 PROPOSAL-pull-config-wedge.md create mode 100644 internal/storage/dolt/pull_config_wedge_test.go create mode 100644 internal/storage/embeddeddolt/pull_config_wedge_test.go create mode 100644 internal/storage/kvkeys/kvkeys.go create mode 100644 internal/storage/kvkeys/kvkeys_test.go diff --git a/PROPOSAL-pull-config-wedge.md b/PROPOSAL-pull-config-wedge.md new file mode 100644 index 000000000..b3f39df32 --- /dev/null +++ b/PROPOSAL-pull-config-wedge.md @@ -0,0 +1,173 @@ +# Proposal: fix `bd dolt pull` "cannot merge with uncommitted changes" wedge + +**Status:** for review (diagnosis + patch). Authored 2026-06-14 from a live repro in +the Wyvern beads DB (server mode, git-protocol remote on Bitbucket). + +## Symptom + +`bd dolt pull` always fails: + +``` +Error 1105 (HY000): cannot merge with uncommitted changes +``` + +`bd doctor` reports `Dolt Locks: config: modified` immediately after every command, +even right after `bd dolt commit`, and even with `--dolt-auto-commit on`. Tracked as +Wyvern wy-71t. Push is unaffected; only pull (the cross-clone *receive* path) is wedged. + +## Root cause — two correct PRs collide over the `config` table + +1. **Memories live in `config`.** `bd remember` stores each persistent memory as a + `kv.memory.` row in the **synced** `config` table. Confirmed via the working + set on the live DB — the only dirty rows were exactly the 5 memories: + + ``` + SELECT to_key, diff_type FROM dolt_diff('main','WORKING','config'); + kv.memory.beads-jsonl-sync-procedure added + kv.memory.bitbucket-altssh-dead added + kv.memory.local-dev-ssl-posture added + kv.memory.react-client-e2e-against-the-local-game-server added + kv.memory.react-e2e-same-character-parallel-logins added + ``` + +2. **`Commit` deliberately excludes `config`** (GH#2455, `store.go` ~1731): + `DOLT_COMMIT('-Am', …)` once swept stale `issue_prefix` changes from concurrent + operations into commits, so `Commit` now stages every dirty table *except* `config`. + +3. **The pre-pull auto-commit uses `Commit`** (GH#2474). Both pull paths try to clean + the working set before merging, but call `s.Commit`, which skips `config`: + + - `internal/storage/dolt/store.go:2309` — `pullFromRemote` (default remote; what + `bd dolt pull` invokes via `st.Pull`) + - `internal/storage/dolt/federation.go:65` — `PullFrom` (named-peer pull) + +**Net effect:** once any memory exists, the `kv.memory.*` rows sit permanently +uncommitted (nothing ever commits `config`), so the pre-pull "clean the working set" +step leaves `config` dirty, and `DOLT_MERGE` refuses to start — the exact case +`abortMerge`'s comment already calls out: *"a merge that REFUSED TO START on a dirty +working set."* GH#2474's intent is silently defeated by GH#2455 whenever `config` is +the dirty table, which is always after the first `bd remember`. + +This is independent of the v42→v51 schema-migration fork (#4259) seen in the same DB; +that was a separate, newer problem. Migrating + force-publishing fixed the fork and +restored `push`, but `pull` stays wedged because of this config issue. + +## Empirical validation (live DB) + +Confirmed the mechanism end-to-end without any source change: + +1. The only dirty table was `config`; the only dirty rows were the 5 `kv.memory.*` + entries (see query above). +2. Explicitly committing config — `CALL DOLT_ADD('config'); CALL DOLT_COMMIT('-m', …)` — + left the working set **clean** (`dolt_diff('HEAD','WORKING','config')` → 0 rows). +3. Running ordinary commands afterward (`bd list`, `bd show`) kept it clean (still 0). + So bd's per-command memory re-sync is **idempotent**: once the rows match HEAD, + re-writing identical values produces no diff. The "re-dirties after every command" + symptom existed *only* because the rows were never committed (Commit skips config), + so they showed as permanently uncommitted. Committing config once breaks the cycle. + +This proves a clean `config` working set is the *only* thing the merge needs, and that +a pre-merge commit of config is a permanent fix, not a per-command band-aid. + +**Confirmed:** after committing config, `bd dolt pull` succeeds ("Pull complete.") +repeatably, `bd dolt push` still works, and `bd doctor` no longer reports +`config: modified`. The wedge is entirely the uncommitted-config condition; the patch +just makes the pull path establish it automatically. (Without the patch the wedge +returns on the next `bd remember`, which adds a fresh uncommitted `kv.memory.*` row.) + +**One wrinkle to verify in the patch:** a standalone `bd dolt commit` (which already +routes through `CommitWithConfig` → `DOLT_COMMIT('-Am', …)`) did **not** visibly clean +`config` in this server-mode DB, whereas the explicit `DOLT_ADD('config')` + +`DOLT_COMMIT('-m', …)` did. That suggests `-Am` may not be staging `config` reliably +under the SQL stored-procedure path here. The patch author should confirm `CommitWithConfig` +actually commits `config` in server mode; if not, the pre-pull step should stage config +explicitly (`CALL DOLT_ADD('config')` then commit) rather than rely on `-Am`. A regression +test should assert `dolt_status` is empty immediately after the pre-pull commit. + +## Fix (primary) — pre-pull commit must include `config` + +`CommitWithConfig` already exists (`store.go:1780`) for exactly this "commit everything, +including intentional config" case — `CommitPending` (`bd dolt commit`) already uses it. +Use it in the two pre-pull sites: + +```diff +--- a/internal/storage/dolt/store.go ++++ b/internal/storage/dolt/store.go +@@ pullFromRemote (GH#2474 pre-pull auto-commit) +- if err := s.Commit(ctx, "auto-commit before pull"); err != nil { ++ // Include config: persistent memories live in config as kv.memory.* rows, ++ // and Commit() excludes config (GH#2455), so they never commit and leave ++ // the working set permanently dirty — DOLT_MERGE then refuses to start. ++ if err := s.CommitWithConfig(ctx, "auto-commit before pull"); err != nil { + if !isDoltNothingToCommit(err) { + return fmt.Errorf("failed to commit pending changes before pull: %w", err) + } + } +``` + +```diff +--- a/internal/storage/dolt/federation.go ++++ b/internal/storage/dolt/federation.go +@@ PullFrom (GH#2474 pre-pull auto-commit) +- if err := s.Commit(ctx, "auto-commit before pull"); err != nil { ++ if err := s.CommitWithConfig(ctx, "auto-commit before pull"); err != nil { + if !isDoltNothingToCommit(err) { + return nil, fmt.Errorf("failed to commit pending changes before pull: %w", err) + } + } +``` + +### Why this does not reintroduce GH#2455 + +GH#2455 guards against `-Am` sweeping a *concurrent writer's* half-applied `issue_prefix` +change into an unrelated commit. The pre-pull commit is different in kind: it is an +explicit user action (`bd dolt pull`) that **requires** a clean working set, and it is +committing *this clone's own* config state as the basis for the merge — exactly what +`CommitPending` already does for `bd dolt commit`. The race window GH#2455 worried about +(another bd process mutating `issue_prefix` at the same instant) is not meaningfully +widened by committing config here versus letting the user's next `bd dolt commit` do it. + +## Fix (optional hardening) — auto-resolve convergent `config` merge conflicts + +With the primary fix, `config` (incl. `kv.memory.*`) becomes a committed, synced table, +so two clones that edit the *same* memory will produce a `config` merge conflict. +`TryAutoResolveMergeConflicts` (`mergesettle.go`) currently handles only `metadata`, +`dependencies`, and `schema_migrations` — **not** `config` — so such a conflict would +stop the pull for the operator. + +Recommend a follow-up that teaches the resolver to auto-resolve `config` conflicts on +machine-convergent keys (e.g. `kv.memory.*`) with `--theirs`, the same convergent policy +already used for the `metadata` table. Out of scope for the wedge fix itself; the primary +change resolves the reported bug. (Until then, divergent same-memory edits are a rare, +operator-visible case rather than a silent wedge.) + +## Alternative considered — move memories to a `dolt_ignore`'d table + +Rejected: memories are meant to **sync** (they round-trip through `bd export | bd import`, +and are part of cross-clone shared state). `dolt_ignore` (as used for wisps) would make +them machine-local and stop syncing them via Dolt — a behavior change, not a fix. + +## Test plan + +Regression test (alongside `pull_conflict_test.go` / `federation_pull_settle_test.go`): + +1. Two clones of a tiny remote-backed DB. +2. On clone A: `bd remember --key t "x"` (dirties `config` with a `kv.memory.*` row, + committing nothing via the normal path). +3. On clone B: create+push an issue. +4. On clone A: `Pull` — **must succeed** (today it returns + `cannot merge with uncommitted changes`), and clone A must end with B's issue *and* + its own memory intact. + +Manual repro on the Wyvern DB: a fresh `bd dolt pull` fails today purely on the 5 +`kv.memory.*` rows; with the patch the pre-pull `CommitWithConfig` commits them and the +merge proceeds. + +## Files + +- `internal/storage/dolt/store.go:2309` (`pullFromRemote`) — primary +- `internal/storage/dolt/federation.go:65` (`PullFrom`) — primary +- `internal/storage/dolt/store.go:1780` (`CommitWithConfig`) — reused, unchanged +- `internal/storage/versioncontrolops/mergesettle.go` (`TryAutoResolveMergeConflicts`) + — optional hardening +``` diff --git a/cmd/bd/doctor/kv.go b/cmd/bd/doctor/kv.go index daf34e0f3..f9e545650 100644 --- a/cmd/bd/doctor/kv.go +++ b/cmd/bd/doctor/kv.go @@ -6,10 +6,11 @@ import ( "strings" "github.com/steveyegge/beads/internal/storage/dolt" + "github.com/steveyegge/beads/internal/storage/kvkeys" ) // kvPrefix matches the prefix used in cmd/bd/kv.go -const kvPrefix = "kv." +const kvPrefix = kvkeys.Prefix // CheckKVSyncStatus checks if KV data exists and reports sync status. func CheckKVSyncStatus(path string) DoctorCheck { diff --git a/cmd/bd/export_test.go b/cmd/bd/export_test.go index 0e33fb733..d44d69d65 100644 --- a/cmd/bd/export_test.go +++ b/cmd/bd/export_test.go @@ -13,6 +13,7 @@ import ( "testing" "time" + "github.com/steveyegge/beads/internal/storage/kvkeys" "github.com/steveyegge/beads/internal/testutil" "github.com/steveyegge/beads/internal/types" ) @@ -586,7 +587,7 @@ func TestExportMemoryDeterminism(t *testing.T) { // Seed 5 memories with keys that would sort differently than insertion order. memKeys := []string{"zeta-config", "alpha-note", "mu-decision", "beta-lesson", "omega-context"} for _, mk := range memKeys { - storageKey := "kv.memory." + mk + storageKey := kvkeys.MemoryConfigKeyPrefix + mk if err := s.SetConfig(ctx, storageKey, "value-for-"+mk); err != nil { t.Fatalf("SetConfig(%s): %v", storageKey, err) } @@ -747,7 +748,7 @@ func TestExportByteStabilityAllRecordTypes(t *testing.T) { // Memories, keyed so insertion order differs from sorted order. for _, mk := range []string{"zeta-config", "alpha-note", "mu-decision", "beta-lesson"} { - if err := s.SetConfig(ctx, "kv.memory."+mk, "value-for-"+mk); err != nil { + if err := s.SetConfig(ctx, kvkeys.MemoryConfigKeyPrefix+mk, "value-for-"+mk); err != nil { t.Fatalf("SetConfig(%s): %v", mk, err) } } @@ -1014,7 +1015,7 @@ func TestExportExcludesMemoriesByDefault(t *testing.T) { // Seed memories. for _, mk := range []string{"secret-api-pattern", "debug-session-notes"} { - storageKey := "kv.memory." + mk + storageKey := kvkeys.MemoryConfigKeyPrefix + mk if err := s.SetConfig(ctx, storageKey, "sensitive-value-for-"+mk); err != nil { t.Fatalf("SetConfig(%s): %v", storageKey, err) } diff --git a/cmd/bd/kv.go b/cmd/bd/kv.go index 5c9742324..f90081c61 100644 --- a/cmd/bd/kv.go +++ b/cmd/bd/kv.go @@ -7,10 +7,12 @@ import ( "strings" "github.com/spf13/cobra" + + "github.com/steveyegge/beads/internal/storage/kvkeys" ) // kvPrefix is prepended to all user keys to separate them from internal config -const kvPrefix = "kv." +const kvPrefix = kvkeys.Prefix // validateKVKey checks if a key is valid for the KV store. // Returns an error if the key is invalid. @@ -22,9 +24,18 @@ func validateKVKey(key string) error { return fmt.Errorf("key cannot be only whitespace") } // Prevent keys that would create nested kv.kv.* prefixes - if strings.HasPrefix(key, "kv.") { + if strings.HasPrefix(key, kvPrefix) { return fmt.Errorf("key cannot start with 'kv.' (would create nested prefix)") } + // Reserve the persistent-memory namespace: a generic memory.* key would + // store to kv.memory.*, indistinguishable from a `bd remember` memory, and + // the merge resolver auto-resolves kv.memory.* conflicts with --theirs + // (GH#2474). Without this guard a user's deliberate kv value could be + // silently overridden by a remote on pull. Keep the namespace owned by + // bd remember / bd forget. + if strings.HasPrefix(key, kvkeys.MemoryPrefix) { + return fmt.Errorf("key cannot start with %q (reserved for persistent memories; use 'bd remember' / 'bd forget')", kvkeys.MemoryPrefix) + } // Prevent keys that look like internal config if strings.HasPrefix(key, "sync.") || strings.HasPrefix(key, "conflict.") || strings.HasPrefix(key, "federation.") || strings.HasPrefix(key, "jira.") || diff --git a/cmd/bd/kv_test.go b/cmd/bd/kv_test.go index f5493e82f..ea1de9d23 100644 --- a/cmd/bd/kv_test.go +++ b/cmd/bd/kv_test.go @@ -203,6 +203,15 @@ func TestValidateKVKey(t *testing.T) { {"linear prefix", "linear.key", true, "reserved prefix"}, {"export prefix", "export.path", true, "reserved prefix"}, {"import prefix", "import.path", true, "reserved prefix"}, + + // memory.* is reserved for `bd remember`: a generic kv.memory.* key is + // indistinguishable from a memory and the merge resolver auto-resolves it + // with --theirs (GH#2474), so it must not be settable via `bd kv set`. + {"memory prefix", "memory.foo", true, "reserved for persistent memories"}, + {"memory prefix slug", "memory.test-wedge", true, "reserved for persistent memories"}, + // A key that merely contains "memory" but does not start with the prefix + // is still valid — only the actual namespace is reserved. + {"memory not a prefix", "my.memory.note", false, ""}, } for _, tc := range testCases { diff --git a/cmd/bd/memory.go b/cmd/bd/memory.go index 52647079a..fe2cc4f95 100644 --- a/cmd/bd/memory.go +++ b/cmd/bd/memory.go @@ -8,10 +8,12 @@ import ( "strings" "github.com/spf13/cobra" + + "github.com/steveyegge/beads/internal/storage/kvkeys" ) // memoryPrefix is prepended (after kvPrefix) to all memory keys. -const memoryPrefix = "memory." +const memoryPrefix = kvkeys.MemoryPrefix // memoryKeyFlag allows explicit key override for bd remember. var memoryKeyFlag string @@ -129,7 +131,7 @@ Examples: } // Filter for kv.memory.* keys - fullPrefix := kvPrefix + memoryPrefix + fullPrefix := kvkeys.MemoryConfigKeyPrefix memories := make(map[string]string) for k, v := range allConfig { if strings.HasPrefix(k, fullPrefix) { diff --git a/cmd/bd/vc.go b/cmd/bd/vc.go index 877e1dfa8..26a836cc9 100644 --- a/cmd/bd/vc.go +++ b/cmd/bd/vc.go @@ -68,8 +68,13 @@ Examples: } // Conclude the merge: an unresolved-then-resolved working set // stays uncommitted otherwise, and the merged-in writes - // bypassed every is_blocked hook (bd-578h9.11). - if err := store.Commit(ctx, fmt.Sprintf("Resolve merge conflicts from %s using %s strategy", branchName, vcMergeStrategy)); err != nil { + // bypassed every is_blocked hook (bd-578h9.11). Use + // CommitMergeResolution, not Commit: server-mode Commit excludes + // config (GH#2455), so a resolved config conflict — routine now + // that kv.* user data syncs through config — would be silently + // dropped, leaving the merge unconcluded and re-wedging the next + // pull/sync (GH#2474). + if err := store.CommitMergeResolution(ctx, fmt.Sprintf("Resolve merge conflicts from %s using %s strategy", branchName, vcMergeStrategy)); err != nil { FatalErrorRespectJSON("conflicts resolved but commit failed: %v", err) } if rs, ok := store.(interface { diff --git a/internal/storage/dolt/federation.go b/internal/storage/dolt/federation.go index 68d331609..b798f9d10 100644 --- a/internal/storage/dolt/federation.go +++ b/internal/storage/dolt/federation.go @@ -62,7 +62,7 @@ func (s *DoltStore) PullFrom(ctx context.Context, peer string) ([]storage.Confli // GH#2474: Auto-commit pending changes before pull to prevent // "cannot merge with uncommitted changes" errors. if !s.readOnly { - if err := s.Commit(ctx, "auto-commit before pull"); err != nil { + if err := s.commitBeforePull(ctx, "auto-commit before pull"); err != nil { if !isDoltNothingToCommit(err) { return nil, fmt.Errorf("failed to commit pending changes before pull: %w", err) } @@ -311,10 +311,13 @@ func (s *DoltStore) Sync(ctx context.Context, peer string, strategy string) (*Sy StartTime: time.Now(), } - // Match PullFrom: federation metadata writes such as add-peer must not - // leave the working set dirty before fetch/merge. + // GH#2474: match PullFrom — commit pending changes before the merge, + // INCLUDING config (where kv.memory.* rows live). Plain Commit excludes + // config (GH#2455), so federation metadata writes such as add-peer plus any + // persistent memories would otherwise leave the working set dirty and wedge + // DOLT_MERGE ("cannot merge with uncommitted changes"). if !s.readOnly { - if err := s.Commit(ctx, "auto-commit before sync"); err != nil { + if err := s.commitBeforePull(ctx, "auto-commit before sync"); err != nil { if !isDoltNothingToCommit(err) { result.Error = fmt.Errorf("failed to commit pending changes before sync: %w", err) return result, result.Error @@ -359,8 +362,12 @@ func (s *DoltStore) Sync(ctx context.Context, peer string, strategy string) (*Sy } result.ConflictsResolved = true - // Commit the resolution - if err := s.Commit(ctx, fmt.Sprintf("Resolve conflicts from %s using %s strategy", peer, strategy)); err != nil { + // Commit the resolution INCLUDING config: the operator chose this + // strategy, and plain Commit excludes config (GH#2455). A config-only + // conflict — routine now that kv.memory.* memories sync through config — + // would otherwise resolve but never commit, leaving the merge + // unconcluded and re-wedging the next sync. + if err := s.CommitMergeResolution(ctx, fmt.Sprintf("Resolve conflicts from %s using %s strategy", peer, strategy)); err != nil { result.Error = fmt.Errorf("failed to commit conflict resolution: %w", err) return result, result.Error } diff --git a/internal/storage/dolt/pull_config_wedge_test.go b/internal/storage/dolt/pull_config_wedge_test.go new file mode 100644 index 000000000..857ab17bd --- /dev/null +++ b/internal/storage/dolt/pull_config_wedge_test.go @@ -0,0 +1,646 @@ +package dolt + +import ( + "context" + "database/sql" + "strings" + "testing" + + "github.com/steveyegge/beads/internal/storage" +) + +// configDirty reports whether the config table has uncommitted changes in the +// working set — the exact condition that makes DOLT_MERGE refuse to start. +func configDirty(t *testing.T, ctx context.Context, db *sql.DB) bool { + t.Helper() + var n int + if err := db.QueryRowContext(ctx, + "SELECT COUNT(*) FROM dolt_status WHERE table_name = 'config'").Scan(&n); err != nil { + t.Fatalf("query dolt_status: %v", err) + } + return n > 0 +} + +// TestCommitBeforePullIncludesConfig is the regression test for the pull config +// wedge: persistent memories live in config as kv.memory.* rows, plain Commit() +// excludes config (GH#2455), so they sit permanently uncommitted and the +// pre-pull "clean the working set" step leaves config dirty — DOLT_MERGE then +// refuses to start ("cannot merge with uncommitted changes"). commitBeforePull +// must stage config explicitly and leave the working set clean. +func TestCommitBeforePullIncludesConfig(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + db := store.db + + // Simulate `bd remember`: a kv.memory.* row in the synced config table. + if _, err := db.ExecContext(ctx, + "INSERT INTO config (`key`, value) VALUES ('kv.memory.test-wedge', 'v1')"); err != nil { + t.Fatalf("insert config memory row: %v", err) + } + + // Plain Commit() leaves config dirty — the wedge precondition. (With only + // config dirty it commits nothing and returns nil.) + if err := store.Commit(ctx, "commit excluding config"); err != nil { + t.Fatalf("Commit: %v", err) + } + if !configDirty(t, ctx, db) { + t.Fatalf("Commit() unexpectedly committed config; the wedge precondition no longer reproduces (did GH#2455's config exclusion change?)") + } + + // commitBeforePull must stage config and leave the working set clean so the + // subsequent merge can start. + if err := store.commitBeforePull(ctx, "auto-commit before pull"); err != nil { + t.Fatalf("commitBeforePull: %v", err) + } + if configDirty(t, ctx, db) { + t.Fatalf("commitBeforePull left config dirty; DOLT_MERGE would still refuse to start") + } +} + +// TestFederationSyncCommitsConfigBeforeFetch is the `bd federation sync` +// analogue of the pull config wedge: Sync auto-commits the working set before +// its merge, and that pre-merge commit must include config. Plain Commit +// (GH#2455) excludes config, so a dirty kv.memory.* row used to survive Sync's +// pre-merge commit and wedge DOLT_MERGE — the same failure this PR fixes for the +// pull paths. The pre-merge commit must leave config clean even though Sync then +// fails at fetch against a nonexistent remote. +func TestFederationSyncCommitsConfigBeforeFetch(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + db := store.db + + // Register a peer whose remote does not exist: Sync runs its pre-merge + // auto-commit, then fails at fetch — far enough to prove config was staged. + peer := &storage.FederationPeer{ + Name: "peer-config-wedge", + RemoteURL: "file:///tmp/beads-no-such-federation-peer", + Sovereignty: "T2", + } + if err := store.AddFederationPeer(ctx, peer); err != nil { + t.Fatalf("add federation peer: %v", err) + } + + // Simulate `bd remember`: a kv.memory.* row dirties the synced config table. + if _, err := db.ExecContext(ctx, + "INSERT INTO config (`key`, value) VALUES ('kv.memory.sync-wedge', 'v1')"); err != nil { + t.Fatalf("insert config memory row: %v", err) + } + if !configDirty(t, ctx, db) { + t.Fatalf("inserted memory row did not dirty config; cannot reproduce the wedge precondition") + } + + if _, err := store.Sync(ctx, peer.Name, ""); err == nil { + t.Fatal("expected sync to fail for nonexistent file remote") + } + if configDirty(t, ctx, db) { + t.Fatal("Sync left config dirty before merge; bd federation sync would wedge DOLT_MERGE (GH#2474)") + } +} + +// TestPullAutoResolveMemoryConfigConflicts verifies that a merge conflict +// limited to kv.memory.* config rows is auto-resolved with "theirs" — the same +// machine-convergent policy used for metadata. Without this, making config a +// synced table (so memories round-trip) would turn same-memory edits across +// clones into an operator-visible pull wedge. +func TestPullAutoResolveMemoryConfigConflicts(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + db := store.db + + // Stage config explicitly: -Am was observed not to stage config reliably + // under the server-mode stored-procedure path, so the test must not depend + // on it to set up the conflict. + commitConfig := func(msg string) { + t.Helper() + if _, err := db.ExecContext(ctx, "CALL DOLT_ADD('config')"); err != nil { + t.Fatalf("dolt add config: %v", err) + } + if _, err := db.ExecContext(ctx, "CALL DOLT_COMMIT('-m', ?)", msg); err != nil { + t.Fatalf("dolt commit: %v", err) + } + } + + value := resolveConfigConflict(t, ctx, store, "kv.memory.shared", "ours", "theirs", commitConfig) + if value != "theirs" { + t.Errorf("resolved memory value = %q, want \"theirs\" (--theirs convergent)", value) + } +} + +// TestPullAutoResolveSkipsNonMemoryConfigConflicts verifies the prefix boundary: +// a config key under kv. but NOT kv.memory. (e.g. a user kv setting, or +// issue_prefix) is a real semantic conflict and must be left for the operator, +// so the whole config table is declined. +func TestPullAutoResolveSkipsNonMemoryConfigConflicts(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + db := store.db + + commitConfig := func(msg string) { + t.Helper() + if _, err := db.ExecContext(ctx, "CALL DOLT_ADD('config')"); err != nil { + t.Fatalf("dolt add config: %v", err) + } + if _, err := db.ExecContext(ctx, "CALL DOLT_COMMIT('-m', ?)", msg); err != nil { + t.Fatalf("dolt commit: %v", err) + } + } + + resolved := tryResolveConfigConflict(t, ctx, store, "kv.custom.setting", "ours", "theirs", commitConfig) + if resolved { + t.Fatalf("non-memory config conflict was auto-resolved; only kv.memory.* keys are safe") + } +} + +// resolveConfigConflict sets up a same-key config conflict on a divergent +// branch, merges it, runs the auto-resolver, and (asserting it resolved) +// commits and returns the resolved value. commitFn stages+commits config. +func resolveConfigConflict(t *testing.T, ctx context.Context, store *DoltStore, key, ours, theirs string, commitFn func(string)) string { + t.Helper() + resolved, db := runConfigConflictMerge(t, ctx, store, key, ours, theirs, commitFn, true) + var value string + if err := db.QueryRowContext(ctx, "SELECT value FROM config WHERE `key` = ?", key).Scan(&value); err != nil { + t.Fatalf("read resolved config value: %v", err) + } + if !resolved { + t.Fatalf("config conflict on %q was not auto-resolved", key) + } + return value +} + +// tryResolveConfigConflict is resolveConfigConflict's negative-path sibling: it +// returns whether the resolver accepted the conflict without requiring it to. +func tryResolveConfigConflict(t *testing.T, ctx context.Context, store *DoltStore, key, ours, theirs string, commitFn func(string)) bool { + t.Helper() + resolved, _ := runConfigConflictMerge(t, ctx, store, key, ours, theirs, commitFn, false) + return resolved +} + +// runConfigConflictMerge builds a same-key config conflict (ours on the current +// branch, theirs on a divergent "remote" branch), merges remote into current +// with conflict-tolerant session flags, and runs tryAutoResolveMergeConflicts. +// When commitOnResolve and the resolver succeeds, the merge tx is committed so +// callers can read the settled value; otherwise the tx is rolled back. +func runConfigConflictMerge(t *testing.T, ctx context.Context, store *DoltStore, key, ours, theirs string, commitFn func(string), commitOnResolve bool) (bool, *sql.DB) { + t.Helper() + db := store.db + + var currentBranch string + if err := db.QueryRowContext(ctx, "SELECT active_branch()").Scan(¤tBranch); err != nil { + t.Fatalf("active_branch: %v", err) + } + + // Our value on the current branch. + if _, err := db.ExecContext(ctx, "INSERT INTO config (`key`, value) VALUES (?, ?)", key, ours); err != nil { + t.Fatalf("insert local config: %v", err) + } + commitFn("local config") + + // Their value on a branch forked from the common ancestor (HEAD~1). + remoteBranch := currentBranch + "_cfgremote" + if _, err := db.ExecContext(ctx, "CALL DOLT_BRANCH(?, 'HEAD~1')", remoteBranch); err != nil { + t.Fatalf("create remote branch: %v", err) + } + defer func() { + db.ExecContext(ctx, "CALL DOLT_CHECKOUT(?)", currentBranch) + db.ExecContext(ctx, "CALL DOLT_BRANCH('-D', ?)", remoteBranch) + }() + if _, err := db.ExecContext(ctx, "CALL DOLT_CHECKOUT(?)", remoteBranch); err != nil { + t.Fatalf("checkout remote branch: %v", err) + } + if _, err := db.ExecContext(ctx, "INSERT INTO config (`key`, value) VALUES (?, ?)", key, theirs); err != nil { + t.Fatalf("insert remote config: %v", err) + } + commitFn("remote config") + if _, err := db.ExecContext(ctx, "CALL DOLT_CHECKOUT(?)", currentBranch); err != nil { + t.Fatalf("checkout current branch: %v", err) + } + + tx, err := db.BeginTx(ctx, nil) + if err != nil { + t.Fatalf("begin tx: %v", err) + } + if _, err := tx.ExecContext(ctx, "SET @@dolt_allow_commit_conflicts = 1"); err != nil { + _ = tx.Rollback() + t.Fatalf("set dolt_allow_commit_conflicts: %v", err) + } + _, mergeErr := tx.ExecContext(ctx, "CALL DOLT_MERGE(?)", remoteBranch) + + // Prove the merge actually produced a config conflict before the resolver + // runs. tryAutoResolveMergeConflicts returns (false, nil) both when it + // DECLINES a conflict and when there is NO conflict at all, so without this + // the negative test (TestPullAutoResolveSkipsNonMemoryConfigConflicts) would + // pass vacuously if the conflict ever stopped forming. + var configConflicts int + if err := tx.QueryRowContext(ctx, + "SELECT COUNT(*) FROM dolt_conflicts WHERE `table` = 'config'").Scan(&configConflicts); err != nil { + _ = tx.Rollback() + t.Fatalf("query config conflicts: %v (mergeErr: %v)", err, mergeErr) + } + if configConflicts == 0 { + _ = tx.Rollback() + t.Fatalf("merge produced no config conflict on %q; the resolve/decline path is not exercised (mergeErr: %v)", key, mergeErr) + } + + resolved, resolveErr := store.tryAutoResolveMergeConflicts(ctx, tx) + if resolveErr != nil { + _ = tx.Rollback() + t.Fatalf("tryAutoResolveMergeConflicts error: %v (mergeErr: %v)", resolveErr, mergeErr) + } + if !resolved { + _ = tx.Rollback() + return false, db + } + if !commitOnResolve { + _ = tx.Rollback() + return true, db + } + if err := tx.Commit(); err != nil { + t.Fatalf("commit after auto-resolve: %v", err) + } + return true, db +} + +// TestCommitBeforePullRefusesUnsafeConfig is the GH#2455 guard on the pre-pull +// auto-commit: it must include this clone's own user kv.* data so it stops +// wedging DOLT_MERGE (GH#2474), but it must NOT sweep an unrelated dirty INTERNAL +// config key such as issue_prefix into an automatic commit — that is the +// stale-config corruption GH#2455 fixed. With a non-kv. config key dirty, +// commitBeforePull must refuse, name the key, leave config uncommitted, and not +// advance HEAD. +func TestCommitBeforePullRefusesUnsafeConfig(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + db := store.db + + headBefore, err := store.GetCurrentCommit(ctx) + if err != nil { + t.Fatalf("get HEAD before: %v", err) + } + + // A non-memory config key dirties config (a stale issue_prefix edit is the + // exact GH#2455 hazard). REPLACE works whether or not the row already exists. + if _, err := db.ExecContext(ctx, + "REPLACE INTO config (`key`, value) VALUES ('issue_prefix', 'unsafe-prefix')"); err != nil { + t.Fatalf("dirty non-memory config row: %v", err) + } + if !configDirty(t, ctx, db) { + t.Fatalf("non-memory config row did not dirty config; cannot reproduce the hazard") + } + + err = store.commitBeforePull(ctx, "auto-commit before pull") + if err == nil { + t.Fatalf("commitBeforePull auto-committed an unsafe dirty config key; GH#2455 stale-config sweep is reintroduced") + } + if !strings.Contains(err.Error(), "issue_prefix") { + t.Errorf("refusal should name the unsafe key issue_prefix; got: %v", err) + } + if !configDirty(t, ctx, db) { + t.Errorf("commitBeforePull must leave the unsafe config row uncommitted for the operator") + } + headAfter, err := store.GetCurrentCommit(ctx) + if err != nil { + t.Fatalf("get HEAD after: %v", err) + } + if headAfter != headBefore { + t.Errorf("commitBeforePull advanced HEAD (%s -> %s) despite refusing unsafe config", headBefore, headAfter) + } +} + +// TestCommitMergeResolutionIncludesConfig proves the bd federation sync +// --strategy resolution commit includes config where plain Commit drops it. +// A config-only resolution is routine now that kv.memory.* memories sync through +// config; if it is not committed the merge stays unconcluded and re-wedges the +// next sync (GH#2474). It uses a non-memory key to show CommitMergeResolution +// commits whatever the operator resolved, unlike the kv.*-screened commitBeforePull. +func TestCommitMergeResolutionIncludesConfig(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + db := store.db + + // A resolved config row left in the working set after --strategy. + if _, err := db.ExecContext(ctx, + "INSERT INTO config (`key`, value) VALUES ('kv.custom.resolved', 'theirs')"); err != nil { + t.Fatalf("insert resolved config row: %v", err) + } + + // Plain Commit excludes config (GH#2455), so it would silently drop the + // resolution — the bug. (With only config dirty it commits nothing.) + if err := store.Commit(ctx, "plain commit excludes config"); err != nil { + t.Fatalf("Commit: %v", err) + } + if !configDirty(t, ctx, db) { + t.Fatalf("plain Commit committed config; cannot demonstrate the resolution-commit gap") + } + + // CommitMergeResolution must commit it, concluding the merge. + if err := store.CommitMergeResolution(ctx, "Resolve conflicts using theirs strategy"); err != nil { + t.Fatalf("CommitMergeResolution: %v", err) + } + if configDirty(t, ctx, db) { + t.Fatalf("CommitMergeResolution left config dirty; a config-only sync resolution would not be committed (GH#2474 re-wedge)") + } +} + +// TestStrategyResolutionCommitsConfigConflict drives an actual config merge +// conflict through the resolution sequence bd federation sync --strategy runs — +// ResolveConflicts(theirs) then the resolution commit — and asserts config is +// committed. Before the fix the resolution committed through config-excluding +// Commit (GH#2455), so a config-only resolution returned success without +// committing, leaving config dirty and re-wedging the next sync (GH#2474). +// +// The conflicted working set is staged in a conflict-tolerant transaction (what +// a conflict-tolerant merge leaves behind) so the test can then exercise the +// real autocommit ResolveConflicts + CommitMergeResolution calls without the +// MaxOpenConns=1 deadlock that holding the tx open would cause. +func TestStrategyResolutionCommitsConfigConflict(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + db := store.db + key := "kv.memory.sync-strategy" + + commitConfig := func(msg string) { + t.Helper() + if _, err := db.ExecContext(ctx, "CALL DOLT_ADD('config')"); err != nil { + t.Fatalf("dolt add config: %v", err) + } + if _, err := db.ExecContext(ctx, "CALL DOLT_COMMIT('-m', ?)", msg); err != nil { + t.Fatalf("dolt commit: %v", err) + } + } + + var currentBranch string + if err := db.QueryRowContext(ctx, "SELECT active_branch()").Scan(¤tBranch); err != nil { + t.Fatalf("active_branch: %v", err) + } + + // Our value on the current branch. + if _, err := db.ExecContext(ctx, "INSERT INTO config (`key`, value) VALUES (?, ?)", key, "ours"); err != nil { + t.Fatalf("insert local config: %v", err) + } + commitConfig("local config") + + // Their value on a branch forked from the common ancestor (HEAD~1). + remoteBranch := currentBranch + "_syncstrategy" + if _, err := db.ExecContext(ctx, "CALL DOLT_BRANCH(?, 'HEAD~1')", remoteBranch); err != nil { + t.Fatalf("create remote branch: %v", err) + } + defer func() { + db.ExecContext(ctx, "CALL DOLT_CHECKOUT(?)", currentBranch) + db.ExecContext(ctx, "CALL DOLT_BRANCH('-D', ?)", remoteBranch) + }() + if _, err := db.ExecContext(ctx, "CALL DOLT_CHECKOUT(?)", remoteBranch); err != nil { + t.Fatalf("checkout remote branch: %v", err) + } + if _, err := db.ExecContext(ctx, "INSERT INTO config (`key`, value) VALUES (?, ?)", key, "theirs"); err != nil { + t.Fatalf("insert remote config: %v", err) + } + commitConfig("remote config") + if _, err := db.ExecContext(ctx, "CALL DOLT_CHECKOUT(?)", currentBranch); err != nil { + t.Fatalf("checkout current branch: %v", err) + } + + // Merge in a conflict-tolerant tx and commit the tx so the conflicted working + // set persists (autocommit rolls a conflicted merge back). This is the state + // the resolution sequence inherits. + func() { + tx, err := db.BeginTx(ctx, nil) + if err != nil { + t.Fatalf("begin merge tx: %v", err) + } + defer func() { _ = tx.Rollback() }() + if _, err := tx.ExecContext(ctx, "SET @@dolt_allow_commit_conflicts = 1"); err != nil { + t.Fatalf("set dolt_allow_commit_conflicts: %v", err) + } + if _, err := tx.ExecContext(ctx, "SET @@dolt_force_transaction_commit = 1"); err != nil { + t.Fatalf("set dolt_force_transaction_commit: %v", err) + } + if _, err := tx.ExecContext(ctx, "CALL DOLT_MERGE(?)", remoteBranch); err != nil { + t.Fatalf("merge: %v", err) + } + var n int + if err := tx.QueryRowContext(ctx, + "SELECT COUNT(*) FROM dolt_conflicts WHERE `table` = 'config'").Scan(&n); err != nil { + t.Fatalf("query config conflicts: %v", err) + } + if n == 0 { + t.Fatalf("merge produced no config conflict; cannot exercise the --strategy path") + } + if err := tx.Commit(); err != nil { + t.Fatalf("commit conflicted working set: %v", err) + } + }() + + // The exact resolution sequence Sync runs for --strategy theirs. + if err := store.ResolveConflicts(ctx, "config", "theirs"); err != nil { + t.Fatalf("ResolveConflicts: %v", err) + } + if err := store.CommitMergeResolution(ctx, "Resolve conflicts using theirs strategy"); err != nil { + t.Fatalf("CommitMergeResolution: %v", err) + } + + if configDirty(t, ctx, db) { + t.Fatalf("config left dirty after --strategy resolution; the merge is unconcluded and re-wedges the next sync (GH#2474)") + } + var value string + if err := db.QueryRowContext(ctx, "SELECT value FROM config WHERE `key` = ?", key).Scan(&value); err != nil { + t.Fatalf("read resolved config value: %v", err) + } + if value != "theirs" { + t.Errorf("resolved config value = %q, want \"theirs\" (--strategy theirs)", value) + } +} + +// TestCommitBeforePullCommitsGenericKV is the regression for the generic kv.* +// pre-pull gap (PR #4412 review attempt 4): `bd kv set foo bar` writes a kv.foo +// row into the synced config table without committing it (SetConfig does not +// commit), and plain Commit excludes config (GH#2455), so the row sits dirty and +// the next server-mode pull/sync would refuse to start. The pre-pull auto-commit +// must include the whole user kv.* namespace — not just kv.memory.* memories — so +// an ordinary `bd kv set` no longer wedges DOLT_MERGE. (The narrower auto-resolve +// policy stays kv.memory.*-only; see TestPullAutoResolveSkipsNonMemoryConfigConflicts.) +func TestCommitBeforePullCommitsGenericKV(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + db := store.db + + headBefore, err := store.GetCurrentCommit(ctx) + if err != nil { + t.Fatalf("get HEAD before: %v", err) + } + + // Simulate `bd kv set custom.setting v1`: a generic (non-memory) kv.* row. + if _, err := db.ExecContext(ctx, + "INSERT INTO config (`key`, value) VALUES ('kv.custom.setting', 'v1')"); err != nil { + t.Fatalf("insert generic kv config row: %v", err) + } + if !configDirty(t, ctx, db) { + t.Fatalf("generic kv row did not dirty config; cannot reproduce the wedge") + } + + // commitBeforePull must stage the kv.* row and leave config clean — the same + // un-wedge it already does for memories — instead of refusing it. + if err := store.commitBeforePull(ctx, "auto-commit before pull"); err != nil { + t.Fatalf("commitBeforePull refused a generic kv.* row; `bd kv set` still wedges pull/sync: %v", err) + } + if configDirty(t, ctx, db) { + t.Fatalf("commitBeforePull left a generic kv.* row dirty; DOLT_MERGE would still refuse to start") + } + headAfter, err := store.GetCurrentCommit(ctx) + if err != nil { + t.Fatalf("get HEAD after: %v", err) + } + if headAfter == headBefore { + t.Fatalf("commitBeforePull did not advance HEAD; the generic kv.* row was not committed") + } +} + +// TestVCMergeStrategyCommitsConfigResolution is the server-mode regression for +// the `bd vc merge --strategy` config-resolution gap (PR #4412 review attempt 4): +// cmd/bd/vc.go concluded the resolution with config-excluding Commit (GH#2455), so +// a resolved INTERNAL config key — issue_prefix is the exact re-wedge the finding +// names, since unlike a kv.memory.* key it does not self-heal on the next pull — +// was dropped, leaving the merge unconcluded and refusing the next pull (GH#2474). +// It drives the same ResolveConflicts + CommitMergeResolution sequence cmd/bd/vc.go +// now runs and asserts config is committed and converged. +// +// The conflicted working set is staged in a conflict-tolerant transaction (what a +// conflict-tolerant merge leaves behind) so the test can then exercise the real +// autocommit ResolveConflicts + CommitMergeResolution calls without the +// MaxOpenConns=1 deadlock that holding the tx open would cause. +func TestVCMergeStrategyCommitsConfigResolution(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + db := store.db + // issue_prefix is an internal key (not kv.*): a resolved value left dirty + // re-wedges the next pull via assertDirtyConfigUserKVOnly, which is precisely + // why the resolution must be committed with config included. + key := "issue_prefix" + + commitConfig := func(msg string) { + t.Helper() + if _, err := db.ExecContext(ctx, "CALL DOLT_ADD('config')"); err != nil { + t.Fatalf("dolt add config: %v", err) + } + if _, err := db.ExecContext(ctx, "CALL DOLT_COMMIT('-m', ?)", msg); err != nil { + t.Fatalf("dolt commit: %v", err) + } + } + + var currentBranch string + if err := db.QueryRowContext(ctx, "SELECT active_branch()").Scan(¤tBranch); err != nil { + t.Fatalf("active_branch: %v", err) + } + + // Our value on the current branch. REPLACE because issue_prefix is seeded at + // bootstrap, so the three-way merge base already carries the key. + if _, err := db.ExecContext(ctx, "REPLACE INTO config (`key`, value) VALUES (?, ?)", key, "ours"); err != nil { + t.Fatalf("insert local config: %v", err) + } + commitConfig("local config") + + // Their value on a branch forked from the common ancestor (HEAD~1). + remoteBranch := currentBranch + "_vcmergestrategy" + if _, err := db.ExecContext(ctx, "CALL DOLT_BRANCH(?, 'HEAD~1')", remoteBranch); err != nil { + t.Fatalf("create remote branch: %v", err) + } + defer func() { + db.ExecContext(ctx, "CALL DOLT_CHECKOUT(?)", currentBranch) + db.ExecContext(ctx, "CALL DOLT_BRANCH('-D', ?)", remoteBranch) + }() + if _, err := db.ExecContext(ctx, "CALL DOLT_CHECKOUT(?)", remoteBranch); err != nil { + t.Fatalf("checkout remote branch: %v", err) + } + if _, err := db.ExecContext(ctx, "REPLACE INTO config (`key`, value) VALUES (?, ?)", key, "theirs"); err != nil { + t.Fatalf("insert remote config: %v", err) + } + commitConfig("remote config") + if _, err := db.ExecContext(ctx, "CALL DOLT_CHECKOUT(?)", currentBranch); err != nil { + t.Fatalf("checkout current branch: %v", err) + } + + // Merge in a conflict-tolerant tx and commit the tx so the conflicted working + // set persists (autocommit rolls a conflicted merge back). This is the state + // the resolution sequence inherits. + func() { + tx, err := db.BeginTx(ctx, nil) + if err != nil { + t.Fatalf("begin merge tx: %v", err) + } + defer func() { _ = tx.Rollback() }() + if _, err := tx.ExecContext(ctx, "SET @@dolt_allow_commit_conflicts = 1"); err != nil { + t.Fatalf("set dolt_allow_commit_conflicts: %v", err) + } + if _, err := tx.ExecContext(ctx, "SET @@dolt_force_transaction_commit = 1"); err != nil { + t.Fatalf("set dolt_force_transaction_commit: %v", err) + } + if _, err := tx.ExecContext(ctx, "CALL DOLT_MERGE(?)", remoteBranch); err != nil { + t.Fatalf("merge: %v", err) + } + var n int + if err := tx.QueryRowContext(ctx, + "SELECT COUNT(*) FROM dolt_conflicts WHERE `table` = 'config'").Scan(&n); err != nil { + t.Fatalf("query config conflicts: %v", err) + } + if n == 0 { + t.Fatalf("merge produced no config conflict on %q; cannot exercise the --strategy path", key) + } + if err := tx.Commit(); err != nil { + t.Fatalf("commit conflicted working set: %v", err) + } + }() + + // The exact resolution sequence cmd/bd/vc.go runs for --strategy theirs. + if err := store.ResolveConflicts(ctx, "config", "theirs"); err != nil { + t.Fatalf("ResolveConflicts: %v", err) + } + if err := store.CommitMergeResolution(ctx, "Resolve merge conflicts using theirs strategy"); err != nil { + t.Fatalf("CommitMergeResolution: %v", err) + } + + if configDirty(t, ctx, db) { + t.Fatalf("config left dirty after bd vc merge --strategy resolution of %q; the merge is unconcluded and re-wedges the next pull (GH#2474)", key) + } + var value string + if err := db.QueryRowContext(ctx, "SELECT value FROM config WHERE `key` = ?", key).Scan(&value); err != nil { + t.Fatalf("read resolved config value: %v", err) + } + if value != "theirs" { + t.Errorf("resolved config value = %q, want \"theirs\" (--strategy theirs)", value) + } +} diff --git a/internal/storage/dolt/store.go b/internal/storage/dolt/store.go index ea24a5ef2..a023952d8 100644 --- a/internal/storage/dolt/store.go +++ b/internal/storage/dolt/store.go @@ -43,6 +43,7 @@ import ( "github.com/steveyegge/beads/internal/storage" "github.com/steveyegge/beads/internal/storage/doltutil" "github.com/steveyegge/beads/internal/storage/issueops" + "github.com/steveyegge/beads/internal/storage/kvkeys" "github.com/steveyegge/beads/internal/storage/schema" "github.com/steveyegge/beads/internal/storage/versioncontrolops" "github.com/steveyegge/beads/internal/types" @@ -1702,6 +1703,30 @@ func (s *DoltStore) commitAuthorString() string { return fmt.Sprintf("%s <%s>", s.committerName, s.committerEmail) } +// configCommitMode controls how commitWorkingSet treats the config table, which +// holds both internal keys (issue_prefix) and synced user data (kv.* keys, +// including kv.memory.* persistent memories). +type configCommitMode int + +const ( + // configExclude skips config entirely (GH#2455): a plain Commit must not + // sweep a concurrent writer's half-applied issue_prefix change into an + // unrelated commit. + configExclude configCommitMode = iota + // configIncludeUserKVOnly stages config for the pre-pull auto-commit, but + // only when every dirty config row is this clone's own user KV data (the + // kv.* namespace, which includes kv.memory.* memories). Any other dirty + // config key — an internal key such as issue_prefix above all — aborts the + // commit with operator guidance so the pull never auto-commits unsafe + // config (GH#2455 + GH#2474). + configIncludeUserKVOnly + // configIncludeAll stages every dirty config row. Used only to conclude a + // merge whose conflicts the operator resolved explicitly (bd federation + // sync --strategy): that resolution is intentional, so a resolved + // issue_prefix (or any config row) must be committed, not dropped. + configIncludeAll +) + // Commit creates a Dolt commit with the given message. // // GH#2455: Stages all dirty tables EXCEPT config, then commits with '-m'. @@ -1711,7 +1736,59 @@ func (s *DoltStore) commitAuthorString() string { // // Callers that intentionally modify config (e.g., CommitPending after // 'bd config set') must call CommitWithConfig instead. -func (s *DoltStore) Commit(ctx context.Context, message string) (retErr error) { +func (s *DoltStore) Commit(ctx context.Context, message string) error { + return s.commitWorkingSet(ctx, message, configExclude) +} + +// commitBeforePull commits the working set ahead of a pull's merge, INCLUDING +// config. The pre-pull auto-commit (GH#2474) must include config because user +// KV data lives there as kv.* rows (persistent memories are the kv.memory.* +// subset) and Commit() deliberately skips config (GH#2455): without this those +// rows sit permanently uncommitted, so the "clean the working set before +// merging" step leaves config dirty and DOLT_MERGE refuses to start ("cannot +// merge with uncommitted changes"). +// +// It includes ONLY this clone's own user kv.* rows: if any other config key is +// dirty (an internal key such as issue_prefix above all) it refuses rather than +// auto-committing it, so the stale-config corruption GH#2455 guards against is +// never re-opened by a pull. Auto-*resolution* of a config conflict stays +// narrower still — only convergent kv.memory.* keys (see +// configConflictsAreMemoryConvergent) — so widening the commit screen to the +// whole kv. namespace cannot auto-resolve a genuine kv.* conflict; it only stops +// generic `bd kv set` writes from wedging the pull. Config is staged explicitly +// (via DOLT_ADD in commitWorkingSet) rather than through CommitWithConfig's +// DOLT_COMMIT('-Am'), which was observed not to stage config reliably under the +// server-mode stored-procedure path. Committing this clone's own kv.* rows as the +// merge basis is the same explicit, user-initiated action CommitPending ('bd dolt +// commit') already performs, so it does not widen the concurrent-writer race +// GH#2455 guards against. +func (s *DoltStore) commitBeforePull(ctx context.Context, message string) error { + return s.commitWorkingSet(ctx, message, configIncludeUserKVOnly) +} + +// CommitMergeResolution concludes a merge whose conflicts were resolved by an +// explicit operator strategy (bd federation sync --strategy / bd vc merge +// --strategy ours|theirs), committing the resolved working set INCLUDING config. +// Plain Commit excludes config (GH#2455), so a config-only resolution — exactly +// the case this change makes routine by syncing kv.* through config — would be +// silently dropped, leaving the merge unconcluded and re-wedging the next +// pull/sync. Unlike commitBeforePull it does not screen config keys: the operator +// chose this resolution, so whichever config rows it touched (issue_prefix +// included) are committed as-is. It satisfies storage.VersionControl so cmd/bd +// concludes bd vc merge --strategy through the same config-inclusive commit +// instead of the config-excluding Commit that would drop the resolution. +func (s *DoltStore) CommitMergeResolution(ctx context.Context, message string) error { + return s.commitWorkingSet(ctx, message, configIncludeAll) +} + +// commitWorkingSet stages the dirty tables reported by dolt_status and commits +// them with '-m'. The config table is staged according to mode: configExclude +// skips it (GH#2455) so a concurrent writer's half-applied issue_prefix change +// is never swept into an unrelated commit; configIncludeUserKVOnly stages it for +// the pre-pull path but refuses when any non-kv. (internal) config key is dirty; +// configIncludeAll stages every dirty config row to conclude an explicit merge +// resolution. +func (s *DoltStore) commitWorkingSet(ctx context.Context, message string, mode configCommitMode) (retErr error) { ctx, span := doltTracer.Start(ctx, "dolt.commit", trace.WithSpanKind(trace.SpanKindClient), trace.WithAttributes(s.doltSpanAttrs()...), @@ -1725,36 +1802,57 @@ func (s *DoltStore) Commit(ctx context.Context, message string) (retErr error) { } defer conn.Close() - // GH#2455: Stage all dirty tables EXCEPT config. Query dolt_status for - // dirty tables and stage each one individually, skipping config to avoid - // sweeping up stale issue_prefix changes from concurrent operations. + // GH#2455: stage each dirty table individually, skipping config unless the + // mode opts it in, to avoid sweeping up stale issue_prefix changes from + // concurrent operations. Query dolt_status first. rows, err := conn.QueryContext(ctx, "SELECT table_name FROM dolt_status") if err != nil { // If dolt_status fails, fall back to nothing (rare edge case). return fmt.Errorf("failed to query dolt_status: %w", err) } var tables []string + configDirty := false for rows.Next() { var table string if err := rows.Scan(&table); err != nil { _ = rows.Close() return fmt.Errorf("failed to scan dolt_status: %w", err) } - if table != "config" { - tables = append(tables, table) + if table == "config" { + configDirty = true + if mode == configExclude { + continue + } } + tables = append(tables, table) } _ = rows.Close() if err := rows.Err(); err != nil { return fmt.Errorf("failed to iterate dolt_status: %w", err) } + // GH#2455 + GH#2474: the pre-pull auto-commit includes config so user kv.* + // writes sync, but it must NOT auto-commit any internal (non-kv.) config key. + // Refuse before staging anything so the merge is never concluded over an + // unsafe config row; the operator commits those explicitly. + if configDirty && mode == configIncludeUserKVOnly { + if err := s.assertDirtyConfigUserKVOnly(ctx, conn); err != nil { + return err + } + } + if len(tables) == 0 { return nil // Nothing to commit (all changes were config-only or dolt_ignore'd) } for _, table := range tables { if _, err := conn.ExecContext(ctx, "CALL DOLT_ADD(?)", table); err != nil { + // config when the mode intentionally includes it is the whole reason + // we stage here: silently skipping a failed DOLT_ADD('config') would + // leave config dirty and re-wedge the merge, so surface it instead. + if table == "config" && mode != configExclude { + return fmt.Errorf("failed to stage config before commit: %w", err) + } // Best effort: some tables may be dolt_ignore'd (e.g., wisps). // DOLT_ADD fails for ignored tables; skip silently. continue @@ -1773,6 +1871,48 @@ func (s *DoltStore) Commit(ctx context.Context, message string) (retErr error) { return nil } +// assertDirtyConfigUserKVOnly returns an error unless every config row dirty in +// the working set is this clone's own user KV data (the kv.* namespace, which +// includes kv.memory.* memories). The pre-pull auto-commit opts config into the +// staged set so user KV writes sync and stop wedging DOLT_MERGE (GH#2474), but +// auto-committing an unrelated dirty internal config key such as issue_prefix +// would re-open the GH#2455 stale-config corruption — that is the operator's +// explicit `bd dolt commit` to make, not the pull's. Screening on the whole kv. +// namespace (not just kv.memory.*) un-wedges generic `bd kv set` writes too: a +// kv.* row is this clone's own data, exactly as safe to auto-commit as a memory, +// and a genuine kv.* merge conflict is still left for the operator because +// auto-resolution stays kv.memory.*-only (configConflictsAreMemoryConvergent). +// config's primary key is `key`, so dolt_diff exposes to_key/from_key; an add or +// delete leaves one side NULL, so COALESCE picks whichever key the change carries. +func (s *DoltStore) assertDirtyConfigUserKVOnly(ctx context.Context, conn *sql.Conn) error { + rows, err := conn.QueryContext(ctx, + "SELECT COALESCE(to_key, from_key) FROM dolt_diff('HEAD', 'WORKING', 'config')") + if err != nil { + return fmt.Errorf("inspect dirty config before pull: %w", err) + } + defer rows.Close() + + var unsafe []string + for rows.Next() { + var key sql.NullString + if err := rows.Scan(&key); err != nil { + return fmt.Errorf("scan dirty config key: %w", err) + } + if key.Valid && !strings.HasPrefix(key.String, kvkeys.Prefix) { + unsafe = append(unsafe, key.String) + } + } + if err := rows.Err(); err != nil { + return fmt.Errorf("iterate dirty config diff: %w", err) + } + if len(unsafe) > 0 { + return fmt.Errorf("refusing to auto-commit %d dirty internal config key(s) before pull: %s; "+ + "only user %s* keys auto-commit before a pull (GH#2455) — commit or revert "+ + "these explicitly with `bd dolt commit` first", len(unsafe), strings.Join(unsafe, ", "), kvkeys.Prefix) + } + return nil +} + // CommitWithConfig creates a Dolt commit that includes the config table. // Use this instead of Commit when the caller intentionally modified config // (e.g., CommitPending after 'bd config set', 'bd init', or 'bd rename-prefix'). @@ -2308,7 +2448,7 @@ func (s *DoltStore) pullFromRemote(ctx context.Context, remote string) (retErr e // (schema init, molecule loading, metadata writes) can dirty the working // set before the user's pull command runs. if !s.readOnly { - if err := s.Commit(ctx, "auto-commit before pull"); err != nil { + if err := s.commitBeforePull(ctx, "auto-commit before pull"); err != nil { // "nothing to commit" is fine — working set is already clean if !isDoltNothingToCommit(err) { return fmt.Errorf("failed to commit pending changes before pull: %w", err) @@ -2705,7 +2845,8 @@ func (s *DoltStore) autoResolveConflictsAfterCLIPull(ctx context.Context) (bool, // tryAutoResolveMergeConflicts auto-resolves merge conflicts that are safe to // resolve without operator input (GH#2466 metadata, #4259 audit-only -// dependency edges, bd-6dnrw.29 schema_migrations vintage rows), returning +// dependency edges, bd-6dnrw.29 schema_migrations vintage rows, GH#2474 +// convergent kv.memory.* config rows), returning // (true, nil) only if ALL conflicts were resolved. The implementation is // shared with the embedded pull path (bd-6dnrw.40); see // versioncontrolops.TryAutoResolveMergeConflicts for the full contract. diff --git a/internal/storage/embeddeddolt/federation.go b/internal/storage/embeddeddolt/federation.go index 386538387..caf82a067 100644 --- a/internal/storage/embeddeddolt/federation.go +++ b/internal/storage/embeddeddolt/federation.go @@ -201,6 +201,17 @@ func (s *EmbeddedDoltStore) Sync(ctx context.Context, peer string, strategy stri StartTime: time.Now(), } + // GH#2474 / bd-578h9.2: commit pending changes before the merge, matching + // embedded Pull/PullRemote/PullFrom and server-mode Sync. Embedded Commit is + // DOLT_COMMIT('-Am'), so it stages config — where kv.memory.* memories live — + // and a leftover dirty working set (e.g. a `bd remember` write) would + // otherwise make DOLT_MERGE refuse to start ("cannot merge with uncommitted + // changes"). CommitPending is a no-op when the working set is already clean. + if _, err := s.CommitPending(ctx, "beads"); err != nil { + result.Error = fmt.Errorf("commit pending before sync: %w", err) + return result, result.Error + } + // Step 1: Fetch if err := s.Fetch(ctx, peer); err != nil { result.Error = fmt.Errorf("fetch failed: %w", err) diff --git a/internal/storage/embeddeddolt/pull_config_wedge_test.go b/internal/storage/embeddeddolt/pull_config_wedge_test.go new file mode 100644 index 000000000..db8589864 --- /dev/null +++ b/internal/storage/embeddeddolt/pull_config_wedge_test.go @@ -0,0 +1,231 @@ +//go:build cgo + +package embeddeddolt_test + +import ( + "context" + "database/sql" + "errors" + "testing" + + "github.com/steveyegge/beads/internal/storage" + "github.com/steveyegge/beads/internal/storage/versioncontrolops" +) + +// configCommittedClean asserts the config table has no uncommitted changes — +// i.e. the preceding commit actually staged config. This is the embedded-engine +// check for the pull config wedge: persistent memories live in config as +// kv.memory.* rows, and a commit that fails to stage config leaves the working +// set permanently dirty so DOLT_MERGE refuses to start. +func configCommittedClean(t *testing.T, ctx context.Context, conn *sql.Conn) { + t.Helper() + var n int + if err := conn.QueryRowContext(ctx, + "SELECT COUNT(*) FROM dolt_status WHERE table_name = 'config'").Scan(&n); err != nil { + t.Fatalf("query dolt_status: %v", err) + } + if n > 0 { + t.Fatalf("config still dirty after commit: the embedded commit path does not stage config, so pulls will wedge") + } +} + +// TestEmbeddedConfigCommitStagesConfig documents the embedded engine's commit +// behavior for config: the embedded store commits via DOLT_COMMIT('-Am') (see +// EmbeddedDoltStore.Commit), and the pre-pull auto-commit relies on that to +// stage memory rows. If this fails, the embedded pre-pull path needs the same +// explicit-config-staging fix the server-mode path got. +func TestEmbeddedConfigCommitStagesConfig(t *testing.T) { + te := newTestEnv(t, "cfgstage") + ctx := t.Context() + conn := openSettleConn(t, ctx, te) + + if _, err := conn.ExecContext(ctx, + "INSERT INTO config (`key`, value) VALUES ('kv.memory.stage-check', 'v1')"); err != nil { + t.Fatalf("insert config memory row: %v", err) + } + if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'commit memory')"); err != nil { + t.Fatalf("commit: %v", err) + } + configCommittedClean(t, ctx, conn) +} + +// TestEmbeddedMergeAndSettleMemoryConfigConflict is the embedded-engine +// counterpart of TestPullAutoResolveMemoryConfigConflicts: a merge conflict +// limited to kv.memory.* config rows is auto-resolved with "theirs", so two +// clones that edited the same memory converge instead of wedging the pull. +func TestEmbeddedMergeAndSettleMemoryConfigConflict(t *testing.T) { + te := newTestEnv(t, "cfgsettle") + ctx := t.Context() + conn := openSettleConn(t, ctx, te) + + if _, err := conn.ExecContext(ctx, + "INSERT INTO config (`key`, value) VALUES ('kv.memory.shared', 'ours')"); err != nil { + t.Fatalf("insert config on main: %v", err) + } + if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'local memory')"); err != nil { + t.Fatalf("commit on main: %v", err) + } + // Guard the setup: if -Am did not stage config, no conflict would form and + // the test would pass vacuously. + configCommittedClean(t, ctx, conn) + + if _, err := conn.ExecContext(ctx, "CALL DOLT_BRANCH('cfgpeer', 'HEAD~1')"); err != nil { + t.Fatalf("create peer branch: %v", err) + } + if _, err := conn.ExecContext(ctx, "CALL DOLT_CHECKOUT('cfgpeer')"); err != nil { + t.Fatalf("checkout peer: %v", err) + } + if _, err := conn.ExecContext(ctx, + "INSERT INTO config (`key`, value) VALUES ('kv.memory.shared', 'theirs')"); err != nil { + t.Fatalf("insert config on peer: %v", err) + } + if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'peer memory')"); err != nil { + t.Fatalf("commit on peer: %v", err) + } + if _, err := conn.ExecContext(ctx, "CALL DOLT_CHECKOUT('main')"); err != nil { + t.Fatalf("checkout main: %v", err) + } + + if err := versioncontrolops.MergeAndSettle(ctx, conn, "cfgpeer"); err != nil { + t.Fatalf("settling memory-config-conflicted merge: %v", err) + } + + var value string + if err := conn.QueryRowContext(ctx, + "SELECT value FROM config WHERE `key` = 'kv.memory.shared'").Scan(&value); err != nil { + t.Fatalf("read resolved config: %v", err) + } + if value != "theirs" { + t.Errorf("resolved memory value = %q, want \"theirs\" (--theirs convergent)", value) + } +} + +// TestEmbeddedMergeAndSettleSkipsNonMemoryConfigConflict verifies the prefix +// boundary: a config key under kv. but NOT kv.memory. is a real semantic +// conflict, so the resolver declines the whole config table and the merge is +// surfaced to the operator as a MergeConflictsError naming config. +func TestEmbeddedMergeAndSettleSkipsNonMemoryConfigConflict(t *testing.T) { + te := newTestEnv(t, "cfgskip") + ctx := t.Context() + conn := openSettleConn(t, ctx, te) + + if _, err := conn.ExecContext(ctx, + "INSERT INTO config (`key`, value) VALUES ('kv.custom.setting', 'ours')"); err != nil { + t.Fatalf("insert config on main: %v", err) + } + if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'local config')"); err != nil { + t.Fatalf("commit on main: %v", err) + } + configCommittedClean(t, ctx, conn) + + if _, err := conn.ExecContext(ctx, "CALL DOLT_BRANCH('cfgskippeer', 'HEAD~1')"); err != nil { + t.Fatalf("create peer branch: %v", err) + } + if _, err := conn.ExecContext(ctx, "CALL DOLT_CHECKOUT('cfgskippeer')"); err != nil { + t.Fatalf("checkout peer: %v", err) + } + if _, err := conn.ExecContext(ctx, + "INSERT INTO config (`key`, value) VALUES ('kv.custom.setting', 'theirs')"); err != nil { + t.Fatalf("insert config on peer: %v", err) + } + if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'peer config')"); err != nil { + t.Fatalf("commit on peer: %v", err) + } + if _, err := conn.ExecContext(ctx, "CALL DOLT_CHECKOUT('main')"); err != nil { + t.Fatalf("checkout main: %v", err) + } + + err := versioncontrolops.MergeAndSettle(ctx, conn, "cfgskippeer") + var mce *versioncontrolops.MergeConflictsError + if !errors.As(err, &mce) { + t.Fatalf("want MergeConflictsError for non-memory config conflict, got: %v", err) + } + foundConfig := false + for _, c := range mce.Conflicts { + if c.Field == "config" { + foundConfig = true + } + } + if !foundConfig { + t.Errorf("captured conflicts %+v do not name the config table", mce.Conflicts) + } + // The local value must survive the aborted merge. + var value string + if err := conn.QueryRowContext(ctx, + "SELECT value FROM config WHERE `key` = 'kv.custom.setting'").Scan(&value); err != nil { + t.Fatalf("read config after abort: %v", err) + } + if value != "ours" { + t.Errorf("local config value = %q after aborted merge, want %q", value, "ours") + } +} + +// statusHasConfig reports whether the config table has staged or unstaged +// working-set changes, observed through the store's own connection — the same +// session Sync commits on. A separate OpenSQL handle is avoided on purpose: +// embedded uncommitted working-set writes are not reliably visible across +// handles until shutdown-flush, so the dirty precondition must be read here. +func statusHasConfig(t *testing.T, ctx context.Context, te *testEnv) bool { + t.Helper() + st, err := te.store.Status(ctx) + if err != nil { + t.Fatalf("Status: %v", err) + } + for _, e := range st.Staged { + if e.Table == "config" { + return true + } + } + for _, e := range st.Unstaged { + if e.Table == "config" { + return true + } + } + return false +} + +// TestEmbeddedSyncCommitsConfigBeforeMerge is the embedded-engine counterpart of +// server-mode TestFederationSyncCommitsConfigBeforeFetch: EmbeddedDoltStore.Sync +// must auto-commit pending changes before its merge, exactly as embedded +// Pull/PullRemote/PullFrom already do. Persistent memories live in config as +// kv.memory.* rows that `bd remember` leaves uncommitted; without the pre-merge +// CommitPending, a dirty config working set survives into DOLT_MERGE and wedges +// `bd federation sync` ("cannot merge with uncommitted changes"). The peer remote +// does not exist, so Sync fails at fetch — but only after the pre-merge commit has +// already staged config, which is exactly what this test pins. +func TestEmbeddedSyncCommitsConfigBeforeMerge(t *testing.T) { + te := newTestEnv(t, "syncwedge") + ctx := t.Context() + + // Register a peer whose file remote does not exist: Sync runs its pre-merge + // auto-commit, then fails at fetch — far enough to prove config was staged. + peer := &storage.FederationPeer{ + Name: "peer-sync-wedge", + RemoteURL: "file:///tmp/beads-no-such-embedded-federation-peer", + Sovereignty: "T2", + } + if err := te.store.AddFederationPeer(ctx, peer); err != nil { + t.Fatalf("add federation peer: %v", err) + } + // Commit the peer registration so the only dirty table going into Sync is the + // config row seeded below — the precise `bd remember` wedge precondition. + if err := te.store.Commit(ctx, "register sync peer"); err != nil { + t.Fatalf("commit peer registration: %v", err) + } + + // Simulate `bd remember`: a kv.memory.* config write, left uncommitted. + if err := te.store.SetConfig(ctx, "kv.memory.sync-wedge", "v1"); err != nil { + t.Fatalf("SetConfig memory row: %v", err) + } + if !statusHasConfig(t, ctx, te) { + t.Fatal("memory write did not dirty config; cannot reproduce the wedge precondition") + } + + if _, err := te.store.Sync(ctx, peer.Name, ""); err == nil { + t.Fatal("expected sync to fail for nonexistent file remote") + } + + if statusHasConfig(t, ctx, te) { + t.Fatal("embedded Sync left config dirty before merge; bd federation sync would wedge DOLT_MERGE (GH#2474)") + } +} diff --git a/internal/storage/embeddeddolt/version_control.go b/internal/storage/embeddeddolt/version_control.go index 140729b74..df842ba44 100644 --- a/internal/storage/embeddeddolt/version_control.go +++ b/internal/storage/embeddeddolt/version_control.go @@ -113,6 +113,15 @@ func (s *EmbeddedDoltStore) CommitWithConfig(ctx context.Context, message string return s.Commit(ctx, message) } +// CommitMergeResolution concludes an operator --strategy merge resolution with +// config included. Embedded Commit already stages everything via DOLT_COMMIT +// ('-Am'), so config is never dropped here the way server-mode Commit drops it +// (GH#2455); this alias satisfies the VersionControl interface so cmd/bd can +// conclude bd vc merge --strategy uniformly across both stores. +func (s *EmbeddedDoltStore) CommitMergeResolution(ctx context.Context, message string) error { + return s.Commit(ctx, message) +} + func (s *EmbeddedDoltStore) AddRemote(ctx context.Context, name, url string) error { return s.withMutatingDBConn(ctx, func(db versioncontrolops.DBConn) error { _, err := db.ExecContext(ctx, "CALL DOLT_REMOTE('add', ?, ?)", name, url) diff --git a/internal/storage/kvkeys/kvkeys.go b/internal/storage/kvkeys/kvkeys.go new file mode 100644 index 000000000..ffdbea697 --- /dev/null +++ b/internal/storage/kvkeys/kvkeys.go @@ -0,0 +1,29 @@ +// Package kvkeys defines the config-table key prefixes that the cmd/bd KV and +// memory commands write and that the storage-layer merge resolver reads, so the +// "kv.memory.* config rows are convergent persistent memories" contract has a +// single source of truth. +// +// cmd/bd is package main and cannot be imported, so before this package the +// prefix lived in three structurally-forced copies (cmd/bd/kv.go, +// cmd/bd/memory.go, and internal/storage/versioncontrolops). A rename of any +// one silently drifted from the others: the merge resolver would stop matching +// real memory keys and the pull/sync config wedge would quietly return for the +// renamed keys with no compile error. Centralizing the prefix here makes that a +// single edit guarded by a contract test. +package kvkeys + +const ( + // Prefix namespaces every user key under the synced config table, keeping + // user data out of internal config keys such as issue_prefix. + Prefix = "kv." + + // MemoryPrefix namespaces persistent `bd remember` memories within Prefix. + MemoryPrefix = "memory." + + // MemoryConfigKeyPrefix is the full config-table key prefix for persistent + // memories (Prefix + MemoryPrefix == "kv.memory."). The merge resolver + // treats a config conflict as a convergent memory conflict only when every + // conflicted key carries this prefix; cmd/bd reserves it so generic + // `bd kv set` keys cannot collide with the `bd remember` namespace. + MemoryConfigKeyPrefix = Prefix + MemoryPrefix +) diff --git a/internal/storage/kvkeys/kvkeys_test.go b/internal/storage/kvkeys/kvkeys_test.go new file mode 100644 index 000000000..218e47cd9 --- /dev/null +++ b/internal/storage/kvkeys/kvkeys_test.go @@ -0,0 +1,19 @@ +package kvkeys + +import "testing" + +// TestMemoryConfigKeyPrefix pins the canonical config-table prefix for +// persistent memories. The merge resolver auto-resolves config conflicts only +// when every conflicted key carries this prefix (GH#2474); if it ever drifts +// from what cmd/bd actually writes, memory conflicts silently fall back to the +// operator and the pull/sync config wedge returns for the renamed keys. This +// test makes such a rename a conscious, caught change rather than a silent one. +func TestMemoryConfigKeyPrefix(t *testing.T) { + if got, want := MemoryConfigKeyPrefix, "kv.memory."; got != want { + t.Fatalf("MemoryConfigKeyPrefix = %q, want %q (renaming it re-wedges memory merge conflicts)", got, want) + } + if MemoryConfigKeyPrefix != Prefix+MemoryPrefix { + t.Fatalf("MemoryConfigKeyPrefix %q must equal Prefix %q + MemoryPrefix %q", + MemoryConfigKeyPrefix, Prefix, MemoryPrefix) + } +} diff --git a/internal/storage/version_control.go b/internal/storage/version_control.go index eaf924785..c2ae6e2cb 100644 --- a/internal/storage/version_control.go +++ b/internal/storage/version_control.go @@ -38,6 +38,13 @@ type VersionControl interface { // Use after intentional config writes (bd init, bd config set, bd rename-prefix). // GH#3216: bootstrap paths must use this to commit issue_prefix. CommitWithConfig(ctx context.Context, message string) error + // CommitMergeResolution concludes a merge whose conflicts the operator + // resolved with an explicit --strategy (bd vc merge / bd federation sync), + // committing the resolved working set INCLUDING config. Plain Commit excludes + // config (GH#2455), so a config-only resolution — routine now that kv.* user + // data syncs through config — would be dropped, leaving the merge unconcluded + // and re-wedging the next pull/sync (GH#2474). + CommitMergeResolution(ctx context.Context, message string) error CommitPending(ctx context.Context, actor string) (bool, error) CommitExists(ctx context.Context, commitHash string) (bool, error) GetCurrentCommit(ctx context.Context) (string, error) diff --git a/internal/storage/versioncontrolops/mergesettle.go b/internal/storage/versioncontrolops/mergesettle.go index 61e7c3a0c..8e2221479 100644 --- a/internal/storage/versioncontrolops/mergesettle.go +++ b/internal/storage/versioncontrolops/mergesettle.go @@ -10,14 +10,25 @@ import ( "strings" "github.com/steveyegge/beads/internal/storage" + "github.com/steveyegge/beads/internal/storage/kvkeys" ) +// memoryConfigKeyPrefix is the config-table key prefix under which `bd remember` +// stores persistent memories. It is sourced from the shared kvkeys package so it +// can never drift from the prefix cmd/bd actually writes (kvkeys.Prefix "kv." + +// kvkeys.MemoryPrefix "memory."), which cmd/bd also reserves against generic +// `bd kv set` keys. Config rows with this prefix are the only config class safe +// to auto-resolve on merge; any other key (issue_prefix above all) is left for +// the operator. +const memoryConfigKeyPrefix = kvkeys.MemoryConfigKeyPrefix + // This file holds the merge-settlement machinery shared by server-mode // DoltStore (which drives it inside an explicit *sql.Tx) and the embedded // store's pull path (which drives it on a pinned autocommit connection via // MergeAndSettle): auto-resolving the conflict classes that are safe without // operator input (GH#2466 metadata, #4259 audit-only dependency edges, -// bd-6dnrw.29 schema_migrations vintage rows) and repairing FK cascade +// bd-6dnrw.29 schema_migrations vintage rows, GH#2474 convergent kv.memory.* +// config rows) and repairing FK cascade // violations (bd-6dnrw.4). All functions take a DBConn, which *sql.Tx, // *sql.Conn, and *sql.DB all satisfy. @@ -184,7 +195,7 @@ func workingSetClean(ctx context.Context, db DBConn) bool { // TryAutoResolveMergeConflicts auto-resolves merge conflicts that are safe to // resolve without operator input, and returns (true, nil) only if ALL conflicts -// were resolved. It handles three classes: +// were resolved. It handles four classes: // // - metadata: machine-local rows (e.g. dolt_auto_push_*) that routinely diverge // across clones (GH#2466). Resolved with "theirs". @@ -204,10 +215,16 @@ func workingSetClean(ctx context.Context, db DBConn) bool { // provenance beats its absence, and the result converges across clones. // Two DIFFERENT non-empty hashes are the #4259 schema fork itself and are // left for the operator (bd doctor reports them as Migration Content Skew). +// - config: persistent memories live in config as kv.memory.* rows (the +// pre-pull auto-commit now commits config so they sync). Like metadata, +// same-key memory edits across clones are machine-convergent: resolved with +// "theirs", so all clones pulling from one remote converge on the remote's +// value. A conflict touching ANY non-memory config key (issue_prefix above +// all) is a real semantic conflict and is left for the operator. // -// Any conflict on another table, or an unresolvable dependencies or -// schema_migrations conflict, returns (false, nil) so the caller fails the pull -// and the operator resolves it. +// Any conflict on another table, or an unresolvable dependencies, +// schema_migrations, or config conflict, returns (false, nil) so the caller +// fails the pull and the operator resolves it. // // The resolved tables are staged but NOT committed: the caller must run // CommitResolvedConflicts after the FK cascade repair, because DOLT_COMMIT @@ -265,6 +282,15 @@ func TryAutoResolveMergeConflicts(ctx context.Context, db DBConn) (bool, error) return false, nil } resolvable = append(resolvable, "schema_migrations") + case "config": + memoryOnly, err := configConflictsAreMemoryConvergent(ctx, db) + if err != nil { + return false, err + } + if !memoryOnly { + return false, nil + } + resolvable = append(resolvable, "config") default: return false, nil } @@ -273,13 +299,29 @@ func TryAutoResolveMergeConflicts(ctx context.Context, db DBConn) (bool, error) // Resolve each safe table and stage only that table (GH#2455). // table is from the fixed allowlist above, never user input. for _, table := range resolvable { - if table == "schema_migrations" { + switch table { + case "schema_migrations": // Row-wise: keep whichever side recorded a content hash, so the // table-level --ours/--theirs choice can never drop one. if err := resolveSchemaMigrationsVintageConflicts(ctx, db); err != nil { return false, err } - } else { + case "config": + // --theirs makes this clone's local kv.memory.* edit lose to the + // remote value (the same convergent trade-off metadata makes). That + // supersession is otherwise undiagnosable, so name the resolved keys + // first. Best-effort: a diagnostics query failure must not abort an + // otherwise-correct resolution. + if keys, kerr := resolvedConfigConflictKeys(ctx, db); kerr == nil && len(keys) > 0 { + fmt.Fprintf(os.Stderr, + "Notice: auto-resolved %d memory config conflict(s) with the remote value (--theirs); "+ + "local edits to %s were superseded\n", + len(keys), strings.Join(keys, ", ")) + } + if _, err := db.ExecContext(ctx, "CALL DOLT_CONFLICTS_RESOLVE('--theirs', 'config')"); err != nil { + return false, fmt.Errorf("failed to resolve config conflicts: %w", err) + } + default: //nolint:gosec // G201: table is one of the hardcoded constants above. if _, err := db.ExecContext(ctx, "CALL DOLT_CONFLICTS_RESOLVE('--theirs', '"+table+"')"); err != nil { return false, fmt.Errorf("failed to resolve %s conflicts: %w", table, err) @@ -302,7 +344,7 @@ func TryAutoResolveMergeConflicts(ctx context.Context, db DBConn) (bool, error) // cascade violation could never settle while the resolver committed first // (bd-578h9.14). func CommitResolvedConflicts(ctx context.Context, db DBConn) error { - if _, err := db.ExecContext(ctx, "CALL DOLT_COMMIT('-m', 'auto-resolve merge conflicts (GH#2466, #4259)')"); err != nil { + if _, err := db.ExecContext(ctx, "CALL DOLT_COMMIT('-m', 'auto-resolve merge conflicts (GH#2466, #4259, GH#2474)')"); err != nil { return fmt.Errorf("failed to commit resolved conflicts: %w", err) } return nil @@ -372,6 +414,68 @@ func dependencyConflictsAreAuditOnly(ctx context.Context, db DBConn) (bool, erro return true, rows.Err() } +// configConflictsAreMemoryConvergent reports whether every conflicted config +// row is a persistent-memory row (key prefixed memoryConfigKeyPrefix). Memories +// are the only config class safe to auto-resolve with --theirs: like metadata, +// all clones pulling from the same remote converge on the remote's value (a +// local edit to the same memory key loses, the same convergent trade-off +// metadata makes). Any other config key in conflict — issue_prefix above all, +// whose stale-value sweep GH#2455 specifically guards against — is a real +// semantic conflict, so the whole config table is left for the operator. +// +// The key column is config's primary key, so a same-key conflict carries the +// identical key on both sides; an add/delete conflict leaves one side NULL. A +// row is convergent only if every key it presents is a memory key. +func configConflictsAreMemoryConvergent(ctx context.Context, db DBConn) (bool, error) { + rows, err := db.QueryContext(ctx, ` + SELECT our_key, their_key FROM dolt_conflicts_config`) + if err != nil { + return false, fmt.Errorf("query config conflicts: %w", err) + } + defer rows.Close() + + for rows.Next() { + var ourKey, theirKey sql.NullString + if err := rows.Scan(&ourKey, &theirKey); err != nil { + return false, fmt.Errorf("scan config conflict: %w", err) + } + for _, k := range []sql.NullString{ourKey, theirKey} { + if k.Valid && !strings.HasPrefix(k.String, memoryConfigKeyPrefix) { + return false, nil + } + } + } + return true, rows.Err() +} + +// resolvedConfigConflictKeys returns the keys of the config rows currently in +// conflict, used only to name the kv.memory.* keys whose local value the +// --theirs auto-resolution is about to supersede. It must be called BEFORE +// DOLT_CONFLICTS_RESOLVE clears dolt_conflicts_config. config's primary key is +// `key`, so a same-key conflict carries the identical key on both sides; an +// add/delete conflict leaves one side NULL, so COALESCE picks whichever side has +// it. +func resolvedConfigConflictKeys(ctx context.Context, db DBConn) ([]string, error) { + rows, err := db.QueryContext(ctx, + "SELECT COALESCE(our_key, their_key) FROM dolt_conflicts_config") + if err != nil { + return nil, err + } + defer rows.Close() + + var keys []string + for rows.Next() { + var key sql.NullString + if err := rows.Scan(&key); err != nil { + return nil, err + } + if key.Valid { + keys = append(keys, key.String) + } + } + return keys, rows.Err() +} + // schemaMigrationsConflictsAreVintageOnly reports whether every conflicted // schema_migrations row is the same migration version present on BOTH sides // whose content hashes are compatible: equal, or NULL/empty on exactly one side