beads/PROPOSAL-pull-config-wedge.md
Steve Yegge 387959695d
fix(dolt): pre-pull commit must include config (kv.memory.* pull-wedge) (#4412)
* fix(dolt): pre-pull commit must include config; auto-resolve convergent kv.memory.* config conflicts

bd remember stores persistent memories as kv.memory.* rows in the synced
config table. Commit() deliberately excludes config (GH#2455, to avoid
sweeping a concurrent writer's stale issue_prefix into an unrelated commit),
so the GH#2474 pre-pull auto-commit at pullFromRemote (store.go) and PullFrom
(federation.go) leaves config permanently dirty once any memory exists. The
"clean the working set before merging" step then leaves config dirty and
DOLT_MERGE refuses to start ("cannot merge with uncommitted changes"). Two
correct PRs colliding over config. Diagnosed from a live server-mode repro
(Wyvern wy-71t); see PROPOSAL-pull-config-wedge.md.

Fix (server mode): add commitBeforePull, which stages config explicitly via
DOLT_ADD rather than relying on CommitWithConfig's DOLT_COMMIT('-Am'). The
proposal author observed -Am not staging config reliably under the server-mode
stored-procedure path, and the new embedded test TestEmbeddedConfigCommitStagesConfig
confirms -Am DOES stage config in the embedded engine — so the unreliability is
server-mode-specific and explicit staging is the robust choice. Commit() and
commitBeforePull now share commitWorkingSet(includeConfig bool).

Hardening (both modes): making config a committed/synced table means two clones
editing the same memory key collide on a config merge conflict. Extend
TryAutoResolveMergeConflicts to auto-resolve config conflicts with --theirs ONLY
when every conflicted row is a kv.memory.* key (machine-convergent, same policy
as metadata); any other config key (issue_prefix above all) is left for the
operator. Without this the primary fix would trade a silent pull wedge for an
operator-visible conflict wedge on the exact bd remember workload.

Embedded pre-pull (CommitPending -> -Am) already stages config and is unchanged;
it gains the resolver hardening.

Tests: server-mode pull_config_wedge_test.go (commit-includes-config + resolver
positive/negative, run under Docker in CI) and embedded pull_config_wedge_test.go
(in-process: -Am-stages-config probe + resolver positive/negative, all green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: add pull-config-wedge diagnosis/proposal (PR #4412)

The diagnosis-and-patch write-up behind this PR: how GH#2455 (Commit excludes
config) and GH#2474 (pre-pull auto-commit) collide over the synced config table,
leaving kv.memory.* rows permanently uncommitted so DOLT_MERGE refuses to start.
Includes the live-DB validation (Wyvern wy-71t) and the rejected alternatives.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(dolt): adopt-pr maintainer fixes for pull/sync config wedge

Maintainer review fixups for PR #4412, squashed into one commit on top of
the contributor's branch.

Pre-merge config wedge (sync): bd federation sync committed with the
config-excluding Commit before its merge while this PR switched the pull
paths to commitBeforePull, so any DB that ran `bd remember` carried dirty
kv.memory.* config rows and hit the "cannot merge with uncommitted changes"
wedge. Sync now uses commitBeforePull too, with a regression test.

Pre-pull auto-commit safety (GH#2455): commitBeforePull no longer stages the
whole config table. It auto-commits only convergent memory rows
(kv.memory.*); if any other config key is dirty (issue_prefix above all) it
refuses with operator guidance instead of sweeping a stale value into the
merge basis, restoring the GH#2455 protection the broad pre-pull staging had
reopened.

Sync --strategy config commit: a config-only conflict resolution committed
through the config-excluding Commit, so it could report success without
creating the merge-resolution commit and re-wedge the next sync. Sync now
commits resolutions through a config-inclusive path (commitMergeResolution).

Embedded sync parity: EmbeddedDoltStore.Sync fetched and merged without the
pre-merge CommitPending guard already used by embedded Pull/PullRemote/PullFrom
and the new server Sync, so the same dirty kv.memory.* working set could still
wedge `bd federation sync` in embedded mode. Sync now commits pending changes
before fetch/merge (embedded Commit is DOLT_COMMIT('-Am'), which stages
config), with an embedded regression test that drives Sync with a dirty
kv.memory.* row.

KV/memory namespace collision: `bd kv set memory.*` stored to kv.memory.*,
indistinguishable from a `bd remember` memory the merge resolver auto-resolves
with --theirs. validateKVKey now reserves the memory. prefix so a generic KV
value can't be silently overridden by the remote on pull.

Shared prefix + tests: centralize the kv.memory. prefix in
internal/storage/kvkeys (used by cmd/bd and the merge resolver) with a
contract test; route the remaining cmd/bd kv. prefix literals (the doctor KV
check, the validateKVKey nested-prefix guard, and the memory listing) through
the kvkeys constants; and strengthen the server-mode non-memory config
conflict test to prove a conflict materialized before the resolver declines it.

Maintainer review fixup for PR #4412.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <ci@beads.test>
2026-06-19 21:31:33 -07:00

8.9 KiB

Proposal: fix bd dolt pull "cannot merge with uncommitted changes" wedge

Status: for review (diagnosis + patch). Authored 2026-06-14 from a live repro in the Wyvern beads DB (server mode, git-protocol remote on Bitbucket).

Symptom

bd dolt pull always fails:

Error 1105 (HY000): cannot merge with uncommitted changes

bd doctor reports Dolt Locks: config: modified immediately after every command, even right after bd dolt commit, and even with --dolt-auto-commit on. Tracked as Wyvern wy-71t. Push is unaffected; only pull (the cross-clone receive path) is wedged.

Root cause — two correct PRs collide over the config table

  1. Memories live in config. bd remember stores each persistent memory as a kv.memory.<slug> row in the synced config table. Confirmed via the working set on the live DB — the only dirty rows were exactly the 5 memories:

    SELECT to_key, diff_type FROM dolt_diff('main','WORKING','config');
    kv.memory.beads-jsonl-sync-procedure                       added
    kv.memory.bitbucket-altssh-dead                            added
    kv.memory.local-dev-ssl-posture                            added
    kv.memory.react-client-e2e-against-the-local-game-server   added
    kv.memory.react-e2e-same-character-parallel-logins         added
    
  2. Commit deliberately excludes config (GH#2455, store.go ~1731): DOLT_COMMIT('-Am', …) once swept stale issue_prefix changes from concurrent operations into commits, so Commit now stages every dirty table except config.

  3. The pre-pull auto-commit uses Commit (GH#2474). Both pull paths try to clean the working set before merging, but call s.Commit, which skips config:

    • internal/storage/dolt/store.go:2309 — pullFromRemote (default remote; what bd dolt pull invokes via st.Pull)
    • internal/storage/dolt/federation.go:65 — PullFrom (named-peer pull)

Net effect: once any memory exists, the kv.memory.* rows sit permanently uncommitted (nothing ever commits config), so the pre-pull "clean the working set" step leaves config dirty, and DOLT_MERGE refuses to start — the exact case abortMerge's comment already calls out: "a merge that REFUSED TO START on a dirty working set." GH#2474's intent is silently defeated by GH#2455 whenever config is the dirty table, which is always after the first bd remember.

This is independent of the v42→v51 schema-migration fork (#4259) seen in the same DB; that was a separate, newer problem. Migrating + force-publishing fixed the fork and restored push, but pull stays wedged because of this config issue.

Empirical validation (live DB)

Confirmed the mechanism end-to-end without any source change:

  1. The only dirty table was config; the only dirty rows were the 5 kv.memory.* entries (see query above).
  2. Explicitly committing config — CALL DOLT_ADD('config'); CALL DOLT_COMMIT('-m', …) — left the working set clean (dolt_diff('HEAD','WORKING','config') → 0 rows).
  3. Running ordinary commands afterward (bd list, bd show) kept it clean (still 0). So bd's per-command memory re-sync is idempotent: once the rows match HEAD, re-writing identical values produces no diff. The "re-dirties after every command" symptom existed only because the rows were never committed (Commit skips config), so they showed as permanently uncommitted. Committing config once breaks the cycle.

This proves a clean config working set is the only thing the merge needs, and that a pre-merge commit of config is a permanent fix, not a per-command band-aid.

Confirmed: after committing config, bd dolt pull succeeds ("Pull complete.") repeatably, bd dolt push still works, and bd doctor no longer reports config: modified. The wedge is entirely the uncommitted-config condition; the patch just makes the pull path establish it automatically. (Without the patch the wedge returns on the next bd remember, which adds a fresh uncommitted kv.memory.* row.)

One wrinkle to verify in the patch: a standalone bd dolt commit (which already routes through CommitWithConfig → DOLT_COMMIT('-Am', …)) did not visibly clean config in this server-mode DB, whereas the explicit DOLT_ADD('config') + DOLT_COMMIT('-m', …) did. That suggests -Am may not be staging config reliably under the SQL stored-procedure path here. The patch author should confirm CommitWithConfig actually commits config in server mode; if not, the pre-pull step should stage config explicitly (CALL DOLT_ADD('config') then commit) rather than rely on -Am. A regression test should assert dolt_status is empty immediately after the pre-pull commit.

Fix (primary) — pre-pull commit must include config

CommitWithConfig already exists (store.go:1780) for exactly this "commit everything, including intentional config" case — CommitPending (bd dolt commit) already uses it. Use it in the two pre-pull sites:

--- a/internal/storage/dolt/store.go
+++ b/internal/storage/dolt/store.go
@@ pullFromRemote (GH#2474 pre-pull auto-commit)
-        if err := s.Commit(ctx, "auto-commit before pull"); err != nil {
+        // Include config: persistent memories live in config as kv.memory.* rows,
+        // and Commit() excludes config (GH#2455), so they never commit and leave
+        // the working set permanently dirty — DOLT_MERGE then refuses to start.
+        if err := s.CommitWithConfig(ctx, "auto-commit before pull"); err != nil {
             if !isDoltNothingToCommit(err) {
                 return fmt.Errorf("failed to commit pending changes before pull: %w", err)
             }
         }
--- a/internal/storage/dolt/federation.go
+++ b/internal/storage/dolt/federation.go
@@ PullFrom (GH#2474 pre-pull auto-commit)
-        if err := s.Commit(ctx, "auto-commit before pull"); err != nil {
+        if err := s.CommitWithConfig(ctx, "auto-commit before pull"); err != nil {
             if !isDoltNothingToCommit(err) {
                 return nil, fmt.Errorf("failed to commit pending changes before pull: %w", err)
             }
         }

Why this does not reintroduce GH#2455

GH#2455 guards against -Am sweeping a concurrent writer's half-applied issue_prefix change into an unrelated commit. The pre-pull commit is different in kind: it is an explicit user action (bd dolt pull) that requires a clean working set, and it is committing this clone's own config state as the basis for the merge — exactly what CommitPending already does for bd dolt commit. The race window GH#2455 worried about (another bd process mutating issue_prefix at the same instant) is not meaningfully widened by committing config here versus letting the user's next bd dolt commit do it.

Fix (optional hardening) — auto-resolve convergent config merge conflicts

With the primary fix, config (incl. kv.memory.*) becomes a committed, synced table, so two clones that edit the same memory will produce a config merge conflict. TryAutoResolveMergeConflicts (mergesettle.go) currently handles only metadata, dependencies, and schema_migrations — not config — so such a conflict would stop the pull for the operator.

Recommend a follow-up that teaches the resolver to auto-resolve config conflicts on machine-convergent keys (e.g. kv.memory.*) with --theirs, the same convergent policy already used for the metadata table. Out of scope for the wedge fix itself; the primary change resolves the reported bug. (Until then, divergent same-memory edits are a rare, operator-visible case rather than a silent wedge.)

Alternative considered — move memories to a dolt_ignore'd table

Rejected: memories are meant to sync (they round-trip through bd export | bd import, and are part of cross-clone shared state). dolt_ignore (as used for wisps) would make them machine-local and stop syncing them via Dolt — a behavior change, not a fix.

Test plan

Regression test (alongside pull_conflict_test.go / federation_pull_settle_test.go):

  1. Two clones of a tiny remote-backed DB.
  2. On clone A: bd remember --key t "x" (dirties config with a kv.memory.* row, committing nothing via the normal path).
  3. On clone B: create+push an issue.
  4. On clone A: Pull — must succeed (today it returns cannot merge with uncommitted changes), and clone A must end with B's issue and its own memory intact.

Manual repro on the Wyvern DB: a fresh bd dolt pull fails today purely on the 5 kv.memory.* rows; with the patch the pre-pull CommitWithConfig commits them and the merge proceeds.

Files

  • internal/storage/dolt/store.go:2309 (pullFromRemote) — primary
  • internal/storage/dolt/federation.go:65 (PullFrom) — primary
  • internal/storage/dolt/store.go:1780 (CommitWithConfig) — reused, unchanged
  • internal/storage/versioncontrolops/mergesettle.go (TryAutoResolveMergeConflicts) — optional hardening