beads/internal/storage/embeddeddolt/pull_config_wedge_test.go
Steve Yegge 387959695d
fix(dolt): pre-pull commit must include config (kv.memory.* pull-wedge) (#4412)
* fix(dolt): pre-pull commit must include config; auto-resolve convergent kv.memory.* config conflicts

bd remember stores persistent memories as kv.memory.* rows in the synced
config table. Commit() deliberately excludes config (GH#2455, to avoid
sweeping a concurrent writer's stale issue_prefix into an unrelated commit),
so the GH#2474 pre-pull auto-commit at pullFromRemote (store.go) and PullFrom
(federation.go) leaves config permanently dirty once any memory exists. The
"clean the working set before merging" step then leaves config dirty and
DOLT_MERGE refuses to start ("cannot merge with uncommitted changes"). Two
correct PRs colliding over config. Diagnosed from a live server-mode repro
(Wyvern wy-71t); see PROPOSAL-pull-config-wedge.md.

Fix (server mode): add commitBeforePull, which stages config explicitly via
DOLT_ADD rather than relying on CommitWithConfig's DOLT_COMMIT('-Am'). The
proposal author observed -Am not staging config reliably under the server-mode
stored-procedure path, and the new embedded test TestEmbeddedConfigCommitStagesConfig
confirms -Am DOES stage config in the embedded engine — so the unreliability is
server-mode-specific and explicit staging is the robust choice. Commit() and
commitBeforePull now share commitWorkingSet(includeConfig bool).

Hardening (both modes): making config a committed/synced table means two clones
editing the same memory key collide on a config merge conflict. Extend
TryAutoResolveMergeConflicts to auto-resolve config conflicts with --theirs ONLY
when every conflicted row is a kv.memory.* key (machine-convergent, same policy
as metadata); any other config key (issue_prefix above all) is left for the
operator. Without this the primary fix would trade a silent pull wedge for an
operator-visible conflict wedge on the exact bd remember workload.

Embedded pre-pull (CommitPending -> -Am) already stages config and is unchanged;
it gains the resolver hardening.

Tests: server-mode pull_config_wedge_test.go (commit-includes-config + resolver
positive/negative, run under Docker in CI) and embedded pull_config_wedge_test.go
(in-process: -Am-stages-config probe + resolver positive/negative, all green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: add pull-config-wedge diagnosis/proposal (PR #4412)

The diagnosis-and-patch write-up behind this PR: how GH#2455 (Commit excludes
config) and GH#2474 (pre-pull auto-commit) collide over the synced config table,
leaving kv.memory.* rows permanently uncommitted so DOLT_MERGE refuses to start.
Includes the live-DB validation (Wyvern wy-71t) and the rejected alternatives.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(dolt): adopt-pr maintainer fixes for pull/sync config wedge

Maintainer review fixups for PR #4412, squashed into one commit on top of
the contributor's branch.

Pre-merge config wedge (sync): bd federation sync committed with the
config-excluding Commit before its merge while this PR switched the pull
paths to commitBeforePull, so any DB that ran `bd remember` carried dirty
kv.memory.* config rows and hit the "cannot merge with uncommitted changes"
wedge. Sync now uses commitBeforePull too, with a regression test.

Pre-pull auto-commit safety (GH#2455): commitBeforePull no longer stages the
whole config table. It auto-commits only convergent memory rows
(kv.memory.*); if any other config key is dirty (issue_prefix above all) it
refuses with operator guidance instead of sweeping a stale value into the
merge basis, restoring the GH#2455 protection the broad pre-pull staging had
reopened.

Sync --strategy config commit: a config-only conflict resolution committed
through the config-excluding Commit, so it could report success without
creating the merge-resolution commit and re-wedge the next sync. Sync now
commits resolutions through a config-inclusive path (commitMergeResolution).

Embedded sync parity: EmbeddedDoltStore.Sync fetched and merged without the
pre-merge CommitPending guard already used by embedded Pull/PullRemote/PullFrom
and the new server Sync, so the same dirty kv.memory.* working set could still
wedge `bd federation sync` in embedded mode. Sync now commits pending changes
before fetch/merge (embedded Commit is DOLT_COMMIT('-Am'), which stages
config), with an embedded regression test that drives Sync with a dirty
kv.memory.* row.

KV/memory namespace collision: `bd kv set memory.*` stored to kv.memory.*,
indistinguishable from a `bd remember` memory the merge resolver auto-resolves
with --theirs. validateKVKey now reserves the memory. prefix so a generic KV
value can't be silently overridden by the remote on pull.

Shared prefix + tests: centralize the kv.memory. prefix in
internal/storage/kvkeys (used by cmd/bd and the merge resolver) with a
contract test; route the remaining cmd/bd kv. prefix literals (the doctor KV
check, the validateKVKey nested-prefix guard, and the memory listing) through
the kvkeys constants; and strengthen the server-mode non-memory config
conflict test to prove a conflict materialized before the resolver declines it.

Maintainer review fixup for PR #4412.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <ci@beads.test>
2026-06-19 21:31:33 -07:00

231 lines
8.9 KiB
Go

//go:build cgo
package embeddeddolt_test
import (
"context"
"database/sql"
"errors"
"testing"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/versioncontrolops"
)
// configCommittedClean asserts the config table has no uncommitted changes —
// i.e. the preceding commit actually staged config. This is the embedded-engine
// check for the pull config wedge: persistent memories live in config as
// kv.memory.* rows, and a commit that fails to stage config leaves the working
// set permanently dirty so DOLT_MERGE refuses to start.
func configCommittedClean(t *testing.T, ctx context.Context, conn *sql.Conn) {
t.Helper()
var n int
if err := conn.QueryRowContext(ctx,
"SELECT COUNT(*) FROM dolt_status WHERE table_name = 'config'").Scan(&n); err != nil {
t.Fatalf("query dolt_status: %v", err)
}
if n > 0 {
t.Fatalf("config still dirty after commit: the embedded commit path does not stage config, so pulls will wedge")
}
}
// TestEmbeddedConfigCommitStagesConfig documents the embedded engine's commit
// behavior for config: the embedded store commits via DOLT_COMMIT('-Am') (see
// EmbeddedDoltStore.Commit), and the pre-pull auto-commit relies on that to
// stage memory rows. If this fails, the embedded pre-pull path needs the same
// explicit-config-staging fix the server-mode path got.
func TestEmbeddedConfigCommitStagesConfig(t *testing.T) {
te := newTestEnv(t, "cfgstage")
ctx := t.Context()
conn := openSettleConn(t, ctx, te)
if _, err := conn.ExecContext(ctx,
"INSERT INTO config (`key`, value) VALUES ('kv.memory.stage-check', 'v1')"); err != nil {
t.Fatalf("insert config memory row: %v", err)
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'commit memory')"); err != nil {
t.Fatalf("commit: %v", err)
}
configCommittedClean(t, ctx, conn)
}
// TestEmbeddedMergeAndSettleMemoryConfigConflict is the embedded-engine
// counterpart of TestPullAutoResolveMemoryConfigConflicts: a merge conflict
// limited to kv.memory.* config rows is auto-resolved with "theirs", so two
// clones that edited the same memory converge instead of wedging the pull.
func TestEmbeddedMergeAndSettleMemoryConfigConflict(t *testing.T) {
te := newTestEnv(t, "cfgsettle")
ctx := t.Context()
conn := openSettleConn(t, ctx, te)
if _, err := conn.ExecContext(ctx,
"INSERT INTO config (`key`, value) VALUES ('kv.memory.shared', 'ours')"); err != nil {
t.Fatalf("insert config on main: %v", err)
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'local memory')"); err != nil {
t.Fatalf("commit on main: %v", err)
}
// Guard the setup: if -Am did not stage config, no conflict would form and
// the test would pass vacuously.
configCommittedClean(t, ctx, conn)
if _, err := conn.ExecContext(ctx, "CALL DOLT_BRANCH('cfgpeer', 'HEAD~1')"); err != nil {
t.Fatalf("create peer branch: %v", err)
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_CHECKOUT('cfgpeer')"); err != nil {
t.Fatalf("checkout peer: %v", err)
}
if _, err := conn.ExecContext(ctx,
"INSERT INTO config (`key`, value) VALUES ('kv.memory.shared', 'theirs')"); err != nil {
t.Fatalf("insert config on peer: %v", err)
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'peer memory')"); err != nil {
t.Fatalf("commit on peer: %v", err)
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_CHECKOUT('main')"); err != nil {
t.Fatalf("checkout main: %v", err)
}
if err := versioncontrolops.MergeAndSettle(ctx, conn, "cfgpeer"); err != nil {
t.Fatalf("settling memory-config-conflicted merge: %v", err)
}
var value string
if err := conn.QueryRowContext(ctx,
"SELECT value FROM config WHERE `key` = 'kv.memory.shared'").Scan(&value); err != nil {
t.Fatalf("read resolved config: %v", err)
}
if value != "theirs" {
t.Errorf("resolved memory value = %q, want \"theirs\" (--theirs convergent)", value)
}
}
// TestEmbeddedMergeAndSettleSkipsNonMemoryConfigConflict verifies the prefix
// boundary: a config key under kv. but NOT kv.memory. is a real semantic
// conflict, so the resolver declines the whole config table and the merge is
// surfaced to the operator as a MergeConflictsError naming config.
func TestEmbeddedMergeAndSettleSkipsNonMemoryConfigConflict(t *testing.T) {
te := newTestEnv(t, "cfgskip")
ctx := t.Context()
conn := openSettleConn(t, ctx, te)
if _, err := conn.ExecContext(ctx,
"INSERT INTO config (`key`, value) VALUES ('kv.custom.setting', 'ours')"); err != nil {
t.Fatalf("insert config on main: %v", err)
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'local config')"); err != nil {
t.Fatalf("commit on main: %v", err)
}
configCommittedClean(t, ctx, conn)
if _, err := conn.ExecContext(ctx, "CALL DOLT_BRANCH('cfgskippeer', 'HEAD~1')"); err != nil {
t.Fatalf("create peer branch: %v", err)
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_CHECKOUT('cfgskippeer')"); err != nil {
t.Fatalf("checkout peer: %v", err)
}
if _, err := conn.ExecContext(ctx,
"INSERT INTO config (`key`, value) VALUES ('kv.custom.setting', 'theirs')"); err != nil {
t.Fatalf("insert config on peer: %v", err)
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'peer config')"); err != nil {
t.Fatalf("commit on peer: %v", err)
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_CHECKOUT('main')"); err != nil {
t.Fatalf("checkout main: %v", err)
}
err := versioncontrolops.MergeAndSettle(ctx, conn, "cfgskippeer")
var mce *versioncontrolops.MergeConflictsError
if !errors.As(err, &mce) {
t.Fatalf("want MergeConflictsError for non-memory config conflict, got: %v", err)
}
foundConfig := false
for _, c := range mce.Conflicts {
if c.Field == "config" {
foundConfig = true
}
}
if !foundConfig {
t.Errorf("captured conflicts %+v do not name the config table", mce.Conflicts)
}
// The local value must survive the aborted merge.
var value string
if err := conn.QueryRowContext(ctx,
"SELECT value FROM config WHERE `key` = 'kv.custom.setting'").Scan(&value); err != nil {
t.Fatalf("read config after abort: %v", err)
}
if value != "ours" {
t.Errorf("local config value = %q after aborted merge, want %q", value, "ours")
}
}
// statusHasConfig reports whether the config table has staged or unstaged
// working-set changes, observed through the store's own connection — the same
// session Sync commits on. A separate OpenSQL handle is avoided on purpose:
// embedded uncommitted working-set writes are not reliably visible across
// handles until shutdown-flush, so the dirty precondition must be read here.
func statusHasConfig(t *testing.T, ctx context.Context, te *testEnv) bool {
t.Helper()
st, err := te.store.Status(ctx)
if err != nil {
t.Fatalf("Status: %v", err)
}
for _, e := range st.Staged {
if e.Table == "config" {
return true
}
}
for _, e := range st.Unstaged {
if e.Table == "config" {
return true
}
}
return false
}
// TestEmbeddedSyncCommitsConfigBeforeMerge is the embedded-engine counterpart of
// server-mode TestFederationSyncCommitsConfigBeforeFetch: EmbeddedDoltStore.Sync
// must auto-commit pending changes before its merge, exactly as embedded
// Pull/PullRemote/PullFrom already do. Persistent memories live in config as
// kv.memory.* rows that `bd remember` leaves uncommitted; without the pre-merge
// CommitPending, a dirty config working set survives into DOLT_MERGE and wedges
// `bd federation sync` ("cannot merge with uncommitted changes"). The peer remote
// does not exist, so Sync fails at fetch — but only after the pre-merge commit has
// already staged config, which is exactly what this test pins.
func TestEmbeddedSyncCommitsConfigBeforeMerge(t *testing.T) {
te := newTestEnv(t, "syncwedge")
ctx := t.Context()
// Register a peer whose file remote does not exist: Sync runs its pre-merge
// auto-commit, then fails at fetch — far enough to prove config was staged.
peer := &storage.FederationPeer{
Name: "peer-sync-wedge",
RemoteURL: "file:///tmp/beads-no-such-embedded-federation-peer",
Sovereignty: "T2",
}
if err := te.store.AddFederationPeer(ctx, peer); err != nil {
t.Fatalf("add federation peer: %v", err)
}
// Commit the peer registration so the only dirty table going into Sync is the
// config row seeded below — the precise `bd remember` wedge precondition.
if err := te.store.Commit(ctx, "register sync peer"); err != nil {
t.Fatalf("commit peer registration: %v", err)
}
// Simulate `bd remember`: a kv.memory.* config write, left uncommitted.
if err := te.store.SetConfig(ctx, "kv.memory.sync-wedge", "v1"); err != nil {
t.Fatalf("SetConfig memory row: %v", err)
}
if !statusHasConfig(t, ctx, te) {
t.Fatal("memory write did not dirty config; cannot reproduce the wedge precondition")
}
if _, err := te.store.Sync(ctx, peer.Name, ""); err == nil {
t.Fatal("expected sync to fail for nonexistent file remote")
}
if statusHasConfig(t, ctx, te) {
t.Fatal("embedded Sync left config dirty before merge; bd federation sync would wedge DOLT_MERGE (GH#2474)")
}
}