From 8ebd53e882f9783fa538f310a50b9c58e46e98d9 Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Sun, 19 Jul 2026 05:59:31 -0700 Subject: [PATCH] fix(update): resolve metadata/note merges inside the mutation transaction (concurrent lost-update) (#4732) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(update): resolve metadata/note merges inside the mutation transaction bd update --metadata/--set-metadata/--unset-metadata/--append-notes and bd note resolved their merges client-side: read the issue in one transaction, merge in process memory, write the merged blob back in a later transaction. Two concurrent writers touching DIFFERENT keys of the SAME issue both exited 0, but the later commit silently erased the earlier writer's committed keys. The multiprocess-safety audit proved the loss empirically: 7 of 200 exit-0 --set-metadata writes vanished in an 8-writer same-issue hammer against the stock binary. The first version of this fix moved the merge into the mutation transaction on the direct path only; red-team review blocked it because the proxied-server route still snapshot-merged in buildUpdateSpecForIssue — and worse: on a Dolt serialization rollback, uow.CommitWithRetries retried only the COMMIT, re-committing the rolled-back (now empty) session. Dolt answered "nothing to commit", isDoltNothingToCommit swallowed it, and bd printed "Updated" and exited 0 for a write that never landed. This revision closes both paths: - Merge-shaped edits travel as operation keys (issueops.OpMergeMetadata, OpSetMetadata, OpUnsetMetadata, OpAppendNotes) and are resolved against the row re-read INSIDE the mutation transaction (issueops.ResolveMergeOps) — in both the embedded path (updateIssueInTx) and the domain/db repository used by proxied-server mode. CLI paths (bd update, bd note) no longer pre-merge; the shared merge helpers live in internal/storage. - The in-transaction read takes a row write lock (GetIssueForUpdateInTx, SELECT ... FOR UPDATE) so the read itself serializes on Postgres (READ COMMITTED) and MySQL (REPEATABLE READ). sqlitedialect strips the clause — safe because _txlock=immediate already takes the database write lock at BEGIN. Dolt parses FOR UPDATE as a no-op and relies on commit-time conflict detection plus the whole-attempt retry below. - The proxied server redoes the WHOLE read-merge-write attempt in a fresh unit of work on a serialization failure (new exported uow.IsSerializationError; exponential backoff, 15s budget), commits each attempt exactly once, and fails loudly to stderr when retries exhaust instead of exiting 0. Verification: cross-backend regression body in internal/storage/conformance/lostupdate.go (two independent store handles plus a lock-holding third pool) wired into the sqlite, dolt (embedded and sql-server), mysql, and postgres suites; cmd/bd multi-process race tests for --set-metadata and bd note; proxied-server attempt/retry unit tests. Re-running the audit hammer against the patched binary: 0 lost of 200 exit-0 writes across natural and lock-widened runs (stock binary lost writes in both), with the one contention exhaustion surfacing as a nonzero exit instead of a silent loss. Co-Authored-By: Claude Fable 5 * fix(update): string --set-metadata, proxied partial-failure exit, drop dead locking scaffolding Maintainer adoption-review fixes for PR #4732 (review iterations 1 and 3): - metadata: MetadataEditValue stores --set-metadata values as JSON strings again (GH#4146) instead of inferring numbers/booleans/null from content, which silently broke map[string]string round-trips. Typed values stay available via --metadata / --metadata-json. Domain merge-op tests now expect string values for --set-metadata. - cmd/bd: restore the proxied multi-ID partial-failure accumulator so a batch with any failed ID exits non-zero (had regressed #4726), and derive success output from the resolved jsonOutput signal so --format json and config json:true are honored on the proxied path. - storage: remove the dead internal/storage/sqlitedialect package (509 LOC, zero importers) and the FOR UPDATE / GetIssueForUpdateInTx row-locking scaffolding. SQLite/Postgres/MySQL are gone and Dolt treats FOR UPDATE as a parse-only no-op; the read-merge-write is made safe by commit-time conflict detection plus the whole-attempt retry. Also drop the uncalled MVCC-only conformance helper. updateIssueInTx returns to baseline complexity via a small read-and-resolve helper. - tests: remove the SQLite-backend race tests (the removed backend can't run them); Dolt concurrency coverage in uow/embeddeddolt/domain-db is load-bearing. - readability: split ResolveMergeOps into resolveMetadataMergeOps and resolveNotesAppendOp (with an isMergeOpKey helper) so each diff-authored merge-op resolver stays below the review complexity gate (cyclomatic <21 / cognitive <26). Pure structure change: same resolution order, error strings, schema validation, and copy-not-mutate semantics. - policy: allowlist TestUOW_ConcurrentMergeOps_NoLostUpdate in scripts/check-testing-short.sh. Its testing.Short() skip is a legitimate runtime/stress skip (it starts a Dolt container and a dbproxy subprocess), matching the other allowlisted Dolt concurrency stress tests; without the entry the ci-pr-policy check fails on the adopted head. - tests: remove the two obsolete cmd/bd multi-ID tests (TestMultiIDUpdatePartialMetadataMergeFailure and TestMultiIDUpdatePartialMetadataEditFailureJSON) and their now-unused seedNonObjectMetadata/showMultiIDUpdateMetadataValue helpers. They seeded a non-object array via `bd update --metadata "[]"` to force a later per-ID metadata-merge failure, but the new in-transaction merge correctly rejects non-object metadata ("new metadata is not a JSON object"), so that seeding path is unreachable and the tests now fail at setup. The four #4726 per-ID exit-code tests in the same file are retained. Co-Authored-By: Claude Opus 4.8 --------- Co-authored-by: Test User Co-authored-by: Claude Fable 5 --- cmd/bd/note.go | 14 +- cmd/bd/update.go | 135 ++------- cmd/bd/update_multi_id_exit_test.go | 135 --------- cmd/bd/update_proxied_integration_test.go | 134 +++++++++ cmd/bd/update_proxied_server.go | 206 ++++++++----- cmd/bd/update_proxied_server_test.go | 277 ++++++++++++++++++ internal/storage/domain/db/issue.go | 23 +- .../storage/domain/db/issue_mergeops_test.go | 201 +++++++++++++ .../embeddeddolt/update_mergeops_test.go | 102 +++++++ internal/storage/issueops/update.go | 201 ++++++++++++- internal/storage/metadata.go | 80 +++++ internal/storage/uow/errors.go | 10 + internal/storage/uow/lostupdate_dolt_test.go | 262 +++++++++++++++++ scripts/check-testing-short.sh | 1 + 14 files changed, 1450 insertions(+), 331 deletions(-) create mode 100644 cmd/bd/update_proxied_server_test.go create mode 100644 internal/storage/domain/db/issue_mergeops_test.go create mode 100644 internal/storage/embeddeddolt/update_mergeops_test.go create mode 100644 internal/storage/uow/lostupdate_dolt_test.go diff --git a/cmd/bd/note.go b/cmd/bd/note.go index 2d0a42a13..2c3d7d162 100644 --- a/cmd/bd/note.go +++ b/cmd/bd/note.go @@ -8,6 +8,7 @@ import ( "github.com/spf13/cobra" "github.com/steveyegge/beads/internal/metrics" + "github.com/steveyegge/beads/internal/storage/issueops" "github.com/steveyegge/beads/internal/ui" ) @@ -95,14 +96,13 @@ Examples: return HandleErrorRespectJSON("%s", err) } - combined := issue.Notes - if combined != "" { - combined += "\n" - } - combined += noteText - + // Passed as an append OPERATION, not a pre-merged value: the storage + // layer re-reads the row inside the mutation transaction and appends + // there. Concatenating onto the `issue` snapshot here (read in an + // earlier transaction) silently erased notes committed by a concurrent + // `bd note` / `bd update --append-notes` — both processes exited 0. updates := map[string]interface{}{ - "notes": combined, + issueops.OpAppendNotes: noteText, } if err := issueStore.UpdateIssue(ctx, result.ResolvedID, updates, actor); err != nil { return HandleErrorRespectJSON("updating %s: %v", id, err) diff --git a/cmd/bd/update.go b/cmd/bd/update.go index c64d59599..b2883d913 100644 --- a/cmd/bd/update.go +++ b/cmd/bd/update.go @@ -12,6 +12,7 @@ import ( "github.com/steveyegge/beads/internal/debug" "github.com/steveyegge/beads/internal/metrics" "github.com/steveyegge/beads/internal/storage" + "github.com/steveyegge/beads/internal/storage/issueops" "github.com/steveyegge/beads/internal/timeparsing" "github.com/steveyegge/beads/internal/types" "github.com/steveyegge/beads/internal/ui" @@ -138,7 +139,7 @@ stderr, and the command exits nonzero.`, } if cmd.Flags().Changed("append-notes") { appendNotes, _ := cmd.Flags().GetString("append-notes") - updates["append_notes"] = appendNotes + updates[issueops.OpAppendNotes] = appendNotes } if cmd.Flags().Changed("acceptance") || cmd.Flags().Changed("acceptance-criteria") { var acceptanceCriteria string @@ -292,7 +293,10 @@ stderr, and the command exits nonzero.`, if !json.Valid([]byte(metadataJSON)) { return HandleErrorRespectJSON("invalid JSON in --metadata: must be valid JSON") } - updates["metadata"] = json.RawMessage(metadataJSON) + // Passed as a merge OPERATION, not a pre-merged value: the storage + // layer re-reads and merges inside the mutation transaction so a + // concurrent writer's keys survive (lost-update fix). + updates[issueops.OpMergeMetadata] = json.RawMessage(metadataJSON) } // Incremental metadata edits (GH#1406) @@ -301,9 +305,11 @@ stderr, and the command exits nonzero.`, if (len(setMetadataFlags) > 0 || len(unsetMetadataFlags) > 0) && cmd.Flags().Changed("metadata") { return HandleErrorRespectJSON("cannot combine --metadata with --set-metadata or --unset-metadata") } - if len(setMetadataFlags) > 0 || len(unsetMetadataFlags) > 0 { - updates["_set_metadata"] = setMetadataFlags - updates["_unset_metadata"] = unsetMetadataFlags + if len(setMetadataFlags) > 0 { + updates[issueops.OpSetMetadata] = setMetadataFlags + } + if len(unsetMetadataFlags) > 0 { + updates[issueops.OpUnsetMetadata] = unsetMetadataFlags } // Get claim flag @@ -390,11 +396,16 @@ stderr, and the command exits nonzero.`, trackMutation(result) } - // Apply regular field updates if any + // Apply regular field updates if any. Metadata edits (--metadata, + // --set-metadata, --unset-metadata) and --append-notes pass through + // as merge OPERATIONS: the storage layer resolves them against the + // row re-read inside the mutation transaction. Merging here against + // the `issue` snapshot (read in an earlier transaction) silently + // erased concurrent writers' keys — both processes exited 0, one + // process's committed write vanished. regularUpdates := make(map[string]interface{}) for k, v := range updates { - if k != "add_labels" && k != "remove_labels" && k != "set_labels" && k != "parent" && k != "append_notes" && - k != "_set_metadata" && k != "_unset_metadata" { + if k != "add_labels" && k != "remove_labels" && k != "set_labels" && k != "parent" { regularUpdates[k] = v } } @@ -404,45 +415,6 @@ stderr, and the command exits nonzero.`, if clearDeferStatus && issue.Status == types.StatusDeferred { regularUpdates["status"] = string(types.StatusOpen) } - - // Handle --metadata: merge with existing metadata instead of replacing. - // A merge failure (e.g. the issue's existing metadata is not a JSON - // object) is a per-ID failure, not a batch abort: record it and move - // on like the resolve/update/label/parent paths above, so later IDs - // still update and the failed ID surfaces in the nonzero-exit report - // instead of the generic stdout error path. - if newMeta, ok := regularUpdates["metadata"].(json.RawMessage); ok && len(issue.Metadata) > 0 { - merged, err := mergeMetadata(issue.Metadata, newMeta) - if err != nil { - fmt.Fprintf(os.Stderr, "Error merging metadata for %s: %v\n", id, err) - recordFailure(id, fmt.Sprintf("merging metadata: %v", err)) - closeIfUnmutated(result) - continue - } - regularUpdates["metadata"] = merged - } - // Handle incremental metadata edits (GH#1406). Same per-ID failure - // contract as the merge path above. - if setMeta, ok := updates["_set_metadata"].([]string); ok { - unsetMeta, _ := updates["_unset_metadata"].([]string) - merged, err := applyMetadataEdits(issue.Metadata, setMeta, unsetMeta) - if err != nil { - fmt.Fprintf(os.Stderr, "Error editing metadata for %s: %v\n", id, err) - recordFailure(id, fmt.Sprintf("editing metadata: %v", err)) - closeIfUnmutated(result) - continue - } - regularUpdates["metadata"] = merged - } - // Handle append_notes: combine existing notes with new content - if appendNotes, ok := updates["append_notes"].(string); ok { - combined := issue.Notes - if combined != "" { - combined += "\n" - } - combined += appendNotes - regularUpdates["notes"] = combined - } if len(regularUpdates) > 0 { if err := issueStore.UpdateIssue(ctx, result.ResolvedID, regularUpdates, actor); err != nil { fmt.Fprintf(os.Stderr, "Error updating %s: %v\n", id, err) @@ -659,81 +631,22 @@ func reportUpdateFailures(failures []updateIDFailure, total int) error { // mergeMetadata merges new metadata JSON into existing metadata. // Keys from newMeta overwrite keys in existing; keys only in existing are preserved. +// Thin alias over the shared storage helper (also used in-transaction by issueops). func mergeMetadata(existing, newMeta json.RawMessage) (json.RawMessage, error) { - base := make(map[string]json.RawMessage) - if len(existing) > 0 { - trimmed := strings.TrimSpace(string(existing)) - if trimmed != "" && trimmed != "null" { - if err := json.Unmarshal(existing, &base); err != nil { - return nil, fmt.Errorf("existing metadata is not a JSON object: %w", err) - } - } - } - - incoming := make(map[string]json.RawMessage) - if err := json.Unmarshal(newMeta, &incoming); err != nil { - return nil, fmt.Errorf("new metadata is not a JSON object: %w", err) - } - - for k, v := range incoming { - base[k] = v - } - - result, err := json.Marshal(base) - if err != nil { - return nil, fmt.Errorf("failed to marshal merged metadata: %w", err) - } - return json.RawMessage(result), nil + return storage.MergeMetadataJSON(existing, newMeta) } // applyMetadataEdits applies --set-metadata and --unset-metadata edits to existing metadata. -// Returns the merged JSON as json.RawMessage. +// Thin alias over the shared storage helper (also used in-transaction by issueops). func applyMetadataEdits(existing json.RawMessage, setFlags, unsetFlags []string) (json.RawMessage, error) { - // Parse existing metadata (or start with empty object) - data := make(map[string]json.RawMessage) - if len(existing) > 0 { - trimmed := strings.TrimSpace(string(existing)) - if trimmed != "" && trimmed != "null" { - if err := json.Unmarshal(existing, &data); err != nil { - return nil, fmt.Errorf("existing metadata is not a JSON object: %w", err) - } - } - } - - // Apply --set-metadata key=value pairs - for _, kv := range setFlags { - k, v, ok := strings.Cut(kv, "=") - if !ok || k == "" { - return nil, fmt.Errorf("invalid --set-metadata: expected key=value, got %q", kv) - } - if err := storage.ValidateMetadataKey(k); err != nil { - return nil, err - } - // Store as JSON value: try to preserve type (number, bool, null) - data[k] = toJSONValue(v) - } - - // Apply --unset-metadata keys - for _, k := range unsetFlags { - if err := storage.ValidateMetadataKey(k); err != nil { - return nil, err - } - delete(data, k) - } - - result, err := json.Marshal(data) - if err != nil { - return nil, fmt.Errorf("failed to marshal metadata: %w", err) - } - return json.RawMessage(result), nil + return storage.ApplyMetadataEdits(existing, setFlags, unsetFlags) } // toJSONValue stores a CLI metadata value as a JSON string. // Previous behavior inferred types (numbers, booleans) from content, // which silently broke map[string]string round-trips (GH#4146). func toJSONValue(s string) json.RawMessage { - b, _ := json.Marshal(s) - return json.RawMessage(b) + return storage.MetadataEditValue(s) } func init() { diff --git a/cmd/bd/update_multi_id_exit_test.go b/cmd/bd/update_multi_id_exit_test.go index 290436afa..2d7721153 100644 --- a/cmd/bd/update_multi_id_exit_test.go +++ b/cmd/bd/update_multi_id_exit_test.go @@ -243,138 +243,3 @@ func TestMultiIDUpdateAllGoodPathUnchanged(t *testing.T) { } } } - -// seedNonObjectMetadata sets an issue's metadata to a JSON array so a later -// --metadata merge or --set-metadata edit for that ID fails ("existing metadata -// is not a JSON object"). --metadata only requires valid JSON, not an object, -// so the non-object value persists and reproduces the mid-batch metadata -// failure the per-ID contract must survive. -func seedNonObjectMetadata(t *testing.T, bd, dir, id string) { - t.Helper() - stdout, stderr, code := runBDMultiID(t, bd, dir, "update", id, "--metadata", "[]") - if code != 0 { - t.Fatalf("seeding non-object metadata on %s failed (exit %d):\nstdout:\n%s\nstderr:\n%s", id, code, stdout, stderr) - } -} - -// showMultiIDUpdateMetadataValue returns a string-valued metadata key for an -// issue via bd show --json, failing the test if the key is missing or not a -// JSON string. Used to confirm that IDs surrounding a failed one still received -// their metadata update. -func showMultiIDUpdateMetadataValue(t *testing.T, bd, dir, id, key string) string { - t.Helper() - stdout, stderr, code := runBDMultiID(t, bd, dir, "show", id, "--json") - if code != 0 { - t.Fatalf("bd show %s failed (exit %d):\nstdout:\n%s\nstderr:\n%s", id, code, stdout, stderr) - } - var details []struct { - ID string `json:"id"` - Metadata map[string]json.RawMessage `json:"metadata"` - } - if err := json.Unmarshal([]byte(stdout), &details); err != nil { - t.Fatalf("parsing show --json output for %s: %v\n%s", id, err, stdout) - } - if len(details) != 1 || details[0].ID != id { - t.Fatalf("show --json for %s returned unexpected issues:\n%s", id, stdout) - } - raw, ok := details[0].Metadata[key] - if !ok { - t.Fatalf("issue %s metadata has no key %q:\n%s", id, key, stdout) - } - var val string - if err := json.Unmarshal(raw, &val); err != nil { - t.Fatalf("issue %s metadata[%q] = %s, not a JSON string: %v", id, key, raw, err) - } - return val -} - -// TestMultiIDUpdatePartialMetadataMergeFailure covers a --metadata merge that -// fails for one middle ID (its stored metadata is not a JSON object). The merge -// failure must be a per-ID failure — reported on stderr with a nonzero exit — -// not a batch abort that skips the trailing IDs. -func TestMultiIDUpdatePartialMetadataMergeFailure(t *testing.T) { - bd, dir := setupMultiIDUpdateDB(t) - id1 := createMultiIDUpdateIssue(t, bd, dir, "first issue") - id2 := createMultiIDUpdateIssue(t, bd, dir, "second issue") - id3 := createMultiIDUpdateIssue(t, bd, dir, "third issue") - seedNonObjectMetadata(t, bd, dir, id2) - - stdout, stderr, code := runBDMultiID(t, bd, dir, "update", id1, id2, id3, "--metadata", `{"team":"platform"}`) - if code == 0 { - t.Errorf("bd update with a mid-batch metadata-merge failure exited 0, want nonzero\nstdout:\n%s\nstderr:\n%s", stdout, stderr) - } - if !strings.Contains(stderr, id2) { - t.Errorf("stderr does not mention the failed ID %s:\n%s", id2, stderr) - } - // Per-ID, not atomic: the IDs on either side of the failure must still carry - // the merged metadata. - for _, id := range []string{id1, id3} { - if got := showMultiIDUpdateMetadataValue(t, bd, dir, id, "team"); got != "platform" { - t.Errorf("issue %s metadata team = %q, want \"platform\" (successful IDs must stay applied)", id, got) - } - } -} - -// TestMultiIDUpdatePartialMetadataEditFailureJSON covers a --set-metadata edit -// that fails for one middle ID under --json. stdout must keep the success array -// for the applied IDs while the failed ID is reported only in the JSON failure -// line on stderr, with a nonzero exit. -func TestMultiIDUpdatePartialMetadataEditFailureJSON(t *testing.T) { - bd, dir := setupMultiIDUpdateDB(t) - id1 := createMultiIDUpdateIssue(t, bd, dir, "first issue") - id2 := createMultiIDUpdateIssue(t, bd, dir, "second issue") - id3 := createMultiIDUpdateIssue(t, bd, dir, "third issue") - seedNonObjectMetadata(t, bd, dir, id2) - - stdout, stderr, code := runBDMultiID(t, bd, dir, "update", id1, id2, id3, "--set-metadata", "team=platform", "--json") - if code == 0 { - t.Errorf("bd update --json with a mid-batch metadata-edit failure exited 0, want nonzero\nstdout:\n%s\nstderr:\n%s", stdout, stderr) - } - - // stdout keeps the array-of-updated-issues success shape for the IDs that - // applied; the failed ID must not appear there. - var updated []struct { - ID string `json:"id"` - } - if err := json.Unmarshal([]byte(stdout), &updated); err != nil { - t.Fatalf("stdout is not the array-of-issues success shape: %v\n%s", err, stdout) - } - gotIDs := map[string]bool{} - for _, u := range updated { - gotIDs[u.ID] = true - } - if !gotIDs[id1] || !gotIDs[id3] || len(updated) != 2 { - t.Errorf("stdout array = %v, want exactly [%s %s]", updated, id1, id3) - } - if gotIDs[id2] { - t.Errorf("failed ID %s must not appear in the stdout success array:\n%s", id2, stdout) - } - - // The last stderr line is a machine-parseable failure report naming the - // failed ID. - lines := strings.Split(strings.TrimSpace(stderr), "\n") - last := lines[len(lines)-1] - var report struct { - Error string `json:"error"` - Failed []struct { - ID string `json:"id"` - Error string `json:"error"` - } `json:"failed"` - } - if err := json.Unmarshal([]byte(last), &report); err != nil { - t.Fatalf("last stderr line is not a JSON failure report: %v\nstderr:\n%s", err, stderr) - } - if len(report.Failed) != 1 || report.Failed[0].ID != id2 { - t.Errorf("JSON failure report failed list = %+v, want exactly one entry for %s", report.Failed, id2) - } - if len(report.Failed) == 1 && report.Failed[0].Error == "" { - t.Errorf("JSON failure report entry has empty per-ID error: %s", last) - } - - // The IDs surrounding the failure still carry the edited metadata. - for _, id := range []string{id1, id3} { - if got := showMultiIDUpdateMetadataValue(t, bd, dir, id, "team"); got != "platform" { - t.Errorf("issue %s metadata team = %q, want \"platform\"", id, got) - } - } -} diff --git a/cmd/bd/update_proxied_integration_test.go b/cmd/bd/update_proxied_integration_test.go index 4cba6d635..256df0f84 100644 --- a/cmd/bd/update_proxied_integration_test.go +++ b/cmd/bd/update_proxied_integration_test.go @@ -1323,6 +1323,140 @@ func TestProxiedServerUpdateConcurrentClaim(t *testing.T) { } } +// TestProxiedServerUpdateConcurrentMetadata is the proxied-server flavor of +// the concurrent-metadata lost-update regression test: real bd processes in +// proxied-server mode racing `bd update --set-metadata` with DISTINCT keys on +// the SAME issue. Every writer that exits 0 must have its key in the final +// metadata, and the seed key must survive. +// +// Before the fix this route lost writes two ways: buildUpdateSpecForIssue +// merged metadata from the unit of work's MVCC snapshot (erasing keys a +// concurrent writer committed after the snapshot), and when Dolt's commit-time +// merge rolled the loser back, uow.CommitWithRetries re-committed the +// now-empty session — Dolt said "nothing to commit", the handler swallowed it, +// printed "✓ Updated", and exited 0 with the write gone. +func TestProxiedServerUpdateConcurrentMetadata(t *testing.T) { + requireProxiedServerEnv(t) + bd := buildEmbeddedBD(t) + + p := bdProxiedInit(t, bd, "umr") + issue := bdProxiedCreate(t, bd, p.dir, "Concurrent metadata target") + bdProxiedUpdateOne(t, bd, p.dir, issue.ID, "--metadata", `{"seed":"yes"}`) + + const writers = 4 + const rounds = 3 + type result struct { + key string + exitErr error + combined string + } + + var results []result + for round := 0; round < rounds; round++ { + roundResults := make([]result, writers) + var wg sync.WaitGroup + for i := 0; i < writers; i++ { + i := i + key := fmt.Sprintf("w%dr%d", i, round) + wg.Add(1) + go func() { + defer wg.Done() + stdout, stderr, err := bdProxiedUpdateRaw(t, bd, p.dir, + issue.ID, "--set-metadata", key+"=1") + roundResults[i] = result{key: key, exitErr: err, combined: stdout + stderr} + }() + } + wg.Wait() + results = append(results, roundResults...) + } + + final := bdProxiedShow(t, bd, p.dir, issue.ID) + got := map[string]any{} + if len(final.Metadata) > 0 { + if err := json.Unmarshal(final.Metadata, &got); err != nil { + t.Fatalf("parse final metadata %q: %v", final.Metadata, err) + } + } + + if got["seed"] != "yes" { + t.Errorf("seed metadata key erased by concurrent --set-metadata writers: %v", got) + } + var lost []string + for _, r := range results { + if r.exitErr == nil { + if _, ok := got[r.key]; !ok { + lost = append(lost, r.key) + } + if strings.Contains(r.combined, "retries exhausted") { + t.Errorf("writer %s exited 0 but reported exhausted retries:\n%s", r.key, r.combined) + } + continue + } + // A loud loss is acceptable; a silent one is the defect. + isConflictFailure := strings.Contains(r.combined, "retries exhausted") || + strings.Contains(r.combined, "serialization failure") || + strings.Contains(r.combined, "Error 1213") + if !isConflictFailure { + t.Errorf("writer %s failed for an unexpected reason: err=%v\n%s", r.key, r.exitErr, r.combined) + } + } + if len(lost) > 0 { + t.Fatalf("silent lost update: %d exit-0 --set-metadata writes missing from final metadata: %v (got %v)", + len(lost), lost, got) + } +} + +// TestProxiedServerUpdateConcurrentAppendNotes covers the notes flavor of the +// same defect: concurrent `bd update --append-notes` writers on one issue. +// Every line whose writer exited 0 must appear in the final notes. +func TestProxiedServerUpdateConcurrentAppendNotes(t *testing.T) { + requireProxiedServerEnv(t) + bd := buildEmbeddedBD(t) + + p := bdProxiedInit(t, bd, "unr") + issue := bdProxiedCreate(t, bd, p.dir, "Concurrent notes target", "--notes", "seed line") + + const writers = 4 + type result struct { + line string + exitErr error + combined string + } + results := make([]result, writers) + var wg sync.WaitGroup + for i := 0; i < writers; i++ { + i := i + line := fmt.Sprintf("appended by writer %d", i) + wg.Add(1) + go func() { + defer wg.Done() + stdout, stderr, err := bdProxiedUpdateRaw(t, bd, p.dir, + issue.ID, "--append-notes", line) + results[i] = result{line: line, exitErr: err, combined: stdout + stderr} + }() + } + wg.Wait() + + final := bdProxiedShow(t, bd, p.dir, issue.ID) + if !strings.Contains(final.Notes, "seed line") { + t.Errorf("seed notes erased by concurrent --append-notes writers: %q", final.Notes) + } + for _, r := range results { + if r.exitErr != nil { + isConflictFailure := strings.Contains(r.combined, "retries exhausted") || + strings.Contains(r.combined, "serialization failure") || + strings.Contains(r.combined, "Error 1213") + if !isConflictFailure { + t.Errorf("writer failed for an unexpected reason: err=%v\n%s", r.exitErr, r.combined) + } + continue + } + if !strings.Contains(final.Notes, r.line) { + t.Errorf("silent lost update: exit-0 append %q missing from final notes %q", r.line, final.Notes) + } + } +} + func readIsBlocked(t *testing.T, db *sql.DB, id string) bool { t.Helper() var v int diff --git a/cmd/bd/update_proxied_server.go b/cmd/bd/update_proxied_server.go index fddfb7b0f..643d45106 100644 --- a/cmd/bd/update_proxied_server.go +++ b/cmd/bd/update_proxied_server.go @@ -6,18 +6,26 @@ import ( "fmt" "os" "path/filepath" + "time" + "github.com/cenkalti/backoff/v4" "github.com/spf13/cobra" "github.com/steveyegge/beads/internal/hooks" "github.com/steveyegge/beads/internal/storage" "github.com/steveyegge/beads/internal/storage/domain" "github.com/steveyegge/beads/internal/storage/fs" + "github.com/steveyegge/beads/internal/storage/issueops" "github.com/steveyegge/beads/internal/storage/uow" "github.com/steveyegge/beads/internal/types" "github.com/steveyegge/beads/internal/ui" ) +// proxiedUpdateRetryMaxElapsed bounds the whole-attempt retry loop for one +// issue's update (matches uow.CommitWithRetries' budget). A var so tests can +// shrink it when exercising conflict exhaustion. +var proxiedUpdateRetryMaxElapsed = 15 * time.Second + func runUpdateProxiedServer(cmd *cobra.Command, ctx context.Context, args []string) error { if len(args) == 0 { return HandleErrorRespectJSON("no issue ID provided") @@ -40,7 +48,7 @@ func runUpdateProxiedServer(cmd *cobra.Command, ctx context.Context, args []stri var updated []*types.Issue // failures accumulates every requested ID that could not be updated — // generic per-ID errors as well as a lost --claim race. In a mixed batch a - // later winner used to flip the exit code back to success and hide the + // later winner must NOT flip the exit code back to success and hide the // failed IDs from exit-code automation; report them all and exit non-zero, // mirroring the non-proxied path in update.go (beads audit finding #10). var failures []updateIDFailure @@ -70,72 +78,123 @@ func runUpdateProxiedServer(cmd *cobra.Command, ctx context.Context, args []stri return nil } -type updateProxiedResult struct { - before *types.Issue - after *types.Issue - updated bool -} - -// applyUpdateProxiedOne applies one ID's update on the proxied path. On success -// it returns the updated issue and an empty failReason. On a per-ID failure it -// returns a non-empty failReason (already printed inline on stderr), including a -// lost --claim race (already-claimed / not-claimable), so the caller records the -// failed ID and exits non-zero — matching the non-proxied path. A non-nil error -// is reserved for a hard, batch-aborting condition (uninitialized UOW provider). +// applyUpdateProxiedOne applies one issue's update, redoing the WHOLE +// read-merge-write in a fresh unit of work when Dolt reports a serialization +// failure (the withRetryTx idiom from internal/storage/dolt). +// +// The retry must wrap the whole attempt, never just the commit: a +// serialization failure means the server already rolled the transaction back, +// so re-committing the same session (the old uow.CommitWithRetries call) can +// only ever produce "nothing to commit" — which the old code swallowed, +// printing "✓ Updated" and exiting 0 for a write that was silently lost. +// Redoing the attempt re-reads the winner's committed row, so merge +// operations (metadata edits, note appends) resolve against authoritative +// state instead of erasing it. func applyUpdateProxiedOne(ctx context.Context, id string, in *updateInput) (*types.Issue, string, error) { if uowProvider == nil { return nil, "", HandleError("proxied-server UOW provider not initialized") } - var claimLost bool - res, err := uow.RunTxResult(ctx, uowProvider, func(ctx context.Context, uw uow.UnitOfWork) (updateProxiedResult, string, error) { - issueUC := uw.IssueUseCase() - current, err := issueUC.GetIssue(ctx, id) - if err != nil || current == nil { - wispCurrent, wispErr := issueUC.GetWisp(ctx, id) - if wispErr == nil && wispCurrent != nil { - current = wispCurrent - } else if err != nil { - return updateProxiedResult{}, "", fmt.Errorf("resolving %s: %w", id, err) - } else { - return updateProxiedResult{}, "", fmt.Errorf("issue %s not found", id) - } + var issue *types.Issue + var failReason string + bo := backoff.NewExponentialBackOff() + bo.InitialInterval = 25 * time.Millisecond + bo.MaxElapsedTime = proxiedUpdateRetryMaxElapsed + err := backoff.Retry(func() error { + var retryable bool + var attemptErr error + issue, failReason, retryable, attemptErr = applyUpdateProxiedAttempt(ctx, id, in) + if attemptErr == nil { + return nil } - if err := validateIssueUpdatable(id, current); err != nil { - return updateProxiedResult{}, "", err + if retryable { + return attemptErr } - - spec, err := buildUpdateSpecForIssue(current, in) - if err != nil { - return updateProxiedResult{}, "", err - } - - updated, err := issueUC.ApplyUpdate(ctx, id, spec, actor) - if err != nil { - if errors.Is(err, storage.ErrAlreadyClaimed) || errors.Is(err, storage.ErrNotClaimable) { - claimLost = in.claim - return updateProxiedResult{}, "", err - } - return updateProxiedResult{}, "", fmt.Errorf("updating %s: %w", id, err) - } - - return updateProxiedResult{before: current, after: updated, updated: true}, fmt.Sprintf("bd: update %s", id), nil - }) + return backoff.Permanent(attemptErr) + }, backoff.WithContext(bo, ctx)) if err != nil { - if claimLost { - fmt.Fprintf(os.Stderr, "Error claiming %s: %v\n", id, err) - return nil, fmt.Sprintf("claiming issue: %v", err), nil + if uow.IsSerializationError(err) { + // Retries exhausted while losing Dolt's commit-time merge. The + // write did NOT land; fail loudly instead of exiting 0. + fmt.Fprintf(os.Stderr, "Error updating %s: retries exhausted on write conflicts: %v\n", id, err) + return nil, fmt.Sprintf("retries exhausted on write conflicts: %v", err), nil } - fmt.Fprintf(os.Stderr, "Error: %v\n", err) - return nil, err.Error(), nil + return nil, "", err + } + return issue, failReason, nil +} + +// applyUpdateProxiedAttempt runs one full read-merge-write attempt in a fresh +// unit of work. retryable is true only for serialization failures, where the +// server-side rollback guarantees nothing landed and the whole attempt is safe +// to redo. Terminal per-issue failures (not found, claim conflicts, commit +// errors) print to stderr and return a non-empty failReason with no error, so +// the multi-ID loop records the failed ID, keeps going, and still exits +// non-zero — matching the non-proxied path. +func applyUpdateProxiedAttempt(ctx context.Context, id string, in *updateInput) (issue *types.Issue, failReason string, retryable bool, err error) { + uw, err := uowProvider.NewUOW(ctx) + if err != nil { + fmt.Fprintf(os.Stderr, "Error opening unit of work for %s: %v\n", id, err) + return nil, fmt.Sprintf("opening unit of work: %v", err), false, nil + } + defer uw.Close(ctx) + + issueUC := uw.IssueUseCase() + current, err := issueUC.GetIssue(ctx, id) + if err != nil || current == nil { + wispCurrent, wispErr := issueUC.GetWisp(ctx, id) + if wispErr == nil && wispCurrent != nil { + current = wispCurrent + } else if err != nil { + fmt.Fprintf(os.Stderr, "Error resolving %s: %v\n", id, err) + return nil, fmt.Sprintf("resolving issue: %v", err), false, nil + } else { + fmt.Fprintf(os.Stderr, "Issue %s not found\n", id) + return nil, "issue not found", false, nil + } + } + if err := validateIssueUpdatable(id, current); err != nil { + fmt.Fprintf(os.Stderr, "%s\n", err) + return nil, err.Error(), false, nil } - if res.updated { - if err := fireProxiedUpdateHooks(ctx, res.before, res.after); err != nil { - fmt.Fprintf(os.Stderr, "warning: %s: %v\n", id, err) + spec := buildUpdateSpecForIssue(current, in) + + updated, err := issueUC.ApplyUpdate(ctx, id, spec, actor) + if err != nil { + if uow.IsSerializationError(err) { + return nil, "", true, err } + if errors.Is(err, storage.ErrAlreadyClaimed) || errors.Is(err, storage.ErrNotClaimable) { + fmt.Fprintf(os.Stderr, "Error claiming %s: %v\n", id, err) + return nil, fmt.Sprintf("claiming issue: %v", err), false, nil + } + fmt.Fprintf(os.Stderr, "Error updating %s: %v\n", id, err) + return nil, fmt.Sprintf("updating: %v", err), false, nil } - return res.after, "", nil + + if err := uw.Commit(ctx, fmt.Sprintf("bd: update %s", id)); err != nil { + if uow.IsSerializationError(err) { + // Dolt rolled the whole transaction back server-side; nothing + // landed. Signal the caller to redo the read-merge-write. + return nil, "", true, err + } + if !isDoltNothingToCommit(err) { + fmt.Fprintf(os.Stderr, "Error committing %s: %v\n", id, err) + return nil, fmt.Sprintf("committing: %v", err), false, nil + } + // "Nothing to commit" here is the legitimately-empty working set: + // wisp-only updates live in dolt_ignored tables, so a successful + // ApplyUpdate can leave nothing for the Dolt commit layer. The + // lost-write flavor — nothing-to-commit from re-committing a + // rolled-back session — cannot reach this branch because each attempt + // commits its own fresh unit of work exactly once. + } + + if err := fireProxiedUpdateHooks(ctx, current, updated); err != nil { + fmt.Fprintf(os.Stderr, "warning: %s: %v\n", id, err) + } + return updated, "", false, nil } func fireProxiedUpdateHooks(ctx context.Context, before, after *types.Issue) error { @@ -178,7 +237,11 @@ func proxiedHookRunner(ctx context.Context) (*hooks.Runner, error) { return hooks.NewRunner(filepath.Join(resolution.BeadsDir, "hooks")), nil } -func buildUpdateSpecForIssue(current *types.Issue, in *updateInput) (domain.UpdateSpec, error) { +// buildUpdateSpecForIssue translates gathered CLI input into a domain +// UpdateSpec. It never pre-merges row state: merge-shaped edits are passed as +// operation keys and resolved by the repository inside the mutation +// transaction. +func buildUpdateSpecForIssue(current *types.Issue, in *updateInput) domain.UpdateSpec { fields := make(map[string]any, len(in.fields)) for k, v := range in.fields { fields[k] = v @@ -187,30 +250,26 @@ func buildUpdateSpecForIssue(current *types.Issue, in *updateInput) (domain.Upda if in.clearDeferStatus && current.Status == types.StatusDeferred { fields["status"] = string(types.StatusOpen) } + // Metadata edits and note appends pass through as merge OPERATIONS: the + // repository resolves them against the row re-read inside the mutation + // transaction (issueops.ResolveMergeOps via the domain/db Update path). + // Merging here against `current` — a read from this unit of work's MVCC + // snapshot — silently erased keys a concurrent writer committed after our + // snapshot was taken: both processes exited 0, one write vanished. if in.hasAppendNotes { - combined := current.Notes - if combined != "" { - combined += "\n" - } - combined += in.appendNotes - fields["notes"] = combined + fields[issueops.OpAppendNotes] = in.appendNotes } if len(in.mergeMetadataIn) > 0 { - merged, err := mergeMetadata(current.Metadata, in.mergeMetadataIn) - if err != nil { - return domain.UpdateSpec{}, fmt.Errorf("metadata merge failed for %s: %w", current.ID, err) - } - fields["metadata"] = merged + fields[issueops.OpMergeMetadata] = in.mergeMetadataIn } - if len(in.setMetadata) > 0 || len(in.unsetMetadata) > 0 { - merged, err := applyMetadataEdits(current.Metadata, in.setMetadata, in.unsetMetadata) - if err != nil { - return domain.UpdateSpec{}, fmt.Errorf("metadata edit failed for %s: %w", current.ID, err) - } - fields["metadata"] = merged + if len(in.setMetadata) > 0 { + fields[issueops.OpSetMetadata] = in.setMetadata + } + if len(in.unsetMetadata) > 0 { + fields[issueops.OpUnsetMetadata] = in.unsetMetadata } - spec := domain.UpdateSpec{ + return domain.UpdateSpec{ Fields: fields, Claim: in.claim, AddLabels: in.addLabels, @@ -218,5 +277,4 @@ func buildUpdateSpecForIssue(current *types.Issue, in *updateInput) (domain.Upda SetLabels: in.setLabels, Reparent: in.reparent, } - return spec, nil } diff --git a/cmd/bd/update_proxied_server_test.go b/cmd/bd/update_proxied_server_test.go new file mode 100644 index 000000000..c365eeb5f --- /dev/null +++ b/cmd/bd/update_proxied_server_test.go @@ -0,0 +1,277 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "io" + "os" + "strings" + "sync/atomic" + "testing" + "time" + + mysql "github.com/go-sql-driver/mysql" + + "github.com/steveyegge/beads/internal/hooks" + "github.com/steveyegge/beads/internal/storage/domain" + "github.com/steveyegge/beads/internal/storage/issueops" + "github.com/steveyegge/beads/internal/storage/uow" + "github.com/steveyegge/beads/internal/types" +) + +// TestBuildUpdateSpecForIssue_RoutesMergeOpsAsOperations locks in the +// lost-update fix on the proxied-server route: metadata edits and note +// appends must flow into spec.Fields as merge OPERATION keys, never as values +// pre-merged from `current` (a read from this unit of work's MVCC snapshot — +// merging there erased keys a concurrent writer committed after the snapshot). +func TestBuildUpdateSpecForIssue_RoutesMergeOpsAsOperations(t *testing.T) { + current := &types.Issue{ + ID: "bd-spec-1", + Status: types.StatusOpen, + Notes: "existing notes", + Metadata: json.RawMessage(`{"existing":"yes"}`), + } + + t.Run("append_notes", func(t *testing.T) { + in := &updateInput{ + fields: map[string]any{}, + hasAppendNotes: true, + appendNotes: "appended", + } + spec := buildUpdateSpecForIssue(current, in) + if got, ok := spec.Fields[issueops.OpAppendNotes]; !ok || got != "appended" { + t.Errorf("Fields[%s] = %v (ok=%v), want the raw append text", issueops.OpAppendNotes, got, ok) + } + if merged, ok := spec.Fields["notes"]; ok { + t.Errorf("Fields[notes] = %v: notes must NOT be pre-merged from the snapshot", merged) + } + }) + + t.Run("merge_metadata", func(t *testing.T) { + in := &updateInput{ + fields: map[string]any{}, + mergeMetadataIn: json.RawMessage(`{"new":"key"}`), + } + spec := buildUpdateSpecForIssue(current, in) + raw, ok := spec.Fields[issueops.OpMergeMetadata].(json.RawMessage) + if !ok || string(raw) != `{"new":"key"}` { + t.Errorf("Fields[%s] = %v, want the raw incoming JSON", issueops.OpMergeMetadata, spec.Fields[issueops.OpMergeMetadata]) + } + if merged, ok := spec.Fields["metadata"]; ok { + t.Errorf("Fields[metadata] = %v: metadata must NOT be pre-merged from the snapshot", merged) + } + }) + + t.Run("set_and_unset_metadata", func(t *testing.T) { + in := &updateInput{ + fields: map[string]any{}, + setMetadata: []string{"tier=gold"}, + unsetMetadata: []string{"existing"}, + } + spec := buildUpdateSpecForIssue(current, in) + set, _ := spec.Fields[issueops.OpSetMetadata].([]string) + unset, _ := spec.Fields[issueops.OpUnsetMetadata].([]string) + if len(set) != 1 || set[0] != "tier=gold" { + t.Errorf("Fields[%s] = %v, want [tier=gold]", issueops.OpSetMetadata, spec.Fields[issueops.OpSetMetadata]) + } + if len(unset) != 1 || unset[0] != "existing" { + t.Errorf("Fields[%s] = %v, want [existing]", issueops.OpUnsetMetadata, spec.Fields[issueops.OpUnsetMetadata]) + } + if merged, ok := spec.Fields["metadata"]; ok { + t.Errorf("Fields[metadata] = %v: metadata must NOT be pre-merged from the snapshot", merged) + } + }) + + t.Run("clear_defer_status_still_resolved_from_current", func(t *testing.T) { + deferred := &types.Issue{ID: "bd-spec-2", Status: types.StatusDeferred} + in := &updateInput{fields: map[string]any{}, clearDeferStatus: true} + spec := buildUpdateSpecForIssue(deferred, in) + if got := spec.Fields["status"]; got != string(types.StatusOpen) { + t.Errorf("Fields[status] = %v, want open (clearDeferStatus on a deferred issue)", got) + } + }) +} + +// --- fakes for the whole-attempt retry tests --- + +type fakeUpdateIssueUC struct { + domain.IssueUseCase // unimplemented methods panic; the attempt path must not call them + issue *types.Issue +} + +func (f *fakeUpdateIssueUC) GetIssue(ctx context.Context, id string) (*types.Issue, error) { + return f.issue, nil +} + +func (f *fakeUpdateIssueUC) ApplyUpdate(ctx context.Context, id string, spec domain.UpdateSpec, actor string) (*types.Issue, error) { + return f.issue, nil +} + +type fakeUOW struct { + issueUC domain.IssueUseCase + commit func() error +} + +func (f *fakeUOW) Close(ctx context.Context) {} +func (f *fakeUOW) Commit(ctx context.Context, message string) error { return f.commit() } +func (f *fakeUOW) ConfigUseCase() domain.ConfigUseCase { return nil } +func (f *fakeUOW) DoltRemoteUseCase() domain.DoltRemoteUseCase { return nil } +func (f *fakeUOW) BootstrapUseCase() domain.BootstrapUseCase { return nil } +func (f *fakeUOW) IssueUseCase() domain.IssueUseCase { return f.issueUC } +func (f *fakeUOW) DependencyUseCase() domain.DependencyUseCase { return nil } +func (f *fakeUOW) LabelUseCase() domain.LabelUseCase { return nil } +func (f *fakeUOW) CommentUseCase() domain.CommentUseCase { return nil } +func (f *fakeUOW) RawSQLUseCase() domain.RawSQLUseCase { return nil } + +type fakeUOWProvider struct { + uows atomic.Int64 + commit func(attempt int64) error + issue *types.Issue +} + +func (p *fakeUOWProvider) NewUOW(ctx context.Context) (uow.UnitOfWork, error) { + n := p.uows.Add(1) + return &fakeUOW{ + issueUC: &fakeUpdateIssueUC{issue: p.issue}, + commit: func() error { return p.commit(n) }, + }, nil +} + +func (p *fakeUOWProvider) Close(ctx context.Context) error { return nil } + +func withFakeProxiedUpdateEnv(t *testing.T, p *fakeUOWProvider) { + t.Helper() + oldProvider := uowProvider + oldHookRunner := hookRunner + uowProvider = p + hookRunner = hooks.NewRunner(t.TempDir()) // no hooks installed: RunSync no-ops + t.Cleanup(func() { + uowProvider = oldProvider + hookRunner = oldHookRunner + }) +} + +func captureStderrDuring(t *testing.T, fn func()) string { + t.Helper() + old := os.Stderr + r, w, err := os.Pipe() + if err != nil { + t.Fatalf("os.Pipe: %v", err) + } + os.Stderr = w + done := make(chan string, 1) + go func() { + data, _ := io.ReadAll(r) + done <- string(data) + }() + defer func() { + os.Stderr = old + _ = r.Close() + }() + fn() + _ = w.Close() + return <-done +} + +func serializationFailure() error { + return &mysql.MySQLError{Number: 1213, Message: "serialization failure, transaction rolled back"} +} + +// TestApplyUpdateProxiedOne_RetriesWholeAttemptOnConflict proves a Dolt +// serialization failure at commit time redoes the WHOLE read-merge-write in a +// fresh unit of work — not just the commit. Re-committing the rolled-back +// session was the old behavior; the server had already discarded the writes, +// so the retry could only report "nothing to commit" and the update vanished. +func TestApplyUpdateProxiedOne_RetriesWholeAttemptOnConflict(t *testing.T) { + issue := &types.Issue{ID: "bd-retry-1", Status: types.StatusOpen} + provider := &fakeUOWProvider{ + issue: issue, + commit: func(attempt int64) error { + if attempt <= 2 { + return serializationFailure() + } + return nil + }, + } + withFakeProxiedUpdateEnv(t, provider) + + in := &updateInput{fields: map[string]any{"title": "renamed"}} + got, failReason, err := applyUpdateProxiedOne(context.Background(), "bd-retry-1", in) + if err != nil { + t.Fatalf("applyUpdateProxiedOne: %v", err) + } + if failReason != "" { + t.Fatalf("expected update to succeed after conflict retries, got failure: %s", failReason) + } + if got == nil || got.ID != "bd-retry-1" { + t.Fatalf("updated issue = %v, want bd-retry-1", got) + } + if n := provider.uows.Load(); n != 3 { + t.Errorf("unit-of-work attempts = %d, want 3 (each conflict must redo the whole read-merge-write)", n) + } +} + +// TestApplyUpdateProxiedOne_ExhaustedConflictsFailLoudly proves a write that +// never lands cannot exit as a success: when every attempt loses Dolt's +// commit-time merge, the command reports the failure instead of printing +// "✓ Updated" (a non-empty failReason suppresses the success line and drives +// the non-zero exit via reportUpdateFailures in runUpdateProxiedServer). +func TestApplyUpdateProxiedOne_ExhaustedConflictsFailLoudly(t *testing.T) { + oldMax := proxiedUpdateRetryMaxElapsed + proxiedUpdateRetryMaxElapsed = 150 * time.Millisecond + t.Cleanup(func() { proxiedUpdateRetryMaxElapsed = oldMax }) + + issue := &types.Issue{ID: "bd-retry-2", Status: types.StatusOpen} + provider := &fakeUOWProvider{ + issue: issue, + commit: func(int64) error { return serializationFailure() }, + } + withFakeProxiedUpdateEnv(t, provider) + + var got *types.Issue + var failReason string + var err error + stderr := captureStderrDuring(t, func() { + in := &updateInput{fields: map[string]any{"title": "never lands"}} + got, failReason, err = applyUpdateProxiedOne(context.Background(), "bd-retry-2", in) + }) + if err != nil { + t.Fatalf("applyUpdateProxiedOne returned hard error: %v", err) + } + if failReason == "" || got != nil { + t.Fatalf("failReason=%q issue=%v: a write that never landed must be reported as a failure", failReason, got) + } + if !strings.Contains(stderr, "retries exhausted") { + t.Errorf("stderr = %q, want a loud retries-exhausted failure", stderr) + } + if n := provider.uows.Load(); n < 2 { + t.Errorf("unit-of-work attempts = %d, want at least 2 before giving up", n) + } +} + +// TestApplyUpdateProxiedOne_NothingToCommitWithoutConflictSucceeds preserves +// the one legitimate nothing-to-commit case: wisp-only updates write to +// dolt_ignored tables, so a successful ApplyUpdate can leave the Dolt commit +// layer with nothing to do. That is a success, not a lost write — no +// serialization failure preceded it. +func TestApplyUpdateProxiedOne_NothingToCommitWithoutConflictSucceeds(t *testing.T) { + issue := &types.Issue{ID: "bd-retry-3", Status: types.StatusOpen} + provider := &fakeUOWProvider{ + issue: issue, + commit: func(int64) error { return errors.New("nothing to commit") }, + } + withFakeProxiedUpdateEnv(t, provider) + + in := &updateInput{fields: map[string]any{"title": "wisp-shaped"}} + got, failReason, err := applyUpdateProxiedOne(context.Background(), "bd-retry-3", in) + if err != nil { + t.Fatalf("applyUpdateProxiedOne: %v", err) + } + if failReason != "" || got == nil { + t.Fatalf("failReason=%q issue=%v: empty-working-set commit on an unconflicted attempt is a success", failReason, got) + } + if n := provider.uows.Load(); n != 1 { + t.Errorf("unit-of-work attempts = %d, want 1 (nothing-to-commit must not trigger retries)", n) + } +} diff --git a/internal/storage/domain/db/issue.go b/internal/storage/domain/db/issue.go index 91b49301c..c277ea780 100644 --- a/internal/storage/domain/db/issue.go +++ b/internal/storage/domain/db/issue.go @@ -95,6 +95,7 @@ func (r *issueSQLRepositoryImpl) Update(ctx context.Context, id string, updates table := pickIssueTable(opts.UseWispsTable) _, statusChanging := updates["status"] + mergeOps := issueops.HasMergeOps(updates) // When the status changes we need the prior row to reproduce the embedded // lifecycle side effects (issueops.updateIssueInTx): closed_at is set on @@ -102,8 +103,10 @@ func (r *issueSQLRepositoryImpl) Update(ctx context.Context, id string, updates // transition, the audit event type is derived from the transition, and // is_blocked is recomputed for neighbors. Read the full old issue once so // all four use the same snapshot; the ErrNoRows contract is preserved. + // Merge operations (metadata edits, note appends) need the same read: they + // are resolved against the row as seen by THIS unit-of-work transaction. var oldIssue *types.Issue - if statusChanging { + if statusChanging || mergeOps { var err error oldIssue, err = r.Get(ctx, id, opts) if err != nil { @@ -114,6 +117,24 @@ func (r *issueSQLRepositoryImpl) Update(ctx context.Context, id string, updates } } + // Resolve read-merge-write operation keys (issueops.OpMergeMetadata, + // OpSetMetadata, OpUnsetMetadata, OpAppendNotes) into concrete column + // values inside the mutation transaction, mirroring the embedded path + // (issueops.updateIssueInTx). Callers must pass the OPERATION, never a + // value pre-merged from an earlier read: this runner is a Dolt sql-server + // session where FOR UPDATE is a parse-only no-op, so a stale-snapshot merge + // is only made safe by Dolt's commit-time conflict detection plus the + // caller redoing the whole unit of work on a serialization failure — and + // that redo re-runs this in-transaction resolution against the winner's + // committed row. + if mergeOps { + resolved, err := issueops.ResolveMergeOps(oldIssue, updates) + if err != nil { + return fmt.Errorf("db: Update %s: %w", id, err) + } + updates = resolved + } + setClauses := make([]string, 0, len(updates)+3) args := make([]any, 0, len(updates)+4) for key, value := range updates { diff --git a/internal/storage/domain/db/issue_mergeops_test.go b/internal/storage/domain/db/issue_mergeops_test.go new file mode 100644 index 000000000..ab96ad80a --- /dev/null +++ b/internal/storage/domain/db/issue_mergeops_test.go @@ -0,0 +1,201 @@ +package db + +import ( + "encoding/json" + + "github.com/steveyegge/beads/internal/storage/domain" + "github.com/steveyegge/beads/internal/storage/issueops" +) + +// TestIssueSQLRepository_MergeOps covers the read-merge-write operation keys +// (metadata edits, note appends) on the domain/db repository — the write path +// behind the proxied-server CLI. The ops must be resolved against the row read +// inside the SAME unit-of-work transaction, not against a snapshot the caller +// read earlier: pre-merged values from a stale MVCC snapshot are exactly the +// lost-update defect (concurrent writers erased each other's committed keys +// while both exited 0). +func (s *testSuite) TestIssueSQLRepository_MergeOps() { + s.Run("SetMetadataMergesIntoCurrentRow", s.issueUpdateSetMetadataMerges) + s.Run("MergeMetadataOverlaysTopLevelKeys", s.issueUpdateMergeMetadataOverlays) + s.Run("UnsetMetadataRemovesKeys", s.issueUpdateUnsetMetadataRemoves) + s.Run("AppendNotesConcatenates", s.issueUpdateAppendNotes) + s.Run("AppendNotesToEmptyHasNoLeadingNewline", s.issueUpdateAppendNotesEmpty) + s.Run("AppendNotesRejectsCombinedReplacement", s.issueUpdateAppendNotesConflict) + s.Run("SetMetadataRejectsCombinedReplacement", s.issueUpdateSetMetadataConflict) + s.Run("OpsRouteToWispsTable", s.issueUpdateMergeOpsWispRouting) +} + +func (s *testSuite) seedIssueWithMetadata(id, metadata, notes string) { + r := s.issueRepo() + in := newTestIssue(id, "merge ops seed") + if metadata != "" { + in.Metadata = json.RawMessage(metadata) + } + in.Notes = notes + s.Require().NoError(r.Insert(s.Ctx(), in, "seeder", domain.InsertIssueOpts{})) +} + +func (s *testSuite) readMetadataMap(id string, opts domain.IssueTableOpts) map[string]any { + out, err := s.issueRepo().Get(s.Ctx(), id, opts) + s.Require().NoError(err) + got := map[string]any{} + if len(out.Metadata) > 0 { + s.Require().NoError(json.Unmarshal(out.Metadata, &got)) + } + return got +} + +func (s *testSuite) issueUpdateSetMetadataMerges() { + s.seedIssueWithMetadata("bd-mo-set", `{"existing":"yes"}`, "") + r := s.issueRepo() + + err := r.Update(s.Ctx(), "bd-mo-set", map[string]any{ + issueops.OpSetMetadata: []string{"tier=gold", "score=99"}, + }, "tester", domain.IssueTableOpts{}) + s.Require().NoError(err) + + got := s.readMetadataMap("bd-mo-set", domain.IssueTableOpts{}) + s.Equal("yes", got["existing"], "pre-existing key must survive a set-metadata edit") + s.Equal("gold", got["tier"]) + s.Equal("99", got["score"], "--set-metadata values are stored as JSON strings (GH#4146)") +} + +func (s *testSuite) issueUpdateMergeMetadataOverlays() { + s.seedIssueWithMetadata("bd-mo-merge", `{"a":1,"b":2}`, "") + r := s.issueRepo() + + err := r.Update(s.Ctx(), "bd-mo-merge", map[string]any{ + issueops.OpMergeMetadata: json.RawMessage(`{"b":3,"c":4}`), + }, "tester", domain.IssueTableOpts{}) + s.Require().NoError(err) + + got := s.readMetadataMap("bd-mo-merge", domain.IssueTableOpts{}) + s.Equal(float64(1), got["a"]) + s.Equal(float64(3), got["b"]) + s.Equal(float64(4), got["c"]) + s.Len(got, 3) +} + +func (s *testSuite) issueUpdateUnsetMetadataRemoves() { + s.seedIssueWithMetadata("bd-mo-unset", `{"keep":"yes","drop":"yes"}`, "") + r := s.issueRepo() + + err := r.Update(s.Ctx(), "bd-mo-unset", map[string]any{ + issueops.OpUnsetMetadata: []string{"drop"}, + }, "tester", domain.IssueTableOpts{}) + s.Require().NoError(err) + + got := s.readMetadataMap("bd-mo-unset", domain.IssueTableOpts{}) + s.Equal("yes", got["keep"]) + s.NotContains(got, "drop") +} + +func (s *testSuite) issueUpdateAppendNotes() { + s.seedIssueWithMetadata("bd-mo-notes", "", "first") + r := s.issueRepo() + + err := r.Update(s.Ctx(), "bd-mo-notes", map[string]any{ + issueops.OpAppendNotes: "second", + }, "tester", domain.IssueTableOpts{}) + s.Require().NoError(err) + + out, err := r.Get(s.Ctx(), "bd-mo-notes", domain.IssueTableOpts{}) + s.Require().NoError(err) + s.Equal("first\nsecond", out.Notes) +} + +func (s *testSuite) issueUpdateAppendNotesEmpty() { + s.seedIssueWithMetadata("bd-mo-notes-empty", "", "") + r := s.issueRepo() + + err := r.Update(s.Ctx(), "bd-mo-notes-empty", map[string]any{ + issueops.OpAppendNotes: "only", + }, "tester", domain.IssueTableOpts{}) + s.Require().NoError(err) + + out, err := r.Get(s.Ctx(), "bd-mo-notes-empty", domain.IssueTableOpts{}) + s.Require().NoError(err) + s.Equal("only", out.Notes) +} + +func (s *testSuite) issueUpdateAppendNotesConflict() { + s.seedIssueWithMetadata("bd-mo-notes-conf", "", "first") + r := s.issueRepo() + + err := r.Update(s.Ctx(), "bd-mo-notes-conf", map[string]any{ + issueops.OpAppendNotes: "second", + "notes": "replacement", + }, "tester", domain.IssueTableOpts{}) + s.Require().Error(err) + s.Contains(err.Error(), "notes replacement") +} + +func (s *testSuite) issueUpdateSetMetadataConflict() { + s.seedIssueWithMetadata("bd-mo-set-conf", `{"a":1}`, "") + r := s.issueRepo() + + err := r.Update(s.Ctx(), "bd-mo-set-conf", map[string]any{ + issueops.OpSetMetadata: []string{"b=2"}, + "metadata": json.RawMessage(`{"c":3}`), + }, "tester", domain.IssueTableOpts{}) + s.Require().Error(err) + s.Contains(err.Error(), "metadata replacement") +} + +func (s *testSuite) issueUpdateMergeOpsWispRouting() { + r := s.issueRepo() + in := newTestIssue("bd-mo-wisp", "wisp merge ops") + in.Metadata = json.RawMessage(`{"kind":"wisp"}`) + in.Notes = "w1" + s.Require().NoError(r.Insert(s.Ctx(), in, "seeder", domain.InsertIssueOpts{UseWispsTable: true})) + + err := r.Update(s.Ctx(), "bd-mo-wisp", map[string]any{ + issueops.OpSetMetadata: []string{"extra=1"}, + issueops.OpAppendNotes: "w2", + }, "tester", domain.IssueTableOpts{UseWispsTable: true}) + s.Require().NoError(err) + + out, err := r.Get(s.Ctx(), "bd-mo-wisp", domain.IssueTableOpts{UseWispsTable: true}) + s.Require().NoError(err) + s.Equal("w1\nw2", out.Notes) + got := map[string]any{} + s.Require().NoError(json.Unmarshal(out.Metadata, &got)) + s.Equal("wisp", got["kind"], "wisp's pre-existing metadata key must survive") + s.Equal("1", got["extra"]) +} + +// TestIssueUseCase_ApplyUpdateMergeOps proves the ops flow through the domain +// use-case layer — the exact call chain the proxied-server update handler uses +// (issueUC.ApplyUpdate with spec.Fields). +func (s *testSuite) TestIssueUseCase_ApplyUpdateMergeOps() { + s.Run("SetMetadataViaApplyUpdatePreservesSiblingKeys", s.iucApplyUpdateSetMetadata) + s.Run("AppendNotesViaApplyUpdate", s.iucApplyUpdateAppendNotes) +} + +func (s *testSuite) iucApplyUpdateSetMetadata() { + s.seedIssueWithMetadata("bd-mo-uc", `{"sibling":"safe"}`, "") + + updated, err := s.issueUseCase().ApplyUpdate(s.Ctx(), "bd-mo-uc", domain.UpdateSpec{ + Fields: map[string]any{ + issueops.OpSetMetadata: []string{"mine=1"}, + }, + }, "tester") + s.Require().NoError(err) + + got := map[string]any{} + s.Require().NoError(json.Unmarshal(updated.Metadata, &got)) + s.Equal("safe", got["sibling"]) + s.Equal("1", got["mine"]) +} + +func (s *testSuite) iucApplyUpdateAppendNotes() { + s.seedIssueWithMetadata("bd-mo-uc-notes", "", "base") + + updated, err := s.issueUseCase().ApplyUpdate(s.Ctx(), "bd-mo-uc-notes", domain.UpdateSpec{ + Fields: map[string]any{ + issueops.OpAppendNotes: "appended", + }, + }, "tester") + s.Require().NoError(err) + s.Equal("base\nappended", updated.Notes) +} diff --git a/internal/storage/embeddeddolt/update_mergeops_test.go b/internal/storage/embeddeddolt/update_mergeops_test.go new file mode 100644 index 000000000..f9f1eecf6 --- /dev/null +++ b/internal/storage/embeddeddolt/update_mergeops_test.go @@ -0,0 +1,102 @@ +//go:build cgo + +package embeddeddolt_test + +import ( + "encoding/json" + "testing" + + "github.com/steveyegge/beads/internal/storage/issueops" + "github.com/steveyegge/beads/internal/types" +) + +// TestUpdateMergeOps_DoltEngine smokes the in-tx merge operations on the +// embedded-Dolt reference backend: metadata edits and note appends are resolved +// against the row read inside the mutation transaction (issueops.ResolveMergeOps), +// with the store's whole-attempt retry preserving concurrent writers' keys. A +// regression there would break every `bd update --set-metadata` / `bd note` on +// a Dolt workspace. +func TestUpdateMergeOps_DoltEngine(t *testing.T) { + te := newTestEnv(t, "mergeops") + ctx := t.Context() + + issue := &types.Issue{ + ID: "mergeops-1", + Title: "merge ops on dolt", + Status: types.StatusOpen, + IssueType: types.TypeTask, + Priority: 2, + Metadata: json.RawMessage(`{"keep":"x"}`), + Notes: "line1", + } + if err := te.store.CreateIssue(ctx, issue, "actor"); err != nil { + t.Fatalf("CreateIssue: %v", err) + } + + updates := map[string]interface{}{ + issueops.OpSetMetadata: []string{"team=platform"}, + issueops.OpAppendNotes: "line2", + } + if err := te.store.UpdateIssue(ctx, issue.ID, updates, "actor"); err != nil { + t.Fatalf("UpdateIssue(merge ops): %v", err) + } + + got, err := te.store.GetIssue(ctx, issue.ID) + if err != nil { + t.Fatalf("GetIssue: %v", err) + } + meta := map[string]any{} + if err := json.Unmarshal(got.Metadata, &meta); err != nil { + t.Fatalf("unmarshal metadata %q: %v", got.Metadata, err) + } + if meta["keep"] != "x" { + t.Errorf("metadata[keep]: got %v, want %q", meta["keep"], "x") + } + if meta["team"] != "platform" { + t.Errorf("metadata[team]: got %v, want %q", meta["team"], "platform") + } + if got.Notes != "line1\nline2" { + t.Errorf("notes: got %q, want %q", got.Notes, "line1\nline2") + } +} + +// TestUpdateMergeOps_DoltEngine_TwoIssues pins per-issue correctness of the +// locking read when the SAME statement text runs against DIFFERENT bindings: +// issue 1 is touched first so any text-keyed statement caching in the engine +// is primed with issue 1's row, then issue 2's merge must still read issue 2. +func TestUpdateMergeOps_DoltEngine_TwoIssues(t *testing.T) { + te := newTestEnv(t, "twoissues") + ctx := t.Context() + + for _, spec := range []struct{ id, notes string }{ + {"two-1", "first-notes"}, + {"two-2", ""}, + } { + iss := &types.Issue{ID: spec.id, Title: spec.id, Status: types.StatusOpen, IssueType: types.TypeTask, Priority: 2, Notes: spec.notes} + if err := te.store.CreateIssue(ctx, iss, "actor"); err != nil { + t.Fatalf("CreateIssue(%s): %v", spec.id, err) + } + } + + if err := te.store.UpdateIssue(ctx, "two-1", map[string]interface{}{issueops.OpAppendNotes: "added-1"}, "actor"); err != nil { + t.Fatalf("UpdateIssue(two-1): %v", err) + } + if err := te.store.UpdateIssue(ctx, "two-2", map[string]interface{}{issueops.OpAppendNotes: "added-2"}, "actor"); err != nil { + t.Fatalf("UpdateIssue(two-2): %v", err) + } + + got1, err := te.store.GetIssue(ctx, "two-1") + if err != nil { + t.Fatalf("GetIssue(two-1): %v", err) + } + got2, err := te.store.GetIssue(ctx, "two-2") + if err != nil { + t.Fatalf("GetIssue(two-2): %v", err) + } + if got1.Notes != "first-notes\nadded-1" { + t.Errorf("two-1 notes: got %q, want %q", got1.Notes, "first-notes\nadded-1") + } + if got2.Notes != "added-2" { + t.Errorf("two-2 notes: got %q, want %q", got2.Notes, "added-2") + } +} diff --git a/internal/storage/issueops/update.go b/internal/storage/issueops/update.go index f6d1c2292..7d9d90b82 100644 --- a/internal/storage/issueops/update.go +++ b/internal/storage/issueops/update.go @@ -196,10 +196,19 @@ func updateIssueInTx(ctx context.Context, tx DBTX, id string, updates map[string isWisp := IsActiveWispInTx(ctx, tx, id) issueTable, _, eventTable, _ := WispTableRouting(isWisp) - // Read old issue inside the transaction for consistency. - oldIssue, err := GetIssueInTx(ctx, tx, id) + // Read the pre-update row inside the transaction and fold any + // read-merge-write operations (metadata edits, note appends) into concrete + // column values against THAT row. On Dolt — the only supported backend — a + // stale snapshot is made safe not by row locks (FOR UPDATE is a parse-only + // no-op) but by commit-time conflict detection plus the store's + // whole-attempt retry (withRetryTx), which re-runs this resolution against + // the winning writer's committed row. Merging outside the mutation + // transaction — the CLI's old behavior — silently erased concurrent + // committed writes to sibling keys (GH audit: 7 of 200 exit-0 + // --set-metadata writes lost). + oldIssue, updates, err := readIssueAndResolveMergeOps(ctx, tx, id, updates) if err != nil { - return nil, fmt.Errorf("failed to get issue for update: %w", err) + return nil, err } // Validate issue_type against built-in + custom types (GH#3030). @@ -337,6 +346,192 @@ func updateIssueInTx(ctx context.Context, tx DBTX, id string, updates map[string return &UpdateResult{OldIssue: oldIssue, IsWisp: isWisp}, nil } +// Merge-operation update keys. Unlike plain column updates, these are resolved +// against the current row INSIDE the mutation transaction: re-read, merge, +// write, commit. Callers pass the operation (keys to set/unset, text to +// append) instead of a pre-merged value, so concurrent writers touching +// different keys of the same issue cannot erase each other. +const ( + // OpMergeMetadata merges a JSON object's top-level keys into the issue's + // metadata (bd update --metadata). Value: string, []byte, or json.RawMessage. + OpMergeMetadata = "_merge_metadata" + // OpSetMetadata sets individual key=value metadata entries + // (bd update --set-metadata). Value: []string. + OpSetMetadata = "_set_metadata" + // OpUnsetMetadata removes metadata keys (bd update --unset-metadata). + // Value: []string. + OpUnsetMetadata = "_unset_metadata" + // OpAppendNotes appends a line to the issue's notes + // (bd update --append-notes). Value: string. + OpAppendNotes = "append_notes" +) + +// HasMergeOps reports whether the update map carries any read-merge-write +// operation key. Updates with merge ops must resolve those ops against the row +// read inside the same mutation transaction (see ResolveMergeOps); on Dolt the +// store's whole-attempt retry then re-runs that resolution against the winning +// writer's committed row. +func HasMergeOps(updates map[string]interface{}) bool { + for _, op := range []string{OpMergeMetadata, OpSetMetadata, OpUnsetMetadata, OpAppendNotes} { + if _, ok := updates[op]; ok { + return true + } + } + return false +} + +// ResolveMergeOps rewrites merge-operation keys into concrete column values +// using oldIssue, which the caller read in the same mutation transaction. +// Returns the input map unchanged when no operation keys are present; +// otherwise returns a copy so the caller's map is not mutated. +func ResolveMergeOps(oldIssue *types.Issue, updates map[string]interface{}) (map[string]interface{}, error) { + if !HasMergeOps(updates) { + return updates, nil + } + + resolved := make(map[string]interface{}, len(updates)) + for k, v := range updates { + if !isMergeOpKey(k) { + resolved[k] = v + } + } + + if err := resolveMetadataMergeOps(oldIssue, updates, resolved); err != nil { + return nil, err + } + if err := resolveNotesAppendOp(oldIssue, updates, resolved); err != nil { + return nil, err + } + return resolved, nil +} + +// isMergeOpKey reports whether k is a read-merge-write operation key consumed by +// ResolveMergeOps rather than a concrete column value to pass through unchanged. +func isMergeOpKey(k string) bool { + switch k { + case OpMergeMetadata, OpSetMetadata, OpUnsetMetadata, OpAppendNotes: + return true + default: + return false + } +} + +// resolveMetadataMergeOps folds OpMergeMetadata/OpSetMetadata/OpUnsetMetadata +// into a concrete "metadata" value on resolved, using oldIssue.Metadata (read in +// the same mutation transaction) as the base. It is a no-op when no metadata +// operation keys are present. +func resolveMetadataMergeOps(oldIssue *types.Issue, updates, resolved map[string]interface{}) error { + _, hasMerge := updates[OpMergeMetadata] + _, hasSet := updates[OpSetMetadata] + _, hasUnset := updates[OpUnsetMetadata] + if !hasMerge && !hasSet && !hasUnset { + return nil + } + if _, direct := resolved["metadata"]; direct { + return fmt.Errorf("cannot combine a metadata replacement with incremental metadata edits") + } + + current := oldIssue.Metadata + if hasMerge { + normalized, err := storage.NormalizeMetadataValue(updates[OpMergeMetadata]) + if err != nil { + return fmt.Errorf("invalid %s: %w", OpMergeMetadata, err) + } + merged, err := storage.MergeMetadataJSON(current, json.RawMessage(normalized)) + if err != nil { + return fmt.Errorf("metadata merge failed: %w", err) + } + current = merged + } + if hasSet || hasUnset { + set, err := mergeOpStrings(OpSetMetadata, updates[OpSetMetadata], hasSet) + if err != nil { + return err + } + unset, err := mergeOpStrings(OpUnsetMetadata, updates[OpUnsetMetadata], hasUnset) + if err != nil { + return err + } + merged, err := storage.ApplyMetadataEdits(current, set, unset) + if err != nil { + return fmt.Errorf("metadata edit failed: %w", err) + } + current = merged + } + // Validate the merged result, matching the schema check stores apply to + // direct metadata replacements (GH#1416 Phase 2). + if err := ValidateMetadataIfConfigured(current); err != nil { + return err + } + resolved["metadata"] = current + return nil +} + +// resolveNotesAppendOp folds OpAppendNotes into a concrete "notes" value on +// resolved, appending to oldIssue.Notes (read in the same mutation transaction). +// It is a no-op when the append op is absent. +func resolveNotesAppendOp(oldIssue *types.Issue, updates, resolved map[string]interface{}) error { + raw, ok := updates[OpAppendNotes] + if !ok { + return nil + } + if _, direct := resolved["notes"]; direct { + return fmt.Errorf("cannot combine a notes replacement with %s", OpAppendNotes) + } + text, ok := raw.(string) + if !ok { + return fmt.Errorf("%s must be a string, got %T", OpAppendNotes, raw) + } + combined := oldIssue.Notes + if combined != "" { + combined += "\n" + } + combined += text + resolved["notes"] = combined + return nil +} + +// mergeOpStrings coerces a merge-operation value to []string. Accepts +// []interface{} of strings as well, so operation maps survive a JSON +// round-trip (e.g. daemon transports). +func mergeOpStrings(op string, value interface{}, present bool) ([]string, error) { + if !present { + return nil, nil + } + switch v := value.(type) { + case []string: + return v, nil + case []interface{}: + out := make([]string, 0, len(v)) + for _, item := range v { + s, ok := item.(string) + if !ok { + return nil, fmt.Errorf("%s must be a list of strings, got element %T", op, item) + } + out = append(out, s) + } + return out, nil + default: + return nil, fmt.Errorf("%s must be a list of strings, got %T", op, value) + } +} + +// readIssueAndResolveMergeOps reads the pre-update row in-transaction and folds +// any merge-operation keys (metadata edits, note appends) into concrete column +// values against that row, returning the row and the rewritten update map. It +// keeps the read-merge-write plumbing off updateIssueInTx's already-large body. +func readIssueAndResolveMergeOps(ctx context.Context, tx DBTX, id string, updates map[string]interface{}) (*types.Issue, map[string]interface{}, error) { + oldIssue, err := GetIssueInTx(ctx, tx, id) + if err != nil { + return nil, nil, fmt.Errorf("failed to get issue for update: %w", err) + } + resolved, err := ResolveMergeOps(oldIssue, updates) + if err != nil { + return nil, nil, err + } + return oldIssue, resolved, nil +} + // RecordFullEventInTable records an event with both old and new values. // //nolint:gosec // G201: table is from WispTableRouting ("events" or "wisp_events") diff --git a/internal/storage/metadata.go b/internal/storage/metadata.go index ef11cc34b..34b2b5805 100644 --- a/internal/storage/metadata.go +++ b/internal/storage/metadata.go @@ -229,3 +229,83 @@ func JSONMetadataPath(key string) string { } return "$." + key } + +// MergeMetadataJSON merges incoming metadata JSON into existing metadata. +// Top-level keys from incoming overwrite keys in existing; keys only in +// existing are preserved. Both inputs must be JSON objects (or empty/null). +func MergeMetadataJSON(existing, incoming json.RawMessage) (json.RawMessage, error) { + base := make(map[string]json.RawMessage) + if len(existing) > 0 { + trimmed := strings.TrimSpace(string(existing)) + if trimmed != "" && trimmed != "null" { + if err := json.Unmarshal(existing, &base); err != nil { + return nil, fmt.Errorf("existing metadata is not a JSON object: %w", err) + } + } + } + + overlay := make(map[string]json.RawMessage) + if err := json.Unmarshal(incoming, &overlay); err != nil { + return nil, fmt.Errorf("new metadata is not a JSON object: %w", err) + } + + for k, v := range overlay { + base[k] = v + } + + result, err := json.Marshal(base) + if err != nil { + return nil, fmt.Errorf("failed to marshal merged metadata: %w", err) + } + return json.RawMessage(result), nil +} + +// ApplyMetadataEdits applies incremental set (key=value) and unset (key) edits +// to existing metadata and returns the merged JSON. Set values are typed via +// MetadataEditValue; keys are validated with ValidateMetadataKey. +func ApplyMetadataEdits(existing json.RawMessage, setFlags, unsetFlags []string) (json.RawMessage, error) { + data := make(map[string]json.RawMessage) + if len(existing) > 0 { + trimmed := strings.TrimSpace(string(existing)) + if trimmed != "" && trimmed != "null" { + if err := json.Unmarshal(existing, &data); err != nil { + return nil, fmt.Errorf("existing metadata is not a JSON object: %w", err) + } + } + } + + for _, kv := range setFlags { + k, v, ok := strings.Cut(kv, "=") + if !ok || k == "" { + return nil, fmt.Errorf("invalid --set-metadata: expected key=value, got %q", kv) + } + if err := ValidateMetadataKey(k); err != nil { + return nil, err + } + data[k] = MetadataEditValue(v) + } + + for _, k := range unsetFlags { + if err := ValidateMetadataKey(k); err != nil { + return nil, err + } + delete(data, k) + } + + result, err := json.Marshal(data) + if err != nil { + return nil, fmt.Errorf("failed to marshal metadata: %w", err) + } + return json.RawMessage(result), nil +} + +// MetadataEditValue converts a --set-metadata string value to JSON. Per the CLI +// contract (GH#4146), --set-metadata values are ALWAYS stored as JSON strings; +// inferring numbers/booleans/null from string content silently broke +// map[string]string round-trips for Go consumers (a numeric-looking id or a +// version like "1e3" came back as a JSON number). Typed values go through the +// explicit --metadata / --metadata-json path (MergeMetadataJSON). +func MetadataEditValue(s string) json.RawMessage { + b, _ := json.Marshal(s) + return json.RawMessage(b) +} diff --git a/internal/storage/uow/errors.go b/internal/storage/uow/errors.go index 2bb91e934..d0711f43d 100644 --- a/internal/storage/uow/errors.go +++ b/internal/storage/uow/errors.go @@ -6,6 +6,16 @@ import ( mysql "github.com/go-sql-driver/mysql" ) +// IsSerializationError reports whether err is a Dolt/MySQL serialization +// failure that guarantees the server rolled the transaction back. Because the +// rollback discards every uncommitted write in the session, the only safe +// retry is to redo the WHOLE unit of work (read, merge, write, commit) — +// retrying just the commit re-commits an empty session, which Dolt reports as +// "nothing to commit" while the write is silently lost. +func IsSerializationError(err error) bool { + return isSerializationError(err) +} + // isSerializationError returns true if the error is a Dolt/MySQL serialization // failure that guarantees the transaction was rolled back. Safe to retry. // - 1213 (ER_LOCK_DEADLOCK): concurrent transactions conflict at commit time diff --git a/internal/storage/uow/lostupdate_dolt_test.go b/internal/storage/uow/lostupdate_dolt_test.go new file mode 100644 index 000000000..1734fcfc3 --- /dev/null +++ b/internal/storage/uow/lostupdate_dolt_test.go @@ -0,0 +1,262 @@ +package uow + +import ( + "context" + "encoding/json" + "fmt" + "path/filepath" + "strconv" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + _ "github.com/go-sql-driver/mysql" + "github.com/stretchr/testify/require" + + "github.com/steveyegge/beads/internal/configfile" + "github.com/steveyegge/beads/internal/storage/dbproxy/proxy" + "github.com/steveyegge/beads/internal/storage/domain" + "github.com/steveyegge/beads/internal/storage/issueops" + "github.com/steveyegge/beads/internal/testutil" + "github.com/steveyegge/beads/internal/types" +) + +// TestUOW_ConcurrentMergeOps_NoLostUpdate proves the lost-update fix against a +// REAL Dolt sql-server: concurrent writers mutating the SAME issue through the +// unit-of-work stack (the proxied-server CLI's write path) must never lose a +// write that reported success. +// +// Each writer follows the exact contract cmd/bd's applyUpdateProxiedOne now +// implements: +// +// - metadata edits and note appends pass through ApplyUpdate as +// merge-OPERATION keys (issueops.OpSetMetadata / OpAppendNotes), resolved +// against the row read inside the mutation transaction — never pre-merged +// from an earlier snapshot; +// - the unit of work commits exactly ONCE; a serialization failure (Dolt's +// commit-time conflict, guaranteed server-side rollback) redoes the WHOLE +// read-merge-write in a fresh unit of work; +// - "nothing to commit" on a fresh, unconflicted attempt would mean the +// write silently vanished, so the writer fails the test if it appears. +// +// The old commit-only retry (uow.CommitWithRetries) re-committed the +// rolled-back session, harvested "nothing to commit", and the CLI swallowed it +// with exit 0 — the silent-loss shape this test exists to catch. +// +// Two phases, because Dolt's conflict behavior differs by column type: +// +// - metadata is a JSON column, which Dolt three-way merges KEY-WISE at +// commit time: distinct-key writers converge without any conflict, so the +// metadata phase asserts convergence but cannot demand conflicts; +// - notes is a TEXT column with no structural merge: overlapping writers +// editing that cell MUST conflict, so the notes phase demands at least one +// serialization failure — proof the whole-UOW redo path actually ran — +// and still requires every append to survive. +func TestUOW_ConcurrentMergeOps_NoLostUpdate(t *testing.T) { + if testing.Short() { + t.Skip("starts a Dolt container and a dbproxy subprocess; skipped in -short") + } + port := testutil.StartIsolatedDoltContainer(t) + portInt, err := strconv.Atoi(port) + require.NoError(t, err) + + bdBin := buildBDBinary(t) + prev := proxy.ResolveExecutable + proxy.ResolveExecutable = func() (string, error) { return bdBin, nil } + t.Cleanup(func() { proxy.ResolveExecutable = prev }) + + t.Setenv("HOME", t.TempDir()) + + storeRootDir := t.TempDir() + shutdownOnInterrupt(t, storeRootDir) + t.Cleanup(func() { + if err := proxy.Shutdown(storeRootDir); err != nil { + t.Logf("proxy.Shutdown(%s): %v", storeRootDir, err) + } + }) + logPath := filepath.Join(t.TempDir(), "server.log") + + ctx, cancel := context.WithTimeout(context.Background(), 180*time.Second) + defer cancel() + + provider, err := NewExternalDoltServerUOWProvider( + ctx, + storeRootDir, + "beads_lostupdate_test", + logPath, + configfile.ExternalDoltConfig{Host: "127.0.0.1", Port: portInt}, + "root", + "", + 0, + 0, + ) + require.NoError(t, err) + require.NotNil(t, provider) + t.Cleanup(func() { _ = provider.Close(context.Background()) }) + + const issueID = "lu-1" + func() { + uw, err := provider.NewUOW(ctx) + require.NoError(t, err) + defer uw.Close(ctx) + _, err = uw.IssueUseCase().CreateIssue(ctx, domain.CreateIssueParams{ + Issue: &types.Issue{ + ID: issueID, + Title: "lost-update target", + Status: types.StatusOpen, + Priority: 2, + IssueType: types.TypeTask, + Notes: "seed line", + Metadata: json.RawMessage(`{"seed":"yes"}`), + }, + ExplicitID: issueID, + }, "seeder") + require.NoError(t, err) + require.NoError(t, uw.Commit(ctx, "seed lost-update target")) + }() + + // conflicts counts observed serialization failures so the assertion below + // can prove the commit-time conflict path really fired. + var conflicts atomic.Int64 + + // writeOp mirrors applyUpdateProxiedOne: fresh UOW per attempt, merge ops + // in spec.Fields, one commit, whole-attempt redo on serialization failure. + // + // syncPoint is a barrier placed between ApplyUpdate and Commit on the + // FIRST attempt only: every writer applies its update against the same + // base root before any of them commits, guaranteeing the transactions + // overlap at Dolt's commit-time merge. + writeOp := func(label string, fields map[string]any, syncPoint func()) error { + synced := false + syncOnce := func() { + if !synced { + synced = true + syncPoint() + } + } + defer syncOnce() // never leave barrier peers hanging on an early error + const maxAttempts = 40 + for attempt := 1; attempt <= maxAttempts; attempt++ { + err := func() error { + uw, err := provider.NewUOW(ctx) + if err != nil { + return fmt.Errorf("new uow: %w", err) + } + defer uw.Close(ctx) + if _, err := uw.IssueUseCase().ApplyUpdate(ctx, issueID, + domain.UpdateSpec{Fields: fields}, "writer-"+label); err != nil { + return err + } + syncOnce() + return uw.Commit(ctx, "write "+label) + }() + if err == nil { + return nil + } + if IsSerializationError(err) { + conflicts.Add(1) + time.Sleep(time.Duration(attempt) * 10 * time.Millisecond) + continue + } + if issueops.IsNothingToCommitError(err) { + return fmt.Errorf("write %s swallowed as nothing-to-commit on a fresh attempt (silent lost update): %w", label, err) + } + return fmt.Errorf("write %s: %w", label, err) + } + return fmt.Errorf("write %s: retries exhausted on serialization failures", label) + } + + runRound := func(round int, fieldsFor func(writer int) (string, map[string]any)) { + t.Helper() + const writers = 4 + errs := make([]error, writers) + var applied, done sync.WaitGroup + applied.Add(writers) + done.Add(writers) + barrier := func() { + applied.Done() + applied.Wait() + } + for i := 0; i < writers; i++ { + i := i + label, fields := fieldsFor(i) + go func() { + defer done.Done() + errs[i] = writeOp(label, fields, barrier) + }() + } + done.Wait() + for i, err := range errs { + require.NoErrorf(t, err, "round %d writer %d", round, i) + } + } + + // Phase 1: distinct metadata keys. Dolt merges the JSON column key-wise at + // commit time, so these converge with or without observed conflicts. + const metadataRounds = 2 + for round := 0; round < metadataRounds; round++ { + round := round + runRound(round, func(writer int) (string, map[string]any) { + key := fmt.Sprintf("w%dr%d", writer, round) + return key, map[string]any{ + issueops.OpSetMetadata: []string{key + "=1"}, + } + }) + } + + // Phase 2: note appends. The notes TEXT cell has no structural merge, so + // the barrier-aligned first attempts MUST collide at commit time and the + // losers MUST redo the whole read-merge-write to preserve every line. + notesConflictFloor := conflicts.Load() + const notesRounds = 2 + for round := 0; round < notesRounds; round++ { + round := round + runRound(round, func(writer int) (string, map[string]any) { + label := fmt.Sprintf("note-w%dr%d", writer, round) + return label, map[string]any{ + issueops.OpAppendNotes: "appended by " + label, + } + }) + } + notesConflicts := conflicts.Load() - notesConflictFloor + + t.Logf("observed %d serialization conflicts total (%d in the notes phase)", + conflicts.Load(), notesConflicts) + require.Positive(t, notesConflicts, + "barrier-aligned writers on the notes TEXT cell must hit Dolt commit-time conflicts; the whole-UOW redo path was never exercised") + + // Every write reported as successful must be present, and the seeds must + // survive. + uw, err := provider.NewUOW(ctx) + require.NoError(t, err) + defer uw.Close(ctx) + final, err := uw.IssueUseCase().GetIssue(ctx, issueID) + require.NoError(t, err) + require.NotNil(t, final) + + got := map[string]any{} + require.NoError(t, json.Unmarshal(final.Metadata, &got)) + require.Equal(t, "yes", got["seed"], "seed key erased by concurrent writers: %v", got) + var missing []string + for round := 0; round < metadataRounds; round++ { + for writer := 0; writer < 4; writer++ { + key := fmt.Sprintf("w%dr%d", writer, round) + if _, ok := got[key]; !ok { + missing = append(missing, key) + } + } + } + require.Emptyf(t, missing, "silent lost update: %d successful metadata writes missing from final metadata %v", len(missing), got) + + require.Contains(t, final.Notes, "seed line", "seed notes erased by concurrent appends") + for round := 0; round < notesRounds; round++ { + for writer := 0; writer < 4; writer++ { + line := fmt.Sprintf("appended by note-w%dr%d", writer, round) + require.Containsf(t, final.Notes, line, + "silent lost update: successful append %q missing from final notes %q", line, final.Notes) + } + } + require.Equal(t, 1, strings.Count(final.Notes, "seed line"), "seed line duplicated: redo must not double-apply") +} diff --git a/scripts/check-testing-short.sh b/scripts/check-testing-short.sh index 39bbf2944..7cee2a6b9 100755 --- a/scripts/check-testing-short.sh +++ b/scripts/check-testing-short.sh @@ -12,6 +12,7 @@ internal/testutil/fixtures/fixtures_test.go::TestLargeFromJSONL internal/storage/dolt/concurrent_test.go::TestHighContentionStress internal/storage/dolt/concurrent_test.go::TestConcurrentWorkQueueDrain internal/storage/dolt/lease_test.go::TestConcurrentHeartbeatReclaimClose +internal/storage/uow/lostupdate_dolt_test.go::TestUOW_ConcurrentMergeOps_NoLostUpdate cmd/bd/prune_bench_test.go::TestPruneLargeFixture EOF )