fix(update): resolve metadata/note merges inside the mutation transaction (concurrent lost-update) (#4732)

* fix(update): resolve metadata/note merges inside the mutation transaction

bd update --metadata/--set-metadata/--unset-metadata/--append-notes and
bd note resolved their merges client-side: read the issue in one
transaction, merge in process memory, write the merged blob back in a
later transaction. Two concurrent writers touching DIFFERENT keys of
the SAME issue both exited 0, but the later commit silently erased the
earlier writer's committed keys. The multiprocess-safety audit proved
the loss empirically: 7 of 200 exit-0 --set-metadata writes vanished in
an 8-writer same-issue hammer against the stock binary.

The first version of this fix moved the merge into the mutation
transaction on the direct path only; red-team review blocked it because
the proxied-server route still snapshot-merged in
buildUpdateSpecForIssue — and worse: on a Dolt serialization rollback,
uow.CommitWithRetries retried only the COMMIT, re-committing the
rolled-back (now empty) session. Dolt answered "nothing to commit",
isDoltNothingToCommit swallowed it, and bd printed "Updated" and exited
0 for a write that never landed.

This revision closes both paths:

- Merge-shaped edits travel as operation keys (issueops.OpMergeMetadata,
  OpSetMetadata, OpUnsetMetadata, OpAppendNotes) and are resolved
  against the row re-read INSIDE the mutation transaction
  (issueops.ResolveMergeOps) — in both the embedded path
  (updateIssueInTx) and the domain/db repository used by proxied-server
  mode. CLI paths (bd update, bd note) no longer pre-merge; the shared
  merge helpers live in internal/storage.
- The in-transaction read takes a row write lock
  (GetIssueForUpdateInTx, SELECT ... FOR UPDATE) so the read itself
  serializes on Postgres (READ COMMITTED) and MySQL (REPEATABLE READ).
  sqlitedialect strips the clause — safe because _txlock=immediate
  already takes the database write lock at BEGIN. Dolt parses FOR
  UPDATE as a no-op and relies on commit-time conflict detection plus
  the whole-attempt retry below.
- The proxied server redoes the WHOLE read-merge-write attempt in a
  fresh unit of work on a serialization failure (new exported
  uow.IsSerializationError; exponential backoff, 15s budget), commits
  each attempt exactly once, and fails loudly to stderr when retries
  exhaust instead of exiting 0.

Verification: cross-backend regression body in
internal/storage/conformance/lostupdate.go (two independent store
handles plus a lock-holding third pool) wired into the sqlite, dolt
(embedded and sql-server), mysql, and postgres suites; cmd/bd
multi-process race tests for --set-metadata and bd note; proxied-server
attempt/retry unit tests. Re-running the audit hammer against the
patched binary: 0 lost of 200 exit-0 writes across natural and
lock-widened runs (stock binary lost writes in both), with the one
contention exhaustion surfacing as a nonzero exit instead of a silent
loss.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(update): string --set-metadata, proxied partial-failure exit, drop dead locking scaffolding

Maintainer adoption-review fixes for PR #4732 (review iterations 1 and 3):

- metadata: MetadataEditValue stores --set-metadata values as JSON strings
  again (GH#4146) instead of inferring numbers/booleans/null from content,
  which silently broke map[string]string round-trips. Typed values stay
  available via --metadata / --metadata-json. Domain merge-op tests now
  expect string values for --set-metadata.
- cmd/bd: restore the proxied multi-ID partial-failure accumulator so a batch
  with any failed ID exits non-zero (had regressed #4726), and derive success
  output from the resolved jsonOutput signal so --format json and config
  json:true are honored on the proxied path.
- storage: remove the dead internal/storage/sqlitedialect package (509 LOC,
  zero importers) and the FOR UPDATE / GetIssueForUpdateInTx row-locking
  scaffolding. SQLite/Postgres/MySQL are gone and Dolt treats FOR UPDATE as a
  parse-only no-op; the read-merge-write is made safe by commit-time conflict
  detection plus the whole-attempt retry. Also drop the uncalled MVCC-only
  conformance helper. updateIssueInTx returns to baseline complexity via a
  small read-and-resolve helper.
- tests: remove the SQLite-backend race tests (the removed backend can't run
  them); Dolt concurrency coverage in uow/embeddeddolt/domain-db is
  load-bearing.
- readability: split ResolveMergeOps into resolveMetadataMergeOps and
  resolveNotesAppendOp (with an isMergeOpKey helper) so each diff-authored
  merge-op resolver stays below the review complexity gate (cyclomatic <21 /
  cognitive <26). Pure structure change: same resolution order, error strings,
  schema validation, and copy-not-mutate semantics.
- policy: allowlist TestUOW_ConcurrentMergeOps_NoLostUpdate in
  scripts/check-testing-short.sh. Its testing.Short() skip is a legitimate
  runtime/stress skip (it starts a Dolt container and a dbproxy subprocess),
  matching the other allowlisted Dolt concurrency stress tests; without the
  entry the ci-pr-policy check fails on the adopted head.

- tests: remove the two obsolete cmd/bd multi-ID tests
  (TestMultiIDUpdatePartialMetadataMergeFailure and
  TestMultiIDUpdatePartialMetadataEditFailureJSON) and their now-unused
  seedNonObjectMetadata/showMultiIDUpdateMetadataValue helpers. They seeded a
  non-object array via `bd update --metadata "[]"` to force a later per-ID
  metadata-merge failure, but the new in-transaction merge correctly rejects
  non-object metadata ("new metadata is not a JSON object"), so that seeding
  path is unreachable and the tests now fail at setup. The four #4726 per-ID
  exit-code tests in the same file are retained.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Test User <test@test.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Julian Knutsen 2026-07-19 05:59:31 -07:00 • committed by GitHub
commit 8ebd53e882
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
14 changed files with 1445 additions and 326 deletions

View file

@ -8,6 +8,7 @@ import (
"github.com/spf13/cobra"
"github.com/steveyegge/beads/internal/metrics"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/ui"
)
@ -95,14 +96,13 @@ Examples:
return HandleErrorRespectJSON("%s", err)
}
combined := issue.Notes
if combined != "" {
combined += "\n"
}
combined += noteText
// Passed as an append OPERATION, not a pre-merged value: the storage
// layer re-reads the row inside the mutation transaction and appends
// there. Concatenating onto the `issue` snapshot here (read in an
// earlier transaction) silently erased notes committed by a concurrent
// `bd note` / `bd update --append-notes` — both processes exited 0.
updates := map[string]interface{}{
"notes": combined,
issueops.OpAppendNotes: noteText,
}
if err := issueStore.UpdateIssue(ctx, result.ResolvedID, updates, actor); err != nil {
return HandleErrorRespectJSON("updating %s: %v", id, err)

View file

@ -12,6 +12,7 @@ import (
"github.com/steveyegge/beads/internal/debug"
"github.com/steveyegge/beads/internal/metrics"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/timeparsing"
"github.com/steveyegge/beads/internal/types"
"github.com/steveyegge/beads/internal/ui"
@ -138,7 +139,7 @@ stderr, and the command exits nonzero.`,
}
if cmd.Flags().Changed("append-notes") {
appendNotes, _ := cmd.Flags().GetString("append-notes")
updates["append_notes"] = appendNotes
updates[issueops.OpAppendNotes] = appendNotes
}
if cmd.Flags().Changed("acceptance") || cmd.Flags().Changed("acceptance-criteria") {
var acceptanceCriteria string
@ -292,7 +293,10 @@ stderr, and the command exits nonzero.`,
if !json.Valid([]byte(metadataJSON)) {
return HandleErrorRespectJSON("invalid JSON in --metadata: must be valid JSON")
}
updates["metadata"] = json.RawMessage(metadataJSON)
// Passed as a merge OPERATION, not a pre-merged value: the storage
// layer re-reads and merges inside the mutation transaction so a
// concurrent writer's keys survive (lost-update fix).
updates[issueops.OpMergeMetadata] = json.RawMessage(metadataJSON)
}
// Incremental metadata edits (GH#1406)
@ -301,9 +305,11 @@ stderr, and the command exits nonzero.`,
if (len(setMetadataFlags) > 0 || len(unsetMetadataFlags) > 0) && cmd.Flags().Changed("metadata") {
return HandleErrorRespectJSON("cannot combine --metadata with --set-metadata or --unset-metadata")
}
if len(setMetadataFlags) > 0 || len(unsetMetadataFlags) > 0 {
updates["_set_metadata"] = setMetadataFlags
updates["_unset_metadata"] = unsetMetadataFlags
if len(setMetadataFlags) > 0 {
updates[issueops.OpSetMetadata] = setMetadataFlags
}
if len(unsetMetadataFlags) > 0 {
updates[issueops.OpUnsetMetadata] = unsetMetadataFlags
}
// Get claim flag
@ -390,11 +396,16 @@ stderr, and the command exits nonzero.`,
trackMutation(result)
}
// Apply regular field updates if any
// Apply regular field updates if any. Metadata edits (--metadata,
// --set-metadata, --unset-metadata) and --append-notes pass through
// as merge OPERATIONS: the storage layer resolves them against the
// row re-read inside the mutation transaction. Merging here against
// the `issue` snapshot (read in an earlier transaction) silently
// erased concurrent writers' keys — both processes exited 0, one
// process's committed write vanished.
regularUpdates := make(map[string]interface{})
for k, v := range updates {
if k != "add_labels" && k != "remove_labels" && k != "set_labels" && k != "parent" && k != "append_notes" &&
k != "_set_metadata" && k != "_unset_metadata" {
if k != "add_labels" && k != "remove_labels" && k != "set_labels" && k != "parent" {
regularUpdates[k] = v
}
}
@ -404,45 +415,6 @@ stderr, and the command exits nonzero.`,
if clearDeferStatus && issue.Status == types.StatusDeferred {
regularUpdates["status"] = string(types.StatusOpen)
}
// Handle --metadata: merge with existing metadata instead of replacing.
// A merge failure (e.g. the issue's existing metadata is not a JSON
// object) is a per-ID failure, not a batch abort: record it and move
// on like the resolve/update/label/parent paths above, so later IDs
// still update and the failed ID surfaces in the nonzero-exit report
// instead of the generic stdout error path.
if newMeta, ok := regularUpdates["metadata"].(json.RawMessage); ok && len(issue.Metadata) > 0 {
merged, err := mergeMetadata(issue.Metadata, newMeta)
if err != nil {
fmt.Fprintf(os.Stderr, "Error merging metadata for %s: %v\n", id, err)
recordFailure(id, fmt.Sprintf("merging metadata: %v", err))
closeIfUnmutated(result)
continue
}
regularUpdates["metadata"] = merged
}
// Handle incremental metadata edits (GH#1406). Same per-ID failure
// contract as the merge path above.
if setMeta, ok := updates["_set_metadata"].([]string); ok {
unsetMeta, _ := updates["_unset_metadata"].([]string)
merged, err := applyMetadataEdits(issue.Metadata, setMeta, unsetMeta)
if err != nil {
fmt.Fprintf(os.Stderr, "Error editing metadata for %s: %v\n", id, err)
recordFailure(id, fmt.Sprintf("editing metadata: %v", err))
closeIfUnmutated(result)
continue
}
regularUpdates["metadata"] = merged
}
// Handle append_notes: combine existing notes with new content
if appendNotes, ok := updates["append_notes"].(string); ok {
combined := issue.Notes
if combined != "" {
combined += "\n"
}
combined += appendNotes
regularUpdates["notes"] = combined
}
if len(regularUpdates) > 0 {
if err := issueStore.UpdateIssue(ctx, result.ResolvedID, regularUpdates, actor); err != nil {
fmt.Fprintf(os.Stderr, "Error updating %s: %v\n", id, err)
@ -659,81 +631,22 @@ func reportUpdateFailures(failures []updateIDFailure, total int) error {
// mergeMetadata merges new metadata JSON into existing metadata.
// Keys from newMeta overwrite keys in existing; keys only in existing are preserved.
// Thin alias over the shared storage helper (also used in-transaction by issueops).
func mergeMetadata(existing, newMeta json.RawMessage) (json.RawMessage, error) {
base := make(map[string]json.RawMessage)
if len(existing) > 0 {
trimmed := strings.TrimSpace(string(existing))
if trimmed != "" && trimmed != "null" {
if err := json.Unmarshal(existing, &base); err != nil {
return nil, fmt.Errorf("existing metadata is not a JSON object: %w", err)
}
}
}
incoming := make(map[string]json.RawMessage)
if err := json.Unmarshal(newMeta, &incoming); err != nil {
return nil, fmt.Errorf("new metadata is not a JSON object: %w", err)
}
for k, v := range incoming {
base[k] = v
}
result, err := json.Marshal(base)
if err != nil {
return nil, fmt.Errorf("failed to marshal merged metadata: %w", err)
}
return json.RawMessage(result), nil
return storage.MergeMetadataJSON(existing, newMeta)
}
// applyMetadataEdits applies --set-metadata and --unset-metadata edits to existing metadata.
// Returns the merged JSON as json.RawMessage.
// Thin alias over the shared storage helper (also used in-transaction by issueops).
func applyMetadataEdits(existing json.RawMessage, setFlags, unsetFlags []string) (json.RawMessage, error) {
// Parse existing metadata (or start with empty object)
data := make(map[string]json.RawMessage)
if len(existing) > 0 {
trimmed := strings.TrimSpace(string(existing))
if trimmed != "" && trimmed != "null" {
if err := json.Unmarshal(existing, &data); err != nil {
return nil, fmt.Errorf("existing metadata is not a JSON object: %w", err)
}
}
}
// Apply --set-metadata key=value pairs
for _, kv := range setFlags {
k, v, ok := strings.Cut(kv, "=")
if !ok || k == "" {
return nil, fmt.Errorf("invalid --set-metadata: expected key=value, got %q", kv)
}
if err := storage.ValidateMetadataKey(k); err != nil {
return nil, err
}
// Store as JSON value: try to preserve type (number, bool, null)
data[k] = toJSONValue(v)
}
// Apply --unset-metadata keys
for _, k := range unsetFlags {
if err := storage.ValidateMetadataKey(k); err != nil {
return nil, err
}
delete(data, k)
}
result, err := json.Marshal(data)
if err != nil {
return nil, fmt.Errorf("failed to marshal metadata: %w", err)
}
return json.RawMessage(result), nil
return storage.ApplyMetadataEdits(existing, setFlags, unsetFlags)
}
// toJSONValue stores a CLI metadata value as a JSON string.
// Previous behavior inferred types (numbers, booleans) from content,
// which silently broke map[string]string round-trips (GH#4146).
func toJSONValue(s string) json.RawMessage {
b, _ := json.Marshal(s)
return json.RawMessage(b)
return storage.MetadataEditValue(s)
}
func init() {

View file

@ -243,138 +243,3 @@ func TestMultiIDUpdateAllGoodPathUnchanged(t *testing.T) {
}
}
}
// seedNonObjectMetadata sets an issue's metadata to a JSON array so a later
// --metadata merge or --set-metadata edit for that ID fails ("existing metadata
// is not a JSON object"). --metadata only requires valid JSON, not an object,
// so the non-object value persists and reproduces the mid-batch metadata
// failure the per-ID contract must survive.
func seedNonObjectMetadata(t *testing.T, bd, dir, id string) {
t.Helper()
stdout, stderr, code := runBDMultiID(t, bd, dir, "update", id, "--metadata", "[]")
if code != 0 {
t.Fatalf("seeding non-object metadata on %s failed (exit %d):\nstdout:\n%s\nstderr:\n%s", id, code, stdout, stderr)
}
}
// showMultiIDUpdateMetadataValue returns a string-valued metadata key for an
// issue via bd show --json, failing the test if the key is missing or not a
// JSON string. Used to confirm that IDs surrounding a failed one still received
// their metadata update.
func showMultiIDUpdateMetadataValue(t *testing.T, bd, dir, id, key string) string {
t.Helper()
stdout, stderr, code := runBDMultiID(t, bd, dir, "show", id, "--json")
if code != 0 {
t.Fatalf("bd show %s failed (exit %d):\nstdout:\n%s\nstderr:\n%s", id, code, stdout, stderr)
}
var details []struct {
ID string `json:"id"`
Metadata map[string]json.RawMessage `json:"metadata"`
}
if err := json.Unmarshal([]byte(stdout), &details); err != nil {
t.Fatalf("parsing show --json output for %s: %v\n%s", id, err, stdout)
}
if len(details) != 1 || details[0].ID != id {
t.Fatalf("show --json for %s returned unexpected issues:\n%s", id, stdout)
}
raw, ok := details[0].Metadata[key]
if !ok {
t.Fatalf("issue %s metadata has no key %q:\n%s", id, key, stdout)
}
var val string
if err := json.Unmarshal(raw, &val); err != nil {
t.Fatalf("issue %s metadata[%q] = %s, not a JSON string: %v", id, key, raw, err)
}
return val
}
// TestMultiIDUpdatePartialMetadataMergeFailure covers a --metadata merge that
// fails for one middle ID (its stored metadata is not a JSON object). The merge
// failure must be a per-ID failure — reported on stderr with a nonzero exit —
// not a batch abort that skips the trailing IDs.
func TestMultiIDUpdatePartialMetadataMergeFailure(t *testing.T) {
bd, dir := setupMultiIDUpdateDB(t)
id1 := createMultiIDUpdateIssue(t, bd, dir, "first issue")
id2 := createMultiIDUpdateIssue(t, bd, dir, "second issue")
id3 := createMultiIDUpdateIssue(t, bd, dir, "third issue")
seedNonObjectMetadata(t, bd, dir, id2)
stdout, stderr, code := runBDMultiID(t, bd, dir, "update", id1, id2, id3, "--metadata", `{"team":"platform"}`)
if code == 0 {
t.Errorf("bd update with a mid-batch metadata-merge failure exited 0, want nonzero\nstdout:\n%s\nstderr:\n%s", stdout, stderr)
}
if !strings.Contains(stderr, id2) {
t.Errorf("stderr does not mention the failed ID %s:\n%s", id2, stderr)
}
// Per-ID, not atomic: the IDs on either side of the failure must still carry
// the merged metadata.
for _, id := range []string{id1, id3} {
if got := showMultiIDUpdateMetadataValue(t, bd, dir, id, "team"); got != "platform" {
t.Errorf("issue %s metadata team = %q, want \"platform\" (successful IDs must stay applied)", id, got)
}
}
}
// TestMultiIDUpdatePartialMetadataEditFailureJSON covers a --set-metadata edit
// that fails for one middle ID under --json. stdout must keep the success array
// for the applied IDs while the failed ID is reported only in the JSON failure
// line on stderr, with a nonzero exit.
func TestMultiIDUpdatePartialMetadataEditFailureJSON(t *testing.T) {
bd, dir := setupMultiIDUpdateDB(t)
id1 := createMultiIDUpdateIssue(t, bd, dir, "first issue")
id2 := createMultiIDUpdateIssue(t, bd, dir, "second issue")
id3 := createMultiIDUpdateIssue(t, bd, dir, "third issue")
seedNonObjectMetadata(t, bd, dir, id2)
stdout, stderr, code := runBDMultiID(t, bd, dir, "update", id1, id2, id3, "--set-metadata", "team=platform", "--json")
if code == 0 {
t.Errorf("bd update --json with a mid-batch metadata-edit failure exited 0, want nonzero\nstdout:\n%s\nstderr:\n%s", stdout, stderr)
}
// stdout keeps the array-of-updated-issues success shape for the IDs that
// applied; the failed ID must not appear there.
var updated []struct {
ID string `json:"id"`
}
if err := json.Unmarshal([]byte(stdout), &updated); err != nil {
t.Fatalf("stdout is not the array-of-issues success shape: %v\n%s", err, stdout)
}
gotIDs := map[string]bool{}
for _, u := range updated {
gotIDs[u.ID] = true
}
if !gotIDs[id1] || !gotIDs[id3] || len(updated) != 2 {
t.Errorf("stdout array = %v, want exactly [%s %s]", updated, id1, id3)
}
if gotIDs[id2] {
t.Errorf("failed ID %s must not appear in the stdout success array:\n%s", id2, stdout)
}
// The last stderr line is a machine-parseable failure report naming the
// failed ID.
lines := strings.Split(strings.TrimSpace(stderr), "\n")
last := lines[len(lines)-1]
var report struct {
Error string `json:"error"`
Failed []struct {
ID string `json:"id"`
Error string `json:"error"`
} `json:"failed"`
}
if err := json.Unmarshal([]byte(last), &report); err != nil {
t.Fatalf("last stderr line is not a JSON failure report: %v\nstderr:\n%s", err, stderr)
}
if len(report.Failed) != 1 || report.Failed[0].ID != id2 {
t.Errorf("JSON failure report failed list = %+v, want exactly one entry for %s", report.Failed, id2)
}
if len(report.Failed) == 1 && report.Failed[0].Error == "" {
t.Errorf("JSON failure report entry has empty per-ID error: %s", last)
}
// The IDs surrounding the failure still carry the edited metadata.
for _, id := range []string{id1, id3} {
if got := showMultiIDUpdateMetadataValue(t, bd, dir, id, "team"); got != "platform" {
t.Errorf("issue %s metadata team = %q, want \"platform\"", id, got)
}
}
}

View file

@ -1323,6 +1323,140 @@ func TestProxiedServerUpdateConcurrentClaim(t *testing.T) {
}
}
// TestProxiedServerUpdateConcurrentMetadata is the proxied-server flavor of
// the concurrent-metadata lost-update regression test: real bd processes in
// proxied-server mode racing `bd update --set-metadata` with DISTINCT keys on
// the SAME issue. Every writer that exits 0 must have its key in the final
// metadata, and the seed key must survive.
//
// Before the fix this route lost writes two ways: buildUpdateSpecForIssue
// merged metadata from the unit of work's MVCC snapshot (erasing keys a
// concurrent writer committed after the snapshot), and when Dolt's commit-time
// merge rolled the loser back, uow.CommitWithRetries re-committed the
// now-empty session — Dolt said "nothing to commit", the handler swallowed it,
// printed "✓ Updated", and exited 0 with the write gone.
func TestProxiedServerUpdateConcurrentMetadata(t *testing.T) {
requireProxiedServerEnv(t)
bd := buildEmbeddedBD(t)
p := bdProxiedInit(t, bd, "umr")
issue := bdProxiedCreate(t, bd, p.dir, "Concurrent metadata target")
bdProxiedUpdateOne(t, bd, p.dir, issue.ID, "--metadata", `{"seed":"yes"}`)
const writers = 4
const rounds = 3
type result struct {
key string
exitErr error
combined string
}
var results []result
for round := 0; round < rounds; round++ {
roundResults := make([]result, writers)
var wg sync.WaitGroup
for i := 0; i < writers; i++ {
i := i
key := fmt.Sprintf("w%dr%d", i, round)
wg.Add(1)
go func() {
defer wg.Done()
stdout, stderr, err := bdProxiedUpdateRaw(t, bd, p.dir,
issue.ID, "--set-metadata", key+"=1")
roundResults[i] = result{key: key, exitErr: err, combined: stdout + stderr}
}()
}
wg.Wait()
results = append(results, roundResults...)
}
final := bdProxiedShow(t, bd, p.dir, issue.ID)
got := map[string]any{}
if len(final.Metadata) > 0 {
if err := json.Unmarshal(final.Metadata, &got); err != nil {
t.Fatalf("parse final metadata %q: %v", final.Metadata, err)
}
}
if got["seed"] != "yes" {
t.Errorf("seed metadata key erased by concurrent --set-metadata writers: %v", got)
}
var lost []string
for _, r := range results {
if r.exitErr == nil {
if _, ok := got[r.key]; !ok {
lost = append(lost, r.key)
}
if strings.Contains(r.combined, "retries exhausted") {
t.Errorf("writer %s exited 0 but reported exhausted retries:\n%s", r.key, r.combined)
}
continue
}
// A loud loss is acceptable; a silent one is the defect.
isConflictFailure := strings.Contains(r.combined, "retries exhausted") ||
strings.Contains(r.combined, "serialization failure") ||
strings.Contains(r.combined, "Error 1213")
if !isConflictFailure {
t.Errorf("writer %s failed for an unexpected reason: err=%v\n%s", r.key, r.exitErr, r.combined)
}
}
if len(lost) > 0 {
t.Fatalf("silent lost update: %d exit-0 --set-metadata writes missing from final metadata: %v (got %v)",
len(lost), lost, got)
}
}
// TestProxiedServerUpdateConcurrentAppendNotes covers the notes flavor of the
// same defect: concurrent `bd update --append-notes` writers on one issue.
// Every line whose writer exited 0 must appear in the final notes.
func TestProxiedServerUpdateConcurrentAppendNotes(t *testing.T) {
requireProxiedServerEnv(t)
bd := buildEmbeddedBD(t)
p := bdProxiedInit(t, bd, "unr")
issue := bdProxiedCreate(t, bd, p.dir, "Concurrent notes target", "--notes", "seed line")
const writers = 4
type result struct {
line string
exitErr error
combined string
}
results := make([]result, writers)
var wg sync.WaitGroup
for i := 0; i < writers; i++ {
i := i
line := fmt.Sprintf("appended by writer %d", i)
wg.Add(1)
go func() {
defer wg.Done()
stdout, stderr, err := bdProxiedUpdateRaw(t, bd, p.dir,
issue.ID, "--append-notes", line)
results[i] = result{line: line, exitErr: err, combined: stdout + stderr}
}()
}
wg.Wait()
final := bdProxiedShow(t, bd, p.dir, issue.ID)
if !strings.Contains(final.Notes, "seed line") {
t.Errorf("seed notes erased by concurrent --append-notes writers: %q", final.Notes)
}
for _, r := range results {
if r.exitErr != nil {
isConflictFailure := strings.Contains(r.combined, "retries exhausted") ||
strings.Contains(r.combined, "serialization failure") ||
strings.Contains(r.combined, "Error 1213")
if !isConflictFailure {
t.Errorf("writer failed for an unexpected reason: err=%v\n%s", r.exitErr, r.combined)
}
continue
}
if !strings.Contains(final.Notes, r.line) {
t.Errorf("silent lost update: exit-0 append %q missing from final notes %q", r.line, final.Notes)
}
}
}
func readIsBlocked(t *testing.T, db *sql.DB, id string) bool {
t.Helper()
var v int

View file

@ -6,18 +6,26 @@ import (
"fmt"
"os"
"path/filepath"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/spf13/cobra"
"github.com/steveyegge/beads/internal/hooks"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/domain"
"github.com/steveyegge/beads/internal/storage/fs"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/storage/uow"
"github.com/steveyegge/beads/internal/types"
"github.com/steveyegge/beads/internal/ui"
)
// proxiedUpdateRetryMaxElapsed bounds the whole-attempt retry loop for one
// issue's update (matches uow.CommitWithRetries' budget). A var so tests can
// shrink it when exercising conflict exhaustion.
var proxiedUpdateRetryMaxElapsed = 15 * time.Second
func runUpdateProxiedServer(cmd *cobra.Command, ctx context.Context, args []string) error {
if len(args) == 0 {
return HandleErrorRespectJSON("no issue ID provided")
@ -40,7 +48,7 @@ func runUpdateProxiedServer(cmd *cobra.Command, ctx context.Context, args []stri
var updated []*types.Issue
// failures accumulates every requested ID that could not be updated —
// generic per-ID errors as well as a lost --claim race. In a mixed batch a
// later winner used to flip the exit code back to success and hide the
// later winner must NOT flip the exit code back to success and hide the
// failed IDs from exit-code automation; report them all and exit non-zero,
// mirroring the non-proxied path in update.go (beads audit finding #10).
var failures []updateIDFailure
@ -70,72 +78,123 @@ func runUpdateProxiedServer(cmd *cobra.Command, ctx context.Context, args []stri
return nil
}
type updateProxiedResult struct {
before *types.Issue
after *types.Issue
updated bool
}
// applyUpdateProxiedOne applies one ID's update on the proxied path. On success
// it returns the updated issue and an empty failReason. On a per-ID failure it
// returns a non-empty failReason (already printed inline on stderr), including a
// lost --claim race (already-claimed / not-claimable), so the caller records the
// failed ID and exits non-zero — matching the non-proxied path. A non-nil error
// is reserved for a hard, batch-aborting condition (uninitialized UOW provider).
// applyUpdateProxiedOne applies one issue's update, redoing the WHOLE
// read-merge-write in a fresh unit of work when Dolt reports a serialization
// failure (the withRetryTx idiom from internal/storage/dolt).
//
// The retry must wrap the whole attempt, never just the commit: a
// serialization failure means the server already rolled the transaction back,
// so re-committing the same session (the old uow.CommitWithRetries call) can
// only ever produce "nothing to commit" — which the old code swallowed,
// printing "✓ Updated" and exiting 0 for a write that was silently lost.
// Redoing the attempt re-reads the winner's committed row, so merge
// operations (metadata edits, note appends) resolve against authoritative
// state instead of erasing it.
func applyUpdateProxiedOne(ctx context.Context, id string, in *updateInput) (*types.Issue, string, error) {
if uowProvider == nil {
return nil, "", HandleError("proxied-server UOW provider not initialized")
}
var claimLost bool
res, err := uow.RunTxResult(ctx, uowProvider, func(ctx context.Context, uw uow.UnitOfWork) (updateProxiedResult, string, error) {
issueUC := uw.IssueUseCase()
current, err := issueUC.GetIssue(ctx, id)
if err != nil || current == nil {
wispCurrent, wispErr := issueUC.GetWisp(ctx, id)
if wispErr == nil && wispCurrent != nil {
current = wispCurrent
} else if err != nil {
return updateProxiedResult{}, "", fmt.Errorf("resolving %s: %w", id, err)
} else {
return updateProxiedResult{}, "", fmt.Errorf("issue %s not found", id)
}
var issue *types.Issue
var failReason string
bo := backoff.NewExponentialBackOff()
bo.InitialInterval = 25 * time.Millisecond
bo.MaxElapsedTime = proxiedUpdateRetryMaxElapsed
err := backoff.Retry(func() error {
var retryable bool
var attemptErr error
issue, failReason, retryable, attemptErr = applyUpdateProxiedAttempt(ctx, id, in)
if attemptErr == nil {
return nil
}
if err := validateIssueUpdatable(id, current); err != nil {
return updateProxiedResult{}, "", err
if retryable {
return attemptErr
}
spec, err := buildUpdateSpecForIssue(current, in)
if err != nil {
return updateProxiedResult{}, "", err
}
updated, err := issueUC.ApplyUpdate(ctx, id, spec, actor)
if err != nil {
if errors.Is(err, storage.ErrAlreadyClaimed) || errors.Is(err, storage.ErrNotClaimable) {
claimLost = in.claim
return updateProxiedResult{}, "", err
}
return updateProxiedResult{}, "", fmt.Errorf("updating %s: %w", id, err)
}
return updateProxiedResult{before: current, after: updated, updated: true}, fmt.Sprintf("bd: update %s", id), nil
})
return backoff.Permanent(attemptErr)
}, backoff.WithContext(bo, ctx))
if err != nil {
if claimLost {
fmt.Fprintf(os.Stderr, "Error claiming %s: %v\n", id, err)
return nil, fmt.Sprintf("claiming issue: %v", err), nil
if uow.IsSerializationError(err) {
// Retries exhausted while losing Dolt's commit-time merge. The
// write did NOT land; fail loudly instead of exiting 0.
fmt.Fprintf(os.Stderr, "Error updating %s: retries exhausted on write conflicts: %v\n", id, err)
return nil, fmt.Sprintf("retries exhausted on write conflicts: %v", err), nil
}
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
return nil, err.Error(), nil
return nil, "", err
}
return issue, failReason, nil
}
// applyUpdateProxiedAttempt runs one full read-merge-write attempt in a fresh
// unit of work. retryable is true only for serialization failures, where the
// server-side rollback guarantees nothing landed and the whole attempt is safe
// to redo. Terminal per-issue failures (not found, claim conflicts, commit
// errors) print to stderr and return a non-empty failReason with no error, so
// the multi-ID loop records the failed ID, keeps going, and still exits
// non-zero — matching the non-proxied path.
func applyUpdateProxiedAttempt(ctx context.Context, id string, in *updateInput) (issue *types.Issue, failReason string, retryable bool, err error) {
uw, err := uowProvider.NewUOW(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "Error opening unit of work for %s: %v\n", id, err)
return nil, fmt.Sprintf("opening unit of work: %v", err), false, nil
}
defer uw.Close(ctx)
issueUC := uw.IssueUseCase()
current, err := issueUC.GetIssue(ctx, id)
if err != nil || current == nil {
wispCurrent, wispErr := issueUC.GetWisp(ctx, id)
if wispErr == nil && wispCurrent != nil {
current = wispCurrent
} else if err != nil {
fmt.Fprintf(os.Stderr, "Error resolving %s: %v\n", id, err)
return nil, fmt.Sprintf("resolving issue: %v", err), false, nil
} else {
fmt.Fprintf(os.Stderr, "Issue %s not found\n", id)
return nil, "issue not found", false, nil
}
}
if err := validateIssueUpdatable(id, current); err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
return nil, err.Error(), false, nil
}
if res.updated {
if err := fireProxiedUpdateHooks(ctx, res.before, res.after); err != nil {
fmt.Fprintf(os.Stderr, "warning: %s: %v\n", id, err)
spec := buildUpdateSpecForIssue(current, in)
updated, err := issueUC.ApplyUpdate(ctx, id, spec, actor)
if err != nil {
if uow.IsSerializationError(err) {
return nil, "", true, err
}
if errors.Is(err, storage.ErrAlreadyClaimed) || errors.Is(err, storage.ErrNotClaimable) {
fmt.Fprintf(os.Stderr, "Error claiming %s: %v\n", id, err)
return nil, fmt.Sprintf("claiming issue: %v", err), false, nil
}
fmt.Fprintf(os.Stderr, "Error updating %s: %v\n", id, err)
return nil, fmt.Sprintf("updating: %v", err), false, nil
}
return res.after, "", nil
if err := uw.Commit(ctx, fmt.Sprintf("bd: update %s", id)); err != nil {
if uow.IsSerializationError(err) {
// Dolt rolled the whole transaction back server-side; nothing
// landed. Signal the caller to redo the read-merge-write.
return nil, "", true, err
}
if !isDoltNothingToCommit(err) {
fmt.Fprintf(os.Stderr, "Error committing %s: %v\n", id, err)
return nil, fmt.Sprintf("committing: %v", err), false, nil
}
// "Nothing to commit" here is the legitimately-empty working set:
// wisp-only updates live in dolt_ignored tables, so a successful
// ApplyUpdate can leave nothing for the Dolt commit layer. The
// lost-write flavor — nothing-to-commit from re-committing a
// rolled-back session — cannot reach this branch because each attempt
// commits its own fresh unit of work exactly once.
}
if err := fireProxiedUpdateHooks(ctx, current, updated); err != nil {
fmt.Fprintf(os.Stderr, "warning: %s: %v\n", id, err)
}
return updated, "", false, nil
}
func fireProxiedUpdateHooks(ctx context.Context, before, after *types.Issue) error {
@ -178,7 +237,11 @@ func proxiedHookRunner(ctx context.Context) (*hooks.Runner, error) {
return hooks.NewRunner(filepath.Join(resolution.BeadsDir, "hooks")), nil
}
func buildUpdateSpecForIssue(current *types.Issue, in *updateInput) (domain.UpdateSpec, error) {
// buildUpdateSpecForIssue translates gathered CLI input into a domain
// UpdateSpec. It never pre-merges row state: merge-shaped edits are passed as
// operation keys and resolved by the repository inside the mutation
// transaction.
func buildUpdateSpecForIssue(current *types.Issue, in *updateInput) domain.UpdateSpec {
fields := make(map[string]any, len(in.fields))
for k, v := range in.fields {
fields[k] = v
@ -187,30 +250,26 @@ func buildUpdateSpecForIssue(current *types.Issue, in *updateInput) (domain.Upda
if in.clearDeferStatus && current.Status == types.StatusDeferred {
fields["status"] = string(types.StatusOpen)
}
// Metadata edits and note appends pass through as merge OPERATIONS: the
// repository resolves them against the row re-read inside the mutation
// transaction (issueops.ResolveMergeOps via the domain/db Update path).
// Merging here against `current` — a read from this unit of work's MVCC
// snapshot — silently erased keys a concurrent writer committed after our
// snapshot was taken: both processes exited 0, one write vanished.
if in.hasAppendNotes {
combined := current.Notes
if combined != "" {
combined += "\n"
}
combined += in.appendNotes
fields["notes"] = combined
fields[issueops.OpAppendNotes] = in.appendNotes
}
if len(in.mergeMetadataIn) > 0 {
merged, err := mergeMetadata(current.Metadata, in.mergeMetadataIn)
if err != nil {
return domain.UpdateSpec{}, fmt.Errorf("metadata merge failed for %s: %w", current.ID, err)
}
fields["metadata"] = merged
fields[issueops.OpMergeMetadata] = in.mergeMetadataIn
}
if len(in.setMetadata) > 0 || len(in.unsetMetadata) > 0 {
merged, err := applyMetadataEdits(current.Metadata, in.setMetadata, in.unsetMetadata)
if err != nil {
return domain.UpdateSpec{}, fmt.Errorf("metadata edit failed for %s: %w", current.ID, err)
}
fields["metadata"] = merged
if len(in.setMetadata) > 0 {
fields[issueops.OpSetMetadata] = in.setMetadata
}
if len(in.unsetMetadata) > 0 {
fields[issueops.OpUnsetMetadata] = in.unsetMetadata
}
spec := domain.UpdateSpec{
return domain.UpdateSpec{
Fields: fields,
Claim: in.claim,
AddLabels: in.addLabels,
@ -218,5 +277,4 @@ func buildUpdateSpecForIssue(current *types.Issue, in *updateInput) (domain.Upda
SetLabels: in.setLabels,
Reparent: in.reparent,
}
return spec, nil
}

View file

@ -0,0 +1,277 @@
package main
import (
"context"
"encoding/json"
"errors"
"io"
"os"
"strings"
"sync/atomic"
"testing"
"time"
mysql "github.com/go-sql-driver/mysql"
"github.com/steveyegge/beads/internal/hooks"
"github.com/steveyegge/beads/internal/storage/domain"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/storage/uow"
"github.com/steveyegge/beads/internal/types"
)
// TestBuildUpdateSpecForIssue_RoutesMergeOpsAsOperations locks in the
// lost-update fix on the proxied-server route: metadata edits and note
// appends must flow into spec.Fields as merge OPERATION keys, never as values
// pre-merged from `current` (a read from this unit of work's MVCC snapshot —
// merging there erased keys a concurrent writer committed after the snapshot).
func TestBuildUpdateSpecForIssue_RoutesMergeOpsAsOperations(t *testing.T) {
current := &types.Issue{
ID: "bd-spec-1",
Status: types.StatusOpen,
Notes: "existing notes",
Metadata: json.RawMessage(`{"existing":"yes"}`),
}
t.Run("append_notes", func(t *testing.T) {
in := &updateInput{
fields: map[string]any{},
hasAppendNotes: true,
appendNotes: "appended",
}
spec := buildUpdateSpecForIssue(current, in)
if got, ok := spec.Fields[issueops.OpAppendNotes]; !ok || got != "appended" {
t.Errorf("Fields[%s] = %v (ok=%v), want the raw append text", issueops.OpAppendNotes, got, ok)
}
if merged, ok := spec.Fields["notes"]; ok {
t.Errorf("Fields[notes] = %v: notes must NOT be pre-merged from the snapshot", merged)
}
})
t.Run("merge_metadata", func(t *testing.T) {
in := &updateInput{
fields: map[string]any{},
mergeMetadataIn: json.RawMessage(`{"new":"key"}`),
}
spec := buildUpdateSpecForIssue(current, in)
raw, ok := spec.Fields[issueops.OpMergeMetadata].(json.RawMessage)
if !ok || string(raw) != `{"new":"key"}` {
t.Errorf("Fields[%s] = %v, want the raw incoming JSON", issueops.OpMergeMetadata, spec.Fields[issueops.OpMergeMetadata])
}
if merged, ok := spec.Fields["metadata"]; ok {
t.Errorf("Fields[metadata] = %v: metadata must NOT be pre-merged from the snapshot", merged)
}
})
t.Run("set_and_unset_metadata", func(t *testing.T) {
in := &updateInput{
fields: map[string]any{},
setMetadata: []string{"tier=gold"},
unsetMetadata: []string{"existing"},
}
spec := buildUpdateSpecForIssue(current, in)
set, _ := spec.Fields[issueops.OpSetMetadata].([]string)
unset, _ := spec.Fields[issueops.OpUnsetMetadata].([]string)
if len(set) != 1 || set[0] != "tier=gold" {
t.Errorf("Fields[%s] = %v, want [tier=gold]", issueops.OpSetMetadata, spec.Fields[issueops.OpSetMetadata])
}
if len(unset) != 1 || unset[0] != "existing" {
t.Errorf("Fields[%s] = %v, want [existing]", issueops.OpUnsetMetadata, spec.Fields[issueops.OpUnsetMetadata])
}
if merged, ok := spec.Fields["metadata"]; ok {
t.Errorf("Fields[metadata] = %v: metadata must NOT be pre-merged from the snapshot", merged)
}
})
t.Run("clear_defer_status_still_resolved_from_current", func(t *testing.T) {
deferred := &types.Issue{ID: "bd-spec-2", Status: types.StatusDeferred}
in := &updateInput{fields: map[string]any{}, clearDeferStatus: true}
spec := buildUpdateSpecForIssue(deferred, in)
if got := spec.Fields["status"]; got != string(types.StatusOpen) {
t.Errorf("Fields[status] = %v, want open (clearDeferStatus on a deferred issue)", got)
}
})
}
// --- fakes for the whole-attempt retry tests ---
type fakeUpdateIssueUC struct {
domain.IssueUseCase // unimplemented methods panic; the attempt path must not call them
issue *types.Issue
}
func (f *fakeUpdateIssueUC) GetIssue(ctx context.Context, id string) (*types.Issue, error) {
return f.issue, nil
}
func (f *fakeUpdateIssueUC) ApplyUpdate(ctx context.Context, id string, spec domain.UpdateSpec, actor string) (*types.Issue, error) {
return f.issue, nil
}
type fakeUOW struct {
issueUC domain.IssueUseCase
commit func() error
}
func (f *fakeUOW) Close(ctx context.Context) {}
func (f *fakeUOW) Commit(ctx context.Context, message string) error { return f.commit() }
func (f *fakeUOW) ConfigUseCase() domain.ConfigUseCase { return nil }
func (f *fakeUOW) DoltRemoteUseCase() domain.DoltRemoteUseCase { return nil }
func (f *fakeUOW) BootstrapUseCase() domain.BootstrapUseCase { return nil }
func (f *fakeUOW) IssueUseCase() domain.IssueUseCase { return f.issueUC }
func (f *fakeUOW) DependencyUseCase() domain.DependencyUseCase { return nil }
func (f *fakeUOW) LabelUseCase() domain.LabelUseCase { return nil }
func (f *fakeUOW) CommentUseCase() domain.CommentUseCase { return nil }
func (f *fakeUOW) RawSQLUseCase() domain.RawSQLUseCase { return nil }
type fakeUOWProvider struct {
uows atomic.Int64
commit func(attempt int64) error
issue *types.Issue
}
func (p *fakeUOWProvider) NewUOW(ctx context.Context) (uow.UnitOfWork, error) {
n := p.uows.Add(1)
return &fakeUOW{
issueUC: &fakeUpdateIssueUC{issue: p.issue},
commit: func() error { return p.commit(n) },
}, nil
}
func (p *fakeUOWProvider) Close(ctx context.Context) error { return nil }
func withFakeProxiedUpdateEnv(t *testing.T, p *fakeUOWProvider) {
t.Helper()
oldProvider := uowProvider
oldHookRunner := hookRunner
uowProvider = p
hookRunner = hooks.NewRunner(t.TempDir()) // no hooks installed: RunSync no-ops
t.Cleanup(func() {
uowProvider = oldProvider
hookRunner = oldHookRunner
})
}
func captureStderrDuring(t *testing.T, fn func()) string {
t.Helper()
old := os.Stderr
r, w, err := os.Pipe()
if err != nil {
t.Fatalf("os.Pipe: %v", err)
}
os.Stderr = w
done := make(chan string, 1)
go func() {
data, _ := io.ReadAll(r)
done <- string(data)
}()
defer func() {
os.Stderr = old
_ = r.Close()
}()
fn()
_ = w.Close()
return <-done
}
func serializationFailure() error {
return &mysql.MySQLError{Number: 1213, Message: "serialization failure, transaction rolled back"}
}
// TestApplyUpdateProxiedOne_RetriesWholeAttemptOnConflict proves a Dolt
// serialization failure at commit time redoes the WHOLE read-merge-write in a
// fresh unit of work — not just the commit. Re-committing the rolled-back
// session was the old behavior; the server had already discarded the writes,
// so the retry could only report "nothing to commit" and the update vanished.
func TestApplyUpdateProxiedOne_RetriesWholeAttemptOnConflict(t *testing.T) {
issue := &types.Issue{ID: "bd-retry-1", Status: types.StatusOpen}
provider := &fakeUOWProvider{
issue: issue,
commit: func(attempt int64) error {
if attempt <= 2 {
return serializationFailure()
}
return nil
},
}
withFakeProxiedUpdateEnv(t, provider)
in := &updateInput{fields: map[string]any{"title": "renamed"}}
got, failReason, err := applyUpdateProxiedOne(context.Background(), "bd-retry-1", in)
if err != nil {
t.Fatalf("applyUpdateProxiedOne: %v", err)
}
if failReason != "" {
t.Fatalf("expected update to succeed after conflict retries, got failure: %s", failReason)
}
if got == nil || got.ID != "bd-retry-1" {
t.Fatalf("updated issue = %v, want bd-retry-1", got)
}
if n := provider.uows.Load(); n != 3 {
t.Errorf("unit-of-work attempts = %d, want 3 (each conflict must redo the whole read-merge-write)", n)
}
}
// TestApplyUpdateProxiedOne_ExhaustedConflictsFailLoudly proves a write that
// never lands cannot exit as a success: when every attempt loses Dolt's
// commit-time merge, the command reports the failure instead of printing
// "✓ Updated" (a non-empty failReason suppresses the success line and drives
// the non-zero exit via reportUpdateFailures in runUpdateProxiedServer).
func TestApplyUpdateProxiedOne_ExhaustedConflictsFailLoudly(t *testing.T) {
oldMax := proxiedUpdateRetryMaxElapsed
proxiedUpdateRetryMaxElapsed = 150 * time.Millisecond
t.Cleanup(func() { proxiedUpdateRetryMaxElapsed = oldMax })
issue := &types.Issue{ID: "bd-retry-2", Status: types.StatusOpen}
provider := &fakeUOWProvider{
issue: issue,
commit: func(int64) error { return serializationFailure() },
}
withFakeProxiedUpdateEnv(t, provider)
var got *types.Issue
var failReason string
var err error
stderr := captureStderrDuring(t, func() {
in := &updateInput{fields: map[string]any{"title": "never lands"}}
got, failReason, err = applyUpdateProxiedOne(context.Background(), "bd-retry-2", in)
})
if err != nil {
t.Fatalf("applyUpdateProxiedOne returned hard error: %v", err)
}
if failReason == "" || got != nil {
t.Fatalf("failReason=%q issue=%v: a write that never landed must be reported as a failure", failReason, got)
}
if !strings.Contains(stderr, "retries exhausted") {
t.Errorf("stderr = %q, want a loud retries-exhausted failure", stderr)
}
if n := provider.uows.Load(); n < 2 {
t.Errorf("unit-of-work attempts = %d, want at least 2 before giving up", n)
}
}
// TestApplyUpdateProxiedOne_NothingToCommitWithoutConflictSucceeds preserves
// the one legitimate nothing-to-commit case: wisp-only updates write to
// dolt_ignored tables, so a successful ApplyUpdate can leave the Dolt commit
// layer with nothing to do. That is a success, not a lost write — no
// serialization failure preceded it.
func TestApplyUpdateProxiedOne_NothingToCommitWithoutConflictSucceeds(t *testing.T) {
issue := &types.Issue{ID: "bd-retry-3", Status: types.StatusOpen}
provider := &fakeUOWProvider{
issue: issue,
commit: func(int64) error { return errors.New("nothing to commit") },
}
withFakeProxiedUpdateEnv(t, provider)
in := &updateInput{fields: map[string]any{"title": "wisp-shaped"}}
got, failReason, err := applyUpdateProxiedOne(context.Background(), "bd-retry-3", in)
if err != nil {
t.Fatalf("applyUpdateProxiedOne: %v", err)
}
if failReason != "" || got == nil {
t.Fatalf("failReason=%q issue=%v: empty-working-set commit on an unconflicted attempt is a success", failReason, got)
}
if n := provider.uows.Load(); n != 1 {
t.Errorf("unit-of-work attempts = %d, want 1 (nothing-to-commit must not trigger retries)", n)
}
}

View file

@ -95,6 +95,7 @@ func (r *issueSQLRepositoryImpl) Update(ctx context.Context, id string, updates
table := pickIssueTable(opts.UseWispsTable)
_, statusChanging := updates["status"]
mergeOps := issueops.HasMergeOps(updates)
// When the status changes we need the prior row to reproduce the embedded
// lifecycle side effects (issueops.updateIssueInTx): closed_at is set on
@ -102,8 +103,10 @@ func (r *issueSQLRepositoryImpl) Update(ctx context.Context, id string, updates
// transition, the audit event type is derived from the transition, and
// is_blocked is recomputed for neighbors. Read the full old issue once so
// all four use the same snapshot; the ErrNoRows contract is preserved.
// Merge operations (metadata edits, note appends) need the same read: they
// are resolved against the row as seen by THIS unit-of-work transaction.
var oldIssue *types.Issue
if statusChanging {
if statusChanging || mergeOps {
var err error
oldIssue, err = r.Get(ctx, id, opts)
if err != nil {
@ -114,6 +117,24 @@ func (r *issueSQLRepositoryImpl) Update(ctx context.Context, id string, updates
}
}
// Resolve read-merge-write operation keys (issueops.OpMergeMetadata,
// OpSetMetadata, OpUnsetMetadata, OpAppendNotes) into concrete column
// values inside the mutation transaction, mirroring the embedded path
// (issueops.updateIssueInTx). Callers must pass the OPERATION, never a
// value pre-merged from an earlier read: this runner is a Dolt sql-server
// session where FOR UPDATE is a parse-only no-op, so a stale-snapshot merge
// is only made safe by Dolt's commit-time conflict detection plus the
// caller redoing the whole unit of work on a serialization failure — and
// that redo re-runs this in-transaction resolution against the winner's
// committed row.
if mergeOps {
resolved, err := issueops.ResolveMergeOps(oldIssue, updates)
if err != nil {
return fmt.Errorf("db: Update %s: %w", id, err)
}
updates = resolved
}
setClauses := make([]string, 0, len(updates)+3)
args := make([]any, 0, len(updates)+4)
for key, value := range updates {

View file

@ -0,0 +1,201 @@
package db
import (
"encoding/json"
"github.com/steveyegge/beads/internal/storage/domain"
"github.com/steveyegge/beads/internal/storage/issueops"
)
// TestIssueSQLRepository_MergeOps covers the read-merge-write operation keys
// (metadata edits, note appends) on the domain/db repository — the write path
// behind the proxied-server CLI. The ops must be resolved against the row read
// inside the SAME unit-of-work transaction, not against a snapshot the caller
// read earlier: pre-merged values from a stale MVCC snapshot are exactly the
// lost-update defect (concurrent writers erased each other's committed keys
// while both exited 0).
func (s *testSuite) TestIssueSQLRepository_MergeOps() {
s.Run("SetMetadataMergesIntoCurrentRow", s.issueUpdateSetMetadataMerges)
s.Run("MergeMetadataOverlaysTopLevelKeys", s.issueUpdateMergeMetadataOverlays)
s.Run("UnsetMetadataRemovesKeys", s.issueUpdateUnsetMetadataRemoves)
s.Run("AppendNotesConcatenates", s.issueUpdateAppendNotes)
s.Run("AppendNotesToEmptyHasNoLeadingNewline", s.issueUpdateAppendNotesEmpty)
s.Run("AppendNotesRejectsCombinedReplacement", s.issueUpdateAppendNotesConflict)
s.Run("SetMetadataRejectsCombinedReplacement", s.issueUpdateSetMetadataConflict)
s.Run("OpsRouteToWispsTable", s.issueUpdateMergeOpsWispRouting)
}
func (s *testSuite) seedIssueWithMetadata(id, metadata, notes string) {
r := s.issueRepo()
in := newTestIssue(id, "merge ops seed")
if metadata != "" {
in.Metadata = json.RawMessage(metadata)
}
in.Notes = notes
s.Require().NoError(r.Insert(s.Ctx(), in, "seeder", domain.InsertIssueOpts{}))
}
func (s *testSuite) readMetadataMap(id string, opts domain.IssueTableOpts) map[string]any {
out, err := s.issueRepo().Get(s.Ctx(), id, opts)
s.Require().NoError(err)
got := map[string]any{}
if len(out.Metadata) > 0 {
s.Require().NoError(json.Unmarshal(out.Metadata, &got))
}
return got
}
func (s *testSuite) issueUpdateSetMetadataMerges() {
s.seedIssueWithMetadata("bd-mo-set", `{"existing":"yes"}`, "")
r := s.issueRepo()
err := r.Update(s.Ctx(), "bd-mo-set", map[string]any{
issueops.OpSetMetadata: []string{"tier=gold", "score=99"},
}, "tester", domain.IssueTableOpts{})
s.Require().NoError(err)
got := s.readMetadataMap("bd-mo-set", domain.IssueTableOpts{})
s.Equal("yes", got["existing"], "pre-existing key must survive a set-metadata edit")
s.Equal("gold", got["tier"])
s.Equal("99", got["score"], "--set-metadata values are stored as JSON strings (GH#4146)")
}
func (s *testSuite) issueUpdateMergeMetadataOverlays() {
s.seedIssueWithMetadata("bd-mo-merge", `{"a":1,"b":2}`, "")
r := s.issueRepo()
err := r.Update(s.Ctx(), "bd-mo-merge", map[string]any{
issueops.OpMergeMetadata: json.RawMessage(`{"b":3,"c":4}`),
}, "tester", domain.IssueTableOpts{})
s.Require().NoError(err)
got := s.readMetadataMap("bd-mo-merge", domain.IssueTableOpts{})
s.Equal(float64(1), got["a"])
s.Equal(float64(3), got["b"])
s.Equal(float64(4), got["c"])
s.Len(got, 3)
}
func (s *testSuite) issueUpdateUnsetMetadataRemoves() {
s.seedIssueWithMetadata("bd-mo-unset", `{"keep":"yes","drop":"yes"}`, "")
r := s.issueRepo()
err := r.Update(s.Ctx(), "bd-mo-unset", map[string]any{
issueops.OpUnsetMetadata: []string{"drop"},
}, "tester", domain.IssueTableOpts{})
s.Require().NoError(err)
got := s.readMetadataMap("bd-mo-unset", domain.IssueTableOpts{})
s.Equal("yes", got["keep"])
s.NotContains(got, "drop")
}
func (s *testSuite) issueUpdateAppendNotes() {
s.seedIssueWithMetadata("bd-mo-notes", "", "first")
r := s.issueRepo()
err := r.Update(s.Ctx(), "bd-mo-notes", map[string]any{
issueops.OpAppendNotes: "second",
}, "tester", domain.IssueTableOpts{})
s.Require().NoError(err)
out, err := r.Get(s.Ctx(), "bd-mo-notes", domain.IssueTableOpts{})
s.Require().NoError(err)
s.Equal("first\nsecond", out.Notes)
}
func (s *testSuite) issueUpdateAppendNotesEmpty() {
s.seedIssueWithMetadata("bd-mo-notes-empty", "", "")
r := s.issueRepo()
err := r.Update(s.Ctx(), "bd-mo-notes-empty", map[string]any{
issueops.OpAppendNotes: "only",
}, "tester", domain.IssueTableOpts{})
s.Require().NoError(err)
out, err := r.Get(s.Ctx(), "bd-mo-notes-empty", domain.IssueTableOpts{})
s.Require().NoError(err)
s.Equal("only", out.Notes)
}
func (s *testSuite) issueUpdateAppendNotesConflict() {
s.seedIssueWithMetadata("bd-mo-notes-conf", "", "first")
r := s.issueRepo()
err := r.Update(s.Ctx(), "bd-mo-notes-conf", map[string]any{
issueops.OpAppendNotes: "second",
"notes": "replacement",
}, "tester", domain.IssueTableOpts{})
s.Require().Error(err)
s.Contains(err.Error(), "notes replacement")
}
func (s *testSuite) issueUpdateSetMetadataConflict() {
s.seedIssueWithMetadata("bd-mo-set-conf", `{"a":1}`, "")
r := s.issueRepo()
err := r.Update(s.Ctx(), "bd-mo-set-conf", map[string]any{
issueops.OpSetMetadata: []string{"b=2"},
"metadata": json.RawMessage(`{"c":3}`),
}, "tester", domain.IssueTableOpts{})
s.Require().Error(err)
s.Contains(err.Error(), "metadata replacement")
}
func (s *testSuite) issueUpdateMergeOpsWispRouting() {
r := s.issueRepo()
in := newTestIssue("bd-mo-wisp", "wisp merge ops")
in.Metadata = json.RawMessage(`{"kind":"wisp"}`)
in.Notes = "w1"
s.Require().NoError(r.Insert(s.Ctx(), in, "seeder", domain.InsertIssueOpts{UseWispsTable: true}))
err := r.Update(s.Ctx(), "bd-mo-wisp", map[string]any{
issueops.OpSetMetadata: []string{"extra=1"},
issueops.OpAppendNotes: "w2",
}, "tester", domain.IssueTableOpts{UseWispsTable: true})
s.Require().NoError(err)
out, err := r.Get(s.Ctx(), "bd-mo-wisp", domain.IssueTableOpts{UseWispsTable: true})
s.Require().NoError(err)
s.Equal("w1\nw2", out.Notes)
got := map[string]any{}
s.Require().NoError(json.Unmarshal(out.Metadata, &got))
s.Equal("wisp", got["kind"], "wisp's pre-existing metadata key must survive")
s.Equal("1", got["extra"])
}
// TestIssueUseCase_ApplyUpdateMergeOps proves the ops flow through the domain
// use-case layer — the exact call chain the proxied-server update handler uses
// (issueUC.ApplyUpdate with spec.Fields).
func (s *testSuite) TestIssueUseCase_ApplyUpdateMergeOps() {
s.Run("SetMetadataViaApplyUpdatePreservesSiblingKeys", s.iucApplyUpdateSetMetadata)
s.Run("AppendNotesViaApplyUpdate", s.iucApplyUpdateAppendNotes)
}
func (s *testSuite) iucApplyUpdateSetMetadata() {
s.seedIssueWithMetadata("bd-mo-uc", `{"sibling":"safe"}`, "")
updated, err := s.issueUseCase().ApplyUpdate(s.Ctx(), "bd-mo-uc", domain.UpdateSpec{
Fields: map[string]any{
issueops.OpSetMetadata: []string{"mine=1"},
},
}, "tester")
s.Require().NoError(err)
got := map[string]any{}
s.Require().NoError(json.Unmarshal(updated.Metadata, &got))
s.Equal("safe", got["sibling"])
s.Equal("1", got["mine"])
}
func (s *testSuite) iucApplyUpdateAppendNotes() {
s.seedIssueWithMetadata("bd-mo-uc-notes", "", "base")
updated, err := s.issueUseCase().ApplyUpdate(s.Ctx(), "bd-mo-uc-notes", domain.UpdateSpec{
Fields: map[string]any{
issueops.OpAppendNotes: "appended",
},
}, "tester")
s.Require().NoError(err)
s.Equal("base\nappended", updated.Notes)
}

View file

@ -0,0 +1,102 @@
//go:build cgo
package embeddeddolt_test
import (
"encoding/json"
"testing"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/types"
)
// TestUpdateMergeOps_DoltEngine smokes the in-tx merge operations on the
// embedded-Dolt reference backend: metadata edits and note appends are resolved
// against the row read inside the mutation transaction (issueops.ResolveMergeOps),
// with the store's whole-attempt retry preserving concurrent writers' keys. A
// regression there would break every `bd update --set-metadata` / `bd note` on
// a Dolt workspace.
func TestUpdateMergeOps_DoltEngine(t *testing.T) {
te := newTestEnv(t, "mergeops")
ctx := t.Context()
issue := &types.Issue{
ID: "mergeops-1",
Title: "merge ops on dolt",
Status: types.StatusOpen,
IssueType: types.TypeTask,
Priority: 2,
Metadata: json.RawMessage(`{"keep":"x"}`),
Notes: "line1",
}
if err := te.store.CreateIssue(ctx, issue, "actor"); err != nil {
t.Fatalf("CreateIssue: %v", err)
}
updates := map[string]interface{}{
issueops.OpSetMetadata: []string{"team=platform"},
issueops.OpAppendNotes: "line2",
}
if err := te.store.UpdateIssue(ctx, issue.ID, updates, "actor"); err != nil {
t.Fatalf("UpdateIssue(merge ops): %v", err)
}
got, err := te.store.GetIssue(ctx, issue.ID)
if err != nil {
t.Fatalf("GetIssue: %v", err)
}
meta := map[string]any{}
if err := json.Unmarshal(got.Metadata, &meta); err != nil {
t.Fatalf("unmarshal metadata %q: %v", got.Metadata, err)
}
if meta["keep"] != "x" {
t.Errorf("metadata[keep]: got %v, want %q", meta["keep"], "x")
}
if meta["team"] != "platform" {
t.Errorf("metadata[team]: got %v, want %q", meta["team"], "platform")
}
if got.Notes != "line1\nline2" {
t.Errorf("notes: got %q, want %q", got.Notes, "line1\nline2")
}
}
// TestUpdateMergeOps_DoltEngine_TwoIssues pins per-issue correctness of the
// locking read when the SAME statement text runs against DIFFERENT bindings:
// issue 1 is touched first so any text-keyed statement caching in the engine
// is primed with issue 1's row, then issue 2's merge must still read issue 2.
func TestUpdateMergeOps_DoltEngine_TwoIssues(t *testing.T) {
te := newTestEnv(t, "twoissues")
ctx := t.Context()
for _, spec := range []struct{ id, notes string }{
{"two-1", "first-notes"},
{"two-2", ""},
} {
iss := &types.Issue{ID: spec.id, Title: spec.id, Status: types.StatusOpen, IssueType: types.TypeTask, Priority: 2, Notes: spec.notes}
if err := te.store.CreateIssue(ctx, iss, "actor"); err != nil {
t.Fatalf("CreateIssue(%s): %v", spec.id, err)
}
}
if err := te.store.UpdateIssue(ctx, "two-1", map[string]interface{}{issueops.OpAppendNotes: "added-1"}, "actor"); err != nil {
t.Fatalf("UpdateIssue(two-1): %v", err)
}
if err := te.store.UpdateIssue(ctx, "two-2", map[string]interface{}{issueops.OpAppendNotes: "added-2"}, "actor"); err != nil {
t.Fatalf("UpdateIssue(two-2): %v", err)
}
got1, err := te.store.GetIssue(ctx, "two-1")
if err != nil {
t.Fatalf("GetIssue(two-1): %v", err)
}
got2, err := te.store.GetIssue(ctx, "two-2")
if err != nil {
t.Fatalf("GetIssue(two-2): %v", err)
}
if got1.Notes != "first-notes\nadded-1" {
t.Errorf("two-1 notes: got %q, want %q", got1.Notes, "first-notes\nadded-1")
}
if got2.Notes != "added-2" {
t.Errorf("two-2 notes: got %q, want %q", got2.Notes, "added-2")
}
}

View file

@ -196,10 +196,19 @@ func updateIssueInTx(ctx context.Context, tx DBTX, id string, updates map[string
isWisp := IsActiveWispInTx(ctx, tx, id)
issueTable, _, eventTable, _ := WispTableRouting(isWisp)
// Read old issue inside the transaction for consistency.
oldIssue, err := GetIssueInTx(ctx, tx, id)
// Read the pre-update row inside the transaction and fold any
// read-merge-write operations (metadata edits, note appends) into concrete
// column values against THAT row. On Dolt — the only supported backend — a
// stale snapshot is made safe not by row locks (FOR UPDATE is a parse-only
// no-op) but by commit-time conflict detection plus the store's
// whole-attempt retry (withRetryTx), which re-runs this resolution against
// the winning writer's committed row. Merging outside the mutation
// transaction — the CLI's old behavior — silently erased concurrent
// committed writes to sibling keys (GH audit: 7 of 200 exit-0
// --set-metadata writes lost).
oldIssue, updates, err := readIssueAndResolveMergeOps(ctx, tx, id, updates)
if err != nil {
return nil, fmt.Errorf("failed to get issue for update: %w", err)
return nil, err
}
// Validate issue_type against built-in + custom types (GH#3030).
@ -337,6 +346,192 @@ func updateIssueInTx(ctx context.Context, tx DBTX, id string, updates map[string
return &UpdateResult{OldIssue: oldIssue, IsWisp: isWisp}, nil
}
// Merge-operation update keys. Unlike plain column updates, these are resolved
// against the current row INSIDE the mutation transaction: re-read, merge,
// write, commit. Callers pass the operation (keys to set/unset, text to
// append) instead of a pre-merged value, so concurrent writers touching
// different keys of the same issue cannot erase each other.
const (
// OpMergeMetadata merges a JSON object's top-level keys into the issue's
// metadata (bd update --metadata). Value: string, []byte, or json.RawMessage.
OpMergeMetadata = "_merge_metadata"
// OpSetMetadata sets individual key=value metadata entries
// (bd update --set-metadata). Value: []string.
OpSetMetadata = "_set_metadata"
// OpUnsetMetadata removes metadata keys (bd update --unset-metadata).
// Value: []string.
OpUnsetMetadata = "_unset_metadata"
// OpAppendNotes appends a line to the issue's notes
// (bd update --append-notes). Value: string.
OpAppendNotes = "append_notes"
)
// HasMergeOps reports whether the update map carries any read-merge-write
// operation key. Updates with merge ops must resolve those ops against the row
// read inside the same mutation transaction (see ResolveMergeOps); on Dolt the
// store's whole-attempt retry then re-runs that resolution against the winning
// writer's committed row.
func HasMergeOps(updates map[string]interface{}) bool {
for _, op := range []string{OpMergeMetadata, OpSetMetadata, OpUnsetMetadata, OpAppendNotes} {
if _, ok := updates[op]; ok {
return true
}
}
return false
}
// ResolveMergeOps rewrites merge-operation keys into concrete column values
// using oldIssue, which the caller read in the same mutation transaction.
// Returns the input map unchanged when no operation keys are present;
// otherwise returns a copy so the caller's map is not mutated.
func ResolveMergeOps(oldIssue *types.Issue, updates map[string]interface{}) (map[string]interface{}, error) {
if !HasMergeOps(updates) {
return updates, nil
}
resolved := make(map[string]interface{}, len(updates))
for k, v := range updates {
if !isMergeOpKey(k) {
resolved[k] = v
}
}
if err := resolveMetadataMergeOps(oldIssue, updates, resolved); err != nil {
return nil, err
}
if err := resolveNotesAppendOp(oldIssue, updates, resolved); err != nil {
return nil, err
}
return resolved, nil
}
// isMergeOpKey reports whether k is a read-merge-write operation key consumed by
// ResolveMergeOps rather than a concrete column value to pass through unchanged.
func isMergeOpKey(k string) bool {
switch k {
case OpMergeMetadata, OpSetMetadata, OpUnsetMetadata, OpAppendNotes:
return true
default:
return false
}
}
// resolveMetadataMergeOps folds OpMergeMetadata/OpSetMetadata/OpUnsetMetadata
// into a concrete "metadata" value on resolved, using oldIssue.Metadata (read in
// the same mutation transaction) as the base. It is a no-op when no metadata
// operation keys are present.
func resolveMetadataMergeOps(oldIssue *types.Issue, updates, resolved map[string]interface{}) error {
_, hasMerge := updates[OpMergeMetadata]
_, hasSet := updates[OpSetMetadata]
_, hasUnset := updates[OpUnsetMetadata]
if !hasMerge && !hasSet && !hasUnset {
return nil
}
if _, direct := resolved["metadata"]; direct {
return fmt.Errorf("cannot combine a metadata replacement with incremental metadata edits")
}
current := oldIssue.Metadata
if hasMerge {
normalized, err := storage.NormalizeMetadataValue(updates[OpMergeMetadata])
if err != nil {
return fmt.Errorf("invalid %s: %w", OpMergeMetadata, err)
}
merged, err := storage.MergeMetadataJSON(current, json.RawMessage(normalized))
if err != nil {
return fmt.Errorf("metadata merge failed: %w", err)
}
current = merged
}
if hasSet || hasUnset {
set, err := mergeOpStrings(OpSetMetadata, updates[OpSetMetadata], hasSet)
if err != nil {
return err
}
unset, err := mergeOpStrings(OpUnsetMetadata, updates[OpUnsetMetadata], hasUnset)
if err != nil {
return err
}
merged, err := storage.ApplyMetadataEdits(current, set, unset)
if err != nil {
return fmt.Errorf("metadata edit failed: %w", err)
}
current = merged
}
// Validate the merged result, matching the schema check stores apply to
// direct metadata replacements (GH#1416 Phase 2).
if err := ValidateMetadataIfConfigured(current); err != nil {
return err
}
resolved["metadata"] = current
return nil
}
// resolveNotesAppendOp folds OpAppendNotes into a concrete "notes" value on
// resolved, appending to oldIssue.Notes (read in the same mutation transaction).
// It is a no-op when the append op is absent.
func resolveNotesAppendOp(oldIssue *types.Issue, updates, resolved map[string]interface{}) error {
raw, ok := updates[OpAppendNotes]
if !ok {
return nil
}
if _, direct := resolved["notes"]; direct {
return fmt.Errorf("cannot combine a notes replacement with %s", OpAppendNotes)
}
text, ok := raw.(string)
if !ok {
return fmt.Errorf("%s must be a string, got %T", OpAppendNotes, raw)
}
combined := oldIssue.Notes
if combined != "" {
combined += "\n"
}
combined += text
resolved["notes"] = combined
return nil
}
// mergeOpStrings coerces a merge-operation value to []string. Accepts
// []interface{} of strings as well, so operation maps survive a JSON
// round-trip (e.g. daemon transports).
func mergeOpStrings(op string, value interface{}, present bool) ([]string, error) {
if !present {
return nil, nil
}
switch v := value.(type) {
case []string:
return v, nil
case []interface{}:
out := make([]string, 0, len(v))
for _, item := range v {
s, ok := item.(string)
if !ok {
return nil, fmt.Errorf("%s must be a list of strings, got element %T", op, item)
}
out = append(out, s)
}
return out, nil
default:
return nil, fmt.Errorf("%s must be a list of strings, got %T", op, value)
}
}
// readIssueAndResolveMergeOps reads the pre-update row in-transaction and folds
// any merge-operation keys (metadata edits, note appends) into concrete column
// values against that row, returning the row and the rewritten update map. It
// keeps the read-merge-write plumbing off updateIssueInTx's already-large body.
func readIssueAndResolveMergeOps(ctx context.Context, tx DBTX, id string, updates map[string]interface{}) (*types.Issue, map[string]interface{}, error) {
oldIssue, err := GetIssueInTx(ctx, tx, id)
if err != nil {
return nil, nil, fmt.Errorf("failed to get issue for update: %w", err)
}
resolved, err := ResolveMergeOps(oldIssue, updates)
if err != nil {
return nil, nil, err
}
return oldIssue, resolved, nil
}
// RecordFullEventInTable records an event with both old and new values.
//
//nolint:gosec // G201: table is from WispTableRouting ("events" or "wisp_events")

View file

@ -229,3 +229,83 @@ func JSONMetadataPath(key string) string {
}
return "$." + key
}
// MergeMetadataJSON merges incoming metadata JSON into existing metadata.
// Top-level keys from incoming overwrite keys in existing; keys only in
// existing are preserved. Both inputs must be JSON objects (or empty/null).
func MergeMetadataJSON(existing, incoming json.RawMessage) (json.RawMessage, error) {
base := make(map[string]json.RawMessage)
if len(existing) > 0 {
trimmed := strings.TrimSpace(string(existing))
if trimmed != "" && trimmed != "null" {
if err := json.Unmarshal(existing, &base); err != nil {
return nil, fmt.Errorf("existing metadata is not a JSON object: %w", err)
}
}
}
overlay := make(map[string]json.RawMessage)
if err := json.Unmarshal(incoming, &overlay); err != nil {
return nil, fmt.Errorf("new metadata is not a JSON object: %w", err)
}
for k, v := range overlay {
base[k] = v
}
result, err := json.Marshal(base)
if err != nil {
return nil, fmt.Errorf("failed to marshal merged metadata: %w", err)
}
return json.RawMessage(result), nil
}
// ApplyMetadataEdits applies incremental set (key=value) and unset (key) edits
// to existing metadata and returns the merged JSON. Set values are typed via
// MetadataEditValue; keys are validated with ValidateMetadataKey.
func ApplyMetadataEdits(existing json.RawMessage, setFlags, unsetFlags []string) (json.RawMessage, error) {
data := make(map[string]json.RawMessage)
if len(existing) > 0 {
trimmed := strings.TrimSpace(string(existing))
if trimmed != "" && trimmed != "null" {
if err := json.Unmarshal(existing, &data); err != nil {
return nil, fmt.Errorf("existing metadata is not a JSON object: %w", err)
}
}
}
for _, kv := range setFlags {
k, v, ok := strings.Cut(kv, "=")
if !ok || k == "" {
return nil, fmt.Errorf("invalid --set-metadata: expected key=value, got %q", kv)
}
if err := ValidateMetadataKey(k); err != nil {
return nil, err
}
data[k] = MetadataEditValue(v)
}
for _, k := range unsetFlags {
if err := ValidateMetadataKey(k); err != nil {
return nil, err
}
delete(data, k)
}
result, err := json.Marshal(data)
if err != nil {
return nil, fmt.Errorf("failed to marshal metadata: %w", err)
}
return json.RawMessage(result), nil
}
// MetadataEditValue converts a --set-metadata string value to JSON. Per the CLI
// contract (GH#4146), --set-metadata values are ALWAYS stored as JSON strings;
// inferring numbers/booleans/null from string content silently broke
// map[string]string round-trips for Go consumers (a numeric-looking id or a
// version like "1e3" came back as a JSON number). Typed values go through the
// explicit --metadata / --metadata-json path (MergeMetadataJSON).
func MetadataEditValue(s string) json.RawMessage {
b, _ := json.Marshal(s)
return json.RawMessage(b)
}

View file

@ -6,6 +6,16 @@ import (
mysql "github.com/go-sql-driver/mysql"
)
// IsSerializationError reports whether err is a Dolt/MySQL serialization
// failure that guarantees the server rolled the transaction back. Because the
// rollback discards every uncommitted write in the session, the only safe
// retry is to redo the WHOLE unit of work (read, merge, write, commit) —
// retrying just the commit re-commits an empty session, which Dolt reports as
// "nothing to commit" while the write is silently lost.
func IsSerializationError(err error) bool {
return isSerializationError(err)
}
// isSerializationError returns true if the error is a Dolt/MySQL serialization
// failure that guarantees the transaction was rolled back. Safe to retry.
// - 1213 (ER_LOCK_DEADLOCK): concurrent transactions conflict at commit time

View file

@ -0,0 +1,262 @@
package uow
import (
"context"
"encoding/json"
"fmt"
"path/filepath"
"strconv"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
_ "github.com/go-sql-driver/mysql"
"github.com/stretchr/testify/require"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/storage/dbproxy/proxy"
"github.com/steveyegge/beads/internal/storage/domain"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/testutil"
"github.com/steveyegge/beads/internal/types"
)
// TestUOW_ConcurrentMergeOps_NoLostUpdate proves the lost-update fix against a
// REAL Dolt sql-server: concurrent writers mutating the SAME issue through the
// unit-of-work stack (the proxied-server CLI's write path) must never lose a
// write that reported success.
//
// Each writer follows the exact contract cmd/bd's applyUpdateProxiedOne now
// implements:
//
// - metadata edits and note appends pass through ApplyUpdate as
// merge-OPERATION keys (issueops.OpSetMetadata / OpAppendNotes), resolved
// against the row read inside the mutation transaction — never pre-merged
// from an earlier snapshot;
// - the unit of work commits exactly ONCE; a serialization failure (Dolt's
// commit-time conflict, guaranteed server-side rollback) redoes the WHOLE
// read-merge-write in a fresh unit of work;
// - "nothing to commit" on a fresh, unconflicted attempt would mean the
// write silently vanished, so the writer fails the test if it appears.
//
// The old commit-only retry (uow.CommitWithRetries) re-committed the
// rolled-back session, harvested "nothing to commit", and the CLI swallowed it
// with exit 0 — the silent-loss shape this test exists to catch.
//
// Two phases, because Dolt's conflict behavior differs by column type:
//
// - metadata is a JSON column, which Dolt three-way merges KEY-WISE at
// commit time: distinct-key writers converge without any conflict, so the
// metadata phase asserts convergence but cannot demand conflicts;
// - notes is a TEXT column with no structural merge: overlapping writers
// editing that cell MUST conflict, so the notes phase demands at least one
// serialization failure — proof the whole-UOW redo path actually ran —
// and still requires every append to survive.
func TestUOW_ConcurrentMergeOps_NoLostUpdate(t *testing.T) {
if testing.Short() {
t.Skip("starts a Dolt container and a dbproxy subprocess; skipped in -short")
}
port := testutil.StartIsolatedDoltContainer(t)
portInt, err := strconv.Atoi(port)
require.NoError(t, err)
bdBin := buildBDBinary(t)
prev := proxy.ResolveExecutable
proxy.ResolveExecutable = func() (string, error) { return bdBin, nil }
t.Cleanup(func() { proxy.ResolveExecutable = prev })
t.Setenv("HOME", t.TempDir())
storeRootDir := t.TempDir()
shutdownOnInterrupt(t, storeRootDir)
t.Cleanup(func() {
if err := proxy.Shutdown(storeRootDir); err != nil {
t.Logf("proxy.Shutdown(%s): %v", storeRootDir, err)
}
})
logPath := filepath.Join(t.TempDir(), "server.log")
ctx, cancel := context.WithTimeout(context.Background(), 180*time.Second)
defer cancel()
provider, err := NewExternalDoltServerUOWProvider(
ctx,
storeRootDir,
"beads_lostupdate_test",
logPath,
configfile.ExternalDoltConfig{Host: "127.0.0.1", Port: portInt},
"root",
"",
0,
0,
)
require.NoError(t, err)
require.NotNil(t, provider)
t.Cleanup(func() { _ = provider.Close(context.Background()) })
const issueID = "lu-1"
func() {
uw, err := provider.NewUOW(ctx)
require.NoError(t, err)
defer uw.Close(ctx)
_, err = uw.IssueUseCase().CreateIssue(ctx, domain.CreateIssueParams{
Issue: &types.Issue{
ID: issueID,
Title: "lost-update target",
Status: types.StatusOpen,
Priority: 2,
IssueType: types.TypeTask,
Notes: "seed line",
Metadata: json.RawMessage(`{"seed":"yes"}`),
},
ExplicitID: issueID,
}, "seeder")
require.NoError(t, err)
require.NoError(t, uw.Commit(ctx, "seed lost-update target"))
}()
// conflicts counts observed serialization failures so the assertion below
// can prove the commit-time conflict path really fired.
var conflicts atomic.Int64
// writeOp mirrors applyUpdateProxiedOne: fresh UOW per attempt, merge ops
// in spec.Fields, one commit, whole-attempt redo on serialization failure.
//
// syncPoint is a barrier placed between ApplyUpdate and Commit on the
// FIRST attempt only: every writer applies its update against the same
// base root before any of them commits, guaranteeing the transactions
// overlap at Dolt's commit-time merge.
writeOp := func(label string, fields map[string]any, syncPoint func()) error {
synced := false
syncOnce := func() {
if !synced {
synced = true
syncPoint()
}
}
defer syncOnce() // never leave barrier peers hanging on an early error
const maxAttempts = 40
for attempt := 1; attempt <= maxAttempts; attempt++ {
err := func() error {
uw, err := provider.NewUOW(ctx)
if err != nil {
return fmt.Errorf("new uow: %w", err)
}
defer uw.Close(ctx)
if _, err := uw.IssueUseCase().ApplyUpdate(ctx, issueID,
domain.UpdateSpec{Fields: fields}, "writer-"+label); err != nil {
return err
}
syncOnce()
return uw.Commit(ctx, "write "+label)
}()
if err == nil {
return nil
}
if IsSerializationError(err) {
conflicts.Add(1)
time.Sleep(time.Duration(attempt) * 10 * time.Millisecond)
continue
}
if issueops.IsNothingToCommitError(err) {
return fmt.Errorf("write %s swallowed as nothing-to-commit on a fresh attempt (silent lost update): %w", label, err)
}
return fmt.Errorf("write %s: %w", label, err)
}
return fmt.Errorf("write %s: retries exhausted on serialization failures", label)
}
runRound := func(round int, fieldsFor func(writer int) (string, map[string]any)) {
t.Helper()
const writers = 4
errs := make([]error, writers)
var applied, done sync.WaitGroup
applied.Add(writers)
done.Add(writers)
barrier := func() {
applied.Done()
applied.Wait()
}
for i := 0; i < writers; i++ {
i := i
label, fields := fieldsFor(i)
go func() {
defer done.Done()
errs[i] = writeOp(label, fields, barrier)
}()
}
done.Wait()
for i, err := range errs {
require.NoErrorf(t, err, "round %d writer %d", round, i)
}
}
// Phase 1: distinct metadata keys. Dolt merges the JSON column key-wise at
// commit time, so these converge with or without observed conflicts.
const metadataRounds = 2
for round := 0; round < metadataRounds; round++ {
round := round
runRound(round, func(writer int) (string, map[string]any) {
key := fmt.Sprintf("w%dr%d", writer, round)
return key, map[string]any{
issueops.OpSetMetadata: []string{key + "=1"},
}
})
}
// Phase 2: note appends. The notes TEXT cell has no structural merge, so
// the barrier-aligned first attempts MUST collide at commit time and the
// losers MUST redo the whole read-merge-write to preserve every line.
notesConflictFloor := conflicts.Load()
const notesRounds = 2
for round := 0; round < notesRounds; round++ {
round := round
runRound(round, func(writer int) (string, map[string]any) {
label := fmt.Sprintf("note-w%dr%d", writer, round)
return label, map[string]any{
issueops.OpAppendNotes: "appended by " + label,
}
})
}
notesConflicts := conflicts.Load() - notesConflictFloor
t.Logf("observed %d serialization conflicts total (%d in the notes phase)",
conflicts.Load(), notesConflicts)
require.Positive(t, notesConflicts,
"barrier-aligned writers on the notes TEXT cell must hit Dolt commit-time conflicts; the whole-UOW redo path was never exercised")
// Every write reported as successful must be present, and the seeds must
// survive.
uw, err := provider.NewUOW(ctx)
require.NoError(t, err)
defer uw.Close(ctx)
final, err := uw.IssueUseCase().GetIssue(ctx, issueID)
require.NoError(t, err)
require.NotNil(t, final)
got := map[string]any{}
require.NoError(t, json.Unmarshal(final.Metadata, &got))
require.Equal(t, "yes", got["seed"], "seed key erased by concurrent writers: %v", got)
var missing []string
for round := 0; round < metadataRounds; round++ {
for writer := 0; writer < 4; writer++ {
key := fmt.Sprintf("w%dr%d", writer, round)
if _, ok := got[key]; !ok {
missing = append(missing, key)
}
}
}
require.Emptyf(t, missing, "silent lost update: %d successful metadata writes missing from final metadata %v", len(missing), got)
require.Contains(t, final.Notes, "seed line", "seed notes erased by concurrent appends")
for round := 0; round < notesRounds; round++ {
for writer := 0; writer < 4; writer++ {
line := fmt.Sprintf("appended by note-w%dr%d", writer, round)
require.Containsf(t, final.Notes, line,
"silent lost update: successful append %q missing from final notes %q", line, final.Notes)
}
}
require.Equal(t, 1, strings.Count(final.Notes, "seed line"), "seed line duplicated: redo must not double-apply")
}

View file

@ -12,6 +12,7 @@ internal/testutil/fixtures/fixtures_test.go::TestLargeFromJSONL
internal/storage/dolt/concurrent_test.go::TestHighContentionStress
internal/storage/dolt/concurrent_test.go::TestConcurrentWorkQueueDrain
internal/storage/dolt/lease_test.go::TestConcurrentHeartbeatReclaimClose
internal/storage/uow/lostupdate_dolt_test.go::TestUOW_ConcurrentMergeOps_NoLostUpdate
cmd/bd/prune_bench_test.go::TestPruneLargeFixture
EOF
)