beads/scripts/check-testing-short.sh
Julian Knutsen 8ebd53e882
fix(update): resolve metadata/note merges inside the mutation transaction (concurrent lost-update) (#4732)
* fix(update): resolve metadata/note merges inside the mutation transaction

bd update --metadata/--set-metadata/--unset-metadata/--append-notes and
bd note resolved their merges client-side: read the issue in one
transaction, merge in process memory, write the merged blob back in a
later transaction. Two concurrent writers touching DIFFERENT keys of
the SAME issue both exited 0, but the later commit silently erased the
earlier writer's committed keys. The multiprocess-safety audit proved
the loss empirically: 7 of 200 exit-0 --set-metadata writes vanished in
an 8-writer same-issue hammer against the stock binary.

The first version of this fix moved the merge into the mutation
transaction on the direct path only; red-team review blocked it because
the proxied-server route still snapshot-merged in
buildUpdateSpecForIssue — and worse: on a Dolt serialization rollback,
uow.CommitWithRetries retried only the COMMIT, re-committing the
rolled-back (now empty) session. Dolt answered "nothing to commit",
isDoltNothingToCommit swallowed it, and bd printed "Updated" and exited
0 for a write that never landed.

This revision closes both paths:

- Merge-shaped edits travel as operation keys (issueops.OpMergeMetadata,
  OpSetMetadata, OpUnsetMetadata, OpAppendNotes) and are resolved
  against the row re-read INSIDE the mutation transaction
  (issueops.ResolveMergeOps) — in both the embedded path
  (updateIssueInTx) and the domain/db repository used by proxied-server
  mode. CLI paths (bd update, bd note) no longer pre-merge; the shared
  merge helpers live in internal/storage.
- The in-transaction read takes a row write lock
  (GetIssueForUpdateInTx, SELECT ... FOR UPDATE) so the read itself
  serializes on Postgres (READ COMMITTED) and MySQL (REPEATABLE READ).
  sqlitedialect strips the clause — safe because _txlock=immediate
  already takes the database write lock at BEGIN. Dolt parses FOR
  UPDATE as a no-op and relies on commit-time conflict detection plus
  the whole-attempt retry below.
- The proxied server redoes the WHOLE read-merge-write attempt in a
  fresh unit of work on a serialization failure (new exported
  uow.IsSerializationError; exponential backoff, 15s budget), commits
  each attempt exactly once, and fails loudly to stderr when retries
  exhaust instead of exiting 0.

Verification: cross-backend regression body in
internal/storage/conformance/lostupdate.go (two independent store
handles plus a lock-holding third pool) wired into the sqlite, dolt
(embedded and sql-server), mysql, and postgres suites; cmd/bd
multi-process race tests for --set-metadata and bd note; proxied-server
attempt/retry unit tests. Re-running the audit hammer against the
patched binary: 0 lost of 200 exit-0 writes across natural and
lock-widened runs (stock binary lost writes in both), with the one
contention exhaustion surfacing as a nonzero exit instead of a silent
loss.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(update): string --set-metadata, proxied partial-failure exit, drop dead locking scaffolding

Maintainer adoption-review fixes for PR #4732 (review iterations 1 and 3):

- metadata: MetadataEditValue stores --set-metadata values as JSON strings
  again (GH#4146) instead of inferring numbers/booleans/null from content,
  which silently broke map[string]string round-trips. Typed values stay
  available via --metadata / --metadata-json. Domain merge-op tests now
  expect string values for --set-metadata.
- cmd/bd: restore the proxied multi-ID partial-failure accumulator so a batch
  with any failed ID exits non-zero (had regressed #4726), and derive success
  output from the resolved jsonOutput signal so --format json and config
  json:true are honored on the proxied path.
- storage: remove the dead internal/storage/sqlitedialect package (509 LOC,
  zero importers) and the FOR UPDATE / GetIssueForUpdateInTx row-locking
  scaffolding. SQLite/Postgres/MySQL are gone and Dolt treats FOR UPDATE as a
  parse-only no-op; the read-merge-write is made safe by commit-time conflict
  detection plus the whole-attempt retry. Also drop the uncalled MVCC-only
  conformance helper. updateIssueInTx returns to baseline complexity via a
  small read-and-resolve helper.
- tests: remove the SQLite-backend race tests (the removed backend can't run
  them); Dolt concurrency coverage in uow/embeddeddolt/domain-db is
  load-bearing.
- readability: split ResolveMergeOps into resolveMetadataMergeOps and
  resolveNotesAppendOp (with an isMergeOpKey helper) so each diff-authored
  merge-op resolver stays below the review complexity gate (cyclomatic <21 /
  cognitive <26). Pure structure change: same resolution order, error strings,
  schema validation, and copy-not-mutate semantics.
- policy: allowlist TestUOW_ConcurrentMergeOps_NoLostUpdate in
  scripts/check-testing-short.sh. Its testing.Short() skip is a legitimate
  runtime/stress skip (it starts a Dolt container and a dbproxy subprocess),
  matching the other allowlisted Dolt concurrency stress tests; without the
  entry the ci-pr-policy check fails on the adopted head.

- tests: remove the two obsolete cmd/bd multi-ID tests
  (TestMultiIDUpdatePartialMetadataMergeFailure and
  TestMultiIDUpdatePartialMetadataEditFailureJSON) and their now-unused
  seedNonObjectMetadata/showMultiIDUpdateMetadataValue helpers. They seeded a
  non-object array via `bd update --metadata "[]"` to force a later per-ID
  metadata-merge failure, but the new in-transaction merge correctly rejects
  non-object metadata ("new metadata is not a JSON object"), so that seeding
  path is unreachable and the tests now fail at setup. The four #4726 per-ID
  exit-code tests in the same file are retained.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Test User <test@test.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 05:59:31 -07:00

56 lines
1.7 KiB
Bash
Executable file

#!/usr/bin/env bash
# Ensure testing.Short() is only used for true runtime/stress/large-fixture skips.
set -euo pipefail
allowed=$(
cat <<'EOF'
internal/hooks/hooks_test.go::TestRunSync_Timeout
internal/hooks/hooks_test.go::TestRunSync_KillsDescendants
internal/testutil/fixtures/fixtures_test.go::TestXLargeDolt
internal/testutil/fixtures/fixtures_test.go::TestLargeFromJSONL
internal/storage/dolt/concurrent_test.go::TestHighContentionStress
internal/storage/dolt/concurrent_test.go::TestConcurrentWorkQueueDrain
internal/storage/dolt/lease_test.go::TestConcurrentHeartbeatReclaimClose
internal/storage/uow/lostupdate_dolt_test.go::TestUOW_ConcurrentMergeOps_NoLostUpdate
cmd/bd/prune_bench_test.go::TestPruneLargeFixture
EOF
)
status=0
while IFS=: read -r file line _; do
file="${file#./}"
if [[ -z "$file" || -z "$line" ]]; then
continue
fi
func=$(
awk -v target="$line" '
NR <= target && /^func [A-Za-z0-9_]+\(/ {
current = $0
sub(/^func /, "", current)
sub(/\(.*/, "", current)
}
END { print current }
' "$file"
)
key="${file}::${func}"
if ! grep -Fxq "$key" <<<"$allowed"; then
printf 'Disallowed testing.Short() at %s:%s in %s\n' "$file" "$line" "${func:-unknown}" >&2
status=1
fi
done < <(find . -type f -name '*.go' -not -path './.git/*' -exec grep -n 'testing\.Short()' {} + || true)
if (( status != 0 )); then
cat >&2 <<'EOF'
testing.Short() is reserved for true runtime, stress, or large-fixture skips.
Use build tags, environment checks, or named wrappers for integration/e2e/API
boundaries instead.
EOF
exit "$status"
fi
printf 'testing.Short() usage is limited to approved runtime/stress/large-fixture skips.\n'