beads/scripts
Julian Knutsen 8ebd53e882
fix(update): resolve metadata/note merges inside the mutation transaction (concurrent lost-update) (#4732)
* fix(update): resolve metadata/note merges inside the mutation transaction

bd update --metadata/--set-metadata/--unset-metadata/--append-notes and
bd note resolved their merges client-side: read the issue in one
transaction, merge in process memory, write the merged blob back in a
later transaction. Two concurrent writers touching DIFFERENT keys of
the SAME issue both exited 0, but the later commit silently erased the
earlier writer's committed keys. The multiprocess-safety audit proved
the loss empirically: 7 of 200 exit-0 --set-metadata writes vanished in
an 8-writer same-issue hammer against the stock binary.

The first version of this fix moved the merge into the mutation
transaction on the direct path only; red-team review blocked it because
the proxied-server route still snapshot-merged in
buildUpdateSpecForIssue — and worse: on a Dolt serialization rollback,
uow.CommitWithRetries retried only the COMMIT, re-committing the
rolled-back (now empty) session. Dolt answered "nothing to commit",
isDoltNothingToCommit swallowed it, and bd printed "Updated" and exited
0 for a write that never landed.

This revision closes both paths:

- Merge-shaped edits travel as operation keys (issueops.OpMergeMetadata,
  OpSetMetadata, OpUnsetMetadata, OpAppendNotes) and are resolved
  against the row re-read INSIDE the mutation transaction
  (issueops.ResolveMergeOps) — in both the embedded path
  (updateIssueInTx) and the domain/db repository used by proxied-server
  mode. CLI paths (bd update, bd note) no longer pre-merge; the shared
  merge helpers live in internal/storage.
- The in-transaction read takes a row write lock
  (GetIssueForUpdateInTx, SELECT ... FOR UPDATE) so the read itself
  serializes on Postgres (READ COMMITTED) and MySQL (REPEATABLE READ).
  sqlitedialect strips the clause — safe because _txlock=immediate
  already takes the database write lock at BEGIN. Dolt parses FOR
  UPDATE as a no-op and relies on commit-time conflict detection plus
  the whole-attempt retry below.
- The proxied server redoes the WHOLE read-merge-write attempt in a
  fresh unit of work on a serialization failure (new exported
  uow.IsSerializationError; exponential backoff, 15s budget), commits
  each attempt exactly once, and fails loudly to stderr when retries
  exhaust instead of exiting 0.

Verification: cross-backend regression body in
internal/storage/conformance/lostupdate.go (two independent store
handles plus a lock-holding third pool) wired into the sqlite, dolt
(embedded and sql-server), mysql, and postgres suites; cmd/bd
multi-process race tests for --set-metadata and bd note; proxied-server
attempt/retry unit tests. Re-running the audit hammer against the
patched binary: 0 lost of 200 exit-0 writes across natural and
lock-widened runs (stock binary lost writes in both), with the one
contention exhaustion surfacing as a nonzero exit instead of a silent
loss.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(update): string --set-metadata, proxied partial-failure exit, drop dead locking scaffolding

Maintainer adoption-review fixes for PR #4732 (review iterations 1 and 3):

- metadata: MetadataEditValue stores --set-metadata values as JSON strings
  again (GH#4146) instead of inferring numbers/booleans/null from content,
  which silently broke map[string]string round-trips. Typed values stay
  available via --metadata / --metadata-json. Domain merge-op tests now
  expect string values for --set-metadata.
- cmd/bd: restore the proxied multi-ID partial-failure accumulator so a batch
  with any failed ID exits non-zero (had regressed #4726), and derive success
  output from the resolved jsonOutput signal so --format json and config
  json:true are honored on the proxied path.
- storage: remove the dead internal/storage/sqlitedialect package (509 LOC,
  zero importers) and the FOR UPDATE / GetIssueForUpdateInTx row-locking
  scaffolding. SQLite/Postgres/MySQL are gone and Dolt treats FOR UPDATE as a
  parse-only no-op; the read-merge-write is made safe by commit-time conflict
  detection plus the whole-attempt retry. Also drop the uncalled MVCC-only
  conformance helper. updateIssueInTx returns to baseline complexity via a
  small read-and-resolve helper.
- tests: remove the SQLite-backend race tests (the removed backend can't run
  them); Dolt concurrency coverage in uow/embeddeddolt/domain-db is
  load-bearing.
- readability: split ResolveMergeOps into resolveMetadataMergeOps and
  resolveNotesAppendOp (with an isMergeOpKey helper) so each diff-authored
  merge-op resolver stays below the review complexity gate (cyclomatic <21 /
  cognitive <26). Pure structure change: same resolution order, error strings,
  schema validation, and copy-not-mutate semantics.
- policy: allowlist TestUOW_ConcurrentMergeOps_NoLostUpdate in
  scripts/check-testing-short.sh. Its testing.Short() skip is a legitimate
  runtime/stress skip (it starts a Dolt container and a dbproxy subprocess),
  matching the other allowlisted Dolt concurrency stress tests; without the
  entry the ci-pr-policy check fails on the adopted head.

- tests: remove the two obsolete cmd/bd multi-ID tests
  (TestMultiIDUpdatePartialMetadataMergeFailure and
  TestMultiIDUpdatePartialMetadataEditFailureJSON) and their now-unused
  seedNonObjectMetadata/showMultiIDUpdateMetadataValue helpers. They seeded a
  non-object array via `bd update --metadata "[]"` to force a later per-ID
  metadata-merge failure, but the new in-transaction merge correctly rejects
  non-object metadata ("new metadata is not a JSON object"), so that seeding
  path is unreachable and the tests now fail at setup. The four #4726 per-ID
  exit-code tests in the same file are retained.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Test User <test@test.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 05:59:31 -07:00
..
bench-ready-indexes feat: choose your own storage backend (Postgres / MySQL / SQLite) — rebased on main 2026-07-10 17:44:37 +00:00
ci docs: go all-in on Mintlify — delete the Docusaurus site and its machinery 2026-07-12 10:37:17 -07:00
hooks feat: Add post-push hook to auto-install bd CLI after git push 2025-10-14 17:23:43 -07:00
migration-test test(migration): assert bd ready/blocked/close on migrated DB (mybd-ihg5) (#4584) 2026-07-07 13:32:37 -07:00
repro-dolt-hang /{cmd,internal,scripts}: use dsn helpers 2026-04-06 15:56:37 -07:00
repro-dolt-prod-timeouts feat: choose your own storage backend (Postgres / MySQL / SQLite) — rebased on main 2026-07-10 17:44:37 +00:00
bump-version.sh refactor: replace bump-version.sh with molecule pointer 2026-01-14 21:53:06 -08:00
check-build-tags.sh docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
check-cli-docs-drift.sh docs(pipeline): pin the CLI docs to the released bd; repair main's broken CLI reference (#4876) 2026-07-17 18:03:28 -07:00
check-doc-flags.sh docs(pipeline): pin the CLI docs to the released bd; repair main's broken CLI reference (#4876) 2026-07-17 18:03:28 -07:00
check-doc-freshness.sh docs: loss-check merge legacy docs/ originals into the site pages 2026-07-12 10:37:07 -07:00
check-go-install-guidance.sh fix(install): use declared module path for go fallback (#4215) 2026-05-27 21:48:24 -07:00
check-migration-hygiene.sh fix(audit): sand down the P3 sharp edges from the June 9-11 audit (bd-578h9.17) 2026-06-11 19:54:21 -04:00
check-testing-short.sh fix(update): resolve metadata/note merges inside the mutation transaction (concurrent lost-update) (#4732) 2026-07-19 05:59:31 -07:00
check-versions.sh docs: go all-in on Mintlify — delete the Docusaurus site and its machinery 2026-07-12 10:37:17 -07:00
clean-test-tmp.sh test(cmd/bd): route sync.Once test temp dirs through TestMain cleanup (#4108) 2026-05-22 16:48:22 -07:00
conformance.sh refactor(storage)!: remove the SQLite backend and dialect 2026-07-18 02:43:56 +00:00
cross-version-smoke-test.sh docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
docs-autofix-push.sh chore: purge deleted docs-pipeline references from release tooling, CI, and repo plumbing 2026-07-12 11:07:26 -07:00
docs.md chore: purge deleted docs-pipeline references from release tooling, CI, and repo plumbing 2026-07-12 11:07:26 -07:00
gen-winres.sh chore: re-land v1.1.0-rc.1 release prep (#4486) 2026-06-25 20:10:18 -07:00
generate-cli-docs.sh docs(pipeline): pin the CLI docs to the released bd; repair main's broken CLI reference (#4876) 2026-07-17 18:03:28 -07:00
generate-newsletter.py Newsletter generator: automated narrative summaries of releases 2026-01-22 16:09:29 -08:00
generate-newsletter_README.md Newsletter generator: automated narrative summaries of releases 2026-01-22 16:09:29 -08:00
gh-body-lint docs: align agent maintainer conventions (#4237) 2026-06-20 20:44:08 -07:00
install-hooks.sh feat: Add post-push hook to auto-install bd CLI after git push 2025-10-14 17:23:43 -07:00
install.sh docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
migrate-jsonl-to-dolt.sh feat: add JSONL-to-Dolt migration script (GH#2276) 2026-03-02 22:05:38 -08:00
migrate-sqlite-to-current.sh docs: add cross-era migration instructions to upgrading guide (#3081) 2026-04-06 20:11:00 -07:00
pr-preflight.sh fix: support PR preflight on gh 2.45 (bd-of1cd) (#4737) 2026-07-12 20:19:32 -07:00
pr_preflight_test.go fix: support PR preflight on gh 2.45 (bd-of1cd) (#4737) 2026-07-12 20:19:32 -07:00
README.md docs: go all-in on Mintlify — delete the Docusaurus site and its machinery 2026-07-12 10:37:17 -07:00
release.sh fix(release): harden shipped release formula adoption (#3668) 2026-05-06 12:14:54 -07:00
release_script_test.go test: port Windows hardening fixes (#4600) 2026-07-07 12:59:50 -07:00
resolve-docs-bd.sh docs(pipeline): pin the CLI docs to the released bd; repair main's broken CLI reference (#4876) 2026-07-17 18:03:28 -07:00
sign-windows.sh docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
test-cgo.sh Rename ICU-only test path to explicit opt-in workflow (bd-b18) (#3291) 2026-04-16 08:54:24 -07:00
test-git-remote.sh /{.github,docs,engdocs,go,internal,scripts}: bump dolt 2026-07-15 13:14:34 -07:00
test-icu-path.sh Rename ICU-only test path to explicit opt-in workflow (bd-b18) (#3291) 2026-04-16 08:54:24 -07:00
test.sh Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
update-nix-vendorhash.sh feat: comprehensive NixOS support improvements (#1024) 2026-01-12 23:46:33 -08:00
update-versions.sh chore: purge deleted docs-pipeline references from release tooling, CI, and repo plumbing 2026-07-12 11:07:26 -07:00
update-winget.sh docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
upgrade-smoke-test.sh test(upgrade-smoke): exercise dependency blocker paths on migrated DB (#4238) 2026-05-28 15:30:21 -07:00
verify-cgo.sh fix: remove ICU runtime dependency from release binaries (#3066) 2026-04-07 09:48:44 -07:00

Beads Scripts

Utility scripts for maintaining the beads project.

ci/

Repository-owned CI command wrappers. These scripts are the source of truth for the target CI tiers; Make targets are aliases for local discoverability.

make ci-pr-core
make ci-pr-policy
make ci-pr-lint
make ci-package-mcp
make ci-package-npm

Each wrapper auto-detects the repository root, sources .buildflags when it invokes Go in the default build mode, and records per-command timing through scripts/ci/lib/timing.sh.

Broad Go test wrappers also source scripts/ci/lib/test-env.sh, which creates a temporary HOME/XDG/Dolt root, isolates Git global/system config, clears runtime Beads/Dolt environment variables, and sets BEADS_TEST_SKIP=dolt before tests run. This keeps local make test and make ci-pr-core results comparable to the fast PR-core contract even on shared agent hosts. Set BEADS_TEST_ENV_RUN_DOLT=1 only when intentionally running the Dolt-dependent tests through these broad wrappers, or BEADS_TEST_ENV_DISABLE=1 when debugging against your real local configuration.

The broad Go wrappers also cap package and test parallelism to 4 by default (GO_TEST_PKG_PARALLEL and GO_TEST_PARALLEL). This avoids turning high-core shared hosts into a different test topology than GitHub Actions.

make ci-pr-policy includes scripts/check-testing-short.sh, which enforces that testing.Short() is only used for runtime, stress, or large-fixture skips. Use build tags, environment checks, or named wrappers for integration/e2e/API boundaries.

Package gate wrappers validate publishable/package-adjacent surfaces:

  • make ci-package-mcp builds or consumes a bd binary, puts it on PATH as bd, then runs locked MCP package uv sync, Ruff, mypy, pytest, and build checks.
  • make ci-package-npm builds or consumes the native binary expected by npm-package/bin/bd, runs the npm package test suite, and checks npm pack --dry-run.

Set BEADS_TEST_BD_BINARY=/path/to/bd for MCP and npm package gates to reuse a prebuilt candidate binary instead of rebuilding it inside the wrapper.

pr-preflight.sh

Read-only PR safety check for agents and maintainers.

# Before implementing or opening a related PR
./scripts/pr-preflight.sh --search "topic keywords" --repo gastownhall/beads

# Before changing, closing, or merging an existing PR
./scripts/pr-preflight.sh 123 --repo gastownhall/beads

It reports contributor/fork status, draft/review/merge/check state, risky diff signals such as .beads/ changes or missing tests, and the required contributor-protection next steps. It does not replace code review or local validation.

gh-body-lint

Lint Markdown files before posting them with gh ... --body-file.

./scripts/gh-body-lint body.md
./scripts/gh-body-lint --fix body.md

The lint catches literal \n sequences, which render poorly on GitHub, and GH#123 references, which do not auto-link like #123 or owner/repo#123.

release.sh (⭐ The Easy Button)

One-command release from version bump to local installation.

Usage

# Full release (does everything)
./scripts/release.sh 0.9.3

# Preview what would happen
./scripts/release.sh 0.9.3 --dry-run

What It Does

This master script automates the entire release process:

  1. ✅ Stops running Dolt servers (avoids version conflicts)
  2. ✅ Runs tests and linting
  3. ✅ Bumps version in all files
  4. ✅ Commits and pushes version bump
  5. ✅ Creates and pushes git tag
  6. ✅ Verifies or opens the Homebrew core formula PR
  7. ✅ Upgrades local Homebrew installation
  8. ✅ Verifies everything works

After this script completes, your system is running the new version!

Examples

# Release version 0.9.3
./scripts/release.sh 0.9.3

# Preview a release (no changes made)
./scripts/release.sh 1.0.0 --dry-run

Prerequisites

  • Clean git working directory
  • All changes committed
  • golangci-lint installed
  • Homebrew installed (for local upgrade)
  • Push access to gastownhall/beads

Output

The script provides colorful, step-by-step progress output:

  • 🟨 Yellow: Current step
  • 🟩 Green: Step completed
  • 🟥 Red: Errors
  • 🟦 Blue: Section headers

What Happens Next

After the script finishes:

  • GitHub Actions builds binaries for all platforms (~5 minutes)
  • PyPI package is published automatically
  • Homebrew core formula is verified or tracked through its canonical PR
  • Users can brew upgrade beads to get the new version after Homebrew merges
  • GitHub Release is created with binaries and changelog

bump-version.sh

Bumps the version number across all beads components in a single command.

Usage

# Show usage
./scripts/bump-version.sh

# Update versions (shows diff, no commit)
./scripts/bump-version.sh 0.9.3

# Update versions and auto-commit
./scripts/bump-version.sh 0.9.3 --commit

What It Does

Updates version in all these files:

  • cmd/bd/version.go - bd CLI version constant
  • plugins/beads/.claude-plugin/plugin.json - Claude plugin version
  • plugins/beads/.codex-plugin/plugin.json - Codex plugin version
  • .claude-plugin/marketplace.json - Claude marketplace plugin version
  • integrations/beads-mcp/pyproject.toml - MCP server version
  • README.md - Alpha status version
  • PLUGIN.md - Version requirements

Features

  • Validates semantic versioning format (MAJOR.MINOR.PATCH)
  • Verifies all versions match after update
  • Shows git diff of changes
  • Auto-commits with standardized message (optional)
  • Cross-platform compatible (macOS and Linux)

Examples

# Bump to 0.9.3 and review changes
./scripts/bump-version.sh 0.9.3
# Review the diff, then manually commit

# Bump to 1.0.0 and auto-commit
./scripts/bump-version.sh 1.0.0 --commit
git push origin main

Why This Script Exists

Previously, version bumps only updated cmd/bd/version.go, leaving other components out of sync. This script ensures all version numbers stay consistent across the project.

Safety

  • Checks for uncommitted changes before proceeding
  • Refuses to auto-commit if there are existing uncommitted changes
  • Validates version format before making any changes
  • Verifies all versions match after update
  • Shows diff for review before commit

sign-windows.sh

Signs Windows executables with an Authenticode certificate using osslsigncode.

Usage

# Sign a Windows executable
./scripts/sign-windows.sh path/to/bd.exe

# Environment variables required for signing:
export WINDOWS_SIGNING_CERT_PFX_BASE64="<base64-encoded-pfx>"
export WINDOWS_SIGNING_CERT_PASSWORD="<certificate-password>"

What It Does

This script is called automatically by GoReleaser during the release process:

  1. Decodes the PFX certificate from base64
  2. Signs the Windows executable using osslsigncode
  3. Timestamps the signature using DigiCert's RFC3161 server
  4. Replaces the original binary with the signed version
  5. Verifies the signature was applied correctly

Prerequisites

  • osslsigncode installed (apt install osslsigncode or brew install osslsigncode)
  • EV code signing certificate exported as PFX file
  • GitHub secrets configured:
    • WINDOWS_SIGNING_CERT_PFX_BASE64 - base64-encoded PFX file
    • WINDOWS_SIGNING_CERT_PASSWORD - certificate password

Graceful Degradation

If the signing secrets are not configured:

  • The script prints a warning and exits successfully
  • GoReleaser continues without signing
  • The release proceeds with unsigned Windows binaries

This allows releases to work before a certificate is acquired.

Why This Script Exists

Windows code signing helps reduce antivirus false positives that affect Go binaries. Kaspersky and other AV software commonly flag unsigned Go executables as potentially malicious due to heuristic detection. See docs/reference/antivirus.md for details.


Future Scripts

Additional maintenance scripts may be added here as needed.