beads/scripts
Steve Yegge e97839a2e1
feat(lease): land the leasing stack + schema v54 (claim-TTL, heartbeat, reclaim) (#4537)
* fix(dolt): retry mutate+commit methods on serialization conflict (1213/1205)

ClaimIssue, ClaimReadyIssue, UpdateIssue and CloseIssue each hand-rolled
their own tx (BeginTx + DOLT_ADD/DOLT_COMMIT + Commit) outside
withRetryTx, so a concurrent writer that lost Dolt's optimistic
commit-time merge surfaced MySQL 1213/1205 as a hard failure instead of
retrying. Dolt has no real row locking — FOR UPDATE / SKIP LOCKED are
parse-only no-ops (https://www.dolthub.com/blog/2023-10-23-hold-my-beer/)
— so retry is the only safety net under N concurrent workers. These four
are the work-queue hot paths: `bd ready --claim`, claim, update, close.

Move each body into a withRetryTx closure (which owns BeginTx and the
final Commit and retries serialization failures with backoff), preserving
the single-tx CAS/mutation + DOLT_COMMIT atomicity exactly. On retry,
ClaimReadyIssue re-scans the ready front from a fresh snapshot and claims
the next available issue.

Tested: TestEmbeddedReadyClaimConcurrent, TestEmbeddedReadyConcurrent,
TestEmbeddedCloseConcurrent, TestEmbeddedUpdateConcurrent and the
non-concurrent embedded Update/Close/Ready tests pass; build + vet clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(dolt): add concurrent work-queue drain test (Gas Station claim invariant)

N workers concurrently dequeue one shared ready-front via ClaimReadyIssue
until empty; asserts every issue claimed exactly once, none stranded, no
double-claims, and no serialization error ever surfaces (withRetryTx absorbs
the 1213/1205 from the claim-CAS cell collision). This is the regression guard
for the multi-agent claim queue that #4491 makes safe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(lease): claim-TTL + heartbeat + reclaim for dead-worker recovery

Gas Station v1.1 (wy-5r9j). A claim was previously permanent: a worker that
died mid-task stranded its issue in_progress forever. This adds a lease so a
claim can expire and be recovered.

Schema (migration 0054): add lease_expires_at, heartbeat_at (DATETIME) and
row_lock (BIGINT) to issues (and wisps, for uniform issueops routing), plus
idx_issues_lease for the reaper scan. assignee doubles as the lease owner.

Lifecycle (shared issueops, so both server-mode DoltStore and EmbeddedDoltStore
get it):
  - Claim stamps lease_expires_at = now + TTL (default 5m, WithLeaseTTL to
    override), heartbeat_at = now, and a fresh row_lock.
  - bd heartbeat <id> — owner-only; pushes the lease forward and rewrites
    row_lock. Fails (ErrNotClaimable/ErrAlreadyClaimed) once the lease is gone.
  - bd reclaim --older-than <dur> — reverts in_progress issues whose lease
    expired more than <dur> ago back to ready (clears assignee/started_at,
    records a lease_reclaimed event). Default grace 2×TTL.

The row_lock trick: Dolt has no row locking and merges concurrent commits
cell-by-cell, so a heartbeat (writes heartbeat_at) racing a reclaim (writes
status) would silently cell-merge into a zombie — an open/unassigned issue that
still carries the worker's fresh heartbeat. Every mutating path (claim, close,
update, heartbeat, reclaim) now rewrites the shared row_lock cell, forcing the
loser to a 1213/1205 serialization conflict that withRetryTx replays. All lease
paths are withRetryTx/withConn wrapped.

Tests (real Dolt): TestRowLockForcesConflictOnDisjointCellWrites proves the bug
without row_lock (silent merge to zombie) and correctness with it (conflict);
TestConcurrentHeartbeatReclaimClose is the integration race (live workers
heartbeat+close vs a continuous reaper vs dead workers) asserting exactly-once,
no lost close, no zombie; plus claim/heartbeat/reclaim unit coverage on both
backends.

Note: lease columns are second-granular DATETIME (Dolt rounds), so sub-second
TTLs are meaningless — production TTL is minutes. Heartbeat writes a Dolt
commit, so cadence must stay ≪ claim cadence to avoid history bloat.

Fixes the test coupling in TestMigration0053PromotesRigWisps, which assumed the
rig-wisp repair was always LatestVersion().

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci+docs: allowlist lease stress tests; scope the row_lock invariant comment

- Add TestConcurrentWorkQueueDrain and TestConcurrentHeartbeatReclaimClose
  to the check-testing-short.sh allowlist; both are real-Dolt runtime/stress
  skips and were failing ci-pr-policy.
- freshRowLock doc said "EVERY mutating path" rewrites row_lock, which
  overstates it. Scope it to status/ownership/lease-mutating paths (the set
  the reclaim/heartbeat races care about) and document the invariant + why
  orthogonal-cell writers (is_blocked, compaction, deps, rename, reopen) are
  intentionally exempt, so a future writer does not silently reintroduce the
  zombie-merge bug.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(show): surface lease_expires_at/heartbeat_at in bd show (--json + text) (wy-9cdw)

Migration 0054 added lease_expires_at/heartbeat_at (and internal row_lock) to
the issues/wisps tables for the claim/heartbeat/reclaim leasing feature, but
bd show could not observe lease state — you had to drop to raw SQL.

- types.Issue: add LeaseExpiresAt/HeartbeatAt (*time.Time, omitempty), matching
  the StartedAt nullable-timestamp convention. row_lock stays unexported.
- sqlbuild.IssueSelectColumns: append lease_expires_at, heartbeat_at to the
  canonical hydration list (flows to every stack via the shared constant).
- issueops.ScanIssueFrom: scan + map the two nullable columns (before the
  trailing extra dests, keeping positional parity across all scan sites).
- bd show text view: emit a muted 'Lease: expires <rel> (heartbeat <rel> ago)'
  line only for in_progress issues holding an active lease; add formatTimeUntil
  as the forward-looking mirror of formatTimeAgo.
- types_test: assert lease keys appear when set, are omitted when nil, and that
  row_lock is never surfaced.

* fix(lease): settle v54 review blockers

Address #4537 review blockers before landing v54: keep heartbeat off wisps in both backends, keep direct status/assignee updates from leaving stale leases, guard the down migration, and emit reclaim JSON as snake_case.

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie

* fix(lease): cover lease fields in tests and hooks

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-03 23:28:55 -07:00
..
bench-ready-indexes test: add perf reference benchmarks (#4032) 2026-05-19 18:39:13 -07:00
ci docs: publish community tools on website (#4299) 2026-06-03 13:44:47 -07:00
hooks feat: Add post-push hook to auto-install bd CLI after git push 2025-10-14 17:23:43 -07:00
migration-test docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
repro-dolt-hang /{cmd,internal,scripts}: use dsn helpers 2026-04-06 15:56:37 -07:00
repro-dolt-prod-timeouts fix(deps): deterministic dependency primary keys for cross-clone merge-safety (#4259) 2026-06-07 14:47:58 -07:00
bump-version.sh refactor: replace bump-version.sh with molecule pointer 2026-01-14 21:53:06 -08:00
check-build-tags.sh chore(build): codify the gms_pure_go build-tag policy (GH#3303) 2026-04-16 12:28:11 -07:00
check-cli-docs-drift.sh fix(docgen): generate CLI docs in one process (#4474) 2026-06-22 18:25:58 -07:00
check-doc-flags.sh fix(docgen): generate CLI docs in one process (#4474) 2026-06-22 18:25:58 -07:00
check-doc-freshness.sh docs: check reference doc freshness (#4181) 2026-05-26 15:59:12 -07:00
check-docs-version.sh chore: re-land v1.1.0-rc.1 release prep (#4486) 2026-06-25 20:10:18 -07:00
check-go-install-guidance.sh fix(install): use declared module path for go fallback (#4215) 2026-05-27 21:48:24 -07:00
check-migration-hygiene.sh fix(audit): sand down the P3 sharp edges from the June 9-11 audit (bd-578h9.17) 2026-06-11 19:54:21 -04:00
check-testing-short.sh feat(lease): land the leasing stack + schema v54 (claim-TTL, heartbeat, reclaim) (#4537) 2026-07-03 23:28:55 -07:00
check-versions.sh Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
clean-test-tmp.sh test(cmd/bd): route sync.Once test temp dirs through TestMain cleanup (#4108) 2026-05-22 16:48:22 -07:00
cross-version-smoke-test.sh docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
docs.md fix(docgen): generate CLI docs in one process (#4474) 2026-06-22 18:25:58 -07:00
gen-winres.sh chore: re-land v1.1.0-rc.1 release prep (#4486) 2026-06-25 20:10:18 -07:00
generate-cli-docs.sh fix(scripts): document CGO override + stderr-warn in CLI docgen guard (#4532) 2026-07-01 23:00:20 -07:00
generate-llms-full.sh /{docs,website,scripts}: holy fuck again 2026-06-03 15:10:34 -07:00
generate-newsletter.py Newsletter generator: automated narrative summaries of releases 2026-01-22 16:09:29 -08:00
generate-newsletter_README.md Newsletter generator: automated narrative summaries of releases 2026-01-22 16:09:29 -08:00
gh-body-lint docs: align agent maintainer conventions (#4237) 2026-06-20 20:44:08 -07:00
install-hooks.sh feat: Add post-push hook to auto-install bd CLI after git push 2025-10-14 17:23:43 -07:00
install.sh fix(install): use declared module path for go fallback (#4215) 2026-05-27 21:48:24 -07:00
migrate-jsonl-to-dolt.sh feat: add JSONL-to-Dolt migration script (GH#2276) 2026-03-02 22:05:38 -08:00
migrate-sqlite-to-current.sh docs: add cross-era migration instructions to upgrading guide (#3081) 2026-04-06 20:11:00 -07:00
pr-preflight.sh Add PR preflight and contributor intake guidance (#3470) 2026-04-24 15:44:06 -07:00
README.md docs: align agent maintainer conventions (#4237) 2026-06-20 20:44:08 -07:00
release.sh fix(release): harden shipped release formula adoption (#3668) 2026-05-06 12:14:54 -07:00
release_script_test.go fix(release): harden shipped release formula adoption (#3668) 2026-05-06 12:14:54 -07:00
sign-windows.sh docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
snapshot-release-docs.sh fix(docgen): generate CLI docs in one process (#4474) 2026-06-22 18:25:58 -07:00
sync-website-docs.sh docs: publish community tools on website (#4299) 2026-06-03 13:44:47 -07:00
test-cgo.sh Rename ICU-only test path to explicit opt-in workflow (bd-b18) (#3291) 2026-04-16 08:54:24 -07:00
test-git-remote.sh /{.github,docs,go,internal,scripts}: bump driver/dolt version to 2.1.4 (#4326) 2026-06-08 11:31:14 -07:00
test-icu-path.sh Rename ICU-only test path to explicit opt-in workflow (bd-b18) (#3291) 2026-04-16 08:54:24 -07:00
test.sh Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
update-nix-vendorhash.sh feat: comprehensive NixOS support improvements (#1024) 2026-01-12 23:46:33 -08:00
update-versions.sh chore: re-land v1.1.0-rc.1 release prep (#4486) 2026-06-25 20:10:18 -07:00
update-winget.sh docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
upgrade-smoke-test.sh test(upgrade-smoke): exercise dependency blocker paths on migrated DB (#4238) 2026-05-28 15:30:21 -07:00
verify-cgo.sh fix: remove ICU runtime dependency from release binaries (#3066) 2026-04-07 09:48:44 -07:00

Beads Scripts

Utility scripts for maintaining the beads project.

ci/

Repository-owned CI command wrappers. These scripts are the source of truth for the target CI tiers; Make targets are aliases for local discoverability.

make ci-pr-core
make ci-pr-policy
make ci-pr-lint
make ci-package-mcp
make ci-package-npm
make ci-website

Each wrapper auto-detects the repository root, sources .buildflags when it invokes Go in the default build mode, and records per-command timing through scripts/ci/lib/timing.sh.

Broad Go test wrappers also source scripts/ci/lib/test-env.sh, which creates a temporary HOME/XDG/Dolt root, isolates Git global/system config, clears runtime Beads/Dolt environment variables, and sets BEADS_TEST_SKIP=dolt before tests run. This keeps local make test and make ci-pr-core results comparable to the fast PR-core contract even on shared agent hosts. Set BEADS_TEST_ENV_RUN_DOLT=1 only when intentionally running the Dolt-dependent tests through these broad wrappers, or BEADS_TEST_ENV_DISABLE=1 when debugging against your real local configuration.

The broad Go wrappers also cap package and test parallelism to 4 by default (GO_TEST_PKG_PARALLEL and GO_TEST_PARALLEL). This avoids turning high-core shared hosts into a different test topology than GitHub Actions.

make ci-pr-policy includes scripts/check-testing-short.sh, which enforces that testing.Short() is only used for runtime, stress, or large-fixture skips. Use build tags, environment checks, or named wrappers for integration/e2e/API boundaries.

Package gate wrappers validate publishable/package-adjacent surfaces:

  • make ci-package-mcp builds or consumes a bd binary, puts it on PATH as bd, then runs locked MCP package uv sync, Ruff, mypy, pytest, and build checks.
  • make ci-package-npm builds or consumes the native binary expected by npm-package/bin/bd, runs the npm package test suite, and checks npm pack --dry-run.
  • make ci-website runs website dependency install, typecheck, llms-full.txt generation, and Docusaurus build.

Set BEADS_TEST_BD_BINARY=/path/to/bd for MCP and npm package gates to reuse a prebuilt candidate binary instead of rebuilding it inside the wrapper.

pr-preflight.sh

Read-only PR safety check for agents and maintainers.

# Before implementing or opening a related PR
./scripts/pr-preflight.sh --search "topic keywords" --repo gastownhall/beads

# Before changing, closing, or merging an existing PR
./scripts/pr-preflight.sh 123 --repo gastownhall/beads

It reports contributor/fork status, draft/review/merge/check state, risky diff signals such as .beads/ changes or missing tests, and the required contributor-protection next steps. It does not replace code review or local validation.

gh-body-lint

Lint Markdown files before posting them with gh ... --body-file.

./scripts/gh-body-lint body.md
./scripts/gh-body-lint --fix body.md

The lint catches literal \n sequences, which render poorly on GitHub, and GH#123 references, which do not auto-link like #123 or owner/repo#123.

release.sh (⭐ The Easy Button)

One-command release from version bump to local installation.

Usage

# Full release (does everything)
./scripts/release.sh 0.9.3

# Preview what would happen
./scripts/release.sh 0.9.3 --dry-run

What It Does

This master script automates the entire release process:

  1. ✅ Stops running Dolt servers (avoids version conflicts)
  2. ✅ Runs tests and linting
  3. ✅ Bumps version in all files
  4. ✅ Commits and pushes version bump
  5. ✅ Creates and pushes git tag
  6. ✅ Updates Homebrew formula
  7. ✅ Upgrades local brew installation
  8. ✅ Verifies everything works

After this script completes, your system is running the new version!

Examples

# Release version 0.9.3
./scripts/release.sh 0.9.3

# Preview a release (no changes made)
./scripts/release.sh 1.0.0 --dry-run

Prerequisites

  • Clean git working directory
  • All changes committed
  • golangci-lint installed
  • Homebrew installed (for local upgrade)
  • Push access to gastownhall/beads

Output

The script provides colorful, step-by-step progress output:

  • 🟨 Yellow: Current step
  • 🟩 Green: Step completed
  • 🟥 Red: Errors
  • 🟦 Blue: Section headers

What Happens Next

After the script finishes:

  • GitHub Actions builds binaries for all platforms (~5 minutes)
  • PyPI package is published automatically
  • Users can brew upgrade beads to get the new version
  • GitHub Release is created with binaries and changelog

bump-version.sh

Bumps the version number across all beads components in a single command.

Usage

# Show usage
./scripts/bump-version.sh

# Update versions (shows diff, no commit)
./scripts/bump-version.sh 0.9.3

# Update versions and auto-commit
./scripts/bump-version.sh 0.9.3 --commit

What It Does

Updates version in all these files:

  • cmd/bd/version.go - bd CLI version constant
  • plugins/beads/.claude-plugin/plugin.json - Claude plugin version
  • plugins/beads/.codex-plugin/plugin.json - Codex plugin version
  • .claude-plugin/marketplace.json - Claude marketplace plugin version
  • integrations/beads-mcp/pyproject.toml - MCP server version
  • README.md - Alpha status version
  • PLUGIN.md - Version requirements

Features

  • Validates semantic versioning format (MAJOR.MINOR.PATCH)
  • Verifies all versions match after update
  • Shows git diff of changes
  • Auto-commits with standardized message (optional)
  • Cross-platform compatible (macOS and Linux)

Examples

# Bump to 0.9.3 and review changes
./scripts/bump-version.sh 0.9.3
# Review the diff, then manually commit

# Bump to 1.0.0 and auto-commit
./scripts/bump-version.sh 1.0.0 --commit
git push origin main

Why This Script Exists

Previously, version bumps only updated cmd/bd/version.go, leaving other components out of sync. This script ensures all version numbers stay consistent across the project.

Safety

  • Checks for uncommitted changes before proceeding
  • Refuses to auto-commit if there are existing uncommitted changes
  • Validates version format before making any changes
  • Verifies all versions match after update
  • Shows diff for review before commit

sign-windows.sh

Signs Windows executables with an Authenticode certificate using osslsigncode.

Usage

# Sign a Windows executable
./scripts/sign-windows.sh path/to/bd.exe

# Environment variables required for signing:
export WINDOWS_SIGNING_CERT_PFX_BASE64="<base64-encoded-pfx>"
export WINDOWS_SIGNING_CERT_PASSWORD="<certificate-password>"

What It Does

This script is called automatically by GoReleaser during the release process:

  1. Decodes the PFX certificate from base64
  2. Signs the Windows executable using osslsigncode
  3. Timestamps the signature using DigiCert's RFC3161 server
  4. Replaces the original binary with the signed version
  5. Verifies the signature was applied correctly

Prerequisites

  • osslsigncode installed (apt install osslsigncode or brew install osslsigncode)
  • EV code signing certificate exported as PFX file
  • GitHub secrets configured:
    • WINDOWS_SIGNING_CERT_PFX_BASE64 - base64-encoded PFX file
    • WINDOWS_SIGNING_CERT_PASSWORD - certificate password

Graceful Degradation

If the signing secrets are not configured:

  • The script prints a warning and exits successfully
  • GoReleaser continues without signing
  • The release proceeds with unsigned Windows binaries

This allows releases to work before a certificate is acquired.

Why This Script Exists

Windows code signing helps reduce antivirus false positives that affect Go binaries. Kaspersky and other AV software commonly flag unsigned Go executables as potentially malicious due to heuristic detection. See docs/ANTIVIRUS.md for details.


Future Scripts

Additional maintenance scripts may be added here as needed.