* perf(storage): cut batch-create round trips (import, bulk create, apply-batch)
A multi-issue create (import, bulk create, CreateIssuesWithFullOptions) ran
~20 SQL statements per issue, and its dependency pass ran a recursive cycle
CTE per edge over the union of both dependency tables, which Dolt cannot
index — every recursion step rescanned the edge tables, so an import whose
edges form chains went superlinear (400 issues: 73 s of 82 s in that CTE).
issueops batch create, same transaction and same stored outcome:
- createBatchCache: one IN-list read of every explicit id's presence in both
planes (replacing two COUNT probes per issue), one read of the batch's
stored labels (each issue's new labels then land in one multi-row INSERT),
created/label_added events buffered and written in one pass under the ids
InsertDerivedEvent would mint (same digest, same lowest free ordinal), and
runs of brand-new rows written with multi-row INSERTs (same ODKU clause)
then finished in order.
- Dependency pass: wisp routing and target presence from batch reads; the
cycle and hierarchy checks walk an in-memory copy of the edge tables,
loaded once and kept in step with every edge the pass writes (a pair it
already holds is re-read, since a clientFoundRows connection reports a
matched duplicate as written).
- createBlockedRecomputeIDs seeds all created edges and expands once.
Shared paths:
- Blocked-state recompute splits ids once per call by whether they own a
dependency row; edgeless ids (every freshly created issue) skip the two
union UPDATEs for one plain unmark, run after the chunk's union statements
on the first pass so the unions see what they saw before.
- appendChildrenInTx reads children per IN-list chunk, not per parent.
- ExecuteCreate hands its BatchContext to the batch body instead of having
it re-read the same config (-6 statements per apply-batch create item).
Not changed: one version-history row per created issue (the version
contract, version_history.go), journal order and content, and the per-row
bodies, which stay the path for singular creates.
Equivalence: internal/storage/createbatchequiv runs an import scenario
(upserts, duplicate ids, wisps, generated ids, cycles, hierarchy conflicts,
dangling/external targets, stale is_blocked, comments, journal and
versioned history on) through the fast and per-row bodies on the embedded
and server engines and compares every written table and the skip reports.
Re-pinned TestLargeBatchApplyStatementCounts*: embedded 7009/14014/846 ->
6396/12790/793, server 7008/14014/846 -> 6396/12791/793 (breakdown in
pinnedEmbeddedStatementCounts).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): batch-create review fixes — lazy edge graph, opt-in recompute probe, insert budget
Review of the batch-create round-trip change (S1-S3 and nits):
- The dependency pass's in-memory graph no longer loads both edge tables.
It reads a node's outgoing edges through the issue_id index the first
time a walk reaches it: the batch's own endpoints up front, then one
IN-list read per BFS level. Memory and reads scale with the subgraph
the batch's walks touch, not the edge tables, so a two-edge create on a
large database no longer materialises every edge. A written edge whose
source was never loaded loads that source (row included).
- The blocked-state no-edge probe runs only when a create asks for it or
when a chunk holds at least 32 ids. Single-issue recomputes on close,
reopen, update and dependency edits issue exactly the statements they
did on main (the reopen sqlmock expectation is back to main's).
- Multi-row issue INSERTs are bounded by an estimated 16 MiB of bound
text as well as 100 rows. When a multi-row statement fails, its rows are
replayed one at a time, and the batch continues if every row lands
instead of returning the original error.
- Nits: document the binary-collation assumption behind Go string
equality; DisableCreateFastPathsForTest documents that it is
process-global and non-parallel, and panics on an overlapping caller;
drop the unreachable skip branch; explain why wisp targets need no
presence read.
Re-pinned TestLargeBatchApplyStatementCounts* classic 40: 793 -> 786
(the +7 probes on dep-add recomputes are gone; net vs main is -60, all of
it ExecuteCreate's config reuse). 356/712 unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): search the batch dependency graph from both ends
The lazy per-node load (previous commit) still walked forward from the
check's start, so on a large stored graph a single create's cycle check
read every node reachable from its target — about 800 statements per
create on a 2000-issue fixture. The cycle and hierarchy questions a batch
asks almost always end at a row the batch just created, whose only
incoming edges are the batch's own.
depGraph now answers each reachability question with a bidirectional
search: forward from the start along outgoing edges (issue_id index),
backward from the goal along incoming edges (the three target-column
indexes, read as one UNION ALL per table), always expanding the smaller
frontier and loading a level's nodes in that direction once. The batch's
endpoints are loaded in both directions up front, so a create's checks
usually finish with no further read (a chain import, too: each new
edge's goal has no incoming edge yet). Rows are keyed by (table, source,
target), so a row read from both ends counts once. An incoming read
ignores a row whose matching column is not its first set target column,
as the CTE's COALESCE does. A written pair that neither loaded end can
vouch for is re-read instead of assumed new.
On a 50k-issue / 100k-edge database, a create with two blocks edges
now runs 27 statements, against the full-scan or per-node walks before
(numbers in the review-fix summary).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): hold batch create to golden digests from the pre-change code
The batch-create equivalence test compared the fast paths with the same
code's per-row bodies, so changes the fast-path switch does not gate —
the recompute seed merge, the IN-list children read, ExecuteCreate's
context reuse, the prepare/finish split — were checked only by older
contract tests. Its batch was also too small to cross a chunk boundary
and never set RejectStaleUpserts.
createbatchequiv now runs four scenarios, each through the fast and
per-row bodies, and checks both against a per-table row count and
SHA-256 recorded from
|
||
|---|---|---|
| .. | ||
| bench-ready-indexes | ||
| check-versions | ||
| checkworkflowtags | ||
| ci | ||
| gitattributespolicy | ||
| hooks | ||
| internal/testbash | ||
| migration-test | ||
| pr-lint | ||
| prlintmake | ||
| repro-be-1he-slow-path | ||
| repro-dolt-hang | ||
| repro-dolt-prod-timeouts | ||
| workflowtags | ||
| bazel-autofix-push.sh | ||
| bazel_autofix_push_test.go | ||
| bazel_cache_mode_test.go | ||
| bazel_farm_workflow_test.go | ||
| bazel_fork_mode_test.go | ||
| bazel_policy_test.go | ||
| bazel_rbe_prewarm_test.go | ||
| bazel_skip_allowlist_test.go | ||
| build-examples.sh | ||
| BUILD.bazel | ||
| bump-version.sh | ||
| check-build-tags.sh | ||
| check-cli-docs-drift.sh | ||
| check-doc-flags.sh | ||
| check-doc-freshness.sh | ||
| check-go-install-guidance.sh | ||
| check-migration-hygiene.sh | ||
| check-testing-short.sh | ||
| check-versions.sh | ||
| check-winget-portable-alias.sh | ||
| check_build_tags_test.go | ||
| check_doc_freshness_test.go | ||
| check_release_cross_compile_test.go | ||
| check_testing_short_test.go | ||
| check_winget_portable_alias_test.go | ||
| ci_actions_policy_test.go | ||
| ci_blacksmith_runner_test.go | ||
| ci_capability_selector_test.go | ||
| ci_f7b_review_test.go | ||
| ci_f7c_advisory_test.go | ||
| ci_workflow_apt_get_timeout_test.go | ||
| ci_workflow_test.go | ||
| clean-test-tmp.sh | ||
| complexity_script_test.go | ||
| conformance.sh | ||
| conformance_script_test.go | ||
| cross-version-smoke-test.sh | ||
| docs-autofix-push.sh | ||
| docs.md | ||
| embedded_shard_count_test.go | ||
| gen-winres.sh | ||
| generate-cli-docs.sh | ||
| generate-newsletter.py | ||
| generate-newsletter_README.md | ||
| gh-body-lint | ||
| host_tools_test.go | ||
| install-hooks.sh | ||
| install.sh | ||
| install_dolt_test.go | ||
| migrate-jsonl-to-dolt.sh | ||
| migrate-legacy-to-current.sh | ||
| mutation-equivalence.sh | ||
| pr-preflight.sh | ||
| pr_lanes_bazel_coverage_test.go | ||
| pr_preflight_test.go | ||
| pr_risk_bazel_coverage_test.go | ||
| pull_dolt_image_test.go | ||
| README.md | ||
| release.sh | ||
| release_script_test.go | ||
| required_suite_test.go | ||
| resolve-docs-bd.sh | ||
| sign-windows.sh | ||
| test-cgo.sh | ||
| test-git-remote.sh | ||
| test-icu-path.sh | ||
| test.sh | ||
| test_script_test.go | ||
| update-nix-vendorhash.sh | ||
| update-versions.sh | ||
| update-winget.sh | ||
| upgrade-smoke-test.sh | ||
| upgrade_smoke_loop_test.go | ||
| verify-cgo.sh | ||
| windows_test_binaries_manifest_test.go | ||
Beads Scripts
Utility scripts for maintaining the beads project.
ci/
Repository-owned CI command wrappers. These scripts are the source of truth for the target CI tiers; Make targets are aliases for local discoverability.
make ci-pr-core
make ci-pr-policy
make ci-pr-lint
make ci-package-mcp
make ci-package-npm
Each wrapper auto-detects the repository root, sources .buildflags when it
invokes Go in the default build mode, and records per-command timing through
scripts/ci/lib/timing.sh.
ci-pr-lint keeps its supported Bash/Make entrypoint but delegates native and
Windows/non-CGO target selection to the checkout-owned scripts/pr-lint Go
driver. This is the same shell-free lint authority used by bd preflight in a
Beads source checkout.
Broad Go test wrappers also source scripts/ci/lib/test-env.sh, which creates a
temporary HOME/XDG/Dolt root, isolates Git global/system config, clears runtime
Beads/Dolt environment variables, and sets BEADS_TEST_SKIP=dolt before tests
run. This keeps local make test and make ci-pr-core results comparable to
the fast PR-core contract even on shared agent hosts. Set
BEADS_TEST_ENV_RUN_DOLT=1 only when intentionally running the Dolt-dependent
tests through these broad wrappers, or BEADS_TEST_ENV_DISABLE=1 when debugging
against your real local configuration.
The broad Go wrappers also cap package and test parallelism to 4 by default
(GO_TEST_PKG_PARALLEL and GO_TEST_PARALLEL). This avoids turning high-core
shared hosts into a different test topology than GitHub Actions.
make ci-pr-policy includes scripts/check-testing-short.sh, which enforces
that testing.Short() is only used for runtime, stress, or large-fixture skips.
Use build tags, environment checks, or named wrappers for integration/e2e/API
boundaries.
Package gate wrappers validate publishable/package-adjacent surfaces:
make ci-package-mcpbuilds or consumes abdbinary, puts it onPATHasbd, then runs locked MCP packageuv sync, Ruff, mypy, pytest, and build checks.make ci-package-npmbuilds or consumes the native binary expected bynpm-package/bin/bd, runs the npm package test suite, and checksnpm pack --dry-run.
Set BEADS_TEST_BD_BINARY=/path/to/bd for MCP and npm package gates to reuse a
prebuilt candidate binary instead of rebuilding it inside the wrapper.
pr-preflight.sh
Read-only PR safety check for agents and maintainers.
# Before implementing or opening a related PR
./scripts/pr-preflight.sh --search "topic keywords" --repo gastownhall/beads
# Before changing, closing, or merging an existing PR
./scripts/pr-preflight.sh 123 --repo gastownhall/beads
It reports contributor/fork status, draft/review/merge/check state, risky diff
signals such as .beads/ changes or missing tests, and the required
contributor-protection next steps. It does not replace code review or local
validation.
Base-branch health is supplemented by a warn-only PR-gate sample: the last 60
completed pull_request-event runs, grouped per workflow. A single workflow
whose decisive runs (>= 5, across >= 3 distinct head branches) are all
failure-class is reported as a broken PR gate — the case where a job that
exists only in the PR workflow is red for every PR while the base branch shows
green. This detector never blocks (deliberately: automation classifies
unrecognized [block] lines as genuine merge blockers and would park merge
lanes on a false positive); it is skipped entirely while the base branch is
red.
gh-body-lint
Lint Markdown files before posting them with gh ... --body-file.
./scripts/gh-body-lint body.md
./scripts/gh-body-lint --fix body.md
The lint catches literal \n sequences, which render poorly on GitHub, and
GH#123 references, which do not auto-link like #123 or owner/repo#123.
release.sh (⭐ The Easy Button)
One-command release from version bump to local installation.
Usage
# Full release (does everything)
./scripts/release.sh 0.9.3
# Preview what would happen
./scripts/release.sh 0.9.3 --dry-run
What It Does
This master script automates the entire release process:
- ✅ Stops running Dolt servers (avoids version conflicts)
- ✅ Runs tests and linting
- ✅ Bumps version in all files
- ✅ Commits and pushes version bump
- ✅ Creates and pushes git tag
- ✅ Verifies or opens the Homebrew core formula PR
- ✅ Upgrades local Homebrew installation
- ✅ Verifies everything works
After this script completes, your system is running the new version!
Examples
# Release version 0.9.3
./scripts/release.sh 0.9.3
# Preview a release (no changes made)
./scripts/release.sh 1.0.0 --dry-run
Prerequisites
- Clean git working directory
- All changes committed
- golangci-lint installed
- Homebrew installed (for local upgrade)
- Push access to gastownhall/beads
Output
The script provides colorful, step-by-step progress output:
- 🟨 Yellow: Current step
- 🟩 Green: Step completed
- 🟥 Red: Errors
- 🟦 Blue: Section headers
What Happens Next
After the script finishes:
- GitHub Actions builds binaries for all platforms (~5 minutes)
- PyPI package is published automatically
- Homebrew core formula is verified or tracked through its canonical PR
- Users can
brew upgrade beadsto get the new version after Homebrew merges - GitHub Release is created with binaries and changelog
bump-version.sh
Bumps the version number across all beads components in a single command.
Usage
# Show usage
./scripts/bump-version.sh
# Update versions (shows diff, no commit)
./scripts/bump-version.sh 0.9.3
# Update versions and auto-commit
./scripts/bump-version.sh 0.9.3 --commit
What It Does
Updates version in all these files:
cmd/bd/version.go- bd CLI version constantplugins/beads/.claude-plugin/plugin.json- Claude plugin versionplugins/beads/.codex-plugin/plugin.json- Codex plugin version.claude-plugin/marketplace.json- Claude marketplace plugin versionintegrations/beads-mcp/pyproject.toml- MCP server versionREADME.md- Alpha status versionPLUGIN.md- Version requirements
Features
- Validates semantic versioning format (MAJOR.MINOR.PATCH)
- Verifies all versions match after update
- Shows git diff of changes
- Auto-commits with standardized message (optional)
- Cross-platform compatible (macOS and Linux)
Examples
# Bump to 0.9.3 and review changes
./scripts/bump-version.sh 0.9.3
# Review the diff, then manually commit
# Bump to 1.0.0 and auto-commit
./scripts/bump-version.sh 1.0.0 --commit
git push origin main
Why This Script Exists
Previously, version bumps only updated cmd/bd/version.go, leaving other components out of sync. This script ensures all version numbers stay consistent across the project.
Safety
- Checks for uncommitted changes before proceeding
- Refuses to auto-commit if there are existing uncommitted changes
- Validates version format before making any changes
- Verifies all versions match after update
- Shows diff for review before commit
sign-windows.sh
Signs Windows executables with an Authenticode certificate using osslsigncode.
Usage
# Sign a Windows executable
./scripts/sign-windows.sh path/to/bd.exe
# Environment variables required for signing:
export WINDOWS_SIGNING_CERT_PFX_BASE64="<base64-encoded-pfx>"
export WINDOWS_SIGNING_CERT_PASSWORD="<certificate-password>"
What It Does
This script is called automatically by GoReleaser during the release process:
- Decodes the PFX certificate from base64
- Signs the Windows executable using osslsigncode
- Timestamps the signature using DigiCert's RFC3161 server
- Replaces the original binary with the signed version
- Verifies the signature was applied correctly
Prerequisites
osslsigncodeinstalled (apt install osslsigncodeorbrew install osslsigncode)- EV code signing certificate exported as PFX file
- GitHub secrets configured:
WINDOWS_SIGNING_CERT_PFX_BASE64- base64-encoded PFX fileWINDOWS_SIGNING_CERT_PASSWORD- certificate password
Graceful Degradation
If the signing secrets are not configured:
- The script prints a warning and exits successfully
- GoReleaser continues without signing
- The release proceeds with unsigned Windows binaries
This allows releases to work before a certificate is acquired.
Why This Script Exists
Windows code signing helps reduce antivirus false positives that affect Go binaries.
Kaspersky and other AV software commonly flag unsigned Go executables as potentially
malicious due to heuristic detection. See docs/reference/antivirus.md for details.
Future Scripts
Additional maintenance scripts may be added here as needed.