beads/scripts
Julian Knutsen ddca3e3ef1
fix(security,ci): redact all DSN password forms in telemetry; require pg/mysql conformance (post-merge #4601) (#4712)
* fix(security,ci): redact all DSN password forms in telemetry; require pg/mysql conformance

Post-merge remediation for #4601 (choose-your-own-backend), reviewed landed
range 812ad6d..3f0474a. Addresses the two gating findings from the post-merge
review.

Security (telemetry DSN leak): scrubArgsForTelemetry only redacted
user:PASS@host userinfo, so `bd init --pg-url` URL `?password=`/`?sslpassword=`
query params and libpq `password=`/`sslpassword=` keyword/value forms leaked
into the bd.args telemetry span before init reached the parser-backed
redaction path. Route --pg-url/--mysql-url values (both `--flag=value` and
`--flag value`) through the parser-backed pgdialect scrubber via a new exported
pgdialect.ScrubDSNString; every other arg keeps the narrow userinfo scrub, so
ordinary text is never over-redacted.

CI (conformance anti-skip): scripts/conformance.sh ran the pg/mysql Tier-1
suites with no skip/PASS detector, so a missing BEADS_*_TEST_URL self-skipped
to green and silently narrowed backend coverage. Add require_backend +
run_sql_conformance mirroring the embedded-Dolt reference guard, and set
BEADS_CONFORMANCE_REQUIRE=postgres,mysql in conformance.yml (which also arms
the existing Tier-2 e2e hard-fail).

Tests: pgdialect TestScrubDSNString; cmd/bd TestScrubArgsForTelemetry (all DSN
password forms x both spellings) and TestScrubArgsForTelemetryLeavesOrdinaryArgs
(no over-redaction + userinfo defense-in-depth). Bash guards unit- and
integration-tested against real `go test` self-skip/required paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): redact federation add-peer --password/-p from telemetry args

The bd.args telemetry root span scrubbed --pg-url/--mysql-url DSN passwords
but not federation add-peer's --password/-p, so `bd federation add-peer
<name> <url> --user u --password <secret>` leaked the plaintext SQL
password verbatim into the exported bd.args attribute and trace logs — the
same sink and harm class the DSN scrub already closed.

Redact by flag identity: secretFlagTokens resolves the secret flag's actual
--long/-short tokens from the running command, so -p is redacted only on the
command that binds it to --password (federation add-peer) and never on the
many commands that bind -p to --priority/--prefix/--parallel. The value is
redacted wholesale across every pflag spelling: --password <v>,
--password=<v>, -p <v>, -p=<v>, and the concatenated -p<v> shorthand.

Tests cover all five spellings, the command-aware token resolution, and the
no-over-redaction guarantee for the overloaded -p on non-secret commands.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): redact raw encoded URL-userinfo passwords in DSN scrubber

Post-merge remediation follow-up review (attempt-4) for #4601: the shared
pgdialect DSN redactor missed percent-encoded URL-userinfo passwords when
called directly — ScrubDSNError in driver.go, and any telemetry/log sink that
scrubs a raw DSN through ScrubDSNString.

dsnPasswordValues added only url.User.Password(), which url.Parse returns
percent-DECODED, so ScrubDSNString("postgres://u:SUPER%2ASECRET@h/db", same)
left the raw "SUPER%2ASECRET" in error text and logs. The query-param branch
already adds both the raw and decoded forms; the userinfo branch now does too,
extracting the raw as-written password structurally (rawURLUserinfoPassword)
so it matches regardless of encoding form (mixed case, over-encoding).

Tests: add encoded-userinfo cases to TestScrubDSNError and TestScrubDSNString
(the latter now checks every leak form, matching TestScrubDSNError). Both new
cases fail on the pre-fix redactor and pass after.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): close five telemetry DSN-redaction bypasses found in review

Test-first fixes, each reproduced against pgx.ParseConfig before fixing:

- percent-encoded query KEYS (pass%77ord=, %70assword=) are live password
  params after url.Query() decoding; compare unescaped keys too
- pflag boolean-shorthand clusters (-qpSECRET parses as -q -p SECRET) leaked
  the raw token; walk the leading letter run to the first secret shorthand
- backslash-escaped whitespace in unquoted libpq values (password=SUPER\ SECRET)
  leaked the tail; value pattern now consumes escaped sequences
- pgx treats \v as DSN whitespace but Go regexp \s does not; spell the
  whitespace class out explicitly
- overlapping secrets (one a prefix of another) leaked the longer tail;
  replace longest-first in ScrubDSNString

Found by paired cross-vendor review (claude fable-5 + codex gpt-5.6-sol) on
PR 4712.

Agent-Signature: claude-fable-5-high on behalf of maphew

---------

Co-authored-by: Eddie the Engineer <test@test.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-12 13:44:35 -07:00
..
bench-ready-indexes feat: choose your own storage backend (Postgres / MySQL / SQLite) — rebased on main 2026-07-10 17:44:37 +00:00
ci docs: publish community tools on website (#4299) 2026-06-03 13:44:47 -07:00
hooks feat: Add post-push hook to auto-install bd CLI after git push 2025-10-14 17:23:43 -07:00
migration-test test(migration): assert bd ready/blocked/close on migrated DB (mybd-ihg5) (#4584) 2026-07-07 13:32:37 -07:00
repro-dolt-hang /{cmd,internal,scripts}: use dsn helpers 2026-04-06 15:56:37 -07:00
repro-dolt-prod-timeouts feat: choose your own storage backend (Postgres / MySQL / SQLite) — rebased on main 2026-07-10 17:44:37 +00:00
bump-version.sh refactor: replace bump-version.sh with molecule pointer 2026-01-14 21:53:06 -08:00
check-build-tags.sh chore(build): codify the gms_pure_go build-tag policy (GH#3303) 2026-04-16 12:28:11 -07:00
check-cli-docs-drift.sh fix(docgen): generate CLI docs in one process (#4474) 2026-06-22 18:25:58 -07:00
check-doc-flags.sh fix(docgen): generate CLI docs in one process (#4474) 2026-06-22 18:25:58 -07:00
check-doc-freshness.sh docs: check reference doc freshness (#4181) 2026-05-26 15:59:12 -07:00
check-docs-version.sh chore: re-land v1.1.0-rc.1 release prep (#4486) 2026-06-25 20:10:18 -07:00
check-go-install-guidance.sh fix(install): use declared module path for go fallback (#4215) 2026-05-27 21:48:24 -07:00
check-migration-hygiene.sh fix(audit): sand down the P3 sharp edges from the June 9-11 audit (bd-578h9.17) 2026-06-11 19:54:21 -04:00
check-testing-short.sh feat(prune): reference-aware bd prune skips closed beads cited by open beads (be-jewoem, be-u2mw2x) (#4023) 2026-07-05 15:57:27 -07:00
check-versions.sh Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
check-website-docs-drift.sh docs(website): refresh committed mirrors and add CI drift gate (#4661) 2026-07-08 08:23:47 -07:00
clean-test-tmp.sh test(cmd/bd): route sync.Once test temp dirs through TestMain cleanup (#4108) 2026-05-22 16:48:22 -07:00
conformance.sh fix(security,ci): redact all DSN password forms in telemetry; require pg/mysql conformance (post-merge #4601) (#4712) 2026-07-12 13:44:35 -07:00
cross-version-smoke-test.sh docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
docs-autofix-push.sh ci(docs): auto-apply CLI docs regeneration to PR branches (#4653) 2026-07-07 22:16:37 -07:00
docs.md fix(docgen): generate CLI docs in one process (#4474) 2026-06-22 18:25:58 -07:00
gen-winres.sh chore: re-land v1.1.0-rc.1 release prep (#4486) 2026-06-25 20:10:18 -07:00
generate-cli-docs.sh fix(scripts): document CGO override + stderr-warn in CLI docgen guard (#4532) 2026-07-01 23:00:20 -07:00
generate-llms-full.sh /{docs,website,scripts}: holy fuck again 2026-06-03 15:10:34 -07:00
generate-newsletter.py Newsletter generator: automated narrative summaries of releases 2026-01-22 16:09:29 -08:00
generate-newsletter_README.md Newsletter generator: automated narrative summaries of releases 2026-01-22 16:09:29 -08:00
gh-body-lint docs: align agent maintainer conventions (#4237) 2026-06-20 20:44:08 -07:00
install-hooks.sh feat: Add post-push hook to auto-install bd CLI after git push 2025-10-14 17:23:43 -07:00
install.sh fix(install): use declared module path for go fallback (#4215) 2026-05-27 21:48:24 -07:00
migrate-jsonl-to-dolt.sh feat: add JSONL-to-Dolt migration script (GH#2276) 2026-03-02 22:05:38 -08:00
migrate-sqlite-to-current.sh docs: add cross-era migration instructions to upgrading guide (#3081) 2026-04-06 20:11:00 -07:00
pr-preflight.sh fix(preflight): treat timed-out base CI as red 2026-07-07 07:15:18 -07:00
README.md docs(release): harden Homebrew formula procedure (#4571) 2026-07-04 10:46:49 -07:00
release.sh fix(release): harden shipped release formula adoption (#3668) 2026-05-06 12:14:54 -07:00
release_script_test.go test: port Windows hardening fixes (#4600) 2026-07-07 12:59:50 -07:00
run-oracle-p.sh feat: choose your own storage backend (Postgres / MySQL / SQLite) — rebased on main 2026-07-10 17:44:37 +00:00
sign-windows.sh docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
snapshot-release-docs.sh fix(docgen): generate CLI docs in one process (#4474) 2026-06-22 18:25:58 -07:00
sync-website-docs.sh docs: publish community tools on website (#4299) 2026-06-03 13:44:47 -07:00
test-cgo.sh Rename ICU-only test path to explicit opt-in workflow (bd-b18) (#3291) 2026-04-16 08:54:24 -07:00
test-git-remote.sh /{.github,docs,go,internal,scripts}: bump dolt min version and driver 2026-07-06 16:04:11 -07:00
test-icu-path.sh Rename ICU-only test path to explicit opt-in workflow (bd-b18) (#3291) 2026-04-16 08:54:24 -07:00
test.sh Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
update-nix-vendorhash.sh feat: comprehensive NixOS support improvements (#1024) 2026-01-12 23:46:33 -08:00
update-versions.sh chore: re-land v1.1.0-rc.1 release prep (#4486) 2026-06-25 20:10:18 -07:00
update-winget.sh docs: update user-facing URLs from steveyegge/beads to gastownhall/beads 2026-04-24 08:17:42 -07:00
upgrade-smoke-test.sh test(upgrade-smoke): exercise dependency blocker paths on migrated DB (#4238) 2026-05-28 15:30:21 -07:00
verify-cgo.sh fix: remove ICU runtime dependency from release binaries (#3066) 2026-04-07 09:48:44 -07:00

Beads Scripts

Utility scripts for maintaining the beads project.

ci/

Repository-owned CI command wrappers. These scripts are the source of truth for the target CI tiers; Make targets are aliases for local discoverability.

make ci-pr-core
make ci-pr-policy
make ci-pr-lint
make ci-package-mcp
make ci-package-npm
make ci-website

Each wrapper auto-detects the repository root, sources .buildflags when it invokes Go in the default build mode, and records per-command timing through scripts/ci/lib/timing.sh.

Broad Go test wrappers also source scripts/ci/lib/test-env.sh, which creates a temporary HOME/XDG/Dolt root, isolates Git global/system config, clears runtime Beads/Dolt environment variables, and sets BEADS_TEST_SKIP=dolt before tests run. This keeps local make test and make ci-pr-core results comparable to the fast PR-core contract even on shared agent hosts. Set BEADS_TEST_ENV_RUN_DOLT=1 only when intentionally running the Dolt-dependent tests through these broad wrappers, or BEADS_TEST_ENV_DISABLE=1 when debugging against your real local configuration.

The broad Go wrappers also cap package and test parallelism to 4 by default (GO_TEST_PKG_PARALLEL and GO_TEST_PARALLEL). This avoids turning high-core shared hosts into a different test topology than GitHub Actions.

make ci-pr-policy includes scripts/check-testing-short.sh, which enforces that testing.Short() is only used for runtime, stress, or large-fixture skips. Use build tags, environment checks, or named wrappers for integration/e2e/API boundaries.

Package gate wrappers validate publishable/package-adjacent surfaces:

  • make ci-package-mcp builds or consumes a bd binary, puts it on PATH as bd, then runs locked MCP package uv sync, Ruff, mypy, pytest, and build checks.
  • make ci-package-npm builds or consumes the native binary expected by npm-package/bin/bd, runs the npm package test suite, and checks npm pack --dry-run.
  • make ci-website runs website dependency install, typecheck, llms-full.txt generation, and Docusaurus build.

Set BEADS_TEST_BD_BINARY=/path/to/bd for MCP and npm package gates to reuse a prebuilt candidate binary instead of rebuilding it inside the wrapper.

pr-preflight.sh

Read-only PR safety check for agents and maintainers.

# Before implementing or opening a related PR
./scripts/pr-preflight.sh --search "topic keywords" --repo gastownhall/beads

# Before changing, closing, or merging an existing PR
./scripts/pr-preflight.sh 123 --repo gastownhall/beads

It reports contributor/fork status, draft/review/merge/check state, risky diff signals such as .beads/ changes or missing tests, and the required contributor-protection next steps. It does not replace code review or local validation.

gh-body-lint

Lint Markdown files before posting them with gh ... --body-file.

./scripts/gh-body-lint body.md
./scripts/gh-body-lint --fix body.md

The lint catches literal \n sequences, which render poorly on GitHub, and GH#123 references, which do not auto-link like #123 or owner/repo#123.

release.sh (⭐ The Easy Button)

One-command release from version bump to local installation.

Usage

# Full release (does everything)
./scripts/release.sh 0.9.3

# Preview what would happen
./scripts/release.sh 0.9.3 --dry-run

What It Does

This master script automates the entire release process:

  1. ✅ Stops running Dolt servers (avoids version conflicts)
  2. ✅ Runs tests and linting
  3. ✅ Bumps version in all files
  4. ✅ Commits and pushes version bump
  5. ✅ Creates and pushes git tag
  6. ✅ Verifies or opens the Homebrew core formula PR
  7. ✅ Upgrades local Homebrew installation
  8. ✅ Verifies everything works

After this script completes, your system is running the new version!

Examples

# Release version 0.9.3
./scripts/release.sh 0.9.3

# Preview a release (no changes made)
./scripts/release.sh 1.0.0 --dry-run

Prerequisites

  • Clean git working directory
  • All changes committed
  • golangci-lint installed
  • Homebrew installed (for local upgrade)
  • Push access to gastownhall/beads

Output

The script provides colorful, step-by-step progress output:

  • 🟨 Yellow: Current step
  • 🟩 Green: Step completed
  • 🟥 Red: Errors
  • 🟦 Blue: Section headers

What Happens Next

After the script finishes:

  • GitHub Actions builds binaries for all platforms (~5 minutes)
  • PyPI package is published automatically
  • Homebrew core formula is verified or tracked through its canonical PR
  • Users can brew upgrade beads to get the new version after Homebrew merges
  • GitHub Release is created with binaries and changelog

bump-version.sh

Bumps the version number across all beads components in a single command.

Usage

# Show usage
./scripts/bump-version.sh

# Update versions (shows diff, no commit)
./scripts/bump-version.sh 0.9.3

# Update versions and auto-commit
./scripts/bump-version.sh 0.9.3 --commit

What It Does

Updates version in all these files:

  • cmd/bd/version.go - bd CLI version constant
  • plugins/beads/.claude-plugin/plugin.json - Claude plugin version
  • plugins/beads/.codex-plugin/plugin.json - Codex plugin version
  • .claude-plugin/marketplace.json - Claude marketplace plugin version
  • integrations/beads-mcp/pyproject.toml - MCP server version
  • README.md - Alpha status version
  • PLUGIN.md - Version requirements

Features

  • Validates semantic versioning format (MAJOR.MINOR.PATCH)
  • Verifies all versions match after update
  • Shows git diff of changes
  • Auto-commits with standardized message (optional)
  • Cross-platform compatible (macOS and Linux)

Examples

# Bump to 0.9.3 and review changes
./scripts/bump-version.sh 0.9.3
# Review the diff, then manually commit

# Bump to 1.0.0 and auto-commit
./scripts/bump-version.sh 1.0.0 --commit
git push origin main

Why This Script Exists

Previously, version bumps only updated cmd/bd/version.go, leaving other components out of sync. This script ensures all version numbers stay consistent across the project.

Safety

  • Checks for uncommitted changes before proceeding
  • Refuses to auto-commit if there are existing uncommitted changes
  • Validates version format before making any changes
  • Verifies all versions match after update
  • Shows diff for review before commit

sign-windows.sh

Signs Windows executables with an Authenticode certificate using osslsigncode.

Usage

# Sign a Windows executable
./scripts/sign-windows.sh path/to/bd.exe

# Environment variables required for signing:
export WINDOWS_SIGNING_CERT_PFX_BASE64="<base64-encoded-pfx>"
export WINDOWS_SIGNING_CERT_PASSWORD="<certificate-password>"

What It Does

This script is called automatically by GoReleaser during the release process:

  1. Decodes the PFX certificate from base64
  2. Signs the Windows executable using osslsigncode
  3. Timestamps the signature using DigiCert's RFC3161 server
  4. Replaces the original binary with the signed version
  5. Verifies the signature was applied correctly

Prerequisites

  • osslsigncode installed (apt install osslsigncode or brew install osslsigncode)
  • EV code signing certificate exported as PFX file
  • GitHub secrets configured:
    • WINDOWS_SIGNING_CERT_PFX_BASE64 - base64-encoded PFX file
    • WINDOWS_SIGNING_CERT_PASSWORD - certificate password

Graceful Degradation

If the signing secrets are not configured:

  • The script prints a warning and exits successfully
  • GoReleaser continues without signing
  • The release proceeds with unsigned Windows binaries

This allows releases to work before a certificate is acquired.

Why This Script Exists

Windows code signing helps reduce antivirus false positives that affect Go binaries. Kaspersky and other AV software commonly flag unsigned Go executables as potentially malicious due to heuristic detection. See docs/ANTIVIRUS.md for details.


Future Scripts

Additional maintenance scripts may be added here as needed.