beads/docs/getting-started/upgrading.md
Julian Knutsen 62d211937b
fix(upgrade-guard): parse the version witness as semver, and stop reading "unreadable" as "legacy" (#5650)
* fix(upgrade-guard): parse the version witness as semver, and stop reading "unreadable" as "legacy"

The cross-era upgrade guard (#4907) refused five production workspaces
outright:

    legacy Dolt server workspace detected; explicit migration is required
    before this bd version can open or modify the workspace.

Every one was current-era. The witness was a Go pseudo-version,
`v1.1.1-0.20260805093327-bf97b73749ac`, and `currentVersionWitness` split
it on "." expecting exactly three fields. It got four. Measured blast
radius before the operator hand-rewrote nine `.local_version` files to
`1.1.0`: every `bd` invocation against five cities failed, taking every
order that shells out to bd with it — one gate sweep alone logged 644
failures in 6 hours, which froze PR review gates entirely (no bead ever
left `needs-review`), stalled triage, and left demand-driven pools at
zero (one city ran 1 session against 8 declared agents). Rewriting the
witness files took the sweep to 0 failures/6h immediately.

Two defects, one in the parse and one in the policy.

## The parse

`.local_version` holds `main.Version` verbatim, and release tooling
injects that by ldflags: goreleaser passes `{{.Version}}`, the release
workflow passes `steps.version.outputs.version`, and other build paths
pass whatever they resolved. So the writer can emit a plain release, a
release candidate, a build carrying metadata, or a Go pseudo-version —
all valid semantic versions, none of them three bare numeric fields. bd
could not read what bd wrote. The writer is right; recording anything
but the true version would destroy the ordering `CompareVersions` needs.
The reader was counting dots.

`classifyVersionWitness` now parses with `golang.org/x/mod/semver`
(already an indirect dependency) and reports an era. `legacyVersionMinor`
shares the parse, so a pre-1.0 pseudo-version or release candidate is now
*correctly* classified as legacy where it used to fall through
unrecognized. Shapes strict semver rejects but a distributor could still
stamp on a legacy build — zero-padded or four-component versions — are
still read as legacy from their major component alone, so the pre-1.0
guard only tightens.

## The policy

Before this change an unparseable witness meant "legacy workspace, refuse
every command". That is the part that turned a trivial parse bug into a
fleet outage, and it is wrong on its own terms: every pre-1.0 bd wrote a
plain X.Y.Z through this same writer, so a string that fails a real semver
parse is affirmative evidence *against* a legacy workspace. The guard was
inferring "legacy" from data that positively excludes it, using a file the
codebase treats as advisory everywhere else — `.local_version` is
gitignored, clone-local, written best-effort, and `bd doctor` downgrades a
bad value to a warning whose suggested fix is "run any bd command", which
the guard made impossible.

So a witness that is *present but unreadable* is now its own era,
unknown: bd warns and opens the workspace. A *missing* witness is
unchanged and still refused — a workspace that never announced itself is
genuinely ambiguous — as is any witness that reads as pre-1.0.

The warning is self-healing rather than permanent noise: the guard runs
in PersistentPreRunE immediately before `trackBdVersion`, which rewrites
the witness whenever it differs from `Version`, so the admitted command
leaves a readable witness behind and the next command is silent.

## Coverage

`TestClassifyVersionWitness` pins the production string, plain and
v-prefixed releases, release candidates, build metadata, the pre-1.0 side
of each, empty, and garbage. `TestVersionWitnessRoundTrip` pins the
contract the guard depends on — everything bd can write, including
`Version` itself, bd reads back and recognizes as current, which also
holds the writer inside the witness reader's bounded size.
`TestLegacyUpgradeGuardStillRefusesPreOneWorkspaces` and the missing-
witness test prove the cross-era guard did not relax. Red before the
parse fix on 15 cases across 5 tests.

Refs GH#5603. That report reaches the same conclusion from a different
writer — Homebrew stamps `HEAD-<shortsha>` into `main.Version` for
`--HEAD` installs — and the unknown era admits those workspaces too
(`TestLegacyUpgradeGuardAdmitsBrewHeadStamp`). It does not silence their
warning: a HEAD stamp is rewritten identically every run, so it never
heals. Silencing it wants the shape recognizer in GH#5625 (anisoptera),
which this deliberately does not duplicate; the two changes compose.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(upgrade-guard): warn+open on a present-but-blank version witness

The legacy-upgrade guard's witness reader collapsed a present-but-blank
`.local_version` (0-byte or whitespace-only, e.g. from an interrupted or
disk-full best-effort write) into the same ("", false) result as a
genuinely missing witness. In server mode with a local Dolt root the
`if ok` reader-gate then routed present-blank onto the missing->refuse
path, hard-refusing a possibly-current workspace as "legacy Dolt server
workspace" with no self-heal -- the exact false-refusal class this PR
removes, left open for the blank-witness shape.

Make legacyUpgradeVersionWitness report presence independently of blank
contents: a present, bounded, regular witness returns ("", true) so the
guard classifies it witnessEraUnknown and warns-and-opens (matching the
present-but-unparseable case and the documented upgrading.md contract),
while a missing, non-regular, or oversized witness stays ("", false) and
still refuses, preserving the pre-1.0 guard safety invariant.

Adds guard-level tests for present-blank (0-byte / newline-only /
spaces+tabs) -> warn+open and missing -> refuse, plus a reader-contract
test pinning present-blank->present and missing/oversized->absent.

Addresses the maintainer review's one major finding (Codex).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Julian Knutsen <ci@beads.test>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-25 00:29:17 -07:00

15 KiB
Raw Blame History

title description
Upgrading Upgrade the bd binary, refresh git hooks, run schema migrations, and handle remote-backed and cross-era databases

How to upgrade bd and keep your projects in sync.

Checking for Updates

# Current version
bd version

# What's new in recent versions
bd info --whats-new
bd info --whats-new --json  # Machine-readable

Short Version

  1. With your current bd, sync remote-backed databases before installing the new binary: bd dolt push bd dolt pull
  2. Back up before migration: bd export --all -o .beads/backup/pre-migrate-$(date +%Y%m%d).jsonl
  3. Upgrade using the command that matches your install method.
  4. After upgrading: bd info --whats-new bd hooks install bd version
  5. If crossing a schema migration on a remote-backed database, only the designated migrator runs: bd migrate bd dolt push

Other clones should install the new binary and run bd bootstrap, not independently migrate. The full procedure is below.

Upgrading

Use the command that matches your install method.

Install method Platforms Command
Quick install script macOS, Linux, FreeBSD curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash
PowerShell installer Windows irm https://raw.githubusercontent.com/gastownhall/beads/main/install.ps1 | iex
Homebrew macOS, Linux brew upgrade beads
go install (server-mode only) macOS, Linux, FreeBSD, Windows CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest
go install (embedded-capable) macOS, Linux, Windows CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest
npm macOS, Linux, Windows npm update -g @beads/bd
bun macOS, Linux, Windows bun install -g --trust @beads/bd
From source (Unix shell) macOS, Linux, FreeBSD git pull && make build

Quick install script (macOS/Linux/FreeBSD)

curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash

PowerShell installer (Windows)

irm https://raw.githubusercontent.com/gastownhall/beads/main/install.ps1 | iex

Homebrew

brew upgrade beads

{/* Canonical Homebrew tap-migration snippet. The installation page links here; keep the two in sync. */} If you still have the old tap formula installed as bd, switch to the Homebrew core formula:

brew uninstall bd
brew untap gastownhall/beads 2>/dev/null || true
brew untap steveyegge/beads 2>/dev/null || true
brew install beads

go install

# Server-mode only
CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest

# Embedded-capable
CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest

From Source

cd beads
git pull
make build
sudo mv bd /usr/local/bin/

After Upgrading

Important: After upgrading, update your hooks:

# 1. Check what changed
bd info --whats-new

# 2. Update git hooks to match new version
bd hooks install

# 3. Check for any outdated hooks
bd info  # Shows warnings if hooks are outdated

# 4. If using Dolt backend, restart the server
bd dolt stop && bd dolt start

Why update hooks? Git hooks are versioned with bd. Outdated hooks may miss export refresh, legacy fallback, or safety fixes.

Database Migrations

After major upgrades, check for database migrations:

# Inspect migration plan (AI agents)
bd migrate --inspect --json

# Preview migration changes
bd migrate --dry-run

# Apply migrations
bd migrate

# Migrate and clean up old files
bd migrate --yes

Remote-backed databases and multiple clones

bd refuses to silently apply pending schema migrations to a database that has a Dolt remote configured. Migrating more than one clone of a shared remote independently forks the schema, after which bd dolt pull can no longer merge — the break is silent and, across a primary-key-reshaping migration, unrecoverable (#4259). The supported flow is: one machine migrates and publishes; every other clone re-clones the migrated database.

This applies to every upgrade that crosses a pending migration on a remote-backed database — the same procedure whether you are moving to a prerelease or to a stable release.

The gate is state-aware by default (#4516): before blocking, bd consults the remote's cached schema state and

  • auto-migrates when the remote is at the same schema version as this clone — no one has migrated yet, so this clone is a safe first-mover (concurrent first-movers converge to identical tables). It reminds you to bd dolt push afterwards.
  • stops and directs you to adopt (bd bootstrap) when the remote has already been migrated by another clone.
  • stops for a human decision when this clone and the remote applied different content for the same migration (a genuine fork), or when the remote's schema state cannot be read from the cached ref.

Set BD_SMART_GATE=0 to opt out and make the gate block unconditionally. The recipes below are the explicit path and work the same in either mode.

Important ordering: once the new binary is installed, a database with pending migrations is gated on every open — bd dolt push and bd dolt pull are refused too, not just bd migrate. So do all syncing with your current binary, before you install the new one.

Back up before you migrate. Schema migrations assume the database matches the shape the previous migrations left behind; real databases sometimes drift (interrupted writes, tooling bugs, very old bootstraps). A JSONL export is cheap, issue-complete, and importable by any bd version:

bd export --all -o .beads/backup/pre-migrate-$(date +%Y%m%d).jsonl

bd export captures issues, not Dolt history or config — for a full snapshot also copy the .beads directory (or dolt backup in server mode) while no bd command is running.

Single clone (including a solo user with a remote):

bd dolt push                              # 1. CURRENT binary: publish all local work
bd export --all -o .beads/backup/pre-migrate.jsonl   # 2. backup (see above)
# 3. install the new binary (see Upgrading above)
bd migrate                                # 4. migrate as the designated migrator
bd dolt push                              # 5. publish the migrated schema
bd version                                # 6. confirm the new version is active

If bd's remote-migrate gate blocks the run, it prints the available options — migrating here as the designated migrator, adopting the remote's already-migrated database, or recovering a fork — and asks for an explicit operator decision. Follow the guidance it prints.

For scripted or CI upgrades where nobody reads the prompt, BD_ALLOW_REMOTE_MIGRATE=1 bd migrate (any boolean true value works) declares this clone the designated migrator and bypasses the gate entirely — including its already-forked checks — so wire it into exactly one clone's upgrade job, never all of them.

Multiple clones sharing one remote:

# 1. With your CURRENT (old) binary, on EVERY clone: publish all work and get in
#    sync, then stop editing until the upgrade is done.
bd dolt push
bd dolt pull

# 2. Designated migrator ONLY: back up, install the new binary, then migrate
#    and publish.
bd export --all -o .beads/backup/pre-migrate.jsonl
bd migrate
bd dolt push

# 3. Every OTHER clone: install the new binary, then ADOPT the migrated database.
#    (bd dolt pull is refused here — the clone still has pending migrations — so
#    re-clone instead. Safe because step 1 already pushed all work.)
bd bootstrap

bd bootstrap replaces the local database, so any work not pushed in step 1 is lost — that is why step 1 publishes everything first. If a clone was instead migrated independently and bd dolt pull later fails with cannot merge because table dependencies has different primary keys in its common ancestor, the schema has already forked — follow the recovery playbook: the pk-fork-refused runbook.

`bd doctor` includes a migration-content-skew check that flags a forked schema against the cached remote ref — a useful post-upgrade verification. It runs in both server and embedded modes.

Cross-era Upgrades

If you're upgrading from a much older version of bd, inspect the storage layout and metadata before running the current binary. A .beads/dolt/ directory alone does not identify a legacy workspace: supported current server mode uses that directory too. Current bd evaluates explicit server metadata, the presence of that local root, and the bounded .local_version witness together. An explicit server selection is not overridden by a stale .beads/embeddeddolt/ repository.

Storage layout Upgrade path
Current embedded metadata with .beads/embeddeddolt/ and no explicit server selection Direct current-era upgrade
Explicit server metadata plus .local_version from v0.55.4 through v0.62.0, whether or not .beads/dolt/ exists Explicit legacy Dolt export/import
Explicit server metadata plus .beads/dolt/ and a witness whose major version is 1 or newer Normal current server-mode upgrade
Explicit server metadata plus .beads/dolt/ and a missing or pre-v1 witness Explicit legacy Dolt export/import
Explicit server metadata plus .beads/dolt/ and a witness that is present but unreadable Normal current server-mode upgrade, with a warning
Explicit server metadata without .beads/dolt/, and a missing, malformed, or non-historical witness Normal current server-mode compatibility path
.beads/dolt/ with missing metadata or persisted dolt_mode blank/embedded Explicit legacy Dolt export/import, except for the configured shared-server compatibility path described below
One .beads/*.db file, such as beads.db or vc.db Sealed SQLite bridge

The witness is whatever bd held in its own version string when it last touched the workspace, so it may be a plain release, a release candidate, a build carrying metadata, or a Go pseudo-version; all of those are read as the version they name. A witness that is present but unreadable is not treated as a legacy marker — no pre-v1 bd could have written one — so bd warns and continues rather than refusing every command. A missing witness stays ambiguous and is still refused.

Current bd refuses recognized historical SQLite and legacy Dolt layouts before opening storage or rewriting metadata. This is intentional: preserve the source and complete the matching explicit migration below.

PostgreSQL and MySQL are removed backends, not supported cross-era upgrade paths. Current bd refuses metadata that selects either backend, and the sealed bridge below accepts SQLite sources only.

.beads/embeddeddolt/: direct upgrade

Upgrade the binary and run:

bd migrate

If the project was initialized before bd init automatically wired git origin as the Dolt remote, verify the remote after upgrading:

bd dolt remote list

When the list is empty, fix it on the machine whose local database is authoritative:

bd export -o .beads/issues.pre-remote.jsonl   # optional issue audit export
bd dolt remote add origin git+ssh://git@github.com/org/repo.git
bd dolt push

Commit the resulting .beads/config.yaml change so other clones can run bd bootstrap or bd dolt pull.

Historical Dolt server mode: explicit migration

Do not run current bd init --force when .beads/dolt/ has missing metadata or persisted dolt_mode is blank/embedded. Those old embedded layouts are never current embedded storage. The same explicit path applies when metadata selects backend: dolt, dolt_mode: server and .local_version records v0.55.4 through v0.62.0.

First take a native snapshot while every writer is stopped. Restore that snapshot to a disposable workspace, export it with the verified historical binary, then import the export into a fresh current project. Keep the original and snapshot unchanged until the cutover has been reviewed. The SQLite sealed-copy helper below does not start or manage a Dolt SQL server.

Explicit server metadata with a v0.55.4–v0.62.0 witness is always refused, including when there is no local .beads/dolt/ root. When that root does exist, the guard admits explicit server mode only with a syntactically valid witness whose major version is 1 or newer; a missing, malformed, or pre-v1 witness fails closed. Without the local root, a missing, malformed, or non-historical witness is admitted only as a compatibility layout.

The configured shared-server compatibility path applies only when persisted metadata is missing or leaves dolt_mode blank/embedded; it does not override an explicit server selection with a local root. Compatibility admission cannot prove that a workspace is modern. If you know it was created by v0.55.4 through v0.62.0, use this explicit bridge even when its witness was lost or damaged. Otherwise, follow the normal bd migrate --dry-run and bd migrate flow for an admitted server workspace.

One .beads/*.db file: sealed SQLite bridge

The old binary stored data in SQLite. The new binary uses Dolt.

Recommended: use the sealed-copy bridge (requires jq):

Stop every process that can write the old workspace before starting. Run this from a source checkout at the exact commit you intend to run; installed binaries do not include repository scripts. Record that commit with git rev-parse HEAD before executing the script. Download the old bd asset only from the official gastownhall/beads release and verify the asset with its published SHA-256:

sha256sum -c checksums.txt --ignore-missing

On macOS or BSD, use shasum -a 256 on the downloaded archive and compare the result with that archive's entry in checksums.txt.

scripts/migrate-legacy-to-current.sh \
  --source /absolute/path/to/old-project \
  --destination /absolute/path/to/old-project-cutover \
  --source-version v0.50.3 \
  --old-bd /absolute/path/to/verified-old-bd \
  --new-bd /absolute/path/to/current-bd \
  --prefix beads

For a source older than v0.49.6, also supply an authenticated v0.49.6 binary:

scripts/migrate-legacy-to-current.sh \
  --source /absolute/path/to/old-project \
  --destination /absolute/path/to/old-project-cutover \
  --source-version v0.17.0 \
  --old-bd /absolute/path/to/verified-old-bd \
  --canonicalizer-bd /absolute/path/to/verified-v0.49.6-bd \
  --new-bd /absolute/path/to/current-bd \
  --prefix beads

Verify the historical binary against its official release checksum before running the bridge. The script verifies each binary's reported version, rejects Dolt and ambiguous layouts, retains a sealed source copy, and compares the candidate export with the canonical historical export. Activate the cutover manually only after reviewing those retained artifacts.

Troubleshooting Upgrades

Hooks out of date

bd hooks install

Database schema changed

bd migrate --dry-run
bd migrate

Recovery after upgrade

If you need to restore from a backup:

bd init
bd backup restore [path] --force

Or pull from a Dolt remote:

bd dolt pull