* feat(roles): issueops.Heartbeater role on the dolt, embedded and uow legs
Adds issueops.Heartbeater.Heartbeat(HeartbeatRequest{Actor, IssueID}) ->
HeartbeatResult{IssueID, Holder, HeartbeatAt, LeaseExpiresAt, Rearmed},
the lease refresh behind `bd heartbeat`, as its own role with its own
accessor.
- One body: internal/storage/issueops.ExecuteHeartbeatInTx validates
(*HeartbeatFieldError for a blank/over-long actor or empty id), refuses
a wisp, then runs RenewLeaseInTx, which is HeartbeatIssueInTx's body
reporting the renewal. HeartbeatIssueInTx is now a wrapper, so the legacy
HeartbeatIssue legs answer the same text and sentinels.
- The refresh is fenced on the holder AND on the live claim: after the
holder-fenced UPDATE, a read in the same transaction requires the issue
to be in_progress under the holder (the `update leases set` prefix and
the one IF() the Postgres dialect rewrites are unchanged; a Postgres port
should expect more serialization retries from that read). A refusal deletes the caller's orphaned lease
row, and the legs commit that refusal.
- A refresh writes only the lease row: no row_lock, version, updated_at,
event, journal entry, Dolt commit or hook. A re-arm (a live hand-doled
claim with no lease row the caller holds) is recorded as a mutation:
row_lock reminted, updated_at stamped, a lease_rearmed event, a journal
entry, a version, a Dolt commit (LeaseRearmVersionCommit) and on_update.
The remint makes a re-arm collide with a racing release or close instead
of leaving an orphaned lease on an open or closed issue (a pre-existing
race on every Dolt route). Rearmed is true only when a row was created:
a same-second refresh (0 changed rows on embedded Dolt) or a holder
spelled differently refreshes the existing row in place and keeps its
granted_at/granted_node. Reported times are rounded to whole seconds,
the stored values; the binds stay unrounded so a same-second refresh is
never a 0-row UPDATE on Dolt.
- Typed refusals: an absent id is ErrNotFound; a foreign holder is
*LeaseConflictError{Holder} wrapping ErrAlreadyClaimed; a wisp
(Ephemeral) or no live claim is *LeaseConflictError wrapping the new
ErrLeaseNotHeld, which also matches ErrNotClaimable.
- Legs: DoltStore and EmbeddedDoltStore through the issue-operation
funnel (DOLT_ADD of the named tables, then DOLT_COMMIT -m, after the SQL
commit); uow.NewHeartbeater through the new uow.RunTxScoped: a refresh or
a refusal commits with the plain SQL COMMIT (no working-set scan, no Dolt
commit, nothing another writer left uncommitted is swept), and a re-arm
commits through the new ScopedCommitter (DOLT_ADD of exactly its tables,
DOLT_COMMIT -m, never -Am). The legacy HeartbeatIssue doors commit a
re-arm the same way and keep a refusal's orphan delete, and the
proxied-server `bd heartbeat` route runs the role's body with the same
scoped commit.
- Census: storage.Storage gains Heartbeater(); HookFiringStore wraps it
(fires on_update for a re-arm only, RoleFiresHooks knows it), telemetry
wraps it, httpapi.timedProvider, uow.notifyingProvider (records a re-arm)
and externaldeps.uowProvider build it; the journal and version guards
list the heartbeat family as entry points; the httpclient unsupported
shell is regenerated and Heartbeater sits on its unsupported allowlist
until the wire lands.
Conformance: RunHeartbeater* (18 cases) on dolt, embedded dolt and uow:
refresh post-state with the stored times pinned to the result; no
row_lock/current_revision/updated_at/events/journal/issue_versions/
version-control/hook change; a guarded write with the pre-heartbeat
revision still lands; foreign holder refused against an aged lease;
refused after close/release/reclaim/status move; wisp refused; hand-doled
claim re-armed as a mutation; same-second heartbeat is not a re-arm;
orphaned leases on released, closed and reassigned issues refused and
deleted; a refresh and a refusal against a dirty working set make no
commit and leave the other writer's change uncommitted, and a re-arm
commits only its own tables; absent id; request rules; no request mutation. The http leg is
waived until the wire lands.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(httpapi): POST /v0/beads/issues/{id}:heartbeat behind issues.heartbeat
Publishes issueops.Heartbeater on bd serve as heartbeatIssue, the served
lease refresh behind `bd heartbeat`.
- Spec: the operation, HeartbeatIssueRequest {actor} and
HeartbeatIssueResponse {id, holder, heartbeat_at, lease_expires_at,
rearmed}; no `revision` (a refresh writes nothing to the row) and no
expected_version (the fence is the holder and the live claim).
issues.heartbeat joins the capability vocabulary and lease_not_held
(409) the Problem.code list; `reason: ephemeral` qualifies it for a
wisp. apigen and the wire-shape golden are regenerated; the golden diff
is additive only, so wire_revision stays 2.
- Server: route row on the custom-method dispatcher, handleHeartbeat in
heartbeat.go (claim's actor rules via bodyActor, unknown member and
query refusals), Config.Heartbeater as a required role (sourceRoles,
roleSourceNames, the per-request uow.HeartbeaterSource accessor).
failHeartbeat maps *LeaseConflictError typed fields: a foreign holder
is the claim's conflict shape (already_claimed + assignee +
issue_status); no live claim, a wisp included, is the new
lease_not_held (+ issue_status, + reason: ephemeral for a wisp), matched
before the generic arms so ErrLeaseNotHeld's ErrNotClaimable never
surfaces as not_claimable. Its detail names the one state the refusing
transaction found (closed, open and not claimed, an ephemeral wisp, ...).
- bd serve wires the role from its peeled store (serveRoleSource gains
Heartbeater); the serve stub stores, the corpus and e2e harnesses, the
served harness and the encode oracle bind it.
- Wire client vocabulary: OpHeartbeatIssue/issues.heartbeat and
MethodHeartbeat are CLASSIFIED (the capability set-equality gate),
ReasonEphemeral is pinned to the server's, and lease_not_held rebuilds
*LeaseConflictError{Status, Ephemeral} wrapping ErrLeaseNotHeld in the
native words (the code-table set-equality gate). No accessor yet.
Tests: heartbeat_test.go (request mapping and projection, edge refusals,
every role refusal with its code, members and state-specific detail, the
code row, the capability). The served-http Heartbeater leg
(served_heartbeater_test.go, 18 cases) runs the contract against a real
bd serve through a test-local dial over the store's own dispatch, and the
http-leg waivers are deleted.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(http): bd heartbeat over http through httpclient.Heartbeater
The client leg of the served lease refresh, and `bd heartbeat` on the
role for every route.
- httpclient.Store.Heartbeater (heartbeater.go) dials POST
/v0/beads/issues/{id}:heartbeat through Store.dispatch, so
issues.heartbeat is preflighted: an older server is refused BEFORE the
dial with *wire.CapabilityError naming the token (unwraps to
*storage.ErrUnsupported{Capability: "issues.heartbeat"}); it never falls
back to a re-claim or a metadata stamp.
- The role's request rules are restated before the dial
(validateHeartbeatRequest, pinned to
storeops.ValidateHeartbeatRequest): blank/over-long actor, empty id, each
a *HeartbeatFieldError.
- Problem mapping: already_claimed (which the shared table rebuilds as the
claim's ClaimConflictError) becomes *issueops.LeaseConflictError naming
the holder and wrapping ErrAlreadyClaimed, in the native words ("issue
already claimed by <holder>"); lease_not_held already arrives as
LeaseConflictError (Ephemeral from `reason`); not_found is ErrNotFound;
invalid_argument on `actor` is the role's field error.
- Idempotent: true, the one write this client marks replayable (a repeat
only moves the expiry later; a replay after a reclaim fails the fence).
- Legacy Store.HeartbeatIssue rides the role. Both leave the generated
shell and completeness_test's unsupported allowlist.
- `bd heartbeat` takes Heartbeater() from the resolved store on the direct
routes (dolt, embedded, http) and from the provider's HeartbeaterSource
on the proxied route (exact id resolution in a read unit of work, then
the role); runHeartbeatProxiedServer and heartbeat_proxied_server.go are
deleted. Success output and exit codes are unchanged; refusal text
changes per route (see CHANGELOG).
Tests: heartbeater_test.go (stub server: preflight refusal naming the
token with no POST, request/answer projection, each refusal's typed
mapping including the wisp's reason, the 503 replay, pre-dial validation,
validator parity, the legacy door). served_heartbeater_test.go runs the
18-case contract through env.subject.Heartbeater(), with
HookedHeartbeater wired off a HookFiringStore. TestE2E_HeartbeatOverHTTP:
real bd against an in-process serve: holder ok with revision and
updated_at unchanged and a guarded write with the pre-heartbeat revision
landing; a non-holder refused against an aged lease that stays untouched;
closed, wisp and unclaimed refused, each naming its own state; an older
server's handshake refused naming issues.heartbeat with no write. Corpus
shapes gc-heartbeat, -non-holder, -closed, -wisp pending until
reference_server.json advertises the token. CHANGELOG entry for the whole
track, including the orphaned-lease race fix.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/38875641041
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
* fix(heartbeat): roll back a no-issue refusal; restate refresh vs re-arm
Review fixes for #7600 (adopt-pr apply-fixes, iterations 1-2).
Legacy HeartbeatIssue doors commit a refusal only when it read the issue.
The dolt and embedded HeartbeatIssue callbacks committed every
LeaseRenewalRefusal, including Found=false (the issue read failed). That
refusal deletes nothing, but it can follow a fast-path UPDATE that
already extended an orphan lease row's expiry, so committing it kept the
extension. issueops.LeaseRenewalRefusalCommits now gates the commit on
Found, the rule the Heartbeater role's HeartbeatRefusalCommits applies.
New tests on both legs pin both arms: a no-issue orphan's expiry is
unchanged, and a released issue's orphan is deleted.
Refresh vs re-arm contract docs. Comments, Long help, the OpenAPI spec
and the generated apigen still said every heartbeat writes only the
lease row, makes no Dolt commit and fires no hook. A re-arm is recorded
as a mutation: it remints the revision, records an event and a version
entry, commits once (LeaseRearmVersionCommit) and fires on_update
through the wrapping hook decorator. Restated in domain, uow, issueops,
storage.go (why RoleFiresHooks answers true, so bd serve takes the
heartbeater off a peeled store), cmd/bd heartbeat and serve, httpapi,
httpclient, the spec (apigen regenerated), and the conformance and
contract-test docs.
Minor review findings:
- The 409 already_claimed and lease_not_held details now end "nothing
on the issue was written": the refusal may have deleted and committed
the caller's orphan lease row.
- docs/multi-agent/federation.md and the CHANGELOG entry describe the
spelling-mismatch heartbeat: the row is refreshed in place and keeps
granted_at and granted_node; only a re-arm stamps this node. The
embedded spelling test pins both columns, and the lease.go comments
say "refreshed in place" instead of "re-arms".
- The refusal's orphan delete judges the holder under actorMatches, as
the in-place refresh does (iteration 2). It matched holder = ? on the
actor as sent, so a dead claim's orphan held under another spelling
of the caller (gastown.mayor, heartbeat as gastown__mayor) survived
the refusal until TTL. The refused closure now reads the row's holder
and deletes that stored holder only when actorMatches(holder, actor).
Both TestHeartbeatIssueCommitsARefusalOnlyWhenItReadTheIssue twins
gain a cross-spelling arm (row deleted) and a control: another
identity's row (gastown.dog-3) survives.
Not changed: `bd heartbeat --json` still omits `rearmed`, since adding
it changes a CLI JSON contract (follow-up). The federation.md paragraph
post-dates the v1.2.2 docs pin, so documenting this main-only behaviour
there is a maintainer call under
engdocs/decisions/2026-07-17-docs-release-pin.md section 1.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(uow): run the scoped re-arm's Dolt commit after the SQL commit
The proxied-server heartbeat commits a re-arm with doltServerTx.CommitScoped,
which ran DOLT_ADD per table and DOLT_COMMIT('-m') INSIDE the open
transaction, before the SQL COMMIT. DOLT_ADD stages whole tables from the
session's transaction root, so that Dolt commit was built from the re-arm's
start snapshot: an issues row another session changed while the re-arm was
open (a write still only in the working set, as in batch mode or before that
writer's own Dolt commit) was staged at its old value, and HEAD disagreed
with the rows every reader sees. This is the LatentLabsSpace/NEXUS#92
ordering the DoltStore funnel (runIssueOperationTxWithMessage) already
avoids, and the review's major finding on this PR.
CommitScoped now commits the SQL transaction first, then stages exactly the
named tables (plus the versioned-history tables) and makes the Dolt commit on
the same pinned connection, the uow twin of doltAddAndCommitPostTx:
- a serialization failure of the SQL COMMIT is still returned, so
RunTxScoped replays the unit of work;
- the Dolt commit runs detached on its own budget and replays only conflicts
that leave nothing behind: serialization failures and Dolt's 1105
"@autocommit transaction rolled back" (new isDoltAutocommitRollbackError,
mirroring the DoltStore classifier of the same name);
- nothing staged, or "nothing to commit", is not an error: a concurrent
writer's Dolt commit already took the row. A "nothing to commit" past the
staged check is logged, as the funnel logs it, so the missing audit line
is explicable;
- any other failure is logged, never returned. The data is committed, and
returning it would make RunTxScoped replay a committed re-arm, which would
then report a refresh. The session is poisoned rather than pooled, since
its state is unknown. uow records no metrics, so that log line is the
drop's only signal; bd.db.post_tx_commit_dropped counts only the funnel's.
Statements go through schema.DrainCall, retiring the local drainCall fork
(review nit).
Tests: five sqlmock tests pin the order (SQL COMMIT before any DOLT_ADD),
the swallow-and-poison path, the 1105 retry, the returned serialization
failure and the empty-commit tolerance; each fails against the previous
CommitScoped. TestIsDoltAutocommitRollbackError carries the DoltStore
classifier's table over to the uow copy, plus the prefix boundary; dropping
the prefix branch, the prefix's period, the 1105 check or the type check
each fails a case. TestHeartbeaterRearmKeepsAConcurrentWrite is the
real-Dolt regression: a change to another issues row made while the
re-arm's transaction is open must be at HEAD after the re-arm's Dolt
commit. Its working-set-only writer fails on the old ordering (dolt
2.1.8); the writer that Dolt-commits its own change passes on both and is
kept as coverage.
Review follow-ups folded in (comments, a file move and a test; the
absorption log line above is the only behaviour change among them):
- the freshRowLock invariant and its row_lock_guard_test.go mirror name the
re-arm as the one deliberate remint without a lifecycle-field change,
load-bearing for the release/close collision;
- httpclient: "a write is never retried" (wire/client.go, stall_retry.go)
and "only a read's answer is yes" (dispatch.go) now allow for the
Idempotent heartbeat, and httpHeartbeater and issueops.HeartbeatResult
document that a replayed heartbeat can report Rearmed false after a
committed re-arm, with the guarded write's revision check as the backstop;
- the proxied route moves back to cmd/bd/heartbeat_proxied_server.go, the
file family cmd/bd/AGENTS.md prescribes;
- the ScopedCommitter and RunTxScoped docs no longer say every other
writer's uncommitted change stays out of the commit: a staged table is
committed whole, so such a change to a named table rides in the re-arm's
commit, as it does in the funnel, and only other tables keep theirs;
- commitStagedTables says its drop is log-only, the absorption is logged,
and the uow classifier gets its table test (see Tests).
Not folded in: the base-owned IssueUseCase.Heartbeat /
IssueSQLRepository.HeartbeatIssue door, dead in production since the
proxied route moved to the Heartbeater role. Its one caller is the
notifying hook-parity test (uow/notifying_test.go), and moving that case
onto the Heartbeater means re-deriving its RenewLease hook semantics, so it
is left for a follow-up.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
12 KiB
| title | description |
|---|---|
| Federation Setup Guide | Configure peer-to-peer sync of beads databases across workspaces with Dolt remotes, sovereignty tiers, and topologies |
Federation enables peer-to-peer synchronization of beads databases between multiple workspaces using Dolt remotes. Each workspace maintains its own database while sharing work items with configured peers.
Overview
Federation uses Dolt's distributed version control capabilities to sync issue data between independent teams or locations. Key benefits:
- Peer-to-peer: No central server required; each town is autonomous
- Database-native versioning: Built on Dolt's version control, not file exports
- Flexible infrastructure: Works with DoltHub, S3, GCS, local paths, or SSH
- Data sovereignty: Configurable tiers for compliance (GDPR, regional laws)
Prerequisites
- Dolt backend: Federation requires the Dolt storage backend (the only supported backend)
Configuration
Enable Federation-Compatible Sync
Edit .beads/config.yaml or ~/.config/bd/config.yaml:
federation:
remote: dolthub://myorg/beads # Primary remote (optional)
sovereignty: T2 # Data sovereignty tier
Or via environment variables:
export BD_FEDERATION_REMOTE="dolthub://myorg/beads"
export BD_FEDERATION_SOVEREIGNTY="T2"
Data Sovereignty Tiers
| Tier | Description | Use Case |
|---|---|---|
| T1 | No restrictions | Public data |
| T2 | Organization-level | Regional/company compliance |
| T3 | Pseudonymous | Identifiers removed |
| T4 | Anonymous | Maximum privacy |
Adding Federation Peers
Use bd federation add-peer to register remote peers:
bd federation add-peer <name> <endpoint>
Peer Name Rules
- Must start with a letter
- Alphanumeric, dash, and underscore only
- Maximum 64 characters
Supported Endpoint Formats
| Format | Example | Description |
|---|---|---|
| DoltHub | dolthub://org/repo |
DoltHub hosted repository |
| Google Cloud | gs://bucket/path |
Google Cloud Storage |
| Amazon S3 | s3://bucket/path |
Amazon S3 |
| Local | file:///path/to/backup |
Local filesystem |
| HTTPS | https://host/path |
HTTPS remote |
| SSH | ssh://host/path |
SSH remote |
| Git SSH | git@host:path |
Git SSH shorthand |
Examples
# Add a staging environment on DoltHub
bd federation add-peer staging dolthub://myorg/staging-beads
# Add a cloud backup
bd federation add-peer backup gs://mybucket/beads-backup
bd federation add-peer backup-s3 s3://mybucket/beads-backup
# Add a local backup
bd federation add-peer local file:///home/user/beads-backup
# Add a partner organization
bd federation add-peer partner-town dolthub://partner-org/beads
Credentials
Peers configured with --user (and optionally --password, otherwise
prompted interactively) store SQL credentials AES-256 encrypted, locally.
Stored credentials are used automatically during sync:
bd federation add-peer town-gamma 192.168.1.100:3306/beads --user sync-bot
JSON Output
For scripting, use the --json flag:
bd --json federation add-peer staging dolthub://myorg/staging-beads
# {"added":"staging","url":"dolthub://myorg/staging-beads","has_auth":false,"sovereignty":""}
Verify Configuration
List configured peers:
bd federation list-peers
Syncing with Peers
Use bd federation sync to pull from and push to peer towns, and
bd federation status to check sync state without transferring data.
# Sync with all peers
bd federation sync
# Sync with a specific peer
bd federation sync --peer town-beta
# Handle conflicts
bd federation sync --strategy theirs # or 'ours'
# Check status (ahead/behind, reachability, conflicts)
bd federation status
bd federation status --peer town-beta
Without --strategy, a sync that hits merge conflicts pauses and reports the
conflicting tables for manual resolution instead of auto-resolving.
Topologies
| Pattern | Description | Use Case |
|---|---|---|
| Hub-spoke | Central hub, satellites sync to hub | Team with central coordination |
| Mesh | All peers sync with each other | Decentralized collaboration |
| Hierarchical | Tree of hubs | Multi-team organizations |
Architecture Notes
How It Works
- Each workspace has its own Dolt database
add-peerregisters a Dolt remote (similar togit remote add)bd federation syncpushes and pulls commits between peers- Conflict resolution follows the configured strategy
When run against a Dolt SQL server, federation uses two ports: MySQL (3306) for multi-writer SQL access, and remotesapi (8080) for peer-to-peer push/pull:
┌─────────────────┐ ┌─────────────────┐
│ Workspace A │◄───────►│ Workspace B │
│ dolt sql-server│ sync │ dolt sql-server│
│ :3306 (sql) │ │ :3306 (sql) │
│ :8080 (remote) │ │ :8080 (remote) │
└─────────────────┘ └─────────────────┘
Multi-Repo Support
Issues track their SourceSystem to identify which federated system created
them. This enables proper attribution and trust chains across organizations.
Connectivity
Remote connectivity is validated on first push/pull operation, not when adding the peer. This allows configuring remotes before infrastructure is ready.
Leases are per-replica
A claim lease (bd ready --claim + bd heartbeat, reaped by bd reclaim) is
only meaningful on the replica that granted it. The leases table is
clone-local and never replicates; what crosses the bridge is the claim's
visibility — status/assignee on the issue row — and that is stale on
every other replica by up to one sync interval.
Two rules follow, and a federated deployment owes both:
- Grace window > sync interval, and lease TTL > sync interval. A TTL or
bd reclaim --older-thangrace shorter than the cadence at which replicas exchange state is meaningless across the bridge: the remote view is a full interval old by construction, so a reaper over there would be judging liveness from data older than the lease itself.bd reclaimdefaults its grace to 2× the lease TTL; raise the TTL (or the grace) above your sync interval, never shrink the interval to fit them. - Reclaim belongs to the granting replica. Each lease records the replica
that granted it, and
bd reclaimskips a lease granted elsewhere, naming it on stderr. Reap dead workers on the machine that hired them.
The guard is opt-in: it arms only where you name this replica.
export BEADS_NODE_ID=mini # per-machine; or `bd config set node_id mini`
Two rules about what to name, both load-bearing:
node_idnames the STORE, not the host. One value per beads database. Hosts that are clients of the same dolt sql-server (BEADS_DOLT_SERVER_HOST, a systemd/Docker server, Hosted Dolt, a VPS) are one replica no matter how many machines they are — give them all the same value, or leave it unset. Give them distinct ids and you rebuild the very fail-closed regression described below: a supervisor would match no worker's lease and reclaim 0 forever. Name a replica only where there is a real sync interval between it and the others.node_idis per-machine, so it must never be committed. The project.beads/config.yamlis a git-tracked file. Anode_idcommitted there propagates one machine's identity to every clone that pulls it, and then every comparison matches: the guard is fully armed and fully inert, andlaptopreapsmini's leases exactly as if they were local — the precise hazard this feature exists to close, now happening while you believe you are protected. That is worse than not setting it at all.bd config set node_idtherefore writes the user-global~/.config/bd/config.yaml, alongside the other per-machine state (sync-state.json,push-state.json,redirect). Use the env var or that command; never hand-addnode_idto.beads/config.yaml.
There is deliberately no hostname fallback. The hostname answers the wrong
question — it names the client process's machine, not the store — and
guessing gets it wrong in the topologies that most need automated reclaim:
with a shared or remote dolt sql-server (BEADS_DOLT_SERVER_HOST, Hosted
Dolt, a VPS) many hosts are clients of ONE store with no sync interval
between them, so a per-hostname identity would stop a supervisor reaping any
worker's lease at all; in a container the hostname is a per-run container ID;
on macOS the transient hostname follows the network. Each of those would
strand work on a deployment with no federation at all — a worse failure than
the one this guard prevents.
So an unset identity degrades to the old behavior (every lease treated as local) rather than failing closed: an upgrade, and any single-store deployment, can never strand a lease the reaper could previously recover. Leases granted before this feature landed likewise carry no replica and stay reclaimable until a heartbeat re-stamps them with a configured node.
bd reclaim --any-replica disarms the guard. It is for a replica that is
permanently gone (or a node that was renamed and now sees its own old leases
as foreign) — not a normal setting, since only the granting machine has a
first-hand view of whether the holder is alive.
A heartbeat proves the holder is alive; it does not move the lease. An ordinary heartbeat only backfills the granting replica when it is still empty — it does not overwrite a row that positively names one. So a lease normally keeps its granting replica for life, and states like these strand one where a local heartbeat is keeping it alive:
- a replica that was renamed (
mini→mini2) heartbeats its own leases, which now read foreign forever; - a foreign lease that arrived through the JSONL interchange, whose holder name also exists locally, gets heartbeated here but stays labelled with the remote node;
- an import that lands on an already-expired local lease row takes the whole row from the snapshot, granting replica included — so this node's own stale lease can come back labelled with a remote one.
(No heartbeat moves an existing lease, including one whose holder is a different spelling of the lease row's holder: that row is refreshed in place and keeps its granting replica. Only a re-arm, which grants a lease to a live claim that had none, stamps this node.)
Recover a stranded lease with bd reclaim --any-replica, once you have
confirmed the granting replica is not still reaping — that confirmation is the
whole point of the guard, so prefer the narrow forms:
bd reclaim --any-replica --id <id> for one issue, or bd unclaim --force <id>; the bare global form reverts every foreign stale lease, live peers
included. bd reclaim names what it declined on stderr — one summary line per
run, with bd -v expanding it to the first 20 leases individually.
Planned Features
The following operation has infrastructure support but is not yet exposed as a command:
bd federation push <peer>/bd federation pull <peer>- single-direction sync with one peer.bd federation syncalready covers the bidirectional case.
Troubleshooting
"requires direct database access"
Federation commands require the Dolt backend with direct database access. Ensure you have the Dolt backend configured for federation operations.
"peer already exists"
A peer with that name is already configured. Use a different name or check
existing peers with bd federation list-peers.
Invalid endpoint format
Ensure your endpoint matches one of the supported formats above. The scheme
must be one of: dolthub://, gs://, s3://, file://, https://, ssh://,
or git SSH format (git@host:path).
General health check
bd doctor --deep
Reference
- Configuration: See Configuration for all federation settings
- Source:
cmd/bd/federation.go - Storage interfaces:
internal/storage/versioned.go - Dolt implementation:
internal/storage/dolt/store.go