BuildReadyWorkWhere emitted clauses for --label (AND-set) and
--exclude-label but none at all for --label-any, so the OR-set filter was
silently dropped by `bd ready` and `bd ready --claim` on every backend
(with or without --parent), while `bd list`/`bd search` honored it.
On an atomic claim the silent drop is dangerous rather than merely wrong:
a worker fencing itself to its own lane
(`bd ready --claim --label-any lane-a --parent epic-1`) would claim
another lane's issue and believe it was fenced. Reproduced on current
main: the new conformance case claims a lane-b issue when lane-a was
requested.
Emit an OR-set membership clause (same shape as the list path in
sqlbuild/filter.go) that AND-combines with Labels, ExcludeLabels and the
parent filter. All claim paths (Dolt, embedded-Dolt, sqlkit SQL family)
funnel through ClaimReadyIssueInTx -> GetReadyWorkInTx ->
BuildReadyWorkWhere, so the single seam fixes every backend.
Tests: a SQL-shape unit test in sqlbuild, plus a behavioral conformance
case (ClaimReadyIssueLabelFilters) asserting the claim is fenced by the
OR-set alone, AND-combined with --label and --parent, and — the safety
property — that an exhausted lane claims NOTHING rather than falling back
to unfenced ready work.
Rebase of PR #4601 (feat/choose-your-own-backend) onto current origin/main,
squashed to a single commit because the original 73-commit history built and then
removed the uowStore derisk spike incrementally, which does not replay cleanly
over the 125 commits main has since advanced (notably #4679, the ready-work
counts fix, which merged to main and is now inherited rather than re-applied).
The full pre-rebase history remains on the original branch tip for reference.
Contents (multi-backend feature + the six review follow-ups):
- bd init --backend=postgres|mysql|sqlite provisioning behind the Dolt-parity
seam; the three SQL backends share *sqlkit.Store over pluggable dialects and
carry a generated typed-unsupported shell for the non-core surface.
- Passwords in init DSNs are redacted before persisting to metadata.json (driver
parser + re-parse verification), with a negative-test matrix proving no secret
survives (pgdialect + mysql).
- The uowStore derisk spike and its report artifacts are removed; the shell
generator now lives in internal/storage/unsupportedgen (neutral name).
- Compile-time guard `var _ storage.DoltStorage = (*Store)(nil)` on each backend
plus a regen-idempotence test keep the generated skip lists honest.
- sqlkit exposes CountReadyWork so every SQL backend satisfies
storage.ReadyWorkCounter and inherits the shared page-pushed ready-counts path.
- Commercial/product names scrubbed from the feature's own code for vendor
neutrality.
Rebase reconciliations: adopted main's metadata-load hardening in the store
factories alongside the backend dispatch; combined the config interface
(GetMetadata + ReconcileVersion) and the conformance registrations (ReadyCounts +
Claim/lease); took main's versions of the shared-infra concurrency (uow
doltserver_tx) and delete-cascade (domain/issue_delete) evolutions; implemented
the interface's new UnclaimIssue on sqlkit (delegating to issueops) and added it
to the skip lists.
Verified: go build (cgo + CGO_ENABLED=0) and go vet clean; unit tests
(sqlbuild/issueops/pgdialect/mysql/unsupportedgen) and the SQLite conformance
suite green (RunAll, incl. ReadyCounts equivalence and Claim/lease on the real
sqlkit backend).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`bd ready --json` had two O(N) hotspots on main. The counts mega-query's
reverse-blocker `rc` subquery keys on COALESCE(depends_on_issue_id, …), an
expression the pure-Go GMS analyzer cannot auto-index, so it re-scanned its
whole materialization once per candidate — O(candidates × blockers). And the
"Showing X of N" total re-ran that entire mega-query UNBOUNDED just to size N.
Changes:
- Generalize one counts builder: sqlbuild.SearchCountsSQL now takes an optional
`ids []string`. Empty ids = the predicate form (unchanged callers); non-empty
ids = the page-pushed form that constrains the driver and every count
subquery to the page, bounding rc by the page instead of the full graph.
Removes the drifted SearchCountsSQL/SearchCountsByIDsSQL duplication; the
projection order is defined once and DepTargetExpr is used throughout.
- issueops.runReadyCountsInTx resolves the ready-ID page with the cheap indexed
ID query, then hydrates counts constrained to those IDs, CHUNKED by
sqlbuild.QueryBatchSize so large pages stay within backend placeholder limits
(no quadratic fallback, no wasted ID pass). The page order is restored so the
result is byte-identical to the unbounded ORDER BY … LIMIT.
- issueops.CountReadyWorkInTx sizes N with indexed COUNT(*)s over the ready
predicate for the issues AND wisps families, minus the wisp-wins overlap —
identical to len(GetReadyWorkWithCounts(Limit=0)) even when wisps exist, with
no mega-query fallback.
- Add ReadyWorkCounter to internal/storage beside the other optional
capabilities; DoltStore and EmbeddedDoltStore implement CountReadyWork.
cmd/bd/ready.go zeroes the limit at the call site and unwraps to the
capability via storage.UnwrapStore, falling back to the unbounded query only
when a store predates it.
- Extract the shared scan/dedupe loop (scanCountsRowsInTx).
- Conformance tests: page counts byte-identical to unbounded, and
CountReadyWork == len(unbounded), with fixtures for wisps and a page that
spans the chunk boundary.
Co-authored-by: Test User <test@test.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(dolt): retry mutate+commit methods on serialization conflict (1213/1205)
ClaimIssue, ClaimReadyIssue, UpdateIssue and CloseIssue each hand-rolled
their own tx (BeginTx + DOLT_ADD/DOLT_COMMIT + Commit) outside
withRetryTx, so a concurrent writer that lost Dolt's optimistic
commit-time merge surfaced MySQL 1213/1205 as a hard failure instead of
retrying. Dolt has no real row locking — FOR UPDATE / SKIP LOCKED are
parse-only no-ops (https://www.dolthub.com/blog/2023-10-23-hold-my-beer/)
— so retry is the only safety net under N concurrent workers. These four
are the work-queue hot paths: `bd ready --claim`, claim, update, close.
Move each body into a withRetryTx closure (which owns BeginTx and the
final Commit and retries serialization failures with backoff), preserving
the single-tx CAS/mutation + DOLT_COMMIT atomicity exactly. On retry,
ClaimReadyIssue re-scans the ready front from a fresh snapshot and claims
the next available issue.
Tested: TestEmbeddedReadyClaimConcurrent, TestEmbeddedReadyConcurrent,
TestEmbeddedCloseConcurrent, TestEmbeddedUpdateConcurrent and the
non-concurrent embedded Update/Close/Ready tests pass; build + vet clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(dolt): add concurrent work-queue drain test (Gas Station claim invariant)
N workers concurrently dequeue one shared ready-front via ClaimReadyIssue
until empty; asserts every issue claimed exactly once, none stranded, no
double-claims, and no serialization error ever surfaces (withRetryTx absorbs
the 1213/1205 from the claim-CAS cell collision). This is the regression guard
for the multi-agent claim queue that #4491 makes safe.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(lease): claim-TTL + heartbeat + reclaim for dead-worker recovery
Gas Station v1.1 (wy-5r9j). A claim was previously permanent: a worker that
died mid-task stranded its issue in_progress forever. This adds a lease so a
claim can expire and be recovered.
Schema (migration 0054): add lease_expires_at, heartbeat_at (DATETIME) and
row_lock (BIGINT) to issues (and wisps, for uniform issueops routing), plus
idx_issues_lease for the reaper scan. assignee doubles as the lease owner.
Lifecycle (shared issueops, so both server-mode DoltStore and EmbeddedDoltStore
get it):
- Claim stamps lease_expires_at = now + TTL (default 5m, WithLeaseTTL to
override), heartbeat_at = now, and a fresh row_lock.
- bd heartbeat <id> — owner-only; pushes the lease forward and rewrites
row_lock. Fails (ErrNotClaimable/ErrAlreadyClaimed) once the lease is gone.
- bd reclaim --older-than <dur> — reverts in_progress issues whose lease
expired more than <dur> ago back to ready (clears assignee/started_at,
records a lease_reclaimed event). Default grace 2×TTL.
The row_lock trick: Dolt has no row locking and merges concurrent commits
cell-by-cell, so a heartbeat (writes heartbeat_at) racing a reclaim (writes
status) would silently cell-merge into a zombie — an open/unassigned issue that
still carries the worker's fresh heartbeat. Every mutating path (claim, close,
update, heartbeat, reclaim) now rewrites the shared row_lock cell, forcing the
loser to a 1213/1205 serialization conflict that withRetryTx replays. All lease
paths are withRetryTx/withConn wrapped.
Tests (real Dolt): TestRowLockForcesConflictOnDisjointCellWrites proves the bug
without row_lock (silent merge to zombie) and correctness with it (conflict);
TestConcurrentHeartbeatReclaimClose is the integration race (live workers
heartbeat+close vs a continuous reaper vs dead workers) asserting exactly-once,
no lost close, no zombie; plus claim/heartbeat/reclaim unit coverage on both
backends.
Note: lease columns are second-granular DATETIME (Dolt rounds), so sub-second
TTLs are meaningless — production TTL is minutes. Heartbeat writes a Dolt
commit, so cadence must stay ≪ claim cadence to avoid history bloat.
Fixes the test coupling in TestMigration0053PromotesRigWisps, which assumed the
rig-wisp repair was always LatestVersion().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci+docs: allowlist lease stress tests; scope the row_lock invariant comment
- Add TestConcurrentWorkQueueDrain and TestConcurrentHeartbeatReclaimClose
to the check-testing-short.sh allowlist; both are real-Dolt runtime/stress
skips and were failing ci-pr-policy.
- freshRowLock doc said "EVERY mutating path" rewrites row_lock, which
overstates it. Scope it to status/ownership/lease-mutating paths (the set
the reclaim/heartbeat races care about) and document the invariant + why
orthogonal-cell writers (is_blocked, compaction, deps, rename, reopen) are
intentionally exempt, so a future writer does not silently reintroduce the
zombie-merge bug.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(show): surface lease_expires_at/heartbeat_at in bd show (--json + text) (wy-9cdw)
Migration 0054 added lease_expires_at/heartbeat_at (and internal row_lock) to
the issues/wisps tables for the claim/heartbeat/reclaim leasing feature, but
bd show could not observe lease state — you had to drop to raw SQL.
- types.Issue: add LeaseExpiresAt/HeartbeatAt (*time.Time, omitempty), matching
the StartedAt nullable-timestamp convention. row_lock stays unexported.
- sqlbuild.IssueSelectColumns: append lease_expires_at, heartbeat_at to the
canonical hydration list (flows to every stack via the shared constant).
- issueops.ScanIssueFrom: scan + map the two nullable columns (before the
trailing extra dests, keeping positional parity across all scan sites).
- bd show text view: emit a muted 'Lease: expires <rel> (heartbeat <rel> ago)'
line only for in_progress issues holding an active lease; add formatTimeUntil
as the forward-looking mirror of formatTimeAgo.
- types_test: assert lease keys appear when set, are omitted when nil, and that
row_lock is never surfaced.
* fix(lease): settle v54 review blockers
Address #4537 review blockers before landing v54: keep heartbeat off wisps in both backends, keep direct status/assignee updates from leaving stale leases, guard the down migration, and emit reclaim JSON as snake_case.
Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie
* fix(lease): cover lease fields in tests and hooks
Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
* Fix rig bead durability (bds-vwz)
* fix: keep rig beads out of infra migration
* fix: repair durable rig wisps
Restore migration 0035 to its frozen upstream shape and add a forward repair migration for existing type=rig rows that were routed into wisps.
Keep type=rig durable while excluding it from ready work, and cover the repair path with CLI-backed schema tests plus Dolt-server tests where available.
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of matt wilkie
* fix: make rig wisp repair migration upgrade-safe
Guard migration 0053 so older workspaces without local wisp tables no-op instead of failing, while keeping the fresh-Dolt CLI migration bundle able to exercise rig wisp repair fixtures.
Regenerate CLI reference docs for the updated infra type help text.
Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie
---------
Co-authored-by: silvanshade <silvanshade@users.noreply.github.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
Move all pure SQL-text builders shared by issueops (classic) and
domain/db (proxied) into one package; both stacks now delegate, so
ready/filter/counts query semantics can no longer drift (Seam A).
Execution orchestration stays per-stack per the extract-don't-unify
decision. Canonical SQL text is classic's; the only proxied-side text
deltas are IN-list whitespace.
Adversarial re-review (bd-6dnrw.47) verified all 10 attack items with
no production divergence, and adds an import-boundary test pinning the
layering (domain must never import issueops/sqlbuild) plus a single-use
constraint note on LabelSearchPlan.MergeInto. Follow-ups filed:
bd-emrs2 (+1-overfetch convention), bd-kzeme (IN-helper consolidation).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>