Commit graph

11,146 commits

Author SHA1 Message Date
Chris Sells
705e7661ab docs(upgrading): key the shared-server read failure to schema v53, not release 1.2.0
#7301 scoped the pre-migration read failure on a shared server to schemas
that "predate 1.2.0". That misses 1.2.2, the recovery re-release of the
1.1 line: its migrations stop at 0053 with no lease migrations, so a 1.2.2
server sits at v53 like 1.1.x. Upgrading such a server's clients to 1.3.1
without `bd migrate schema` makes `bd list`, `bd show` and `bd ready` fail
with `table not found: leases`.

Key the caveat to schema v53 or older, naming 1.1.x and 1.2.2, which
matches the section's own "A 1.2.2 or 1.1.x database sits at schema v53".

Refs #6929, #7284, #7301
2026-10-06 11:22:18 -07:00
Chris Sells
f05b2b0a6a
docs: scope the shared-server read failure to pre-1.2.0 schemas; add eight community tools (#7301)
* docs(upgrading): scope the shared-server read failure to pre-1.2.0 schemas

#7284 said most reads fail between upgrading a shared server's clients
and running `bd migrate schema`. That only holds for a schema older than
1.2.0: the leases table ships with main migration 0055 and ignored 0012
and 0016, all in v1.2.0. From 1.2.x, reads keep working and only writes
are refused (probed against a 1.2.1 server with 1.3.1 and main builds).

Restore the original wording and add the pre-1.2.0 caveat, naming only
the observed `bd list` failure from #6929.

Refs #6929

* docs: add beads-viewer (Paseo plugin) to community tools

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit ad28cf00ba)

* Add LazyBeads (lb) to community tools

- List lesliesrussell/lazybeads under Terminal UIs
- Note that it is unrelated to the earlier lazybeads under Historical

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 1aa2b645e2)

* docs(community-tools): update the beads-web entry

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit aa177110b3)

* docs(community-tools): mention the beads-web tree view

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QebAVaD36wUemAsdBx8wwJ
(cherry picked from commit e0eb482a11)

* Updated community tools documentation with Beads Tracker

Applied changes from BB to 1 file

Files modified:
- docs/community-tools.md

Updated community tools documentation

Applied changes from BB to 1 file

Files modified:
- docs/community-tools.md

Updated community tools documentation

Applied changes from BB to 1 file

Files modified:
- docs/community-tools.md

(cherry picked from commit 75d0f8de0c)

* docs: add b9s to community tools (TUI and web UI)

(cherry picked from commit 8bb6ab6913)

* docs: add btui to community tools (btui-bb2)

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Andreas Dellrud
(cherry picked from commit 81da372c63)

* docs: mention released btui Beads 1.3 support (btui-0fm)

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Andreas Dellrud
(cherry picked from commit 8f27445bee)

* docs: add beadside to community tools

(cherry picked from commit 2c2cb1fd5c)

* docs: add Beads for Obsidian to community tools

(cherry picked from commit cb82d067d8)

---------

Co-authored-by: cuongntr <cuongntr@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Leslie Russell <leslierussell@gmail.com>
Co-authored-by: weselow <598746@gmail.com>
Co-authored-by: Beyond Better <git@beyondbetter.app>
Co-authored-by: vanderheijden86 <vanderheijden86@gmail.com>
Co-authored-by: Andreas Dellrud <andreas.dellrud@gmail.com>
Co-authored-by: michal.psencik <misa.psencik@gmail.com>
Co-authored-by: Rome-1 <romethorstenson@gmail.com>
2026-10-06 10:56:39 -07:00
Chris Sells
b1bee123ba
docs(upgrading): reads fail too between upgrading clients and migrating a shared server (#7284)
The shared-server upgrade steps said an upgraded client keeps reading the
old schema and only has writes refused. In practice most reads fail as
well: bd list, bd show and bd ready stop with "table not found: leases"
because 1.3 reads expect tables the old schema lacks. Say so, and tell
operators to run the migration as soon as the clients are upgraded
rather than treating the window as merely degraded.

Refs #6929, #7038
2026-10-05 21:58:43 -07:00
Chris Sells
384001a22e
docs(upgrading): adopt a migrated remote without relying on bd bootstrap (#7280)
The upgrade docs told every non-migrating clone to adopt the migrated
database with `bd bootstrap`, claiming it replaces the local database.
On a clone that already has one, bootstrap prints "Nothing to do." and
exits 0, leaving the clone on the old schema (#6687).

Document the flow that works: every clone backs up with the old binary,
other clones `bd dolt pull` on the old binary after the migrator pushes
and then upgrade; a clone that already upgraded moves its local database
aside, re-clones with `bd bootstrap`, and re-imports the backup. Align
the gate description, shared-server note, and init-safety summary.

Also add a post-migration step to run `bd recompute-blocked` once per
database, since migration 0059 can over-set is_blocked and silently hide
ready work (#7037).

Refs #6687, #7037
2026-10-05 21:46:05 -07:00
Julian Knutsen
e72cd8b315
ci: merge-queue readiness for the required gates; cache test results (fresh nightly) (#7264)
* ci: make the required gates merge-queue ready

The default-branch ruleset requires "CI Gate / Required" and "PR Risk
Gate / Required". Prepare both for a GitHub merge queue (ALLGREEN,
squash), so enabling the ruleset's merge_queue rule is the only step left.

- pr.yml, pr-risk.yml: merge_group pinned to types [checks_requested].
- D2: bazel-coverage covers every merge_group run (same repository, CI
  secrets, actor github-merge-queue[bot], bazel.yml mode remote), so the
  retired legacy tiers stay retired in the queue and ci-gate requires
  their Bazel lanes to have run remotely and passed; a missing secret or
  the farm switch off is red, as on a same-repo PR.
- Diff bases on merge_group: the package gates diff
  merge_group.base_sha..head_sha (not HEAD^, which misses all but the
  last commit of a rebase-method entry), the .beads/issues.jsonl guard
  now runs on merge_group against base_sha (ci-gate no longer excuses
  its skip), and check-doc-flags passes BD_DOCS_DIFF_BASE so CLI-docs
  drift already on main cannot eject every queued PR.
- Test result reuse: on a merge_group run and on attempt 2+ of a PR run,
  the six always-execute Bazel lanes append --config=reuse-test-results
  (--cache_test_results=auto), so tests whose action key is unchanged
  reuse the farm's recorded PASS and only what the merge changed runs.
  Failures are never reused, eviction retries stay 0, and each PR's first
  attempt, push to main and nightly still execute every test.
- scripts/ci_merge_queue_test.go simulates merge_group through the shared
  evalGHExpr evaluator (now with true/false/null literals, hyphenated
  identifiers and case-insensitive ==) and the workflows' own scripts:
  triggers and gate names, concurrency groups, every pull_request field
  read, the coverage and rbe decisions, result reuse, the package-gate,
  .beads and doc-drift diff bases.
- engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md: new "Merge Queue" section (how
  to queue, settings, what runs, result reuse rationale, ejection and
  flakes, rollback).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci: cache Bazel test results except nightly; queue trust and sizing

Review follow-ups to the merge-queue readiness change.

- Test results (S1): --nocache_test_results also stops Bazel writing
  results, so a PR's first run never fed the cache and the merge-queue
  reuse config could only hit on later reuse runs. Drop it from
  test:embedded, test:doltserver-proxied, test:doltserver-integration and
  test:sole-run (which keeps --experimental_remote_cache_eviction_retries=0),
  and drop test:reuse-test-results and BAZEL_REUSE_RESULTS. Results are now
  cached (read and write) on PRs, merge groups and pushes to main.
  nightly.yml passes the new fresh-test-results input, so every `bazel
  test` appends --config=fresh (--nocache_test_results) and the suite
  re-executes nightly. The docker lane keeps --nocache_test_results.
  Measured locally with a disk cache: a default run writes and the next
  reads ("(cached) PASSED"); a fresh run neither reads nor writes.
- Security headers (S2): bazel.yml now states that a merge_group run
  executes the queued commit's workflows, so a queued fork or Dependabot
  PR's CI edits run with every CI secret, the pre-warm app key included
  (merging-level trust), and records the maintainer rule: review fork
  changes to .github/, .bazelrc, tools/bazel, MODULE.bazel, BUILD files
  and CI scripts before queueing. Same note in the topology doc.
- Queue sizing (S3): recommend max_entries_to_build 3 and
  check_response_timeout_minutes 60 (max_entries_to_merge 5, ALLGREEN,
  SQUASH unchanged).
- Nits: the .beads guard fails closed on a missing diff base, and its
  test runs under GitHub's default `bash -e`; evalGHExpr follows GitHub's
  loose equality (true == 'true' is false, false == '' is true, missing
  context keys are null) with a typed-context variant; the
  detect-package-gates comment notes the SQUASH/HEAD^ equivalence and
  path-filter scope; the doc covers re-run reuse, flake re-sampling via
  dispatch, stale queue runs and how to confirm reuse on the farm.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 13:54:15 -07:00
Julian Knutsen
4493eb8d23
ci(pr): run same-repo macOS preflight/doc-freshness legs on Blacksmith macOS (#7231)
pr-preflight-platforms and check-doc-freshness-platforms now resolve their
same-repo-macos marker like the Linux/Windows legs: blacksmith-6vcpu-macos-26
for same-repo PRs and merge_group, GitHub-hosted macos-latest for forks,
Dependabot and every other event. The label is pinned to macos-26 because
GitHub's macos-latest is macos-26-arm64 today, so both paths run the same
OS/arch; 6 vCPU is already 2x macos-latest's 3 vCPU.

Blacksmith cannot see GitHub-saved caches, so main.yml gains
blacksmith-macos-go-build-cache: a push-to-main-only saver on the same label
(same job guard as the other Blacksmith savers) that builds ./cmd/bd and runs
the shared warm-non-race-cache.sh, saving the module cache and a non-race
GOCACHE keyed by go.sum + UTC day. The PR legs stay restore-only. main.yml's
GitHub-hosted macOS full-suite leg is unchanged and keeps seeding the fork
path.

Policy tests: the macOS marker arm is evaluated for same-repo/merge_group/
fork/deleted-head/dependabot/push/schedule and must match the flat macOS
const; the saver joins the PR-guard and per-day-key sweeps and the cache
ownership inventory; TestBlacksmithMacOSSaverMatchesPRLegs pins the
label/path/key/warm-up contract between saver and PR legs.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 13:19:49 -07:00
Julian Knutsen
685cf848bd
fix(gate): a budget spent during the intent probe still gets the final attempt (#7237)
A farm run of TestAcquireAllExhaustedBudgetFinalAttemptIgnoresIntent
(#7225) failed with "held by a bd maintenance operation queued for
exclusive access" for a gate that was free: a real product race that broke
the guarantee that queued maintenance can delay a shared acquirer but never
fail it.

Acquire decided whether an attempt was final from the clock at the top of
the iteration, then probed the intent lock (open + flock + reading the
sidecar). On a starved machine the probe can take longer than the budget
that was left, so the iteration deferred as "not final", found the budget
spent, and returned ErrBusy without ever trying the gate. Now a deferral
can never end the acquisition: if the budget ran out while deferring, the
loop makes one more attempt at the gate that ignores the queue, and only a
gate that is really held can fail it.

Tests:
- TestSharedFinalAttemptSurvivesStallDuringIntentProbe stalls the
  acquirer past its deadline right after the deferral decision (a
  test-only hook), so the race is reproduced deterministically for both
  Acquire and AcquireAll; it fails with the farm's exact message without
  the fix.
- TestAcquireAllExhaustedBudgetFinalAttemptIgnoresIntent no longer
  releases gate a from a timer racing the budget (a second load-dependent
  failure mode seen in the repro); a is released synchronously after the
  call.

Under load (Bazel server pinned to 2 CPUs with taskset, 32 concurrent test
jobs, race detector): the original test failed 71/200 runs (-test.count=20);
with the fix both tests pass 200/200 (-test.count=10), and the whole
workspacegate_test passes 200/200.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 12:57:09 -07:00
Julian Knutsen
dbaa4631ca
feat(roles): BatchGetter role and POST issues:batchGet (#7248)
* feat(roles): BatchGetter role and issues:batchGet route (S7)

Adds BatchGetter.GetMany(ids) -> {Issues, Missing}, a one-snapshot,
order-preserving, deduped, 1000-id-capped batch read, wired through all
three library legs (dolt, embedded dolt, uow) behind the existing
hook/telemetry decorator chain, plus the POST /v0/beads/issues:batchGet
HTTP route (spec, apigen, wire-shape golden, issues.batchGet capability
token). Conformance contract (RunBatchGetter*, 12 cases) passes on all
three legs against real backends. No HTTP client leg yet (waits on the
generator / PR #7247); left as a documented seam on BatchGetter's doc
comment.

CLI GetIssuesByIDs call sites are left unmigrated this slice and marked
with TODO(batchgetter) comments: every call site reads an unbounded id
list today, and GetMany's 1000-id cap has no chunking/fallback in any
caller yet, so swapping them in now would add a new, unhandled failure
mode to already-shipped CLI paths (export, import, molecule port,
ready-explain, wisp sweep).

Verified via three live-mutate guards against
internal/storage/issueops/batchget.go (disabling the cap check, dropping
the Missing append, and replacing ordered iteration with map iteration):
each turns the embedded-dolt contract red, confirming the cap, Missing
reporting, and order-preservation assertions actually catch a break.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(roles): close S7 review gaps — revision on wire, BUILD sync, handler tests

Addresses the S7 Opus review's FIX-FIRST items on BatchGetter/issues:batchGet:

HIGH
- batchGetResponse now projects apigen.BatchGetIssue (types.BatchGetIssue +
  NewBatchGetIssue), carrying `revision` per issue the way IssueDetails does;
  spec, apigen and the wire-shape golden regenerated (additive, no
  wire_revision bump).
- make bazel-sync: the 12 files missing from BUILD.bazel srcs are back in
  sync across 9 BUILD.bazel files.
- New internal/httpapi/batch_get_test.go: success incl. revision, missing
  ids, empty arrays not null, unknown member refused, missing/null ids
  refused, query parameter refused, 1001 ids refused with the full request
  reaching the role, 1000 ids accepted.

MED
- RunBatchGetterAcceptsExactlyTheCap pins the cap boundary (> vs >=).
- RunBatchGetterHydratesLabels pins Labels survive projection.
- RunBatchGetterReadsOneSnapshot renamed to
  RunBatchGetterSharesOneReadStructurally with a doc that no longer
  overclaims atomicity; a real interleaving test needs a hook on the frozen
  roleFixtureKit, out of scope here.
- Doc comments across ExecuteGetMany, GetManyResult.Issues and the
  BatchGetter interface now say hydration is labels only.

LOW
- failBatchGetErr's doc no longer claims blank entries are "already caught".
- PR #7247 references kept.

Re-ran build/vet/gofmt/lint, make api-check, make bazel-sync(-check), the
full internal/httpapi suite, and all three conformance legs (dolt via the
pinned local server, embeddeddolt, uow) plus targeted cmd/bd serve tests —
all green. Live-mutate-guard reverified: cap >=, dropping Missing from the
response, and clearing Labels each now fail the relevant test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(roles): sync client vocabulary and repo gates with issues:batchGet

The S7 route landed on the server without the sibling tables that hold
the rest of the tree set-equal with the server's surface, so five
tests and two manifest gates went red on the PR head:

- httpclient/wire: mirror batchGetIssues into the client vocabulary,
  CLASSIFIED, not wired, the way the wire wave and #7250's
  issues.count.scope landed: OpBatchGetIssues, its issues.batchGet
  token on opCapability, its identity row, and the first-slice skew
  dimension 35 -> 36 (TestCapabilityTableMatchesTheServerRouteTable).
- httpclient/wire: BatchGetIssue is the first golden.json schema with
  a `revision` member that needs no legacy-integer tolerance — the
  route postdates #6053 and types.BatchGetIssue declares Revision a
  string, and a pre-#6053 server never advertises issues.batchGet, so
  Preflight refuses before any body is decoded. Exempt it with that
  reason in a new revisionSchemaExempt table rather than adding an
  issues[] rewrite no legacy server can reach; the gate refuses an
  entry that is both covered and exempt, a stale exemption, and an
  empty reason.
- httpclient/encode: give the oracle's role set a BatchGetter stub —
  httpapi.Listen requires the whole set, so all three oracle tests
  failed to bind.
- storage/uow: exempt IssueUseCase.GetMany from the notifying-parity
  partition as a read, the reason hook_batch_getter.go gives.
- shard manifest: place TestBatchGetterContract in the 15-shard
  embedded storage block (generator --write, incremental).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(roles): pin the BatchGetter pure tier and the spec ids caps

Review fixes, iteration 1. The review approved with no blocker or major
finding and three minors, and this commit takes all three. It changes
tests and one comment only, and every new assertion was mutation-probed
with go test -overlay.

- storage/issueops: add batchget_test.go, the database-free tier that
  FinishGetMany's doc already claimed ("pinned without a database") but
  that no test supplied. It pins:
  - the cap is checked before the blank scan (an over-cap request with
    a blank at position 0 answers *TooManyIDsError);
  - the bound is inclusive;
  - dedupe keeps the first mention, does not trim, and never writes
    through the caller's slice;
  - Issues and Missing both follow request order;
  - both slices are never nil;
  - ExecuteGetMany refuses one id repeated past the cap before any
    query (a sqlmock with no expectations).
  Ten mutants were each killed by exactly their intended test, with the
  rest of the package green: last-mention, sort, trim, in-place dedupe,
  blank-before-cap, >=, nil Missing, nil Issues, dedupe-before-validate
  and validate-after-query.
- httpapi: TestSpecDefaultsMatchSharedConstants now checks ids.maxItems
  against the enforced bound on the two request bodies that carry an
  `ids` list: batchGetIssues against issueops.MaxGetManyIDs, and
  deleteIssues against maxDeleteIDs. Each body is reached through its
  operation's own requestBody $ref. Setting either constant to 999 turns
  only this test red. The spec's other list caps (batch
  create/close/apply items, dependency edge lists) stay ungated, as
  before.
- cmd/bd/serve.go: the batchGetter field comment said its accessor
  "does NOT recurse through the hook decorator". HookFiringStore.
  BatchGetter does recurse to the inner getter; it just doesn't wrap
  the result. The comment now says that.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-05 12:09:18 -07:00
Julian Knutsen
0b1e9657ce
fix(cmd/bd): classify delete's pre-flight not-found as precondition_failed (#7256)
bd delete's single-id path resolves the row with
resolveAndGetIssueForMutation BEFORE calling deleter.Delete() and the
per-id lock fence #7244 added. On a real Dolt sql-server, a same-token
--if-revision racer that loses that fence can find the row already
gone at that pre-flight check, which returned an unclassified exit 1
("issue %s not found") instead of routing through
classifyIfRevisionFailure/reportIfRevisionFailure like every other
--if-revision loser. This is why TestSharedServerDeleteIfRevisionSingleWinner/same_token
is flaky.

classifyIfRevisionFailure already had a storage.ErrNotFound branch
anticipating exactly this gap (mc-zndi7.73); delete.go's pre-flight
check just needed to call it when --if-revision is set, reporting
precondition_failed / ExitGuardMismatch (13) instead of a bare
"not found".

Checked the proxied delete path (delete_proxied_server.go): it has no
separate pre-flight check and already routes deleter.Delete()'s error
through reportIfRevisionFailure, so no gap there.

Adds:
- TestClassifyIfRevisionFailureNotFoundSentinel: unit test pinning
  classifyIfRevisionFailure/reportIfRevisionFailure's handling of the
  bare storage.ErrNotFound sentinel.
- TestIfRevisionDeletePreflightGoneIsPreconditionFailed: embedded CLI
  test that deterministically reproduces the same pre-flight code path
  a losing racer hits (delete the row out from under a stale
  --if-revision token), without depending on actual concurrency.

Fixes mc-zndi7.81.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-05 11:20:24 -07:00
Julian Knutsen
c1b35aed3c
fix(storage): serialize guarded deletes against each other on a Dolt sql-server (#7244)
* fix(dolt): serialize guarded deletes against each other (mc-zndi7.73)

On a Dolt sql-server, two concurrent `bd delete --if-revision R --force`
requests for the same row were both landing: Dolt's commit-time merge
treats two concurrent transactions that each delete the same row as
identical diffs and lands both with no conflict, so every same-token
racer exited 0 instead of exactly one. A delete racing an update/close
already produced a real write conflict (the racing write's row_lock
bump is a genuine edit Dolt's merge must reconcile against the delete),
but deleter.go used withWriteTx (no retry), so the loser surfaced a raw
Dolt Error 1213 instead of being retried and reclassified as a version
mismatch.

Fix: a per-id delete fence (GET_LOCK/RELEASE_LOCK, Dolt/MySQL built-in
session primitives) serializes guarded deletes on a real Dolt
sql-server, so a same-token loser's existence probe now runs only after
the winner's delete has already committed and finds the row gone,
failing through the ordinary NotFoundError path. The fence pins ONE
*sql.Conn for GET_LOCK, the retried write, and RELEASE_LOCK, because
setupTestStore's tests (and any store configured with a pool of one)
run with MaxOpenConns=1, and a two-connection design would deadlock
under it. withRetryTx/commitWriteTx are generalized (withRetryTxOn/
commitWriteTxOn) to run their shared retry-and-commit body against
either the pool or this one pinned connection. deleter.go also now
uses withRetryTx instead of the non-retrying withWriteTx, so a delete
racing an update/close is retried like every other guarded write.
classifyIfRevisionFailure (cmd/bd/if_revision.go) gains a
storage.ErrNotFound case, mapping the row-vanished-from-under-the-guard
outcome to precondition_failed/ExitGuardMismatch on both the proxied
routes (which have no pre-write existence check) and this same-token
delete race.

No schema or migration change: GET_LOCK/RELEASE_LOCK are built-in
Dolt/MySQL session primitives, the same idiom
schema.MigrateUpWithLock already uses for migrations, reimplemented
standalone in internal/storage/dolt/delete_fence.go to avoid borrowing
the schema package's migration-flavored timeout and error sentinels.

Adds real concurrency race tests in the style of #7233's
shared_server_if_revision_concurrency_test.go: same-token deletes,
delete vs close, delete vs update, on both the shared-server and
proxied routes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(uow): serialize guarded deletes against each other on the proxied route (mc-zndi7.73)

internal/storage/dolt's new per-id delete fence only covers the
shared-server/direct-CLI route. The proxied-server (and bd serve)
route goes through internal/storage/uow instead, a completely
separate deleter implementation that shares no code with the dolt
package, so it had the exact same bug: two concurrent guarded deletes
of the same row both landed, because Dolt's commit-time merge treats
two identical "row absent" diffs as non-conflicting.

Fix: a twin fence (internal/storage/uow/delete_fence.go) applied
around the write whenever req.ExpectedVersion is set. Unlike the
dolt-package fence, this one uses two independent connections (one for
GET_LOCK/RELEASE_LOCK, one drawn normally by the retried write) rather
than pinning a single connection: no cmd/bd call site ever constrains
this package's pool to one connection, so a dedicated lock connection
cannot starve the write. The fence unwraps d.provider via the existing
uow.UnwrapProvider helper first, because the real proxied chain always
wraps the bare *doltSQLProvider in a notifyingProvider and then an
externaldeps uowProvider before handing it to NewDeleter.

No code changes were needed in cmd/bd/if_revision.go: a losing
racer's existence probe already returns the same NotFoundError/
storage.ErrNotFound classifyIfRevisionFailure already maps to
precondition_failed for the shared-server race.

Also fixes internal/storage/dolt/version_scope_completeness_test.go's
TestTxMintingWrappersScopeVersionedHistory, broken by the prior
mc-zndi7.73 commit's withRetryTxOn/commitWriteTxOn split: the BeginTx
call (and the versioned-history scope call beside it) moved from
commitWriteTx's own body down into commitWriteTxOn, so the guard's
hardcoded txMintingWrappers list needed to name the new function that
actually mints the transaction.

No schema or migration change: GET_LOCK/RELEASE_LOCK are the same
built-in Dolt/MySQL session primitives the dolt-package fence uses.

Agent-Signature: claude-code-claude-sonnet-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore(bazel): sync BUILD files for mc-zndi7.73's new delete-fence sources

make bazel-sync, picking up internal/storage/dolt/delete_fence.go and
the two cmd/bd race-test files added by the prior commit (missed
there) plus this commit's new internal/storage/uow/delete_fence.go.

Agent-Signature: claude-code-claude-sonnet-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-05 09:39:44 -07:00
Julian Knutsen
c9a36b248c
fix(dbproxy): wait out a live starter's spawn marker instead of a fixed 5s (#7249)
Shutdown advances the stop epoch, then waits while a live starter's
proxy.spawn marker covers the release-lock-before-exec window. That wait
shared shutdownConfirmDeadline (5s) with the kill/confirm phases, but the
starter legitimately holds the marker longer: it waits up to openDeadline
for the child to take proxy.lock (which clears the marker) and then up to
shutdownConfirmDeadline each to kill the child and clear the marker. On a
loaded host (CI -race runners) a slow fork/exec pushed the start past 5s
and `bd dolt stop` reported "proxy.Shutdown partial: backend stopped;
proxy left running: timeout (5s) waiting for spawn marker" for a start that
was still progressing and about to abort on the advanced epoch. This is the
TestShutdownWaitsForStartDelayedBeforeExec flake.

The marker is only treated as active while its owner's verified identity
is alive (a dead owner's marker is removed on sight), so bound the wait by
the starter's own budgets (shutdownSpawnWaitDeadline = openDeadline +
2*shutdownConfirmDeadline) and give the post-start stop phases a fresh
shutdownConfirmDeadline once the marker resolves.

Tests: TestShutdownWaitsForLiveStarterPastConfirmDeadline holds the start
before exec past shutdownConfirmDeadline; it fails deterministically with
the CI signature before this change. Both spawn/stop tests share a helper
and use generous hang-detector timeouts.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 08:53:11 -07:00
Julian Knutsen
5b44f40160
test(proxied): reap the outage bridge before rebinding its TCP port (#7238)
TestProxiedServerOutageReconnectAcceptanceMatrix/external-tcp flaked in the
Bazel proxied-server tier with "Condition never satisfied: bd reconnect after
upstream restart" (5 CI failures, every one external-tcp, never the unix
case).

The restore step SIGKILLed the old socat front and immediately started a
replacement on the same TCP port. Kill only queues the signal: on a loaded
host the old socat can still hold the listener when the replacement calls
bind, so the replacement exits with EADDRINUSE. Its stderr went nowhere and
the "remained alive" check passed on the unreaped pid, so the endpoint
refused every connection and each bd attempt correctly failed with the
proxy's upstream-unreachable report until the 10s window ran out. The unix
case removes and rebinds a fresh socket path, so it cannot hit this.

The bridge is now a small type that tracks socat's exit: stop() kills and
waits for exit (the listener is released by then), and startOutageBridge
waits for socat's own "listening on" notice (-d -d), retrying a socat that
exits before listening. The reconnect loop reports the last bd error and the
bridge log on failure instead of a bare "condition never satisfied".

Product behaviour is unchanged: the proxy reports a refused upstream and
reconnects as soon as the endpoint is back.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 08:42:50 -07:00
Julian Knutsen
edafcbce1b
fix(httpclient): map Counter scope params and issues.count.scope capability (#7250)
#7210 (OSS HTTP client wire/encode layers) and #7199 (Counter scope —
ParentID, NoParent, ExcludeTypes, ExcludeStatus behind issues.count.scope)
merged back to back without a rebase, leaving main red on the client's
parity gates: the count encoder table left the four new CountRequest
members unclassified, the published parent/no_parent/exclude_type/
exclude_status parameters had no driving field, and the client's
capability vocabulary lacked the server's new behavior token.

- encode: countTable maps the four scope members as params, spelled as
  the server's countFilters decodes them; countFilters emits them.
- encode/skew: --no-parent / --exclude-status flag mapping for the
  unknown_parameter refusal.
- wire: CapCountScope mirrors httpapi.CapIssuesCountScope and joins
  behaviorCapabilities; the vocabulary test pins it by name and order.
- roundtrip: scope members round-trip through the server decoder
  (ParentID and NoParent in separate cases, as the role refuses both).

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 08:31:08 -07:00
Julian Knutsen
6d7fecaa3e
feat(httpclient): OSS HTTP client wire/encode layers (contributed from bd-enterprise under MIT) (#7210)
* feat(httpapi): wire-shape signal — wire_revision, floor header, golden digest, param tokens

S0 of the HTTP role-parity program: give a client a single place to learn
whether this build's wire shape has moved under it, and give CI a gate that
catches an undocumented move.

- GET /v0/beads/context reports wire_revision (bumps only on a non-additive
  response-shape change) and min_client_wire_revision. An optional
  Bd-Wire-Revision request header lets a client declare the revision it was
  built for; checkWireRevision refuses a declared revision below the floor
  with 400 invalid_argument/reason=wire_revision_unsupported and a
  min_wire_revision field, on every route but /healthz (wireRevisionExempt) —
  including the identity handshake, where catching the mismatch is cheapest.
  Absent header is today's behavior; malformed value is invalid_value, never
  wire_revision_unsupported. The floor lives in Server.minClientWireRevision
  (defaulting to the MinClientWireRevision constant) so a test can raise it.
- internal/httpapi/wireshape: a golden digest over every response's every
  content-type schema (including application/problem+json) across the whole
  spec, keyed by schema+member. TestWireShapeDigest fails closed when a
  member's JSON name or type changes without wire_revision bumping to match;
  regenerate with `go run ./internal/httpapi/wireshape/cmd/gendigest`.
- GET /v0/beads/issues' `sort` parameter now carries the issues.listSort
  capability token, the same convention `capabilities` entries use.
  TestNewParameterOnExistingOperationHasABehaviorToken requires every new
  parameter on an existing operation to carry one, checked against a
  write-once grandfather baseline (testdata/pretoken_parameters.json) of
  parameters that predate the rule; TestUntokenedParameterRuleFires proves
  the checker itself against a synthetic fixture.
- Docs: engdocs/design/bd-serve-v0.md gets a "The wire-shape signal" section;
  CHANGELOG.md gets an Added entry.

Live-mutated and reverted three RED demonstrations during development:
retyping api_version in the spec failed TestWireShapeDigest; short-circuiting
checkWireRevision's floor comparison failed
TestWireRevisionBelowFloorRefusesEveryEnforcedRoute; adding an untokened
query parameter to listIssues failed
TestNewParameterOnExistingOperationHasABehaviorToken. All three now pass
against the unmutated tree.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): rename issues.listSort to issues.list.sort, harden token rule

Opus review of 12c5144ca (HIGH-1, MEDIUM-5):

- Rename the capability token issues.listSort -> issues.list.sort to match
  bd-enterprise's client (httpstore/wire/handshake.go:121, DESIGN.txt S4e),
  and widen capabilityToken to accept an optional third segment.
- untokenedParameters() now requires the extracted token to be one
  Capabilities() actually serves, not just backtick-shaped text. This
  catches real false positives that existed in the spec already
  (workapi.DefaultReadyLimit, status.custom) which only shape-matched the
  old regex; they're added to the pretoken baseline as genuinely
  pre-existing untokened parameters.
- Extend the rule to request body members (new
  testdata/pretoken_request_body_members.json baseline) and exempt
  brand-new operations via a frozen testdata/pretoken_operations.json
  snapshot.
- Add TestPretokenBaselinesNeverGrow pinning all three baseline sizes so
  a future change can't hand-add an exemption instead of a token.
- Update spec prose for the widened rule scope.

go test ./internal/httpapi/... and the reviewer's pmut.py all pass as
expected: plain/bogus modes go RED, reuse (real-but-unrelated token) stays
green, matching the review's literal ask ("a token must be one advertised
by Capabilities()").

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(wireshape): close four digest-coverage gaps, add gendigest write guard

Opus review of 12c5144ca (HIGH-2, MEDIUM-4). The digest only tracked
Type/Format/Enum/Required on direct properties, missing:

- nullable: true (never read at all)
- array item shape (Issue.labels items string->integer was invisible;
  only object-shaped items got recursed into)
- additionalProperties value shape (IssueCount.groups)
- request bodies entirely (Compute() only walked responses, so
  DeleteIssuesRequest.expected_version string->integer, the other half
  of #6053, was invisible)

Also added defensive oneOf flattening alongside the existing allOf merge
(dormant today -- grep confirms this spec uses oneOf nowhere -- but no
longer silently invisible if that changes).

Extracted the golden-vs-computed comparison into wireshape.Compare(),
shared by TestWireShapeDigest and the new wireshape.SafeToWrite(), which
gendigest now calls before writing: it refuses to overwrite golden.json
with changed/removed entries unless the candidate's wire_revision is
strictly greater than the existing golden's. Pure additions always write.
TestSafeToWrite exercises both guard directions.

golden.json is regenerated. This single regen is a one-time exception to
the new guard, done by writing the digest directly rather than through
gendigest: confirmed via a throwaway diff (not checked in) that every one
of the 57 "changed" entries differs only in the NEW fields the wider walk
now populates (ItemType/ItemFormat/Nullable/AdditionalProps*) -- every
pre-existing Type/Format/Required/Enum value is byte-identical. Nothing on
the wire changed; the measurement got more precise. Bumping
CurrentWireRevision for this would have written a false "the protocol
changed here" entry into the revision history table. Going forward, any
REAL change the guard now reports uses the normal bump-then-regenerate
path, same as before.

Verified against the reviewer's mut.py: nested_type, nullable,
nullable_issue, array_items, addlprops, request_body and format all go RED
on TestWireShapeDigest, each reverted after confirming.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): make bd_version wire_revision inference advisory

Opus review of 12c5144ca (HIGH-3). The spec told a client inferring
wire_revision from bd_version to use the release cutoff >= 1.3.0, and said
nothing about what happens when that inference is wrong -- which it
demonstrably is for real builds already in the field:

- main builds up to commit b3ef65c85 report bd_version 1.2.2 (below any
  1.3.0 cutoff) while already sending the string-typed revision/
  expected_version shape (#6053).
- 1.3.0-rc.N pre-release builds sort BELOW the release cutoff 1.3.0 under
  plain string comparison despite already carrying the string shape.
- The bd-enterprise compatibility line at d3ab32773462 reports bd_version
  1.1.0 while also sending strings.

Moved the cutoff to >= 1.3.0-rc.1 (compared as semver-with-prerelease,
which at least captures the rc builds), documented both named exceptions
inline, and made the whole inference explicitly ADVISORY: a client must
accept revision/expected_version as either a string or a number whenever
wire_revision is absent (or probe directly) rather than let the inferred
value choose a decoder, and must never refuse solely because the inferred
revision is 0. Mirrored into engdocs/design/bd-serve-v0.md.

This is spec/doc prose only -- there is no server-side inference code to
change, since this server always sends wire_revision directly; the
inference is guidance for clients reading a pre-signal server's response
(this server, past builds, or an unrelated implementation).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): LOW-7/8/9/10 review follow-ups for wire-revision signal

LOW-7: WireRevisionUnsupported now also sets Problem.wire_revision (this
server's CurrentWireRevision) and Problem.bd_version on the
wire_revision_unsupported refusal, alongside the existing
min_wire_revision, so a client logging the refusal has this server's
full self-description without a second /v0/beads/context call.

LOW-8: gazelle + make bazel-sync add the BUILD.bazel entries the prior
commits were missing: wire_revision.go/wire_revision_test.go in
internal/httpapi's srcs, new BUILD.bazel files for wireshape and
wireshape/cmd/gendigest, and a testdata glob on httpapi's go_test so
the pretoken_*.json baselines are visible under `bazel test`. Verified
with `bazel build //internal/httpapi/...` and
`bazel test //internal/httpapi/... --test_env=BEADS_TEST_SKIP=dolt`
(both green) and `python3 tools/bazel/go_srcs.py --check` (clean).

LOW-9: narrowed CHANGELOG.md and engdocs/design/bd-serve-v0.md to
describe the digest's actual final coverage (format/enum/required/
nullable, array items, additionalProperties, request bodies, $ref/
allOf/oneOf recursion, the SafeToWrite revision-bump guard) and the
token rule's actual final shape (three pinned baselines, request-body
members, Capabilities()-membership validation, new-operation
exemption) instead of the original, narrower claims.

LOW-10: Bd-Wire-Revision now parses via a strict decimal-only
parseStrictNonNegativeInt (no leading '+', no leading zero except a
bare "0") instead of strconv.Atoi alone, so two different header
bytes can no longer name the same revision. New malformed-header test
cases: "+1", "01", "007".

Regenerated apigen/types.gen.go (make api-gen) for the new Problem
fields and wireshape/testdata/golden.json (go run
.../wireshape/cmd/gendigest) for the two new additive Problem members;
the latter wrote cleanly through SafeToWrite itself, since both are
pure additions.

go test ./... and bazel test //internal/httpapi/... both green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): re-review LOW follow-ups — retirement test, gendigest -init, docs

1. TestRetiredRevisionsStayBelowTheFloor pins the PRESENCE guarantee the
   wire_revision property doc relies on: CurrentWireRevision >= 2 (so 0
   and 1 stay permanently retired) and MinClientWireRevision <=
   CurrentWireRevision (so this build's own floor never refuses every
   request against itself).

2. gendigest now refuses to create a golden.json that does not exist yet
   unless run with -init. Previously a missing file (moved testdata,
   wrong path, accidental delete) was indistinguishable from a real first
   run and silently wrote a fresh golden with no revision history checked
   against anything. writeGolden() is extracted out of main() so this is
   covered directly: TestWriteGoldenRequiresInitWhenMissing,
   TestWriteGoldenInitCreatesWhenMissing,
   TestWriteGoldenIgnoresInitWhenGoldenExists (the existing-golden path is
   still governed only by SafeToWrite, -init or not).

3. Narrowed the two remaining stale claims that still described the
   digest's original, narrower coverage: the wire_revision property's
   prose in openapi.v0.yaml (regenerated into apigen/types.gen.go via
   make api-gen) and wireshape/doc.go now both list nullable, array item
   shape, additionalProperties value shape, request bodies, and
   $ref/allOf/oneOf recursion, plus the new -init guard.

go test ./internal/httpapi/... and make api-check both green;
bazel test //internal/httpapi/... (httpapi, wireshape, and the new
gendigest test target) green; python3 tools/bazel/go_srcs.py --check
clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(wireshape): cover operation parameters in the wire-shape digest

The digest previously walked only response and request-body schemas, so a
type, enum, required, or removal change on one of the spec's 138 operation
parameters (123 query, 14 path, 1 header) passed the gate silently — the
open item from PR review on #7182.

Each parameter is now recorded as its own Entry, keyed by
"param:"+operationId / location+":"+name (operation-scoped, not
schema-scoped, since two operations' same-named parameter are independent
wire contracts), capturing type, array item shape, enum, required, style,
explode and default. Adding a parameter stays additive; removing,
retyping, narrowing its enum, or changing required/style/explode/default
now trips SafeToWrite exactly like a response or request-body change.

Verified against the live spec: retyping listIssues' `limit` query
parameter from integer to string now fails TestWireShapeDigest (confirmed
red, then reverted — not part of this commit). golden.json regenerated via
gendigest; the new parameter entries are a pure addition, no wire_revision
bump needed. Added TestParameterMutationsAreCaught (remove/retype/narrow-
enum against real golden entries) and updated wireshape's doc.go, the
`wire_revision` spec prose, and the CHANGELOG to describe the new
coverage.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(wireshape): review fixes — stale-golden and lowered-revision arms, effective param serialization, digest boundary

Maintainer fixup from the adopt-pr review loop (iterations 1 and 2), on
top of e17ba924f.

- goldenDrift (A, I): TestWireShapeDigest's comparison moves into a
  goldenDrift helper. An addition alongside a moved wire_revision no
  longer passes with a stale golden: it asks for a regenerate, as a
  changed or removed entry after a bump already did. Every message now
  names response/request-body members AND parameters, and TestGoldenDrift
  pins each arm. This takes only the test half of the stranded fixup
  (adopt-pr/bd-47kdge/stranded-fixup-528cd6095); its doc half, which says
  parameters are outside the digest, is false since e17ba924f and is not
  replayed (J).
- Lowered wire_revision (D): a CurrentWireRevision LOWER than the
  golden's is a wrong constant, not a stale golden, so goldenDrift says to
  restore it instead of regenerating, and SafeToWrite (so gendigest)
  refuses any write at a lower revision, whatever the entries.
- Effective style/explode (B): walkParam records style and explode with
  OpenAPI 3.0's defaults filled in (form for query/cookie, simple for
  path/header; explode defaults to style == form). Adding `explode: false`
  to a query array that relied on form's implicit explode is now a
  changed entry, while spelling out a default changes nothing.
  testdata/golden.json is regenerated with -init: 111 parameter entries
  change only in style/explode (96 to form/true, 15 to simple), 828
  entries before and after, wire_revision stays 2. The golden is new in
  this PR and unreleased, so this needs no bump.
- TestWalkParam (H): a package-internal table test
  (wireshape_internal_test.go) pins every field walkParam extracts,
  through a computeFrom(document, revision) seam that Compute now wraps:
  per-location defaults, $ref'd parameters and items, format, nullable and
  default, content parameters, and the nil returns.
  TestEffectiveSerializationIsCompared pins B end to end through Compare
  and SafeToWrite. BUILD.bazel embeds the library in the go_test, which is
  gazelle's own output.
- Required inputs and the digest boundary (C, E, F; docs only): all five
  prose surfaces (openapi.v0.yaml's wire_revision description and the
  regenerated apigen/types.gen.go, wireshape/doc.go, CHANGELOG.md,
  engdocs/design/bd-serve-v0.md) now say that a new REQUIRED request
  input is recorded as an ordinary addition and needs its own review,
  that value constraints and member-level composition are outside the
  digest, and that object and content parameters are recorded only as
  their container. bd-serve-v0.md also gains the parameter coverage it
  was missing.
- G: a brand-new operation is permanently exempt from the
  capability-token rule; its own review decides its capability story.
- Iteration 2 (nit): walkParam's comment now says that the location
  default it records as a `content` parameter's style and explode is a
  placeholder, since OpenAPI 3.0 applies style and explode only to a
  parameter described by `schema`. All 138 operation parameters in the
  spec use `schema`, so the golden does not change.

Validated: go test ./internal/httpapi/... (gms_pure_go), make api-check,
go vet for linux/windows/darwin, gofmt, git diff --check, gazelle on the
package (no drift), make bazel-sync-check, and bazel test
//internal/httpapi/wireshape/... in the local processwrapper sandbox
(2 of 2 pass). Overlay mutation probes: dropping the added-after-bump
arm, the lowered-revision arm, SafeToWrite's lowered check, or the
effective-default fill each turns the new tests red.
Iteration 2 changes only that comment; go test
./internal/httpapi/wireshape/... (gms_pure_go), go vet, gofmt and git
diff --check pass again.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(httpclient): lift the HTTP client wire/encode layers from bd-enterprise (MIT contribution)

Contributed by gascity from bd-enterprise (github.com/gascity/bd-enterprise,
internal/enterprise/httpstore @ 49d1df2f6) to OSS beads under the MIT
license, with the owner's explicit approval. Every lifted file carries a
provenance header naming its source path.

Lifted into internal/httpclient:
- wire/    the HTTP transport, lazy handshake, project-identity stamping,
           problem-code decoding, retry behaviour and the capability vocabulary
- encode/  request/response encoders for the role operations
- credential*.go, ca*.go, target*.go   the bearer credential ladder,
           host-scoped CA trust and the http_target.json sidecar

Not lifted (stays enterprise-only): the hosted gateway (httpgw/gwauth),
box-login, storage profiles and gasworks.

Deferred to later slices (kept aside, not in this tree): the store transport
seam (WireClient/WireDialer/process-wide dialer, which goes to S3 with
dial.go), the enterprise-only BatchCloseRequest.claim_next encoder
(ClaimNextBody; the OSS server has no claim_next on batch close), and the
tests that need the store/dial layer.

This commit is the mechanical lift: import paths rewritten, package renamed,
two compile gaps removed. The production code builds. Known follow-up in the
next commit: reconcile the client's vocabularies and encoder tables with the
OSS spec (e.g. the OSS problem code notes_overwrite_refused, policy/sort
differences), restore the test helpers the lifted tests need, and wire in the
S0 wire_revision handling and S1 OpenOptions credentials.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(httpclient): compile the lifted client and reconcile with the OSS wire

The lift in ab11cb1f8 carried relative test paths and vocabulary tables
from bd-enterprise's internal/enterprise/httpstore (4 directories below
repo root) into internal/httpclient (3 below), and several tables
described enterprise-only server behavior OSS does not serve. Reconcile:

- Add testTarget test helper (written fresh) so internal/httpclient
  compiles and its tests pass.
- Fix four relative paths off by one "../" from the package relocation
  (skew_test.go, ledger_render_test.go, bijection_test.go x2).
- skew_test.go: importsHTTPClient now matches internal/httpclient instead
  of the retired internal/enterprise/httpstore path; the cmd/bd sweep no
  longer hard-fails when it legitimately finds zero matching files (cmd/bd
  does not import this client yet - that's S3's wiring).
- encode/table.go, encode/ledger_fields.go: removed a Reserved "reverse"
  entry and a refused ExcludeIDs field claimed for listIssues/ReadyRequest;
  OSS's listIssues never accepts "reverse" (confirmed against
  internal/httpapi/reads.go and routes.go's CapIssuesListSort, which gates
  `sort` only) and issueops.ReadyRequest carries no ExcludeIDs member in
  OSS. Each removal is documented in place.
- encode/table.go, encode/encode.go, encode/roundtrip_test.go: countIssues
  publishes has_metadata_key and include_ephemeral that CountRequest
  carries and the table/encoder/round-trip corpus all silently dropped;
  mapped and wired through.
- encode/ledger.go: ~70 ledger rows cited enterprise store-conformance and
  cmd/bd subprocess-parity tests (dual-run reader, claim races, flag-mode
  refusals) that don't exist in this package and belong to S3's store/dial
  seam and cmd/bd wiring. Re-pinned to a documented TODO(S3) sentinel
  (pinnedByS3Conformance) rather than deleted, per the ledger's own
  well-formedness discipline.
- wire/handshake.go, wire/vocabulary_test.go: added CapBatchApplyLarge
  (issues.batchApplyLarge) alongside the server's behaviorCapabilities.
- wire/problem.go: mapped the notes_overwrite_refused problem code to
  issueops.ErrNotesOverwrite.

internal/httpclient/... is fully green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(httpclient): wire-revision handshake gate and batch cap from capability

S0 integration (task #2): Client.Handshake now reads ContextResponse's
wire_revision/min_client_wire_revision and refuses with a typed
*WireRevisionSkewError before any post-handshake call when the server's wire
shape falls outside [ClientMinWireRevision, ClientWireRevision] -- mirroring
DESIGN.txt sec 4's "Client rule". A decoded 0 is treated as "the server
omitted the member" per the permanently-retired-values rule, never as a value
needing refusal. The client now also stamps Bd-Wire-Revision on every request
(client.go's stampRequest), and problem.go gained the per-request mirror:
*WireRevisionUnsupportedError, decoded from the reason:"wire_revision_unsupported"
extension members, for the case where the SERVER refuses this client's
declared revision as too old. Both error types are deliberately distinct --
one is this client declining after reading a 200, the other is the server
refusing a request outright -- and neither branches on bd_version (only
assigns it into error text), so TestNoFeatureBranchesOnBdVersionOrSchemaVersion
stays green.

Also added legacyRevisionFields/decodeRevisionToken to problem.go: a
precondition_failed refusal's expected_version/actual_version, typed *string
on the generated Problem, now tolerates a pre-#6053 server sending either as a
bare JSON integer rather than the decimal string every server since has used,
recovering the field the primary decode silently drops rather than losing it.

Batch cap (task #3): ApplyBatch reads the handshake snapshot on every call and
enforces 100 items unless the server advertises issues.batchApplyLarge, in
which case the ceiling is issueops.MaxApplyBatchItems (1000) -- never a bare
compiled constant. A plan over the applicable cap refuses before the dial: via
the existing *CapabilityError machinery when advertising the capability would
help (D7 case 2), or via a new *BatchTooLargeError (unwraps to
issueops.ErrValidation) when it would not. BatchTooLargeError lives in
handshake.go rather than beside ApplyBatch because every exported method in
writes.go is swept by TestEveryOperationMethodRoutesThroughTheSharedDispatch as
an operation-dispatch method.

All three internal/httpclient/... packages build and `go test` passes with
TMPDIR=/data/projects/.scratch/s2t BEADS_TEST_SKIP=dolt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(httpclient): REQUIRE-mode credential resolution bridge to backends.Credential

Task #4: adds the knob internal/storage/backends/backends.go's OpenOptions.Credential
doc explicitly deferred ("S1 does not add a RequireCredential knob to enforce
the multi-tenant rule mechanically -- that is deferred to S2").

ProvidedCredential wraps an embedder-supplied wire.CredentialProvider (via its
httpclient alias) as a backends.Credential, so a multi-tenant embedder (for
example gc, Gas City: one process serving many workspaces) can carry a
per-workspace credential through backends.OpenOptions without the backends
package needing to know this package's CredentialProvider shape.

ResolveCredential(opts, base, require) is the bridge itself:
  - nil Credential, require=false: NewBearerProvider(base), the existing
    single-tenant ambient bearer ladder (unchanged).
  - nil Credential, require=true: ErrCredentialRequired -- the multi-tenant
    fail-closed case backends.go's doc names: a missing per-workspace
    credential must be a loud refusal, not a silent fall-through to
    process-global ambient env that cannot tell one tenant's workspace from
    another's.
  - a ProvidedCredential with a non-nil Provider: returned directly,
    regardless of require.
  - a ProvidedCredential with a nil Provider, or any other
    backends.Credential this package does not recognise: refused -- the
    former as ErrCredentialRequired (a present-but-empty wrapper must not
    look like "a credential was supplied"), the latter wrapping
    backends.ErrUnsupportedCredential per the OpenWith contract
    internal/storage/backends/open_with_test.go already pins
    (TestBackendOpenWithRefusesUnsupportedCredentialType).

No bd-enterprise source was copied for this file; it is new code written
against the OSS backends.go contract and the existing BearerProvider/
CredentialProvider types already lifted into this package. Live-mutated all
three guards (the require branch, the ProvidedCredential type assertion, and
the nil-Provider check) and confirmed each produces a RED test before
reverting.

go build/vet/gofmt clean; `go test ./internal/httpclient/...` passes with
TMPDIR=/data/projects/.scratch/s2t BEADS_TEST_SKIP=dolt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(httpclient): S2 task #6 wire-revision/legacy-decode coverage, fix legacyRevisionFields bug

Add httptest coverage for the two wire-revision mechanisms task #2 added
(commit 7ff68b29d8):

- handshake_wire_revision_test.go: the client-initiated skew gate
  (checkWireRevision/WireRevisionSkewError) and the per-request
  Bd-Wire-Revision header stamp. Live-mutated both switch arms in
  checkWireRevision and the stamped header value; confirmed RED, reverted.

- problem_wire_revision_test.go: the server-initiated
  wire_revision_unsupported refusal (reason-before-code dispatch priority)
  and the legacy-integer revision-token fallback.

Writing the legacy-decode tests surfaced a real bug in legacyRevisionFields:
its presence guard treated any non-nil *string field as "already decoded,"
but encoding/json's indirect() pre-allocates a pointer to "" for a *string
field before discovering a type mismatch (e.g. a bare JSON integer). A
type-mismatched expected_version/actual_version therefore arrived as a
non-nil pointer to the empty string, not nil -- exactly the one case the
fallback exists to recover, and exactly the case the old guard skipped.
Fixed by adding revisionNeedsFallback(p) (nil-or-empty), used in place of
the bare nil checks. No legitimate server response uses a literal
empty-string revision token, so this changes no other behavior.
Live-mutated the fixed guard back to the old nil-only checks; confirmed RED
on both recovery tests, reverted.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(httpclient): S2 task #6 ApplyBatch item-cap coverage

Add writes_batch_cap_test.go: httptest coverage for ApplyBatch's
capability-gated item ceiling (task #3), separate from the generic
two-speed "is issues.batchApply routed at all" gate writes_twospeed_test.go
already covers.

Cases: at-or-under the 100-item floor succeeds without the capability;
over the floor without issues.batchApplyLarge refuses with a
*CapabilityError before any dial past the handshake; the capability raises
the ceiling to issueops.MaxApplyBatchItems (1000), both at the new ceiling
and strictly between the old floor and it; over the absolute ceiling
refuses with *BatchTooLargeError regardless of the capability, also
without dialing the batch-apply endpoint.

Live-mutated all three guards in ApplyBatch (the limit computation, the
n > limit check, and the capability-vs-absolute-ceiling branch); confirmed
each produces a distinct RED, reverted.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(httpclient): S2 task #6 live TLS coverage for wrong-CA refusal

Add ca_wrong_ca_test.go: an end-to-end httptest TLS dial proving
TransportFor's configured CA pool is actually enforced at the wire, not
just recorded on the transport struct. ca_test.go already proves the
latter (TestTransportForReplacesRootPoolNotExtends, via
x509.CertPool.Equal) and proves the RIGHT CA dials successfully
(TestTransportForFileRotationClosesSupersededIdleConnections's live GETs);
neither exercises a server whose leaf a DIFFERENT CA signed.

TestTransportForWrongCARefusesTheTLSHandshake dials a server through a
transport built from an unrelated CA file and asserts the handshake fails
with x509.UnknownAuthorityError. TestTransportForRightCASucceedsOverTLS is
the positive control, using the same (newly-exercised) ca.startServer
helper that was defined but had no caller before this test -- without it,
a mutation that drops RootCAs entirely (falling back to the system pool)
would still pass the wrong-CA test for the wrong reason; the positive
control catches that case. Live-mutated transportForFile to drop RootCAs;
confirmed the positive control goes RED, reverted.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): gazelle BUILD files for the lifted httpclient package tree

Task #7: bazel run //:gazelle, now that internal/httpclient and its
encode/wire subpackages exist in the OSS tree (ab11cb1f8) and have gained
this S2 slice's new files. No existing BUILD.bazel files changed; these
three packages previously had none.

Note for S3: bazel test //internal/httpclient/... fails on tests that read
sibling source trees by relative path at runtime
(TestLedgerRowsAreWellFormed, TestEveryEncoderTableParameterIsPublished,
TestDivergenceLedgerDocMatchesLedger in encode, and
TestNoFeatureBranchesOnBdVersionOrSchemaVersion,
TestEveryOperationMethodRoutesThroughTheSharedDispatch,
TestEveryOperationMethodIsCoveredByTheTwoSpeedTable in wire) -- paths like
../../../cmd/bd and ../../httpapi/spec/openapi.v0.yaml resolve under `go
test`'s working directory but not Bazel's sandbox, which gazelle cannot
infer `data` attributes for. Pre-existing in code from task #1/#2, not
introduced by this commit; `go test ./internal/httpclient/...`,
`go vet ./...`, `gofmt -l .`, `golangci-lint run ./internal/httpclient/...`,
and `make api-check` are all clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpclient/wire): tolerate bare-integer revision on success-path responses

Review HIGH #2: against a fake pre-#6053 server (one whose ContextResponse
omits wire_revision), CloseIssue and its revision-bearing siblings
(ApplyBatch, ReleaseIssue, ReopenIssue, UpdateIssue) failed with an untyped
`json: cannot unmarshal number ... revision of type string`. S0's
string-or-number tolerance (problem.go's legacyRevisionFields) only covered
the problem+json error path, never the success path.

Adds revision_tolerance.go: a pre-unmarshal rewrite that walks a response
body and turns a bare JSON number at a "revision" key into its exact decimal
string, scoped to the five apigen response types that carry the member and
gated on the handshake having seen no wire_revision at all. Left
apigen/types.gen.go and the OpenAPI spec untouched (both are
regenerate-and-diff gated by `make api-check`), rather than introducing a
custom Revision JSON type through codegen, to keep the fix's blast radius to
the wire package's own reader.

Updates ClientMinWireRevision's doc comment (handshake.go) to describe
tolerance on both halves of the wire instead of overclaiming the error-path
fix alone covered it. Adds revision_tolerance_test.go: a read (ListReadyWork)
and a guarded write (CloseIssue with ExpectedVersion) against a fake
pre-#6053 server, plus a modern-server control proving the rewrite never
fires when wire_revision is present. Mutation-tested: disabling the rewrite
reproduces the exact untyped UnmarshalTypeError the review reported.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): sandbox internal/httpclient's source-reading tests (Bazel gate green)

Review HIGH #1: the Bazel lane was red because three tests under
internal/httpclient read sibling/cross-package files by relative OS path at
runtime, which bazel test's sandbox does not materialize without an explicit
`data` dependency.

- internal/httpclient/encode/BUILD.bazel: encode_test gets
  data = ["//cmd/bd:go_srcs", "//internal/httpapi/spec:openapi.v0.yaml",
  "//:engdocs/design/http-divergence-ledger.md"] + glob(["*.go"]), covering
  bijection_test.go's cmd/bd scan and ledger_render_test.go's golden-doc read.
- internal/httpclient/wire/BUILD.bazel: wire_test gets
  data = ["//cmd/bd:go_srcs", "//internal/httpclient:go_srcs"] + glob(["*.go"]),
  covering skew_test.go's AST sweep of both trees.
- internal/httpclient/BUILD.bazel gains a go_srcs filegroup via `make
  bazel-sync` (tools/bazel/go_srcs.py): added "internal/httpclient" to the
  script's managed PACKAGES list rather than hand-writing the block.
- internal/httpapi/spec/BUILD.bazel and the root BUILD.bazel gain
  exports_files for openapi.v0.yaml and engdocs/design/http-divergence-ledger.md,
  scoped to //internal/httpclient/encode:__pkg__.
- bijection_test.go's loadPublishedParameters now reads the wire contract
  through spec.OpenAPIV0() (the package's existing go:embed) instead of a
  relative os.ReadFile path, per review suggestion -- the embed makes the
  parameter set reachable without a `data` dependency on the YAML at all; the
  openapi.v0.yaml data entry above stays as defense in depth for any other
  reader the package gains.

Ran via `bazel run //:gazelle` then `python3 tools/bazel/go_srcs.py`
(bazel-sync-check passes). `bazel test //internal/httpclient/...` is fully
green: 3/3 test targets pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(httpclient): scrub private/enterprise infra references from code and docs

Review MEDIUM #3. Provenance headers ("Contributed by gascity from
bd-enterprise...") stay by design -- this removes references to private
infrastructure and unported internals from comment bodies and generated docs:

- wire/doc.go: dropped the internal/enterprise/hostedbeads and
  internal/enterprise/storageprofile build-tag comparison (neither exists in
  this OSS tree) and the dangling citation of the never-written
  engdocs/design/http-client-backend.md, replacing both with an accurate
  description of this package's own build-tag-free status and a pointer to
  the divergence ledger that does exist.
- credential.go: CredentialProvider's doc no longer names gwauth/DPoP/gateway
  auto-detection by their private spelling; describes the interface generically.
- ca.go: dropped "cherry-to-AWS RTT" (generalized to "a typical cross-region
  RTT") and the cmd/bd/enterprise_connect.go file reference (generalized to
  "a `bd connect --ca-file X`-style command").
- ca_permissions_unix.go / ca_permissions_unix_test.go: ~/.config/gascity and
  /etc/gascity example paths replaced with ~/.config/bd and /etc/bd.
- encode/skew_test.go: beads.corp replaced with bd.example.invalid.
- encode/ledger_render.go + ledger_render_test.go: the generated-doc preamble
  and regenerate-hint strings now name internal/httpclient/encode (this
  package) instead of internal/enterprise/httpstore/encode; regenerated
  engdocs/design/http-divergence-ledger.md via `go test
  ./internal/httpclient/encode -run TestDivergenceLedgerDocMatchesLedger
  -update-ledger-doc` rather than hand-editing it.

Final grep for enterprise/cherry/gascity/corp/gwauth under internal/httpclient
and engdocs turns up only: provenance headers (kept deliberately), this
session's own "no bd-enterprise source copied" disclaimers, and a handful of
generic "enterprise-tagged"/"enterprise server" references describing a
build-tag or commercial-tier server concept in the abstract -- none name a
private host, path, or credential.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(httpclient): review LOW items -- S3 TODO, env-driven CA test, lockstep doc

Review LOW #4-#6:

- wire/skew_test.go: TODO(S3) on the branch where the cmd/bd AST sweep is
  vacuous (no store/dial wiring yet), naming the slice that closes it.
- ca_wrong_ca_test.go: TestTransportForWrongCAViaEnvRefusesTheTLSHandshake,
  the BEADS_HTTP_CA_FILE env-driven sibling of the existing direct-CAFile
  wrong-CA test -- same live-TLS refusal, resolved through resolveCAFile's
  host-scoped env syntax instead of Target.CAFile.
- vocabulary_test.go: expanded the ClientWireRevision/CurrentWireRevision
  comment to explain why this repo holds them in strict lockstep (client and
  server ship from the same commit) rather than merely <=, distinguishing it
  from the real cross-version compatibility ClientMinWireRevision exists for.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpclient/wire): re-review HIGH-a/b -- cover IssueDetails, stop recursing into metadata

HIGH-a: the legacy-revision set was missing apigen.IssueDetails (GET
/v0/beads/issues/{id}'s body, an alias for types.IssueDetails that a grep for
"type ... struct" never sees). getIssue is a baseline op that can dispatch
with no cached handshake at all, so serverPredatesRevisionStrings now treats
"no snapshot" the same as "WireRevision == 0" instead of only the latter.
revisionBearingResponse's set is now pinned exhaustively against
internal/httpapi/wireshape/testdata/golden.json -- the same committed digest
TestWireShapeDigest guards -- by a new test
(TestRevisionBearingResponseCoversEveryWireShapeSchemaWithARevisionMember), so
a future schema gaining a `revision` member can't silently miss coverage
again. New coverage: both "with" and "without" a prior handshake for getIssue,
and an explicit items[].revision test for ApplyBatchResponse.

HIGH-b: the previous fix recursed into every nested object looking for a
bare-number "revision" key, which reached caller-owned blobs like
CloseIssueResponse.issue.metadata -- real user data shaped like
{"revision":3} came back silently coerced to a string, on every server the
broadened gate above now also covers. tolerateLegacyRevisionNumbers no longer
recurses at all: it rewrites only the body's own top-level "revision", and,
for *apigen.ApplyBatchResponse specifically, each item's top-level
"revision" in the top-level "items" array. New
TestCloseIssueLegacyRewriteNeverTouchesIssueMetadata proves a metadata
document with "revision" keys at three nesting depths round-trips
byte-for-byte unchanged.

Mutation-tested both guards by hand (not committed): dropping the
IssueDetails case turns TestRevisionBearingResponseCoversEveryWireShapeSchemaWithARevisionMember
and both getIssue tests red with the exact missing-coverage/decode-failure
messages; reinstating a fully recursive rewrite turns
TestCloseIssueLegacyRewriteNeverTouchesIssueMetadata red, showing the
metadata's revision keys silently rewritten at all three depths. Both
reverted clean before this commit.

internal/httpapi/wireshape/BUILD.bazel and internal/httpclient/wire/BUILD.bazel
export/declare testdata/golden.json as data for wire_test's new exhaustiveness
test.

Verified: go test ./internal/httpclient/... (ok, one pre-existing unrelated
flake in ca_wrong_ca_test.go confirmed not reproducible across 8 reruns),
golangci-lint run ./internal/httpclient/... (0 issues), make api-check (no
apigen drift), bazel test //internal/httpclient/... //internal/httpapi/wireshape/...
(5/5 PASSED).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpclient): stop the wrong-CA TLS tests racing Happy Eyeballs

Root cause: startServer rewrites its httptest TLS server's URL to the
hostname "localhost" so SNI is actually sent, but the server itself only
binds 127.0.0.1. net/http's default dialer does RFC 6555 Happy Eyeballs
for a hostname with more than one address family, racing a tcp6 dial to
::1:<port> against the real tcp4 dial. On this host, many unrelated
processes (JVM tooling) are bound to high ::1 ports, so the tcp6 race
occasionally "won" against a listener that had nothing to do with the
test, and the client spoke TLS to it instead -- surfacing as "tls: first
record does not look like a TLS handshake" rather than the expected
x509.UnknownAuthorityError. Confirmed via ss -tlnp captured at the moment
of failure (a java process holding the exact port raced) and via an
immediate manual redial to 127.0.0.1 on that port succeeding against the
real, still-live test server.

Fix: forceIPv4Loopback (new shared helper in ca_test.go) rewrites the
returned *http.Transport's DialContext to dial tcp4 only. SNI and the
request Host are unaffected, since crypto/tls and net/http derive both
from the URL, never from which address family carried the bytes. Wired
into every TLS fixture test in the package (ca_wrong_ca_test.go's three
tests, and ca_test.go's rotation test) -- grepped the rest of the repo
for the same "rewrite to localhost for SNI" pattern and found no other
occurrences.

Also fixed a latent bug this surfaced: TestTransportForWrongCAViaEnvRefusesTheTLSHandshake
called TransportFor(Target{}) with a nil BaseURL, so caHostMatches always
returned false and the CAFileEnv path was never actually exercised --
the nil RoundTripper was silently masked by http.Client's fallback to
http.DefaultTransport, which also failed to verify (via system roots)
for an unrelated reason. Now passes Target{BaseURL: u} so the test
exercises the host-scoped env-resolution path it documents.

go test -race -count=50 -run WrongCA: 8/8 clean. Whole package
go test -race -count=20: clean, twice. bazel test, golangci-lint,
make api-check: all green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpclient): repair CI-red tests and apply review fixes (adopt-pr #7210)

Three defects on the contributor head, none of them present on the base:

- test/docsync's TestEngdocsAndRootMarkdownLinks fails: the generated
  divergence-ledger doc links http-client-backend.md, an architecture
  spec this repository does not carry. Name it in a code span instead,
  in ledgerDocPreamble, and regenerate the doc (-update-ledger-doc).

- TestCAFileHygieneRefusesGroupWritableFile fails under umask 022:
  os.WriteFile's 0o660 passes through the umask, so the file lands 0640,
  is never group-writable, and TransportForFile accepts it. Chmod it
  explicitly. Its acceptance twin had the same gap on its file (only the
  explicitly chmodded directory reached the user-private-group
  exception), so chmod that file too: a mutant that drops the exception
  for files alone passed the old test under umask 022 and fails the new
  one.

- That acceptance twin calls the unix-only isUserPrivateGroup from
  ca_test.go, which carries no build constraint, so the package's tests
  do not compile for GOOS=windows. Move it, otherwise verbatim, into
  ca_permissions_unix_test.go beside the other isUserPrivateGroup tests.

Review fixes (iteration 1):

- Encode dispatched through nine case arms of op/shape conjunctions
  (cyclomatic 24). Switch on an (op, shape) struct key instead, and move
  the getIssue and search-bridge arms into encodeGetIssue and
  encodeSearch. The arms are disjoint, so dispatch and every error string
  are unchanged. Cyclomatic 24 -> 10.

- EventStream.Next kept four accumulators and the per-field switch in
  one loop (cognitive 27). Gather the accumulators into sseEvent and
  fold each field line, per-event cap included, in applyField. Next
  cognitive 27 -> 12; applyField is 11/11.

- The CA hygiene walk joined a relative symlink target onto the textual
  containing directory, so with a/link -> ../b/c and b/c/x -> ../y it
  resolved a/y where the kernel opens b/y: a valid CA was refused, or a
  hygienic file at a/y was checked and read instead. readLinkTarget joins
  onto the directory's EvalSymlinks path, taken after that directory's
  own ancestors passed the hop-by-hop check. The regression test plants
  a decoy at a/y; a lexical-join mutant reads it and fails.

- SaveTarget wrote a relative CAFile that every later LoadTarget
  refuses. Both now go through checkCAFileAbsolute, so SaveTarget refuses
  first, with LoadTarget's own message, and writes nothing.

- Document the cached transport TransportFor and TransportForFile return
  as shared and read-only.

Review fixes (iteration 2):

- The walk still dropped a ".." inside a symlink target textually:
  filepath.Join cleans a relative target, and an absolute target's
  ancestors come from filepath.Dir, while the kernel takes that ".."
  from wherever the component before it really leads. Both reviews
  rated the relative leaf as a wrong-file read, and it is one: with
  x/d -> ../e/s, x/L -> d/../y read a decoy at x/y where the kernel
  opens e/y. The other three shapes are hygiene bypasses: an absolute
  leaf target x/d/../y, and an ancestor link to d/../dir or x/d/../dir
  with the CA at x/L/y, each read a file in a world-writable e that the
  walk never checked. readLinkTarget now refuses a target with a ".."
  after any other component, relative or absolute; a leading run of
  ".." is still taken from the directory's physical path. The test pins
  all four shapes behind a control that refuses e/y named directly, and
  a table pins hasInteriorDotDot's boundary. Mutants that drop the
  refusal, limit it to relative targets, refuse every "..", or count
  "." as a component each fail.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
2026-10-05 02:58:34 -07:00
Julian Knutsen
4af58c97fd
feat(count): Counter scope — ParentID, NoParent, ExcludeTypes, ExcludeStatus (issues.count.scope) (#7199)
* feat(httpapi): wire-shape signal — wire_revision, floor header, golden digest, param tokens

S0 of the HTTP role-parity program: give a client a single place to learn
whether this build's wire shape has moved under it, and give CI a gate that
catches an undocumented move.

- GET /v0/beads/context reports wire_revision (bumps only on a non-additive
  response-shape change) and min_client_wire_revision. An optional
  Bd-Wire-Revision request header lets a client declare the revision it was
  built for; checkWireRevision refuses a declared revision below the floor
  with 400 invalid_argument/reason=wire_revision_unsupported and a
  min_wire_revision field, on every route but /healthz (wireRevisionExempt) —
  including the identity handshake, where catching the mismatch is cheapest.
  Absent header is today's behavior; malformed value is invalid_value, never
  wire_revision_unsupported. The floor lives in Server.minClientWireRevision
  (defaulting to the MinClientWireRevision constant) so a test can raise it.
- internal/httpapi/wireshape: a golden digest over every response's every
  content-type schema (including application/problem+json) across the whole
  spec, keyed by schema+member. TestWireShapeDigest fails closed when a
  member's JSON name or type changes without wire_revision bumping to match;
  regenerate with `go run ./internal/httpapi/wireshape/cmd/gendigest`.
- GET /v0/beads/issues' `sort` parameter now carries the issues.listSort
  capability token, the same convention `capabilities` entries use.
  TestNewParameterOnExistingOperationHasABehaviorToken requires every new
  parameter on an existing operation to carry one, checked against a
  write-once grandfather baseline (testdata/pretoken_parameters.json) of
  parameters that predate the rule; TestUntokenedParameterRuleFires proves
  the checker itself against a synthetic fixture.
- Docs: engdocs/design/bd-serve-v0.md gets a "The wire-shape signal" section;
  CHANGELOG.md gets an Added entry.

Live-mutated and reverted three RED demonstrations during development:
retyping api_version in the spec failed TestWireShapeDigest; short-circuiting
checkWireRevision's floor comparison failed
TestWireRevisionBelowFloorRefusesEveryEnforcedRoute; adding an untokened
query parameter to listIssues failed
TestNewParameterOnExistingOperationHasABehaviorToken. All three now pass
against the unmutated tree.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): rename issues.listSort to issues.list.sort, harden token rule

Opus review of 12c5144ca (HIGH-1, MEDIUM-5):

- Rename the capability token issues.listSort -> issues.list.sort to match
  bd-enterprise's client (httpstore/wire/handshake.go:121, DESIGN.txt S4e),
  and widen capabilityToken to accept an optional third segment.
- untokenedParameters() now requires the extracted token to be one
  Capabilities() actually serves, not just backtick-shaped text. This
  catches real false positives that existed in the spec already
  (workapi.DefaultReadyLimit, status.custom) which only shape-matched the
  old regex; they're added to the pretoken baseline as genuinely
  pre-existing untokened parameters.
- Extend the rule to request body members (new
  testdata/pretoken_request_body_members.json baseline) and exempt
  brand-new operations via a frozen testdata/pretoken_operations.json
  snapshot.
- Add TestPretokenBaselinesNeverGrow pinning all three baseline sizes so
  a future change can't hand-add an exemption instead of a token.
- Update spec prose for the widened rule scope.

go test ./internal/httpapi/... and the reviewer's pmut.py all pass as
expected: plain/bogus modes go RED, reuse (real-but-unrelated token) stays
green, matching the review's literal ask ("a token must be one advertised
by Capabilities()").

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(wireshape): close four digest-coverage gaps, add gendigest write guard

Opus review of 12c5144ca (HIGH-2, MEDIUM-4). The digest only tracked
Type/Format/Enum/Required on direct properties, missing:

- nullable: true (never read at all)
- array item shape (Issue.labels items string->integer was invisible;
  only object-shaped items got recursed into)
- additionalProperties value shape (IssueCount.groups)
- request bodies entirely (Compute() only walked responses, so
  DeleteIssuesRequest.expected_version string->integer, the other half
  of #6053, was invisible)

Also added defensive oneOf flattening alongside the existing allOf merge
(dormant today -- grep confirms this spec uses oneOf nowhere -- but no
longer silently invisible if that changes).

Extracted the golden-vs-computed comparison into wireshape.Compare(),
shared by TestWireShapeDigest and the new wireshape.SafeToWrite(), which
gendigest now calls before writing: it refuses to overwrite golden.json
with changed/removed entries unless the candidate's wire_revision is
strictly greater than the existing golden's. Pure additions always write.
TestSafeToWrite exercises both guard directions.

golden.json is regenerated. This single regen is a one-time exception to
the new guard, done by writing the digest directly rather than through
gendigest: confirmed via a throwaway diff (not checked in) that every one
of the 57 "changed" entries differs only in the NEW fields the wider walk
now populates (ItemType/ItemFormat/Nullable/AdditionalProps*) -- every
pre-existing Type/Format/Required/Enum value is byte-identical. Nothing on
the wire changed; the measurement got more precise. Bumping
CurrentWireRevision for this would have written a false "the protocol
changed here" entry into the revision history table. Going forward, any
REAL change the guard now reports uses the normal bump-then-regenerate
path, same as before.

Verified against the reviewer's mut.py: nested_type, nullable,
nullable_issue, array_items, addlprops, request_body and format all go RED
on TestWireShapeDigest, each reverted after confirming.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): make bd_version wire_revision inference advisory

Opus review of 12c5144ca (HIGH-3). The spec told a client inferring
wire_revision from bd_version to use the release cutoff >= 1.3.0, and said
nothing about what happens when that inference is wrong -- which it
demonstrably is for real builds already in the field:

- main builds up to commit b3ef65c85 report bd_version 1.2.2 (below any
  1.3.0 cutoff) while already sending the string-typed revision/
  expected_version shape (#6053).
- 1.3.0-rc.N pre-release builds sort BELOW the release cutoff 1.3.0 under
  plain string comparison despite already carrying the string shape.
- The bd-enterprise compatibility line at d3ab32773462 reports bd_version
  1.1.0 while also sending strings.

Moved the cutoff to >= 1.3.0-rc.1 (compared as semver-with-prerelease,
which at least captures the rc builds), documented both named exceptions
inline, and made the whole inference explicitly ADVISORY: a client must
accept revision/expected_version as either a string or a number whenever
wire_revision is absent (or probe directly) rather than let the inferred
value choose a decoder, and must never refuse solely because the inferred
revision is 0. Mirrored into engdocs/design/bd-serve-v0.md.

This is spec/doc prose only -- there is no server-side inference code to
change, since this server always sends wire_revision directly; the
inference is guidance for clients reading a pre-signal server's response
(this server, past builds, or an unrelated implementation).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): LOW-7/8/9/10 review follow-ups for wire-revision signal

LOW-7: WireRevisionUnsupported now also sets Problem.wire_revision (this
server's CurrentWireRevision) and Problem.bd_version on the
wire_revision_unsupported refusal, alongside the existing
min_wire_revision, so a client logging the refusal has this server's
full self-description without a second /v0/beads/context call.

LOW-8: gazelle + make bazel-sync add the BUILD.bazel entries the prior
commits were missing: wire_revision.go/wire_revision_test.go in
internal/httpapi's srcs, new BUILD.bazel files for wireshape and
wireshape/cmd/gendigest, and a testdata glob on httpapi's go_test so
the pretoken_*.json baselines are visible under `bazel test`. Verified
with `bazel build //internal/httpapi/...` and
`bazel test //internal/httpapi/... --test_env=BEADS_TEST_SKIP=dolt`
(both green) and `python3 tools/bazel/go_srcs.py --check` (clean).

LOW-9: narrowed CHANGELOG.md and engdocs/design/bd-serve-v0.md to
describe the digest's actual final coverage (format/enum/required/
nullable, array items, additionalProperties, request bodies, $ref/
allOf/oneOf recursion, the SafeToWrite revision-bump guard) and the
token rule's actual final shape (three pinned baselines, request-body
members, Capabilities()-membership validation, new-operation
exemption) instead of the original, narrower claims.

LOW-10: Bd-Wire-Revision now parses via a strict decimal-only
parseStrictNonNegativeInt (no leading '+', no leading zero except a
bare "0") instead of strconv.Atoi alone, so two different header
bytes can no longer name the same revision. New malformed-header test
cases: "+1", "01", "007".

Regenerated apigen/types.gen.go (make api-gen) for the new Problem
fields and wireshape/testdata/golden.json (go run
.../wireshape/cmd/gendigest) for the two new additive Problem members;
the latter wrote cleanly through SafeToWrite itself, since both are
pure additions.

go test ./... and bazel test //internal/httpapi/... both green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): re-review LOW follow-ups — retirement test, gendigest -init, docs

1. TestRetiredRevisionsStayBelowTheFloor pins the PRESENCE guarantee the
   wire_revision property doc relies on: CurrentWireRevision >= 2 (so 0
   and 1 stay permanently retired) and MinClientWireRevision <=
   CurrentWireRevision (so this build's own floor never refuses every
   request against itself).

2. gendigest now refuses to create a golden.json that does not exist yet
   unless run with -init. Previously a missing file (moved testdata,
   wrong path, accidental delete) was indistinguishable from a real first
   run and silently wrote a fresh golden with no revision history checked
   against anything. writeGolden() is extracted out of main() so this is
   covered directly: TestWriteGoldenRequiresInitWhenMissing,
   TestWriteGoldenInitCreatesWhenMissing,
   TestWriteGoldenIgnoresInitWhenGoldenExists (the existing-golden path is
   still governed only by SafeToWrite, -init or not).

3. Narrowed the two remaining stale claims that still described the
   digest's original, narrower coverage: the wire_revision property's
   prose in openapi.v0.yaml (regenerated into apigen/types.gen.go via
   make api-gen) and wireshape/doc.go now both list nullable, array item
   shape, additionalProperties value shape, request bodies, and
   $ref/allOf/oneOf recursion, plus the new -init guard.

go test ./internal/httpapi/... and make api-check both green;
bazel test //internal/httpapi/... (httpapi, wireshape, and the new
gendigest test target) green; python3 tools/bazel/go_srcs.py --check
clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(wireshape): cover operation parameters in the wire-shape digest

The digest previously walked only response and request-body schemas, so a
type, enum, required, or removal change on one of the spec's 138 operation
parameters (123 query, 14 path, 1 header) passed the gate silently — the
open item from PR review on #7182.

Each parameter is now recorded as its own Entry, keyed by
"param:"+operationId / location+":"+name (operation-scoped, not
schema-scoped, since two operations' same-named parameter are independent
wire contracts), capturing type, array item shape, enum, required, style,
explode and default. Adding a parameter stays additive; removing,
retyping, narrowing its enum, or changing required/style/explode/default
now trips SafeToWrite exactly like a response or request-body change.

Verified against the live spec: retyping listIssues' `limit` query
parameter from integer to string now fails TestWireShapeDigest (confirmed
red, then reverted — not part of this commit). golden.json regenerated via
gendigest; the new parameter entries are a pure addition, no wire_revision
bump needed. Added TestParameterMutationsAreCaught (remove/retype/narrow-
enum against real golden entries) and updated wireshape's doc.go, the
`wire_revision` spec prose, and the CHANGELOG to describe the new
coverage.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(wireshape): review fixes — stale-golden and lowered-revision arms, effective param serialization, digest boundary

Maintainer fixup from the adopt-pr review loop (iterations 1 and 2), on
top of e17ba924f.

- goldenDrift (A, I): TestWireShapeDigest's comparison moves into a
  goldenDrift helper. An addition alongside a moved wire_revision no
  longer passes with a stale golden: it asks for a regenerate, as a
  changed or removed entry after a bump already did. Every message now
  names response/request-body members AND parameters, and TestGoldenDrift
  pins each arm. This takes only the test half of the stranded fixup
  (adopt-pr/bd-47kdge/stranded-fixup-528cd6095); its doc half, which says
  parameters are outside the digest, is false since e17ba924f and is not
  replayed (J).
- Lowered wire_revision (D): a CurrentWireRevision LOWER than the
  golden's is a wrong constant, not a stale golden, so goldenDrift says to
  restore it instead of regenerating, and SafeToWrite (so gendigest)
  refuses any write at a lower revision, whatever the entries.
- Effective style/explode (B): walkParam records style and explode with
  OpenAPI 3.0's defaults filled in (form for query/cookie, simple for
  path/header; explode defaults to style == form). Adding `explode: false`
  to a query array that relied on form's implicit explode is now a
  changed entry, while spelling out a default changes nothing.
  testdata/golden.json is regenerated with -init: 111 parameter entries
  change only in style/explode (96 to form/true, 15 to simple), 828
  entries before and after, wire_revision stays 2. The golden is new in
  this PR and unreleased, so this needs no bump.
- TestWalkParam (H): a package-internal table test
  (wireshape_internal_test.go) pins every field walkParam extracts,
  through a computeFrom(document, revision) seam that Compute now wraps:
  per-location defaults, $ref'd parameters and items, format, nullable and
  default, content parameters, and the nil returns.
  TestEffectiveSerializationIsCompared pins B end to end through Compare
  and SafeToWrite. BUILD.bazel embeds the library in the go_test, which is
  gazelle's own output.
- Required inputs and the digest boundary (C, E, F; docs only): all five
  prose surfaces (openapi.v0.yaml's wire_revision description and the
  regenerated apigen/types.gen.go, wireshape/doc.go, CHANGELOG.md,
  engdocs/design/bd-serve-v0.md) now say that a new REQUIRED request
  input is recorded as an ordinary addition and needs its own review,
  that value constraints and member-level composition are outside the
  digest, and that object and content parameters are recorded only as
  their container. bd-serve-v0.md also gains the parameter coverage it
  was missing.
- G: a brand-new operation is permanently exempt from the
  capability-token rule; its own review decides its capability story.
- Iteration 2 (nit): walkParam's comment now says that the location
  default it records as a `content` parameter's style and explode is a
  placeholder, since OpenAPI 3.0 applies style and explode only to a
  parameter described by `schema`. All 138 operation parameters in the
  spec use `schema`, so the golden does not change.

Validated: go test ./internal/httpapi/... (gms_pure_go), make api-check,
go vet for linux/windows/darwin, gofmt, git diff --check, gazelle on the
package (no drift), make bazel-sync-check, and bazel test
//internal/httpapi/wireshape/... in the local processwrapper sandbox
(2 of 2 pass). Overlay mutation probes: dropping the added-after-bump
arm, the lowered-revision arm, SafeToWrite's lowered check, or the
effective-default fill each turns the new tests red.
Iteration 2 changes only that comment; go test
./internal/httpapi/wireshape/... (gms_pure_go), go vet, gofmt and git
diff --check pass again.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(count): add Counter scope fields (S8) — ParentID, NoParent, ExcludeTypes, ExcludeStatus

Extends issueops.CountRequest with ParentID, NoParent, ExcludeTypes and
ExcludeStatus, closing the gap where bd count could not express the
filters bd list already has, forcing downstream consumers like gc to
fall back to listing everything just to get a count.

- issueops.Counter: four new CountRequest fields, documented against the
  List equivalents they mirror (ParentID/ExcludeTypes) or newly define
  (NoParent, ExcludeStatus has no List counterpart). Status "all" is
  pinned as equivalent to unset, matching BuildListFilter's own "all" and
  preserving count/list cardinality parity (the role's reason to exist).
- internal/workapi.BuildCountFilter: maps the four fields onto the one
  types.IssueFilter builder all three storage legs (dolt, embeddeddolt,
  uow) already share via internal/storage/issueops and sqlbuild, so no
  per-leg storage code was needed.
- cmd/bd: --parent, --no-parent, --exclude-type, --exclude-status flags.
- HTTP: adds parent/no_parent/exclude_type/exclude_status to GET
  /v0/beads/issues:count, behind the new issues.count.scope behavior
  token (routes.go), wired in reads.go's countFilters, documented in
  openapi.v0.yaml with per-parameter tokens and an extended capabilities
  description. gendigest confirms a byte-identical golden.json: query
  parameters are outside the wire-shape digest's scope, so this is a
  zero-diff, no-revision-bump change as expected.
- backend/conformance: four new Counter contract cases (ParentID scoping,
  NoParent's edge-only semantics vs ParentID's edge-or-dotted-id match,
  ExcludeTypes composing with include_infra's own exclusions, and
  ExcludeStatus), run identically against all three storage legs via a
  new CounterFixture.AddDependency hook shared with ReaderFixture.

Verified: go build ./...; go test ./internal/httpapi/... ./internal/workapi/...
./issueops/... and the Counter contract on all three legs (dolt via
testcontainers, embeddeddolt, uow via local dolt binary); make api-gen
(additive-only apigen diff) and make api-check; go vet and golangci-lint
clean; gofmt clean; gazelle diff empty (no new files/imports). Mutation
testing: dropping issues.count.scope from behaviorCapabilities turns
TestCapabilitiesAdvertiseEveryImplementedOperation,
TestSpecCapabilityVocabularyMatchesTheRouteTable and
TestNewParameterOnExistingOperationHasABehaviorToken red; no-op'ing
ExcludeStatus in BuildCountFilter turns
TestBuildCountFilterExcludeStatusTakesNamesAsWritten and the
embeddeddolt ExcludeStatusNarrowsThePredicate conformance case red. Both
reverted and reconfirmed green.

Deviation: DESIGN.txt's literal text would make count --status all an
ErrValidation; shipped behind a test and a doc comment instead, since
"all" already means "every status" on both count and list and rejecting
it would break the count/list parity this role exists to guarantee.

S3/S4 note: no OSS HTTP client exists yet to enforce capability-token
skew. A doc comment on CapIssuesCountScope in routes.go specifies the
required behavior once one lands: check issues.count.scope against the
cached handshake capabilities and refuse locally with a typed
ErrUnsupported{Capability} before sending any of the four new params to
an older server.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(count): S8 review follow-ups — refusal parity, exclude-status validation, doc fixes, parity tests

Addresses the Opus review's six required changes for S8 (bd count scope
fields):

1. BuildCountFilter now refuses ParentID+NoParent together as ErrValidation,
   matching bd list's CLI refusal wording; wired through the role, the CLI,
   and invalid_argument classification in internal/httpapi/reads.go.
2. Status:"all" is documented in DESIGN.txt as every status, matching List's
   no-filter reading (count/list cardinality parity) — a coordinator decision
   recorded in beads-design/DESIGN.txt, outside this worktree.
3. ExcludeStatus names are now validated against the workspace vocabulary
   (built-in statuses plus workspace-custom ones); an unknown name is
   ErrValidation instead of a silent no-op, since a typo here would overcount.
4. Fixed openapi.v0.yaml and issueops/reader.go doc claims about bd list
   --parent's default tree-walk vs. the one-level role answer that bd count
   --parent agrees with (confirmed by tracing list_show_filter_modes.go); fixed
   an Oxford-comma nit in routes.go and the spec.
5. Added storage-level count == len(List) parity tests for ParentID, NoParent
   and ExcludeTypes (List has no ExcludeStatus counterpart), plus wisp-child
   and closed-child contract cases, across all three backends (dolt,
   embeddeddolt, uow). CounterFixture gained a List hook for this.
6. Extended the routes.go skew note: gc's ErrCountUnsupported fallback to List
   is now named explicitly, with a note that S3/S4 must add a skew test that
   masks the token and asserts the fallback fires.

Also fixed RunCounterExcludeStatusNarrowsThePredicate's now-stale "unrecognized
status excludes nothing" assertion to expect ErrValidation instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(count): PR #7199 review follow-ups — exclude-status config load, ParentID doc swap

Addresses bee's REQUEST-CHANGES on gastownhall/beads#7199 (S8):

1. BLOCKER: storecounter.filter and uow.countFilter only loaded the
   workspace list config when IncludeInfra was set, so BuildCountFilter's
   ExcludeStatus validation refused a workspace's own custom status unless
   the caller also passed the unrelated IncludeInfra flag. Both legs now
   load the config when IncludeInfra OR len(ExcludeStatus) > 0, with
   comments corrected to say so. Added leg-level regression tests (the
   conformance fixture cannot drive a workspace-configured custom status)
   on embeddeddolt and uow, reproducing the reviewer's CLI repro directly,
   plus an HTTP-level probe in cmd/bd's real-Dolt proxied count test.
2. issueops.ReadyRequest.ParentID's comment had been overwritten with the
   one-level direct-children wording that belongs to ListRequest.ParentID;
   Ready actually walks all descendants via GetDescendantIDsInTx (GH#3396).
   Restored the descendants comment on ReadyRequest.ParentID and moved the
   one-level explanation to ListRequest.ParentID, which had none.

Verified: make api-check; dolt-server, embeddeddolt and uow Counter
contract legs (BEADS_TEST_DOLT_SERVER=local with a pinned dolt binary for
uow; Docker for the dolt-server leg); the new proxied HTTP exclude_status
probe (BEADS_TEST_PROXIED_SERVER=1).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore(wireshape): add the countIssues scope parameters to the wire-shape golden

#7182 now digests operation parameters. The four new issues:count parameters
(parent, no_parent, exclude_type, exclude_status) are additive entries, so no
wire_revision bump is needed; regenerated with gendigest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(conformance): register the S8 count-scope cases with RunRoleContracts (adopt-pr #7199)

The PR adds nine RunCounter* contract cases to counter_contract.go and wires
them into the three in-tree TestCounterContract legs, but not into the
Counter row of roleContractCases. TestRoleContractCasesCoverEveryContractCase
derives the expected set from source, so it fails on the PR head, and an
external backend running RunRoleContracts would never see the new cases.

Register them in source-declaration order (after
RunCounterAnUnknownStatusMatchesNothing), which the same guard enforces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(count): correct the S8 scope docs the review found wrong (adopt-pr #7199)

Comment and spec-description fixes only; no behavior change. Review-loop
iterations 1 and 2 of the #7199 relaunch.

- ListRequest.ParentID (issueops/reader.go) said "DIRECT children only
  ... the same one-level clause". The sqlbuild clause has two arms and
  neither walks: the parent-child edge arm stops at one level, but the
  dotted-prefix arm (`id LIKE CONCAT(?, '.%')` for rows with no
  parent-child edge) reaches every edge-less dotted id at any depth, which
  RunReaderListParentReachesEveryDescendantAndOnlyItsOwn pins with an
  edge-less X.1.1. State both arms, and say the CLI tree view re-runs the
  clause against the store once per level (findAllDescendants calls
  SearchIssues, not the role). Document ListRequest.NoParent, which had no
  doc. The contributor's ReadyRequest.ParentID text is kept; only its
  cross-reference to this field changes from "one-level" to "non-walking".
- CountRequest.NoParent (issueops/counter.go) and the count flag comment
  (cmd/bd/count.go) said NoParent counts rows with "neither" an edge nor a
  dotted prefix. Its SQL checks only the edge table, so an edge-less X.1
  is counted by both --parent X and --no-parent, as bd list lists it.
- RunCounterExcludeStatusNarrowsThePredicate's header said an
  unrecognized name "excludes nothing rather than failing" while its body
  asserts ErrValidation. The header now matches the body.
- The OpCountIssues entry in problem.go's operationCodes said the role has
  "exactly one reachable refusal". It has three: an invalid metadata key,
  an unrecognized exclude_status name, and parent with no_parent (named
  no_parent), all classified by invalidFilterParam. The code set is
  unchanged.
- BuildCountFilter's header said cfg is "only read under IncludeInfra".
  It is also read to validate ExcludeStatus against the workspace's
  custom statuses, which is why both callers load it when ExcludeStatus is
  non-empty.
- The countIssues `parent` description in openapi.v0.yaml carried the
  same one-level claim as ListRequest.ParentID; it now states both arms.
  types.gen.go is regenerated (make api-gen); make api-check passes.

Review fixes (iteration 2), still comment and spec text only:
- The iteration-1 problem.go rewrite called all three count refusals
  ErrValidation. Only the exclude_status and parent-with-no_parent
  refusals wrap issueops.ErrValidation (internal/workapi/count.go). The
  metadata-key refusal is ValidateMetadataFilters' plain fmt.Errorf
  chain, so errors.Is(err, ErrValidation) is false for it; it reaches the
  wire as a 400 only because invalidFilterParam matches message prefixes.
  The comment now says that, so a later move to typed errors cannot trust
  it and demote the metadata-key refusal to a 500. The sentinel itself is
  not added here: ValidateMetadataFilters is shared with the list and
  ready builders.
- invalidFilterParam's comment on the count rows pointed at the
  metadata-key rows as "above"; they come after it in the switch.
- The countIssues `parent` description said the CLI tree mode walks
  "every edge-linked descendant". findAllDescendants re-runs the full
  two-arm predicate per level, so it also descends through edge-less
  dotted ids; it now says "every descendant reachable through either
  arm". types.gen.go is regenerated; make api-check passes.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(wireshape): re-record the count scope params under the base's param digest (adopt-pr #7199)

The base branch (feat/wire-revision-signal, #7182 be1f0bad72) now digests
the effective style/explode of scalar query parameters. This PR's golden
entries for countIssues query:parent and query:no_parent were written by
the old algorithm, so after the base refresh TestWireShapeDigest reported
them as "changed without a wire_revision bump".

No bump is owed: all four count-scope entries (parent, no_parent,
exclude_type, exclude_status) are additions this PR introduces and are
absent from the base golden. golden.json was reset to the base copy and
regenerated with `go run ./internal/httpapi/wireshape/cmd/gendigest`. The
result is four pure additions against the base (+35/-0); parent and
no_parent gain "style": "form" and "explode": true, and wire_revision
stays 2.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: place the PR's new embedded counter test in the shard manifest

TestCounterExcludeStatusAcceptsAWorkspaceCustomStatusWithoutIncludeInfra,
added to internal/storage/embeddeddolt in f23bf5ddbf, is a new top-level
embedded storage test. The 15-shard block of
.github/scripts/embedded-storage-test-shards.txt must list every
discovered test exactly once. Without the entry,
TestStorageEmbeddedShardManifestGeneratorNotStale (scripts/) fails, both
on the reviewed head 89a387de6c and on the merge. CI never ran the check
on this PR: pr.yml runs only for PRs based on main, release/** or
hotfix/**, and this PR was based on feat/wire-revision-signal until the
retarget.

Generated with
`python3 scripts/ci/gen_embedded_storage_shard_manifest.py 15 --weights=duration --write`.
The incremental write adds this one line, in shard 4, and moves no other
test.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.local>
2026-10-05 02:57:14 -07:00
Julian Knutsen
aaeaedc386
fix(test): retry contended migration lock when re-running schema init on the shared test DB (#7235)
TestSchemaMigrationDoesNotCommitPreExistingStagedData flaked in the
server-Dolt storage tier (shard 4/16) with:

  initSchemaOnDB: schema migration: schema: acquire migration lock:
  schema migration lock unavailable: timeout

Every setupTestStore caller shares testSharedDB, and MigrateUpWithLock's
GET_LOCK name is database-scoped, so a test that re-runs schema init
contends with every parallel test's setup-time init for the same lock.
setupTestStore already uses initSchemaOnDBWithRetry for exactly this
reason, but schema_migration_dirty_test.go (4 sites) and
schema_version_test.go (2 sites) called the bare initSchemaOnDB, which
gives up after a single 5s wait. On a loaded runner a neighbour holds the
lock longer than that and the test fails. Production code retries this
case (isRetryableError covers schema.IsMigrationLockError), so this is
test-side only.

Route those calls through a rerunSchemaInitOnTestStore helper that uses
initSchemaOnDBWithRetry. It retries only a contended lock or a transient
connection error, never a migration result, so the asserted outcomes
(including the expected "pre-existing dirty tables changed" error) are
unchanged.

Seen on four unrelated PR branches in three days. Bazel repro (shard-4
test set, server pinned to 6 CPUs, --local_test_jobs=6): before 1/16
runs failed with the CI signature; after 32/32 passed.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 02:21:42 -07:00
Julian Knutsen
cd2507735e
test(cli): close mc-zndi7.76's --if-revision single-winner test gaps (#7233)
* test(cli): close mc-zndi7.76's --if-revision single-winner test gaps

T4.4 exercised the Update guard against the library only; embedded CLI
tests never catch a CLI wiring bypass because bd's local exclusive
flock serializes every mutation regardless of whether --if-revision is
actually threaded through. This closes the remaining mc-zndi7.76 gaps:

- gap 1 (mutant MA): new Lifecycle contract cases
  (RunLifecycleUpdateExpectedVersionSingleWinnerWithDisjointColumnsUnderConcurrency,
  RunLifecycleCloseExpectedVersionSingleWinnerUnderConcurrency,
  RunLifecycleUpdateExpectedVersionSingleWinnerAcrossUpdateAndCloseUnderConcurrency),
  wired into all three backends (dolt, embeddeddolt, uow).
- gap 2 (mutant MC): a new dolt-server-tier CLI concurrency test,
  TestSharedServerIfRevisionSingleWinner, spawning N real bd
  subprocesses that race update/close/assign --if-revision against one
  shared, unlocked Dolt server. Delete is deliberately excluded --
  its production guard is independently known-broken under real
  concurrency and is tracked as its own schema-touching fix,
  mc-zndi7.73.
- gap 4 (mutants ML/MO/MD1/MD2): close's --if-revision bypass now has
  CLI-level coverage for --continue/--suggest-next/--claim-next
  rejection, cascade-delete fencing, and a proxied-route stale-guard
  refusal.

TDD: hand-applying mutant MC (dropping update.go's ExpectedVersion
wiring) reproduces the bug report's symptom exactly -- 10/10
concurrent racers exit 0 instead of 1 winner + 9 guard refusals.
Restoring the real code returns to exactly 1 winner, confirmed across
4 repeated real-container runs plus -race.

Gates: gofmt, go vet, go build (both tags), golangci-lint (native,
0 issues), bazel go_srcs.py --check, and targeted go test runs
(embeddeddolt/dolt/uow lifecycle contract suites and the full set of
cmd/bd if-revision tests) all green, with and without -race, against a
real dolthub/dolt-sql-server:2.2.0 container.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): sync cmd/bd BUILD.bazel srcs for the new if-revision test files

make bazel-sync-check only validates the go_srcs.py-managed filegroups,
not gazelle's own go_test srcs list, so the two new test files from the
previous commit were invisible to Bazel. CI's BUILD-sync and go-test
equivalence gates caught it; this is gazelle's own generated diff,
applied via the CI job's uploaded bazel-sync-patch artifact.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-05 01:57:10 -07:00
Julian Knutsen
0773e6334e
docs(agents): contributor-only agent instructions and documented-issue intake (#7223)
Makes the repo's agent and contributor surface describe developing beads, not using bd, and requires every PR to link a documented issue.

- Contributor-only root AGENTS.md with a routing table; cmd/bd/AGENTS.md and engdocs/AGENTS.md seam files; CLAUDE.md symlinks so Claude Code loads nested files.
- Bug and feature issue forms, a PR template that requires Closes #, CONTRIBUTING sections for the triage and pipeline labels, and the opt-in fork-cache during dev setup.
- ROADMAP.md for 1.4: versioned beads (#6132), Graph Preview 2 (#7170), contributor onboarding (#7228).
- docsync guards: root AGENTS.md at or under 20 KB; every AGENTS.md symlinked, routed, and with resolving links, paths, and guard-test names.

Agent-Signature: claude-code-claude-opus-5-5-high on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 08:43:53 +00:00
Julian Knutsen
ec72c8c563
ci(bazel): drop //cmd/bd:bd_test from the integration lane where the cmd/bd Dolt-server tier is required (#7230)
bazel-integration (--config=integration) and bazel-cmd-dolt
(--config=doltserver-cmd, //cmd/bd:bd_dolt_server_test) both run the
integration-build cmd/bd bd_test; the second without BEADS_TEST_SKIP=dolt,
so it is a superset. Once pr.yml requires bazel-cmd-dolt, the integration
lane's bd_test run is a duplicate.

bazel.yml gains a boolean workflow_call input, cmd-dolt-required (default
false). When true, bazel-integration tests //... -//cmd/bd:bd_test. pr.yml
passes it, and a policy test pins it to pr.yml's BAZEL_CMD_DOLT_REQUIRED
(a `with:` cannot read env), so the drop only happens where ci-gate
requires BAZEL_CMD_DOLT. bazel-farm.yml, nightly.yml, push and dispatch
leave it unset and keep bd_test. While the flag is "false" (current main)
nothing changes; with #7226's flip the input flips with it.

Coverage: both lanes share the same `if:` (modes remote, fork-ro, fork-rw,
cache; Dependabot included) and bazel-gate.sh accepts their skips in the
same modes (skip, local). TestBazelIntegrationExcludesBdTestOnlyWhereCmdDoltCovers
pins that, the exact exclusion, the flag/input equality, the gate's
conditional requirement, and that no other caller passes the input.

Rollback: commit BAZEL_CMD_DOLT_REQUIRED "false" and cmd-dolt-required:
false together in pr.yml.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 01:19:47 -07:00
Julian Knutsen
15d74c08ff
ci: require the cmd/bd Dolt-server tier (BAZEL_CMD_DOLT_REQUIRED=true) (#7226)
Advisory since #7185. On the farm it passed on every PR run based on
current main; its only failures (3) were branches predating #7219 and
reproduced a real test bug that #7219 fixed. Flip the committed flag so
CI Gate / Required enforces BAZEL_CMD_DOLT. Rollback: set it back to
"false".

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 01:04:39 -07:00
Julian Knutsen
630eae9b0c
ci(pr): share restored GOCACHE with the native init credential step (#7227)
pr-preflight-platforms' "Exercise native init gateway credential shell"
step had no GOCACHE env, so it fell back to the runner's empty default
cache and cold-compiled all of ./cmd/bd's test binary on every PR, right
after the hook-timeout step had compiled the identical binary (same tags,
same platform-default CGO) into the restored non-race cache. Job logs:
213s of a ~350s macos-latest leg, ~117s on Windows; on Linux the rerun
with a shared cache is a relink only (~8s).

- pr.yml: point that step, and the three Windows-only Go steps, at the
  restored non-race GOCACHE.
- main.yml: the macOS test leg (the only macOS non-race seeder) now runs
  warm-non-race-cache.sh, so the PR macOS legs restore the test packages
  and test-only deps they compile, not just the non-test ./cmd/bd graph.
- Policy tests: every run step in pr-preflight-platforms and
  check-doc-freshness-platforms must use the restored GOCACHE, and the
  macOS seeder must warm before it saves.

No test selection, matrix, or gate wiring changes.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 01:01:23 -07:00
Julian Knutsen
b4c76a676b
fix(gate): ordinary commands wait for maintenance operations, fairly; init waits up to 60s (#7225)
* fix(gate): ordinary commands wait out maintenance, with writer fairness

Ordinary bd commands take the workspace (and physical-root) gate SHARED and
never waited for it, so while another process held it EXCLUSIVELY — bd init
(~8s), backup restore, migrate, bootstrap — they failed at once with "a
maintenance operation is running". In shared-server mode every project
shares one physical-root gate, so another project's bd init broke every bd
command on the machine for its duration.

Ordinary commands now wait:
- up to 15s by default; BEADS_GATE_WAIT_TIMEOUT (Go duration or bare
  seconds, same convention as BEADS_INIT_GATE_TIMEOUT) overrides it, and 0
  restores fail-fast;
- one "waiting for another bd process ..." notice after 2s (shared with bd
  init's notice);
- Ctrl-C aborts the wait. A canceled wait used to fall through to the
  shared fail-open path ("continuing ungated"); it is now a hard error;
- past the bound, the same error naming the holder, the wait, and the knob.

Git hooks stay fail-fast: `bd hooks run` and the bd export/import children
the hooks spawn run with BD_GIT_HOOK=1, treat failure as a warning, and must
not stall a commit or checkout for 15s.

Writer fairness (internal/workspacegate). flock/LockFileEx give no queueing,
and acquisition is non-blocking polling, so a steady stream of overlapping
shared holders could keep a waiting exclusive acquirer (bd init) out past
its bound. A per-gate intent lock (<gate>.intent) fixes that: a waiting
exclusive acquirer whose first try is busy takes the intent lock
exclusively while it polls, and waiting shared acquirers probe it before
every attempt and wait while it is held. The exclusive waiter then waits
only for in-flight holders to drain. The intent is an OS lock, so a crashed
waiter's intent dies with it; its advisory sidecar is ignored when the lock
is free, and intent-lock errors other than "busy" degrade to the old
behavior. Fail-fast acquisitions (Wait <= 0) ignore the queue entirely, so
hooks and BEADS_GATE_WAIT_TIMEOUT=0 behave exactly as before. The intent
lock is taken only while acquiring its gate, directly before it in
AcquireAll's sorted order, so it adds no lock-order cycles.

Nested bd (bd orphans running bd close, hooks fired by git that bd runs)
would deadlock against the queue: the child queues behind an exclusive
waiter that is waiting for the child's parent. A bd process holding its
gates shared exports BEADS_GATE_SHARED_HOLDER_PID; a child whose recorded
ancestor is alive skips the queue (it cannot help the waiter anyway).

Tests: workspacegate fairness (a continuous shared stream no longer starves
an exclusive waiter; verified to fail without the queue), intent lifecycle,
stale intent from a SIGKILLed process, cancellation while queued,
InheritedSharedHold. cmd/bd: holder released after ~3s lets a command
through with one notice; past-bound error text; fail-fast modes (0, hooks);
Ctrl-C is not fail-open; a new command queues behind a waiting bd init
while a child of a shared holder does not; env parsing.
TestSharedServerConcurrent runs a late bd init of a new project during the
client workloads, so both directions are exercised against a real server.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(gate): a queued exclusive waiter may delay shared commands, never fail them

Review follow-up (B1). A doomed exclusive waiter — bd init or restore
blocked behind a long-lived shared holder such as `bd list --watch`,
`bd events tail --follow`, an open `bd edit` editor, or a beads.OpenGated
embedder — held the writer-fairness intent lock for its whole budget, so
every NEW ordinary command deferred to it and failed with "queued for
exclusive access" even though nothing held the gate exclusively. Before
the fairness change those commands ran normally.

Two bounds fix it while keeping the fairness win:
- An exclusive waiter drops the intent lock after maxIntentHold (10s)
  without getting the gate and keeps polling unqueued. In-flight ordinary
  commands drain far faster than that.
- The final attempt of a waiting shared acquisition tries the gate even
  while intent is held, so a shared acquisition fails only when the gate
  itself is held exclusively.

With the final attempt ignoring intent, AcquireAll's 1ns floor for an
exhausted budget is dead code (a Wait-0 attempt behaves the same), so it is
back to 0. queuedDetail drops an intent sidecar left by a dead process (on
Windows its removal can fail while another process is reading it).

Tests: the doomed-waiter scenario (both bounds; each fails with its bound
removed), intent lock order across a two-gate set in both directions, an
exhausted AcquireAll budget, two exclusive waiters handing off the intent,
and BEADS_GATE_SHARED_HOLDER_PID inherited through a real subprocess.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(gate): 30s shared wait, 60s init wait, accurate errors, hook pinning

Review follow-ups for the shared gate wait (S1-S5 and nits), on top of the
workspacegate fix that keeps a queued exclusive waiter from failing
ordinary commands.

- Ordinary commands wait up to 30s by default (was 15s). Queued
  maintenance operations run one after another (two inits fanned out
  across shared-server projects hold the gate back to back, 10-15s each
  under load) and a waiting command waits for all of them. Documented in
  configuration.md and CHANGELOG, including BEADS_GATE_WAIT_TIMEOUT=0 for
  scripts that relied on fail-fast.
- bd init waits up to 60s by default (was 30s). Safe only now that a
  waiting init stops queueing others after 10s.
- Error text: a shared acquisition now fails only when the gate is held
  exclusively, so "running" is accurate; in shared-server mode it says
  "the shared server (possibly from another project)", and the detail
  still names the holder when known.
- hooks.go: the hookSubprocessEnv / bd import comments listed BD_GIT_HOOK's
  consumers without the gate wait, which now reads it to stay fail-fast.
  TestHookSubprocessEnvKeepsGateFailFast pins that hook subprocesses carry
  it and therefore never wait on the gate.
- bd edit's $EDITOR (which can outlive bd by hours) no longer inherits
  BEADS_GATE_SHARED_HOLDER_PID.
- Test hygiene: the cmd/bd test binary never advertises the marker
  (in-process command tests run beside tests that spawn bd) and clears any
  inherited one in TestMain; the notice-delay test runs in 1s, not 3s.
- TestSharedServerConcurrent's late init keeps a 30s bound so the 60s
  default cannot mask a fairness regression.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 00:03:48 -07:00
Julian Knutsen
ba12ffa74e
perf(issueops): indexed dep-add cycle/hierarchy walks; pin blocked-state lookup plans; split waits-for spawner checks (#7224)
* perf(issueops): index each step of the per-edge cycle and hierarchy walks

The single-edge cycle and hierarchy probes (AddDependencyInTx, i.e. bd dep
add and apply-batch dep_add items, the domain stack's HasCycle /
ValidateBlockingHierarchy, and apply-batch's end-gate hierarchy check) joined
the recursive frontier against a UNION of dependencies and wisp_dependencies.
Dolt cannot index that derived table, so every recursion step rescanned every
matching edge: on a 50k-issue / 100k-edge database one dep add took minutes
(an apply-batch of 1 create + 2 dep adds ~10 min).

Give each dependency table its own recursive member joined directly on
d.issue_id, so each step is an index lookup, and pin the lookup plan with
JOIN_ORDER/LOOKUP_JOIN hints: without them the Dolt sql-server planner picks
a type-index scan join while table statistics are not yet available (right
after a commit), back to minutes. Same reachable set (UNION distinct merges
the members per step), same edge types, same error paths, and still one
statement per probe.

Adds a "depadd" createbatchequiv scenario with a golden digest recorded from
the pre-change code: every cycle/hierarchy probe over 17 nodes x 3 table
sets (mixed tables, every target column, external and missing ids), then 20
AddDependencyInTx calls covering cycles through both tables, hierarchy
conflicts in both directions, self, missing endpoints, type conflict,
external and non-scheduling edges.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(issueops): pin lookup joins in the batched blocked-state recompute

The batched mark/unmark UPDATEs decide membership through the scoped
should-be-blocked union. On the Dolt sql-server its planner intermittently
(depending on when the background statistics refresh lands) turned the
union's issues legs around: it drove them from an index scan of every open
issue through (is_blocked, status) and probed dependencies per row. Over an
import's few thousand uncommitted issues that is ~3 s per leg and 6-10 s per
mark/unmark statement for each 200-id chunk, where the lookup plan takes
~0.1 s; an import of 5000 issues with edges took ~28 min and came close to
the 10 s read timeout. EXPLAIN on the server engine showed the two plans;
two of three identical 2000-issue imports hit the slow one (4-5 min vs 14 s).

Pin each joined leg of the scoped union to drive from the batch's own
dependency rows and look the target/parent up by primary key
(JOIN_ORDER/LOOKUP_JOIN hints, ignored by MySQL). The SQL is otherwise
unchanged, so results are identical (createbatchequiv golden digests,
including is_blocked, pass on both engines) and the statement count is the
same. The unscoped union used by full repair and the doctor count walks
whole tables and keeps the planner's choice.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(storage): pin the hinted graph-walk and blocked-state plans with EXPLAIN

The JOIN_ORDER/LOOKUP_JOIN hints on the per-edge reachability walks and the
batched blocked-state recompute were only pinned textually: the suite passed
with them deleted or naming a wrong alias. TestGraphWalkPlansHonorJoinHints
(embedded) records the statements the production entry points send
(WouldCreateSchedulingCycleInTx, CheckBlockingHierarchyInTx,
RecomputeIsBlockedInTx) through a DBTX wrapper, EXPLAINs exactly those, and
asserts each recursive member is a lookup join keyed by the frontier node and
each joined union leg a primary-key lookup, never an is_blocked/status scan.
Deleting a hint or misspelling its alias fails it.

The test found a remaining planner trap in the ancestor walk: with
"d.type = 'parent-child'" in the join condition, the embedded planner
satisfied the wisp_dependencies member through the type index keyed by the
constant, a lookup returning every parent-child row of the table per
frontier node. The type filter now sits in each member's projection (a row
of another type yields a NULL node, which joins nothing and never equals the
goal), so issue_id is the join's only predicate. The depadd golden, recorded
from the pre-change code, still matches.

Also corrects the hint comments: MySQL honors JOIN_ORDER (the order it forces
is the wanted one) and ignores the unknown LOOKUP_JOIN with a warning.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(createbatchequiv): check run scenarios against the golden only; probe a stored parent cycle

A run scenario (depadd) drives code the create fast paths do not gate, so
its fast-vs-per-row comparison compared a body with itself. Apply it once,
check it against its golden digest (the pre-change code's answers), and fail
if the run recorded nothing.

depadd now plants a parent-child 2-cycle straight into dependencies, as
legacy or merged data can hold even though every write path refuses it, and
probes across it plus three adds touching it, so termination of the walks on
stored cycles is covered. Golden re-recorded from a93498c81 (pre-change
walks); the current code matches it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(issueops): index the waits-for gate's child lookups; waits-for bench fixture

The large-graph benchmark gains a formula-shaped fan-out
(batchfixtures.WaitsForInserts: one spawner, 1000 parent-child children,
20 waiters over every waits-for gate shape) and two shapes over it:
create+waits-for and closing a spawner child (which recomputes every waiter
through the waits-for gate).

It was not a statistics-dependent plan flip but a consistently bad plan:
each "spawner has an open/closed child" EXISTS matched both spawner columns
with one OR, which no index serves, so every correlated evaluation scanned
the edge table. On 50k issues / 100k edges, closing one child took ~39 s
per close on embedded, and on the server create+waits-for hit the 30-minute
read timeout. The gate now splits each EXISTS per spawner column (one per
dependency table and column, each served by its target-column index) and
pins the child lookup by primary key (JOIN_ORDER/LOOKUP_JOIN).
EXISTS(a OR b) is EXISTS(a) OR EXISTS(b), so the result is unchanged. The
template is shared with the unscoped full repair, where a per-row correlated
lookup is the right plan too.

Close spawner child per close: emb 39.5 s -> 3.3 s; create+waits-for emb
2.1 s -> 0.46 s, srv >30 min (read timeout) -> 0.29 s; srv close spawner
child 3.2 s (not reached before).

Adds a "waitsfor" createbatchequiv scenario, golden recorded from
a93498c81 (pre-change gate): issue and wisp spawners with issue and wisp
children, waiters in all five gate shapes, then closes, reopens, a scoped
recompute and the full repair, snapshotting every is_blocked. A mutation
dropping the wisp-spawner legs fails it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(createbatchequiv): compare dependency metadata canonicalized across engines

The embedded engine and the Dolt sql-server serialize a JSON metadata value
differently (key order, spacing), so the waitsfor golden recorded on embedded
failed on the server for its gate metadata alone. Re-serialize each
dependency row's metadata through encoding/json before comparing. Rows with
'{}' metadata are unchanged, so the other goldens keep their digests;
waitsfor is re-recorded from a93498c81 and now passes on both engines.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): auto-sync BUILD files

Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37272437653
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-04 23:43:32 -07:00
Julian Knutsen
330c3de7d5
fix(cli): guarded close replays molecule auto-close and open-children warning (#7222)
* fix(cli): guarded close replays molecule auto-close and open-children warning

mc-zndi7.75: runCloseDirectIfRevision/runCloseProxiedIfRevision bypass bd
close's batch architecture entirely (BatchCloseItem carries no per-item
ExpectedVersion), so they also bypassed two state-derived post-close
contracts that the batch route performs for every close:

1. Molecule auto-close (autoCloseCompletedMolecule / its proxied twin
   autoCloseProxiedCompletedMolecule). Without replaying it here, a guarded
   close of a molecule's last step left the molecule root stranded open
   forever.
2. The "warning: closing X with N open child issue(s) still active" line
   printed when --force waives the engine's open-children refusal.

Both routes now replay this work after a successful guarded close, exactly
as close.go's batch path and closeProxiedRunPostClose do, on both a real
close and an idempotent re-close (the proxied post-close work runs in its
own unit of work, mirroring autoCloseProxiedCompletedMolecule's existing
batch-route contract).

The third item in mc-zndi7.75 — gating on_close hook firing on Changed — is
deliberately NOT implemented here: hookIssueOperations.Close (embedded) and
recordingIssueUC.CloseIssueChecked (proxied) both fire on_close
unconditionally on success, including an idempotent re-close, by explicit,
independently pinned design in internal/storage/hook_issue_operations_test.go
("close no-op still fires close" — "Do not 'unify' this either") and
internal/storage/uow/notifying_test.go ("a re-close still reports the close"
— "THE BATCH COMPOSITIONS DISAGREE WITH THIS ONE, deliberately"). Only the
batch closers gate on Changed, to stop a replayed teardown batch from
re-running on_close once per item on every pass (ga-2yaqp.1) — a hazard this
single-id guarded close doesn't share. Reusing that gating here would
contradict both pinned invariants rather than fix a bug, so it's documented
in place and left for human review instead of changed unilaterally.

Adds CLI integration tests for both routes: guarded re-close healing a
stranded-open molecule root, and guarded --force close warning on open
children.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): sync cmd/bd BUILD.bazel for the new --if-revision close tests

make bazel-sync (bazel run //:gazelle) after adding
close_if_revision_embedded_test.go and close_if_revision_proxied_test.go.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-04 22:35:09 -07:00
Julian Knutsen
5439be97ba
feat(httpapi): wire-shape signal — wire_revision, client floor, golden shape digest, param tokens (#7182)
* feat(httpapi): wire-shape signal — wire_revision, floor header, golden digest, param tokens

S0 of the HTTP role-parity program: give a client a single place to learn
whether this build's wire shape has moved under it, and give CI a gate that
catches an undocumented move.

- GET /v0/beads/context reports wire_revision (bumps only on a non-additive
  response-shape change) and min_client_wire_revision. An optional
  Bd-Wire-Revision request header lets a client declare the revision it was
  built for; checkWireRevision refuses a declared revision below the floor
  with 400 invalid_argument/reason=wire_revision_unsupported and a
  min_wire_revision field, on every route but /healthz (wireRevisionExempt) —
  including the identity handshake, where catching the mismatch is cheapest.
  Absent header is today's behavior; malformed value is invalid_value, never
  wire_revision_unsupported. The floor lives in Server.minClientWireRevision
  (defaulting to the MinClientWireRevision constant) so a test can raise it.
- internal/httpapi/wireshape: a golden digest over every response's every
  content-type schema (including application/problem+json) across the whole
  spec, keyed by schema+member. TestWireShapeDigest fails closed when a
  member's JSON name or type changes without wire_revision bumping to match;
  regenerate with `go run ./internal/httpapi/wireshape/cmd/gendigest`.
- GET /v0/beads/issues' `sort` parameter now carries the issues.listSort
  capability token, the same convention `capabilities` entries use.
  TestNewParameterOnExistingOperationHasABehaviorToken requires every new
  parameter on an existing operation to carry one, checked against a
  write-once grandfather baseline (testdata/pretoken_parameters.json) of
  parameters that predate the rule; TestUntokenedParameterRuleFires proves
  the checker itself against a synthetic fixture.
- Docs: engdocs/design/bd-serve-v0.md gets a "The wire-shape signal" section;
  CHANGELOG.md gets an Added entry.

Live-mutated and reverted three RED demonstrations during development:
retyping api_version in the spec failed TestWireShapeDigest; short-circuiting
checkWireRevision's floor comparison failed
TestWireRevisionBelowFloorRefusesEveryEnforcedRoute; adding an untokened
query parameter to listIssues failed
TestNewParameterOnExistingOperationHasABehaviorToken. All three now pass
against the unmutated tree.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): rename issues.listSort to issues.list.sort, harden token rule

Opus review of 12c5144ca (HIGH-1, MEDIUM-5):

- Rename the capability token issues.listSort -> issues.list.sort to match
  bd-enterprise's client (httpstore/wire/handshake.go:121, DESIGN.txt S4e),
  and widen capabilityToken to accept an optional third segment.
- untokenedParameters() now requires the extracted token to be one
  Capabilities() actually serves, not just backtick-shaped text. This
  catches real false positives that existed in the spec already
  (workapi.DefaultReadyLimit, status.custom) which only shape-matched the
  old regex; they're added to the pretoken baseline as genuinely
  pre-existing untokened parameters.
- Extend the rule to request body members (new
  testdata/pretoken_request_body_members.json baseline) and exempt
  brand-new operations via a frozen testdata/pretoken_operations.json
  snapshot.
- Add TestPretokenBaselinesNeverGrow pinning all three baseline sizes so
  a future change can't hand-add an exemption instead of a token.
- Update spec prose for the widened rule scope.

go test ./internal/httpapi/... and the reviewer's pmut.py all pass as
expected: plain/bogus modes go RED, reuse (real-but-unrelated token) stays
green, matching the review's literal ask ("a token must be one advertised
by Capabilities()").

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(wireshape): close four digest-coverage gaps, add gendigest write guard

Opus review of 12c5144ca (HIGH-2, MEDIUM-4). The digest only tracked
Type/Format/Enum/Required on direct properties, missing:

- nullable: true (never read at all)
- array item shape (Issue.labels items string->integer was invisible;
  only object-shaped items got recursed into)
- additionalProperties value shape (IssueCount.groups)
- request bodies entirely (Compute() only walked responses, so
  DeleteIssuesRequest.expected_version string->integer, the other half
  of #6053, was invisible)

Also added defensive oneOf flattening alongside the existing allOf merge
(dormant today -- grep confirms this spec uses oneOf nowhere -- but no
longer silently invisible if that changes).

Extracted the golden-vs-computed comparison into wireshape.Compare(),
shared by TestWireShapeDigest and the new wireshape.SafeToWrite(), which
gendigest now calls before writing: it refuses to overwrite golden.json
with changed/removed entries unless the candidate's wire_revision is
strictly greater than the existing golden's. Pure additions always write.
TestSafeToWrite exercises both guard directions.

golden.json is regenerated. This single regen is a one-time exception to
the new guard, done by writing the digest directly rather than through
gendigest: confirmed via a throwaway diff (not checked in) that every one
of the 57 "changed" entries differs only in the NEW fields the wider walk
now populates (ItemType/ItemFormat/Nullable/AdditionalProps*) -- every
pre-existing Type/Format/Required/Enum value is byte-identical. Nothing on
the wire changed; the measurement got more precise. Bumping
CurrentWireRevision for this would have written a false "the protocol
changed here" entry into the revision history table. Going forward, any
REAL change the guard now reports uses the normal bump-then-regenerate
path, same as before.

Verified against the reviewer's mut.py: nested_type, nullable,
nullable_issue, array_items, addlprops, request_body and format all go RED
on TestWireShapeDigest, each reverted after confirming.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): make bd_version wire_revision inference advisory

Opus review of 12c5144ca (HIGH-3). The spec told a client inferring
wire_revision from bd_version to use the release cutoff >= 1.3.0, and said
nothing about what happens when that inference is wrong -- which it
demonstrably is for real builds already in the field:

- main builds up to commit b3ef65c85 report bd_version 1.2.2 (below any
  1.3.0 cutoff) while already sending the string-typed revision/
  expected_version shape (#6053).
- 1.3.0-rc.N pre-release builds sort BELOW the release cutoff 1.3.0 under
  plain string comparison despite already carrying the string shape.
- The bd-enterprise compatibility line at d3ab32773462 reports bd_version
  1.1.0 while also sending strings.

Moved the cutoff to >= 1.3.0-rc.1 (compared as semver-with-prerelease,
which at least captures the rc builds), documented both named exceptions
inline, and made the whole inference explicitly ADVISORY: a client must
accept revision/expected_version as either a string or a number whenever
wire_revision is absent (or probe directly) rather than let the inferred
value choose a decoder, and must never refuse solely because the inferred
revision is 0. Mirrored into engdocs/design/bd-serve-v0.md.

This is spec/doc prose only -- there is no server-side inference code to
change, since this server always sends wire_revision directly; the
inference is guidance for clients reading a pre-signal server's response
(this server, past builds, or an unrelated implementation).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): LOW-7/8/9/10 review follow-ups for wire-revision signal

LOW-7: WireRevisionUnsupported now also sets Problem.wire_revision (this
server's CurrentWireRevision) and Problem.bd_version on the
wire_revision_unsupported refusal, alongside the existing
min_wire_revision, so a client logging the refusal has this server's
full self-description without a second /v0/beads/context call.

LOW-8: gazelle + make bazel-sync add the BUILD.bazel entries the prior
commits were missing: wire_revision.go/wire_revision_test.go in
internal/httpapi's srcs, new BUILD.bazel files for wireshape and
wireshape/cmd/gendigest, and a testdata glob on httpapi's go_test so
the pretoken_*.json baselines are visible under `bazel test`. Verified
with `bazel build //internal/httpapi/...` and
`bazel test //internal/httpapi/... --test_env=BEADS_TEST_SKIP=dolt`
(both green) and `python3 tools/bazel/go_srcs.py --check` (clean).

LOW-9: narrowed CHANGELOG.md and engdocs/design/bd-serve-v0.md to
describe the digest's actual final coverage (format/enum/required/
nullable, array items, additionalProperties, request bodies, $ref/
allOf/oneOf recursion, the SafeToWrite revision-bump guard) and the
token rule's actual final shape (three pinned baselines, request-body
members, Capabilities()-membership validation, new-operation
exemption) instead of the original, narrower claims.

LOW-10: Bd-Wire-Revision now parses via a strict decimal-only
parseStrictNonNegativeInt (no leading '+', no leading zero except a
bare "0") instead of strconv.Atoi alone, so two different header
bytes can no longer name the same revision. New malformed-header test
cases: "+1", "01", "007".

Regenerated apigen/types.gen.go (make api-gen) for the new Problem
fields and wireshape/testdata/golden.json (go run
.../wireshape/cmd/gendigest) for the two new additive Problem members;
the latter wrote cleanly through SafeToWrite itself, since both are
pure additions.

go test ./... and bazel test //internal/httpapi/... both green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpapi): re-review LOW follow-ups — retirement test, gendigest -init, docs

1. TestRetiredRevisionsStayBelowTheFloor pins the PRESENCE guarantee the
   wire_revision property doc relies on: CurrentWireRevision >= 2 (so 0
   and 1 stay permanently retired) and MinClientWireRevision <=
   CurrentWireRevision (so this build's own floor never refuses every
   request against itself).

2. gendigest now refuses to create a golden.json that does not exist yet
   unless run with -init. Previously a missing file (moved testdata,
   wrong path, accidental delete) was indistinguishable from a real first
   run and silently wrote a fresh golden with no revision history checked
   against anything. writeGolden() is extracted out of main() so this is
   covered directly: TestWriteGoldenRequiresInitWhenMissing,
   TestWriteGoldenInitCreatesWhenMissing,
   TestWriteGoldenIgnoresInitWhenGoldenExists (the existing-golden path is
   still governed only by SafeToWrite, -init or not).

3. Narrowed the two remaining stale claims that still described the
   digest's original, narrower coverage: the wire_revision property's
   prose in openapi.v0.yaml (regenerated into apigen/types.gen.go via
   make api-gen) and wireshape/doc.go now both list nullable, array item
   shape, additionalProperties value shape, request bodies, and
   $ref/allOf/oneOf recursion, plus the new -init guard.

go test ./internal/httpapi/... and make api-check both green;
bazel test //internal/httpapi/... (httpapi, wireshape, and the new
gendigest test target) green; python3 tools/bazel/go_srcs.py --check
clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(wireshape): cover operation parameters in the wire-shape digest

The digest previously walked only response and request-body schemas, so a
type, enum, required, or removal change on one of the spec's 138 operation
parameters (123 query, 14 path, 1 header) passed the gate silently — the
open item from PR review on #7182.

Each parameter is now recorded as its own Entry, keyed by
"param:"+operationId / location+":"+name (operation-scoped, not
schema-scoped, since two operations' same-named parameter are independent
wire contracts), capturing type, array item shape, enum, required, style,
explode and default. Adding a parameter stays additive; removing,
retyping, narrowing its enum, or changing required/style/explode/default
now trips SafeToWrite exactly like a response or request-body change.

Verified against the live spec: retyping listIssues' `limit` query
parameter from integer to string now fails TestWireShapeDigest (confirmed
red, then reverted — not part of this commit). golden.json regenerated via
gendigest; the new parameter entries are a pure addition, no wire_revision
bump needed. Added TestParameterMutationsAreCaught (remove/retype/narrow-
enum against real golden entries) and updated wireshape's doc.go, the
`wire_revision` spec prose, and the CHANGELOG to describe the new
coverage.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(wireshape): review fixes — stale-golden and lowered-revision arms, effective param serialization, digest boundary

Maintainer fixup from the adopt-pr review loop (iterations 1 and 2), on
top of e17ba924f.

- goldenDrift (A, I): TestWireShapeDigest's comparison moves into a
  goldenDrift helper. An addition alongside a moved wire_revision no
  longer passes with a stale golden: it asks for a regenerate, as a
  changed or removed entry after a bump already did. Every message now
  names response/request-body members AND parameters, and TestGoldenDrift
  pins each arm. This takes only the test half of the stranded fixup
  (adopt-pr/bd-47kdge/stranded-fixup-528cd6095); its doc half, which says
  parameters are outside the digest, is false since e17ba924f and is not
  replayed (J).
- Lowered wire_revision (D): a CurrentWireRevision LOWER than the
  golden's is a wrong constant, not a stale golden, so goldenDrift says to
  restore it instead of regenerating, and SafeToWrite (so gendigest)
  refuses any write at a lower revision, whatever the entries.
- Effective style/explode (B): walkParam records style and explode with
  OpenAPI 3.0's defaults filled in (form for query/cookie, simple for
  path/header; explode defaults to style == form). Adding `explode: false`
  to a query array that relied on form's implicit explode is now a
  changed entry, while spelling out a default changes nothing.
  testdata/golden.json is regenerated with -init: 111 parameter entries
  change only in style/explode (96 to form/true, 15 to simple), 828
  entries before and after, wire_revision stays 2. The golden is new in
  this PR and unreleased, so this needs no bump.
- TestWalkParam (H): a package-internal table test
  (wireshape_internal_test.go) pins every field walkParam extracts,
  through a computeFrom(document, revision) seam that Compute now wraps:
  per-location defaults, $ref'd parameters and items, format, nullable and
  default, content parameters, and the nil returns.
  TestEffectiveSerializationIsCompared pins B end to end through Compare
  and SafeToWrite. BUILD.bazel embeds the library in the go_test, which is
  gazelle's own output.
- Required inputs and the digest boundary (C, E, F; docs only): all five
  prose surfaces (openapi.v0.yaml's wire_revision description and the
  regenerated apigen/types.gen.go, wireshape/doc.go, CHANGELOG.md,
  engdocs/design/bd-serve-v0.md) now say that a new REQUIRED request
  input is recorded as an ordinary addition and needs its own review,
  that value constraints and member-level composition are outside the
  digest, and that object and content parameters are recorded only as
  their container. bd-serve-v0.md also gains the parameter coverage it
  was missing.
- G: a brand-new operation is permanently exempt from the
  capability-token rule; its own review decides its capability story.
- Iteration 2 (nit): walkParam's comment now says that the location
  default it records as a `content` parameter's style and explode is a
  placeholder, since OpenAPI 3.0 applies style and explode only to a
  parameter described by `schema`. All 138 operation parameters in the
  spec use `schema`, so the golden does not change.

Validated: go test ./internal/httpapi/... (gms_pure_go), make api-check,
go vet for linux/windows/darwin, gofmt, git diff --check, gazelle on the
package (no drift), make bazel-sync-check, and bazel test
//internal/httpapi/wireshape/... in the local processwrapper sandbox
(2 of 2 pass). Overlay mutation probes: dropping the added-after-bump
arm, the lowered-revision arm, SafeToWrite's lowered check, or the
effective-default fill each turns the new tests red.
Iteration 2 changes only that comment; go test
./internal/httpapi/wireshape/... (gms_pure_go), go vet, gofmt and git
diff --check pass again.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-04 22:29:12 -07:00
Julian Knutsen
516e0b4bf0
ci: autofix pushes with the gastownhall-autofix App token (no more approve-and-run) (#7215)
* ci(autofix): push bot fixes with a scoped GitHub App token

github-actions[bot] pushes left PR checks action_required instead of
retriggering, and the DOCS_AUTOFIX_TOKEN fallback secret was never
configured. bazel-autofix.yml and docs-autofix.yml now mint a short-lived,
repo-scoped (contents:write) installation token for the gastownhall-autofix
App via the `autofix` environment on same-repo PRs, falling back to the
default workflow token (and its manual re-run note) when the App token
isn't available; fork PRs are unaffected and still only get the apply-recipe
comment.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(ci): job environment parses as a name or a mapping; gofmt

ciWorkflowJob.Environment was a string, so every test that loads all
workflows failed on deploy-pages-redirect.yml's mapping form
(environment: {name, url}).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-04 21:27:06 -07:00
Julian Knutsen
57eee41274
fix(cli): --if-revision outranks the reassign fence; recognize bare ErrVersionMismatch (#7216)
* fix(cli): --if-revision outranks the reassign fence; recognize bare ErrVersionMismatch

Fixes two post-merge review findings on #7203 (max review 37acca96db):

mc-zndi7.74: a lost --if-revision race on an assignee edit reported the
bd-98s5c live-claim reassign fence's policy refusal (exit 1, no machine
code) instead of precondition_failed (exit 13), because update.go,
update_proxied_server.go, assign.go and mutate_proxied_server.go each ran
an advisory pre-read fence check against their own earlier issue snapshot
before the properly-ordered guarded write underneath (which already checks
ExpectedVersion before the fence) got a chance to run. Added
ifRevisionAlreadyStale to skip the advisory check when the caller's own
pre-read is already stale against --if-revision, letting the guarded write
report the correct outcome.

mc-zndi7.78: classifyIfRevisionFailure recognized a revision mismatch only
via the typed *issueops.VersionMismatchError (errors.As). bd-enterprise's
HTTP client maps a 409 precondition_failed to the bare
issueops.ErrVersionMismatch sentinel instead, which bypassed the classifier
and surfaced as a generic error. Added an errors.Is fallback that reports
precondition_failed with expected_revision from the --if-revision flag
value and current_revision omitted (gascity tolerates the omission).

New regression tests (embedded + proxied routes) reproduce the .74 race
deterministically: claim an issue as a foreign holder to advance its
revision past a caller's stale --if-revision, then assert the guarded
write reports precondition_failed rather than falling through to the
live-claim refusal naming the holder. Both fail on an unpatched
git-archive copy of origin/main and pass on this branch.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): auto-sync BUILD files

Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37258672420
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-04 21:19:40 -07:00
Julian Knutsen
298e08c1ff
fix(test): adapt watermark-throttle test to #7044's single pre-sync read (#7219)
TestMaybeAutoBackup_WalkErrorArmsIntervalThrottle pinned the walk-error
re-arm in maybeAutoBackup by forcing runBackupExport's old post-sync
GetCurrentCommit call to fail. #7044 (874e2866f) collapsed that into a
single pre-sync CurrentCommit read, so the forced failure point no
longer exists and the test failed deterministically on every run.

Swap the failure point for one that still exists after #7044: the sync
itself succeeds (still proving fail-open — BackupDatabase gets called),
but the destination goes read-only immediately afterward, so
runBackupExport's post-sync saveBackupState (the call that would record
the new watermark) fails and never reaches disk. The only persisted
watermark write left standing is maybeAutoBackup's own walk-error
re-arm, so the test is still mutation-sensitive to removing it
(verified locally: deleting the re-arm reproduces the old failure).

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-04 21:07:11 -07:00
Rongjun GENG
3c65c99c29
ci: trigger pull_request CI on hotfix/** branches, not just release/** (#7148) (#7198)
pr.yml and pr-risk.yml (and the three advisory workflows that already
list release/** beside it -- conformance.yml, cross-version-smoke.yml,
regression.yml) trigger pull_request only on main and release/**. A PR
based on hotfix/1.3.1 (or any sibling hotfix/** line) matched neither
pattern, so it ran almost no CI: #7146, a backport to hotfix/1.3.1,
showed 17 checks -- 15 cross-version historical smokes and triage/label
jobs, 2 skipped -- no Lint, no unit shards, no embedded tier, no CI
Gate / Required, no PR Risk Gate / Required.

pr.yml's own comment states exactly why release/** is there:
release-prep and release-fix PRs are based on the release branch, not
main, and without it they run almost no CI before the tag build. A
hotfix line is in the identical position. Added hotfix/** alongside
release/** in all five files.

New TestPullRequestWorkflowsTriggerOnHotfixBranches pins hotfix/**
presence in each file's pull_request.branches list -- red before this
change (observed [main release/**] in all five), green after. The
existing TestPRRiskBazelCoverageJob equality check (pr.yml and
pr-risk.yml's pull_request triggers must match) still passes, since
both were updated identically. go vet/gofmt clean.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
2026-10-04 20:30:49 -07:00
Rongjun GENG
874e2866ff
fix(backup): anchor watermark before snapshot (#7044)
Record the commit read before BackupToDir as the newest revision guaranteed
to be in the snapshot. A commit landing during sync then remains pending for
the next backup instead of being hidden by an advanced watermark.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Rongjun GENG
2026-10-04 20:01:43 -07:00
Rongjun GENG
5db60f42ac
fix(purge): preserve transitive live dependents (#7041)
Propagate sweep protection through closed candidates so an open descendant
keeps every candidate ancestor in its protecting dependency chain. Cover the
shared helper, dry-run, real sweep, and all storage contract wirings.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Rongjun GENG
2026-10-04 20:01:35 -07:00
Bee
4524f4a428
test(cmd/bd): take the flag-reset baseline after a reset, not from live values (#7190) (#7194)
TestResetCommandFlagsRestoresEveryFlag read its expected values from the
live flag variables. The root pre-run fills some of those from config
without marking the flag Changed (doltAutoCommit, main.go:1224), so when
an in-process command ran earlier in the binary the baseline for
--dolt-auto-commit was "on", and the reset under test correctly put it
back to "". PR Core (wrapper timing) has failed on that assertion on
every fork PR since #7185.

The test now calls resetCommandFlags before taking its baseline.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-04 20:00:37 -07:00
Julian Knutsen
a93498c81f
test(kv): pin proxied kv list --prefix fast path and sibling-key e2e (bd-s5k6) (#7204)
Follow-up to #6380 (bd kv list --prefix). Both capability-discovery hops
fall back silently, and both modes re-filter in-process with
kvPairsWithPrefix, so a lost SQL-side fast path printed the same rows and
failed no existing test.

- Extract wireProxiedUOWProvider, the unit-of-work twin of
  wireStorageDecorators, so the proxied provider chain that main.go builds
  is unit-testable; main.go now calls it. No behavior change.
- TestKVListProxiedPrefixReadSurvivesProviderChain drives
  runKVListProxiedServer through that chain (hook runner on and off) over a
  prefix-capable stub use case and fails on any GetAllConfig scan;
  TestKVListProxiedPrefixAbsentCapabilityFallsBack keeps it honest.
- TestEmbeddedKV/kv_list_prefix_json and
  TestProxiedServerKV/list_prefix_json run the real command with sibling
  keys (mail.dog.* vs mail.doggerel.*) and an unmatched prefix.
- bdKVListJSON now fails on output with no JSON object instead of returning
  an empty map, so a "no pairs" check cannot pass vacuously.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 14:09:55 -07:00
Julian Knutsen
5e951a2ff3
perf(issueops): batch create/import without per-row round trips; on-demand graph checks (#7205)
* perf(storage): cut batch-create round trips (import, bulk create, apply-batch)

A multi-issue create (import, bulk create, CreateIssuesWithFullOptions) ran
~20 SQL statements per issue, and its dependency pass ran a recursive cycle
CTE per edge over the union of both dependency tables, which Dolt cannot
index — every recursion step rescanned the edge tables, so an import whose
edges form chains went superlinear (400 issues: 73 s of 82 s in that CTE).

issueops batch create, same transaction and same stored outcome:
- createBatchCache: one IN-list read of every explicit id's presence in both
  planes (replacing two COUNT probes per issue), one read of the batch's
  stored labels (each issue's new labels then land in one multi-row INSERT),
  created/label_added events buffered and written in one pass under the ids
  InsertDerivedEvent would mint (same digest, same lowest free ordinal), and
  runs of brand-new rows written with multi-row INSERTs (same ODKU clause)
  then finished in order.
- Dependency pass: wisp routing and target presence from batch reads; the
  cycle and hierarchy checks walk an in-memory copy of the edge tables,
  loaded once and kept in step with every edge the pass writes (a pair it
  already holds is re-read, since a clientFoundRows connection reports a
  matched duplicate as written).
- createBlockedRecomputeIDs seeds all created edges and expands once.

Shared paths:
- Blocked-state recompute splits ids once per call by whether they own a
  dependency row; edgeless ids (every freshly created issue) skip the two
  union UPDATEs for one plain unmark, run after the chunk's union statements
  on the first pass so the unions see what they saw before.
- appendChildrenInTx reads children per IN-list chunk, not per parent.
- ExecuteCreate hands its BatchContext to the batch body instead of having
  it re-read the same config (-6 statements per apply-batch create item).

Not changed: one version-history row per created issue (the version
contract, version_history.go), journal order and content, and the per-row
bodies, which stay the path for singular creates.

Equivalence: internal/storage/createbatchequiv runs an import scenario
(upserts, duplicate ids, wisps, generated ids, cycles, hierarchy conflicts,
dangling/external targets, stale is_blocked, comments, journal and
versioned history on) through the fast and per-row bodies on the embedded
and server engines and compares every written table and the skip reports.

Re-pinned TestLargeBatchApplyStatementCounts*: embedded 7009/14014/846 ->
6396/12790/793, server 7008/14014/846 -> 6396/12791/793 (breakdown in
pinnedEmbeddedStatementCounts).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(storage): batch-create review fixes — lazy edge graph, opt-in recompute probe, insert budget

Review of the batch-create round-trip change (S1-S3 and nits):

- The dependency pass's in-memory graph no longer loads both edge tables.
  It reads a node's outgoing edges through the issue_id index the first
  time a walk reaches it: the batch's own endpoints up front, then one
  IN-list read per BFS level. Memory and reads scale with the subgraph
  the batch's walks touch, not the edge tables, so a two-edge create on a
  large database no longer materialises every edge. A written edge whose
  source was never loaded loads that source (row included).
- The blocked-state no-edge probe runs only when a create asks for it or
  when a chunk holds at least 32 ids. Single-issue recomputes on close,
  reopen, update and dependency edits issue exactly the statements they
  did on main (the reopen sqlmock expectation is back to main's).
- Multi-row issue INSERTs are bounded by an estimated 16 MiB of bound
  text as well as 100 rows. When a multi-row statement fails, its rows are
  replayed one at a time, and the batch continues if every row lands
  instead of returning the original error.
- Nits: document the binary-collation assumption behind Go string
  equality; DisableCreateFastPathsForTest documents that it is
  process-global and non-parallel, and panics on an overlapping caller;
  drop the unreachable skip branch; explain why wisp targets need no
  presence read.

Re-pinned TestLargeBatchApplyStatementCounts* classic 40: 793 -> 786
(the +7 probes on dep-add recomputes are gone; net vs main is -60, all of
it ExecuteCreate's config reuse). 356/712 unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(storage): search the batch dependency graph from both ends

The lazy per-node load (previous commit) still walked forward from the
check's start, so on a large stored graph a single create's cycle check
read every node reachable from its target — about 800 statements per
create on a 2000-issue fixture. The cycle and hierarchy questions a batch
asks almost always end at a row the batch just created, whose only
incoming edges are the batch's own.

depGraph now answers each reachability question with a bidirectional
search: forward from the start along outgoing edges (issue_id index),
backward from the goal along incoming edges (the three target-column
indexes, read as one UNION ALL per table), always expanding the smaller
frontier and loading a level's nodes in that direction once. The batch's
endpoints are loaded in both directions up front, so a create's checks
usually finish with no further read (a chain import, too: each new
edge's goal has no incoming edge yet). Rows are keyed by (table, source,
target), so a row read from both ends counts once. An incoming read
ignores a row whose matching column is not its first set target column,
as the CTE's COALESCE does. A written pair that neither loaded end can
vouch for is re-read instead of assumed new.

On a 50k-issue / 100k-edge database, a create with two blocks edges
now runs 27 statements, against the full-scan or per-node walks before
(numbers in the review-fix summary).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(storage): hold batch create to golden digests from the pre-change code

The batch-create equivalence test compared the fast paths with the same
code's per-row bodies, so changes the fast-path switch does not gate —
the recompute seed merge, the IN-list children read, ExecuteCreate's
context reuse, the prepare/finish split — were checked only by older
contract tests. Its batch was also too small to cross a chunk boundary
and never set RejectStaleUpserts.

createbatchequiv now runs four scenarios, each through the fast and
per-row bodies, and checks both against a per-table row count and
SHA-256 recorded from ebe3b6bcb (before the fast paths existed;
CREATEBATCHEQUIV_GOLDEN_DIR re-records them):
- small: the original scenario;
- import458 and import458-reject-stale: the review's 458-issue batch
  (every chunk boundary, wisps and cross-plane edges, cycles, hierarchy
  conflicts, duplicate ids across deferred runs, generated and
  hierarchical ids, stale upserts rejected and accepted), plus new edges
  whose blocked-state fallout reaches stored rows outside the batch (a
  spawner's waiter, an upserted parent's existing child);
- apply: an apply-batch through ExecuteCreate, dep adds and an update.

The recorded digests match on the embedded and server engines (the
server test now sets the fixture's prefix, so generated ids agree).
Dropping the spawner-waiter seeding is invisible to the fast-vs-per-row
comparison and caught by the golden check.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(storage): large-graph create benchmark

batchbench measures creates against a database already holding a large
dependency graph (batchfixtures.LargeGraphInserts: N issues, ~2N
acyclic edges with wide reachable sets and a deep parent-child forest,
committed as a Dolt version): ExecuteCreate under an existing parent with
an existing blocker, a store create with two blocks edges, and an
apply-batch of creates each with a parent-child and a blocks dep_add.
It logs wall time, statements and Go allocation per shape on the
embedded and server engines. Opt in with BEADS_BENCH_LARGE_GRAPH=<issues>
(and BEADS_BENCH_LARGE_GRAPH_CREATES); it asserts nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(storage): read incoming dependency edges one target column at a time

The bidirectional graph search read a node's incoming edges with one
UNION ALL over the three target columns per table. Dolt plans that as a
scan: 0.7 s per read on a 100k-edge table, where each leg alone is a 1 ms
lookup on its own index (EXPLAIN shows IndexedTableAccess per leg). Issue
the three legs as separate reads into one row set; a row matched through
two columns still counts once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(storage): run the 458-issue equivalence scenarios outside the race lane

The 458-issue golden scenarios took the embedded tier's shard over its
20-minute limit under -race (the in-process engine's own instrumentation;
about 35 s each without it). They move to
TestCreateBatchFastPathsMatchPerRowLarge_Embedded, skipped under -race and
added to nightly.yml's non-race embedded step, following the precedent of
the 712-item statement-count and 1000-item wall-clock cases. The light
scenarios (small, apply) stay in TestCreateBatchFastPathsMatchPerRow_Embedded
on every lane. The server backend (built without -race in its full
suite) runs both: TestCreateBatchFastPathsMatchPerRow_Dolt and, as its own
top-level test so the shard manifest's hash places it on another shard
(about four minutes there), TestCreateBatchFastPathsMatchPerRowLarge_Dolt.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 13:56:12 -07:00
Julian Knutsen
37acca96db
feat(cli): add --if-revision compare-and-swap guard to update/close/assign/delete (#7203)
* feat(cli): add --if-revision compare-and-swap guard to update/close/assign/delete

Adds --if-revision <int64> to `bd update`, `bd close`, `bd assign` and
`bd delete`, reusing the existing opaque row_lock token published as
IssueDetails.Revision and the library's own ExpectedVersion compare-and-swap
support. A mismatch writes nothing and exits 13 (ExitGuardMismatch); with
--json the stderr envelope carries integer expected_revision/current_revision
fields deployed callers decode directly, never a quoted string. An
unsupported backend reports conditional_write_unsupported and exits 1
instead of silently dropping the guard. The guard composes with
--if-assignee/--if-status (every guard must hold) and is refused alongside
label or parent edits and alongside more than one id, matching the existing
--if-assignee rules. `bd assign` is newly routed through the checked
Lifecycle path to gain CAS support it previously had none of at all.

Implements Item 4 (compare-and-swap through the CLI, gastownhall/beads#4682)
from the BEADS-JOURNAL-PLAN.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore(bazel): regenerate cmd/bd BUILD.bazel for if_revision/close_if_revision

gazelle hadn't picked up if_revision.go, close_if_revision.go,
if_revision_test.go, and if_revision_embedded_test.go, so Bazel's bd_lib
and bd_test targets failed to build with "undefined: parseIfRevisionFlag"
and friends while plain `go build`/`go test` worked fine. Ran `make
bazel-sync` (bazel run //:gazelle; go_srcs.py; bazel mod tidy) to pick up
the new files; only cmd/bd/BUILD.bazel's srcs lists changed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(cli): update update-guard parity strings for --if-revision

TestParityUpdateGuardsRequireFieldUpdate and TestParityUpdateGuardsRejectClaim
pinned the exact stderr text of update.go's pre-write guard short-circuits.
Adding --if-revision to those guard messages (update.go/update_input.go)
changed "--if-assignee/--if-status" to "--if-assignee/--if-status/--if-revision"
in both, which this PR's own change to the guard text had not updated here —
caught by CI's dolt-server tier, not by the targeted if_revision* test runs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-04 13:38:01 -07:00
Julian Knutsen
fa3d50eb19
ci(bazel): zstd for the anonymous fork cache when it advertises it (rbe-west E4) (#7202)
* ci(bazel): zstd cache transfers for the anonymous fork cache only, behind RBE_CACHE_ZSTD (E4)

E4 of the rbe-west egress reduction (design A.5), reworked for the
read-only zstd server shape. rbe-west's trusted MAIN and OSS schedulers
(:443 to :50052) and rbe-fork (:8444) advertise no compressor, and Bazel
refuses a remote that does not advertise zstd. So only the anonymous fork
cache (rbe-cache :8443, nativelink-anon, which keeps zstd) may ask for it.

- **`write-bazelrc.sh`:** no `--remote_cache_compression` on the trusted
  remote-exec or rbe-fork paths. In mode cache, with `RBE_CACHE_ZSTD=1`, it
  adds `build:fork-cache --remote_cache_compression`. Empty or 0 adds
  nothing; another value warns and stays off, rather than failing every
  cache lane.
- **`bazel.yml`:** every setup-bazel caller passes
  `RBE_CACHE_ZSTD: ${{ needs.rbe.outputs.mode == 'cache' &&
  vars.RBE_CACHE_ZSTD || '' }}`, so only cache runs see the variable.
  Merging changes nothing, and rollback is the variable, with no revert.
- **`.bazelrc`:** fork-cache no longer sets the flag; a comment points at
  the gate.

Tests:
- `TestBazelCacheModeReachesTheRC` evaluates `vars.` right-hand sides and
  simulates `RBE_CACHE_ZSTD=1`. Mode cache's rc carries exactly the
  fork-cache zstd line; the remote, fork-ro, fork-rw and local modes carry
  no compression at all.
- `TestSetupBazelRCWriter` runs `RBE_CACHE_ZSTD` = 1, 0, empty and a bad
  value in mode cache, and remote-exec with 1 (never compressed).
  `TestSetupBazelRCWriterForkMode` pins rbe-fork's exact lines with
  `RBE_CACHE_ZSTD=1`.
- The bazel.yml setup-env pin includes `RBE_CACHE_ZSTD`.
- `checkBazelrcForkCache` rejects the flag in any `.bazelrc` config, with
  fork-cache and remote-exec fixtures.
- New `TestBazelRemoteCacheCompressionOnlyForkCache` scans `.bazelrc`, the
  workflows and the local action files. The flag must appear once: the
  writer's fork-cache echo.

**Merge order:** after rbe-west's anonymous process advertises zstd. Even
then the flag stays off until `RBE_CACHE_ZSTD=1` is set. If rbe-cache stops
advertising zstd, fork-cache's fallback flags make lanes build locally.

**Action keys:** unchanged; the flag only changes ByteStream resource
names (`compressed-blobs/zstd/`).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): zstd for the fork cache by a GetCapabilities probe, not a repository variable (E4)

Review finding 3: GitHub gives fork pull_request runs no repository
variables (bazel.yml: forks "see no secrets and no vars"), so
vars.RBE_CACHE_ZSTD was always empty exactly where the switch matters. Fork
run 37220594372 logs RBE_VAR_ON: false. The variable could never turn zstd
on for fork-cache clients, and once on it could not follow rbe-cache
rolling zstd back.

- New setup-bazel/cache-zstd-probe.sh, a byte copy of gascity's
  tools/rbe/cache-zstd-probe.sh: one unary REAPI GetCapabilities call
  (instance oss) to rbe-cache :8443 over curl's HTTP/2, decoded with
  python3's standard library. It exits 0 only when
  cache_capabilities.supported_compressors lists ZSTD, the field Bazel
  checks. Any other result exits nonzero: not listed, an HTTP or gRPC
  error, a refused connection, a timeout (connect 3s, total 5s), an
  unparsable answer, or no curl or python3. It writes nothing to stdout
  and one verdict line to stderr. No new action and no new dependency.
- write-bazelrc.sh: in mode cache, `if bash .../cache-zstd-probe.sh` adds
  build:fork-cache --remote_cache_compression. The probe never fails the
  script. RBE_CACHE_ZSTD and its bad-value warning are gone.
- bazel.yml: the ten RBE_CACHE_ZSTD env lines are gone (back to the
  pre-E4 setup env).

Tests:
- New TestCacheZstdProbe runs the real script against a TLS HTTP/2
  stand-in for rbe-cache, which checks the exact request. Passes: zstd
  advertised (packed, and unpacked among others). Fails: rbe-cache's real
  2026-10-04 answer (no compressors), deflate only, zstd for batch updates
  only, zstd outside cache_capabilities, a gRPC error trailer, no
  grpc-status, trailers-only UNIMPLEMENTED, HTTP 502, a compressed or
  truncated message, a truncated field, not gRPC, a timeout, a refused
  connection, an untrusted certificate. It also pins the probe's endpoint
  and instance to .bazelrc's fork-cache.
- TestSetupBazelRCWriter: the fork cache gets exactly the zstd line after
  --config=fork-cache against a zstd stand-in, and none against the live
  answer or a refused port. Remote-exec and rbe-fork, with a zstd
  stand-in, never probe and never carry compression.
- TestBazelCacheModeReachesTheRC again rejects any vars. in setup-bazel's
  env, and no mode's rc carries compression (the probe is refused there).
- The writer harnesses default RBE_CACHE_PROBE_URL to a refused loopback
  port, so no test reaches rbe-cache.

Live check, 2026-10-04: rbe-cache answers without supported_compressors,
so the writer's mode cache rc has no flag until the E1 cutover.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 12:26:54 -07:00
Steve Yegge
c4305ff01e
kv: add 'bd kv list --prefix' — SQL-side key-prefix filter for scoped reads (wy-1rrwt8) (#6380)
* wy-1rrwt8: add 'bd kv list --prefix' — SQL-side key-prefix filter for scoped kv reads

Downstream mail transports read the whole kv store (3.4MB, ~1988 mail.*
envelopes on the wyvern fleet) for every inbox render. This adds a --prefix
flag to 'bd kv list' that pushes the filter into SQL on stores that support
it, via an optional domain.ConfigPrefixReader interface discovered by type
assertion — no existing storage interface or test double changes shape, and
every path falls back to GetAllConfig plus in-process filtering.

- issueops: ConfigPrefixLikeQuery + ConfigPrefixPattern (LIKE with explicit
  ESCAPE '!' — the engine does not honor backslash as a default escape) and
  GetConfigByPrefixInTx.
- domain: ConfigPrefixReader optional interface; use-case impl delegates to
  the repo fast path or filters GetAllConfig.
- domain/db, dolt, embeddeddolt: SQL-side GetConfigByPrefix.
- telemetry: instrumented passthrough preserving the capability.
- cmd/bd: --prefix on kv list, both classic and proxied-server paths;
  kvPairsWithPrefix re-checks on every path so output is identical whichever
  path served the read.

* build(bazel): list the PR's new test files in go_test srcs (make bazel-sync)

kv_prefix_test.go (cmd/bd) and config_prefix_test.go
(internal/storage/issueops) are new, untagged test files. Both packages'
BUILD.bazel files were generated on main after the PR's commit was written,
and the rebase onto main did not re-run gazelle, so neither go_test listed
them: bd_test and issueops_test reported PASSED with "no tests to run" for
TestKVPairsWithPrefix and TestConfigPrefixPattern, and bazel.yml's "BUILD
files in sync" step fails on that drift in the required bazel-test lane.
Generated by `make bazel-sync`, which rewrote only these two rows. Both files
import only the standard library, so no deps change.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(kv): engage the --prefix SQL fast path through the decorator chain

Maintainer review fixup for PR #6380.

The SQL push-down this PR exists to deliver was unreachable on bd's default
direct/embedded path. `bd kv list --prefix` asserted `store.(configPrefixReader)`
on the outermost decorator, but wireStorageDecorators unconditionally interposes
externaldeps.Store and (hooks on, the default) HookFiringStore, both of which
embed the storage.DoltStorage *interface* — and GetConfigByPrefix is
deliberately not declared there. The assertion could never succeed, so every
direct-mode prefix read silently took the GetAllConfig full-table fallback.
Output was identical, so nothing errored, logged, or failed a test, while the
flag help and command Long text asserted the opposite.

This carries the prior review lineage's fixup 9ca3caeb9, stranded when the PR
was relaunched, plus what this review iteration found missing from it:

- cmd/bd: discover the capability through storage.UnwrapStore, the repo's
  documented idiom for optional-interface assertions (~50 existing call
  sites), in configPrefixReaderFor. The direct-mode read itself is extracted
  as readKVListConfig so the regression test drives the command's own read:
  with only the helper under test, reverting the call site to the bare
  `store.(configPrefixReader)` assertion still left the whole suite green.
- cmd/bd: chain-level regression test over wireStorageDecorators covering all
  four telemetry/hooks configurations, plus the negative case proving a store
  without the capability is still reported incapable. The stub counts full
  reads, so a fallback fails the test even though it returns the same rows.
  Verified non-vacuous: a bare assertion in the helper or in
  readKVListConfig, a disabled fast path, a dropped kv. prefix, or a method
  added only to the mirror interface each fail all four configurations; a
  peel that stops at InstrumentedStorage fails the telemetry-on negative case.
- dolt, embeddeddolt, telemetry: compile-time pins against
  domain.ConfigPrefixReader, so method-set drift on any implementation is a
  build error instead of a silent reversion to full scans. The cmd/bd
  mirror's own pin checks one direction only (a method added just to the
  mirror still compiles); its comment now says so and names the test that
  covers the other direction.
- domain/db: delegate GetConfigByPrefix to issueops.GetConfigByPrefixInTx like
  the sibling stores, instead of a second scan loop over the shared query
  constant.
- telemetry: record the deliberate trade — peel-to-raw keeps the standard
  idiom and leaves the instrumented forwarder off the cmd/bd path.
- bazel: add //internal/storage/domain to the telemetry_test deps (make
  bazel-sync); without it the telemetry pin's import fails strict deps.

Co-authored-by: dog <dog@ghosttrack.com>
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: dog <dog@ghosttrack.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:12:58 -07:00
Julian Knutsen
2f1e8be3ca
feat(backends): registry Remote + OpenWith options; beads.OpenBestAvailableWith for embedders (#7188)
* feat(backends): registry Remote + optional OpenWith(ctx, dir, OpenOptions)

Slice S1: backends.Backend gains Remote bool and an optional
OpenWith(ctx, beadsDir, OpenOptions{Credential, HTTPClient, UserAgent}) so a
registered extension backend can honor a per-open credential instead of
dialing with a process-global client — the motivating case is one embedder
process serving many workspaces with distinct credentials. Open keeps
working unmodified for every existing backend: Backend.OpenWithOptions calls
OpenWith when present, otherwise falls back to Open and ignores
HTTPClient/UserAgent, but refuses (ErrCredentialWithoutOpenWith) rather than
silently dropping a non-nil Credential.

Public beads.OpenBestAvailableWith(ctx, beadsDir, opts) carries this through
the SDK surface (mirrored in beads_nocgo.go and aliased in backend/backend.go
and beads.go); OpenBestAvailable is now that call with a zero OpenOptions,
identical to its prior behavior for Dolt, embedded Dolt, and every registered
backend. The plain-Dolt path (no registered backend) applies the same
Credential refusal, since Dolt has no per-open credential seam either.

doltserver.ResolvePhysicalRoots now checks backends.IsRemote(cfg.GetBackend())
before every Dolt-mode branch, including the shared-server check (which reads
only env/config.yaml and never consults cfg) — without this ordering, a global
BEADS_DOLT_SHARED_SERVER=1 would hijack a remote-backend workspace into
reporting a bogus local Dolt root it will never open.

Tests: backends_test new file open_with_test.go covers OpenWithOptions
dispatch/back-compat/Credential refusal and Remote/IsRemote; root package
open_backend_selection_test.go adds a fake OpenWith registrant proving
dispatch and option pass-through through OpenBestAvailableWith, zero-options
back-compat for every path (registered w/ and w/o OpenWith, plain Dolt), and
the Credential-without-OpenWith refusal on both paths; doltserver
physical_root_test.go adds the registered-remote-backend case including the
shared-server-override ordering regression test. Verified under both cgo and
nocgo builds. Live-mutated each new guard (the Credential refusal, the
OpenWith dispatch, the ResolvePhysicalRoots remote-backend case, and the
plain-Dolt Credential refusal) one at a time and confirmed RED before
reverting to GREEN.

No HTTP client code added; that is a later slice.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fixup(backends): address Opus review findings on OpenWith (M1-M3, L1-L2)

- M1: frame cmd/bd store-open errors for remote backends as unreachable
  server failures instead of Dolt-shaped "failed to open database"
  advice (openStoreErrorMessage, backendNameForErrorFraming), with tests.
- M2: rewrite the OpenBestAvailable/OpenBestAvailableWith back-compat
  subtest to assert fixed expected outcomes (error identity/message,
  returned store behavior) now that the two functions are identical,
  instead of comparing them to each other.
- M3: document and enforce the OpenWith contract: an implementation that
  receives a non-nil Credential it cannot type-assert to its expected
  type must refuse with the new ErrUnsupportedCredential sentinel,
  verified with a fake registrant.
- L1: document that a nil OpenOptions.Credential means the backend's
  default/ambient auth, and that multi-tenant embedders must pass a
  non-nil Credential (RequireCredential deferred to S2).
- L2: fail closed on HTTPClient/UserAgent passed to a backend without
  OpenWith, mirroring the existing Credential refusal, via new
  ErrHTTPClientWithoutOpenWith/ErrUserAgentWithoutOpenWith sentinels.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): auto-sync BUILD files

Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37195230844
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.

* fix(backends): keep the store-open error chain through the remote framing

Maintainer review fixes for the adopted PR (iteration 1).

openStoreErrorMessage returned a string, so ensureStoreActiveWithContext
rebuilt its error from text (%s) where base wrapped the open error with %w,
and the chain ended at the CLI's framing. Prime tells a store-open timeout
apart from a generic "storage unavailable" only through
errors.Is(err, context.DeadlineExceeded) on what ensureStoreActiveWithContext
returns (cmd/bd/prime.go), so the flattening silently downgraded that
diagnostic for every backend, Dolt included, not just the new Remote ones.

Replace the helper with openStoreError, which returns an error wrapping err
with %w on both arms. The message text is unchanged: "failed to open
database: <err>" for Dolt and any registered backend that is not Remote,
"failed to reach remote backend %q: <err>" for a Remote one. direct_mode.go
wraps the result with %w ahead of its Hint line; main.go keeps printing it
through HandleError("%v", ...), which only formats.

Tests:

- TestOpenStoreErrorFramesRemoteBackends (renamed from
  TestOpenStoreErrorMessageFramesRemoteBackends) now also asserts errors.Is
  on every arm, and the exact original text for the non-Remote arms.
- TestEnsureStoreActiveKeepsOpenErrorChain drives the real
  ensureStoreActiveWithContext against a registered backend whose open
  outlives its context, for a local and a Remote backend, and requires
  errors.Is(err, context.DeadlineExceeded) plus the framing prefix and the
  Hint line.
- TestFormatMemoriesForPrimeTimesOutOpeningRegisteredBackend is the
  unstubbed twin of TestFormatMemoriesForPrimeTimesOutOpeningStore and
  requires the timeout banner, not the "storage unavailable" one.

Measured at this head: turning direct_mode.go's outer %w into %v reddens
both chain-test rows and the prime test; turning the helper's %w into %v
reddens the framing test as well.

Non-gating cleanup in the same area:

- beads_cgo.go, beads_nocgo.go: OpenBestAvailableWith's doc and dispatch
  comments said only a non-nil Credential is refused. The code also refuses
  a non-nil HTTPClient and a non-empty UserAgent, each with its own
  ErrXWithoutOpenWith sentinel.
- internal/doltserver/physical_root.go: the Mode doc lists "remote-backend",
  and the RemoteBackend doc covers a registered Remote backend as well as a
  Dolt server on a non-local host.
- open_backend_selection_test.go: the plain-Dolt refusal test is a table
  over Credential, HTTPClient and UserAgent (renamed
  TestOpenBestAvailableWithRefusesPerOpenOptionsOnPlainDolt). Disabling the
  nocgo HTTPClient arm or the cgo UserAgent arm reddens only that row. The
  zero-options test's comment had the delegation backwards and pointed at
  beads.go; OpenBestAvailable delegates to OpenBestAvailableWith in
  beads_cgo.go and beads_nocgo.go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
2026-10-04 09:34:36 -07:00
Julian Knutsen
4cf36b0776
fix(doltserver): auto-started dolt must own its port; move a taken bd-chosen port (#7201)
* refactor(doltserver): share dolt's startup-output watcher with the plain server launcher

Move the ready-line / port-in-use watcher that the proxied DoltServer uses
(startupWatch, doltReadyLine, doltPortInUseTexts, logLevelEmitsReadyLine,
ErrPortInUse) into internal/doltserver as StartupWatch and friends, so the
plain server-mode Start can read the same signals. dbproxy/server already
imports internal/doltserver, so it now uses them from there; its
ErrPortInUse is the same error value. No behavior change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(doltserver): prove the auto-started dolt owns its port; move a taken ephemeral port

The plain server-mode Start (bd init --server, auto-start, bd dolt start)
had the same blind readiness check the managed-local proxied backend had
before #7184, plus a blind exit check:

- It picked an ephemeral port, closed the probe listener and launched dolt,
  then slept 200ms and asked whether the child was still alive. dolt only
  reaches its port check after its own startup, which under load takes
  longer than that, so a lost port was never caught there. Worse, the child
  had been Release()d unreaped, and a zombie still answers kill(pid, 0), so
  an exit was invisible for the rest of the bd process.
- It then waited for any MySQL greeting on the port. If another process
  took the port in between, either it was mute (a plain listener) and Start
  ran out its ready timeout -- the direct-local-server auto-start timeout seen
  under port churn -- or it was another dolt, and Start reported success
  against a server it did not start.

Now each attempt launches the child with a goroutine that waits on (and
reaps) it, and awaitOwnedListener accepts a greeting only once the
listener is shown to be the child's: dolt's ready line in the child's
output when its log level emits it (debug), or else the listening socket
belonging to the child's process tree (Linux, via /proc/net/tcp and
/proc/<pid>/fd). The child's output is read from the server log past the
offset it started at, through the StartupWatch shared with dbproxy, so
"Port N already in use." (printed at log_level warning too) ends the
attempt at once as ErrPortInUse, and a child exit ends it as a failure.

Port policy follows #7184's contract: only a port bd chose itself moves.
An ephemeral port that is lost is retried on a fresh one (up to the
existing 10 attempts); an explicitly configured port (env, config.yaml,
metadata, dolt config) stays pinned and fails with ErrPortInUse and a
remedy. Nothing operator-owned is rewritten: the generated
dolt-server-config.yaml is bd's own state file and was already rendered
per attempt. Where ownership cannot be proven (non-Linux, quiet log
level) the greeting decides as before, but the child's exit and
port-in-use report still end the wait.

Tests: a fake dolt (this test binary, behind a PATH shim) that spends
700ms starting up and then binds like dolt does. A foreign greeter on the
port is not accepted (ErrPortInUse); the child's own listener is; a child
exit is reported promptly; and the real Start, handed a taken first
ephemeral port, comes up on a fresh one (fails before this change: Start
adopted the foreign port).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(doltserver): isolate user config in the DefaultConfig tests

Four DefaultConfig tests failed under -tags integration on a host whose
user config sets dolt.shared-server: true. The integration TestMain, unlike
the plain one, does not redirect HOME, so:
- TestDefaultConfigPrecedenceChain and TestDefaultConfigPortSource call
  config.Initialize, which merged ~/.beads/config.yaml.
- TestDefaultConfigReturnsZeroForStandalone and
  TestDefaultConfigPortFileTakesPrecedence read whatever config an earlier
  test had loaded from the real HOME.
Shared mode then sent DefaultConfig to the shared server's state dir and
its port file ("port=3308 source=port_file, want 5002/config_yaml").

isolateUserConfig points HOME, USERPROFILE and XDG_CONFIG_HOME at an empty
dir, the way TestDefaultConfig_NonzeroPortAlwaysHasSource already does.
It also resets the loaded config before the test and again after it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(doltserver): auto-start review fixes: record the server at launch, sharper ownership, port policy by source

Follow-up to 0acb1041b, addressing review.

B1: write the PID and port files again right after each launch, as before
that commit, and remove them on every failure path (port conflict,
not-ready, remotesapi). Previously a bd interrupted during the readiness
wait left a dolt holding the database lock that nothing could find. The
next start, bd dolt stop and killall all saw nothing. Both files carry the
real port, so IsRunning never stops a starting server as "port unknown".

S1: a child gone from /proc is known not to own the port; it is no longer
"unknown", which would let a foreign greeting through. The child's log is
re-read on every check, not only after its exit has been observed, because
the reaper goroutine can lag the death.

S2: if no listener on the port is listed although one just answered, /proc
is incomplete (WSL1, some sandboxes, forwarding). That is now "unknown"
(fallback to the greeting), not "foreign". Also "unknown": a /proc that is
not this PID namespace's.

S3:
- bd's port-file port moves like an ephemeral one outside shared-server
  mode, since bd chose it (GH#4052).
- Operator-configured ports and the shared-server port stay pinned.
- The pinned error now says where the port came from (env, config.yaml,
  dolt config, metadata, --server-port, shared default, shared port file)
  instead of always "configured explicitly".

S4: a killed child is waited for (up to 5s) before the next attempt or the
return, so it has released the database lock. Its process group is killed
too (unix), which reaches a dolt behind a non-exec wrapper.

S5: when /proc proves the listener belongs to another process, the wait
returns ErrPortInUse at once. It no longer sits out a slow child, which
could outlast the timeout and turn a movable port into a hard failure.

NITs:
- Debug-mode timeout names the missing ready line and the way out.
- The log tail restarts if the log is truncated under it.
- waitForReady is removed; its probe tests now drive awaitOwnedListener.
- Stale daemonization comment fixed in cmd/bd.
- CONFIG_PROC_CHILDREN caveat documented.

Tests:
- fake-/proc ownership cases: owned, foreign, child gone, no listener row,
  unreadable fd table;
- /proc/net/tcp parsing;
- proven-foreign ends the wait at once;
- port-in-use in the log counts before the exit is seen;
- ready-line proof, with and without the line;
- Start records PID+port while waiting;
- Start removes them after a failed start;
- pinned env port: one launch, ErrPortInUse, source in the message;
- port-file port moves;
- remedy wording per source.
The Start tests now isolate the user config instead of skipping.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(doltserver): de-flake two startup tests under load

- TestStart_RecoversWhenEphemeralPortIsTaken: on Linux, /proc proves the
  taken port foreign as soon as the greeting arrives, so the first child
  can be killed before the fake records its launch. Assert two port
  allocations and the final launch instead of the first launch.
- TestAwaitOwnedListener_ReadyLine: give the "no ready line" case 8s, so
  the fake has time to start and greet on a loaded host.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 09:22:03 -07:00
Julian Knutsen
f2eb56a0dd
fix(init): wait up to 30s for the workspace gate so concurrent shared-server inits queue (#7200)
In shared-server mode every project's physical root is the one shared dolt
data dir, and bd init holds that dir's gate EXCLUSIVELY for its whole run
(~8s). It waited only exclusiveGateWait (5s) for it, so two concurrent
`bd init --shared-server` runs in different projects, or an init during
another project's long command, failed with "bd init refuses to run over
live bd activity".

acquireInitMutationGate now uses its own acquisition options:
- Wait: 30s by default, overridable with BEADS_INIT_GATE_TIMEOUT (Go
  duration or bare seconds, same convention as BEADS_PRIME_TIMEOUT and
  BEADS_FSCK_TIMEOUT); ctx cancellation still aborts the wait.
- One stderr notice ("waiting for another bd process on the shared
  server (...)...") once the wait passes 2s, instead of immediately.
- Past the bound, the same refusal, now naming the budget and the knob.

Exclusivity is unchanged. The 5s exclusiveGateWait is also used by
backup restore, bootstrap, migrate and the backup lock; those keep it, so
the change is scoped to init via acquireExclusiveWorkspaceGatesWithOptions.

TestSharedServerConcurrent goes back to fully concurrent inits (manual runs
with many dirs raise BEADS_INIT_GATE_TIMEOUT to match). New unit tests: a
holder released after 6s lets init through with one notice; a holder past
the bound gives the clear ErrBusy error; cancellation; env parsing.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 09:16:33 -07:00
Wilhelm Uschtrin
721e42e680
fix(notion): make the query page bound configurable and honour Retry-After (#6006)
* fix(notion): make the query page bound configurable and honour Retry-After

QueryDataSource paginates up to maxQueryPages (50) x maxPageSize (100), so a
data source larger than 5000 rows cannot be synced at all: the call returns an
error rather than partial results, and because every CLI narrowing flag is
applied after it returns, no flag works around it. Sync dies in both directions
at once. Refs #6004.

Two changes, and the second is what makes the first safe:

- Client.MaxQueryPages / WithMaxQueryPages lets a caller raise the bound for a
  large data source. Default behaviour is unchanged when unset. The error at the
  bound now names the ceiling in rows as well as pages, since rows is the number
  a caller can compare against their own data source.

- doRequest now retries 429 and 529 honouring Retry-After, with exponential
  fallback clamped at 30s. Notion enforces ~3 requests/second per connection, so
  raising the page bound without this would trade a clear failure for an
  intermittent one. Other 5xx retry only for GET/DELETE — replaying a POST that
  may already have been applied server-side is how duplicates get made.

The request body is now marshalled once and the reader rebuilt per attempt; a
retry cannot reuse a drained body.

Tests cover the configured bound, the unchanged default, Retry-After being
obeyed exactly, GET retrying on 5xx, and POST not retrying. Verified they bite:
forcing maxRequestAttempts to 1 fails exactly the two retry tests, and ignoring
the configured bound fails exactly the bound test.

make build and make fmt-check pass; internal/notion and internal/tracker pass
with -tags gms_pure_go. golangci-lint was not run locally (not installed on this
machine); CI will cover it.

* fix(notion): use US spelling so misspell passes

The repo pins misspell to locale: US, and CI flagged "honouring" in
internal/notion/client.go. Corrected there, and in the two occurrences in
client_test.go - those are not linted today (run: tests: false) but leaving
British spellings in place only plants a failure for whenever that changes.

Verified with misspell v0.8.0 -locale US: the pre-fix content reports both
occurrences, the fixed content reports none. internal/notion tests pass with
-tags gms_pure_go, including the renamed test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4YFBBWN33GZZ2BDxWRvjT

* fix(notion): address review — 529 verb gate, ctx-aware retry wait, CLI wiring

MAJOR 1. retryableStatus returned early on 429 *or* 529 before the verb gate, so
529 was retryable for every verb — including CreatePage and CreateDatabase. A
529 can come back after Notion's edge already handed the write to the origin, so
a replay creates a second row for one bd issue, and ensureRemoteIndex is
last-write-wins on the bd ID, so nothing surfaces it. 429 stays verb-agnostic
(Notion rejects rate-limited requests before processing them); 529 now sits
under the GET/DELETE gate with the other 5xx. Three tests pin the boundary: a
creating POST makes one attempt on 529, a GET retries on 529, and a creating
POST still retries on 429.

MAJOR 2. The retry wait polled ctx.Done() with a default: and then slept, so
cancellation was ignored for up to maxRetryDelay per attempt — and
QueryDataSource pays that per page, scaling with the very bound this change
lets callers raise. Replaced with wait(ctx, d): a blocking select over
ctx.Done() and a channel from a c.after hook, time.After by default.

MAJOR 3. The bound is now reachable from the CLI. Tracker.Init reads
notion.max_query_pages / NOTION_MAX_QUERY_PAGES and threads it into the client;
unset keeps the client's own default, and a value that is set but unusable is an
error rather than a silent fallback — the operator sets that key precisely
because sync is already failing on the bound. The pagination error names those
levers instead of a Go method a CLI user cannot call.

Also: assert every attempt carries a non-empty body (a drained reader would send
an empty second one, and nothing noticed); drop the unreachable lastErr branch
by breaking out of the loop; pin retry exhaustion at exactly maxRequestAttempts
with the last attempt's error; table-test parseRetryAfter across the nine
interesting inputs; name the 529 literal; hardcode 50 in the default-bound test
so lowering the default fails it; atomic counters in tests.

Every new test verified to bite by reverting each fix in turn: each mutation
fails exactly the tests naming it, against a clean baseline.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(notion): refuse a Retry-After longer than the client will wait

Separate commit because this one is a judgement call, not a defect fix — drop it
on its own if you would rather keep clamp-and-retry.

With Retry-After: 3600 the old path clamped to 30s and retried four times, so a
server asking for an hour got four more requests inside two minutes — the window
it told us to stay out of, and the way that window gets extended. Now the call
returns immediately, naming the delay the server asked for, and the caller
decides.

A Retry-After equal to maxRetryDelay is still honoured and waited out; only a
longer one is refused. The underlying API error is wrapped, so nothing is lost
to a caller inspecting it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(notion): key retry on side-effect safety, jitter own backoff, honour both Retry-After forms

Maintainer review fixes for PR #6006 (review iterations 1 and 2). All changes are
inside internal/notion; no new files.

F1 (major) — the 5xx/529 retry arm could never fire on POST
/data_sources/{id}/query, the paginating endpoint this PR exists to protect,
while the MethodDelete half of the allowlist had no call site at all (a census of
the client's verbs returns 3 GET / 3 POST / 2 PATCH, zero DELETE). Replaced the
verb gate with a retrySafety value each call site declares, so the property being
tested is side-effect safety rather than a proxy for it, and dropped the dead
MethodDelete arm. Only the read-only query POST changes behaviour: the creating
POSTs and both PATCHes stay non-replayable on every status except 429, exactly as
before. TestClientDoesNotRetryServerErrorOnPost exercised the query endpoint
while its "may already have been applied server-side" rationale could not be true
there, so it is retargeted at CreatePage and renamed
...OnCreatingPost, pairing with the existing 529 twin;
TestClientRetriesServerErrorOnQueryPost pins the new behaviour.

F2 (minor) — the exponential fallback had no jitter, unlike both sibling tracker
clients, so N processes tripping the same ~3 req/s limit retried in lockstep.
Jitter is applied to the self-chosen ladder only, never to a server-mandated
Retry-After, mirroring the split in internal/jira/client.go:419-424.

F3 (minor) — parseRetryAfter collapsed absent, HTTP-date and a legal
"Retry-After: 0" into one sentinel, so a date-form wait routed around the
maxRetryDelay refusal its delta-seconds twin is subject to. It now parses both
RFC 9110 forms and returns a presence flag; the doc comment's old justification
went with the behaviour it described.

F4 (nit) — maxRequestAttempts' comment claimed coverage of "transiently failing"
requests, which the transport arm does not have.

F5 (nit) — the invalid-value error names NOTION_MAX_QUERY_PAGES as well as the
config key, an abandoned wait now carries the API failure that caused it
alongside ctx's error, and the long-Retry-After refusal is evaluated before the
exhaustion break so the operator still gets it on the final attempt. That last
pin is a sibling test rather than a case inside
TestClientExhaustsRetriesAndReturnsTheLastError as the review suggested, to keep
that test a single-purpose pin on the loop's exit arithmetic.

Iteration 2 (non-gating, both in parseRetryAfter, which F3 already reworks) —
a delta-seconds value too large for a Duration now saturates instead of letting
the multiply wrap: "9223372037" wrapped to -2562047h47m16.7s, which is under
maxRetryDelay, so the refusal let it through to immediate retries. The date
form already saturated via time.Until. The unchecked multiply predates this
fixup (it is at the contributor's head) and is shared by linear/jira/gitlab,
which are left to a follow-up. TestClientRefusesLongRetryAfterInHTTPDateForm
drives a date-form header through the client, so the date form reaching the
refusal is pinned by the loop rather than only by TestParseRetryAfterHTTPDate's
arithmetic.

Validation: internal/notion 85 PASS / 0 FAIL / 0 SKIP; go build ./... and go vet
./cmd/bd (the sole importer) clean; CGO_ENABLED=0 builds; gofmt clean;
golangci-lint 0 issues both unscoped and --new-from-rev at the upstream base, and
the added //nolint:gosec is load-bearing (removing it reproduces exactly the G404
it names). Each fix is mutation-proved: reverting it reds its own pin while the
neighbouring pins stay green, including the two creating-POST controls under F1,
both directions of F2's jitter split, and both sides of the saturation bound.
Keying the refusal to delta-seconds only reds the new date-form test alone,
while TestParseRetryAfterHTTPDate stays green.

Moving the new delay logic into retryDelay rather than nesting it in the loop
took doRequest — the run's worst_changed function — from cyclomatic 21 /
cognitive 32 to 19 / 28 (gocyclo, gocognit), out of the major cyclomatic band
instead of further into it. No BUILD.bazel change is owed: no file was added and
both added imports (math, math/rand/v2) are stdlib.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
2026-10-04 07:16:26 -07:00
matt wilkie
e249a5a539
fix(routing): bind role detection to the bd -C target repo (re-port of #4242) (#5243)
* fix(routing): bind role detection to the -C target repo (re-port of #4242)

Role detection previously called routing.DetectUserRole(".") directly,
which resolves against the caller's CWD rather than the repo selected
via `bd -C <target>` (BEADS_DIR-driven). This meant auto-routing and
role-based behavior could silently apply the wrong repo's role when
invoked with -C.

Introduce activeRepoPathForRouting(), which prefers
beads.GetRepoContext().RepoRoot (the BEADS_DIR-bound target) and falls
back to the parent of beads.FindBeadsDir(), then to ".". Route role
detection in cmd/bd/create.go and cmd/bd/routing_read.go through the
new detectUserRoleForActiveRepo() helper instead of calling
routing.DetectUserRole(".") directly.

Faithful re-port of gastownhall/beads#4242 (osamu2001) onto current
main: determineAutoRoutedRepoPath's signature changed to
(string, routing.RoutingRule) since the PR was opened (89a4d6ed8 /
#4866), so the one-line call-site change re-applies inside the new
function body, and the new routing_read_test.go coverage destructures
both return values. All other ported content (routing_role.go,
context_binding_test.go, the create.go call site) is unchanged from
the original PR.

Co-authored-by: Osamu Okano <okano.osamu@gmail.com>
Agent-Signature: claude-fable-5-high on behalf of maphew

* test(routing): fix the sole CI failure in the re-ported #4242 test

TestActiveRepoPathForRoutingFallsBackToBeadsDirParent created its fake
project under this repo's tree, so beads.GetRepoContext() resolved a
real repo root from the caller's CWD and the FindBeadsDir fallback the
test claimed to cover was never exercised — an unreachable assertion
that CI reproduced as the PR's only failing test.

Rewrite it to target a directory that is genuinely outside any git
repo: targetDir now comes from t.TempDir() (its .beads/config.yaml
still written via writeTestConfigYAML), and the caller CWD is also
moved to a fresh t.TempDir() via t.Chdir() so GetRepoContext() can't
resolve through the CWD either. Both resolve under os.TempDir(), which
internal/beads/context.go's isPathInSafeBoundary explicitly admits, so
this stays safe under the existing boundary check.

This makes safeTempDirForContextTest (the manual git-repo-relative
temp dir helper the original test used to dodge the same problem)
dead code; delete it.

Co-authored-by: Osamu Okano <okano.osamu@gmail.com>
Agent-Signature: claude-fable-5-high on behalf of maphew

* fix(routing): a .beads/redirect must not move role detection to the storage root

Cross-vendor review of e8600e732 caught a regression the re-port would have
introduced for redirect users: with a .beads/redirect pointing at storage
outside any git repo, RepoContext.RepoRoot is the redirect target's parent,
so beads.role would be read from the storage location instead of the
workspace repo — silently bypassing contributor routing.

The two external-BeadsDir shapes now get their opposite answers explicitly:
an explicit BEADS_DIR selection (what bd -C plumbs) binds role detection to
the selected project (the GH#4242 fix), while a followed redirect keeps it
on the CWD workspace repo (pre-existing semantics — a redirect relocates
storage, not the project). Mutation-tested: with the guard removed, the new
test fails by resolving to the storage root.

Co-authored-by: Osamu Okano <okano.osamu@gmail.com>
Agent-Signature: claude-fable-5-high on behalf of maphew

* fix(routing): read BEADS_DIR selection provenance at startup, not the live env var

The self-review on this PR found a startup-path gap: rootCmd's
PersistentPreRunE calls prepareSelectedCommandContext for every command,
which sets BEADS_DIR to the resolved target — redirects included. By the
time role detection ran, activeRepoPathForRouting's os.Getenv("BEADS_DIR")
check was therefore always true, classifying a plain .beads/redirect as
explicit selection and binding beads.role to the storage root instead of
the workspace repo.

Capture whether BEADS_DIR was present in the environment at process start,
and treat selection as explicit only for that case or a -C flag. The new
TestActiveRepoPathForRoutingSurvivesStartupRebindAcrossRedirect goes
through the real startup rebind with a real redirect and fails on the
previous logic; the two tests that simulate a caller-exported BEADS_DIR
now pin their provenance explicitly.

Agent-Signature: claude-fable-5-high on behalf of maphew

* fix(routing): .env-provided BEADS_DIR is user selection, mark its provenance

Cross-vendor review of the previous commit caught a regression it would
have introduced: loadBeadsSelectionEnvFile intentionally lets a project's
.beads/.env route commands via BEADS_DIR, and that selection happens after
the process-start snapshot. Flip the provenance flag when the selector
loader sets BEADS_DIR, so .env routing keeps binding role detection to the
selected project. TestActiveRepoPathForRoutingHonorsEnvFileSelection pins
it end to end through loadSelectionEnvironment.

Agent-Signature: claude-fable-5-high on behalf of maphew

* test(routing): make the BeadsDir-parent fallback test order-robust on macOS

The macOS CI leg failed on /private/var vs /var: the test asserted a raw
string path while resolution can hand back the canonicalized spelling of
the same directory depending on which branch wins under test ordering.
Compare through EvalSymlinks like the sibling redirect tests, and pin
selection provenance in the two tests that had not declared theirs.

Agent-Signature: claude-fable-5-high on behalf of maphew

* ci: retrigger - dolt-install/download infra outage (curl 56/503) killed 4 jobs

Agent-Signature: claude-fable-5-high on behalf of maphew

* build(bazel): enumerate cmd/bd/routing_role.go in bd_lib srcs

The PR adds cmd/bd/routing_role.go, whose detectUserRoleForActiveRepo is
called from create.go and routing_read.go. cmd/bd/BUILD.bazel lists
go_library srcs explicitly, so the unenumerated new file broke the Bazel
library compile:

    cmd/bd/create.go:303:21: undefined: detectUserRoleForActiveRepo
    cmd/bd/routing_read.go:52:19: undefined: detectUserRoleForActiveRepo

Generated by `bazel run //:gazelle` (one srcs row, idempotent; no deps row
owed because the library already carries internal/beads and
internal/routing). Base parity at 89348699b0 was 336/336 non-test .go files
enumerated in cmd/bd, so the gap was PR-introduced, not pre-existing.

* fix(routing): bind role to the selected workspace across a .beads/redirect

Maintainer review fixup for the adopted PR (review iterations 1 to 3).
Iteration 1 addressed the one gating major plus both minors and both nits
from its review synthesis. Iteration 2 addressed the major, the minor and the
nit that the next synthesis raised against that fixup, and iteration 3
addresses the minor and the nit raised against iteration 2; see "Review
iteration 2" and "Review iteration 3" below.

The major: routing_role.go ordered its guard `explicitBeadsSelection() ||
!rc.IsRedirected`, but every beads-dir resolver follows a .beads/redirect before
that guard can run (ExplicitBeadsDir via FollowRedirect, FindBeadsDir, and -C's
own resolution). So an explicit selection of a REDIRECTED workspace arrived
already rewritten to the storage root, short-circuited the redirect doctrine the
PR had just written, and bound beads.role to wherever the DATA lives — the same
misroute GH#4242 reported, one resolution step further in. Worse, it was silent:
`git config --get beads.role` succeeds outside a repo by reading GLOBAL config,
and routing.DetectUserRole returns on that first success, so its own "beads.role
not configured" notice never fired and a contributor workspace could be routed
with the user's global role with no diagnostic on any stream.

The remedy has to be anchored per shape, because the two selection shapes carry
the pre-redirect directory in different places:

  - `bd -C <dir>`: applyChangeDirSelection has already rewritten BEADS_DIR to the
    resolved target, so only the -C argument still names the chosen project.
    CWD-anchored GetRedirectInfo() cannot see this case at all — under -C the CWD
    is some unrelated directory — so this arm asks GetRedirectInfoFrom(dir),
    which is documented for exactly this case (GH#5509) and deliberately does not
    consult BEADS_DIR.
  - exported BEADS_DIR: here the CWD *is* the selected workspace, so the
    CWD-anchored probe is the correct one. The selection is vacuous when it
    merely names that workspace's own redirect, under both spellings of it:
    BEADS_DIR=<workspace>/.beads and BEADS_DIR=<storage>/.beads both arrive as
    the storage dir (the bd-wayc3 tooling shape), so both fall through to
    rc.CWDRepoRoot.

An explicit selection of a genuinely different, non-redirected project still
wins — that is the GH#4242 fix itself, and a control test pins it.

Also in this commit:

  - A role path that is not a git repository now warns instead of quietly
    inheriting global beads.role (new beads.RepoRootFor answers both "what is
    the repo root for this dir" and "is this a repo at all").
  - A .env-provided BEADS_DIR records selection provenance on the broad loader
    too, not only via loadBeadsSelectionEnvFile. That loader early-returns
    whenever BEADS_DB or BD_DB is already exported, and loadBeadsEnvFile then
    imports the same .env line, so one .env line meant target-role or CWD-role
    depending on the caller's unrelated BEADS_DB export. Internal rebinds are
    unaffected: the marking is conditional on BEADS_DIR having been unset, which
    is what separates user selection from prepareSelectedCommandContext's
    per-command rebind.
  - The four-way role-path choice is logged via debug.Logf, since the downstream
    "failed to detect user role" warning fires on none of those branches.
  - The PR's new routing test neutralizes BEADS_ROUTING_MODE / BD_ROUTING_MODE,
    which viper binds onto the routing.mode key and which reds the test on any
    host that exports them.
  - routing_role.go now cross-references RepoContext.Role(), the second,
    known-divergent role oracle in internal/beads. No behavior change there.

Tests: the explicit x redirect cell was the one combination the PR's new tests
left empty. Both probe shapes land as tests, plus the `bd -C <redirected-ws>`
shape, a non-redirected -C control, the non-repo warning, and both provenance
polarities. Each new pin was verified to FAIL against the pre-fix code.

Review iteration 2:

  - Major: the non-repo warning above ran inside detectUserRoleForActiveRepo,
    which create.go and determineAutoRoutedRepoPath called before resolving the
    routing config. Routing is off by default and only auto mode reads the
    role, so in any non-git beads project every bd create/list/ready printed a
    4-line warning about a role nothing used. The leak also broke six
    TestParity* characterization tests at the iteration-1 head:
    CreateHumanOutput, CreateOnOccupiedIDRefuses, CreateParentNotFoundExits1,
    CreateMissingDepTargetIsFatal, CreateJSONErrorShape and
    ReopenErrorExits1Silently. Detection now runs only when the fully resolved
    config can use its answer: new RoutingConfig.UsesUserRole (auto mode, no
    explicit override, a role-specific repo configured) gates the new
    detectUserRoleForRouting, which both call sites invoke after building the
    config, so the legacy contributor.auto_route=true fallback to auto counts
    where a raw routing.mode check would miss it. Side effect: routing's own
    GH#2950 "beads.role not configured" notice is also skipped when routing
    cannot use the role, as docs/multi-agent/routing.md already promises when
    it calls `routing.mode explicit` "disabling role detection entirely".
  - Minor: for an exported BEADS_DIR, selectedWorkspaceRepoRoot called the
    selection vacuous when the CWD-anchored GetRedirectInfo reported a redirect
    to it. With no redirect in the CWD repo, GetRedirectInfo falls back to the
    raw BEADS_DIR, so a FOREIGN redirected workspace vouched for its own
    redirect and the CWD repo's role was bound. The check now also requires the
    redirect to sit in the CWD repo's own .beads (RedirectInfo.LocalDir). A
    foreign redirected selection falls through to its storage root, which is
    the answer a live command already reaches after the pre-RunE rebind, so
    unit and command behavior agree. Binding the foreign workspace itself would
    need the raw startup BEADS_DIR, which nothing keeps; the routing_role.go
    comment records that limit.
  - Nit: the non-repo warning prints at most once per process. An ID lookup
    falls back to auto-routing once per missing ID, and each fallback re-ran
    detection.
  - Test isolation: the PR makes role detection read the process-wide
    beads.GetRepoContext() cache, a sync.Once. Three existing tests chdir
    between temp repos without resetting it and so read beads.role from an
    earlier test's deleted repo: TestResolveCloseTargets/
    contributor_mixed_local_and_routed,
    TestDetermineAutoRoutedRepoPath_ContributorToPlanning and
    TestOpenRoutedReadStore_ContributorRouting fail in a broad cmd/bd run at
    86b8abff3f and 8dbfc6c721 and pass at base 89348699b0. The failure is
    order-dependent, and the tests skip without a Dolt test server. Each now
    rebuilds the cache after its chdir, as does the MaintainerToPlanning
    sibling, which has the same exposure.

Iteration-2 tests: a property test pins UsesUserRole to
DetermineTargetRepoWithRule (true exactly when the answer varies by role,
across unset/auto/explicit modes and every override and repo combination). A
table test covers routing unset, explicit mode, auto without a role repo,
auto, and legacy auto_route. Two more cover the foreign redirected BEADS_DIR
cell and repeated detection (the once-guard). Each pin was mutation-checked:
dropping the gate, gating on the raw mode, or dropping the LocalDir guard,
the once-guard, the role-repo clause or the override clause turns a test red.

Validated (iteration 1): gofmt, go build ./..., go vet ./cmd/bd/
./internal/beads/, focused go test over cmd/bd role/routing/envfile/context
tests and ./internal/beads/ ./internal/routing/, gazelle (0 srcs rows owed),
and bazel build //cmd/bd:bd_lib //internal/beads:beads.

Validated (iteration 2): gofmt, git diff --check, go build ./..., go vet
./cmd/bd/ ./internal/routing/ ./internal/beads/ with and without cgo,
golangci-lint v2.10.1 --new-from-rev=HEAD on cmd/bd and internal/routing (0
issues), go test ./internal/routing/ ./internal/beads/, and a broad cmd/bd
routing/role/redirect/context/Parity run. The only reds on this host,
TestInitRedirect/RedirectCreatesDBInTarget,
TestBareParentWorktreeCoreCommandsWithoutRedirect and
TestServeIssueRolesComeFromBeneathTheHookDecorator, fail the same way at base
89348699b0. No files were added and the only new import is stdlib, so no
BUILD.bazel change is owed.

Review iteration 3:

  - Minor: the iteration-2 guard counted a redirect as the CWD workspace's
    own only when it sat in the CWD git root's .beads (Dir(LocalDir) ==
    rc.CWDRepoRoot), which equates the workspace with the git root. From a
    workspace nested below its repo root (<repo>/sub/.beads/redirect), or
    from a git worktree that inherits its main checkout's redirect,
    exporting that workspace's own .beads therefore moved role detection
    from the CWD repo to the storage root: the env-dependent inconsistency
    the iteration-1 vacuous rule exists to remove. Dropping the guard would
    not have fixed it. When the repo root's .beads holds no redirect,
    GetRedirectInfo checks BEADS_DIR itself, and prepareSelectedCommandContext
    has rebound BEADS_DIR to the storage .beads before RunE. After that
    rebind GetRedirectInfo finds no redirect in either shape, so the
    iteration-1 rule bound the storage root there too. New
    beads.GetDiscoveredRedirectInfo is GetRedirectInfo without the
    BEADS_DIR override: the redirect discovery finds from the CWD (the repo
    root's .beads, then the worktree-local redirect, the main checkout's
    .beads for a worktree, and the walk-up from the CWD), via
    findDiscoveredLocalBeadsDir, the BEADS_DIR-free half of
    findLocalBeadsDir. The exported-BEADS_DIR arm now calls a selection
    vacuous exactly when that discovered redirect exists and targets the
    selected storage. A foreign redirected selection still falls through to
    its storage root, because discovery from the CWD finds no redirect or
    one with a different target.
  - Nit: the foreign-selection test's comment now says only the exported
    arm pins the guard. The rebound storage .beads holds no redirect, so
    the rebind arm only checks that both states of the variable agree.

Iteration-3 tests: nested-workspace and worktree-without-own-.beads cells,
each with BEADS_DIR unset, as exported, and as rebound before RunE. Against
the unchanged 07c7373dfb code the four set-BEADS_DIR cells fail and both
unset controls pass. A new test pins the target comparison: from a
redirected workspace, an exported BEADS_DIR naming a different,
non-redirected project still binds that project.
TestGetDiscoveredRedirectInfo pins the new API against GetRedirectInfo in
both directions (a nested redirect that BEADS_DIR masks, and a BEADS_DIR
redirect the CWD does not hold). Five mutants each turn a test red: probing
with GetRedirectInfo again (the iteration-1 rule, which fails both rebind
cells and the foreign test), dropping the target comparison, dropping the
worktree-fallback tier, letting discovery honor BEADS_DIR, and restoring the
git-root guard.

Validated (iteration 3): gofmt, git diff --check, go build ./..., go vet
./cmd/bd/ ./internal/beads/ ./internal/routing/, golangci-lint v2.10.1
--new-from-rev=HEAD on cmd/bd and internal/beads (0 issues), go test
./internal/beads/ ./internal/routing/, and a focused cmd/bd run over the
role, routing, redirect, envfile and context-binding tests against the
Dolt test server (47 passed, 0 failed, 0 skipped). No files were added,
so no BUILD.bazel change is owed.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Osamu Okano <okano.osamu@gmail.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-04 05:47:27 -07:00
Julian Knutsen
b184dae412
fix(dbproxy): managed-local dolt readiness proves port ownership; recover onto a free port (#7184)
* fix(dbproxy): prove the managed dolt owns its port; move a taken Beads-chosen port

The managed-local backend's dolt port comes from a bind-and-close
(proxy.PickFreePort) when bd writes the generated config. The dolt
sql-server binds it only later, after the proxy child has run dolt
config and init and booted the server. If another process takes the port
in that gap, dolt logs "Port N already in use." and exits. But
DoltServer.waitReady only dialed the port, so it accepted the other
process's listener as ready. The proxy then published proxy.pid, its
backend health check saw dolt gone, and it shut down. bd init's first
ping got "unexpected EOF" and every retry after that got "connection
refused". Under parallel tests this hit any test that starts its own
managed backend.

Readiness now requires proof that the process Start launched owns the
port:
- DoltServer sends the child's stdout/stderr through a watcher that
  forwards to the server log. A dial counts only after dolt's own
  "Server ready. Accepting connections." line has appeared on that
  output. That line is info level, and every generated config uses
  info. A config with a quieter log_level has no such signal and keeps
  the dial-only check.
- A child that exits first is reported as the new server.ErrPortInUse
  when its output said its port was taken.

Recovery: DoltServer.SetPortRepicker lets Start move the config to a
fresh port, re-read it, and relaunch, up to 5 ports. dolt config/init
run only on the first attempt. bd's repicker,
repickManagedProxiedServerPort, needs to tell a port Beads chose from
one an operator set, so generated configs now record the chosen port
on a "# beads-chosen-listener-port: N" line after the managed marker.
The repicker moves the port only when the file is Beads-generated,
carries that record, and its listener.port is still the recorded port.
Otherwise the port is treated as pinned and Start fails with
ErrPortInUse. That includes configs generated before this change. It
rewrites only listener.port and the record, through a yaml.Node so the
rest of the file is kept, and writes the file atomically.
db-proxy-child installs it for the local backend.

The proxy's waitForServerReady also re-checks Running after its dial.
Start has already proved ownership, and a running dolt holds its
listener, so only a backend that exited between the two checks could
let a dial reach another process.

Tests:
- server: a held port is not accepted as ready (ErrPortInUse; this
  test fails on main), a repicker brings the server up on a new port
  that the pid record and DSN both use, and a declining repicker ends
  Start with ErrPortInUse.
- cmd/bd: the generated config records its port, a Beads-chosen port
  moves while hand edits survive, and a hand-set port, a config
  without the record, an operator config, and a stale in-use port are
  all refused with the file left untouched.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(dbproxy): move a taken managed port via a runtime config, never config.yaml

This reworks cd00c5acb after review. That commit recovered from a taken
Beads-chosen port by rewriting the managed config.yaml. That broke the
file's own contract ("Never rewritten afterward; hand edits are
preserved", #4986), and #5024 had deferred that contract change to an
RFC.

dolt sql-server has no command-line way around the file: with --config,
-P/--port is silently ignored (checked against dolt 2.1.8 and 2.2.0, in
both argument orders). So on ErrPortInUse the proxy child now writes
proxy-child.runtime-config.yaml (server.RuntimeConfigFileName) in the
server root. That file is the operator config with only listener.port
moved to a free port. The child relaunches dolt from it and records the
effective port where runtime readers already look: the backend pid
record and the DoltServer's dial/DSN config. The runtime config is
removed when the server stops, when Start fails, and by the next Start.
config.yaml is never written.

- server.PortRepicker becomes server.PortConflictPolicy. It only decides
  whether the port may move; the server does the move.
- bd's policy (managedPortConflictPolicy) allows a move when the file
  carries the managed marker and either:
  - has the port record and listener.port still equals it, or
  - is byte-identical to the config bd generated before the record
    existed, so existing unedited workspaces recover too.
- Everything else is pinned: operator configs, hand-set ports,
  hand-edited legacy files, and configs with ${...} placeholders (a
  runtime copy would put the expanded values on disk).
- The record line now says plainly that Beads runs on a free port
  without editing the file, and that deleting the line or changing
  listener.port pins the port. The record parser now requires the
  digits to end at a space or line end and to be in 1..65535.
- ErrPortInUse messages say what to do: free port P, set listener.port
  in <path> to a free port, or (for a generated file) delete it so
  Beads regenerates it. The give-up message names the last port tried.
- A dolt that never logs the ready line is still not accepted. A
  grace-period fallback would let a foreign listener through while dolt
  is still starting. The timeout error now names the missing line, says
  whether anything answered, and points at the escape hatch: at
  log_level warning, readiness is dial-only.
- Port-conflict detection matches go-mysql-server's "Port host:port
  already in use." and the Windows bind error, and requires the SQL port
  in the kernel's bind error, so a metrics or MCP listener failure is
  not taken for the SQL port.
- Spawn error paths reap with cmd.Wait(). cmd.WaitDelay is set, and a
  comment notes that the backend's output is now a pipe into the proxy
  child.

Tests:
- Output-watcher table tests: split, one-byte and CRLF writes, output
  past the 4 KiB tail, the MCP ready line, port boundaries, the
  go-mysql-server and Windows forms, another listener's bind error, the
  unix-socket warning, and nil log.
- A real dolt at log_level warning starts through dial-only readiness.
- With a stand-in dolt (the test binary in fake mode, wired in
  TestMain): Start gives up after maxStartPortAttempts, a missing ready
  line is diagnosed, plus a positive control.
- With real dolt: a move keeps config.yaml byte-identical, runs from the
  runtime config, and the runtime config is removed on Stop. A declining
  policy and the no-policy remedy are covered.
- The bd policy's allow and pin cases.
- The managed-local lifecycle test's config/pid port check accepts the
  runtime config.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(dbproxy): ask the port policy once, about the operator's port; copy raw config text

Re-review of the runtime-config rework found a regression. After the
first move, Start asked the PortConflictPolicy about the runtime port.
bd's policy re-reads config.yaml, whose listener.port is still the
operator's port, so it refused with "listener.port is A, not the in-use
port B". The "up to 5 ports" retry therefore never ran in production;
the fake-dolt test missed it because its policy always allowed.

Start now asks the policy once per Start, on the first conflict, about
the operator config's port. Ports Start picked itself are its own to move
again. Start also re-reads the operator config up front, so every Start
begins from that config's port.

The runtime config is now built from config.yaml's raw text
(servercfg.NewYamlConfig) instead of the env-interpolated
YamlConfigFromFile result. dolt interpolates the runtime file when it
reads it, so the old way expanded placeholders twice: "$$" escapes lost a
level and placeholder values landed on disk. The server reads the
runtime file back with interpolation for its own Dial/DSN view. Since
placeholders no longer expand on disk, bd's policy stops refusing
configs that contain them.

Tests:
- bd's real policy driving a real DoltServer with a stand-in dolt that
  reports every port taken: Start makes five launches on five distinct
  ports, starting with config.yaml's, then gives up. It is not refused
  on the second move, and config.yaml stays unchanged.
- A frozen byte-for-byte literal of the config bd v1.3.0/v1.3.1
  generates. The legacy render must equal it, and the policy must find
  it eligible.
- The runtime config keeps placeholders and "$$" verbatim and is mode
  0600, while the server dials the interpolated host.
- The give-up test asserts the policy is asked once, about the
  operator's port.
- The missing-ready-line test timeout is raised to 30s for the slow
  fake-dolt exec under race.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(dbproxy): classify a child that exits before its identity is captured

TestManagedPortConflictPolicy_ConsecutiveCollisions failed on the farm
with "capture child birth identity: procid: process is no longer
running", after 2 launches instead of 5. Its stand-in dolt reports
"Port N already in use." and exits immediately, and it exited before
Start captured its birth identity. Start treated that capture failure as
fatal, so an ErrPortInUse that Start could recover from became a hard
error. A real dolt that finds its port taken can exit just as fast, so
this race exists in the product, not only in the test.

Every spawn step before the child is handed to its supervising
goroutine (identity capture, root identity, pid record) now fails
through failSpawn. failSpawn kills and reaps the child; reaping also
drains its output into the watcher. The child's own exit then decides
the error:
- the child reported its port taken: ErrPortInUse, so the normal policy
  and retry path runs;
- the child exited by itself for another reason: "exited (status N)
  before startup completed";
- the child was alive and we killed it: the step's own error, as
  before.

On Windows, a killed child reports an ordinary exit code, so there an
exit cannot be told from a kill. Only the port-in-use classification
applies there.

procid.Capture is reached through a package var so tests can make the
capture lose the race deterministically.

Tests, using a shell-script stand-in dolt that starts in milliseconds:
- a fast port-in-use exit that beats the capture is ErrPortInUse, and
  with a policy it is retried up to the 5-port budget;
- a fast exit for another reason reports its status;
- a capture failure while the child is alive stays fatal and kills the
  child;
- ten immediate port-in-use exits with the real capture are all
  ErrPortInUse.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 04:24:32 -07:00
Julian Knutsen
9c5a3fabd4
test: run TestSharedServerConcurrent in the cmd/bd Dolt lane; honest size/shards for uow_test (#7189)
* test(cmd/bd): run TestSharedServerConcurrent in the doltserver-cmd lane

TestSharedServerConcurrent was gated on BEADS_TEST_SHARED_SERVER=1, which
no CI job or Bazel target sets, so it skipped everywhere (the only
non-lane skip in //cmd/bd:bd_dolt_server_test). It predates the workspace
gate (#5093), and once enabled it failed immediately: in shared-server
mode every project's physical root is the one shared dolt dir, and bd init
holds that root's gate EXCLUSIVELY with a 5s wait, so concurrent inits of
different projects refuse each other ("bd init refuses to run over live bd
activity on this workspace"). That exclusivity is deliberate, and a single
init takes ~8s here, so the test now inits its projects one at a time; the
concurrent multi-writer workload phase, which is what the test exists for,
is unchanged.

- Also run when BEADS_TEST_SHARED_SERVER_CONCURRENT=1 (distinct from
  BEADS_TEST_SHARED_SERVER, which scripts/test.sh also reads to start a
  shared server).
- bd_dolt_server_test sets it with BEADS_TEST_SS_DIRS=2 and
  BEADS_TEST_SS_CLIENTS=6: a bounded smoke (~50s, 3 writers per database)
  against the lane's hermetic local dolt sql-server on an ephemeral port.
- Under BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 a server that cannot start
  fails the test instead of skipping it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): size and shard //internal/storage/uow:uow_test for its configless run

uow_test timed out at the default medium 300s whenever it ran without a
lane config (`bazel test //internal/storage/uow:uow_test`, also on
origin/main). No CI lane runs it that way: --config=ci and
--config=integration set BEADS_TEST_SKIP=dolt, so its Dolt-backed tests
skip and the target takes 3-14s there (BEP of recent ci/integration
runs), and the full Dolt workload runs as uow_dolt_test (8 shards, slowest
~90s remotely on main) and uow_docker_test (4 shards, large).

Without a config nothing skips: every newTestUOWProvider test starts its
own dolt sql-server (~6-8s apiece) and the package needs ~7 min serially;
TestBatchApplyContract alone is ~1 min because BoundsTheItemCount applies
1000+1001 real items by design. That is the variants' workload, not a slow
test to fix, so give the base target the variants' shape: size = "large"
(900s) and shard_count = 4. Locally, on a host at load ~185, the shards
took 162-309s (unsharded it was killed at 300s); under --config=ci the
four shards still pass in seconds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 04:00:49 -07:00
Julian Knutsen
ebe3b6bcb0
test,ci(cmd/bd): run the Dolt-gated cmd/bd suite (advisory Bazel lane), fix in-process runner leaks and rotted tests; init --quiet stays quiet (#7185)
* test(cmd/bd): reset cobra flags between in-process runs; keep them out of parallel tests

runBDInProcess and runBDInProcessAllowError execute the package-level
rootCmd tree again and again in one process, and cobra never resets a
flag between Execute calls: a value one invocation parsed (create's
--title, --labels, list's --status, ...) and its Changed bit stayed on the
FlagSet, so the next invocation saw it, and a slice flag that had been
set appended to the stale value instead of replacing it. With a Dolt test
server (the only setup in which these tests run) that failed about 20
TestCLI_* tests depending on suite order.

TestMain now snapshots every flag of the tree (snapshotCommandFlags,
after all init()s registered them, before any test runs) and both runners
put the tree back before and after each run (resetCommandFlags): the
pointee of each flag.Value is restored (for pflag's scalar types that is
the bound variable; for slice types it also clears the internal "changed"
bit, so the next Set replaces), slice contents via SliceValue.Replace,
and Changed=false. TestResetCommandFlagsRestoresEveryFlag sets every flag
of the real tree and checks the reset. The ad-hoc --title reset in
TestCLI_CreateRejectsEmptyTitle_ProxiedServerMode is no longer needed.

Seven TestCLI_Comment* tests called t.Parallel() around the runners.
inProcessMutex serializes the runners only against each other, so their
chdir, stdio swap, env and config re-initialization raced every other
parallel test (data races on config globals; TestCobraParallelPolicyGuard
failing to read its own source files after a concurrent chdir). They run
serially now, and the guard treats the two runners like rootCmd.Execute.

Also two stale TestCLI_Create* cases: DryRunWithRigPrefix exercised a
create --prefix flag that no longer exists, and the flag-like-title cases
passed "--help" or "--foo-bar" where cobra parses them as flags; they now
pass the title after "--", the only way it reaches create's own guard.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): fix the ADO round-trip and bulk-delete tests that rotted unseen

Both need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd (BEADS_TEST_SKIP=dolt everywhere), so they failed unnoticed:

- TestADORoundTrip* stored ado.pat in the database. ado.pat is a secret
  read only from config.yaml or AZURE_DEVOPS_PAT (config.IsYamlOnlyKey),
  so the tracker found no PAT. The fixture sets AZURE_DEVOPS_PAT.
- TestBulkDeleteNoResurrection expected GetIssue to return (nil, nil) for
  a deleted issue; the Dolt store reports storage.ErrNotFound.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): fix parity, completion and notion tests that only fail with a Dolt server

These are Dolt-gated (integration tag + test server), so no CI lane ran
them and they rotted:

- TestParityCreate*/TestParityReopenErrorExits1Silently: bd create
  auto-routes via routing.DetectUserRole("."), which warns "beads.role
  not configured (GH#2950)" on stderr when the working directory's git
  config has no beads.role. The suite pins the configured case, so the
  parity env now sets beads.role=maintainer in a test-owned global git
  config ($XDG_CONFIG_HOME/git/config; the role lookup scrubs every
  GIT_CONFIG_* variable, so those cannot be used).
- TestIssueIDCompletion: issueIDCompletion resolves a database path
  before looking at the open store and returns nothing when there is
  none; the test now sets dbPath to its store's path.
- TestIssueIDCompletion_UsesWorktreeFallbackWhenStoreNil and
  TestGetNotionConfigReadsDBPathWhenStoreUnset: the code under test
  reopens the store from metadata.json, which names a database but not
  the per-test branch of the shared database that newTestStore uses, so
  the reopened store saw none of the test's rows. They now use their own
  database (newTestStoreIsolatedDB).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): fix stale doctor/e2e/autocommit/readonly Dolt-gated tests

These need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd, so they rotted unseen:

- TestE2E_DoctorFixMetadataRoundtrip, TestE2E_MigrateDoltMetadata,
  TestDoctorCheckHealthReportsVersionMismatchOnRepoLocalPort: bd init now
  defaults to embedded mode, where `bd sql` and `bd dolt start` are
  refused. Init with --server on a repo-local server of their own
  (repoLocalServerEnv: the suite's shared-server port vars cleared, server
  stopped at cleanup) and start it explicitly, since BEADS_TEST_MODE=1 never
  auto-starts one. MigrateDoltMetadata's DELETE also needed `key` quoted.
- TestDoctor{Conventions,Pollution}MaxRows_EnvOnly_Exits2: bdRunRaw drops
  every BEADS_* variable, so the subprocess hit dolt.New's test-database
  firewall (exit 1) before the row cap; pass BEADS_TEST_SERVER=1.
- TestDoltAutoCommit_On_WritesAdvanceHead: the embedded engine keeps its
  data under .beads/embeddeddolt, not .beads/dolt; findDoltRepoDir looks
  in both.
- TestValidateCheck_AllClean: bd doctor validate runs five checks since
  the cross-table duplicates check was added.
- TestConfigValidateReadOnlyIsHermetic: readonly commands take a shared
  workspace gate, whose lock file (.beads/dolt.gate.lock) workspacegate
  creates on first use by design; compare the .beads tree without gate
  lock files.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): undo each in-process execution's env and storage-mode leaks

Every in-process run of the bd command tree (runBDInProcess and the many
tests that call rootCmd.Execute directly) leaves process state behind:
PersistentPreRun exports the selected workspace as BEADS_DIR, loads its
.beads/.env, and records the workspace's mode in serverMode /
proxiedServerMode. With a Dolt test server, later tests in the same shard
then resolved a deleted fixture ("no active beads workspace found",
"migration sidecar is missing"), ran subprocess bd against the wrong
workspace ("recorded as proxied-server in .beads/metadata.json"), or took
the proxied path in RunE-level tests ("proxied-server UOW provider not
initialized", a nil-UOW panic in reopenProxiedResolve that killed a shard).

TestMain now registers a cobra initializer/finalizer pair
(installExecuteIsolation) that snapshots the environment and the two mode
globals when an execution starts and restores them when it ends; the
runners' restoreEnvSnapshot shares the helper. TestDoltConfigEnvironmentOverrides
used os.Setenv + defer os.Unsetenv, which deleted the suite's
BEADS_DOLT_SERVER_PORT (the test server's port) for every later test; it
uses t.Setenv now.

Full integration-tagged cmd/bd suite against a hermetic dolt (16 shards):
18 order-dependent failures and the shard panic gone; the 27 remaining
failures all fail alone too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(init): keep --quiet quiet through the Claude/Codex/Cursor auto-setup

`bd init --quiet` ("Suppress output") still printed the project
integration installers' progress to stdout ("Installing Claude hooks for
this project...", "Claude Code integration installed", ...): the
installers write to os.Stdout unconditionally. Point stdout at the null
device around them while quiet; their failures keep going to stderr as
before. Found by TestInitCommand's quiet case, which asserts no output and
had not run in CI (Dolt-gated).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(init): bring the Dolt-gated init tests up to the current init contract

These tests need a Dolt test server, so no CI lane ran them and they
drifted from the product:

- Embedded is init's default mode, so the server-mode assertions
  (.beads/dolt, opening the database on the test server, dolt_mode
  "server" in metadata.json) now pass --server (TestInitCommand,
  TestInitAlreadyInitialized, TestInitRedirect, TestInitBEADS_DIR,
  TestInitDoltMetadata, TestInitDoltMetadataNoGit, TestInitDatabaseFlag's
  metadata subtests). The in-process ones reset rootCmd's flags around
  themselves so --server cannot leak into later in-process inits.
- Each server init gets its own --database: the suite shares one test
  server, and a "test" database another test created belongs to another
  project (init refuses: PROJECT IDENTITY MISMATCH).
- The marker tests seed a current-era version witness: the legacy-upgrade
  guard refuses a non-empty .beads/dolt without one.
- The redirect test provisions the canonical target with config.yaml: bd
  ignores a redirect to a directory with no project files (#4692).
- Shared-server inits get a free port of their own
  (sharedServerTestEnv): the inherited BEADS_DOLT_SERVER_PORT is the
  suite's test server, which a shared-server start refuses as another
  project's, and the default shared port is 3308, a developer's live
  server.
- --shared-server's config.yaml is now nested YAML (dolt: shared-server:
  true); parse it instead of matching the old flat line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): run the Dolt-gated tests that skipped on a bare dolt.New

setupTestDB (config, kv and Linear round-trip tests), setupGatedTestDB
(mol ready --gated) and the rename-prefix and bd sql tests opened stores
with dolt.New(&dolt.Config{Path: ...}): no server port, no
CreateIfMissing, so against a running test server they got "database ...
not found" and skipped as "Dolt server not available". They now use the
suite's newTestStore helpers (skip only when no test server runs), and
what that exposed is fixed test-side:

- TestCustomStatusConfig: custom_statuses is read back ORDER BY name;
  look statuses up by name / expect name order.
- TestLinearRoundTripCoreFields, TestLinearPullMilestonesCreatesEpicHierarchy:
  linear.api_key is a yaml-only secret (LINEAR_API_KEY), push now needs an
  explicit linear.state_map, and batch push uses TeamLabels and
  issueBatchCreate, which the mock server now answers.
- TestSqlCommand: bd sql refuses in the default embedded mode; set
  serverMode for the server-backed store, and stop discarding RunE errors.
- TestE2E_AutoStartedRepoLocalServerPersistsAcrossCommands skipped on
  this process's embedded mode although every bd it runs is a subprocess
  on a --server workspace; gate on the dolt binary instead and stop the
  auto-started server at cleanup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): no fixed "unused" port, no swallowed JSON, no leaked front-door server

Three more things the Dolt-backed cmd/bd run exposed:

- TestTestServerConnection, TestSharedServerCfgNilHonorsSharedServer and
  the two TestLegacyGuard* e2e tests pointed bd at 127.0.0.1:59999 as a
  port nothing listens on. Any process on the host may hold it; a
  listener that accepts and resets turns the expected "connection
  refused" / "Dolt server unreachable" into another error. They take a
  just-freed port from closedLoopbackPort (testutil.FindFreePort) now.
- TestDoltSetConfigJSONOutput and TestDoltSetConfigWithUpdateConfig
  created .beads 0755, so bd's permissions warning landed in the JSON
  under test and the tests skipped ("output not pure JSON") every run.
  The fixture uses 0700, and non-JSON output fails instead of skipping.
- TestMigrateDoltModeFrontDoor's closing show/list auto-start a
  repo-local server after the reverse migration; nothing stopped it, so
  the suite's leak check failed (when the opt-in
  BEADS_TEST_MIGRATION_FRONTDOOR suite runs). It stops it at cleanup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): advisory cmd/bd Dolt-server tier (bazel-cmd-dolt)

No CI lane ran cmd/bd's Dolt-gated tests (TestCLI_*, the init and
store-backed suites, the doctor/e2e tests): main.yml's integration shards,
nightly's Full Test Suite, and Bazel's --config=prcore/ci and
--config=integration all pass BEADS_TEST_SKIP=dolt, so cmd/bd's TestMain
never starts a test server and those tests skip; the proxied and embedded
tiers run only their manifests' tests. They rotted unseen.

//cmd/bd:bd_dolt_server_test (tag dolt-server-cmd) runs the whole
integration-tagged cmd/bd suite, the integration build's bd_test, against
a hermetic dolt sql-server: BEADS_TEST_DOLT_SERVER=local starts it from
the pinned dolt in the test wrapper's runfiles, and
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 fails the target instead of skipping
into a cached pass. No test selection: the Go binary shards itself over
every top-level test (32 shards), so each runs exactly once, and
check_testcases.py fails a shard that ran none. --config=doltserver-cmd
has exactly --config=integration's build flags, so the two lanes share
bd_test's compile; prcore/ci and integration exclude the tag.

bazel.yml's new bazel-cmd-dolt job runs it exactly where bazel-integration
runs (same if: remotely on the farm for same-repo PRs, fork-ro/fork-rw,
and locally with the read-only cache in mode cache for forks). It is
advisory: job-level continue-on-error keeps a failure out of the call's
aggregate (BAZEL), and pr.yml's ci-gate reads its result (BAZEL_CMD_DOLT)
but requires it only while the committed top-level flag
BAZEL_CMD_DOLT_REQUIRED is "true" (now "false"; flipping it is the one
change needed). bazel-gate.sh accepts its skip in modes skip and local.

Policy tests: a third lane category in TestBazelLaneIsGatedAlongsideLegacy
(bazelFlagGatedLanes: gate env wired, never statically required, added by
the run script exactly when the pinned flag is "true"), the gate
simulation over every mode with the flag off (advisory: nothing it
reports reddens the gate) and on (exactly a gated lane), the cache-mode
and execution-mode checks, the tag taxonomy and prcore/integration
exclusions, and TestBazelCmdDoltJob (job steps, timeouts, .bazelrc
config equal to integration's build with no dolt skip or test selection,
the target's dolt-server rules, and bd_test's env repeated).

Measured locally (all fixes in this series, 32 shards, 16 at a time on a
shared host): 3267 top-level tests, all shards pass, max shard 313s (one
5001-issue export test, 214s, sets it), mean 100s.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(init): hand the project installers a writer instead of swapping os.Stdout

The --quiet fix earlier in this series pointed the process-global
os.Stdout at the null device around the Claude/Codex/Cursor installers.
That works, but it mutates a global from inside a cobra Run: anything else
writing to stdout meanwhile (a goroutine, a future concurrent installer)
would vanish, and a concurrent read of os.Stdout is a data race.

The installers already write through env structs with stdout/stderr
fields. setup.ProjectInstallOutput carries the two writers, and
InstallClaudeProjectTo / InstallCodexProjectTo / InstallCursorProjectTo
use them (the old entry points keep the process streams). init passes
QuietProjectInstallOutput() under --quiet: progress discarded, error lines
still on stderr. The proxied-server init path (domain adapter
InstallClaudeProject now takes quiet) does the same, so its
"Installing Claude hooks..." is quiet too.

Consistent error policy under --quiet: errors still reach stderr. init's
own "Warning: failed to setup <installer>" lines were suppressed under
--quiet while the installers' "Error:" lines were not, and some installer
failures (Cursor rules) print nothing themselves; the warnings now print
whether or not quiet, like the installers' errors.

Coverage in required lanes: TestProjectInstallersReportToTheGivenOutput
(cmd/bd/setup) and TestEmbeddedInitA/basic, which now asserts that
`bd init --quiet` with agents enabled installs the Claude hooks and prints
none of the installers' progress (verified to fail with the quiet output
disabled).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(export): exercise the incremental-export fallback with a small threshold

TestTryIncrementalExport_ThresholdExceededFallsBack seeded
incrementalExportThreshold+1 = 5001 issues to cross the threshold. Batch
create is one transaction but still several statements per issue, so under
-race against the shared test server it took ~214 s, the longest test in
cmd/bd's Dolt-backed suite by far. The behaviour under test is only
"more changed issues than the threshold -> full export fallback".

incrementalExportThreshold becomes a var (same value), and the test, which
is not parallel, sets it to 20 and seeds 21 issues: 214 s -> 0.6 s.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): tighten the flag-reset self-test, guards and canaries

Review follow-ups:
- TestResetCommandFlagsRestoresEveryFlag no longer skips a flag it cannot
  set: a flag type without a sample value, a Set error, or a non-pointer
  Value (which resetCommandFlags could not restore) is an error, so a new
  flag type cannot go untested. resetCommandFlags documents that it also
  resets flag-bound package globals.
- TestCobraParallelPolicyGuard treats .ExecuteC/.ExecuteContext/
  .ExecuteContextC like .Execute (the per-execution isolation writes the
  mode globals; a parallel execution would race).
- TestConfigValidateReadOnlyIsHermetic ignores only the gate a readonly
  command takes (.beads/dolt.gate.lock and its .info sidecar), not any
  "*.gate.lock*" file, so a new unexpected gate still trips the canary.
- TestCLI_CreateRejectsFlagLikeTitles: a bare `bd create --foo-bar` must
  fail and create no issue (bd-2c0's intent), next to the "--" cases.
- TestCustomStatusConfig cites why statuses come back in name order: the
  normalized custom_statuses table (#2961, 8537d943b) has no ordinal and is
  read ORDER BY name, like custom_types; the test predates it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the cmd/bd Dolt-server tier in 16 shards

Each shard pays ~30-40 s of fixed cost (TestMain plus its own dolt
sql-server), so 32 shards spent roughly 40% of the lane's executor time on
setup and added 32 farm actions to every PR touching cmd/bd. With the
5001-issue export test cut to under a second, 16 shards measure (locally,
--nocache_test_results): max 186 s, mean 137 s, 2,187 executor-seconds
(vs ~2,800-3,200 at 32), all 3,267 top-level tests, every shard passing.
TestBazelCmdDoltJob pins the count; bazel.yml's timing comment is
restated (cache mode ~20-30 minutes).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 03:06:54 -07:00
Julian Knutsen
683993660f
fix(proxy): fail fast with a clear MySQL error when an external upstream is unreachable (#7181)
* fix(proxy): fail fast with a MySQL error when the external upstream is unreachable

A proxied-server command against a stopped external Dolt upstream stalled
~20-30s before failing with a bare "invalid connection". The local db proxy
stays up, so on a refused backend dial (or a front such as stunnel/socat
that accepts and closes because its own upstream is gone) it simply closed
the client connection. The MySQL driver reports that as invalid
connection / unexpected EOF, which the uow bootstrap ping (#6003) classifies
as a transient drop and retries with exponential backoff for its whole 30s
budget. Retrying cannot help: the proxy already proved the upstream at its
own startup, and the outage is the upstream's.

The proxy now answers such a connection itself with a MySQL ERR packet
(2003, HY000) in place of the server greeting, naming the upstream and the
cause, exactly as a real server refuses a connection before the handshake.
The driver surfaces it as a *mysql.MySQLError, which the bootstrap retry
already treats as permanent, so the command fails in ~0.1s with an
actionable message. Only shapes that prove the upstream is not serving
qualify: ECONNREFUSED/ENOENT/EHOSTUNREACH/ENETUNREACH on dial, or a backend
that reaches EOF having sent zero bytes (a MySQL server always speaks
first). A dial timeout or a drop after the greeting keeps the plain close
and the client's transient retry.

External backends only: a managed (local sidecar) backend is owned by the
proxy and its dial can be refused transiently while bd drives it (the
full proxied lane caught TestProxiedServerInitExplicitBeadsDirIsAuthoritative
hitting exactly that mid-init when this applied to every backend), so
managed backends keep the bare close and the client's bootstrap retry.
TestExternalProxyOutageAndRestoreReDialsTCPAndUnix asserted the old
silent close during an outage; it now asserts the ERR packet.

With the latency fixed, TestProxiedOutageReconnectAcceptanceMatrix is
renamed into the proxied discovery pattern as
TestProxiedServerOutageReconnectAcceptanceMatrix and placed in the Bazel
30-shard block by gen_proxied_shard_manifest.py's incremental --write
(shard 8). The frozen legacy 15-shard block is untouched; the test
hash-distributes there like every other test added since it was frozen.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(proxy,uow): retry the proxy's upstream-outage report for ~1s; hedge the zero-byte case

Review follow-up to the fail-fast change.

A zero-byte close from the upstream is not proof it is down: Dolt closes
queued connections without a byte when max_connections/back_log are full,
when a queued connection times out, and while it shuts down for a
restart; an L4 front does the same while its target is replaced. Its
report now says "closed the connection before the MySQL greeting (down,
restarting, or at its connection limit)" instead of "unreachable", and
the comment no longer claims proof.

Refusals and endpoint churn also lost #6003's retry entirely, so an
external Dolt restart or a Service with no ready endpoints failed a
command at once. The client now retries the proxy's report
(proxy.IsUpstreamOutageError: 2003 with the "beads db proxy: " prefix)
for upstreamOutageRetryWindow (1s) from the first time it sees it, with
the backoff clamped so the last attempt lands at the window's end; other
transient errors keep the 30s budget. A real outage still fails in about
a second, inside the acceptance test's 3s bound.

Also: the dial report drops Go call-site wrapping ("dial tcp H:P:
connection refused"), truncation keeps UTF-8 intact, the comment states
how the packet differs from a real server's pre-handshake ERR (keeps the
'#'+SQLSTATE marker for vitess; 2003 is a client code chosen on purpose),
a Stats counter for reported outages, and the isTransientPingError
paragraph is reflowed and corrected.

Tests: pingWithRetry retries the report inside the window and gives up
at its end without using the full budget; openDB against a front that
plays the proxy (its real packet) and comes back 300ms after the first
connection succeeds in one call, and fails in about the window when it
never does; NewProxyServer enables the report for external backends
only; the dial message carries no Go type names; rune-safe truncation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(proxied): fail, not skip, socat-bridged subtests in the Bazel lane

The external-unix front-door matrix subtest and both outage/reconnect
matrix subtests bridge an external endpoint with socat and skipped when
it was missing, so they could stop running without anyone noticing (the
parent test still passes). requireSocat now fails them when
BEADS_TEST_REQUIRE_SOCAT=1, which //cmd/bd:bd_proxied_test sets (its
remote workers have socat; a recent main run's external-unix subtest
passed there). The legacy GitHub 15-shard jobs have no socat and leave
it unset, so the subtests keep skipping there. The check now runs before
the upstream Dolt server is started, so a skip no longer pays for one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(proxied): measured weight for the outage matrix; pin it to legacy shard 1

Give TestProxiedServerOutageReconnectAcceptanceMatrix a duration entry
(80s: local runs measured 55-79s alone, 60-70s in its shard, and review
68-109s under load) and re-place it with the generator's incremental
--write: it moves from 30/8 to 30/11, one line, nothing else moved. The
heaviest estimated shard is now 435s against a ~357s median; a repack
would even that out but moves most assignments, so it is left to a
separate CI-only change that refreshes all durations together.

Legacy 15-shard block: the test would hash onto shard 14, already ~13.5
of the 15 min timeout on main. It is pinned to shard 1 (~6.5 min in
recent main runs) under the block's reviewed-pin rule; it skips there
today (no socat), and the pin keeps shard 14 safe if that changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 02:37:58 -07:00
Julian Knutsen
b8a9545cbe
ci(setup-bazel): 4 GB client heap on every runner (R4) (#7174)
The Bazel client's JVM defaults to 25% of RAM, about 2 GB on the 8 GB
2 vCPU Blacksmith runners every remote lane runs on. //... analysis fits
today with little headroom, and the plan moves more lanes and targets
onto these runners. write-bazelrc.sh now writes
`startup --host_jvm_args=-Xmx4g`: a startup option, so it changes no
action key and only restarts the server. Verified with Bazel 9.2.0:
`bazel --bazelrc=<generated rc> info max-heap-size` reports 4294MB.

gascity's W1 byte-copies this composite, so the change lands here first.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 01:04:39 -07:00
Julian Knutsen
c41079528d
ci: pin the Actions allow-list for the selected-and-SHA-pinned policy (R3) (#7175)
beads allows every action today (allowed_actions: all, no SHA pinning),
so a fork PR's workflow can call any action, including Blacksmith's
sticky-disk action that commits state later trusted runs restore.
.github/allowed-actions.json is the exact body of the selected-actions
setting: GitHub-owned actions plus gascity's list plus the third-party
actions beads uses. That includes DeterminateSystems/nix-installer-action,
which determinate-nix-action calls, and peter-evans/create-pull-request.

Every uses: under .github is already pinned to a full commit SHA. The new
ci_actions_policy_test.go keeps it that way: each reference is local or
owner/repo[/path]@<40-hex>, and allowed by the file, so the repository
setting can switch to allowed_actions=selected with
sha_pinning_required=true without breaking a workflow. Negative controls:
an @v6 tag ref and an unlisted owner/repo each fail the test.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 01:03:40 -07:00
Julian Knutsen
b1a0690249
ci: pin four hash-placed proxied tests off overloaded legacy shards 5 and 3 (#7172)
Main's "Test (proxied-server cmd 5/15)" has hit the 15m -test.timeout in
6 of the 9 Main runs since #7129 where the shard ran to completion.
That PR renamed 12 proxied tests into TestProxiedServer* discovery.
They are not in the frozen 15-shard legacy block, so they hash-place,
and two landed on shard 5:
TestProxiedServerStorageClass (nine parallel inits, the heaviest of the
12) and TestProxiedServerServeRefusesAStaleCheckpointWithGone. Shard 5
went from 430-620s of job time to 800-950s. In those runs the timeout
fired while TestProxiedServerMolReadyGated's parallel subtests were still
queued behind the rest of the shard. That test is the victim, not a
pathological one. Shard 3 (also hash-loaded) runs 810-870s, within about
a minute of the same limit.

Measured locally (4 pinned CPUs, GOMAXPROCS=4), shard 5 took 897s with
its current test set and 524s with the pre-#7129 set.

Pin those two tests, plus shard 3's two heaviest hash-placed tests
(TestProxiedServerListComments and
TestProxiedServerServeRefusesAStreamOnADisabledJournal), onto the legacy
shards with the lowest measured job time: 14, 7, 4 and 1. Every test
already listed in the block stays where it is, so shard 5 is back to its
pre-#7129 test set. pr-risk.yml's fork jobs read the same block, so they
get the same relief. The script's 15m timeout, the job and step
timeouts, and the Bazel 30-shard block are unchanged. The block header
and the generator's legacy header now say a reviewed pin is allowed.

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 00:15:08 -07:00
Julian Knutsen
cdda0dffa5
test(cmd/bd): drop dead unsupported-storage test, CWD-independent bd fallback build, stop BEADS_DIR leaking between in-process runs (#7171)
* test(delete): drop stale TestDeleteIssueUnsupportedStorage

The test asserted that deleteIssue returns "delete operation not supported
by this storage backend" when the global store is nil. That error came
from a type assertion to a deleter interface that a5da57e39 (Phase 7:
remove storage factory/memory backend/provider abstraction) removed:
store is now storage.DoltStorage, whose method set always includes
DeleteIssue, so there is no unsupported backend left to report. With the
assertion gone, the test's nil store is just a nil-interface method call
and panics at deleteIssue, killing the whole cmd/bd test binary.

The nil store is not reachable in production: deleteIssue's only caller
(bd doctor --check=pollution --clean) has already queried the store to
find the issues it deletes. So the test constructs an impossible state
for a removed behavior; delete it rather than reintroduce a dead branch.

No CI lane ran it: it needs the integration tag and a Dolt test server
(testDoltServerPort != 0), and every integration lane that builds
cmd/bd (main.yml integration cmd/bd shards, nightly Full Test Suite,
bazel --config=integration) sets BEADS_TEST_SKIP=dolt, while the
proxied/dolt-server lanes run only their TestProxiedServer*/
TestServerMode* manifests. It only ran in local integration runs with
Docker or a matching dolt.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): build the fallback bd by package path, independent of CWD

When BEADS_TEST_BD_BINARY is unset, buildEmbeddedBD and its sibling
helpers fall back to `go build -tags gms_pure_go -o X .`, which builds
whatever package is in the process's working directory. That is cmd/bd
only if the first caller of the sync.Once has not chdir'd into a fixture
and the test binary was started in cmd/bd. A precompiled test binary run
from the repo root (the pr-risk pattern) builds the root library package
instead: go writes a non-executable archive to X and the test dies with
"fork/exec .../bd: permission denied" (reproduced with TestEmbeddedCount).

Add goBuildBDCommand: the same non-race -tags gms_pure_go build CI
prebuilds BEADS_TEST_BD_BINARY with, naming
github.com/steveyegge/beads/cmd/bd explicitly and running in cmd/bd's
source directory, resolved at package init (this file's compiled-in path,
else the start-up working directory) before any test can chdir. Use it in
buildEmbeddedBD and the other copies of the pattern (cli_fast,
doctor_repair, explicit_db_nodb, init_permissions, scripttest,
shared_server, buildBDForInitTests); migrationFrontDoorBinary keeps its
untagged cgo build but gets the same package path and directory.
TestGoBuildBDCommandIsCWDIndependent checks it from a chdir'd temp dir
with go list (no link).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): stop in-process bd from leaking BEADS_DIR into later tests

TestResolvedConfigRepoRoot's two CWD-discovery subtests ("uses local
workspace from subdirectory", "uses worktree fallback when local .beads
is absent") failed in full cmd/bd runs with "no active beads workspace
found" but passed alone. Cause: runBDInProcess / runBDInProcessAllowError
run rootCmd.Execute in the test process, and PersistentPreRun exports the
selected workspace with a raw os.Setenv("BEADS_DIR", ...) that nothing
restores. After any TestCLI_* test, BEADS_DIR still names that test's
since-deleted bd-cli-test-* fixture; FindBeadsDir honors an explicit
BEADS_DIR before walking from CWD, finds nothing there, and reports no
workspace. Minimal repro: -run '^(TestCLI_Ready|TestResolvedConfigRepoRoot)$'.

The leak also broke later TestCLI_* tests themselves: with
-run '^(TestCLI_.*|TestResolvedConfigRepoRoot)$' against a Dolt test
server, failures drop from 47 to 20 with this change and no new ones (the
remaining 20 are leftover cobra flag values between in-process runs, a
separate issue).

- runBDInProcess(AllowError): snapshot os.Environ() around Execute and
  restore exactly the keys it changed (restoreEnvSnapshot).
- TestResolvedConfigRepoRoot: clear BEADS_DIR (t.Setenv) in the two
  subtests that test CWD discovery, so neither a leak nor a developer's
  exported BEADS_DIR decides them.

Not host-specific (HOME, global git config and ceilings play no part),
and CI never saw it: the TestCLI_* leakers need a Dolt test server, and
every CI lane that runs cmd/bd's integration tests sets
BEADS_TEST_SKIP=dolt, so they skip and BEADS_DIR is never set.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(delete): expect storage.ErrNotFound after delete in TestDeleteIssueWrapper

"successful issue deletion" asserted that GetIssue returns (nil, nil)
for a deleted issue; the Dolt store reports a missing issue as
storage.ErrNotFound, so the subtest failed with "GetIssue failed: not
found: issue test-xxx" whenever it ran. Like TestDeleteIssueUnsupportedStorage
it needs the integration tag and a Dolt test server, which no CI lane
provides to cmd/bd (BEADS_TEST_SKIP=dolt), so it rotted unseen.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 23:44:37 -07:00