Makes the repo's agent and contributor surface describe developing beads, not using bd, and requires every PR to link a documented issue.
- Contributor-only root AGENTS.md with a routing table; cmd/bd/AGENTS.md and engdocs/AGENTS.md seam files; CLAUDE.md symlinks so Claude Code loads nested files.
- Bug and feature issue forms, a PR template that requires Closes #, CONTRIBUTING sections for the triage and pipeline labels, and the opt-in fork-cache during dev setup.
- ROADMAP.md for 1.4: versioned beads (#6132), Graph Preview 2 (#7170), contributor onboarding (#7228).
- docsync guards: root AGENTS.md at or under 20 KB; every AGENTS.md symlinked, routed, and with resolving links, paths, and guard-test names.
Agent-Signature: claude-code-claude-opus-5-5-high on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bazel-integration (--config=integration) and bazel-cmd-dolt
(--config=doltserver-cmd, //cmd/bd:bd_dolt_server_test) both run the
integration-build cmd/bd bd_test; the second without BEADS_TEST_SKIP=dolt,
so it is a superset. Once pr.yml requires bazel-cmd-dolt, the integration
lane's bd_test run is a duplicate.
bazel.yml gains a boolean workflow_call input, cmd-dolt-required (default
false). When true, bazel-integration tests //... -//cmd/bd:bd_test. pr.yml
passes it, and a policy test pins it to pr.yml's BAZEL_CMD_DOLT_REQUIRED
(a `with:` cannot read env), so the drop only happens where ci-gate
requires BAZEL_CMD_DOLT. bazel-farm.yml, nightly.yml, push and dispatch
leave it unset and keep bd_test. While the flag is "false" (current main)
nothing changes; with #7226's flip the input flips with it.
Coverage: both lanes share the same `if:` (modes remote, fork-ro, fork-rw,
cache; Dependabot included) and bazel-gate.sh accepts their skips in the
same modes (skip, local). TestBazelIntegrationExcludesBdTestOnlyWhereCmdDoltCovers
pins that, the exact exclusion, the flag/input equality, the gate's
conditional requirement, and that no other caller passes the input.
Rollback: commit BAZEL_CMD_DOLT_REQUIRED "false" and cmd-dolt-required:
false together in pr.yml.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Advisory since #7185. On the farm it passed on every PR run based on
current main; its only failures (3) were branches predating #7219 and
reproduced a real test bug that #7219 fixed. Flip the committed flag so
CI Gate / Required enforces BAZEL_CMD_DOLT. Rollback: set it back to
"false".
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pr-preflight-platforms' "Exercise native init gateway credential shell"
step had no GOCACHE env, so it fell back to the runner's empty default
cache and cold-compiled all of ./cmd/bd's test binary on every PR, right
after the hook-timeout step had compiled the identical binary (same tags,
same platform-default CGO) into the restored non-race cache. Job logs:
213s of a ~350s macos-latest leg, ~117s on Windows; on Linux the rerun
with a shared cache is a relink only (~8s).
- pr.yml: point that step, and the three Windows-only Go steps, at the
restored non-race GOCACHE.
- main.yml: the macOS test leg (the only macOS non-race seeder) now runs
warm-non-race-cache.sh, so the PR macOS legs restore the test packages
and test-only deps they compile, not just the non-test ./cmd/bd graph.
- Policy tests: every run step in pr-preflight-platforms and
check-doc-freshness-platforms must use the restored GOCACHE, and the
macOS seeder must warm before it saves.
No test selection, matrix, or gate wiring changes.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(gate): ordinary commands wait out maintenance, with writer fairness
Ordinary bd commands take the workspace (and physical-root) gate SHARED and
never waited for it, so while another process held it EXCLUSIVELY — bd init
(~8s), backup restore, migrate, bootstrap — they failed at once with "a
maintenance operation is running". In shared-server mode every project
shares one physical-root gate, so another project's bd init broke every bd
command on the machine for its duration.
Ordinary commands now wait:
- up to 15s by default; BEADS_GATE_WAIT_TIMEOUT (Go duration or bare
seconds, same convention as BEADS_INIT_GATE_TIMEOUT) overrides it, and 0
restores fail-fast;
- one "waiting for another bd process ..." notice after 2s (shared with bd
init's notice);
- Ctrl-C aborts the wait. A canceled wait used to fall through to the
shared fail-open path ("continuing ungated"); it is now a hard error;
- past the bound, the same error naming the holder, the wait, and the knob.
Git hooks stay fail-fast: `bd hooks run` and the bd export/import children
the hooks spawn run with BD_GIT_HOOK=1, treat failure as a warning, and must
not stall a commit or checkout for 15s.
Writer fairness (internal/workspacegate). flock/LockFileEx give no queueing,
and acquisition is non-blocking polling, so a steady stream of overlapping
shared holders could keep a waiting exclusive acquirer (bd init) out past
its bound. A per-gate intent lock (<gate>.intent) fixes that: a waiting
exclusive acquirer whose first try is busy takes the intent lock
exclusively while it polls, and waiting shared acquirers probe it before
every attempt and wait while it is held. The exclusive waiter then waits
only for in-flight holders to drain. The intent is an OS lock, so a crashed
waiter's intent dies with it; its advisory sidecar is ignored when the lock
is free, and intent-lock errors other than "busy" degrade to the old
behavior. Fail-fast acquisitions (Wait <= 0) ignore the queue entirely, so
hooks and BEADS_GATE_WAIT_TIMEOUT=0 behave exactly as before. The intent
lock is taken only while acquiring its gate, directly before it in
AcquireAll's sorted order, so it adds no lock-order cycles.
Nested bd (bd orphans running bd close, hooks fired by git that bd runs)
would deadlock against the queue: the child queues behind an exclusive
waiter that is waiting for the child's parent. A bd process holding its
gates shared exports BEADS_GATE_SHARED_HOLDER_PID; a child whose recorded
ancestor is alive skips the queue (it cannot help the waiter anyway).
Tests: workspacegate fairness (a continuous shared stream no longer starves
an exclusive waiter; verified to fail without the queue), intent lifecycle,
stale intent from a SIGKILLed process, cancellation while queued,
InheritedSharedHold. cmd/bd: holder released after ~3s lets a command
through with one notice; past-bound error text; fail-fast modes (0, hooks);
Ctrl-C is not fail-open; a new command queues behind a waiting bd init
while a child of a shared holder does not; env parsing.
TestSharedServerConcurrent runs a late bd init of a new project during the
client workloads, so both directions are exercised against a real server.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(gate): a queued exclusive waiter may delay shared commands, never fail them
Review follow-up (B1). A doomed exclusive waiter — bd init or restore
blocked behind a long-lived shared holder such as `bd list --watch`,
`bd events tail --follow`, an open `bd edit` editor, or a beads.OpenGated
embedder — held the writer-fairness intent lock for its whole budget, so
every NEW ordinary command deferred to it and failed with "queued for
exclusive access" even though nothing held the gate exclusively. Before
the fairness change those commands ran normally.
Two bounds fix it while keeping the fairness win:
- An exclusive waiter drops the intent lock after maxIntentHold (10s)
without getting the gate and keeps polling unqueued. In-flight ordinary
commands drain far faster than that.
- The final attempt of a waiting shared acquisition tries the gate even
while intent is held, so a shared acquisition fails only when the gate
itself is held exclusively.
With the final attempt ignoring intent, AcquireAll's 1ns floor for an
exhausted budget is dead code (a Wait-0 attempt behaves the same), so it is
back to 0. queuedDetail drops an intent sidecar left by a dead process (on
Windows its removal can fail while another process is reading it).
Tests: the doomed-waiter scenario (both bounds; each fails with its bound
removed), intent lock order across a two-gate set in both directions, an
exhausted AcquireAll budget, two exclusive waiters handing off the intent,
and BEADS_GATE_SHARED_HOLDER_PID inherited through a real subprocess.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(gate): 30s shared wait, 60s init wait, accurate errors, hook pinning
Review follow-ups for the shared gate wait (S1-S5 and nits), on top of the
workspacegate fix that keeps a queued exclusive waiter from failing
ordinary commands.
- Ordinary commands wait up to 30s by default (was 15s). Queued
maintenance operations run one after another (two inits fanned out
across shared-server projects hold the gate back to back, 10-15s each
under load) and a waiting command waits for all of them. Documented in
configuration.md and CHANGELOG, including BEADS_GATE_WAIT_TIMEOUT=0 for
scripts that relied on fail-fast.
- bd init waits up to 60s by default (was 30s). Safe only now that a
waiting init stops queueing others after 10s.
- Error text: a shared acquisition now fails only when the gate is held
exclusively, so "running" is accurate; in shared-server mode it says
"the shared server (possibly from another project)", and the detail
still names the holder when known.
- hooks.go: the hookSubprocessEnv / bd import comments listed BD_GIT_HOOK's
consumers without the gate wait, which now reads it to stay fail-fast.
TestHookSubprocessEnvKeepsGateFailFast pins that hook subprocesses carry
it and therefore never wait on the gate.
- bd edit's $EDITOR (which can outlive bd by hours) no longer inherits
BEADS_GATE_SHARED_HOLDER_PID.
- Test hygiene: the cmd/bd test binary never advertises the marker
(in-process command tests run beside tests that spawn bd) and clears any
inherited one in TestMain; the notice-delay test runs in 1s, not 3s.
- TestSharedServerConcurrent's late init keeps a 30s bound so the 60s
default cannot mask a fairness regression.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(issueops): index each step of the per-edge cycle and hierarchy walks
The single-edge cycle and hierarchy probes (AddDependencyInTx, i.e. bd dep
add and apply-batch dep_add items, the domain stack's HasCycle /
ValidateBlockingHierarchy, and apply-batch's end-gate hierarchy check) joined
the recursive frontier against a UNION of dependencies and wisp_dependencies.
Dolt cannot index that derived table, so every recursion step rescanned every
matching edge: on a 50k-issue / 100k-edge database one dep add took minutes
(an apply-batch of 1 create + 2 dep adds ~10 min).
Give each dependency table its own recursive member joined directly on
d.issue_id, so each step is an index lookup, and pin the lookup plan with
JOIN_ORDER/LOOKUP_JOIN hints: without them the Dolt sql-server planner picks
a type-index scan join while table statistics are not yet available (right
after a commit), back to minutes. Same reachable set (UNION distinct merges
the members per step), same edge types, same error paths, and still one
statement per probe.
Adds a "depadd" createbatchequiv scenario with a golden digest recorded from
the pre-change code: every cycle/hierarchy probe over 17 nodes x 3 table
sets (mixed tables, every target column, external and missing ids), then 20
AddDependencyInTx calls covering cycles through both tables, hierarchy
conflicts in both directions, self, missing endpoints, type conflict,
external and non-scheduling edges.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(issueops): pin lookup joins in the batched blocked-state recompute
The batched mark/unmark UPDATEs decide membership through the scoped
should-be-blocked union. On the Dolt sql-server its planner intermittently
(depending on when the background statistics refresh lands) turned the
union's issues legs around: it drove them from an index scan of every open
issue through (is_blocked, status) and probed dependencies per row. Over an
import's few thousand uncommitted issues that is ~3 s per leg and 6-10 s per
mark/unmark statement for each 200-id chunk, where the lookup plan takes
~0.1 s; an import of 5000 issues with edges took ~28 min and came close to
the 10 s read timeout. EXPLAIN on the server engine showed the two plans;
two of three identical 2000-issue imports hit the slow one (4-5 min vs 14 s).
Pin each joined leg of the scoped union to drive from the batch's own
dependency rows and look the target/parent up by primary key
(JOIN_ORDER/LOOKUP_JOIN hints, ignored by MySQL). The SQL is otherwise
unchanged, so results are identical (createbatchequiv golden digests,
including is_blocked, pass on both engines) and the statement count is the
same. The unscoped union used by full repair and the doctor count walks
whole tables and keeps the planner's choice.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): pin the hinted graph-walk and blocked-state plans with EXPLAIN
The JOIN_ORDER/LOOKUP_JOIN hints on the per-edge reachability walks and the
batched blocked-state recompute were only pinned textually: the suite passed
with them deleted or naming a wrong alias. TestGraphWalkPlansHonorJoinHints
(embedded) records the statements the production entry points send
(WouldCreateSchedulingCycleInTx, CheckBlockingHierarchyInTx,
RecomputeIsBlockedInTx) through a DBTX wrapper, EXPLAINs exactly those, and
asserts each recursive member is a lookup join keyed by the frontier node and
each joined union leg a primary-key lookup, never an is_blocked/status scan.
Deleting a hint or misspelling its alias fails it.
The test found a remaining planner trap in the ancestor walk: with
"d.type = 'parent-child'" in the join condition, the embedded planner
satisfied the wisp_dependencies member through the type index keyed by the
constant, a lookup returning every parent-child row of the table per
frontier node. The type filter now sits in each member's projection (a row
of another type yields a NULL node, which joins nothing and never equals the
goal), so issue_id is the join's only predicate. The depadd golden, recorded
from the pre-change code, still matches.
Also corrects the hint comments: MySQL honors JOIN_ORDER (the order it forces
is the wanted one) and ignores the unknown LOOKUP_JOIN with a warning.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(createbatchequiv): check run scenarios against the golden only; probe a stored parent cycle
A run scenario (depadd) drives code the create fast paths do not gate, so
its fast-vs-per-row comparison compared a body with itself. Apply it once,
check it against its golden digest (the pre-change code's answers), and fail
if the run recorded nothing.
depadd now plants a parent-child 2-cycle straight into dependencies, as
legacy or merged data can hold even though every write path refuses it, and
probes across it plus three adds touching it, so termination of the walks on
stored cycles is covered. Golden re-recorded from a93498c81 (pre-change
walks); the current code matches it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(issueops): index the waits-for gate's child lookups; waits-for bench fixture
The large-graph benchmark gains a formula-shaped fan-out
(batchfixtures.WaitsForInserts: one spawner, 1000 parent-child children,
20 waiters over every waits-for gate shape) and two shapes over it:
create+waits-for and closing a spawner child (which recomputes every waiter
through the waits-for gate).
It was not a statistics-dependent plan flip but a consistently bad plan:
each "spawner has an open/closed child" EXISTS matched both spawner columns
with one OR, which no index serves, so every correlated evaluation scanned
the edge table. On 50k issues / 100k edges, closing one child took ~39 s
per close on embedded, and on the server create+waits-for hit the 30-minute
read timeout. The gate now splits each EXISTS per spawner column (one per
dependency table and column, each served by its target-column index) and
pins the child lookup by primary key (JOIN_ORDER/LOOKUP_JOIN).
EXISTS(a OR b) is EXISTS(a) OR EXISTS(b), so the result is unchanged. The
template is shared with the unscoped full repair, where a per-row correlated
lookup is the right plan too.
Close spawner child per close: emb 39.5 s -> 3.3 s; create+waits-for emb
2.1 s -> 0.46 s, srv >30 min (read timeout) -> 0.29 s; srv close spawner
child 3.2 s (not reached before).
Adds a "waitsfor" createbatchequiv scenario, golden recorded from
a93498c81 (pre-change gate): issue and wisp spawners with issue and wisp
children, waiters in all five gate shapes, then closes, reopens, a scoped
recompute and the full repair, snapshotting every is_blocked. A mutation
dropping the wisp-spawner legs fails it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(createbatchequiv): compare dependency metadata canonicalized across engines
The embedded engine and the Dolt sql-server serialize a JSON metadata value
differently (key order, spacing), so the waitsfor golden recorded on embedded
failed on the server for its gate metadata alone. Re-serialize each
dependency row's metadata through encoding/json before comparing. Rows with
'{}' metadata are unchanged, so the other goldens keep their digests;
waitsfor is re-recorded from a93498c81 and now passes on both engines.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37272437653
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(cli): guarded close replays molecule auto-close and open-children warning
mc-zndi7.75: runCloseDirectIfRevision/runCloseProxiedIfRevision bypass bd
close's batch architecture entirely (BatchCloseItem carries no per-item
ExpectedVersion), so they also bypassed two state-derived post-close
contracts that the batch route performs for every close:
1. Molecule auto-close (autoCloseCompletedMolecule / its proxied twin
autoCloseProxiedCompletedMolecule). Without replaying it here, a guarded
close of a molecule's last step left the molecule root stranded open
forever.
2. The "warning: closing X with N open child issue(s) still active" line
printed when --force waives the engine's open-children refusal.
Both routes now replay this work after a successful guarded close, exactly
as close.go's batch path and closeProxiedRunPostClose do, on both a real
close and an idempotent re-close (the proxied post-close work runs in its
own unit of work, mirroring autoCloseProxiedCompletedMolecule's existing
batch-route contract).
The third item in mc-zndi7.75 — gating on_close hook firing on Changed — is
deliberately NOT implemented here: hookIssueOperations.Close (embedded) and
recordingIssueUC.CloseIssueChecked (proxied) both fire on_close
unconditionally on success, including an idempotent re-close, by explicit,
independently pinned design in internal/storage/hook_issue_operations_test.go
("close no-op still fires close" — "Do not 'unify' this either") and
internal/storage/uow/notifying_test.go ("a re-close still reports the close"
— "THE BATCH COMPOSITIONS DISAGREE WITH THIS ONE, deliberately"). Only the
batch closers gate on Changed, to stop a replayed teardown batch from
re-running on_close once per item on every pass (ga-2yaqp.1) — a hazard this
single-id guarded close doesn't share. Reusing that gating here would
contradict both pinned invariants rather than fix a bug, so it's documented
in place and left for human review instead of changed unilaterally.
Adds CLI integration tests for both routes: guarded re-close healing a
stranded-open molecule root, and guarded --force close warning on open
children.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): sync cmd/bd BUILD.bazel for the new --if-revision close tests
make bazel-sync (bazel run //:gazelle) after adding
close_if_revision_embedded_test.go and close_if_revision_proxied_test.go.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* feat(httpapi): wire-shape signal — wire_revision, floor header, golden digest, param tokens
S0 of the HTTP role-parity program: give a client a single place to learn
whether this build's wire shape has moved under it, and give CI a gate that
catches an undocumented move.
- GET /v0/beads/context reports wire_revision (bumps only on a non-additive
response-shape change) and min_client_wire_revision. An optional
Bd-Wire-Revision request header lets a client declare the revision it was
built for; checkWireRevision refuses a declared revision below the floor
with 400 invalid_argument/reason=wire_revision_unsupported and a
min_wire_revision field, on every route but /healthz (wireRevisionExempt) —
including the identity handshake, where catching the mismatch is cheapest.
Absent header is today's behavior; malformed value is invalid_value, never
wire_revision_unsupported. The floor lives in Server.minClientWireRevision
(defaulting to the MinClientWireRevision constant) so a test can raise it.
- internal/httpapi/wireshape: a golden digest over every response's every
content-type schema (including application/problem+json) across the whole
spec, keyed by schema+member. TestWireShapeDigest fails closed when a
member's JSON name or type changes without wire_revision bumping to match;
regenerate with `go run ./internal/httpapi/wireshape/cmd/gendigest`.
- GET /v0/beads/issues' `sort` parameter now carries the issues.listSort
capability token, the same convention `capabilities` entries use.
TestNewParameterOnExistingOperationHasABehaviorToken requires every new
parameter on an existing operation to carry one, checked against a
write-once grandfather baseline (testdata/pretoken_parameters.json) of
parameters that predate the rule; TestUntokenedParameterRuleFires proves
the checker itself against a synthetic fixture.
- Docs: engdocs/design/bd-serve-v0.md gets a "The wire-shape signal" section;
CHANGELOG.md gets an Added entry.
Live-mutated and reverted three RED demonstrations during development:
retyping api_version in the spec failed TestWireShapeDigest; short-circuiting
checkWireRevision's floor comparison failed
TestWireRevisionBelowFloorRefusesEveryEnforcedRoute; adding an untokened
query parameter to listIssues failed
TestNewParameterOnExistingOperationHasABehaviorToken. All three now pass
against the unmutated tree.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi): rename issues.listSort to issues.list.sort, harden token rule
Opus review of 12c5144ca (HIGH-1, MEDIUM-5):
- Rename the capability token issues.listSort -> issues.list.sort to match
bd-enterprise's client (httpstore/wire/handshake.go:121, DESIGN.txt S4e),
and widen capabilityToken to accept an optional third segment.
- untokenedParameters() now requires the extracted token to be one
Capabilities() actually serves, not just backtick-shaped text. This
catches real false positives that existed in the spec already
(workapi.DefaultReadyLimit, status.custom) which only shape-matched the
old regex; they're added to the pretoken baseline as genuinely
pre-existing untokened parameters.
- Extend the rule to request body members (new
testdata/pretoken_request_body_members.json baseline) and exempt
brand-new operations via a frozen testdata/pretoken_operations.json
snapshot.
- Add TestPretokenBaselinesNeverGrow pinning all three baseline sizes so
a future change can't hand-add an exemption instead of a token.
- Update spec prose for the widened rule scope.
go test ./internal/httpapi/... and the reviewer's pmut.py all pass as
expected: plain/bogus modes go RED, reuse (real-but-unrelated token) stays
green, matching the review's literal ask ("a token must be one advertised
by Capabilities()").
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(wireshape): close four digest-coverage gaps, add gendigest write guard
Opus review of 12c5144ca (HIGH-2, MEDIUM-4). The digest only tracked
Type/Format/Enum/Required on direct properties, missing:
- nullable: true (never read at all)
- array item shape (Issue.labels items string->integer was invisible;
only object-shaped items got recursed into)
- additionalProperties value shape (IssueCount.groups)
- request bodies entirely (Compute() only walked responses, so
DeleteIssuesRequest.expected_version string->integer, the other half
of #6053, was invisible)
Also added defensive oneOf flattening alongside the existing allOf merge
(dormant today -- grep confirms this spec uses oneOf nowhere -- but no
longer silently invisible if that changes).
Extracted the golden-vs-computed comparison into wireshape.Compare(),
shared by TestWireShapeDigest and the new wireshape.SafeToWrite(), which
gendigest now calls before writing: it refuses to overwrite golden.json
with changed/removed entries unless the candidate's wire_revision is
strictly greater than the existing golden's. Pure additions always write.
TestSafeToWrite exercises both guard directions.
golden.json is regenerated. This single regen is a one-time exception to
the new guard, done by writing the digest directly rather than through
gendigest: confirmed via a throwaway diff (not checked in) that every one
of the 57 "changed" entries differs only in the NEW fields the wider walk
now populates (ItemType/ItemFormat/Nullable/AdditionalProps*) -- every
pre-existing Type/Format/Required/Enum value is byte-identical. Nothing on
the wire changed; the measurement got more precise. Bumping
CurrentWireRevision for this would have written a false "the protocol
changed here" entry into the revision history table. Going forward, any
REAL change the guard now reports uses the normal bump-then-regenerate
path, same as before.
Verified against the reviewer's mut.py: nested_type, nullable,
nullable_issue, array_items, addlprops, request_body and format all go RED
on TestWireShapeDigest, each reverted after confirming.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi): make bd_version wire_revision inference advisory
Opus review of 12c5144ca (HIGH-3). The spec told a client inferring
wire_revision from bd_version to use the release cutoff >= 1.3.0, and said
nothing about what happens when that inference is wrong -- which it
demonstrably is for real builds already in the field:
- main builds up to commit b3ef65c85 report bd_version 1.2.2 (below any
1.3.0 cutoff) while already sending the string-typed revision/
expected_version shape (#6053).
- 1.3.0-rc.N pre-release builds sort BELOW the release cutoff 1.3.0 under
plain string comparison despite already carrying the string shape.
- The bd-enterprise compatibility line at d3ab32773462 reports bd_version
1.1.0 while also sending strings.
Moved the cutoff to >= 1.3.0-rc.1 (compared as semver-with-prerelease,
which at least captures the rc builds), documented both named exceptions
inline, and made the whole inference explicitly ADVISORY: a client must
accept revision/expected_version as either a string or a number whenever
wire_revision is absent (or probe directly) rather than let the inferred
value choose a decoder, and must never refuse solely because the inferred
revision is 0. Mirrored into engdocs/design/bd-serve-v0.md.
This is spec/doc prose only -- there is no server-side inference code to
change, since this server always sends wire_revision directly; the
inference is guidance for clients reading a pre-signal server's response
(this server, past builds, or an unrelated implementation).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi): LOW-7/8/9/10 review follow-ups for wire-revision signal
LOW-7: WireRevisionUnsupported now also sets Problem.wire_revision (this
server's CurrentWireRevision) and Problem.bd_version on the
wire_revision_unsupported refusal, alongside the existing
min_wire_revision, so a client logging the refusal has this server's
full self-description without a second /v0/beads/context call.
LOW-8: gazelle + make bazel-sync add the BUILD.bazel entries the prior
commits were missing: wire_revision.go/wire_revision_test.go in
internal/httpapi's srcs, new BUILD.bazel files for wireshape and
wireshape/cmd/gendigest, and a testdata glob on httpapi's go_test so
the pretoken_*.json baselines are visible under `bazel test`. Verified
with `bazel build //internal/httpapi/...` and
`bazel test //internal/httpapi/... --test_env=BEADS_TEST_SKIP=dolt`
(both green) and `python3 tools/bazel/go_srcs.py --check` (clean).
LOW-9: narrowed CHANGELOG.md and engdocs/design/bd-serve-v0.md to
describe the digest's actual final coverage (format/enum/required/
nullable, array items, additionalProperties, request bodies, $ref/
allOf/oneOf recursion, the SafeToWrite revision-bump guard) and the
token rule's actual final shape (three pinned baselines, request-body
members, Capabilities()-membership validation, new-operation
exemption) instead of the original, narrower claims.
LOW-10: Bd-Wire-Revision now parses via a strict decimal-only
parseStrictNonNegativeInt (no leading '+', no leading zero except a
bare "0") instead of strconv.Atoi alone, so two different header
bytes can no longer name the same revision. New malformed-header test
cases: "+1", "01", "007".
Regenerated apigen/types.gen.go (make api-gen) for the new Problem
fields and wireshape/testdata/golden.json (go run
.../wireshape/cmd/gendigest) for the two new additive Problem members;
the latter wrote cleanly through SafeToWrite itself, since both are
pure additions.
go test ./... and bazel test //internal/httpapi/... both green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi): re-review LOW follow-ups — retirement test, gendigest -init, docs
1. TestRetiredRevisionsStayBelowTheFloor pins the PRESENCE guarantee the
wire_revision property doc relies on: CurrentWireRevision >= 2 (so 0
and 1 stay permanently retired) and MinClientWireRevision <=
CurrentWireRevision (so this build's own floor never refuses every
request against itself).
2. gendigest now refuses to create a golden.json that does not exist yet
unless run with -init. Previously a missing file (moved testdata,
wrong path, accidental delete) was indistinguishable from a real first
run and silently wrote a fresh golden with no revision history checked
against anything. writeGolden() is extracted out of main() so this is
covered directly: TestWriteGoldenRequiresInitWhenMissing,
TestWriteGoldenInitCreatesWhenMissing,
TestWriteGoldenIgnoresInitWhenGoldenExists (the existing-golden path is
still governed only by SafeToWrite, -init or not).
3. Narrowed the two remaining stale claims that still described the
digest's original, narrower coverage: the wire_revision property's
prose in openapi.v0.yaml (regenerated into apigen/types.gen.go via
make api-gen) and wireshape/doc.go now both list nullable, array item
shape, additionalProperties value shape, request bodies, and
$ref/allOf/oneOf recursion, plus the new -init guard.
go test ./internal/httpapi/... and make api-check both green;
bazel test //internal/httpapi/... (httpapi, wireshape, and the new
gendigest test target) green; python3 tools/bazel/go_srcs.py --check
clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(wireshape): cover operation parameters in the wire-shape digest
The digest previously walked only response and request-body schemas, so a
type, enum, required, or removal change on one of the spec's 138 operation
parameters (123 query, 14 path, 1 header) passed the gate silently — the
open item from PR review on #7182.
Each parameter is now recorded as its own Entry, keyed by
"param:"+operationId / location+":"+name (operation-scoped, not
schema-scoped, since two operations' same-named parameter are independent
wire contracts), capturing type, array item shape, enum, required, style,
explode and default. Adding a parameter stays additive; removing,
retyping, narrowing its enum, or changing required/style/explode/default
now trips SafeToWrite exactly like a response or request-body change.
Verified against the live spec: retyping listIssues' `limit` query
parameter from integer to string now fails TestWireShapeDigest (confirmed
red, then reverted — not part of this commit). golden.json regenerated via
gendigest; the new parameter entries are a pure addition, no wire_revision
bump needed. Added TestParameterMutationsAreCaught (remove/retype/narrow-
enum against real golden entries) and updated wireshape's doc.go, the
`wire_revision` spec prose, and the CHANGELOG to describe the new
coverage.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(wireshape): review fixes — stale-golden and lowered-revision arms, effective param serialization, digest boundary
Maintainer fixup from the adopt-pr review loop (iterations 1 and 2), on
top of e17ba924f.
- goldenDrift (A, I): TestWireShapeDigest's comparison moves into a
goldenDrift helper. An addition alongside a moved wire_revision no
longer passes with a stale golden: it asks for a regenerate, as a
changed or removed entry after a bump already did. Every message now
names response/request-body members AND parameters, and TestGoldenDrift
pins each arm. This takes only the test half of the stranded fixup
(adopt-pr/bd-47kdge/stranded-fixup-528cd6095); its doc half, which says
parameters are outside the digest, is false since e17ba924f and is not
replayed (J).
- Lowered wire_revision (D): a CurrentWireRevision LOWER than the
golden's is a wrong constant, not a stale golden, so goldenDrift says to
restore it instead of regenerating, and SafeToWrite (so gendigest)
refuses any write at a lower revision, whatever the entries.
- Effective style/explode (B): walkParam records style and explode with
OpenAPI 3.0's defaults filled in (form for query/cookie, simple for
path/header; explode defaults to style == form). Adding `explode: false`
to a query array that relied on form's implicit explode is now a
changed entry, while spelling out a default changes nothing.
testdata/golden.json is regenerated with -init: 111 parameter entries
change only in style/explode (96 to form/true, 15 to simple), 828
entries before and after, wire_revision stays 2. The golden is new in
this PR and unreleased, so this needs no bump.
- TestWalkParam (H): a package-internal table test
(wireshape_internal_test.go) pins every field walkParam extracts,
through a computeFrom(document, revision) seam that Compute now wraps:
per-location defaults, $ref'd parameters and items, format, nullable and
default, content parameters, and the nil returns.
TestEffectiveSerializationIsCompared pins B end to end through Compare
and SafeToWrite. BUILD.bazel embeds the library in the go_test, which is
gazelle's own output.
- Required inputs and the digest boundary (C, E, F; docs only): all five
prose surfaces (openapi.v0.yaml's wire_revision description and the
regenerated apigen/types.gen.go, wireshape/doc.go, CHANGELOG.md,
engdocs/design/bd-serve-v0.md) now say that a new REQUIRED request
input is recorded as an ordinary addition and needs its own review,
that value constraints and member-level composition are outside the
digest, and that object and content parameters are recorded only as
their container. bd-serve-v0.md also gains the parameter coverage it
was missing.
- G: a brand-new operation is permanently exempt from the
capability-token rule; its own review decides its capability story.
- Iteration 2 (nit): walkParam's comment now says that the location
default it records as a `content` parameter's style and explode is a
placeholder, since OpenAPI 3.0 applies style and explode only to a
parameter described by `schema`. All 138 operation parameters in the
spec use `schema`, so the golden does not change.
Validated: go test ./internal/httpapi/... (gms_pure_go), make api-check,
go vet for linux/windows/darwin, gofmt, git diff --check, gazelle on the
package (no drift), make bazel-sync-check, and bazel test
//internal/httpapi/wireshape/... in the local processwrapper sandbox
(2 of 2 pass). Overlay mutation probes: dropping the added-after-bump
arm, the lowered-revision arm, SafeToWrite's lowered check, or the
effective-default fill each turns the new tests red.
Iteration 2 changes only that comment; go test
./internal/httpapi/wireshape/... (gms_pure_go), go vet, gofmt and git
diff --check pass again.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(autofix): push bot fixes with a scoped GitHub App token
github-actions[bot] pushes left PR checks action_required instead of
retriggering, and the DOCS_AUTOFIX_TOKEN fallback secret was never
configured. bazel-autofix.yml and docs-autofix.yml now mint a short-lived,
repo-scoped (contents:write) installation token for the gastownhall-autofix
App via the `autofix` environment on same-repo PRs, falling back to the
default workflow token (and its manual re-run note) when the App token
isn't available; fork PRs are unaffected and still only get the apply-recipe
comment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): job environment parses as a name or a mapping; gofmt
ciWorkflowJob.Environment was a string, so every test that loads all
workflows failed on deploy-pages-redirect.yml's mapping form
(environment: {name, url}).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cli): --if-revision outranks the reassign fence; recognize bare ErrVersionMismatch
Fixes two post-merge review findings on #7203 (max review 37acca96db):
mc-zndi7.74: a lost --if-revision race on an assignee edit reported the
bd-98s5c live-claim reassign fence's policy refusal (exit 1, no machine
code) instead of precondition_failed (exit 13), because update.go,
update_proxied_server.go, assign.go and mutate_proxied_server.go each ran
an advisory pre-read fence check against their own earlier issue snapshot
before the properly-ordered guarded write underneath (which already checks
ExpectedVersion before the fence) got a chance to run. Added
ifRevisionAlreadyStale to skip the advisory check when the caller's own
pre-read is already stale against --if-revision, letting the guarded write
report the correct outcome.
mc-zndi7.78: classifyIfRevisionFailure recognized a revision mismatch only
via the typed *issueops.VersionMismatchError (errors.As). bd-enterprise's
HTTP client maps a 409 precondition_failed to the bare
issueops.ErrVersionMismatch sentinel instead, which bypassed the classifier
and surfaced as a generic error. Added an errors.Is fallback that reports
precondition_failed with expected_revision from the --if-revision flag
value and current_revision omitted (gascity tolerates the omission).
New regression tests (embedded + proxied routes) reproduce the .74 race
deterministically: claim an issue as a foreign holder to advance its
revision past a caller's stale --if-revision, then assert the guarded
write reports precondition_failed rather than falling through to the
live-claim refusal naming the holder. Both fail on an unpatched
git-archive copy of origin/main and pass on this branch.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37258672420
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
TestMaybeAutoBackup_WalkErrorArmsIntervalThrottle pinned the walk-error
re-arm in maybeAutoBackup by forcing runBackupExport's old post-sync
GetCurrentCommit call to fail. #7044 (874e2866f) collapsed that into a
single pre-sync CurrentCommit read, so the forced failure point no
longer exists and the test failed deterministically on every run.
Swap the failure point for one that still exists after #7044: the sync
itself succeeds (still proving fail-open — BackupDatabase gets called),
but the destination goes read-only immediately afterward, so
runBackupExport's post-sync saveBackupState (the call that would record
the new watermark) fails and never reaches disk. The only persisted
watermark write left standing is maybeAutoBackup's own walk-error
re-arm, so the test is still mutation-sensitive to removing it
(verified locally: deleting the re-arm reproduces the old failure).
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
pr.yml and pr-risk.yml (and the three advisory workflows that already
list release/** beside it -- conformance.yml, cross-version-smoke.yml,
regression.yml) trigger pull_request only on main and release/**. A PR
based on hotfix/1.3.1 (or any sibling hotfix/** line) matched neither
pattern, so it ran almost no CI: #7146, a backport to hotfix/1.3.1,
showed 17 checks -- 15 cross-version historical smokes and triage/label
jobs, 2 skipped -- no Lint, no unit shards, no embedded tier, no CI
Gate / Required, no PR Risk Gate / Required.
pr.yml's own comment states exactly why release/** is there:
release-prep and release-fix PRs are based on the release branch, not
main, and without it they run almost no CI before the tag build. A
hotfix line is in the identical position. Added hotfix/** alongside
release/** in all five files.
New TestPullRequestWorkflowsTriggerOnHotfixBranches pins hotfix/**
presence in each file's pull_request.branches list -- red before this
change (observed [main release/**] in all five), green after. The
existing TestPRRiskBazelCoverageJob equality check (pr.yml and
pr-risk.yml's pull_request triggers must match) still passes, since
both were updated identically. go vet/gofmt clean.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Record the commit read before BackupToDir as the newest revision guaranteed
to be in the snapshot. A commit landing during sync then remains pending for
the next backup instead of being hidden by an advanced watermark.
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Rongjun GENG
Propagate sweep protection through closed candidates so an open descendant
keeps every candidate ancestor in its protecting dependency chain. Cover the
shared helper, dry-run, real sweep, and all storage contract wirings.
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Rongjun GENG
TestResetCommandFlagsRestoresEveryFlag read its expected values from the
live flag variables. The root pre-run fills some of those from config
without marking the flag Changed (doltAutoCommit, main.go:1224), so when
an in-process command ran earlier in the binary the baseline for
--dolt-auto-commit was "on", and the reset under test correctly put it
back to "". PR Core (wrapper timing) has failed on that assertion on
every fork PR since #7185.
The test now calls resetCommandFlags before taking its baseline.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Follow-up to #6380 (bd kv list --prefix). Both capability-discovery hops
fall back silently, and both modes re-filter in-process with
kvPairsWithPrefix, so a lost SQL-side fast path printed the same rows and
failed no existing test.
- Extract wireProxiedUOWProvider, the unit-of-work twin of
wireStorageDecorators, so the proxied provider chain that main.go builds
is unit-testable; main.go now calls it. No behavior change.
- TestKVListProxiedPrefixReadSurvivesProviderChain drives
runKVListProxiedServer through that chain (hook runner on and off) over a
prefix-capable stub use case and fails on any GetAllConfig scan;
TestKVListProxiedPrefixAbsentCapabilityFallsBack keeps it honest.
- TestEmbeddedKV/kv_list_prefix_json and
TestProxiedServerKV/list_prefix_json run the real command with sibling
keys (mail.dog.* vs mail.doggerel.*) and an unmatched prefix.
- bdKVListJSON now fails on output with no JSON object instead of returning
an empty map, so a "no pairs" check cannot pass vacuously.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* perf(storage): cut batch-create round trips (import, bulk create, apply-batch)
A multi-issue create (import, bulk create, CreateIssuesWithFullOptions) ran
~20 SQL statements per issue, and its dependency pass ran a recursive cycle
CTE per edge over the union of both dependency tables, which Dolt cannot
index — every recursion step rescanned the edge tables, so an import whose
edges form chains went superlinear (400 issues: 73 s of 82 s in that CTE).
issueops batch create, same transaction and same stored outcome:
- createBatchCache: one IN-list read of every explicit id's presence in both
planes (replacing two COUNT probes per issue), one read of the batch's
stored labels (each issue's new labels then land in one multi-row INSERT),
created/label_added events buffered and written in one pass under the ids
InsertDerivedEvent would mint (same digest, same lowest free ordinal), and
runs of brand-new rows written with multi-row INSERTs (same ODKU clause)
then finished in order.
- Dependency pass: wisp routing and target presence from batch reads; the
cycle and hierarchy checks walk an in-memory copy of the edge tables,
loaded once and kept in step with every edge the pass writes (a pair it
already holds is re-read, since a clientFoundRows connection reports a
matched duplicate as written).
- createBlockedRecomputeIDs seeds all created edges and expands once.
Shared paths:
- Blocked-state recompute splits ids once per call by whether they own a
dependency row; edgeless ids (every freshly created issue) skip the two
union UPDATEs for one plain unmark, run after the chunk's union statements
on the first pass so the unions see what they saw before.
- appendChildrenInTx reads children per IN-list chunk, not per parent.
- ExecuteCreate hands its BatchContext to the batch body instead of having
it re-read the same config (-6 statements per apply-batch create item).
Not changed: one version-history row per created issue (the version
contract, version_history.go), journal order and content, and the per-row
bodies, which stay the path for singular creates.
Equivalence: internal/storage/createbatchequiv runs an import scenario
(upserts, duplicate ids, wisps, generated ids, cycles, hierarchy conflicts,
dangling/external targets, stale is_blocked, comments, journal and
versioned history on) through the fast and per-row bodies on the embedded
and server engines and compares every written table and the skip reports.
Re-pinned TestLargeBatchApplyStatementCounts*: embedded 7009/14014/846 ->
6396/12790/793, server 7008/14014/846 -> 6396/12791/793 (breakdown in
pinnedEmbeddedStatementCounts).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): batch-create review fixes — lazy edge graph, opt-in recompute probe, insert budget
Review of the batch-create round-trip change (S1-S3 and nits):
- The dependency pass's in-memory graph no longer loads both edge tables.
It reads a node's outgoing edges through the issue_id index the first
time a walk reaches it: the batch's own endpoints up front, then one
IN-list read per BFS level. Memory and reads scale with the subgraph
the batch's walks touch, not the edge tables, so a two-edge create on a
large database no longer materialises every edge. A written edge whose
source was never loaded loads that source (row included).
- The blocked-state no-edge probe runs only when a create asks for it or
when a chunk holds at least 32 ids. Single-issue recomputes on close,
reopen, update and dependency edits issue exactly the statements they
did on main (the reopen sqlmock expectation is back to main's).
- Multi-row issue INSERTs are bounded by an estimated 16 MiB of bound
text as well as 100 rows. When a multi-row statement fails, its rows are
replayed one at a time, and the batch continues if every row lands
instead of returning the original error.
- Nits: document the binary-collation assumption behind Go string
equality; DisableCreateFastPathsForTest documents that it is
process-global and non-parallel, and panics on an overlapping caller;
drop the unreachable skip branch; explain why wisp targets need no
presence read.
Re-pinned TestLargeBatchApplyStatementCounts* classic 40: 793 -> 786
(the +7 probes on dep-add recomputes are gone; net vs main is -60, all of
it ExecuteCreate's config reuse). 356/712 unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): search the batch dependency graph from both ends
The lazy per-node load (previous commit) still walked forward from the
check's start, so on a large stored graph a single create's cycle check
read every node reachable from its target — about 800 statements per
create on a 2000-issue fixture. The cycle and hierarchy questions a batch
asks almost always end at a row the batch just created, whose only
incoming edges are the batch's own.
depGraph now answers each reachability question with a bidirectional
search: forward from the start along outgoing edges (issue_id index),
backward from the goal along incoming edges (the three target-column
indexes, read as one UNION ALL per table), always expanding the smaller
frontier and loading a level's nodes in that direction once. The batch's
endpoints are loaded in both directions up front, so a create's checks
usually finish with no further read (a chain import, too: each new
edge's goal has no incoming edge yet). Rows are keyed by (table, source,
target), so a row read from both ends counts once. An incoming read
ignores a row whose matching column is not its first set target column,
as the CTE's COALESCE does. A written pair that neither loaded end can
vouch for is re-read instead of assumed new.
On a 50k-issue / 100k-edge database, a create with two blocks edges
now runs 27 statements, against the full-scan or per-node walks before
(numbers in the review-fix summary).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): hold batch create to golden digests from the pre-change code
The batch-create equivalence test compared the fast paths with the same
code's per-row bodies, so changes the fast-path switch does not gate —
the recompute seed merge, the IN-list children read, ExecuteCreate's
context reuse, the prepare/finish split — were checked only by older
contract tests. Its batch was also too small to cross a chunk boundary
and never set RejectStaleUpserts.
createbatchequiv now runs four scenarios, each through the fast and
per-row bodies, and checks both against a per-table row count and
SHA-256 recorded from ebe3b6bcb (before the fast paths existed;
CREATEBATCHEQUIV_GOLDEN_DIR re-records them):
- small: the original scenario;
- import458 and import458-reject-stale: the review's 458-issue batch
(every chunk boundary, wisps and cross-plane edges, cycles, hierarchy
conflicts, duplicate ids across deferred runs, generated and
hierarchical ids, stale upserts rejected and accepted), plus new edges
whose blocked-state fallout reaches stored rows outside the batch (a
spawner's waiter, an upserted parent's existing child);
- apply: an apply-batch through ExecuteCreate, dep adds and an update.
The recorded digests match on the embedded and server engines (the
server test now sets the fixture's prefix, so generated ids agree).
Dropping the spawner-waiter seeding is invisible to the fast-vs-per-row
comparison and caught by the golden check.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): large-graph create benchmark
batchbench measures creates against a database already holding a large
dependency graph (batchfixtures.LargeGraphInserts: N issues, ~2N
acyclic edges with wide reachable sets and a deep parent-child forest,
committed as a Dolt version): ExecuteCreate under an existing parent with
an existing blocker, a store create with two blocks edges, and an
apply-batch of creates each with a parent-child and a blocks dep_add.
It logs wall time, statements and Go allocation per shape on the
embedded and server engines. Opt in with BEADS_BENCH_LARGE_GRAPH=<issues>
(and BEADS_BENCH_LARGE_GRAPH_CREATES); it asserts nothing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): read incoming dependency edges one target column at a time
The bidirectional graph search read a node's incoming edges with one
UNION ALL over the three target columns per table. Dolt plans that as a
scan: 0.7 s per read on a 100k-edge table, where each leg alone is a 1 ms
lookup on its own index (EXPLAIN shows IndexedTableAccess per leg). Issue
the three legs as separate reads into one row set; a row matched through
two columns still counts once.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): run the 458-issue equivalence scenarios outside the race lane
The 458-issue golden scenarios took the embedded tier's shard over its
20-minute limit under -race (the in-process engine's own instrumentation;
about 35 s each without it). They move to
TestCreateBatchFastPathsMatchPerRowLarge_Embedded, skipped under -race and
added to nightly.yml's non-race embedded step, following the precedent of
the 712-item statement-count and 1000-item wall-clock cases. The light
scenarios (small, apply) stay in TestCreateBatchFastPathsMatchPerRow_Embedded
on every lane. The server backend (built without -race in its full
suite) runs both: TestCreateBatchFastPathsMatchPerRow_Dolt and, as its own
top-level test so the shard manifest's hash places it on another shard
(about four minutes there), TestCreateBatchFastPathsMatchPerRowLarge_Dolt.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(cli): add --if-revision compare-and-swap guard to update/close/assign/delete
Adds --if-revision <int64> to `bd update`, `bd close`, `bd assign` and
`bd delete`, reusing the existing opaque row_lock token published as
IssueDetails.Revision and the library's own ExpectedVersion compare-and-swap
support. A mismatch writes nothing and exits 13 (ExitGuardMismatch); with
--json the stderr envelope carries integer expected_revision/current_revision
fields deployed callers decode directly, never a quoted string. An
unsupported backend reports conditional_write_unsupported and exits 1
instead of silently dropping the guard. The guard composes with
--if-assignee/--if-status (every guard must hold) and is refused alongside
label or parent edits and alongside more than one id, matching the existing
--if-assignee rules. `bd assign` is newly routed through the checked
Lifecycle path to gain CAS support it previously had none of at all.
Implements Item 4 (compare-and-swap through the CLI, gastownhall/beads#4682)
from the BEADS-JOURNAL-PLAN.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* chore(bazel): regenerate cmd/bd BUILD.bazel for if_revision/close_if_revision
gazelle hadn't picked up if_revision.go, close_if_revision.go,
if_revision_test.go, and if_revision_embedded_test.go, so Bazel's bd_lib
and bd_test targets failed to build with "undefined: parseIfRevisionFlag"
and friends while plain `go build`/`go test` worked fine. Ran `make
bazel-sync` (bazel run //:gazelle; go_srcs.py; bazel mod tidy) to pick up
the new files; only cmd/bd/BUILD.bazel's srcs lists changed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(cli): update update-guard parity strings for --if-revision
TestParityUpdateGuardsRequireFieldUpdate and TestParityUpdateGuardsRejectClaim
pinned the exact stderr text of update.go's pre-write guard short-circuits.
Adding --if-revision to those guard messages (update.go/update_input.go)
changed "--if-assignee/--if-status" to "--if-assignee/--if-status/--if-revision"
in both, which this PR's own change to the guard text had not updated here —
caught by CI's dolt-server tier, not by the targeted if_revision* test runs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): zstd cache transfers for the anonymous fork cache only, behind RBE_CACHE_ZSTD (E4)
E4 of the rbe-west egress reduction (design A.5), reworked for the
read-only zstd server shape. rbe-west's trusted MAIN and OSS schedulers
(:443 to :50052) and rbe-fork (:8444) advertise no compressor, and Bazel
refuses a remote that does not advertise zstd. So only the anonymous fork
cache (rbe-cache :8443, nativelink-anon, which keeps zstd) may ask for it.
- **`write-bazelrc.sh`:** no `--remote_cache_compression` on the trusted
remote-exec or rbe-fork paths. In mode cache, with `RBE_CACHE_ZSTD=1`, it
adds `build:fork-cache --remote_cache_compression`. Empty or 0 adds
nothing; another value warns and stays off, rather than failing every
cache lane.
- **`bazel.yml`:** every setup-bazel caller passes
`RBE_CACHE_ZSTD: ${{ needs.rbe.outputs.mode == 'cache' &&
vars.RBE_CACHE_ZSTD || '' }}`, so only cache runs see the variable.
Merging changes nothing, and rollback is the variable, with no revert.
- **`.bazelrc`:** fork-cache no longer sets the flag; a comment points at
the gate.
Tests:
- `TestBazelCacheModeReachesTheRC` evaluates `vars.` right-hand sides and
simulates `RBE_CACHE_ZSTD=1`. Mode cache's rc carries exactly the
fork-cache zstd line; the remote, fork-ro, fork-rw and local modes carry
no compression at all.
- `TestSetupBazelRCWriter` runs `RBE_CACHE_ZSTD` = 1, 0, empty and a bad
value in mode cache, and remote-exec with 1 (never compressed).
`TestSetupBazelRCWriterForkMode` pins rbe-fork's exact lines with
`RBE_CACHE_ZSTD=1`.
- The bazel.yml setup-env pin includes `RBE_CACHE_ZSTD`.
- `checkBazelrcForkCache` rejects the flag in any `.bazelrc` config, with
fork-cache and remote-exec fixtures.
- New `TestBazelRemoteCacheCompressionOnlyForkCache` scans `.bazelrc`, the
workflows and the local action files. The flag must appear once: the
writer's fork-cache echo.
**Merge order:** after rbe-west's anonymous process advertises zstd. Even
then the flag stays off until `RBE_CACHE_ZSTD=1` is set. If rbe-cache stops
advertising zstd, fork-cache's fallback flags make lanes build locally.
**Action keys:** unchanged; the flag only changes ByteStream resource
names (`compressed-blobs/zstd/`).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): zstd for the fork cache by a GetCapabilities probe, not a repository variable (E4)
Review finding 3: GitHub gives fork pull_request runs no repository
variables (bazel.yml: forks "see no secrets and no vars"), so
vars.RBE_CACHE_ZSTD was always empty exactly where the switch matters. Fork
run 37220594372 logs RBE_VAR_ON: false. The variable could never turn zstd
on for fork-cache clients, and once on it could not follow rbe-cache
rolling zstd back.
- New setup-bazel/cache-zstd-probe.sh, a byte copy of gascity's
tools/rbe/cache-zstd-probe.sh: one unary REAPI GetCapabilities call
(instance oss) to rbe-cache :8443 over curl's HTTP/2, decoded with
python3's standard library. It exits 0 only when
cache_capabilities.supported_compressors lists ZSTD, the field Bazel
checks. Any other result exits nonzero: not listed, an HTTP or gRPC
error, a refused connection, a timeout (connect 3s, total 5s), an
unparsable answer, or no curl or python3. It writes nothing to stdout
and one verdict line to stderr. No new action and no new dependency.
- write-bazelrc.sh: in mode cache, `if bash .../cache-zstd-probe.sh` adds
build:fork-cache --remote_cache_compression. The probe never fails the
script. RBE_CACHE_ZSTD and its bad-value warning are gone.
- bazel.yml: the ten RBE_CACHE_ZSTD env lines are gone (back to the
pre-E4 setup env).
Tests:
- New TestCacheZstdProbe runs the real script against a TLS HTTP/2
stand-in for rbe-cache, which checks the exact request. Passes: zstd
advertised (packed, and unpacked among others). Fails: rbe-cache's real
2026-10-04 answer (no compressors), deflate only, zstd for batch updates
only, zstd outside cache_capabilities, a gRPC error trailer, no
grpc-status, trailers-only UNIMPLEMENTED, HTTP 502, a compressed or
truncated message, a truncated field, not gRPC, a timeout, a refused
connection, an untrusted certificate. It also pins the probe's endpoint
and instance to .bazelrc's fork-cache.
- TestSetupBazelRCWriter: the fork cache gets exactly the zstd line after
--config=fork-cache against a zstd stand-in, and none against the live
answer or a refused port. Remote-exec and rbe-fork, with a zstd
stand-in, never probe and never carry compression.
- TestBazelCacheModeReachesTheRC again rejects any vars. in setup-bazel's
env, and no mode's rc carries compression (the probe is refused there).
- The writer harnesses default RBE_CACHE_PROBE_URL to a refused loopback
port, so no test reaches rbe-cache.
Live check, 2026-10-04: rbe-cache answers without supported_compressors,
so the writer's mode cache rc has no flag until the E1 cutover.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* wy-1rrwt8: add 'bd kv list --prefix' — SQL-side key-prefix filter for scoped kv reads
Downstream mail transports read the whole kv store (3.4MB, ~1988 mail.*
envelopes on the wyvern fleet) for every inbox render. This adds a --prefix
flag to 'bd kv list' that pushes the filter into SQL on stores that support
it, via an optional domain.ConfigPrefixReader interface discovered by type
assertion — no existing storage interface or test double changes shape, and
every path falls back to GetAllConfig plus in-process filtering.
- issueops: ConfigPrefixLikeQuery + ConfigPrefixPattern (LIKE with explicit
ESCAPE '!' — the engine does not honor backslash as a default escape) and
GetConfigByPrefixInTx.
- domain: ConfigPrefixReader optional interface; use-case impl delegates to
the repo fast path or filters GetAllConfig.
- domain/db, dolt, embeddeddolt: SQL-side GetConfigByPrefix.
- telemetry: instrumented passthrough preserving the capability.
- cmd/bd: --prefix on kv list, both classic and proxied-server paths;
kvPairsWithPrefix re-checks on every path so output is identical whichever
path served the read.
* build(bazel): list the PR's new test files in go_test srcs (make bazel-sync)
kv_prefix_test.go (cmd/bd) and config_prefix_test.go
(internal/storage/issueops) are new, untagged test files. Both packages'
BUILD.bazel files were generated on main after the PR's commit was written,
and the rebase onto main did not re-run gazelle, so neither go_test listed
them: bd_test and issueops_test reported PASSED with "no tests to run" for
TestKVPairsWithPrefix and TestConfigPrefixPattern, and bazel.yml's "BUILD
files in sync" step fails on that drift in the required bazel-test lane.
Generated by `make bazel-sync`, which rewrote only these two rows. Both files
import only the standard library, so no deps change.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(kv): engage the --prefix SQL fast path through the decorator chain
Maintainer review fixup for PR #6380.
The SQL push-down this PR exists to deliver was unreachable on bd's default
direct/embedded path. `bd kv list --prefix` asserted `store.(configPrefixReader)`
on the outermost decorator, but wireStorageDecorators unconditionally interposes
externaldeps.Store and (hooks on, the default) HookFiringStore, both of which
embed the storage.DoltStorage *interface* — and GetConfigByPrefix is
deliberately not declared there. The assertion could never succeed, so every
direct-mode prefix read silently took the GetAllConfig full-table fallback.
Output was identical, so nothing errored, logged, or failed a test, while the
flag help and command Long text asserted the opposite.
This carries the prior review lineage's fixup 9ca3caeb9, stranded when the PR
was relaunched, plus what this review iteration found missing from it:
- cmd/bd: discover the capability through storage.UnwrapStore, the repo's
documented idiom for optional-interface assertions (~50 existing call
sites), in configPrefixReaderFor. The direct-mode read itself is extracted
as readKVListConfig so the regression test drives the command's own read:
with only the helper under test, reverting the call site to the bare
`store.(configPrefixReader)` assertion still left the whole suite green.
- cmd/bd: chain-level regression test over wireStorageDecorators covering all
four telemetry/hooks configurations, plus the negative case proving a store
without the capability is still reported incapable. The stub counts full
reads, so a fallback fails the test even though it returns the same rows.
Verified non-vacuous: a bare assertion in the helper or in
readKVListConfig, a disabled fast path, a dropped kv. prefix, or a method
added only to the mirror interface each fail all four configurations; a
peel that stops at InstrumentedStorage fails the telemetry-on negative case.
- dolt, embeddeddolt, telemetry: compile-time pins against
domain.ConfigPrefixReader, so method-set drift on any implementation is a
build error instead of a silent reversion to full scans. The cmd/bd
mirror's own pin checks one direction only (a method added just to the
mirror still compiles); its comment now says so and names the test that
covers the other direction.
- domain/db: delegate GetConfigByPrefix to issueops.GetConfigByPrefixInTx like
the sibling stores, instead of a second scan loop over the shared query
constant.
- telemetry: record the deliberate trade — peel-to-raw keeps the standard
idiom and leaves the instrumented forwarder off the cmd/bd path.
- bazel: add //internal/storage/domain to the telemetry_test deps (make
bazel-sync); without it the telemetry pin's import fails strict deps.
Co-authored-by: dog <dog@ghosttrack.com>
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: dog <dog@ghosttrack.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(backends): registry Remote + optional OpenWith(ctx, dir, OpenOptions)
Slice S1: backends.Backend gains Remote bool and an optional
OpenWith(ctx, beadsDir, OpenOptions{Credential, HTTPClient, UserAgent}) so a
registered extension backend can honor a per-open credential instead of
dialing with a process-global client — the motivating case is one embedder
process serving many workspaces with distinct credentials. Open keeps
working unmodified for every existing backend: Backend.OpenWithOptions calls
OpenWith when present, otherwise falls back to Open and ignores
HTTPClient/UserAgent, but refuses (ErrCredentialWithoutOpenWith) rather than
silently dropping a non-nil Credential.
Public beads.OpenBestAvailableWith(ctx, beadsDir, opts) carries this through
the SDK surface (mirrored in beads_nocgo.go and aliased in backend/backend.go
and beads.go); OpenBestAvailable is now that call with a zero OpenOptions,
identical to its prior behavior for Dolt, embedded Dolt, and every registered
backend. The plain-Dolt path (no registered backend) applies the same
Credential refusal, since Dolt has no per-open credential seam either.
doltserver.ResolvePhysicalRoots now checks backends.IsRemote(cfg.GetBackend())
before every Dolt-mode branch, including the shared-server check (which reads
only env/config.yaml and never consults cfg) — without this ordering, a global
BEADS_DOLT_SHARED_SERVER=1 would hijack a remote-backend workspace into
reporting a bogus local Dolt root it will never open.
Tests: backends_test new file open_with_test.go covers OpenWithOptions
dispatch/back-compat/Credential refusal and Remote/IsRemote; root package
open_backend_selection_test.go adds a fake OpenWith registrant proving
dispatch and option pass-through through OpenBestAvailableWith, zero-options
back-compat for every path (registered w/ and w/o OpenWith, plain Dolt), and
the Credential-without-OpenWith refusal on both paths; doltserver
physical_root_test.go adds the registered-remote-backend case including the
shared-server-override ordering regression test. Verified under both cgo and
nocgo builds. Live-mutated each new guard (the Credential refusal, the
OpenWith dispatch, the ResolvePhysicalRoots remote-backend case, and the
plain-Dolt Credential refusal) one at a time and confirmed RED before
reverting to GREEN.
No HTTP client code added; that is a later slice.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fixup(backends): address Opus review findings on OpenWith (M1-M3, L1-L2)
- M1: frame cmd/bd store-open errors for remote backends as unreachable
server failures instead of Dolt-shaped "failed to open database"
advice (openStoreErrorMessage, backendNameForErrorFraming), with tests.
- M2: rewrite the OpenBestAvailable/OpenBestAvailableWith back-compat
subtest to assert fixed expected outcomes (error identity/message,
returned store behavior) now that the two functions are identical,
instead of comparing them to each other.
- M3: document and enforce the OpenWith contract: an implementation that
receives a non-nil Credential it cannot type-assert to its expected
type must refuse with the new ErrUnsupportedCredential sentinel,
verified with a fake registrant.
- L1: document that a nil OpenOptions.Credential means the backend's
default/ambient auth, and that multi-tenant embedders must pass a
non-nil Credential (RequireCredential deferred to S2).
- L2: fail closed on HTTPClient/UserAgent passed to a backend without
OpenWith, mirroring the existing Credential refusal, via new
ErrHTTPClientWithoutOpenWith/ErrUserAgentWithoutOpenWith sentinels.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37195230844
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
* fix(backends): keep the store-open error chain through the remote framing
Maintainer review fixes for the adopted PR (iteration 1).
openStoreErrorMessage returned a string, so ensureStoreActiveWithContext
rebuilt its error from text (%s) where base wrapped the open error with %w,
and the chain ended at the CLI's framing. Prime tells a store-open timeout
apart from a generic "storage unavailable" only through
errors.Is(err, context.DeadlineExceeded) on what ensureStoreActiveWithContext
returns (cmd/bd/prime.go), so the flattening silently downgraded that
diagnostic for every backend, Dolt included, not just the new Remote ones.
Replace the helper with openStoreError, which returns an error wrapping err
with %w on both arms. The message text is unchanged: "failed to open
database: <err>" for Dolt and any registered backend that is not Remote,
"failed to reach remote backend %q: <err>" for a Remote one. direct_mode.go
wraps the result with %w ahead of its Hint line; main.go keeps printing it
through HandleError("%v", ...), which only formats.
Tests:
- TestOpenStoreErrorFramesRemoteBackends (renamed from
TestOpenStoreErrorMessageFramesRemoteBackends) now also asserts errors.Is
on every arm, and the exact original text for the non-Remote arms.
- TestEnsureStoreActiveKeepsOpenErrorChain drives the real
ensureStoreActiveWithContext against a registered backend whose open
outlives its context, for a local and a Remote backend, and requires
errors.Is(err, context.DeadlineExceeded) plus the framing prefix and the
Hint line.
- TestFormatMemoriesForPrimeTimesOutOpeningRegisteredBackend is the
unstubbed twin of TestFormatMemoriesForPrimeTimesOutOpeningStore and
requires the timeout banner, not the "storage unavailable" one.
Measured at this head: turning direct_mode.go's outer %w into %v reddens
both chain-test rows and the prime test; turning the helper's %w into %v
reddens the framing test as well.
Non-gating cleanup in the same area:
- beads_cgo.go, beads_nocgo.go: OpenBestAvailableWith's doc and dispatch
comments said only a non-nil Credential is refused. The code also refuses
a non-nil HTTPClient and a non-empty UserAgent, each with its own
ErrXWithoutOpenWith sentinel.
- internal/doltserver/physical_root.go: the Mode doc lists "remote-backend",
and the RemoteBackend doc covers a registered Remote backend as well as a
Dolt server on a non-local host.
- open_backend_selection_test.go: the plain-Dolt refusal test is a table
over Credential, HTTPClient and UserAgent (renamed
TestOpenBestAvailableWithRefusesPerOpenOptionsOnPlainDolt). Disabling the
nocgo HTTPClient arm or the cgo UserAgent arm reddens only that row. The
zero-options test's comment had the delegation backwards and pointed at
beads.go; OpenBestAvailable delegates to OpenBestAvailableWith in
beads_cgo.go and beads_nocgo.go.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
* refactor(doltserver): share dolt's startup-output watcher with the plain server launcher
Move the ready-line / port-in-use watcher that the proxied DoltServer uses
(startupWatch, doltReadyLine, doltPortInUseTexts, logLevelEmitsReadyLine,
ErrPortInUse) into internal/doltserver as StartupWatch and friends, so the
plain server-mode Start can read the same signals. dbproxy/server already
imports internal/doltserver, so it now uses them from there; its
ErrPortInUse is the same error value. No behavior change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(doltserver): prove the auto-started dolt owns its port; move a taken ephemeral port
The plain server-mode Start (bd init --server, auto-start, bd dolt start)
had the same blind readiness check the managed-local proxied backend had
before #7184, plus a blind exit check:
- It picked an ephemeral port, closed the probe listener and launched dolt,
then slept 200ms and asked whether the child was still alive. dolt only
reaches its port check after its own startup, which under load takes
longer than that, so a lost port was never caught there. Worse, the child
had been Release()d unreaped, and a zombie still answers kill(pid, 0), so
an exit was invisible for the rest of the bd process.
- It then waited for any MySQL greeting on the port. If another process
took the port in between, either it was mute (a plain listener) and Start
ran out its ready timeout -- the direct-local-server auto-start timeout seen
under port churn -- or it was another dolt, and Start reported success
against a server it did not start.
Now each attempt launches the child with a goroutine that waits on (and
reaps) it, and awaitOwnedListener accepts a greeting only once the
listener is shown to be the child's: dolt's ready line in the child's
output when its log level emits it (debug), or else the listening socket
belonging to the child's process tree (Linux, via /proc/net/tcp and
/proc/<pid>/fd). The child's output is read from the server log past the
offset it started at, through the StartupWatch shared with dbproxy, so
"Port N already in use." (printed at log_level warning too) ends the
attempt at once as ErrPortInUse, and a child exit ends it as a failure.
Port policy follows #7184's contract: only a port bd chose itself moves.
An ephemeral port that is lost is retried on a fresh one (up to the
existing 10 attempts); an explicitly configured port (env, config.yaml,
metadata, dolt config) stays pinned and fails with ErrPortInUse and a
remedy. Nothing operator-owned is rewritten: the generated
dolt-server-config.yaml is bd's own state file and was already rendered
per attempt. Where ownership cannot be proven (non-Linux, quiet log
level) the greeting decides as before, but the child's exit and
port-in-use report still end the wait.
Tests: a fake dolt (this test binary, behind a PATH shim) that spends
700ms starting up and then binds like dolt does. A foreign greeter on the
port is not accepted (ErrPortInUse); the child's own listener is; a child
exit is reported promptly; and the real Start, handed a taken first
ephemeral port, comes up on a fresh one (fails before this change: Start
adopted the foreign port).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(doltserver): isolate user config in the DefaultConfig tests
Four DefaultConfig tests failed under -tags integration on a host whose
user config sets dolt.shared-server: true. The integration TestMain, unlike
the plain one, does not redirect HOME, so:
- TestDefaultConfigPrecedenceChain and TestDefaultConfigPortSource call
config.Initialize, which merged ~/.beads/config.yaml.
- TestDefaultConfigReturnsZeroForStandalone and
TestDefaultConfigPortFileTakesPrecedence read whatever config an earlier
test had loaded from the real HOME.
Shared mode then sent DefaultConfig to the shared server's state dir and
its port file ("port=3308 source=port_file, want 5002/config_yaml").
isolateUserConfig points HOME, USERPROFILE and XDG_CONFIG_HOME at an empty
dir, the way TestDefaultConfig_NonzeroPortAlwaysHasSource already does.
It also resets the loaded config before the test and again after it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(doltserver): auto-start review fixes: record the server at launch, sharper ownership, port policy by source
Follow-up to 0acb1041b, addressing review.
B1: write the PID and port files again right after each launch, as before
that commit, and remove them on every failure path (port conflict,
not-ready, remotesapi). Previously a bd interrupted during the readiness
wait left a dolt holding the database lock that nothing could find. The
next start, bd dolt stop and killall all saw nothing. Both files carry the
real port, so IsRunning never stops a starting server as "port unknown".
S1: a child gone from /proc is known not to own the port; it is no longer
"unknown", which would let a foreign greeting through. The child's log is
re-read on every check, not only after its exit has been observed, because
the reaper goroutine can lag the death.
S2: if no listener on the port is listed although one just answered, /proc
is incomplete (WSL1, some sandboxes, forwarding). That is now "unknown"
(fallback to the greeting), not "foreign". Also "unknown": a /proc that is
not this PID namespace's.
S3:
- bd's port-file port moves like an ephemeral one outside shared-server
mode, since bd chose it (GH#4052).
- Operator-configured ports and the shared-server port stay pinned.
- The pinned error now says where the port came from (env, config.yaml,
dolt config, metadata, --server-port, shared default, shared port file)
instead of always "configured explicitly".
S4: a killed child is waited for (up to 5s) before the next attempt or the
return, so it has released the database lock. Its process group is killed
too (unix), which reaches a dolt behind a non-exec wrapper.
S5: when /proc proves the listener belongs to another process, the wait
returns ErrPortInUse at once. It no longer sits out a slow child, which
could outlast the timeout and turn a movable port into a hard failure.
NITs:
- Debug-mode timeout names the missing ready line and the way out.
- The log tail restarts if the log is truncated under it.
- waitForReady is removed; its probe tests now drive awaitOwnedListener.
- Stale daemonization comment fixed in cmd/bd.
- CONFIG_PROC_CHILDREN caveat documented.
Tests:
- fake-/proc ownership cases: owned, foreign, child gone, no listener row,
unreadable fd table;
- /proc/net/tcp parsing;
- proven-foreign ends the wait at once;
- port-in-use in the log counts before the exit is seen;
- ready-line proof, with and without the line;
- Start records PID+port while waiting;
- Start removes them after a failed start;
- pinned env port: one launch, ErrPortInUse, source in the message;
- port-file port moves;
- remedy wording per source.
The Start tests now isolate the user config instead of skipping.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(doltserver): de-flake two startup tests under load
- TestStart_RecoversWhenEphemeralPortIsTaken: on Linux, /proc proves the
taken port foreign as soon as the greeting arrives, so the first child
can be killed before the fake records its launch. Assert two port
allocations and the final launch instead of the first launch.
- TestAwaitOwnedListener_ReadyLine: give the "no ready line" case 8s, so
the fake has time to start and greet on a loaded host.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In shared-server mode every project's physical root is the one shared dolt
data dir, and bd init holds that dir's gate EXCLUSIVELY for its whole run
(~8s). It waited only exclusiveGateWait (5s) for it, so two concurrent
`bd init --shared-server` runs in different projects, or an init during
another project's long command, failed with "bd init refuses to run over
live bd activity".
acquireInitMutationGate now uses its own acquisition options:
- Wait: 30s by default, overridable with BEADS_INIT_GATE_TIMEOUT (Go
duration or bare seconds, same convention as BEADS_PRIME_TIMEOUT and
BEADS_FSCK_TIMEOUT); ctx cancellation still aborts the wait.
- One stderr notice ("waiting for another bd process on the shared
server (...)...") once the wait passes 2s, instead of immediately.
- Past the bound, the same refusal, now naming the budget and the knob.
Exclusivity is unchanged. The 5s exclusiveGateWait is also used by
backup restore, bootstrap, migrate and the backup lock; those keep it, so
the change is scoped to init via acquireExclusiveWorkspaceGatesWithOptions.
TestSharedServerConcurrent goes back to fully concurrent inits (manual runs
with many dirs raise BEADS_INIT_GATE_TIMEOUT to match). New unit tests: a
holder released after 6s lets init through with one notice; a holder past
the bound gives the clear ErrBusy error; cancellation; env parsing.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(notion): make the query page bound configurable and honour Retry-After
QueryDataSource paginates up to maxQueryPages (50) x maxPageSize (100), so a
data source larger than 5000 rows cannot be synced at all: the call returns an
error rather than partial results, and because every CLI narrowing flag is
applied after it returns, no flag works around it. Sync dies in both directions
at once. Refs #6004.
Two changes, and the second is what makes the first safe:
- Client.MaxQueryPages / WithMaxQueryPages lets a caller raise the bound for a
large data source. Default behaviour is unchanged when unset. The error at the
bound now names the ceiling in rows as well as pages, since rows is the number
a caller can compare against their own data source.
- doRequest now retries 429 and 529 honouring Retry-After, with exponential
fallback clamped at 30s. Notion enforces ~3 requests/second per connection, so
raising the page bound without this would trade a clear failure for an
intermittent one. Other 5xx retry only for GET/DELETE — replaying a POST that
may already have been applied server-side is how duplicates get made.
The request body is now marshalled once and the reader rebuilt per attempt; a
retry cannot reuse a drained body.
Tests cover the configured bound, the unchanged default, Retry-After being
obeyed exactly, GET retrying on 5xx, and POST not retrying. Verified they bite:
forcing maxRequestAttempts to 1 fails exactly the two retry tests, and ignoring
the configured bound fails exactly the bound test.
make build and make fmt-check pass; internal/notion and internal/tracker pass
with -tags gms_pure_go. golangci-lint was not run locally (not installed on this
machine); CI will cover it.
* fix(notion): use US spelling so misspell passes
The repo pins misspell to locale: US, and CI flagged "honouring" in
internal/notion/client.go. Corrected there, and in the two occurrences in
client_test.go - those are not linted today (run: tests: false) but leaving
British spellings in place only plants a failure for whenever that changes.
Verified with misspell v0.8.0 -locale US: the pre-fix content reports both
occurrences, the fixed content reports none. internal/notion tests pass with
-tags gms_pure_go, including the renamed test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4YFBBWN33GZZ2BDxWRvjT
* fix(notion): address review — 529 verb gate, ctx-aware retry wait, CLI wiring
MAJOR 1. retryableStatus returned early on 429 *or* 529 before the verb gate, so
529 was retryable for every verb — including CreatePage and CreateDatabase. A
529 can come back after Notion's edge already handed the write to the origin, so
a replay creates a second row for one bd issue, and ensureRemoteIndex is
last-write-wins on the bd ID, so nothing surfaces it. 429 stays verb-agnostic
(Notion rejects rate-limited requests before processing them); 529 now sits
under the GET/DELETE gate with the other 5xx. Three tests pin the boundary: a
creating POST makes one attempt on 529, a GET retries on 529, and a creating
POST still retries on 429.
MAJOR 2. The retry wait polled ctx.Done() with a default: and then slept, so
cancellation was ignored for up to maxRetryDelay per attempt — and
QueryDataSource pays that per page, scaling with the very bound this change
lets callers raise. Replaced with wait(ctx, d): a blocking select over
ctx.Done() and a channel from a c.after hook, time.After by default.
MAJOR 3. The bound is now reachable from the CLI. Tracker.Init reads
notion.max_query_pages / NOTION_MAX_QUERY_PAGES and threads it into the client;
unset keeps the client's own default, and a value that is set but unusable is an
error rather than a silent fallback — the operator sets that key precisely
because sync is already failing on the bound. The pagination error names those
levers instead of a Go method a CLI user cannot call.
Also: assert every attempt carries a non-empty body (a drained reader would send
an empty second one, and nothing noticed); drop the unreachable lastErr branch
by breaking out of the loop; pin retry exhaustion at exactly maxRequestAttempts
with the last attempt's error; table-test parseRetryAfter across the nine
interesting inputs; name the 529 literal; hardcode 50 in the default-bound test
so lowering the default fails it; atomic counters in tests.
Every new test verified to bite by reverting each fix in turn: each mutation
fails exactly the tests naming it, against a clean baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(notion): refuse a Retry-After longer than the client will wait
Separate commit because this one is a judgement call, not a defect fix — drop it
on its own if you would rather keep clamp-and-retry.
With Retry-After: 3600 the old path clamped to 30s and retried four times, so a
server asking for an hour got four more requests inside two minutes — the window
it told us to stay out of, and the way that window gets extended. Now the call
returns immediately, naming the delay the server asked for, and the caller
decides.
A Retry-After equal to maxRetryDelay is still honoured and waited out; only a
longer one is refused. The underlying API error is wrapped, so nothing is lost
to a caller inspecting it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(notion): key retry on side-effect safety, jitter own backoff, honour both Retry-After forms
Maintainer review fixes for PR #6006 (review iterations 1 and 2). All changes are
inside internal/notion; no new files.
F1 (major) — the 5xx/529 retry arm could never fire on POST
/data_sources/{id}/query, the paginating endpoint this PR exists to protect,
while the MethodDelete half of the allowlist had no call site at all (a census of
the client's verbs returns 3 GET / 3 POST / 2 PATCH, zero DELETE). Replaced the
verb gate with a retrySafety value each call site declares, so the property being
tested is side-effect safety rather than a proxy for it, and dropped the dead
MethodDelete arm. Only the read-only query POST changes behaviour: the creating
POSTs and both PATCHes stay non-replayable on every status except 429, exactly as
before. TestClientDoesNotRetryServerErrorOnPost exercised the query endpoint
while its "may already have been applied server-side" rationale could not be true
there, so it is retargeted at CreatePage and renamed
...OnCreatingPost, pairing with the existing 529 twin;
TestClientRetriesServerErrorOnQueryPost pins the new behaviour.
F2 (minor) — the exponential fallback had no jitter, unlike both sibling tracker
clients, so N processes tripping the same ~3 req/s limit retried in lockstep.
Jitter is applied to the self-chosen ladder only, never to a server-mandated
Retry-After, mirroring the split in internal/jira/client.go:419-424.
F3 (minor) — parseRetryAfter collapsed absent, HTTP-date and a legal
"Retry-After: 0" into one sentinel, so a date-form wait routed around the
maxRetryDelay refusal its delta-seconds twin is subject to. It now parses both
RFC 9110 forms and returns a presence flag; the doc comment's old justification
went with the behaviour it described.
F4 (nit) — maxRequestAttempts' comment claimed coverage of "transiently failing"
requests, which the transport arm does not have.
F5 (nit) — the invalid-value error names NOTION_MAX_QUERY_PAGES as well as the
config key, an abandoned wait now carries the API failure that caused it
alongside ctx's error, and the long-Retry-After refusal is evaluated before the
exhaustion break so the operator still gets it on the final attempt. That last
pin is a sibling test rather than a case inside
TestClientExhaustsRetriesAndReturnsTheLastError as the review suggested, to keep
that test a single-purpose pin on the loop's exit arithmetic.
Iteration 2 (non-gating, both in parseRetryAfter, which F3 already reworks) —
a delta-seconds value too large for a Duration now saturates instead of letting
the multiply wrap: "9223372037" wrapped to -2562047h47m16.7s, which is under
maxRetryDelay, so the refusal let it through to immediate retries. The date
form already saturated via time.Until. The unchecked multiply predates this
fixup (it is at the contributor's head) and is shared by linear/jira/gitlab,
which are left to a follow-up. TestClientRefusesLongRetryAfterInHTTPDateForm
drives a date-form header through the client, so the date form reaching the
refusal is pinned by the loop rather than only by TestParseRetryAfterHTTPDate's
arithmetic.
Validation: internal/notion 85 PASS / 0 FAIL / 0 SKIP; go build ./... and go vet
./cmd/bd (the sole importer) clean; CGO_ENABLED=0 builds; gofmt clean;
golangci-lint 0 issues both unscoped and --new-from-rev at the upstream base, and
the added //nolint:gosec is load-bearing (removing it reproduces exactly the G404
it names). Each fix is mutation-proved: reverting it reds its own pin while the
neighbouring pins stay green, including the two creating-POST controls under F1,
both directions of F2's jitter split, and both sides of the saturation bound.
Keying the refusal to delta-seconds only reds the new date-form test alone,
while TestParseRetryAfterHTTPDate stays green.
Moving the new delay logic into retryDelay rather than nesting it in the loop
took doRequest — the run's worst_changed function — from cyclomatic 21 /
cognitive 32 to 19 / 28 (gocyclo, gocognit), out of the major cyclomatic band
instead of further into it. No BUILD.bazel change is owed: no file was added and
both added imports (math, math/rand/v2) are stdlib.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
* fix(routing): bind role detection to the -C target repo (re-port of #4242)
Role detection previously called routing.DetectUserRole(".") directly,
which resolves against the caller's CWD rather than the repo selected
via `bd -C <target>` (BEADS_DIR-driven). This meant auto-routing and
role-based behavior could silently apply the wrong repo's role when
invoked with -C.
Introduce activeRepoPathForRouting(), which prefers
beads.GetRepoContext().RepoRoot (the BEADS_DIR-bound target) and falls
back to the parent of beads.FindBeadsDir(), then to ".". Route role
detection in cmd/bd/create.go and cmd/bd/routing_read.go through the
new detectUserRoleForActiveRepo() helper instead of calling
routing.DetectUserRole(".") directly.
Faithful re-port of gastownhall/beads#4242 (osamu2001) onto current
main: determineAutoRoutedRepoPath's signature changed to
(string, routing.RoutingRule) since the PR was opened (89a4d6ed8 /
#4866), so the one-line call-site change re-applies inside the new
function body, and the new routing_read_test.go coverage destructures
both return values. All other ported content (routing_role.go,
context_binding_test.go, the create.go call site) is unchanged from
the original PR.
Co-authored-by: Osamu Okano <okano.osamu@gmail.com>
Agent-Signature: claude-fable-5-high on behalf of maphew
* test(routing): fix the sole CI failure in the re-ported #4242 test
TestActiveRepoPathForRoutingFallsBackToBeadsDirParent created its fake
project under this repo's tree, so beads.GetRepoContext() resolved a
real repo root from the caller's CWD and the FindBeadsDir fallback the
test claimed to cover was never exercised — an unreachable assertion
that CI reproduced as the PR's only failing test.
Rewrite it to target a directory that is genuinely outside any git
repo: targetDir now comes from t.TempDir() (its .beads/config.yaml
still written via writeTestConfigYAML), and the caller CWD is also
moved to a fresh t.TempDir() via t.Chdir() so GetRepoContext() can't
resolve through the CWD either. Both resolve under os.TempDir(), which
internal/beads/context.go's isPathInSafeBoundary explicitly admits, so
this stays safe under the existing boundary check.
This makes safeTempDirForContextTest (the manual git-repo-relative
temp dir helper the original test used to dodge the same problem)
dead code; delete it.
Co-authored-by: Osamu Okano <okano.osamu@gmail.com>
Agent-Signature: claude-fable-5-high on behalf of maphew
* fix(routing): a .beads/redirect must not move role detection to the storage root
Cross-vendor review of e8600e732 caught a regression the re-port would have
introduced for redirect users: with a .beads/redirect pointing at storage
outside any git repo, RepoContext.RepoRoot is the redirect target's parent,
so beads.role would be read from the storage location instead of the
workspace repo — silently bypassing contributor routing.
The two external-BeadsDir shapes now get their opposite answers explicitly:
an explicit BEADS_DIR selection (what bd -C plumbs) binds role detection to
the selected project (the GH#4242 fix), while a followed redirect keeps it
on the CWD workspace repo (pre-existing semantics — a redirect relocates
storage, not the project). Mutation-tested: with the guard removed, the new
test fails by resolving to the storage root.
Co-authored-by: Osamu Okano <okano.osamu@gmail.com>
Agent-Signature: claude-fable-5-high on behalf of maphew
* fix(routing): read BEADS_DIR selection provenance at startup, not the live env var
The self-review on this PR found a startup-path gap: rootCmd's
PersistentPreRunE calls prepareSelectedCommandContext for every command,
which sets BEADS_DIR to the resolved target — redirects included. By the
time role detection ran, activeRepoPathForRouting's os.Getenv("BEADS_DIR")
check was therefore always true, classifying a plain .beads/redirect as
explicit selection and binding beads.role to the storage root instead of
the workspace repo.
Capture whether BEADS_DIR was present in the environment at process start,
and treat selection as explicit only for that case or a -C flag. The new
TestActiveRepoPathForRoutingSurvivesStartupRebindAcrossRedirect goes
through the real startup rebind with a real redirect and fails on the
previous logic; the two tests that simulate a caller-exported BEADS_DIR
now pin their provenance explicitly.
Agent-Signature: claude-fable-5-high on behalf of maphew
* fix(routing): .env-provided BEADS_DIR is user selection, mark its provenance
Cross-vendor review of the previous commit caught a regression it would
have introduced: loadBeadsSelectionEnvFile intentionally lets a project's
.beads/.env route commands via BEADS_DIR, and that selection happens after
the process-start snapshot. Flip the provenance flag when the selector
loader sets BEADS_DIR, so .env routing keeps binding role detection to the
selected project. TestActiveRepoPathForRoutingHonorsEnvFileSelection pins
it end to end through loadSelectionEnvironment.
Agent-Signature: claude-fable-5-high on behalf of maphew
* test(routing): make the BeadsDir-parent fallback test order-robust on macOS
The macOS CI leg failed on /private/var vs /var: the test asserted a raw
string path while resolution can hand back the canonicalized spelling of
the same directory depending on which branch wins under test ordering.
Compare through EvalSymlinks like the sibling redirect tests, and pin
selection provenance in the two tests that had not declared theirs.
Agent-Signature: claude-fable-5-high on behalf of maphew
* ci: retrigger - dolt-install/download infra outage (curl 56/503) killed 4 jobs
Agent-Signature: claude-fable-5-high on behalf of maphew
* build(bazel): enumerate cmd/bd/routing_role.go in bd_lib srcs
The PR adds cmd/bd/routing_role.go, whose detectUserRoleForActiveRepo is
called from create.go and routing_read.go. cmd/bd/BUILD.bazel lists
go_library srcs explicitly, so the unenumerated new file broke the Bazel
library compile:
cmd/bd/create.go:303:21: undefined: detectUserRoleForActiveRepo
cmd/bd/routing_read.go:52:19: undefined: detectUserRoleForActiveRepo
Generated by `bazel run //:gazelle` (one srcs row, idempotent; no deps row
owed because the library already carries internal/beads and
internal/routing). Base parity at 89348699b0 was 336/336 non-test .go files
enumerated in cmd/bd, so the gap was PR-introduced, not pre-existing.
* fix(routing): bind role to the selected workspace across a .beads/redirect
Maintainer review fixup for the adopted PR (review iterations 1 to 3).
Iteration 1 addressed the one gating major plus both minors and both nits
from its review synthesis. Iteration 2 addressed the major, the minor and the
nit that the next synthesis raised against that fixup, and iteration 3
addresses the minor and the nit raised against iteration 2; see "Review
iteration 2" and "Review iteration 3" below.
The major: routing_role.go ordered its guard `explicitBeadsSelection() ||
!rc.IsRedirected`, but every beads-dir resolver follows a .beads/redirect before
that guard can run (ExplicitBeadsDir via FollowRedirect, FindBeadsDir, and -C's
own resolution). So an explicit selection of a REDIRECTED workspace arrived
already rewritten to the storage root, short-circuited the redirect doctrine the
PR had just written, and bound beads.role to wherever the DATA lives — the same
misroute GH#4242 reported, one resolution step further in. Worse, it was silent:
`git config --get beads.role` succeeds outside a repo by reading GLOBAL config,
and routing.DetectUserRole returns on that first success, so its own "beads.role
not configured" notice never fired and a contributor workspace could be routed
with the user's global role with no diagnostic on any stream.
The remedy has to be anchored per shape, because the two selection shapes carry
the pre-redirect directory in different places:
- `bd -C <dir>`: applyChangeDirSelection has already rewritten BEADS_DIR to the
resolved target, so only the -C argument still names the chosen project.
CWD-anchored GetRedirectInfo() cannot see this case at all — under -C the CWD
is some unrelated directory — so this arm asks GetRedirectInfoFrom(dir),
which is documented for exactly this case (GH#5509) and deliberately does not
consult BEADS_DIR.
- exported BEADS_DIR: here the CWD *is* the selected workspace, so the
CWD-anchored probe is the correct one. The selection is vacuous when it
merely names that workspace's own redirect, under both spellings of it:
BEADS_DIR=<workspace>/.beads and BEADS_DIR=<storage>/.beads both arrive as
the storage dir (the bd-wayc3 tooling shape), so both fall through to
rc.CWDRepoRoot.
An explicit selection of a genuinely different, non-redirected project still
wins — that is the GH#4242 fix itself, and a control test pins it.
Also in this commit:
- A role path that is not a git repository now warns instead of quietly
inheriting global beads.role (new beads.RepoRootFor answers both "what is
the repo root for this dir" and "is this a repo at all").
- A .env-provided BEADS_DIR records selection provenance on the broad loader
too, not only via loadBeadsSelectionEnvFile. That loader early-returns
whenever BEADS_DB or BD_DB is already exported, and loadBeadsEnvFile then
imports the same .env line, so one .env line meant target-role or CWD-role
depending on the caller's unrelated BEADS_DB export. Internal rebinds are
unaffected: the marking is conditional on BEADS_DIR having been unset, which
is what separates user selection from prepareSelectedCommandContext's
per-command rebind.
- The four-way role-path choice is logged via debug.Logf, since the downstream
"failed to detect user role" warning fires on none of those branches.
- The PR's new routing test neutralizes BEADS_ROUTING_MODE / BD_ROUTING_MODE,
which viper binds onto the routing.mode key and which reds the test on any
host that exports them.
- routing_role.go now cross-references RepoContext.Role(), the second,
known-divergent role oracle in internal/beads. No behavior change there.
Tests: the explicit x redirect cell was the one combination the PR's new tests
left empty. Both probe shapes land as tests, plus the `bd -C <redirected-ws>`
shape, a non-redirected -C control, the non-repo warning, and both provenance
polarities. Each new pin was verified to FAIL against the pre-fix code.
Review iteration 2:
- Major: the non-repo warning above ran inside detectUserRoleForActiveRepo,
which create.go and determineAutoRoutedRepoPath called before resolving the
routing config. Routing is off by default and only auto mode reads the
role, so in any non-git beads project every bd create/list/ready printed a
4-line warning about a role nothing used. The leak also broke six
TestParity* characterization tests at the iteration-1 head:
CreateHumanOutput, CreateOnOccupiedIDRefuses, CreateParentNotFoundExits1,
CreateMissingDepTargetIsFatal, CreateJSONErrorShape and
ReopenErrorExits1Silently. Detection now runs only when the fully resolved
config can use its answer: new RoutingConfig.UsesUserRole (auto mode, no
explicit override, a role-specific repo configured) gates the new
detectUserRoleForRouting, which both call sites invoke after building the
config, so the legacy contributor.auto_route=true fallback to auto counts
where a raw routing.mode check would miss it. Side effect: routing's own
GH#2950 "beads.role not configured" notice is also skipped when routing
cannot use the role, as docs/multi-agent/routing.md already promises when
it calls `routing.mode explicit` "disabling role detection entirely".
- Minor: for an exported BEADS_DIR, selectedWorkspaceRepoRoot called the
selection vacuous when the CWD-anchored GetRedirectInfo reported a redirect
to it. With no redirect in the CWD repo, GetRedirectInfo falls back to the
raw BEADS_DIR, so a FOREIGN redirected workspace vouched for its own
redirect and the CWD repo's role was bound. The check now also requires the
redirect to sit in the CWD repo's own .beads (RedirectInfo.LocalDir). A
foreign redirected selection falls through to its storage root, which is
the answer a live command already reaches after the pre-RunE rebind, so
unit and command behavior agree. Binding the foreign workspace itself would
need the raw startup BEADS_DIR, which nothing keeps; the routing_role.go
comment records that limit.
- Nit: the non-repo warning prints at most once per process. An ID lookup
falls back to auto-routing once per missing ID, and each fallback re-ran
detection.
- Test isolation: the PR makes role detection read the process-wide
beads.GetRepoContext() cache, a sync.Once. Three existing tests chdir
between temp repos without resetting it and so read beads.role from an
earlier test's deleted repo: TestResolveCloseTargets/
contributor_mixed_local_and_routed,
TestDetermineAutoRoutedRepoPath_ContributorToPlanning and
TestOpenRoutedReadStore_ContributorRouting fail in a broad cmd/bd run at
86b8abff3f and 8dbfc6c721 and pass at base 89348699b0. The failure is
order-dependent, and the tests skip without a Dolt test server. Each now
rebuilds the cache after its chdir, as does the MaintainerToPlanning
sibling, which has the same exposure.
Iteration-2 tests: a property test pins UsesUserRole to
DetermineTargetRepoWithRule (true exactly when the answer varies by role,
across unset/auto/explicit modes and every override and repo combination). A
table test covers routing unset, explicit mode, auto without a role repo,
auto, and legacy auto_route. Two more cover the foreign redirected BEADS_DIR
cell and repeated detection (the once-guard). Each pin was mutation-checked:
dropping the gate, gating on the raw mode, or dropping the LocalDir guard,
the once-guard, the role-repo clause or the override clause turns a test red.
Validated (iteration 1): gofmt, go build ./..., go vet ./cmd/bd/
./internal/beads/, focused go test over cmd/bd role/routing/envfile/context
tests and ./internal/beads/ ./internal/routing/, gazelle (0 srcs rows owed),
and bazel build //cmd/bd:bd_lib //internal/beads:beads.
Validated (iteration 2): gofmt, git diff --check, go build ./..., go vet
./cmd/bd/ ./internal/routing/ ./internal/beads/ with and without cgo,
golangci-lint v2.10.1 --new-from-rev=HEAD on cmd/bd and internal/routing (0
issues), go test ./internal/routing/ ./internal/beads/, and a broad cmd/bd
routing/role/redirect/context/Parity run. The only reds on this host,
TestInitRedirect/RedirectCreatesDBInTarget,
TestBareParentWorktreeCoreCommandsWithoutRedirect and
TestServeIssueRolesComeFromBeneathTheHookDecorator, fail the same way at base
89348699b0. No files were added and the only new import is stdlib, so no
BUILD.bazel change is owed.
Review iteration 3:
- Minor: the iteration-2 guard counted a redirect as the CWD workspace's
own only when it sat in the CWD git root's .beads (Dir(LocalDir) ==
rc.CWDRepoRoot), which equates the workspace with the git root. From a
workspace nested below its repo root (<repo>/sub/.beads/redirect), or
from a git worktree that inherits its main checkout's redirect,
exporting that workspace's own .beads therefore moved role detection
from the CWD repo to the storage root: the env-dependent inconsistency
the iteration-1 vacuous rule exists to remove. Dropping the guard would
not have fixed it. When the repo root's .beads holds no redirect,
GetRedirectInfo checks BEADS_DIR itself, and prepareSelectedCommandContext
has rebound BEADS_DIR to the storage .beads before RunE. After that
rebind GetRedirectInfo finds no redirect in either shape, so the
iteration-1 rule bound the storage root there too. New
beads.GetDiscoveredRedirectInfo is GetRedirectInfo without the
BEADS_DIR override: the redirect discovery finds from the CWD (the repo
root's .beads, then the worktree-local redirect, the main checkout's
.beads for a worktree, and the walk-up from the CWD), via
findDiscoveredLocalBeadsDir, the BEADS_DIR-free half of
findLocalBeadsDir. The exported-BEADS_DIR arm now calls a selection
vacuous exactly when that discovered redirect exists and targets the
selected storage. A foreign redirected selection still falls through to
its storage root, because discovery from the CWD finds no redirect or
one with a different target.
- Nit: the foreign-selection test's comment now says only the exported
arm pins the guard. The rebound storage .beads holds no redirect, so
the rebind arm only checks that both states of the variable agree.
Iteration-3 tests: nested-workspace and worktree-without-own-.beads cells,
each with BEADS_DIR unset, as exported, and as rebound before RunE. Against
the unchanged 07c7373dfb code the four set-BEADS_DIR cells fail and both
unset controls pass. A new test pins the target comparison: from a
redirected workspace, an exported BEADS_DIR naming a different,
non-redirected project still binds that project.
TestGetDiscoveredRedirectInfo pins the new API against GetRedirectInfo in
both directions (a nested redirect that BEADS_DIR masks, and a BEADS_DIR
redirect the CWD does not hold). Five mutants each turn a test red: probing
with GetRedirectInfo again (the iteration-1 rule, which fails both rebind
cells and the foreign test), dropping the target comparison, dropping the
worktree-fallback tier, letting discovery honor BEADS_DIR, and restoring the
git-root guard.
Validated (iteration 3): gofmt, git diff --check, go build ./..., go vet
./cmd/bd/ ./internal/beads/ ./internal/routing/, golangci-lint v2.10.1
--new-from-rev=HEAD on cmd/bd and internal/beads (0 issues), go test
./internal/beads/ ./internal/routing/, and a focused cmd/bd run over the
role, routing, redirect, envfile and context-binding tests against the
Dolt test server (47 passed, 0 failed, 0 skipped). No files were added,
so no BUILD.bazel change is owed.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Osamu Okano <okano.osamu@gmail.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(dbproxy): prove the managed dolt owns its port; move a taken Beads-chosen port
The managed-local backend's dolt port comes from a bind-and-close
(proxy.PickFreePort) when bd writes the generated config. The dolt
sql-server binds it only later, after the proxy child has run dolt
config and init and booted the server. If another process takes the port
in that gap, dolt logs "Port N already in use." and exits. But
DoltServer.waitReady only dialed the port, so it accepted the other
process's listener as ready. The proxy then published proxy.pid, its
backend health check saw dolt gone, and it shut down. bd init's first
ping got "unexpected EOF" and every retry after that got "connection
refused". Under parallel tests this hit any test that starts its own
managed backend.
Readiness now requires proof that the process Start launched owns the
port:
- DoltServer sends the child's stdout/stderr through a watcher that
forwards to the server log. A dial counts only after dolt's own
"Server ready. Accepting connections." line has appeared on that
output. That line is info level, and every generated config uses
info. A config with a quieter log_level has no such signal and keeps
the dial-only check.
- A child that exits first is reported as the new server.ErrPortInUse
when its output said its port was taken.
Recovery: DoltServer.SetPortRepicker lets Start move the config to a
fresh port, re-read it, and relaunch, up to 5 ports. dolt config/init
run only on the first attempt. bd's repicker,
repickManagedProxiedServerPort, needs to tell a port Beads chose from
one an operator set, so generated configs now record the chosen port
on a "# beads-chosen-listener-port: N" line after the managed marker.
The repicker moves the port only when the file is Beads-generated,
carries that record, and its listener.port is still the recorded port.
Otherwise the port is treated as pinned and Start fails with
ErrPortInUse. That includes configs generated before this change. It
rewrites only listener.port and the record, through a yaml.Node so the
rest of the file is kept, and writes the file atomically.
db-proxy-child installs it for the local backend.
The proxy's waitForServerReady also re-checks Running after its dial.
Start has already proved ownership, and a running dolt holds its
listener, so only a backend that exited between the two checks could
let a dial reach another process.
Tests:
- server: a held port is not accepted as ready (ErrPortInUse; this
test fails on main), a repicker brings the server up on a new port
that the pid record and DSN both use, and a declining repicker ends
Start with ErrPortInUse.
- cmd/bd: the generated config records its port, a Beads-chosen port
moves while hand edits survive, and a hand-set port, a config
without the record, an operator config, and a stale in-use port are
all refused with the file left untouched.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(dbproxy): move a taken managed port via a runtime config, never config.yaml
This reworks cd00c5acb after review. That commit recovered from a taken
Beads-chosen port by rewriting the managed config.yaml. That broke the
file's own contract ("Never rewritten afterward; hand edits are
preserved", #4986), and #5024 had deferred that contract change to an
RFC.
dolt sql-server has no command-line way around the file: with --config,
-P/--port is silently ignored (checked against dolt 2.1.8 and 2.2.0, in
both argument orders). So on ErrPortInUse the proxy child now writes
proxy-child.runtime-config.yaml (server.RuntimeConfigFileName) in the
server root. That file is the operator config with only listener.port
moved to a free port. The child relaunches dolt from it and records the
effective port where runtime readers already look: the backend pid
record and the DoltServer's dial/DSN config. The runtime config is
removed when the server stops, when Start fails, and by the next Start.
config.yaml is never written.
- server.PortRepicker becomes server.PortConflictPolicy. It only decides
whether the port may move; the server does the move.
- bd's policy (managedPortConflictPolicy) allows a move when the file
carries the managed marker and either:
- has the port record and listener.port still equals it, or
- is byte-identical to the config bd generated before the record
existed, so existing unedited workspaces recover too.
- Everything else is pinned: operator configs, hand-set ports,
hand-edited legacy files, and configs with ${...} placeholders (a
runtime copy would put the expanded values on disk).
- The record line now says plainly that Beads runs on a free port
without editing the file, and that deleting the line or changing
listener.port pins the port. The record parser now requires the
digits to end at a space or line end and to be in 1..65535.
- ErrPortInUse messages say what to do: free port P, set listener.port
in <path> to a free port, or (for a generated file) delete it so
Beads regenerates it. The give-up message names the last port tried.
- A dolt that never logs the ready line is still not accepted. A
grace-period fallback would let a foreign listener through while dolt
is still starting. The timeout error now names the missing line, says
whether anything answered, and points at the escape hatch: at
log_level warning, readiness is dial-only.
- Port-conflict detection matches go-mysql-server's "Port host:port
already in use." and the Windows bind error, and requires the SQL port
in the kernel's bind error, so a metrics or MCP listener failure is
not taken for the SQL port.
- Spawn error paths reap with cmd.Wait(). cmd.WaitDelay is set, and a
comment notes that the backend's output is now a pipe into the proxy
child.
Tests:
- Output-watcher table tests: split, one-byte and CRLF writes, output
past the 4 KiB tail, the MCP ready line, port boundaries, the
go-mysql-server and Windows forms, another listener's bind error, the
unix-socket warning, and nil log.
- A real dolt at log_level warning starts through dial-only readiness.
- With a stand-in dolt (the test binary in fake mode, wired in
TestMain): Start gives up after maxStartPortAttempts, a missing ready
line is diagnosed, plus a positive control.
- With real dolt: a move keeps config.yaml byte-identical, runs from the
runtime config, and the runtime config is removed on Stop. A declining
policy and the no-policy remedy are covered.
- The bd policy's allow and pin cases.
- The managed-local lifecycle test's config/pid port check accepts the
runtime config.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(dbproxy): ask the port policy once, about the operator's port; copy raw config text
Re-review of the runtime-config rework found a regression. After the
first move, Start asked the PortConflictPolicy about the runtime port.
bd's policy re-reads config.yaml, whose listener.port is still the
operator's port, so it refused with "listener.port is A, not the in-use
port B". The "up to 5 ports" retry therefore never ran in production;
the fake-dolt test missed it because its policy always allowed.
Start now asks the policy once per Start, on the first conflict, about
the operator config's port. Ports Start picked itself are its own to move
again. Start also re-reads the operator config up front, so every Start
begins from that config's port.
The runtime config is now built from config.yaml's raw text
(servercfg.NewYamlConfig) instead of the env-interpolated
YamlConfigFromFile result. dolt interpolates the runtime file when it
reads it, so the old way expanded placeholders twice: "$$" escapes lost a
level and placeholder values landed on disk. The server reads the
runtime file back with interpolation for its own Dial/DSN view. Since
placeholders no longer expand on disk, bd's policy stops refusing
configs that contain them.
Tests:
- bd's real policy driving a real DoltServer with a stand-in dolt that
reports every port taken: Start makes five launches on five distinct
ports, starting with config.yaml's, then gives up. It is not refused
on the second move, and config.yaml stays unchanged.
- A frozen byte-for-byte literal of the config bd v1.3.0/v1.3.1
generates. The legacy render must equal it, and the policy must find
it eligible.
- The runtime config keeps placeholders and "$$" verbatim and is mode
0600, while the server dials the interpolated host.
- The give-up test asserts the policy is asked once, about the
operator's port.
- The missing-ready-line test timeout is raised to 30s for the slow
fake-dolt exec under race.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(dbproxy): classify a child that exits before its identity is captured
TestManagedPortConflictPolicy_ConsecutiveCollisions failed on the farm
with "capture child birth identity: procid: process is no longer
running", after 2 launches instead of 5. Its stand-in dolt reports
"Port N already in use." and exits immediately, and it exited before
Start captured its birth identity. Start treated that capture failure as
fatal, so an ErrPortInUse that Start could recover from became a hard
error. A real dolt that finds its port taken can exit just as fast, so
this race exists in the product, not only in the test.
Every spawn step before the child is handed to its supervising
goroutine (identity capture, root identity, pid record) now fails
through failSpawn. failSpawn kills and reaps the child; reaping also
drains its output into the watcher. The child's own exit then decides
the error:
- the child reported its port taken: ErrPortInUse, so the normal policy
and retry path runs;
- the child exited by itself for another reason: "exited (status N)
before startup completed";
- the child was alive and we killed it: the step's own error, as
before.
On Windows, a killed child reports an ordinary exit code, so there an
exit cannot be told from a kill. Only the port-in-use classification
applies there.
procid.Capture is reached through a package var so tests can make the
capture lose the race deterministically.
Tests, using a shell-script stand-in dolt that starts in milliseconds:
- a fast port-in-use exit that beats the capture is ErrPortInUse, and
with a policy it is retried up to the 5-port budget;
- a fast exit for another reason reports its status;
- a capture failure while the child is alive stays fatal and kills the
child;
- ten immediate port-in-use exits with the real capture are all
ErrPortInUse.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): run TestSharedServerConcurrent in the doltserver-cmd lane
TestSharedServerConcurrent was gated on BEADS_TEST_SHARED_SERVER=1, which
no CI job or Bazel target sets, so it skipped everywhere (the only
non-lane skip in //cmd/bd:bd_dolt_server_test). It predates the workspace
gate (#5093), and once enabled it failed immediately: in shared-server
mode every project's physical root is the one shared dolt dir, and bd init
holds that root's gate EXCLUSIVELY with a 5s wait, so concurrent inits of
different projects refuse each other ("bd init refuses to run over live bd
activity on this workspace"). That exclusivity is deliberate, and a single
init takes ~8s here, so the test now inits its projects one at a time; the
concurrent multi-writer workload phase, which is what the test exists for,
is unchanged.
- Also run when BEADS_TEST_SHARED_SERVER_CONCURRENT=1 (distinct from
BEADS_TEST_SHARED_SERVER, which scripts/test.sh also reads to start a
shared server).
- bd_dolt_server_test sets it with BEADS_TEST_SS_DIRS=2 and
BEADS_TEST_SS_CLIENTS=6: a bounded smoke (~50s, 3 writers per database)
against the lane's hermetic local dolt sql-server on an ephemeral port.
- Under BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 a server that cannot start
fails the test instead of skipping it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): size and shard //internal/storage/uow:uow_test for its configless run
uow_test timed out at the default medium 300s whenever it ran without a
lane config (`bazel test //internal/storage/uow:uow_test`, also on
origin/main). No CI lane runs it that way: --config=ci and
--config=integration set BEADS_TEST_SKIP=dolt, so its Dolt-backed tests
skip and the target takes 3-14s there (BEP of recent ci/integration
runs), and the full Dolt workload runs as uow_dolt_test (8 shards, slowest
~90s remotely on main) and uow_docker_test (4 shards, large).
Without a config nothing skips: every newTestUOWProvider test starts its
own dolt sql-server (~6-8s apiece) and the package needs ~7 min serially;
TestBatchApplyContract alone is ~1 min because BoundsTheItemCount applies
1000+1001 real items by design. That is the variants' workload, not a slow
test to fix, so give the base target the variants' shape: size = "large"
(900s) and shard_count = 4. Locally, on a host at load ~185, the shards
took 162-309s (unsharded it was killed at 300s); under --config=ci the
four shards still pass in seconds.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): reset cobra flags between in-process runs; keep them out of parallel tests
runBDInProcess and runBDInProcessAllowError execute the package-level
rootCmd tree again and again in one process, and cobra never resets a
flag between Execute calls: a value one invocation parsed (create's
--title, --labels, list's --status, ...) and its Changed bit stayed on the
FlagSet, so the next invocation saw it, and a slice flag that had been
set appended to the stale value instead of replacing it. With a Dolt test
server (the only setup in which these tests run) that failed about 20
TestCLI_* tests depending on suite order.
TestMain now snapshots every flag of the tree (snapshotCommandFlags,
after all init()s registered them, before any test runs) and both runners
put the tree back before and after each run (resetCommandFlags): the
pointee of each flag.Value is restored (for pflag's scalar types that is
the bound variable; for slice types it also clears the internal "changed"
bit, so the next Set replaces), slice contents via SliceValue.Replace,
and Changed=false. TestResetCommandFlagsRestoresEveryFlag sets every flag
of the real tree and checks the reset. The ad-hoc --title reset in
TestCLI_CreateRejectsEmptyTitle_ProxiedServerMode is no longer needed.
Seven TestCLI_Comment* tests called t.Parallel() around the runners.
inProcessMutex serializes the runners only against each other, so their
chdir, stdio swap, env and config re-initialization raced every other
parallel test (data races on config globals; TestCobraParallelPolicyGuard
failing to read its own source files after a concurrent chdir). They run
serially now, and the guard treats the two runners like rootCmd.Execute.
Also two stale TestCLI_Create* cases: DryRunWithRigPrefix exercised a
create --prefix flag that no longer exists, and the flag-like-title cases
passed "--help" or "--foo-bar" where cobra parses them as flags; they now
pass the title after "--", the only way it reaches create's own guard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix the ADO round-trip and bulk-delete tests that rotted unseen
Both need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd (BEADS_TEST_SKIP=dolt everywhere), so they failed unnoticed:
- TestADORoundTrip* stored ado.pat in the database. ado.pat is a secret
read only from config.yaml or AZURE_DEVOPS_PAT (config.IsYamlOnlyKey),
so the tracker found no PAT. The fixture sets AZURE_DEVOPS_PAT.
- TestBulkDeleteNoResurrection expected GetIssue to return (nil, nil) for
a deleted issue; the Dolt store reports storage.ErrNotFound.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix parity, completion and notion tests that only fail with a Dolt server
These are Dolt-gated (integration tag + test server), so no CI lane ran
them and they rotted:
- TestParityCreate*/TestParityReopenErrorExits1Silently: bd create
auto-routes via routing.DetectUserRole("."), which warns "beads.role
not configured (GH#2950)" on stderr when the working directory's git
config has no beads.role. The suite pins the configured case, so the
parity env now sets beads.role=maintainer in a test-owned global git
config ($XDG_CONFIG_HOME/git/config; the role lookup scrubs every
GIT_CONFIG_* variable, so those cannot be used).
- TestIssueIDCompletion: issueIDCompletion resolves a database path
before looking at the open store and returns nothing when there is
none; the test now sets dbPath to its store's path.
- TestIssueIDCompletion_UsesWorktreeFallbackWhenStoreNil and
TestGetNotionConfigReadsDBPathWhenStoreUnset: the code under test
reopens the store from metadata.json, which names a database but not
the per-test branch of the shared database that newTestStore uses, so
the reopened store saw none of the test's rows. They now use their own
database (newTestStoreIsolatedDB).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix stale doctor/e2e/autocommit/readonly Dolt-gated tests
These need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd, so they rotted unseen:
- TestE2E_DoctorFixMetadataRoundtrip, TestE2E_MigrateDoltMetadata,
TestDoctorCheckHealthReportsVersionMismatchOnRepoLocalPort: bd init now
defaults to embedded mode, where `bd sql` and `bd dolt start` are
refused. Init with --server on a repo-local server of their own
(repoLocalServerEnv: the suite's shared-server port vars cleared, server
stopped at cleanup) and start it explicitly, since BEADS_TEST_MODE=1 never
auto-starts one. MigrateDoltMetadata's DELETE also needed `key` quoted.
- TestDoctor{Conventions,Pollution}MaxRows_EnvOnly_Exits2: bdRunRaw drops
every BEADS_* variable, so the subprocess hit dolt.New's test-database
firewall (exit 1) before the row cap; pass BEADS_TEST_SERVER=1.
- TestDoltAutoCommit_On_WritesAdvanceHead: the embedded engine keeps its
data under .beads/embeddeddolt, not .beads/dolt; findDoltRepoDir looks
in both.
- TestValidateCheck_AllClean: bd doctor validate runs five checks since
the cross-table duplicates check was added.
- TestConfigValidateReadOnlyIsHermetic: readonly commands take a shared
workspace gate, whose lock file (.beads/dolt.gate.lock) workspacegate
creates on first use by design; compare the .beads tree without gate
lock files.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): undo each in-process execution's env and storage-mode leaks
Every in-process run of the bd command tree (runBDInProcess and the many
tests that call rootCmd.Execute directly) leaves process state behind:
PersistentPreRun exports the selected workspace as BEADS_DIR, loads its
.beads/.env, and records the workspace's mode in serverMode /
proxiedServerMode. With a Dolt test server, later tests in the same shard
then resolved a deleted fixture ("no active beads workspace found",
"migration sidecar is missing"), ran subprocess bd against the wrong
workspace ("recorded as proxied-server in .beads/metadata.json"), or took
the proxied path in RunE-level tests ("proxied-server UOW provider not
initialized", a nil-UOW panic in reopenProxiedResolve that killed a shard).
TestMain now registers a cobra initializer/finalizer pair
(installExecuteIsolation) that snapshots the environment and the two mode
globals when an execution starts and restores them when it ends; the
runners' restoreEnvSnapshot shares the helper. TestDoltConfigEnvironmentOverrides
used os.Setenv + defer os.Unsetenv, which deleted the suite's
BEADS_DOLT_SERVER_PORT (the test server's port) for every later test; it
uses t.Setenv now.
Full integration-tagged cmd/bd suite against a hermetic dolt (16 shards):
18 order-dependent failures and the shard panic gone; the 27 remaining
failures all fail alone too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(init): keep --quiet quiet through the Claude/Codex/Cursor auto-setup
`bd init --quiet` ("Suppress output") still printed the project
integration installers' progress to stdout ("Installing Claude hooks for
this project...", "Claude Code integration installed", ...): the
installers write to os.Stdout unconditionally. Point stdout at the null
device around them while quiet; their failures keep going to stderr as
before. Found by TestInitCommand's quiet case, which asserts no output and
had not run in CI (Dolt-gated).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(init): bring the Dolt-gated init tests up to the current init contract
These tests need a Dolt test server, so no CI lane ran them and they
drifted from the product:
- Embedded is init's default mode, so the server-mode assertions
(.beads/dolt, opening the database on the test server, dolt_mode
"server" in metadata.json) now pass --server (TestInitCommand,
TestInitAlreadyInitialized, TestInitRedirect, TestInitBEADS_DIR,
TestInitDoltMetadata, TestInitDoltMetadataNoGit, TestInitDatabaseFlag's
metadata subtests). The in-process ones reset rootCmd's flags around
themselves so --server cannot leak into later in-process inits.
- Each server init gets its own --database: the suite shares one test
server, and a "test" database another test created belongs to another
project (init refuses: PROJECT IDENTITY MISMATCH).
- The marker tests seed a current-era version witness: the legacy-upgrade
guard refuses a non-empty .beads/dolt without one.
- The redirect test provisions the canonical target with config.yaml: bd
ignores a redirect to a directory with no project files (#4692).
- Shared-server inits get a free port of their own
(sharedServerTestEnv): the inherited BEADS_DOLT_SERVER_PORT is the
suite's test server, which a shared-server start refuses as another
project's, and the default shared port is 3308, a developer's live
server.
- --shared-server's config.yaml is now nested YAML (dolt: shared-server:
true); parse it instead of matching the old flat line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): run the Dolt-gated tests that skipped on a bare dolt.New
setupTestDB (config, kv and Linear round-trip tests), setupGatedTestDB
(mol ready --gated) and the rename-prefix and bd sql tests opened stores
with dolt.New(&dolt.Config{Path: ...}): no server port, no
CreateIfMissing, so against a running test server they got "database ...
not found" and skipped as "Dolt server not available". They now use the
suite's newTestStore helpers (skip only when no test server runs), and
what that exposed is fixed test-side:
- TestCustomStatusConfig: custom_statuses is read back ORDER BY name;
look statuses up by name / expect name order.
- TestLinearRoundTripCoreFields, TestLinearPullMilestonesCreatesEpicHierarchy:
linear.api_key is a yaml-only secret (LINEAR_API_KEY), push now needs an
explicit linear.state_map, and batch push uses TeamLabels and
issueBatchCreate, which the mock server now answers.
- TestSqlCommand: bd sql refuses in the default embedded mode; set
serverMode for the server-backed store, and stop discarding RunE errors.
- TestE2E_AutoStartedRepoLocalServerPersistsAcrossCommands skipped on
this process's embedded mode although every bd it runs is a subprocess
on a --server workspace; gate on the dolt binary instead and stop the
auto-started server at cleanup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): no fixed "unused" port, no swallowed JSON, no leaked front-door server
Three more things the Dolt-backed cmd/bd run exposed:
- TestTestServerConnection, TestSharedServerCfgNilHonorsSharedServer and
the two TestLegacyGuard* e2e tests pointed bd at 127.0.0.1:59999 as a
port nothing listens on. Any process on the host may hold it; a
listener that accepts and resets turns the expected "connection
refused" / "Dolt server unreachable" into another error. They take a
just-freed port from closedLoopbackPort (testutil.FindFreePort) now.
- TestDoltSetConfigJSONOutput and TestDoltSetConfigWithUpdateConfig
created .beads 0755, so bd's permissions warning landed in the JSON
under test and the tests skipped ("output not pure JSON") every run.
The fixture uses 0700, and non-JSON output fails instead of skipping.
- TestMigrateDoltModeFrontDoor's closing show/list auto-start a
repo-local server after the reverse migration; nothing stopped it, so
the suite's leak check failed (when the opt-in
BEADS_TEST_MIGRATION_FRONTDOOR suite runs). It stops it at cleanup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): advisory cmd/bd Dolt-server tier (bazel-cmd-dolt)
No CI lane ran cmd/bd's Dolt-gated tests (TestCLI_*, the init and
store-backed suites, the doctor/e2e tests): main.yml's integration shards,
nightly's Full Test Suite, and Bazel's --config=prcore/ci and
--config=integration all pass BEADS_TEST_SKIP=dolt, so cmd/bd's TestMain
never starts a test server and those tests skip; the proxied and embedded
tiers run only their manifests' tests. They rotted unseen.
//cmd/bd:bd_dolt_server_test (tag dolt-server-cmd) runs the whole
integration-tagged cmd/bd suite, the integration build's bd_test, against
a hermetic dolt sql-server: BEADS_TEST_DOLT_SERVER=local starts it from
the pinned dolt in the test wrapper's runfiles, and
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 fails the target instead of skipping
into a cached pass. No test selection: the Go binary shards itself over
every top-level test (32 shards), so each runs exactly once, and
check_testcases.py fails a shard that ran none. --config=doltserver-cmd
has exactly --config=integration's build flags, so the two lanes share
bd_test's compile; prcore/ci and integration exclude the tag.
bazel.yml's new bazel-cmd-dolt job runs it exactly where bazel-integration
runs (same if: remotely on the farm for same-repo PRs, fork-ro/fork-rw,
and locally with the read-only cache in mode cache for forks). It is
advisory: job-level continue-on-error keeps a failure out of the call's
aggregate (BAZEL), and pr.yml's ci-gate reads its result (BAZEL_CMD_DOLT)
but requires it only while the committed top-level flag
BAZEL_CMD_DOLT_REQUIRED is "true" (now "false"; flipping it is the one
change needed). bazel-gate.sh accepts its skip in modes skip and local.
Policy tests: a third lane category in TestBazelLaneIsGatedAlongsideLegacy
(bazelFlagGatedLanes: gate env wired, never statically required, added by
the run script exactly when the pinned flag is "true"), the gate
simulation over every mode with the flag off (advisory: nothing it
reports reddens the gate) and on (exactly a gated lane), the cache-mode
and execution-mode checks, the tag taxonomy and prcore/integration
exclusions, and TestBazelCmdDoltJob (job steps, timeouts, .bazelrc
config equal to integration's build with no dolt skip or test selection,
the target's dolt-server rules, and bd_test's env repeated).
Measured locally (all fixes in this series, 32 shards, 16 at a time on a
shared host): 3267 top-level tests, all shards pass, max shard 313s (one
5001-issue export test, 214s, sets it), mean 100s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(init): hand the project installers a writer instead of swapping os.Stdout
The --quiet fix earlier in this series pointed the process-global
os.Stdout at the null device around the Claude/Codex/Cursor installers.
That works, but it mutates a global from inside a cobra Run: anything else
writing to stdout meanwhile (a goroutine, a future concurrent installer)
would vanish, and a concurrent read of os.Stdout is a data race.
The installers already write through env structs with stdout/stderr
fields. setup.ProjectInstallOutput carries the two writers, and
InstallClaudeProjectTo / InstallCodexProjectTo / InstallCursorProjectTo
use them (the old entry points keep the process streams). init passes
QuietProjectInstallOutput() under --quiet: progress discarded, error lines
still on stderr. The proxied-server init path (domain adapter
InstallClaudeProject now takes quiet) does the same, so its
"Installing Claude hooks..." is quiet too.
Consistent error policy under --quiet: errors still reach stderr. init's
own "Warning: failed to setup <installer>" lines were suppressed under
--quiet while the installers' "Error:" lines were not, and some installer
failures (Cursor rules) print nothing themselves; the warnings now print
whether or not quiet, like the installers' errors.
Coverage in required lanes: TestProjectInstallersReportToTheGivenOutput
(cmd/bd/setup) and TestEmbeddedInitA/basic, which now asserts that
`bd init --quiet` with agents enabled installs the Claude hooks and prints
none of the installers' progress (verified to fail with the quiet output
disabled).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(export): exercise the incremental-export fallback with a small threshold
TestTryIncrementalExport_ThresholdExceededFallsBack seeded
incrementalExportThreshold+1 = 5001 issues to cross the threshold. Batch
create is one transaction but still several statements per issue, so under
-race against the shared test server it took ~214 s, the longest test in
cmd/bd's Dolt-backed suite by far. The behaviour under test is only
"more changed issues than the threshold -> full export fallback".
incrementalExportThreshold becomes a var (same value), and the test, which
is not parallel, sets it to 20 and seeds 21 issues: 214 s -> 0.6 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): tighten the flag-reset self-test, guards and canaries
Review follow-ups:
- TestResetCommandFlagsRestoresEveryFlag no longer skips a flag it cannot
set: a flag type without a sample value, a Set error, or a non-pointer
Value (which resetCommandFlags could not restore) is an error, so a new
flag type cannot go untested. resetCommandFlags documents that it also
resets flag-bound package globals.
- TestCobraParallelPolicyGuard treats .ExecuteC/.ExecuteContext/
.ExecuteContextC like .Execute (the per-execution isolation writes the
mode globals; a parallel execution would race).
- TestConfigValidateReadOnlyIsHermetic ignores only the gate a readonly
command takes (.beads/dolt.gate.lock and its .info sidecar), not any
"*.gate.lock*" file, so a new unexpected gate still trips the canary.
- TestCLI_CreateRejectsFlagLikeTitles: a bare `bd create --foo-bar` must
fail and create no issue (bd-2c0's intent), next to the "--" cases.
- TestCustomStatusConfig cites why statuses come back in name order: the
normalized custom_statuses table (#2961, 8537d943b) has no ordinal and is
read ORDER BY name, like custom_types; the test predates it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the cmd/bd Dolt-server tier in 16 shards
Each shard pays ~30-40 s of fixed cost (TestMain plus its own dolt
sql-server), so 32 shards spent roughly 40% of the lane's executor time on
setup and added 32 farm actions to every PR touching cmd/bd. With the
5001-issue export test cut to under a second, 16 shards measure (locally,
--nocache_test_results): max 186 s, mean 137 s, 2,187 executor-seconds
(vs ~2,800-3,200 at 32), all 3,267 top-level tests, every shard passing.
TestBazelCmdDoltJob pins the count; bazel.yml's timing comment is
restated (cache mode ~20-30 minutes).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(proxy): fail fast with a MySQL error when the external upstream is unreachable
A proxied-server command against a stopped external Dolt upstream stalled
~20-30s before failing with a bare "invalid connection". The local db proxy
stays up, so on a refused backend dial (or a front such as stunnel/socat
that accepts and closes because its own upstream is gone) it simply closed
the client connection. The MySQL driver reports that as invalid
connection / unexpected EOF, which the uow bootstrap ping (#6003) classifies
as a transient drop and retries with exponential backoff for its whole 30s
budget. Retrying cannot help: the proxy already proved the upstream at its
own startup, and the outage is the upstream's.
The proxy now answers such a connection itself with a MySQL ERR packet
(2003, HY000) in place of the server greeting, naming the upstream and the
cause, exactly as a real server refuses a connection before the handshake.
The driver surfaces it as a *mysql.MySQLError, which the bootstrap retry
already treats as permanent, so the command fails in ~0.1s with an
actionable message. Only shapes that prove the upstream is not serving
qualify: ECONNREFUSED/ENOENT/EHOSTUNREACH/ENETUNREACH on dial, or a backend
that reaches EOF having sent zero bytes (a MySQL server always speaks
first). A dial timeout or a drop after the greeting keeps the plain close
and the client's transient retry.
External backends only: a managed (local sidecar) backend is owned by the
proxy and its dial can be refused transiently while bd drives it (the
full proxied lane caught TestProxiedServerInitExplicitBeadsDirIsAuthoritative
hitting exactly that mid-init when this applied to every backend), so
managed backends keep the bare close and the client's bootstrap retry.
TestExternalProxyOutageAndRestoreReDialsTCPAndUnix asserted the old
silent close during an outage; it now asserts the ERR packet.
With the latency fixed, TestProxiedOutageReconnectAcceptanceMatrix is
renamed into the proxied discovery pattern as
TestProxiedServerOutageReconnectAcceptanceMatrix and placed in the Bazel
30-shard block by gen_proxied_shard_manifest.py's incremental --write
(shard 8). The frozen legacy 15-shard block is untouched; the test
hash-distributes there like every other test added since it was frozen.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(proxy,uow): retry the proxy's upstream-outage report for ~1s; hedge the zero-byte case
Review follow-up to the fail-fast change.
A zero-byte close from the upstream is not proof it is down: Dolt closes
queued connections without a byte when max_connections/back_log are full,
when a queued connection times out, and while it shuts down for a
restart; an L4 front does the same while its target is replaced. Its
report now says "closed the connection before the MySQL greeting (down,
restarting, or at its connection limit)" instead of "unreachable", and
the comment no longer claims proof.
Refusals and endpoint churn also lost #6003's retry entirely, so an
external Dolt restart or a Service with no ready endpoints failed a
command at once. The client now retries the proxy's report
(proxy.IsUpstreamOutageError: 2003 with the "beads db proxy: " prefix)
for upstreamOutageRetryWindow (1s) from the first time it sees it, with
the backoff clamped so the last attempt lands at the window's end; other
transient errors keep the 30s budget. A real outage still fails in about
a second, inside the acceptance test's 3s bound.
Also: the dial report drops Go call-site wrapping ("dial tcp H:P:
connection refused"), truncation keeps UTF-8 intact, the comment states
how the packet differs from a real server's pre-handshake ERR (keeps the
'#'+SQLSTATE marker for vitess; 2003 is a client code chosen on purpose),
a Stats counter for reported outages, and the isTransientPingError
paragraph is reflowed and corrected.
Tests: pingWithRetry retries the report inside the window and gives up
at its end without using the full budget; openDB against a front that
plays the proxy (its real packet) and comes back 300ms after the first
connection succeeds in one call, and fails in about the window when it
never does; NewProxyServer enables the report for external backends
only; the dial message carries no Go type names; rune-safe truncation.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(proxied): fail, not skip, socat-bridged subtests in the Bazel lane
The external-unix front-door matrix subtest and both outage/reconnect
matrix subtests bridge an external endpoint with socat and skipped when
it was missing, so they could stop running without anyone noticing (the
parent test still passes). requireSocat now fails them when
BEADS_TEST_REQUIRE_SOCAT=1, which //cmd/bd:bd_proxied_test sets (its
remote workers have socat; a recent main run's external-unix subtest
passed there). The legacy GitHub 15-shard jobs have no socat and leave
it unset, so the subtests keep skipping there. The check now runs before
the upstream Dolt server is started, so a skip no longer pays for one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(proxied): measured weight for the outage matrix; pin it to legacy shard 1
Give TestProxiedServerOutageReconnectAcceptanceMatrix a duration entry
(80s: local runs measured 55-79s alone, 60-70s in its shard, and review
68-109s under load) and re-place it with the generator's incremental
--write: it moves from 30/8 to 30/11, one line, nothing else moved. The
heaviest estimated shard is now 435s against a ~357s median; a repack
would even that out but moves most assignments, so it is left to a
separate CI-only change that refreshes all durations together.
Legacy 15-shard block: the test would hash onto shard 14, already ~13.5
of the 15 min timeout on main. It is pinned to shard 1 (~6.5 min in
recent main runs) under the block's reviewed-pin rule; it skips there
today (no socat), and the pin keeps shard 14 safe if that changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Bazel client's JVM defaults to 25% of RAM, about 2 GB on the 8 GB
2 vCPU Blacksmith runners every remote lane runs on. //... analysis fits
today with little headroom, and the plan moves more lanes and targets
onto these runners. write-bazelrc.sh now writes
`startup --host_jvm_args=-Xmx4g`: a startup option, so it changes no
action key and only restarts the server. Verified with Bazel 9.2.0:
`bazel --bazelrc=<generated rc> info max-heap-size` reports 4294MB.
gascity's W1 byte-copies this composite, so the change lands here first.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
beads allows every action today (allowed_actions: all, no SHA pinning),
so a fork PR's workflow can call any action, including Blacksmith's
sticky-disk action that commits state later trusted runs restore.
.github/allowed-actions.json is the exact body of the selected-actions
setting: GitHub-owned actions plus gascity's list plus the third-party
actions beads uses. That includes DeterminateSystems/nix-installer-action,
which determinate-nix-action calls, and peter-evans/create-pull-request.
Every uses: under .github is already pinned to a full commit SHA. The new
ci_actions_policy_test.go keeps it that way: each reference is local or
owner/repo[/path]@<40-hex>, and allowed by the file, so the repository
setting can switch to allowed_actions=selected with
sha_pinning_required=true without breaking a workflow. Negative controls:
an @v6 tag ref and an unlisted owner/repo each fail the test.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Main's "Test (proxied-server cmd 5/15)" has hit the 15m -test.timeout in
6 of the 9 Main runs since #7129 where the shard ran to completion.
That PR renamed 12 proxied tests into TestProxiedServer* discovery.
They are not in the frozen 15-shard legacy block, so they hash-place,
and two landed on shard 5:
TestProxiedServerStorageClass (nine parallel inits, the heaviest of the
12) and TestProxiedServerServeRefusesAStaleCheckpointWithGone. Shard 5
went from 430-620s of job time to 800-950s. In those runs the timeout
fired while TestProxiedServerMolReadyGated's parallel subtests were still
queued behind the rest of the shard. That test is the victim, not a
pathological one. Shard 3 (also hash-loaded) runs 810-870s, within about
a minute of the same limit.
Measured locally (4 pinned CPUs, GOMAXPROCS=4), shard 5 took 897s with
its current test set and 524s with the pre-#7129 set.
Pin those two tests, plus shard 3's two heaviest hash-placed tests
(TestProxiedServerListComments and
TestProxiedServerServeRefusesAStreamOnADisabledJournal), onto the legacy
shards with the lowest measured job time: 14, 7, 4 and 1. Every test
already listed in the block stays where it is, so shard 5 is back to its
pre-#7129 test set. pr-risk.yml's fork jobs read the same block, so they
get the same relief. The script's 15m timeout, the job and step
timeouts, and the Bazel 30-shard block are unchanged. The block header
and the generator's legacy header now say a reviewed pin is allowed.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(delete): drop stale TestDeleteIssueUnsupportedStorage
The test asserted that deleteIssue returns "delete operation not supported
by this storage backend" when the global store is nil. That error came
from a type assertion to a deleter interface that a5da57e39 (Phase 7:
remove storage factory/memory backend/provider abstraction) removed:
store is now storage.DoltStorage, whose method set always includes
DeleteIssue, so there is no unsupported backend left to report. With the
assertion gone, the test's nil store is just a nil-interface method call
and panics at deleteIssue, killing the whole cmd/bd test binary.
The nil store is not reachable in production: deleteIssue's only caller
(bd doctor --check=pollution --clean) has already queried the store to
find the issues it deletes. So the test constructs an impossible state
for a removed behavior; delete it rather than reintroduce a dead branch.
No CI lane ran it: it needs the integration tag and a Dolt test server
(testDoltServerPort != 0), and every integration lane that builds
cmd/bd (main.yml integration cmd/bd shards, nightly Full Test Suite,
bazel --config=integration) sets BEADS_TEST_SKIP=dolt, while the
proxied/dolt-server lanes run only their TestProxiedServer*/
TestServerMode* manifests. It only ran in local integration runs with
Docker or a matching dolt.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): build the fallback bd by package path, independent of CWD
When BEADS_TEST_BD_BINARY is unset, buildEmbeddedBD and its sibling
helpers fall back to `go build -tags gms_pure_go -o X .`, which builds
whatever package is in the process's working directory. That is cmd/bd
only if the first caller of the sync.Once has not chdir'd into a fixture
and the test binary was started in cmd/bd. A precompiled test binary run
from the repo root (the pr-risk pattern) builds the root library package
instead: go writes a non-executable archive to X and the test dies with
"fork/exec .../bd: permission denied" (reproduced with TestEmbeddedCount).
Add goBuildBDCommand: the same non-race -tags gms_pure_go build CI
prebuilds BEADS_TEST_BD_BINARY with, naming
github.com/steveyegge/beads/cmd/bd explicitly and running in cmd/bd's
source directory, resolved at package init (this file's compiled-in path,
else the start-up working directory) before any test can chdir. Use it in
buildEmbeddedBD and the other copies of the pattern (cli_fast,
doctor_repair, explicit_db_nodb, init_permissions, scripttest,
shared_server, buildBDForInitTests); migrationFrontDoorBinary keeps its
untagged cgo build but gets the same package path and directory.
TestGoBuildBDCommandIsCWDIndependent checks it from a chdir'd temp dir
with go list (no link).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): stop in-process bd from leaking BEADS_DIR into later tests
TestResolvedConfigRepoRoot's two CWD-discovery subtests ("uses local
workspace from subdirectory", "uses worktree fallback when local .beads
is absent") failed in full cmd/bd runs with "no active beads workspace
found" but passed alone. Cause: runBDInProcess / runBDInProcessAllowError
run rootCmd.Execute in the test process, and PersistentPreRun exports the
selected workspace with a raw os.Setenv("BEADS_DIR", ...) that nothing
restores. After any TestCLI_* test, BEADS_DIR still names that test's
since-deleted bd-cli-test-* fixture; FindBeadsDir honors an explicit
BEADS_DIR before walking from CWD, finds nothing there, and reports no
workspace. Minimal repro: -run '^(TestCLI_Ready|TestResolvedConfigRepoRoot)$'.
The leak also broke later TestCLI_* tests themselves: with
-run '^(TestCLI_.*|TestResolvedConfigRepoRoot)$' against a Dolt test
server, failures drop from 47 to 20 with this change and no new ones (the
remaining 20 are leftover cobra flag values between in-process runs, a
separate issue).
- runBDInProcess(AllowError): snapshot os.Environ() around Execute and
restore exactly the keys it changed (restoreEnvSnapshot).
- TestResolvedConfigRepoRoot: clear BEADS_DIR (t.Setenv) in the two
subtests that test CWD discovery, so neither a leak nor a developer's
exported BEADS_DIR decides them.
Not host-specific (HOME, global git config and ceilings play no part),
and CI never saw it: the TestCLI_* leakers need a Dolt test server, and
every CI lane that runs cmd/bd's integration tests sets
BEADS_TEST_SKIP=dolt, so they skip and BEADS_DIR is never set.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(delete): expect storage.ErrNotFound after delete in TestDeleteIssueWrapper
"successful issue deletion" asserted that GetIssue returns (nil, nil)
for a deleted issue; the Dolt store reports a missing issue as
storage.ErrNotFound, so the subtest failed with "GetIssue failed: not
found: issue test-xxx" whenever it ran. Like TestDeleteIssueUnsupportedStorage
it needs the integration tag and a Dolt test server, which no CI lane
provides to cmd/bd (BEADS_TEST_SKIP=dolt), so it rotted unseen.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
TestSharedServerModeServeGlobalReportsTheServedDatabase was parked outside
the proxied discovery pattern because `bd serve --global` exited 1 on the
#5920 shared-store migration guard (v65 -> v69). The guard was right; the
test setup was wrong.
The workspace was a plain `bd init --server --server-port <test server>`
that only flipped BEADS_DOLT_SHARED_SERVER=1 afterwards. Shared-server
mode deliberately ignores the workspace's own dolt_server_port and
resolves the machine-wide shared server (BEADS_DOLT_SERVER_PORT, the
shared port file, ..., then the 3308 default), so every --global command
dialled 127.0.0.1:3308 rather than the hermetic test server. On CI nothing
listens there; on a developer machine running a real shared server, the
test read that server's own beads_global (created by an older bd at v65)
and serve asked to migrate it. beads_global was also never provisioned on
the test server, because only `bd init --shared-server` does that.
Set the workspace up the way an operator joins an externally-managed
shared server: `bd init --shared-server --external` (provisions
beads_global on the test server at the current schema) plus
BEADS_DOLT_SERVER_PORT naming that server. No product change: the guard
correctly refuses an un-consented migration of a shared database, and a
freshly created one (version 0) passes it.
Rename the test into the TestServerMode* discovery pattern and add it to
the Bazel 30-shard block via the generator's incremental --write. The
frozen legacy 15-shard block is untouched; the name hash-distributes there
(shard 12/15).
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
TestProxiedServerSQLDatabaseFlag (and its sibling
TestProxiedServerMultiStatementSQL) flaked in the proxied-server tier with
bd init --proxied-server failed: exit status 1
[mysql] packets.go:58 unexpected EOF
Error: failed to open uow provider: uow: ping db: dial tcp ...: connection refused
Both called bdProxiedInit with no backend flags, so every run spawned a
managed-local dolt sql-server. Its port is picked bind-and-close
(proxy.PickFreePort) when bd init writes the managed config and bound by
dolt only after the proxy child runs dolt config/init and boots the
server. Under the parallel tier another process can bind that port in the
gap. dolt then logs "Port N already in use." and exits, but
DoltServer.waitReady and the proxy's waitForServerReady only dial the
port, so they accept the foreign listener and the proxy publishes
proxy.pid. Its 100ms backend health check then sees dolt gone and shuts
the proxy down: init's first ping gets EOF, and every retry for the
rest of the 30s budget gets connection refused.
These tests cover bd sql, not the managed-local lifecycle, so move them
to newSharedProxiedProject like most of the suite. The shared server's
startup (testutil.localDoltServer) waits for dolt's own ready line and
retries on a new port, so it does not have this race. The second
database name now comes from the project's unique database, because the
server is shared.
The product race in the managed-local backend (dial-only readiness plus a
bind-and-close port) is unchanged. The code already marks it as deferred
in PickFreePort.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(git): canonicalize PinNoRepositoryUnderForTesting root so the pin matches on macOS
On macOS t.TempDir()/os.TempDir() live under /var/folders/..., but /var is
a symlink to /private/var and getcwd(2) reports /private/var/folders/....
underPinnedRootForTesting compared the raw pin against os.Getwd() with a
string prefix test, so the pin never matched there and
TestPinNoRepositoryUnderForTestingSurvivesResetCaches failed on every Main
macOS run (GetRepoRoot() returned the /private/var repo instead of "").
Store the pin both as given (absolute) and symlink-resolved, and match the
working directory in both raw and resolved form. Canonicalization can only
widen the fence, never narrow it (e.g. Bazel runfiles symlink trees keep
matching exactly as before).
TestPinNoRepositoryUnderForTestingMatchesThroughSymlinks reproduces the
macOS shape on any platform with a symlinked repo root; it fails without
this change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): make run-go-test-binary.sh and install-golangci-lint.sh parse under bash 3.2
The Main macOS job runs the scripts package tests with macOS's /bin/bash
3.2, where two constructs abort under `set -u`:
- run-go-test-binary.sh expanded the usually-empty PASSTHROUGH array bare;
bash < 4.4 treats that as an unbound variable ("PASSTHROUGH[*]: unbound
variable"), failing TestTestScriptPrebuiltTestBinaryContract and
TestRunGoTestBinaryDefaultsTimeoutWhenOmitted. Use the ${arr[@]+...}
guard.
- install-golangci-lint.sh used `declare -A`; bash 3.2 parses it as an
indexed array and `[amd64]=` dies as an arithmetic lookup ("amd64:
unbound variable"), failing
TestInstallGolangciLintFailsClosedOnChecksumMismatch. Replace it with a
case function and update TestGolangciLintInstallScriptPinned's pin check.
Reproduced and verified against a bash 3.2.57 build: all four tests fail
on origin/main and pass with this change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
actions/create-github-app-token (v3.2.0, pinned) deprecates its app-id
input in favor of client-id. rbe-prewarm's mint step now passes the
bazel-allocator App's public Client ID (Iv23ligrqVEhlamZnoPU, from
GET /apps/bazel-allocator) as a reviewed literal: it is not a credential,
and a literal keeps it policy-tested with the rest of the step instead of
in an unversioned repository variable. The private key stays a secret.
RBE_POOL_APP_ID is now unused, so its workflow_call declaration in
bazel.yml and its pass-through in pr.yml and nightly.yml are removed,
and a policy test rejects any workflow mentioning it again. The
repository secret can be deleted.
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md's "rbe-west Pre-warm" section now
documents the client-id, the single remaining secret, and
RBE_PREWARM_WORKERS (default 1, clamped to 4, 0 = kill switch, repo
variable currently 2). It also no longer cites a maintainer's local
planning-notes path; a new tracked-file policy test fails if one
reappears.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The prebuilt twins of the Windows liveness and worktree-remove jobs passed
alongside the native ones on every PR since F4 (#7135): #7139, #7145,
#7147, #7155, #7156, #7159, #7160 — 1-1.5 min vs 3-8 min native. Flip the
committed flag so ci-gate enforces the prebuilt pair; the native pair keeps
running as advisory and is the rollback (set the flag back to "false").
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): best-effort pre-warm rbe-west's OSS worker pool on PR open
bazel.yml's Bazel lanes each separately wait ~120s at "N actions, 0
running" while gastownhall/gascity's rbe-west NativeLink worker pool
scales up from zero. Add an advisory rbe-prewarm job (never a `needs`
of any lane, never in bazel-gate.sh's vocabulary, job-level
continue-on-error so it can never fail bazel.yml's own run) that
mints a short-lived installation token for a "bazel-allocator" GitHub
App (installed on gastownhall/gascity only, Actions read/write +
Metadata read) and dispatches gastownhall/gascity's
rbe-worker-pool.yml ahead of time, only when this run's lanes will
actually target rbe-west's "oss" instance (modes remote and fork-rw;
fork-ro targets a separate pool this doesn't serve).
Adds RBE_POOL_APP_ID / RBE_POOL_APP_PRIVATE_KEY as new optional
workflow_call secrets, threaded through every caller (pr.yml,
bazel-farm.yml, nightly.yml); either secret missing (the app not yet
provisioned) or repo var RBE_PREWARM_WORKERS=0 disables pre-warming
cleanly. Policy tests (scripts/bazel_rbe_prewarm_test.go, plus updates
to the existing bazel.yml structural tests) pin the mode gate, the
credential scope, the dispatch target, and that the job can never gate
or fail a run; each invariant was hand mutation-tested. Docs added to
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md ("rbe-west Pre-warm").
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): close rbe-prewarm token-theft path, fix stale secret-count docs (B1)
Security review of bdef342d5 found a BLOCKER (B1): the rbe-prewarm job
checked out inputs.checkout-sha (the fork head, reachable via
bazel-farm.yml's pull_request_target path) and then ran
.github/scripts/rbe-prewarm.sh with the minted gastownhall/gascity
dispatch token present. Script-file edits don't require workflow-approval
the way workflow-file edits do, so a VIP fork or same-repo branch pusher
could edit that script to steal the shared bazel-allocator key.
Fix: delete the checkout step and the .sh file; inline the dispatch logic
into the step's own `run:` block, whose bytes always come from the
trusted (approval-gated) workflow definition even under
pull_request_target. Mint the token in the immediately preceding step;
nothing after the mint step touches a repo-relative path. Stop forwarding
RBE_POOL_APP_ID/RBE_POOL_APP_PRIVATE_KEY from bazel-farm.yml (that fork
path now gets a cold start instead of pre-warming) and narrow the job's
`if:` to mode == 'remote' only (fork-rw never carried secrets anyway, so
it was dead weight - S1).
Also addresses the review's should-fix items:
- S2: document the widened trust boundary (the bazel-allocator key is
shared with gascity's own rbe-west scaler; leak blast radius; rotation
breaks the scaler, so the kill switch is unset-secret /
RBE_PREWARM_WORKERS=0, not rotation) as an accepted risk in bazel.yml's
header and engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md.
- S3: fix stale "exactly four secrets" claims (pr.yml/nightly.yml pass 6;
bazel-farm.yml now correctly passes 4 again) and the false "never a
long-lived credential" claim.
- S4: flag the continue-on-error-preserves-caller-success assumption as
reasoned-through but not live-validated in this sandbox.
- N1: drop the unused RBE_PREWARM_IDLE_MINUTES knob entirely.
- N4: strengthen the dispatch-target test to check exact gh command
strings and ban hardcoded repo/workflow literals outside $POOL_REPO/
$POOL_WORKFLOW.
Test changes (scripts/bazel_rbe_prewarm_test.go rewritten; ci_workflow_test.go,
bazel_farm_workflow_test.go, bazel_cache_mode_test.go updated for the new
job shape): new TestRBEPrewarmNoCheckoutStep,
TestRBEPrewarmNothingAfterMintTouchesARepoPath,
TestRBEPrewarmTokenOnlyInDispatchStep, and
TestRBEPrewarmAppSecretsOnlyExpectedCallers directly target the B1
mutation class; TestRBEPrewarmDispatchScriptBehavior keeps the fake-gh
behavioral coverage for the now-inlined dispatch script. Verified by hand
that both of the reviewer's mutations (widen the `if:` back to include
fork-rw; add a step after mint running a checked-out script) are now
caught.
Verification: go test ./scripts/..., go vet, gofmt, actionlint on the
four touched workflow files, make ci-pr-policy, make bazel-sync-check,
and `bazel test //scripts:scripts_test` all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
TestProxiedServerShowWatch/deleted_issue_polls_stay_quiet flaked with
"stdout grew after the delete" (PR #7139 run 37146882037, shard 25/30).
exec.Cmd copies Stdout and Stderr through two independent goroutines
reading two independent OS pipes, so there is no happens-before
relationship between the stderr "Watching for changes..." banner and
the render's own trailing bytes landing in the stdout buffer.
proxiedRenderIssue's final line is an unconditional trailing
fmt.Println, written after dependency/comment lookups, so the first
render's tail can still be in flight on the stdout pipe after the
banner already confirms the render started; capturing a "no growth"
baseline right after the banner is racy.
Reproduced directly via Bazel (--runs_per_test=40..50,
--local_test_jobs up to 80): 2/40 and 2/50 failures, with diagnostics
confirming "after" was just the tail of the first render's trailing
DESCRIPTION/blank-line output.
Fix the test's synchronization instead of the product: ping
watchBannerWriter.renders on every write (not just banner increments)
and add waitForQuiet, which blocks until a stream has been idle for a
bounded window. The delete subtest now waits for stdout to go quiet
after the banner before taking its "before" baseline, so the first
render has fully landed before growth is measured. No product code
changed; the "no output growth after delete" property is unchanged.
Verified: 60/60 and 80/80 runs clean post-fix (0/140), versus the
pre-fix reproductions above.
Note: an independent upstream PR (#7141, bee/show-watch-stdout-race)
also fixes this race via a different os.Pipe-based approach; this
commit addresses it independently per this worktree's task scope.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): embedded-Dolt tier 20/5 -> 40/14 duration-balanced shards (F1)
Give the Bazel-only bazel-embedded job its own duration-balanced manifest
blocks for //cmd/bd:bd_embedded_test (20 -> 40 shards) and
//internal/storage/embeddeddolt:embeddeddolt_embedded_test (5 -> 14
shards), decoupled from PR Risk's/main.yml's legacy frozen 20/5-shard
fork/push jobs, mirroring F2's bd_proxied_test precedent.
- Measured every TestEmbedded* / embeddeddolt test's wall time in
isolation; split 21 cmd-tier and 1 storage-tier single top-level test
over 180s into disjoint sub-tests (3 irreducible exceptions left:
TestEmbeddedUpdateConcurrent 475s, TestEmbeddedCloseConcurrent 308s,
TestLargeBatchApplyStatementCounts712_Embedded 261s).
- New scripts/ci/gen_embedded_{cmd,storage}_shard_manifest.py +
_embedded_shard_manifest_lib.py generate/check duration-balanced
manifest blocks (LPT packing, incremental --write/--repack, exact-once
--check), chosen at N=40/M=14 where the heaviest shard reaches the
irreducible-test floor.
- Fixed 18 stale legacy 20-shard cmd manifest entries (and 1 storage
entry) left over from earlier test splits, which would have hard-failed
every shard of PR Risk's real test-embedded-cmd jobs on their next run.
- New scripts/embedded_shard_count_test.go policy tests tie the
Bazel-only block's shard count to bazel.yml's check_shard_coverage.py
argument and catch generator staleness; generalized
TestBazelRetiredLanesCheckListedTestsRan's live-shard-count dispatch
from bd_proxied_test-only to a map covering both embedded targets;
removed TestBazelEmbeddedJobMirrorsEmbeddedTier's now-invalid
BUILD.bazel-shard-count-equals-PR-Risk-matrix assertion.
- Verified: go test ./scripts/..., bazel test //scripts:scripts_test,
actionlint, make ci-pr-policy, make bazel-sync-check all pass; full
local bazel test of both targets at 40/14 shards passes 2/2; mutation
testing confirms each new/changed policy test fails exactly when it
should.
See ~/beads-bazel-plan/f1/impl-report.md for full detail.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(embedded-dolt): skip 712-item statement-count assertion under -race (F1)
TestLargeBatchApplyStatementCounts712_Embedded genuinely costs ~260-300s
under -race (confirmed by solo re-measurement: 298.60s) -- 14014 real SQL
statement round-trips through the race-instrumented in-process Dolt engine,
not an artifact and not splittable (one indivisible ApplyBatchInTx
transaction). It was the storage tier's single-test floor for CI shard
packing.
Apply the exact precedent TestLargeBatchApplyWallClock_Embedded already set
for its 1000-item shape: skip this assertion under raceEnabled (the cost is
race-instrumentation overhead on the Dolt engine's own internal
goroutine/lock machinery, not this test's own logic) and extend nightly.yml's
"Embedded Dolt batch-apply suite (non-race)" step's -run regex so the full
pinned 14014-statement baseline still runs, non-race, every night. Verified
directly: under -race the assertion now skips in 0.00s; without -race it
still runs and asserts Total=14015 (within the existing +/-2 tolerance) in
36.76s.
Updates scripts/ci_workflow_test.go's
TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite to match the extended
regex.
Storage tier's new single-test ceiling under the per-PR -race lane drops
from ~260.7s to 152.38s (TestBatchApplyContract).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): correct cmd-tier long-pole durations, re-pack embedded manifolds (F1)
Investigated coordinator item 2 (single-test long poles): TestEmbeddedUpdateConcurrent
(recorded 475.39s) and TestEmbeddedCloseConcurrent (recorded 308.06s), plus
the two other >180s cmd-tier entries from the same capture
(TestEmbeddedInitA 196.94s, TestEmbeddedEpicCloseEligibleReason 192.47s).
Root cause: NOT host contention, NOT backoff/retry sleeps (bdRunWithFlockRetry
contributes negligibly), NOT race-instrumented-subprocess overhead alone.
These numbers were captured via `bazel --nohome_rc test --config=embedded`
with the then-current 20-shard legacy manifest (~10-11 tests/shard) and
`-test.parallel=4`. Go's t.Parallel() starts each parallel test's elapsed
clock when its function is invoked, before the Parallel() call blocks it
waiting for a free one-of-4 slot -- so a test queued behind several other
heavy shard-mates has that queueing wait baked into its own recorded
`--- PASS: TestX (Ys)` time. This is shard-composition-dependent scheduling
delay, not a cost intrinsic to the test.
Confirmed two ways in an isolated clone (not this worktree, per the
no-plain-go-test-from-a-beads-checkout rule):
- Solo (no rivals at all): 151.83s / 106.48s / 63.58s / 66.08s.
- All four together with the real -race -test.parallel=4 flags and exactly
4 tests requested (so nobody queues -- parallel slots == test count):
141.34s / 113.80s / 69.79s / 74.63s.
Both agree closely and both are far below the capture-time numbers.
F1's LPT packer already isolates the heaviest tests into shards with few or
no other heavy rivals (TestEmbeddedUpdateConcurrent lands alone in its own
40-shard-block shard), so under the new manifest these tests see little to
no queueing delay in real CI. None of the four needed splitting, parallel
subtests, or raceEnabled-scaled N: once correctly measured, none exceeds the
180s single-test guideline (new corrected weights: 155.0 / 115.0 / 72.0 /
77.0s). Corrected scripts/ci/embedded_cmd_test_durations.json accordingly
(with a doc-comment explaining the methodology and both confirmations) and
re-packed the 40-shard Bazel-only block (--repack --write); --check passes
(exact-once coverage of all 241 discovered tests). The frozen legacy 20-shard
block is untouched and independently re-verified exact-once by simulating
embedded-test-shard.sh across all 20 shards (241/241 tests, no
duplicates/misses/errors).
New cmd-tier single-test ceiling: 179.47s (TestEmbeddedCloseAlreadyClosed,
likely itself somewhat inflated by the same mechanism but already under the
180s line as recorded, so no action needed).
Also updates scripts/ci/embedded_storage_test_durations.json: the prior
commit's raceEnabled-skip for TestLargeBatchApplyStatementCounts712_Embedded
means it now costs ~0s in the per-PR -race lane bazel-embedded actually
packs for (not its real ~260-300s non-race cost, which nightly.yml still
exercises) -- weight set to 1.0 with a doc comment tying it to the skip in
large_batch_apply_measure_test.go. Re-packed the 14-shard Bazel-only block
accordingly; --check passes (exact-once coverage of all 387 discovered
tests, excluding TestMain/TestConformance per that file's existing
convention). The frozen legacy 5-shard block is untouched and independently
re-verified exact-once via embedded-storage-test-shard.sh simulation
(388/388 tests, no duplicates/misses/errors).
Verification: go test ./scripts/... (includes the nightly.yml policy test),
make ci-pr-policy, make bazel-sync-check, actionlint all pass. Real local
`bazel --nohome_rc test --config=embedded //cmd/bd:bd_embedded_test
//internal/storage/embeddeddolt:embeddeddolt_embedded_test`
(--local_test_jobs=8, both targets together, on a shared/noisy sandbox host)
passes both targets; per-shard wall time from each shard's test.xml: cmd/bd
max 218.9s (shard 23/40, min 20.6s), storage max 251.1s (shard 5/14, min
0.98s) -- down from the prior oversubscribed sweep's 783.41s / 351.94s and
well under the single-test-floor-based ~475s projection this slice started
from. Projected tier lane time (max of the two targets, real dedicated CI
runners should do at least this well): ~251s, under the 5-minute target.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(embedded): un-vacuous TestEmbeddedDepB edges, split epic-close test, trim concurrency stress N (F1 review)
Coordinator review follow-ups on the F1 embedded-Dolt resharding slice:
- TestEmbeddedDepB (one of TestEmbeddedDep's two CI-speed split halves) was
asserting its tree_* subtests against an edgeless graph, passing
vacuously; restores the same parent-child/blocks edges TestEmbeddedDepA
builds so those subtests exercise real multi-node traversal (review S2).
- TestEmbeddedEpicCloseEligibleReason split into
TestEmbeddedEpicCloseEligibleReasonFlag/...Output for CI shard balance,
matching the project's established single-concern split pattern.
- TestEmbeddedUpdateConcurrent/TestEmbeddedCloseConcurrent's
issuesPerWorker reduced 5->2: under real concurrent-shard-load
measurement these were genuine (not queueing-artifact) serialized-write
costs; 2 preserves every invariant the test checks (duplicate-ID
detection across all workers, one non-decreasing-list-count check per
worker) at a fraction of the serialized work.
- large712ShapeName: the 712-shape's race skip and its zero-match guard now
share one named constant instead of two independent literal shape-name
strings, so a future rename of the shape can't silently skip the pinned
statement-count assertion under race (review N2).
- Comment-only fixes (review N5) in files referencing test names since
renamed/split by this slice's earlier work: close_proxied_integration_test.go,
graph_apply_test.go, storage_class_proxied_integration_test.go,
tests/regression/discovery_test.go, plus doc-comment path cleanups (N1:
drop unverified ~/beads-bazel-plan paths/bead ids in favor of the
committed duration JSON files and engdocs/TESTING.md).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): re-derive embedded-Dolt durations under concurrent load; cap cmd shards at Bazel's 50-shard limit (F1 round 2)
Coordinator review required re-deriving every per-test duration weight from
a single, consistent, CI-realistic measurement (all shards running
concurrently, -test.parallel=4, race-enabled) instead of the prior pass's
mix of crowded-shard captures and solo/4-together spot-corrections, and
required measuring per-shard wall time via BEP testAttemptDuration rather
than a shard's outer test.xml <testsuites time> (which can include a
different, already-finished shard-mate's setup/teardown).
- scripts/ci/embedded_cmd_test_durations.json (242 tests) and
scripts/ci/embedded_storage_test_durations.json (387 tests) rewritten in
full from one consistent sweep. Every entry is now a fresh measurement;
none is a carried-over capture-time value or an unverified estimate. Both
files' _comment blocks retract the earlier "queueing delay baked into a
test's own reported elapsed time" theory (testing.go's (*T).Parallel does
not add wait time to a blocked test's own clock) and document the real
methodology.
- Only one cmd-tier test remains over 200s (TestEmbeddedListFiltering,
239.70s), already under the ~240s ceiling; no further top-level splits
were needed this round.
- New finding: Bazel hard-caps shard_count at 50 per sh_test-like rule
("Having more than 50 shards is indicative of poor test organization";
a native analysis-time limit, not configurable). The previous 88-shard
plan for //cmd/bd:bd_embedded_test (chosen as the LPT-packed floor before
this limit was discovered) is reverted to 50, Bazel's real ceiling.
Repacking the fresh weights at 50 shards gives a heaviest shard of
409.6s -- above the single-test floor of 239.7s, because 50 bins is not
enough headroom over the suite's ~20145 measured test-seconds to flatten
every bin to that floor. This makes the <=5 min lane target unreachable
for the cmd tier via resharding alone within Bazel's native per-rule
limit; see ~/beads-bazel-plan/f1/impl-report.md section 3c for the full
writeup and the two remaining levers (suite-wide speedups or a
multi-Bazel-rule split), neither attempted in this slice.
- internal/storage/embeddeddolt gets 15 Bazel-only shards (unaffected by
the 50-shard cap): LPT-packed heaviest shard 225.3s, within target.
- cmd/bd/BUILD.bazel, internal/storage/embeddeddolt/BUILD.bazel,
.github/workflows/bazel.yml, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md,
scripts/embedded_shard_count_test.go, scripts/pr_risk_bazel_coverage_test.go,
scripts/ci/gen_embedded_cmd_shard_manifest.py updated to the corrected
50/15 shard counts (and, for the generator, N3: --write without an
explicit total_shards now errors instead of silently regenerating the
frozen legacy 20-shard block).
- .github/scripts/embedded-{cmd,storage}-test-shards.txt: re-packed
Bazel-only blocks at 50/15; frozen legacy 20/5 blocks untouched except for
N5 fixes to stale entries left by this slice's earlier test splits
(TestShardScriptsListOnlyRealTests catches this class of bug; both
legacy blocks reconfirmed exact-once).
- .github/workflows/nightly.yml / scripts/ci_workflow_test.go (N4): the
embedded-dolt batch-apply nightly step now runs with `if: ${{
!cancelled() }}` and its own explicit `-timeout 20m`, independent of the
job-level timeout, with both halves now asserted by
TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite.
Verified: gofmt -l clean, go vet ./scripts/... clean, go test ./scripts/...
green, both generators' --check pass at 50/15 (and legacy 20/5 remain
exact-once via the frozen-block invariant).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): rename embedded manifest staleness tests out of the ^TestEmbedded skip
The allowlisted go-test leg runs with -skip ^TestEmbedded, so tests named
TestEmbedded*ShardManifestGeneratorNotStale never ran there and the leg
failed with "matched no test that ran".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci: move same-repo PR Linux/Windows/macOS legs to Blacksmith, shard preflight (F7b)
PR-path jobs for same-repo PRs/merge_group now run on Blacksmith instead
of GitHub-hosted runners, following F4/F7a's ternary-expression and
restore-only-cache patterns; forks/Dependabot are unaffected:
- scripts-go-checks, pr-lint-wrapper: same-repo Blacksmith 4vcpu Linux.
- check-doc-freshness-platforms, pr-preflight-platforms: restructured
from a top-level matrix.os list to an include:-only "matrix marker"
shape (runner: same-repo-linux/-windows/-macos) so each host can
resolve an independent same-repo Blacksmith selection via one chained
ternary runs-on expression.
- Six standalone Windows jobs (test-windows-liveness,
worktree-remove-windows, test-windows-small,
test-windows-liveness-prebuilt, worktree-remove-windows-prebuilt,
windows-make-shell) move to same-repo Blacksmith Windows
(blacksmith-{2,4}vcpu-windows-2025, public beta). CGO/mingw-w64 and
Chocolatey preinstall are unconfirmed on that image; both risks are
flagged with one-line rollback instructions at the call sites.
- pr-preflight-platforms' Windows leg is split into two shards (common
steps on shard 1, the three Windows-only heavy steps on shard 2) to
shorten its critical path; shard 2 downloads Go modules directly
instead of sharing shard 1's runner/cache.
main.yml gains cache savers for the newly same-repo-Blacksmith PR
consumers, since Blacksmith and GitHub-hosted runners cannot see each
other's actions/cache entries even for identical keys:
- blacksmith-go-build-cache warms the Blacksmith-selection race/non-race
GOCACHEs scripts-go-checks and pr-preflight-platforms/check-doc-
freshness-platforms' ubuntu legs need, push-to-main only. Its non-race
leg shares scripts/ci/warm-non-race-cache.sh with build-artifacts, and
compiles tools/bazel/run_allowlisted_go_tests.py's allowlisted
packages via its new --compile-only flag, so the two paths and the
warm-up can never drift from what they actually run.
- pr-lint-wrapper, go-vet-cache, test-windows gain a venue: [blacksmith,
github] matrix leg to seed both runner selections pr.yml's
same-repo-Blacksmith-or-fork-hosted consumers can land on.
Updates the policy-test suite (ci_workflow_test.go,
ci_blacksmith_runner_test.go, pr_lanes_bazel_coverage_test.go) to cover
the new runner selections, matrix-marker shape, shard gating, and cache
topology, including a dedicated inventory/writer check for
blacksmith-go-build-cache.
Deferred follow-up (not done here): sharding check-doc-freshness-
platforms and the standalone Windows jobs, and moving/sharding macOS
legs beyond what check-doc-freshness-platforms'/pr-preflight-platforms'
matrix-marker form already covers.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): F7b review response -- msys2/Blacksmith-saver guards, macOS stays hosted (B1/B2/S3)
Addresses ~/beads-bazel-plan/f7b/review.md's two cache-safety blockers and
one follow-up gap found after F7b moved same-repo PR Linux/Windows/macOS
legs onto Blacksmith and added push-to-main Blacksmith cache seeders:
- B1: windows-make-shell's msys2/setup-msys2 step caches its installed
packages by default, which implicitly SAVED a GitHub Actions cache entry
from a same-repo PR the moment it ran on a Blacksmith runner (confirmed
via a real cache-list entry from a refs/pull/.../merge ref). Disabled with
a fail-closed `cache: ${{ runner.environment == 'github-hosted' }}`, and
generalized the sweep (TestBlacksmithReachablePRJobsDisableDefaultCachingActions)
to cover any default-caching setup-* action on a Blacksmith-reachable job,
not just this one.
- B2: blacksmith-go-build-cache, pr-lint-wrapper, go-vet-cache and
test-windows (main.yml) had no (or an incomplete) same-repo push-to-main
guard, so a same-repo fork with push access to its own main could run --
and save a cache entry from -- a paid Blacksmith job. Added the exact
push-to-main/same-repo `if:` F7c's own blacksmith-setup-go-cache already
uses, pinned by TestBlacksmithSaverJobsGuardedAgainstPullRequest.
- S3: the macOS leg of pr-preflight-platforms/check-doc-freshness-platforms
resolved to Blacksmith for same-repo PRs; Blacksmith bills macOS at
roughly 20x its Linux/Windows rate and no push-to-main Blacksmith-macOS
saver exists, so that leg would pay the premium for a guaranteed cold
cache. macOS now stays unconditionally on GitHub-hosted macos-latest.
Also pins S7 (blacksmith-go-build-cache/go-vet-cache/test-windows's cache
keys are bounded by go.sum+UTC-day, not github.sha) and S6/S8 (their
20-minute timeouts), both already in place but previously unpinned, and
replaces run_allowlisted_go_tests.py's fragile GO_TEST_FLAGS[-2:] slice with
an explicit BUILD_FLAGS constant (N1).
Re-ran the reviewer's 15 mutations (~/beads-bazel-plan/f7b/review/mutate.py):
all now killed or structurally moot (the Windows preflight shard split this
review asked to revert-unless-helpful was reverted, in an earlier commit on
this branch, removing the vulnerability surface 6 of them targeted).
scripts/BUILD.bazel: add the new ci_f7b_review_test.go to scripts_test's
srcs -- bazel test caught it missing (file existed on disk but was not
compiled into the Bazel-built test binary).
Verification: go vet/test/build, gofmt, bazel --nohome_rc test
//scripts:scripts_test, actionlint -shellcheck, shellcheck, make
ci-pr-policy, make bazel-sync-check all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): drop local review-doc path from comment
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(advisory): move Conformance/Regression/Migration/Smoke/Docs/Proxied-Local to Blacksmith, fold matrices, path-filter, drop redundant nix-build PR trigger (F7c)
Reduces per-PR GitHub-hosted job load for advisory (non-required) workflows
per the "PR CI -> 5 min" plan, slice F7c. None of these workflows gate the
required CI Gate / PR Risk Gate checks.
- conformance.yml: same-repo PR/merge_group runs move to Blacksmith
(4vcpu); PR trigger gets the shared "upgrade-relevant code" path filter
(any non-test Go change or build input); also now runs on push to main.
- regression.yml: fold detect-regression-need into regression's first step
(gated via `if:`), eliminating a whole job; move to Blacksmith.
- migration-test.yml: fold the 14-version x cache-identity matrix into 3
shards (src/pre-dolt/dolt-runtime), looping every version in a shard via
a new "Verify explicit historical upgrades" step so a bad version does
not hide the rest; gains the shared path filter; move to Blacksmith.
- cross-version-smoke.yml: fold the 1-job-per-version matrix into chunks of
5 (delegated to upgrade-smoke-test.sh's native SMOKE_VERSIONS support);
gains the shared path filter; move to Blacksmith.
- docs-mintlify.yml, proxied-local-smoke.yml: move to Blacksmith
(2vcpu/4vcpu); add missing timeout-minutes where absent.
- nix-build.yml: drop the pull_request trigger — pr-risk.yml's required
test-nix job (nix run + nix flake check) is a strict superset of
`nix build .#default`, pinned by a new regression test.
- main.yml: add blacksmith-setup-go-cache, a push-to-main-only job that
warms setup-go's implicit cache with cache: true, since Blacksmith
runners cannot see GitHub-saved actions/cache entries that the six
workflows above now rely on via setup-go's own caching. Carves a scoped,
documented exception into assertPinnedGoCacheActions for this one job;
every other setup-go step in main.yml/pr.yml/pr-risk.yml still must pin
cache: false.
Forks and Dependabot PRs are unaffected (still ubuntu-latest, no secrets),
via the same conditional runner expression used elsewhere in the repo.
New policy tests in scripts/ci_f7c_advisory_test.go pin: path-filter
identity across the three filtered workflows, full version-set coverage
of the migration-harness fold (no scenario dropped), the per-shard/per-
chunk retry-all-versions-even-on-failure loop, fork/dependabot runner
selection semantics, no Blacksmith job reads secrets, and that the new
cache-seed job exists and that its six consumers don't disable the cache
it warms. All new tests and their underlying behaviors were mutation-
tested (each fails when the behavior it pins is reverted).
Caveat for rollout: the Blacksmith cache-seed job assumes Blacksmith
labels have branch-protected cache isolation from fork/untrusted PRs at
the org level (spec-f7.md Group B precondition) — this needs confirming
with an org admin before merge, since the seed job's cache becomes shared
infrastructure the six advisory workflows implicitly trust.
Replayed the last 60 merged PRs' changed-file lists against the new path
filter: 20/60 would now skip Conformance/Migration/Smoke (matches the
spec's estimate), and none of the 60 touched non-test Go code while being
classified as skip.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,advisory): F7c review B1/B2 - pin migration fallback, restore-only Blacksmith caches
B1: migration-test.yml's non-Blacksmith historical-upgrades fallback was
`ubuntu-latest`, which actions/runner-images#14748 schedules to move from
24.04 to 26.04 starting 2026-10-19. 26.04 drops libicu74, which the v0.55.4
fixture's ICU ABI depends on (and which "Install test dependencies" installs
explicitly). Pin the fallback to the literal `ubuntu-24.04` label instead, and
add the matching sameRepoBlacksmith4vcpuNoble constant so the pin itself is
tested. Confirmed via `git show <pre-F7c>` that none of the other five
workflows F7c touched ever had a pinned-OS fallback, so this regression was
scoped to migration-test.yml only.
B2: Blacksmith cannot see GitHub-hosted cache entries, and a same-repo PR
writing a cache entry another PR or main later reads back unverified is a
poisoning vector. Make every advisory consumer (conformance, regression,
migration-test, cross-version-smoke, docs-mintlify, proxied-local-smoke)
disable setup-go's own implicit cache on self-hosted runners and restore-only
from a self-defined `blacksmith-sg-v1-...` key namespace (not setup-go's own
implicit key, so there is no dependency on Blacksmith's $ImageOS matching
GitHub-hosted resolution). Only main.yml's new blacksmith-setup-go-cache job
(push-to-main only, always trusted) saves: it always restores, unconditionally
re-runs every warm-up command (no longer gated on a cache hit, so a stale or
partial restore self-heals), and always saves under the same key. Delete the
now-unneeded goCacheImplicitCacheExceptions carve-out in ci_workflow_test.go.
Also convert cross-version-smoke.yml's smoke-binaries-* and regression.yml's
regression-baseline-* caches from a monolithic actions/cache (auto-saves on
any miss) into an explicit restore (always, modulo the job's own gate) / save
(gated off pull_request) pair, closing the same poisoning vector for
per-release binaries. migration-test.yml's historical-dolt-* cache is left
unchanged: it is already checksum-verified on every read by
scripts/migration-test/lib/binary.sh.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci,advisory): F7c review SF - behavioral tests, path filters, F7a naming coordination
Replace substring-matching tests with behavioral ones that execute the real
embedded bash/jq against synthetic input, closing the reviewer's 12 surviving
mutations:
- TestMigrationHarnessLoopExecutesBehaviorally runs the real "Verify explicit
historical upgrades" step against a stub run.sh: every version in a shard
still runs after an earlier one fails (closes `exit 0`/`break` mutations),
failures are ::error:: annotated and written to $GITHUB_STEP_SUMMARY, a bad
or empty SHARD_VERSIONS fails closed without ever invoking run.sh, and a
hung version is bounded by `timeout --kill-after` without hiding the version
queued after it. The step's env now exports SHARD so its run: body has no
literal ${{ }} GH-expression syntax left for a test harness to choke on.
- TestMigrationHarnessCacheHashMatchesDerivation pins each shard's cache-hash
against sha256(join(",", <shard's cache-identity strings>))[:16], recovered
from the pre-fold 14-entry table (git show 362eecc6bf), closing a stale-
cache-hash mutation.
- TestMigrationHarnessDoltRuntimeStepGatedByShard pins the exact gate
expression for "Install Dolt test runtime".
- TestCrossVersionSmokeChunksEveryResolvedVersion now executes the real jq
chunking and join programs (via the new ghExtractJQProgram helper) against
synthetic version lists of several sizes, and asserts the smoke step's run:
command is an exact match - closing jq-slice-off-by-one and
dropped-SMOKE_VERSIONS mutations a substring check could not catch.
- TestRegressionStepsAfterDetectAreGated pins that every step after
regression.yml's detect step stays gated on
steps.detect.outputs.run_regression == 'true', closing an
ungated-regression-step mutation.
- TestBlacksmithSetupGoSeedExistsForAdvisoryConsumers,
TestAdvisoryBlacksmithConsumersAreCacheRestoreOnly (replacing
TestAdvisoryBlacksmithConsumersKeepImplicitSetupGoCache),
TestBlacksmithSetupGoCacheKeysMatchAcrossSeederAndConsumers, and
TestAdvisoryBinaryCachesAreRestoreAlwaysSavePRGated cover the B2
restore-only/seeder-saves cache topology end to end, including a
seeder/consumer key-match check and scoping the "no save in a consumer job"
check to the blacksmith-sg-v1- namespace specifically so migration-test.yml's
legitimate, checksum-verified historical-dolt-* cache is not flagged.
Path filters: add scripts/migrate-legacy-to-current.sh to migration-test.yml
(the harness invokes it directly but it lives outside scripts/migration-test/)
and internal/storage/embeddeddolt/conformance_test.go plus backend/conformance/**
to conformance.yml, with a new TestConformancePushPathsMatchPullRequestPaths
pinning push.paths and pull_request.paths stay identical.
Coordination with F7a (ci/f7a-blacksmith-fold, lands before F7c): F7a
independently defines its own sameRepoBlacksmith4vcpu/2vcpu constants and a
TestSameRepoBlacksmithExpressionSemantics, and is moving a real GH-Actions
expression evaluator into a shared scripts test helper file. Per the
coordinator, rename this branch's same-named test to
TestF7cAdvisorySameRepoBlacksmithExpressionSemantics (deliberately left as the
same simple literal mirror it always was, not a competing evaluator) so it
does not collide with F7a's helper on rebase; at rebase time this test should
be reduced or replaced in favor of F7a's shared evaluator.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(ci): update stale nix-build PR-trigger and renamed check-name references (F7c review S6)
F7c (bb81136203) dropped nix-build.yml's pull_request trigger as redundant
with PR Risk's test-nix job, and folded cross-version-smoke.yml's one-job-
per-version matrix into one job per 5-version chunk. Update the docs that
still described the old topology:
- CONTRIBUTING.md: nix build .#default no longer runs on PRs; point at PR
Risk's required test-nix job instead.
- update-vendor-hash.yml: same correction in its header comment and the
GITHUB_TOKEN-retrigger caveat.
- engdocs/CI_TEST_SURFACE_AUDIT.md: migration-test.yml now also runs on PRs
(path-filtered, advisory) across 3 shards instead of 14 per-version legs;
nix-build.yml's row now reflects push-to-main-only.
- engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md: replace `Upgrade smoke
(<version> -> candidate)` with the current `Upgrade smoke (chunk N)` job
name in both the "do not require directly" list and the Cross-Version
Smoke section; rewrite the Nix Build section to describe nix-build.yml as
push-to-main-only (no longer a PR check to warn about requiring directly)
and remove the now-unreachable `nix build .#default` bullet from the
"do not require" list.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(advisory): rebase onto F7a shared Blacksmith helper, drop duplicate const/evaluator
F7a (PR #7139, ci/f7a-blacksmith-fold) landed its own sameRepoBlacksmith4vcpu
const and TestSameRepoBlacksmithExpressionSemantics, plus a real GitHub
Actions expression evaluator (evalGHExpr/mustEvalGHRunsOn) in a new shared
scripts/ci_blacksmith_runner_test.go. This branch rebased onto
origin/ci/f7a-blacksmith-fold and reconciles the two:
- Removed this branch's duplicate sameRepoBlacksmith4vcpu definition in
ci_workflow_test.go (rebase conflict); it's now served solely by F7a's
shared const. sameRepoBlacksmith4vcpuNoble (F7c's B1 fix, fallback pinned
to ubuntu-24.04 instead of ubuntu-latest) is kept as its own const, as
instructed, since it is F7c-specific and not shared.
- Replaced TestF7cAdvisorySameRepoBlacksmithExpressionSemantics's
hand-written Go mirror of the same-repo Blacksmith boolean logic with real
evaluation of each F7c advisory job's actual, as-parsed runs-on expression
text via F7a's evalGHExpr/mustEvalGHRunsOn, across the same event-shape
truth table F7a's own test uses (same-repo/fork/dependabot/deleted-fork-
head/merge_group/push/pull_request_target/schedule/workflow_dispatch).
This covers all 8 F7c advisory Blacksmith jobs, including
migration-test.yml's historical-upgrades job falling back to the literal
ubuntu-24.04 label rather than ubuntu-latest. Confirmed via a cp-backed
mutation (flipping the Noble fallback back to ubuntu-latest) that this new
test, unlike the deleted hand-written mirror, actually catches a
regression in the real expression text.
- Merged two more rebase conflicts in ci_workflow_test.go: the
assertNoUnmanagedGoCacheSteps main.yml allowlist (F7a's
windows-test-binaries-cache plus F7c's blacksmith-setup-go-cache) and a
subsequent non-conflicting-but-incomplete 3-way merge of
TestSameRepoBlacksmithRunners' main.yml entry (added
windows-test-binaries-cache alongside blacksmith-setup-go-cache so the "no
other Blacksmith label leaks" sweep covers both).
Did not merge TestBlacksmithAdvisoryJobsReadNoSecrets into F7a's
TestBlacksmithJobsReadNoSecrets (ci_workflow_test.go) even though the shared
helper file's doc comment suggests it; out of scope for this round per the
coordinator's explicit instructions, left as a follow-up.
Re-ran full verification after rebase + reconciliation: go build/vet/test
./scripts/..., bazel --nohome_rc test //scripts:scripts_test, actionlint
-shellcheck=shellcheck across all touched workflows, make ci-pr-policy, make
bazel-sync-check - all green. gofmt -l clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,advisory): close F7c re-review blocker X1 and S1-S5/N1 gaps
Re-review (~/beads-bazel-plan/f7c/rereview.md) found one blocker and five
should-fix gaps against ci/f7c-advisory-workflows:
X1 (blocker): migration-test.yml's historical-dolt-* cache was still a bare
actions/cache that auto-saves on any key miss on Blacksmith for same-repo
PRs. tar -P extraction plus lib/binary.sh's cp -f following a planted
symlink can redirect a write before sha256 verification ever runs, so the
"checksum-verified" exemption was unsound. Converted to the same
restore-always/save-off-PR split as smoke-binaries-*/regression-baseline-*,
dropped the exemption comments, and added it to advisoryBinaryCaches.
S1: TestRegressionDetectStepBehavioral executes regression.yml's detect
step's actual bash against a scratch git repo, closing mutation M13 (detect
step hard-wired to emit run_regression=false).
S2: TestBlacksmithReachableAdvisoryJobsNeverSaveACache sweeps every job in
every F7c advisory workflow (plus main.yml) whose runs-on resolves to a
Blacksmith label under a same-repo pull_request/merge_group context, and
fails on any bare actions/cache, any un-gated actions/cache/save, or any
setup-go/setup-node/setup-python step that would still cache there. Also
added a job-level `if: github.event_name == 'push' && github.ref ==
'refs/heads/main' && github.repository == 'gastownhall/beads'` guard to
main.yml's seeder job (TestBlacksmithSeederGuardedAgainstPullRequest) so a
future pull_request/merge_group trigger on main.yml can't turn the seeder
into a same-repo-PR-reachable cache writer.
S3: advisoryBinaryCaches' save-gate check is now exact-equality, not
strings.Contains, so a `... || true` gate can no longer slip past it.
S4: conformance.yml's path filters now include
internal/storage/embeddeddolt/test_fixture_test.go, which holds TestMain
for the whole package (moved out of conformance_test.go) and was previously
excluded by the blanket !**_test.go negation.
S5: main.yml's blacksmith-setup-go-cache seeder keyed its save on
github.sha, so every push to main - regardless of whether go.sum changed -
wrote a new multi-GB ~/go/pkg/mod + ~/.cache/go-build entry, risking LRU
eviction of F7a's own Blacksmith-seeded caches. Added a "Compute cache
date" step and keyed on the UTC calendar day instead, capping writes to at
most once per day; the same step/key was added to all 6 consumers so their
restore key stays byte-identical to the seeder's save key
(TestBlacksmithSetupGoCacheKeysMatchAcrossSeederAndConsumers).
N1: flipped `cache: ${{ runner.environment != 'self-hosted' }}` to
`cache: ${{ runner.environment == 'github-hosted' }}` on setup-go's own
implicit cache across all 7 call sites, so an empty/unknown
runner.environment value fails closed (caching off) instead of open.
Verification: go build/vet/test ./scripts/..., bazel --nohome_rc test
//scripts:scripts_test, actionlint -shellcheck=shellcheck
.github/workflows/*.yml, make ci-pr-policy, make bazel-sync-check all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,advisory): close merge_group gap in binary-cache save gates
Running TestBlacksmithReachableAdvisoryJobsNeverSaveACache against every
Blacksmith trust context it is reachable under (instead of stopping at the
first match found, which made the check's coverage depend on Go's
randomized map iteration order) surfaced a real gap the S2 sweep was meant
to close: smoke-binaries-*, regression-baseline-* and historical-dolt-*'s
save gates were all `if: github.event_name != 'pull_request'`, a deny-list
that also admits a hypothetical future merge_group event. All three jobs'
runs-on expressions already resolve to a Blacksmith label under
merge_group, so a merge_group run would have been able to write those
caches too - the same same-repo-writer risk X1 closed for pull_request,
just not currently reachable since none of these three workflows has an
`on: merge_group:` trigger today.
Changed all three save gates to an exact allow-list of the triggers they
actually intend to write from (push, workflow_dispatch) rather than a
deny-list of the one trigger they don't. Updated
TestAdvisoryBinaryCachesAreRestoreAlwaysSavePRGated's wantSaveIf strings to
match, and made TestBlacksmithReachableAdvisoryJobsNeverSaveACache check
every blacksmithTrustContexts entry per job (sorted for determinism)
instead of breaking on the first Blacksmith-reachable context, so this
class of context-dependent gate gap can't hide behind map iteration order
again.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci,advisory): directly assert main.yml has no PR-reachable trigger
TestBlacksmithSeederGuardedAgainstPullRequest's job-level guard defuses the
vulnerability structurally, but a mutation that only edits main.yml's on:
block (adding pull_request:) without touching the job-level if: was not
caught by any existing test (confirmed: mutate.py's N6 probe survived).
Add TestMainWorkflowHasNoSameRepoPRReachableTrigger, the direct half of
F7c review fix S2's "pin the seeder's trust boundary" ask: assert main.yml's
on: has none of pull_request/pull_request_target/merge_group/workflow_run,
and still has push. Confirmed this kills N6 (manually applied and reverted
via a cp backup).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37159233834
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
TestAddWispDep_InfraWispToInfraWisp/agent flaked under
//internal/storage/dolt:dolt_server_full_test shard 12/16 with "failed
to initialize branch-local ignored schema: schema migration: schema:
acquire migration lock: schema migration lock unavailable: timeout"
(job 111272709796).
Root cause: schema.MigrationLockName keys the GET_LOCK name by
database name only, not by branch. setupTestStore gives every test its
own Dolt branch (testutil.StartTestBranch) but all of them share one
database (testSharedDB), so their branch-local schema passes
(initSchemaOnDB, called after the branch checkout to materialize
branch-local ignored-migration state) all contend for the identical
named lock. AcquireMigrationLock's GET_LOCK wait is a hard 5s budget
(migrationLockAcquireTimeoutSeconds) — too short to always win under
host load with multiple contenders.
Production already has a deterministic answer to this exact shape of
contention: initSchemaOnDBWithRetry wraps the same call in a 30s
exponential-backoff retry whose isRetryableError check treats
schema.IsMigrationLockError as transient ("concurrent bd processes
converge instead of failing one unlucky waiter" — see store.go). New()
already goes through that retrying path for a store's initial schema
init; only setupTestStore's second, branch-local pass called the bare,
non-retrying initSchemaOnDB, skipping the resilience production
relies on for the identical lock. TestConcurrentInitSchema already
demonstrates this same contention/retry shape against a fresh
database with 20 concurrent callers, corroborating the mechanism.
Fix: call initSchemaOnDBWithRetry instead of initSchemaOnDB for the
branch-local pass. This is not a timeout bump — it reuses production's
existing contended-lock retry budget instead of inventing a new wait.
Verification: reproduction attempts (up to 300 concurrent Bazel test
actions, and a cgroup-throttled run limiting the whole job to a ~1.5
CPU-core budget while running up to 80 concurrent actions, confirmed
throttling via cpu.stat) did not trip the pre-fix race on this
higher-core sandbox hardware, unlike the original 4-vCPU CI runner;
root cause is established via static analysis plus the matching
TestConcurrentInitSchema precedent instead. Post-fix, 40 runs under
the same cgroup-throttled load passed cleanly (0 failures).
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Under Bazel, prlintmake_test resolved go.mod's toolchain (go1.26.7)
through whatever go the executor has on PATH. When that is another
release, gofmt-bin.sh's GOTOOLCHAIN=go1.26.7 fallback downloads the
toolchain from proxy.golang.org on every run, and fails where actions
have no network: rbe-west's fork pool runs every action in its own
network namespace with loopback only (NETNS=1). Three tests failed
there (TestGofmtBinMatchesGoModToolchain, TestGofmtBinIgnoresPathGofmt,
TestFmtCheckReportsGofmtVersion) and passed with NETNS=0.
The registered SDK is go.mod's toolchain (bazel_policy_test pins them),
so declare it as data and have testGo put it first on PATH. gofmt-bin.sh
then takes its "host go already is the pinned one" path and fetches
nothing. Plain `go test` is unchanged.
Co-authored-by: gascity-agent <agent@gascity.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: F7a same-repo Blacksmith moves and job folds (pr.yml, pr-risk.yml)
Move cache-free same-repo pr.yml/pr-risk.yml jobs onto the org's Blacksmith
runner pool (forks/Dependabot stay ubuntu-latest), and fold several
independent single-purpose jobs into fewer multi-step jobs, to cut same-repo
PR queue time without changing either required gate's name or its token
contract:
- fast-checks: folds check-build-tags, check-version-consistency,
check-migration-hygiene, check-no-beads-changes and fmt-check into one
Blacksmith job with five independent steps; ci-gate reads each step's
outcome via job outputs into the same CI_GATE_REQUIRED token the standalone
job used, plus a FAST_CHECKS backstop token for a lost job.
- check-release-target-cross-compilation: 8 per-target matrix legs collapse
to 2 (unix/desktop), driven by the new scripts/ci/release-targets.txt
manifest (kept in sync with .goreleaser.yml) and
scripts/ci/check-release-cross-compile.sh.
- advisory-reports: folds build-examples and complexity-report (both
already advisory) into one job, keeping their pre-fold
step-level continue-on-error/timeout design.
- windows-make-shell: 3-way host matrix folds into one job with an
aggregator step reproducing the old native/MSYS2/Cygwin pass/fail rules;
test-windows-doltversion and test-windows-dbproxy-server fold into
test-windows-small (neither was gated).
- Runner moves: fast-checks, advisory-reports,
check-release-target-cross-compilation, check-doc-flags,
pr-policy-wrapper, test-dolt-server-fingerprint (pr.yml) and test-nix
(pr-risk.yml) move to the same-repo Blacksmith expression at their
measured vCPU size.
Policy tests: extend TestSameRepoBlacksmithRunners for the moved/added jobs,
add TestSameRepoBlacksmithExpressionSemantics, TestBlacksmithJobsReadNoSecrets
and TestPRCIGateFastChecksTokens, retarget the complexity/build-examples and
dolt-server-fingerprint tests, rewrite
TestReleaseTargetCrossCompilationMatrixMatchesGoreleaser against the new
manifest, and teach the ci-gate simulation helper to resolve
needs.fast-checks.outputs.*. Document the moves/folds and the Blacksmith-off
revert set in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): windows-make-shell/test-windows-small fail natively, raise advisory-reports headroom (review SF-1, SF-2, N-2, N-3)
- windows-make-shell: drop continue-on-error from install/exercise steps
(kept only on the Cygwin setup step), delete the hand-written "Fail if a
required Windows Make shell leg failed" aggregator, and delete the unused
native/msys2/cygwin job outputs. A failing leg now reds the job via native
GitHub Actions propagation instead of relying on an aggregator step that a
mutation showed could silently undershoot (SF-1, N-3).
- test-windows-small: same fix - drop continue-on-error from doltversion/
dbproxy-server steps and delete their aggregator step.
- Add timeout-minutes: 10 to the MSYS2 and Cygwin "Set up ... GNU Make"
steps, so a hung installer download can no longer take out the whole job
(SF-2).
- advisory-reports: raise timeout-minutes from 45 to 60 for more headroom
above the sum of its step timeouts (N-2).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): real GH-expression evaluator shared with F7c, behavioral xcompile test, secret/run-command coverage (review SF-3, SF-5, SF-6, N-1, N-4, N-6)
- New scripts/ci_blacksmith_runner_test.go: a real GitHub Actions expression
evaluator (tokenizer + recursive-descent parser for ==, !=, &&, ||, !,
parens, string literals, dotted lookups, with GitHub's short-circuit-
returns-operand && / || semantics) that evaluates the REAL pinned
sameRepoBlacksmith{2,4,8}vcpu expression strings from pr.yml/pr-risk.yml,
replacing the old TestSameRepoBlacksmithExpressionSemantics' tautological
hand-written Go mirror (SF-3). This file is the shared helper F7c's own
TestSameRepoBlacksmithExpressionSemantics should rebase onto - see its
doc comment for the "Public surface" (the three consts, evalGHExpr, and
mustEvalGHRunsOn) (SF-5).
- ci_workflow_test.go: drop the old consts/test now living in the shared
file; extend TestBlacksmithJobsReadNoSecrets to also scan job.If and
step.If for secret references, not just env/with/run (N-4; container/
services credentials and matrix-marker resolution stay deferred to F7b);
pin fast-checks' build-tags step invoking all 3 scripts and the
migration-hygiene BASE_SHA env, and add
TestPRWindowsSmallDbproxyServerIsCGODisabled (N-6); add
TestPRWindowsMakeShellLegsFailTheJob and
TestPRWindowsMakeShellSetupStepsHaveTimeouts pinning the SF-1/SF-2 fixes.
- New scripts/check_release_cross_compile_test.go: behavioral test for
check-release-cross-compile.sh using a fake `go` on PATH - asserts a
failing target propagates as non-zero, every target in the group is still
attempted, every invocation passes CGO_ENABLED=0 and the gms_pure_go tag,
and an unknown group fails loudly with zero invocations (SF-6).
- check-release-cross-compile.sh: document the deliberate sequential-build
(vs. the spec's 2-wide suggestion) choice and its revisit trigger (N-1).
- scripts/BUILD.bazel: add the two new test files to scripts_test's explicit
srcs list (not a glob); verified against `make bazel-sync-check`.
All new/changed tests mutation-tested: each production-code mutation was
caught by the relevant test, then reverted via `cp` from a /tmp backup and
diff-verified byte-identical (never git stash / git checkout --).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(ci): fix stale check names and aggregator description in CI_REQUIRED_CHECK_TOPOLOGY.md (review N-7)
- Replace the stale "Check build-tag policy", "Check version consistency",
"Check for .beads changes" and "Check formatting" entries under "Do not
require these existing check names" with the actual post-fold name,
"Fast checks (build tags, versions, migrations, beads diff, fmt)", and add
the cross-compilation and windows-make-shell fold's actual check names.
- Rewrite the windows-make-shell fold description: it no longer has a
hand-written aggregator step (deleted per SF-1 in the prior commit), and
"the separate cygwin job" is now described correctly as a pre-fold matrix
leg, not a job.
- Update the advisory-reports timeout mention from 45 to 60 minutes (N-2).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(ci): drop local design-doc path from topology doc
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): fork and Dependabot PRs execute remotely on rbe-fork when rbe-west's mint is open (F9)
Fork and Dependabot pull_request runs get no secrets, no OIDC token and no
repository variables, so rbe-west decides for them. bazel.yml's rbe job asks
rbe-fork-mint's /v1/status (rbe-mint.ops.gascity.com:8444) once per run:
- open, tier ro: mode fork-ro (instance oss-fork, the fork pool, nothing
cached for anyone);
- open, tier rw (allowlisted author's own fork, allowlisted trigger): mode
fork-rw (instance oss, the OSS pool);
- closed, refusing, unreachable or anything else: mode cache, exactly as
today (local execution plus the anonymous read-only cache).
In the fork modes every lane runs, remotely, from a GitHub-hosted runner:
setup-bazel makes an EC P-256 key and a CSR, uploads the CSR as an artifact
of this run (the mint's proof), gets a ~2 h certificate
(fork-credential.sh, a byte copy of gascity's tools/rbe/fork-credential.sh
from F6) and writes an rc for grpcs://rbe-fork.ops.gascity.com:8444 with
the mint's instance. A lane whose certificate names another tier than the
rbe job fails ("re-run all jobs"). Fork lanes add --config=sole-run like
mode remote.
BAZEL_COVERS_FORKS ships "false" in pr.yml and pr-risk.yml: forks and
Dependabot keep their legacy tiers, and the Bazel lanes run beside them
(remotely while rbe-fork is open). F13 commits "true"; then a covered fork
run in mode cache is red, like a covered same-repo PR without the secret.
Safe while rbe-fork is dark: with the gate closed the mint's port resets
connections and the status probe falls back to mode cache in seconds; with
the mint up but disabled, canaried or rw off it answers open false or 403,
also mode cache.
Not in this change (later slices): bazel-farm.yml and its inputs stay (F12),
bazel.yml keeps its push trigger and pr.yml/pr-risk.yml gain none (F14).
keep their go build fallback on GitHub-hosted runners.
Tests: the rbe decision against a stubbed mint (ro, rw, closed, rw closed,
canary, busy, garbage, unknown tier, unreachable); the coverage and gate
matrices learn the fork modes, BAZEL_COVERS_FORKS and mint answers that
change between runs; TestBazelCacheModeReachesTheRC drives setup-bazel's
fork steps against a stand-in mint and checks each lane's rc;
fork-credential.sh's client contract and write-bazelrc.sh's fork mode and
exclusivity; the endpoint pin is allowed as exactly its one line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): fork-credential.sh retries the mint's 429 and never sleeps after the last try
A 429 (per-minute rate, live-certificate quota) may pass on its own like a
502, so it is retried with the same 10/20/30 s backoff (one minute, the
rate window); the loop no longer sleeps after the fourth failed attempt.
The cert request and the rbe job's /v1/status probe give up connecting
after 5 s, so a closed gate (dropped SYNs) costs seconds, not minutes.
Byte-identical to gascity's tools/rbe/fork-credential.sh.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: gascity-agent <agent@gascity.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(F4): advisory Linux-cross-built Windows test binaries, macOS cache restore
Side-by-side rollout slice of the PR CI -> 5 min plan (F4):
- Add pr.yml `windows-test-binaries`: a Linux job that cross-compiles
bd.exe and two go-test-c binaries (cgo and nocgo) for windows/amd64
via mingw (CC/CXX=x86_64-w64-mingw32-*, matching release.yml's
goreleaser setup), driven by a committed manifest
(scripts/ci/windows-test-binaries.txt) and build script
(scripts/ci/build-windows-test-binaries.sh). Artifacts are uploaded
and consumed by two new advisory "-prebuilt" twins
(test-windows-liveness-prebuilt, worktree-remove-windows-prebuilt)
that run the identical test selection/flags/env (including
BEADS_TEST_REPO_ROOT) against the prebuilt binaries via
scripts/ci/run-go-test-binary.sh and scripts/test.sh's new
BEADS_TEST_PREBUILT_TEST_BINARY short-circuit.
- The original native Windows jobs are untouched and remain required;
the new jobs are advisory only (not in CI_GATE_REQUIRED) for this
rollout phase. Flipping to the cross-built path later is a one-line
CI_GATE_REQUIRED change.
- Add main.yml `windows-test-binaries-cache`: seeds the Go build cache
for the cross-compile toolchain/packages on a schedule, mirroring
the existing module/build cache warmers; reorder build-artifacts'
warm-cache step to precede the module-cache save for a cleaner
build -> warm -> save grouping.
- Restore macOS's native build-cache save step in main.yml (lost
during an earlier job consolidation).
- Remove the non-required "Test (macos-latest)" full-suite job from
PRs; the native macOS gate jobs still run, so required coverage is
unaffected. This does drop a non-required pre-merge macOS race
signal; see docs/notes for the accepted tradeoff.
- Add scripts/windows_test_binaries_manifest_test.go: policy tests
pinning the manifest's three advisory-phase entries, pinning the
mingw toolchain to release.yml's, checking consumer jobs only
reference manifest-listed binaries, checking upload/download
artifact names and `needs` match, checking the prebuilt jobs select
exactly the same tests (-run literals) as their native counterparts,
and checking none of the new jobs reference secrets (fork safety).
All mutation-tested.
- Add two scripts/ci_workflow_apt_get_timeout_test.go budget entries
for the new "Install mingw cross-compiler" steps.
- Fix two latent Bazel-only scripts_test failures surfaced while
wiring the new test file into scripts/BUILD.bazel: materialize
cmd/bd's sources via the existing //cmd/bd:go_srcs filegroup so
sourceRepoRoot(t)-based package-directory checks work under Bazel's
runfiles root, and strip Bazel's own TEST_TIMEOUT env var before
exercising scripts/test.sh's hardcoded default in
test_script_test.go (name collision with test.sh's own override
knob).
Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint; shellcheck; make ci-pr-policy;
make bazel-sync-check; a from-scratch /tmp clone cross-build using
the exact job commands (cold 130.2s, warm/cache-hit 30.3s, PE32+
x86-64 validity via file/objdump, byte-identical SHA-256 across
cold/warm/traced runs); wine -test.list confirms
TestPrimeBinaryPortfolio/TestIsServerProbablyRunning* present in the
cgo binary and the full TestWorktreeRemove* family (including
Windows-specific variants) present in the nocgo binary.
Scoping note: only 2 of the spec's 4 consumer jobs (liveness,
worktree-remove) get prebuilt twins, and only 3 of 8 spec-listed
manifest binaries are built this slice; preflight, doc-freshness and
credential-fixture consumers are deferred as documented follow-up.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): address F4 review findings (SF-1..SF-6 + nits)
Addresses the F4 review (~/beads-bazel-plan/f4/review.md): no blockers, six
substantive findings plus several nits.
- SF-1: main.yml's windows-test-binaries-cache job now seeds the Windows
cross-compile cache from both runner selections pr.yml's consumer job can
land on (Blacksmith for same-repo PRs/merge_group, ubuntu-latest for
forks/Dependabot) via a 2-leg matrix with identical keys/paths, instead of
a single ubuntu-latest leg Blacksmith runners can never see.
- SF-2: removed the two consumer-less "Warm non-race Go build cache
(test-only packages)" steps from main.yml (build-artifacts, test macOS
leg); they had no real consumer and cost ~270s on main's macOS leg for
nothing.
- SF-3: scripts/ci/run-go-test-binary.sh now injects the same
-test.timeout=10m default (when -timeout is omitted) and unconditional
-test.paniconexit0 flag `go test` itself injects; added
timeout-minutes to test-windows-liveness-prebuilt; added a direct
regression test (TestRunGoTestBinaryDefaultsTimeoutWhenOmitted) for the
omitted-timeout branch, which no path through scripts/test.sh can reach
(test.sh always supplies its own -timeout default first).
- SF-4: scripts/test.sh's BEADS_TEST_PREBUILT_TEST_BINARY branch now runs
run-go-test-binary.sh as a child process (capturing and re-exiting its
status) instead of `exec`-ing it, so the EXIT trap (beads_test_env_cleanup)
still runs and the per-run BEADS_TEST_ENV_ROOT temp dir (and, with
BEADS_TEST_SHARED_SERVER=1, the shared dolt sql-server) is cleaned up
instead of leaked/orphaned.
- SF-5: closed the five surviving mutations (cgo/nocgo swap between the
Windows prebuilt twins, dropped -count=1, an unguarded twin in
ci-gate.needs, a goreleaser compiler change, a PKGDIR change) with
dedicated policy tests, and replaced the three-edits-per-job required/
advisory toggle with a single committed WINDOWS_PREBUILT_REQUIRED flag
(pr.yml top-level env) in the style of the existing BAZEL_RETIRES_LEGACY_*
flags: both Windows pairs are now unconditionally in ci-gate.needs and
always evaluated, and only which pair lands in CI_GATE_REQUIRED vs.
CI_GATE_SKIPPED_OK is decided by the flag, in one place.
- SF-6: dropped the //cmd/bd:go_srcs data dep from scripts_test; the
prebuilt-binary contract test now exercises ./scripts/ci (already a data
dep of this target) instead of ./cmd/bd, since it never builds or runs the
real package.
- Nits: removed dangling ~/beads-bazel-plan references and a nonexistent
test name from committed comments; log the resolved mingw-w64 version
when cross-compiling; removed a no-op shellcheck disable comment;
documented why run-go-test-binary.sh's BEADS_TEST_REPO_ROOT re-export
doesn't race beads_test_env_enter's unset; rewrote the pr.yml tombstone
comment for the removed macOS PR job to state the actual, narrower
tradeoff instead of an overclaiming "main.yml covers this" statement.
Verified: bazel --nohome_rc test //scripts:scripts_test passes; actionlint
clean on pr.yml/main.yml; shellcheck clean on all edited scripts (only
pre-existing, unrelated warnings remain elsewhere); make bazel-sync-check
and make ci-pr-policy both pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(scripts): isolate prebuilt-binary contract test from ambient hermetic env
TestTestScriptPrebuiltTestBinaryContract's "cleans up BEADS_TEST_ENV_ROOT
after the binary exits" subtest (added for F4 review finding SF-4) passed
locally but failed in CI's scripts-go-checks/scripts-test leg.
scripts/ci/scripts-go-test.sh (the CI driver for this package's `go test
./scripts/...` run) calls beads_test_env_enter in its own shell before
invoking go test, so BEADS_TEST_ENV_ACTIVE=1 and an outer
BEADS_TEST_ENV_ROOT are already present in this test binary's own
os.Environ(). runTestScriptPrebuiltBinary built the scripts/test.sh
subprocess's env from os.Environ() (minus TEST_TIMEOUT), so that ambient
state leaked through. Inside the subprocess, scripts/test.sh's own
beads_test_env_enter call (test.sh:17) saw the already-active guard and
no-opped: no new root, no new EXIT trap armed in that process - so the
fake binary observed the outer, still-alive root, and the subtest's
"directory is gone after test.sh exits" assertion failed, even though the
SF-4 fix (run the prebuilt binary as a child instead of exec'ing it) is
correct.
Every real production caller of scripts/test.sh's prebuilt path (pr.yml's
Windows liveness/worktree-remove prebuilt steps) runs it as the first
command of a fresh GitHub Actions `run:` step, never nested under an
already-entered hermetic env, so this nesting is purely an artifact of
this test binary's own ambient environment under scripts-go-test.sh, not
a real CI behavior gap.
Strip BEADS_TEST_ENV_ACTIVE/_ROOT/_DISABLE/_KEEP (alongside the existing
TEST_TIMEOUT filter) from the child env in runTestScriptPrebuiltBinary so
the subtest deterministically exercises scripts/test.sh's real top-level,
non-nested invocation regardless of what ambient hermetic-env state this
test binary itself happens to be running under.
Verified: reproduced the CI failure locally by exporting
BEADS_TEST_ENV_ACTIVE=1/BEADS_TEST_ENV_ROOT before `go test ./scripts/...`;
confirmed the fix resolves it; re-ran bazel test //scripts:scripts_test
and scripts/ci/scripts-go-test.sh directly, both pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(lint,scripts): fold duplicate Lint job; matrix PR Lint and scripts-go-checks (F5)
Lint (golangci-lint-action, only-new-issues) was functionally identical to
pr-lint-wrapper's native leg (H7): same pinned binary, config, flags, and
diff scope. Fold it into the wrapper in both pr.yml and main.yml, and drop
LINT from pr.yml's ci-gate.
pr-lint-wrapper becomes a 3-leg matrix (native/windows/darwin, fail-fast:
false) under the same PR_LINT_WRAPPER gate id, installing golangci-lint
v2.10.1 from a pinned, sha256-verified release binary
(scripts/ci/install-golangci-lint.sh) instead of go install/the Action, with
a per-leg golangci-lint cache (pr.yml restores only; main.yml's matrix is
the sole saver). scripts-go-checks becomes a 3-leg matrix
(scripts-test/vet/allowlisted, fail-fast: false) under the same
SCRIPTS_GO_CHECKS gate id, giving go vet its own build cache instead of
sharing the race build cache. main.yml gains a go-vet-cache job to warm that
cache on push; build-artifacts remains the Go module cache's sole saver.
Splitting each job's previously-sequential passes into parallel matrix legs,
combined with removing the duplicate Lint job outright, takes wall-clock
time off the PR gate's critical path.
Updates scripts/ci_workflow_test.go and scripts/pr_lanes_bazel_coverage_test.go
for the new matrices, cache topology, and the removed LINT id; adds
TestGolangciLintInstallScriptPinned. Updates engdocs/LINTING.md and the live
prose in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md (its disclaimed "historical
snapshot" gate-wiring block is left as-is).
Verified: go test ./scripts/..., bazel --nohome_rc test //scripts:scripts_test
(after adding engdocs/LINTING.md and .githooks/pre-commit as scripts_test
data), actionlint, shellcheck, make ci-pr-policy, make bazel-sync-check.
Confirmed lint-leg parity against current main with the pinned binary (0
issues PR-scoped; the same two pre-existing unparam findings full-tree,
unchanged) and scripts-go-checks leg parity standalone. Mutation-tested the
new/updated policy assertions with six deliberate breaks (dropped matrix
leg, reintroduced LINT, corrupted the pinned sha256, removed a cache-save
step, dropped the vet leg, reintroduced the lint job); each was caught with
an on-target failure.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): review-fix pass on F5's lint/scripts-go-checks matrix (SF-1..SF-5, N-1..N-8)
Addresses all SHOULD-FIX and NIT findings from the review of 7e858e2714
(no BLOCKERs):
- SF-1: lint/vet cache saves now run on !cancelled() instead of bare
cache-miss, so a red lint/vet run (main is currently lint-red on two
pre-existing, unrelated unparam findings) still warms the cache for
PR legs.
- SF-2: ci-measurements.yml now exports golangci-lint's install dir onto
PATH in the same step, since $GITHUB_PATH only takes effect in later
steps.
- SF-3: adds a behavioral test (fake curl/tar) proving the install
script's sha256 verification fails closed.
- SF-4: drops the commit sha from the golangci-lint cache key (saves
only on config/dependency change, not every push) and trims the
windows/darwin legs to cache only the small golangci-lint result
cache, not their large GOCACHE.
- SF-5: removes the vet cache's dead non-race-build-cache fallback
restore key (different path list, can never hit) in both pr.yml and
main.yml.
- N-1..N-8: doc/comment accuracy fixes, restores the "moved code reads
as new" caveat, adds a module-cache restore step and timeout-minutes
to pr-lint-wrapper/go-vet-cache, pins fetch-depth/GOCACHE via new
policy-test assertions, and flags a superseded doc snapshot.
New commit on top of 7e858e2714; existing commit trailers are left
untouched per explicit instruction. Not pushed, no PR opened.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37105481015
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* ci(bazel): raise proxied-server tier from 15 to 30 shards (F2)
bazel.yml's bazel-proxied lane (//cmd/bd:bd_proxied_test, run only with
remote execution) was sharing PR Risk's legacy 15-shard, frozen
bd-init-cost-proxy manifest block, so its longest shard was the long
pole of the Bazel-only lanes. Give it its own 30-shard block in
.github/scripts/proxied-cmd-test-shards.txt, bin-packed
longest-processing-time-first by measured wall-time
(scripts/ci/proxied_test_durations.json, captured from two bazel-proxied
job failures' testlogs) instead of the bd-init count proxy, via
scripts/ci/gen_proxied_shard_manifest.py's new --weights=duration mode.
Coverage is proven equal to the full discovered test set by two
independent methods (sorted-name diff and an actual 30-shard invocation
of proxied-test-shard.sh in list-only mode), both exact matches with no
duplicates.
cmd/bd:bd_proxied_test's shard_count moves to 30; PR Risk's and
main.yml's own test-proxied-cmd jobs keep their 15-shard matrix and
frozen manifest block untouched, per D2's retirement design (the Bazel
lane only needs to run the same tests, not the same split). Updated the
two coupled policy tests (scripts/ci_workflow_test.go's
TestBazelDoltServerTiersMirrorPRRisk, scripts/pr_risk_bazel_coverage_test.go's
TestBazelRetiredLanesCheckListedTestsRan) to allow the Bazel target's
shard_count to differ from its legacy job's own matrix size, and
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md / engdocs/TESTING.md to document
the split.
Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint on bazel.yml; make bazel-sync-check;
make ci-pr-policy; 3 representative shards (1, 15, 30 of 30) run
directly against Bazel-built race bd_test/bd_for_tests binaries and the
pinned dolt 2.2.0 release, all green, 55-81s wall time each (the
duration data's absolute seconds are from a contention-era sample and
run far faster here; the LPT balance itself, not the absolute SLA, is
the property that matters).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): close fork-PR gap in proxied 30-shard manifest validation (review B1/S1)
TestShardScriptsListOnlyRealTests only ran proxied-test-shard.sh at PR
Risk's legacy 15-shard matrix, never at the Bazel-only bazel-proxied
lane's 30-shard manifest block. bazel-proxied only runs in mode=remote
(skipped on fork PRs), so a fork PR could rename, duplicate, or corrupt
a TestProxiedServer*/TestServerMode* manifest entry in the 30-block and
merge green via go test ./scripts/... (which runs on every PR), only
breaking the required lane afterward on main and every same-repo PR.
Fix: the test now computes every shard total it must validate per
script (this job's own PR Risk matrix size, every distinct total found
in that script's own committed manifest, and - for proxied-test-shard.sh
- cmd/bd:bd_proxied_test's live shard_count from BUILD.bazel) and runs
the full per-shard validation loop at each one, relying on the script's
own duplicate/stale-name guards. It also sums each total's "manifest: N"
counts across all shards and fails if they sum to zero, catching a
silently-deleted block that would otherwise fall back to 100% hashing
undetected.
Also fold in S1: replace the independently hard-coded `30` overrides in
ci_workflow_test.go and pr_risk_bazel_coverage_test.go with
bazelProxiedShardCount(t), a new shared helper that reads
cmd/bd:bd_proxied_test's shard_count live from cmd/bd/BUILD.bazel. This
turns the check_shard_coverage.py argument comparison in
pr_risk_bazel_coverage_test.go into a genuine cross-file pin (BUILD.bazel
vs bazel.yml) instead of two literals that could drift together
unnoticed. ci_workflow_test.go's tiers loop no longer asserts
BUILD.bazel's shard_count against a value read from the same file
(tautological); it instead asserts pr-risk.yml's and main.yml's legacy
test-proxied-cmd matrices equal each other, restoring the pin S1 found
removed on this branch relative to base.
Verified against the reviewer's mutations (duplicate entry, bogus
entry, whole-block deletion, stale rename, matrix divergence, BUILD
shard_count drift, bazel.yml arg drift): each now fails go test
./scripts/..., where it previously passed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): add generator --write/--check and freeze legacy manifest block (review S3/N1/N4/N5)
gen_proxied_shard_manifest.py:
- Add --write (rewrite only the target total_shards block in place,
leaving every other block's header/body text untouched) and --check
(regenerate in memory and diff against the committed block, exiting
1 with the exact --write command needed if stale). Both operate via
new split_blocks()/write_block()/check_block() helpers that round-trip
the real manifest file losslessly.
- N1: fix the duration-weights header's false claim about the
fallback ratio.
- N5: warn on stderr when proxied_test_durations.json has duration
captures for tests that no longer exist in cmd/bd/*_test.go (renamed,
split, or deleted since the last capture).
proxied-cmd-test-shards.txt:
- N4: mark the legacy 15-shard block's header as frozen for
pr-risk.yml's/main.yml's legacy fork/push jobs: it is deliberately
not kept in sync with newly-added tests (those hash-fallback
instead), so regenerating it is not routine maintenance. Only the
header text changed; all 149 explicit assignment lines and the
"known blind spots" callouts are untouched (verified: a fresh
--write would pack all 164 currently-discovered tests explicitly,
silently changing which shard runs which test for this block, which
is exactly what "frozen" means to avoid).
- Regenerate the Bazel-only 30-shard block's header via
`gen_proxied_shard_manifest.py 30 --weights=duration --write` to
describe --write/--check instead of the old manual-copy-paste
instructions. proxied_test_durations.json has not changed yet (S2
is still in progress), so this run only touched header text; the 164
assignment lines are unchanged.
pr_risk_bazel_coverage_test.go:
- Add TestProxiedShardManifestGeneratorNotStale: runs
gen_proxied_shard_manifest.py 30 --weights=duration --check under
plain `go test` (skipped under `bazel test`, whose runfiles hold
neither the generator's sources nor cmd/bd) so a stale 30-block
(e.g. committed by hand, or left behind after adding tests without
regenerating) fails scripts-go-checks on every PR, including forks.
tools/bazel/equivalence_allowlist.txt: allowlist the new test's guarded
Bazel skip.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): rename 14 undiscovered proxied-gated tests; N2/N3/N6 doc fixes
cmd/bd/*_test.go: 14 tests call requireProxiedServerEnv/
requireSharedProxiedServer but their names did not match the shard
scripts' discovery regex ^Test(ProxiedServer|ServerMode), so no CI lane
ever set BEADS_TEST_PROXIED_SERVER=1 for them and they silently skipped
everywhere (pre-existing gap, not caused by F2; list was in
~/beads-bazel-plan/f2/review/undiscovered_gated.txt).
Fixed by renaming rather than widening discovery: 162 of the 164
already-discovered tests call one of the two gate functions in the first
few lines of their body, so name and gate already correlate almost
perfectly, but the 14 tests' names are each styled differently with no
single regex term that would cover all of them without a hand-maintained
list. Renaming lets the existing discovery mechanism, hash-fallback
behavior, and both shard-coverage policy tests pick them up with zero
other changes. Checked each of the 14 names for cross-references first;
each appeared only in its own file plus at most one doc-comment in a
sibling file, both updated alongside the rename. No collisions with
existing test names.
Discovery went from 164 to 178 tests. Regenerated the Bazel-only 30-shard
block (gen_proxied_shard_manifest.py 30 --weights=duration --write) to
pack the 14 new tests; this is the first --write this round to actually
change assignments, not just header text. The frozen legacy 15-shard
block (N4) was deliberately left alone: the 14 renamed tests now
hash-fallback into it, exactly like every other test added since it was
last generated.
cmd/bd/BUILD.bazel (N6): restore the bd_proxied_test comment sentence
bazel.yml's mirror comment kept but BUILD.bazel's had dropped: PR Risk
builds its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd
twin uses build-artifacts' non-race bd-cmd-test, so this lane is the
stricter of the two.
.github/workflows/bazel.yml, .bazelrc (N2/N3): remove stale timing
language ("30 shards of ~5-10 minutes" / ".bazelrc's leftover "15 shards
of ~10 minutes") never reconciled with this task's own local
representative-shard measurements (55-81s/shard); replaced with the
concurrency argument (30 simultaneous dolt sql-servers don't fit a
GitHub-hosted runner regardless of any single shard's wall time). Also
removes the "within 5s of each other" balance claim (S2): the underlying
duration weights are contaminated by -test.parallel overlap in the
top-level <testsuite time> figures they were derived from; a corrected
re-measurement is in progress (separate commit once it lands).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied duration weights without overlap double-counting (review S2)
scripts/ci/proxied_test_durations.json's 178 entries are now each a
clean, single-job (no concurrent heavy bazel/dolt job on the box),
-test.parallel=1 measurement: a fresh manifest-aligned 30-shard sweep's
own <testsuite time> per top-level test where errors="0" (including
failing-but-complete suites, whose elapsed time is still trustworthy),
plus, for the handful too slow to finish inside Bazel's default
per-target timeout, a `bazel run` leaf-subtest-only sum that never adds
a parallel group's own PASS/FAIL line on top of its already-overlapping
children's lines. An earlier pass at this data summed every PASS/FAIL
line indiscriminately, double-counting parallel subtest groups.
A first attempt at re-measuring the slow stragglers overlapped in
wall-clock time with a second, unrelated heavy sweep and came out 2-3x
inflated versus the same tests measured cleanly; that contaminated data
was discarded in favor of isolated re-runs. TestProxiedServerDep2
initially looked hung during its isolated re-run (15+ minutes with no
PASS/FAIL output and no CPU growth in the top-level test binary) but
was not: each of its subtests does real `bd init --proxied-server`
work in a short-lived child process, and Go buffers a parallel
subtest group's output until the group's last member finishes. Letting
it run to completion (2357.72s) rather than guessing avoided reporting
bad data.
seconds_per_init_fallback is recomputed as the weighted average
seconds-per-init across all 178 measured tests (sum of durations / sum
of inits), for use only when a brand-new test has no entry yet.
Re-packing the 30-shard manifest with this corrected data changes the
shard-balance picture: TestProxiedServerDep2 and TestProxiedServerClose
are each large enough, alone, to dominate their own shard (~2358s and
~1105s locally) while every other shard clusters around 741-876s. The
"within 5s" claim this replaces was never earned by the underlying
data; .github/workflows/bazel.yml now documents the real, uneven
spread and flags that the two outlier shards' local numbers already
exceed the job's 25m step timeout on their own merits (unverified
against RBE's actual per-shard throughput, since this job has only
ever run remote).
Verified: go vet ./scripts/..., go test ./scripts/... (includes the
manifest/coverage policy tests), actionlint, make bazel-sync-check,
make ci-pr-policy all pass. Exact-once coverage re-confirmed: 178
manifest assignments for 178 discovered tests, zero duplicates, zero
gaps.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): apply re-review candidate fixes; revert 2 unready renames (R-B1)
Re-review found the prior round's 14 renames would turn the required
Bazel + legacy 15-shard lanes red once discovered:
- TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix/external-unix and
TestProxiedServerOutageReconnectAcceptanceMatrix/external-unix-socket
built a dolt.sock path under t.TempDir(), which nests Bazel's hermetic
tmp-root plus the (64-byte-truncated) test name deep enough to exceed
sockaddr_un's 108-byte sun_path limit. Both now use a new shortSocketPath
helper (a short os.MkdirTemp("", "bds") directory) instead.
- runProxiedDeadline raced a deadline-watcher goroutine against
exec.Cmd.Start()/Run()'s write to cmd.Process, caught by -race. Replaced
with cmd.Cancel, which exec.CommandContext only invokes after Start, so
the read is ordered after Start's write.
With those two fixes applied, TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix
passes outright. TestProxiedServerOutageReconnectAcceptanceMatrix's race is
fixed, but both its subtests still fail a real "exceeded 3s bound" assertion
against a stopped external upstream -- a genuine latency issue, not a test
bug -- so it is reverted back to its original name
(TestProxiedOutageReconnectAcceptanceMatrix) with a comment explaining why,
pending a bead against the underlying latency.
TestServerModeSharedServeGlobalReportsTheServedDatabase also reverted back
to TestSharedServerModeServeGlobalReportsTheServedDatabase: `bd serve
--global` refuses to auto-apply 4 pending schema migrations (v65->v69) to a
shared/proxied server database and exits 1 with a consent-required usage
message instead of starting, per the #5920 migration-consent guard. Also a
genuine product/test-fixture question, not fixed here; reverted with a
comment, pending a bead.
The other 12 of the original 14 renames are kept; all 12 verified passing
in both lanes (see commits that follow for the manifest update and full
verification).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied weights from parallel=4 CI-identical sweep (R-S1)
The prior round's duration data (committed in 6be9605d9f) was captured at
-test.parallel=1 (serial). That fixed a real double-counting bug, but
serial timings don't model this lane, which always runs -test.parallel=4
(--config=doltserver-proxied's own setting). Re-review measured near-zero
correlation (r=0.08) between the serial-derived predicted shard load and
actual parallel=4 wall time -- e.g. TestProxiedServerRecomputeBlocked was
weighted 812s serially but measured 33s actual at parallel=4.
Replaced with a full fresh capture: `bazel --nohome_rc test
--config=doltserver-proxied //cmd/bd:bd_proxied_test` (local execution
only, the exact config bazel.yml's proxied-server tier uses), 30/30 shards
passing, 0 failures, 0 races. Every one of the 176 discovered tests'
top-level <testsuite time> was captured directly from this run's
bazel-testlogs test.xml files -- no leaf-summation workaround was needed
this time, since none of the 176 hit Bazel's per-target default timeout
running alongside the rest of its shard.
Re-ran `gen_proxied_shard_manifest.py 30 --weights=duration --repack
--write` with these corrected weights: a deliberate full reweight, not an
incremental add, since every weight changed. The new packing clusters all
30 shards within a few seconds of ~355-360s each (176 tests summing to
10712s / 30), a dramatic improvement over the serial data's wrongly
implied two-outlier-shard spread (TestProxiedServerDep2 alone at ~2358s,
TestProxiedServerClose alone at ~1105s -- both artifacts of serial
timing, not real per-shard cost). Updated bazel.yml's paragraph to report
this honestly instead of the prior, false "exceeds 25-minute timeout"
claim, which was itself downstream of the serial/parallel mismatch.
Verified: the 12 tests kept renamed in the previous commit, plus
TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix's socket-path fix,
all pass under the new 30-shard Bazel manifest (this capture run) and
under the legacy 15-shard hash-fallback script (ran the 7 shards their
hash lands in: 2, 3, 4, 5, 8, 10, 11 -- all pass).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): incremental --write, --repack, name-coverage --check (R-S2)
The prior --check compared the committed 30-shard block byte-for-byte
against a fresh from-scratch LPT pack, and --write always did a full
repack. That meant any weight drift (even from proxied_test_durations.json
alone, with no test added/renamed/removed) failed --check, and two
unrelated PRs each adding one proxied test would both force a full
repack and conflict on unrelated shard lines -- re-review demonstrated
one new test moving 73 assignment lines under the old full-repack
behavior.
Redesigned:
- --check now verifies exact-once NAME coverage of the currently
discovered TestProxiedServer*/TestServerMode* set against the
committed block -- missing, stale, or duplicated names fail with an
actionable message -- but does NOT compare shard assignments, so a
weights-only change no longer forces --check to fail.
- --write is now incremental by default: every already-assigned test
keeps its current shard, stale (no longer discovered) names are
dropped, and only newly-discovered names are LPT-placed onto whichever
shard is currently lightest. A full from-scratch LPT pack now requires
the explicit --repack flag (used in the preceding commit for the
deliberate full reweight; ordinary new-test additions should use the
incremental default).
Mutation-tested directly (not just reviewed): verified --check correctly
flags a stale name as "stale/unknown" and a newly added test as "missing"
with the exact --write command to fix it; verified incremental --write
touches only the lines for the added/removed names; verified --repack
--write produces a materially different (full rebalance) diff from the
incremental default.
Updated TestProxiedShardManifestGeneratorNotStale's comment in
pr_risk_bazel_coverage_test.go to describe the new name-coverage-only
--check semantics (it previously described the old byte-identical-packing
behavior; the test's own assertion -- exit code only -- was already
correct and unaffected).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>