Commit graph

392 commits

Author SHA1 Message Date
Eddie the Engineer
fe0a03d020 Merge latest base into adopted PR head (adopt-pr base refresh, merge mode) 2026-10-08 10:42:38 +00:00
Bee
2cae0c1ad8
fix(storage): a parent-child edge propagates only the parent's exogenous blockedness (#6506) (#6602)
* fix(storage): parent-child cascade propagates only exogenous blockedness (wy-3eb07b, #6506)

A parent-child edge used to cascade the parent's is_blocked bit to its
children WHATEVER set it. So the "close gate on the epic" idiom — P carries
blocks edges onto its own children C1 and C2 so P cannot close before them —
darkened the very children P was waiting for. Neither child could reach
bd ready, so neither could be worked, so neither could close: a permanent
lock, and the one direction cmd/bd/dep.go does not refuse.

The contract (#6506): a parent-child edge propagates only EXOGENOUS
blockedness. P counts as blocked FOR CHILD C iff P has a blocking reason —
an open blocks/conditional-blocks target, or a live waits-for gate — whose
target is NOT one of P's own parent-child children, or P's own parent is
(recursively) exogenously blocked. P's OWN is_blocked is unchanged: it really
does depend on open children. Only the propagation narrows.

DESIGN: a second-pass predicate on the parent row, not WITH RECURSIVE.
Both parent-child legs of shouldBeBlockedIDsUnionScopedSQL keep their
p.is_blocked = 1 conjunct and gain parentCascadesBlockedSQL, which says "p's
blockedness is not solely explained by p's own children":

    EXISTS (blocking reason of p whose target is not p's child)
      OR NOT EXISTS (blocking reason of p whose target IS p's child)

The second disjunct carries the contract's recursive clause through the
EXISTING fixpoint instead of through SQL recursion: a parent that is blocked
with no child-targeted reason of its own is dark either exogenously or
because ITS parent is, and either way its children inherit — one level per
pass, exactly as the cascade always converged.

A recursive CTE was the other candidate and is rejected on cost, not on
engine support. Its seed set is the direct-exogenous set, which includes the
waits-for gate, and the seed cannot be confined by the batch scope (%[3]s)
because a batch's ancestors are by definition outside the batch. That would
put an UNSCOPED whole-table gate evaluation into every batched mark/unmark —
the statement shape #6288 removed, now on every write instead of only on the
full repair. The predicate here is correlated to a parent that is already
known blocked, so it probes idx_dependencies_issue for a handful of rows.
The scope splice, the DISTINCT waits-for barrier and the doctor-count
lockstep are all untouched.

One gap this form does not express: a parent that is BOTH blocked by its own
open child AND darkened by an exogenously blocked ancestor is not
distinguishable from one blocked only by its children, so its children stay
bright. That is an under-propagation in a corner the contract would darken;
it needs a second stored bit (or the CTE above) to see, and it cannot
recreate the lock this commit removes.

Also: AffectedByDepChange{,ForWisp}InTx now seeds the parent's other children
on a parent-child edge change (appendSiblingsUnderParentInTx). Writing that
edge reclassifies the parent's blocking reason on the new child from
exogenous to child-derived, which flips the answer for the SIBLINGS — the
close-gate epic is built one child at a time, and the second edge is what
un-darkens the first. Without it the incremental write path and the full
repair disagree, which the new tests catch as a nonzero doctor count.

Tests (internal/storage/dolt/blocked_consistency_test.go), all four seeded
through the real write path and each verified RED against the pre-fix SQL
except the control:
  - ParentBlockedOnlyByItsOwnChildren: the repro, plus a GetReadyWork
    assertion that the children are workable again.
  - ExogenousParentStillCascades: the control — the fix narrows the cascade,
    it does not remove it. Green before and after.
  - ThreeLevelChainBothWays: GP exogenous darkens P and C; GP blocked only by
    its child P darkens neither. Repaired from an inverted plane too, since
    the chain is the case that needs more than one pass.
  - WaitsForGateOverOwnChildren: a gate whose spawner is the parent's own
    child does not darken; one on an outside spawner does.

TestRecomputeAllIsBlocked_CascadesThroughParentChild is unchanged and green,
including its single-pass lower-bound count — which is why the legs keep the
p.is_blocked = 1 conjunct rather than deriving the cascade from the graph
alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(dep): warn when a parent blocks on its own child (wy-3eb07b, #6506)

`bd dep add P C` where C is already P's parent-child child is the "close gate
on the epic" idiom: P carries a blocks edge onto its own child so P cannot
close before it. dep.go refuses the OTHER direction (a child blocking on its
dotted-ID parent, "would create a deadlock") and says nothing about this one,
which is the direction that used to deadlock for real — the cascade darkened
the child, so it could never reach bd ready, be worked, or close.

Since the cascade fix the edge is harmless, so this is an advisory, not a
refusal: it names what the edge now means (the parent stays blocked; the
child and its siblings are NOT hidden from bd ready) so an operator does not
have to rediscover the contract from a query.

It fires only for blocks/conditional-blocks, only after the edge has actually
landed and committed, and stays silent on any read error — an advisory must
not turn a successful write into an error, and must not precede a refusal for
an edge that was never written (the hierarchy and cycle guards reject this
pair whenever both edges already exist, in either order, which is why the
shape usually arrives one edge at a time or through import).

Unlike the D1 implicit-blocks warning it is NOT gated on an interactive
stderr: that one fires on the documented-default majority path, where
training callers to ignore stderr is the risk; this one fires on a rare
deliberate shape, and an agent wiring a close gate is exactly the caller who
should read what it does.

Wired into both direct paths (`bd dep add` and `bd dep --blocks`); the
proxied-server paths return earlier and are untouched.

Tests beside the existing dep-add warning tests: the predicate table
(including the two controls — right id with a blocks edge is not a child, and
the parent's other children do not match), the message content, and the type
gate proving it returns before touching the store.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(doctor): report parent-blocks-on-own-child edges (wy-3eb07b, #6506)

A new "Parent Close Gates" check lists every blocking edge from a parent onto
one of its own parent-child children — the close-gate idiom, where P blocks
on C1 and C2 so P cannot close before them.

INFORMATIONAL, with no --fix and no effect on the overall result. Before the
cascade fix these edges were a trap: the parent's blocked bit propagated to
its children whatever set it, so the gate hid the very children it was
waiting for and nothing under it could become ready. The cascade now
propagates only exogenous blockedness, so the edges are legal and usually
deliberate. What an operator wants from doctor is therefore the inventory —
which beads in this store use the idiom, and where a pre-6506 store's
mysteriously invisible children came from — not a repair, and certainly not a
warning that would train people to remove a working gate.

It is the mirror of the existing "Child-Parent Dependencies" check, which
finds a CHILD blocking on its PARENT (still a deadlock, still fixable). The
two differ in how they match: that one reads dotted-ID ancestry, this one
matches real parent-child edges, because the close gate is wired between
unrelated ids.

Tests beside the child-parent ones: the empty inventory (with an ordinary
hierarchy present, which must not be mistaken for a gate), and the reported
case with two controls in the same fixture — an exogenous blocker on the
parent and an ordinary blocker on the child, neither of which may appear.

Gate note: these live in package cmd/bd/doctor, which the brief's cmd/bd
selector does not reach; `go test ./cmd/bd/doctor/` was run in full.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* revert(dep): drop the unreachable parent-blocks-own-child warning (wy-3eb07b, #6506)

This reverts 34b7f5b42. The warning it added could never print.

Its precondition is "a blocks edge from P onto C succeeded AND C is already
P's parent-child child", and the store refuses exactly that edge.
CheckBlockingHierarchyInTx (internal/storage/issueops/dependencies.go) rejects
a blocker that is the source's own descendant, and it sits OUTSIDE the
SkipCycleCheck gate, so no caller opts out of it; batch_apply.go:533 applies it
again on the DependencyEditor path `bd dep add` actually uses. Written in the
other order, the parent-child edge that closes the pair is refused as a cycle.
So the reverted code paid a GetDependentsWithMetadata read on EVERY
blocks/conditional-blocks `bd dep add` to emit something no operator could
ever see, and its 102-line test pinned a predicate instead of a behaviour.

What an operator DOES hit is the refusal — and the cascade fix left its stated
reason false. It read "blocked status cascades to descendants, so <blocker>
would inherit the block and never close". Since #6506 a parent-child edge
propagates only the parent's exogenous blockedness, so the descendant does not
inherit and nothing deadlocks. The refusal still stands, because a blocks edge
is not how the hierarchy says "I close after my children" — so the reason now
names the form that is: a waits-for gate over the children, `bd gate`.

The same false sentence was the OpenAPI description of
Problem.blocker_is_ancestor, so it is corrected there too, with types.gen.go
regenerated by `go generate ./internal/httpapi/apigen/` to keep `make
api-check` green.

The doctor "Parent Close Gates" check from 4c5f2f77b is KEPT. A warning on a
write path that refuses the write is noise; an inventory of the edges a store
already carries — from import, from merge, from the pre-refusal past — is what
an operator can act on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(storage): the parent-child exogeneity test is over the whole SUBTREE (wy-3eb07b, #6506)

B1 of the adversarial review: 648ab0af6 tested "is this blocking reason's
target one of my own children" as a DIRECT parent-child child, but #6506's
contract is about blockedness originating outside the parent's own SUBTREE.
At depth 1 the fix worked; at depth 2 it left the permanent lock alive.

  P --blocks--> G,  pc C -> P,  pc C2 -> P,  pc G -> C

G is not P's child, so P read exogenous, so P darkened C, so C darkened G —
and G is the only row whose close can free P. Exactly the shape-1 mutual lock
the issue says no contract intends, one level down. Pinned as
TestParentChildCascade_ParentBlocksItsOwnGrandchild, verified RED at
4c5f2f77b: p/c/c2/g all blocked, all three children missing from ready work,
and the full repair changing nothing.

The test is now a real ancestor walk (subtreeWalkDepth levels of indexed
LEFT JOINs up from the reason's target), so a reason anywhere under the
blocker counts as its own.

Also B2 — the cost regression, which is the same edit, because the shape that
makes the test correct and the shape that makes it affordable are one
decision. The union's parent-child legs no longer carry a per-row correlated
EXISTS. They subtract an uncorrelated set instead:

    AND p.id NOT IN (parents explained by their own subtree)

and the UNBATCHED statements (full repair, doctor count) do not even spell
that set as a subquery: it is read ONCE per fixpoint pass and bound as ids.
A repair pass is four statements and each would otherwise derive the same set
twice; reading it once is the whole difference. The batched mark/unmark keep
the spliced subquery, where it is confined to the batch's own parents by the
existing scope splice and costs a few rows.

MEASURED, gastownhall/beads#6506 bar: full repair and doctor count <= 1.5x
origin/main, write path <= 1.2x. Committed benchmark
(internal/storage/dolt/blocked_scale_bench_test.go, -short-skipped), 1000
issues / 2870 deps / 333 blocked rows, embedded dolt 2.2.0, run INTERLEAVED
main/head to cancel machine drift, four pairs, medians:

                             origin/main 71c4cd08b   this branch   ratio
  full repair                 48 71 43 46 -> 47 ms   66 56 54 55 -> 55 ms  1.18x
  doctor count                55 62 44 53 -> 54 ms   67 61 57 57 -> 57 ms  1.06x
  write path, per CloseIssue  37 30 21 22 -> 26 ms   30 23 16 24 -> 23 ms  0.91x

All three inside the bar; the worst single PAIR is 1.38x on the repair and
1.30x on the count. The two sides disagree on blocked rows by construction
(604 on main, 333 here) — that IS the fix.

NOT inside a bar, and disclosed rather than hidden: the benchmark's fourth
timing, a 200-id RecomputeIsBlockedForIDsInTx (the bulk/import batched path,
which no metric in the review covered), is 53-63 ms on main against 400-556 ms
here — 6.9x at the medians. The cause is structural and measured: a batched
pass runs many small statements, each deriving the set twice, and the set
costs a few ms however it is written. The per-pass read that fixes the
unbatched statements makes the SMALL-batch case worse, not better (one
whole-store read costs more than the subqueries a two-id batch pays), so the
fix is a per-batch-scoped read threaded through expandBatchTemplate's
placeholder plumbing — a separate change, and the reason this timing is
committed rather than left in a scratch file.

Three shapes were built and measured before this one, all at the same scale:

  - the review's own: per-row correlated EXISTS with two more inside,
    3.3-3.7x full repair, 5.2x doctor count;
  - the set as a DISTINCT derived table LEFT JOINed on p.id, which is what B2
    recommended: ~30x. dolt plans a join operand as the inner side of a nested
    loop and rebuilds it per outer row — the correlated shape in a derived
    table's clothes. This is the one finding that contradicts the review, and
    it is why the set is subtracted with NOT IN instead;
  - WITH RECURSIVE for the ancestor walk: blew the driver read timeout on the
    full repair. A recursive CTE pays a query's overhead per ITERATION; the
    seeded walk cost 45 ms per evaluation against 6 ms for four levels of
    joins, which is why the walk has a horizon (subtreeWalkDepth = 4, epic ->
    sub-epic -> leg -> task). Past it a reason reads as exogenous, which
    propagates MORE and hides nothing the old code showed.

Seeding: AffectedByDepChange{,ForWisp}InTx now walks UP from a parent-child
edge's target, not just to it. Hanging G under C changes nothing about C and
everything about any ANCESTOR of C that blocks on G, so each such ancestor's
sibling set is reseeded — otherwise the incremental write path and the full
repair disagree, which the new test catches as a nonzero doctor count. The
walk is filtered to ancestors that carry a blocking reason at all: without
that filter a 70-deep chain reseeds every ancestor's subtree on every edge
written into it, which timed out TestGetReadyWork_ParentFilterReturnsDeepDescendants.

CHANGELOG now names the limitation the review ruled acceptable but undocumented
(a sub-epic blocked both by its own children and by an exogenously blocked
ancestor keeps its children visible), marked tracked upstream; bee files the
follow-up issue at PR time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(doctor): rune-safe Detail truncation, Category on the N/A returns (wy-3eb07b, #6506)

The "Parent Close Gates" check joins its inventory with U+2192, three bytes
wide, and then cut the joined string at byte 200. Two of every three offsets
land mid-rune, so `bd doctor --json` could carry invalid UTF-8 in Detail — a
display nuisance in the terminal, but a value a JSON consumer has to decode.
truncateDetail walks back to a rune boundary first; the test drives every
entry count from 1 to 120 so the cut lands at all three offsets within the
arrow and asserts utf8.ValidString on each.

The same byte cut appears twice more in this file (:101, :561) on ASCII-only
details. Left alone here: they are not this bead's, and changing them would
put a behaviour change in two checks a reviewer of #6506 has no reason to be
reading.

The check's two "N/A" early returns also omitted Category, unlike its three
real returns, which drops them out of the categorised report. Both now carry
CategoryMetadata.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf(storage): read the batched exogeneity set once per batch, not per statement (wy-3eb07b, #6506)

D1, the PR blocker from the round-1 review. The batched mark/unmark templates
spliced parentsExplainedBySubtreeSQL — the "which blocked parents are explained
only by their own subtree" query — into both of their parent-child legs,
batch-scoped. The scope made it look cheap. It was not: dolt pays for a
`p.id NOT IN (subquery)` once per leg per STATEMENT, and a batched pass is a
mark and an unmark, so one 200-id RecomputeIsBlockedForIDsInTx derived the set
four times and measured 6.9x origin/main (53-63 ms -> 400-556 ms). That is the
bulk/import path and the cost shape #6288 removed.

The set is now READ once per batch and BOUND into both statements as ids —
the same trade the unbatched full repair already makes per pass — which is why
markBlockedTemplate/unmarkBlockedTemplate are built per batch now: the number
of bound ids is a property of the batch. In front of the read sits a cheap
indexed guard, batchParentKindsInTx: a parent KIND with no parent-child row in
the batch has an empty candidate set, hence an empty exogeneity set, so its
read is pure overhead, and the read is a ten-join walk under two CTEs that
dolt spends ~8 ms planning whether it returns rows or not. Most batches name
no wisp parent at all; a batch of roots names neither. Without the guard the
2-id case measured 1.9x, with it 0.94x.

expandBatchTemplate now WALKS the template instead of counting "%s": the batch
scope and the bound exogeneity ids alternate inside the union (leg 3's scope,
leg 3's NOT IN, leg 4's scope, leg 4's NOT IN), so only text order says where
each group goes. It returns an error on a ?/bound mismatch rather than handing
the engine an opaque argument-count failure. The %s budget is back to 6 from
8, re-pinned, with two new pins beside it: the ? group is exactly the bound
exogeneity ids (none for an empty set), and the mismatch is refused.

KEPT, on round-1's measurements: the NOT IN subtraction (the derived-table
JOIN measured ~30x in dolt) and the 4-level LEFT JOIN ancestor walk (WITH
RECURSIVE blew the driver read timeout).

MEASURED. n=1000 issues / 2870 deps / 333-604 blocked, embedded dolt 2.2.0,
TestBlockedStateScaleTiming, SIX interleaved origin/main(71c4cd08b)/head pairs,
medians, worst single pair in brackets. The bench now times three batch sizes
instead of one, because the per-statement and per-id costs pull opposite ways:

  metric                     main     head    ratio          bar
  full repair               49 ms    57 ms    1.16x [1.32x]  <= 1.5x  MET
  doctor count (consistent) 48 ms    57 ms    1.18x [1.32x]  <= 1.5x  MET
  write path, per CloseIssue 25 ms   26 ms    1.03x [2.59x]  <= 1.2x  MET
  batched recompute, 200 id 54 ms    69 ms    1.28x [1.57x]  <= 1.5x  MET
  batched recompute,  20 id 17 ms    22 ms    1.29x [1.50x]  (measured)
  batched recompute,   2 id 22 ms    21 ms    0.94x [1.03x]  (measured)

All four stated bars are met on medians. The worst-pair column is at or inside
this box's own noise: over the same six runs origin/main's OWN spread is 2.07x
on the full repair, 2.33x on the write path, 1.92x on the 2-id batch and 1.19x
on the 200-id batch, so a single pair's ratio is not evidence at this
resolution and the medians are what the bars are read off.

DISCLOSED, not a stated bar and not fixed here: the bench FIXTURE BUILD —
2870 AddDependency calls plus 240 CloseIssue through the store — is 75 s ->
102 s, 1.36x [1.61x]. That is the dependency-ADD write path, and the cost is
round 1's incremental seeding (AffectedByDepChangeInTx now walks up to every
reason-carrying ancestor and reseeds its whole sibling set), which is what
buys the incremental/full-repair agreement pinned by the lockstep test in the
next commit. Per-edge it is sub-millisecond; it shows up here because the
fixture writes the hierarchy of a 1000-issue store edge by edge.

GATE: go build ./... ; go vet ./internal/storage/issueops/ ; go test
./internal/storage/issueops/ (ok, includes the rewritten template and sqlmock
pins) ; BEADS_TEST_EMBEDDED_DOLT=1 go test ./internal/storage/dolt/ -run
'ParentChildCascade|RecomputeAllIsBlocked' (ok) ; gofmt -l clean. The full
enumerated gate is on the bead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(storage): pin both sides of the subtree-walk horizon, and the incremental/full-repair lockstep (wy-3eb07b, #6506)

D2 and D3 from the round-1 supervisor ruling.

D2 — THE HORIZON IS NOW PINNED, not commented. "Inside my own subtree" is
decided by a fixed 4-level ancestor walk (subtreeWalkDepth), because a
recursive walk cost 7.5x per evaluation and blew the driver read timeout. Two
tests that differ by ONE EDGE straddle it:

  _ReasonAtTheWalkHorizon    P blocks a descendant exactly 4 pc levels down.
                             The walk reaches P, so the reason is
                             subtree-derived and the chain comes bright.
                             RED on origin/main (4 rows dark that the fix
                             brightens) — so it is not vacuous.
  _ReasonPastTheWalkHorizon  one level further. The reason reads as EXOGENOUS,
                             so the whole chain stays dark, which is
                             origin/main's own answer — and that test PASSES
                             UNCHANGED ON origin/main (verified in a temp
                             worktree at 71c4cd08b, since removed). It also
                             asserts no row of the shape appears in
                             GetReadyWork: no new dark rows AND no new visible
                             rows past the horizon, checkable rather than
                             argued. Raising the constant makes it fail and
                             name the row that changed.

Both also repair from a fully inverted plane, which is the leg that needs the
fixpoint rather than the write path's seeding. The CHANGELOG now names the
4-level horizon beside the existing limitation sentence, and the constant's own
doc names the two pins.

CAN THE HORIZON BE RAISED? Measured, and the answer is "not on this evidence".
Same bench, same box, full repair / doctor count / 20-id batch / 200-id batch:

  depth 4   57 ms /  58 ms / 21 ms /  69 ms     (shipping)
  depth 6  126 ms / 118 ms / 52 ms / 101 ms     2.2x, TWICE reproduced
  depth 8   59 ms /  74 ms / 24 ms /  68 ms     ~= depth 4, TWICE reproduced

The curve is NON-MONOTONIC: 6 falls off a cliff and 8 comes back. That is a
dolt planner artifact, not a cost model, and raising a semantics-bearing
constant on the strength of a number nobody can explain is the wrong trade —
depth 8 also makes every past-horizon shape newly bright, which is a behavior
change and not this round's business. Left at 4, with the numbers recorded for
whoever picks the knob up.

D3 — THE LOCKSTEP. _IncrementalMatchesFullRepairAcrossEdits walks one graph
through the mutations that reclassify a blocking reason WITHOUT touching the
row that carries it, and after every one asserts both the exact flags and
assertConvergedAndStable: detection counts 0 AND a full repair changes 0 rows.
An under-seeded incremental path (the AffectedByDepChange{,ForWisp}InTx
ancestor walk round 1 added) fails here as a nonzero repair over a graph the
write path just finished maintaining. The mutations: re-parent the reason's
target INTO the blocker's subtree (which is how the shape arrives — the store
refuses a blocks edge straight onto an existing own descendant, and that
refusal's reason text is what 3a5203979 corrected), close the descendant,
reopen it, re-parent it back OUT, remove the blocks edge. The
into/out-of-subtree steps are the ones that flip rows the edge does not name
(the parent's other children), which is exactly what
appendSiblingsUnderAncestorsInTx exists to seed. RED on origin/main for the
semantic values; origin/main's own incremental/full-repair agreement holds
there, so what this pins is the agreement, not the numbers.

GATE: BEADS_TEST_EMBEDDED_DOLT=1 go test ./internal/storage/dolt/ -run
'ParentChildCascade' -count=1 — 8/8 ok; gofmt -l clean; go vet clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(doctor): the close-gate inventory walks the same four parent-child levels as the cascade (wy-3eb07b, #6506)

The "Parent Close Gates" check listed only DIRECT parent->child gates, so the
fix's own headline shape (a parent that blocks on its grandchild) never
appeared in the inventory. The check now joins a descendant-pairs set built
to the same four parent-child levels the cascade uses for "inside my own
subtree" (issueops subtreeWalkDepth), as plain joins over the dependency
union rather than a correlated subquery, which dolt declined to plan once
rows were present. Messages say "own-descendant"; the tests add a grandchild
gate (2 listed, the exogenous edge still not) and assert Category.

CHANGELOG cites the nested-gap follow-up, gastownhall/beads#6601.

Tests run: go build ./... 0; go vet ./cmd/bd/doctor/ 0; gofmt -l clean;
env -u BEADS_ACTOR go test ./cmd/bd/doctor/ -run 'ParentBlocksOwnChild|TruncateDetail' -count=1 PASS (3/3).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* ci: allowlist TestBlockedStateScaleTiming's -short skip (wy-3eb07b, #6506)

scripts/check-testing-short.sh reserves testing.Short() for runtime, stress
and large-fixture skips and keys them by file::func. The scale-timing test
added in this PR builds a 1000-issue / ~2000-edge dolt fixture and times the
blocked-state paths over it: a large-fixture skip, and PR Core runs with
-short, so it needs the entry. One line, the same shape as the
TestUOW_ConcurrentMergeOps_NoLostUpdate row from #4732.

Local: ./scripts/ci/pr-policy.sh rc 0 (check testing.Short boundaries
succeeded; the other nine legs unchanged).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wy-3eb07b: build(bazel): register blocked_scale_bench_test.go in internal/storage/dolt

The rebase carried the bench test but not its BUILD.bazel entry, so the
Bazel test lane's gazelle sync and Go test equivalence steps failed while
every Go lane was green. This is CI's own bazel-sync patch for 043f9c47b9,
applied unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(blocked): resolve explained parents through their ancestry; walk ancestors on hinted per-table members

Maintainer fixup for #6602 (adopt-pr review iteration 1, quality scorecard
760/1000, request_changes; amended at iteration 2, 906/1000, approve). It
fixes the review's two major findings, the doctor check's message and its
swallowed query failure, the stale core-builder doc, and the stale
test-comment names. At iteration 2 it also takes both nits and corrects a
CHANGELOG sentence this fixup had made false.

- Mixed-arm leak (major). The explained-parent read decides from a
  parent's own reason rows, and the fixpoint carries the recursive clause
  only for a parent with no reason row of its own. With Q blocked on an
  outside X, P a child of Q, and P blocking on its own child C, P read as
  explained and C came bright under an epic still gated by X, on the
  write path and in full repair alike. pruneExplainedByAncestryInTx now
  shrinks every read to the greatest set whose members' blocked parents
  are all members. A full-repair read is complete, so a blocked parent it
  did not return is unexplained. A batch-scoped read climbs instead,
  reading the blocked parents above the batch by id (explainedAmongInTx).
  The suggested one-level NOT EXISTS is not enough: put a second close
  gate between them (R blocked on X, pc Q -> R, Q blocks on P, pc P -> Q,
  P blocks on C) and P's blocked parent Q is in the set, so R's gate
  never reaches C.
- ancestorChainInTx (major) walked the derived UNION of both dependency
  tables that #7224 removed from isAncestorInTx, on every parent-child
  write and merge replay. It now walks reachabilityMembers, the hinted
  per-table recursive members split out of reachabilityQuery, with the
  reason filter kept in the outer SELECT. TestGraphWalkPlansHonorJoinHints
  (embedded) gains an ancestor-chain subtest that pins the lookup plan;
  the old walk fails it.
- New dolt tests: TestParentChildCascade_OwnGateUnderExogenousParent
  (write path, full repair, and X closed and reopened),
  _NestedGatesBothWays (the two-gate chain both ways, then a close of the
  children's own blocker, which re-evaluates them through a batch whose
  read covers nothing above their parents), and
  _WispGateUnderExogenousParent (a wisp gate under an issue epic, an
  issue gate under a wisp epic, then an explained wisp gate the walk must
  read by id above such a batch). Five mutants of the prune were each
  killed: no prune (the PR head), a one-level check, an over-prune, a
  scoped read treated as complete, and a scoped read that trusts every
  parent above it. The eight existing cascade tests pass under all five.
  Iteration 2 killed two more, each by its own section of the wisp test
  alone: dropping the lookup of a wisp parent from blockedParentEdgesInTx,
  and dropping explainedAmongInTx's wisp leg, which had survived all
  eleven cascade tests.
- The doctor check no longer says the children "stay in bd ready", and
  a failed or timed-out inventory query is now a warning carrying the
  error, as in validation.go's other checks, instead of an OK that reads
  as "no gates". TestCheckParentBlocksOwnChildDB_QueryFailureWarns pins
  it with a closed handle, and it fails with the old status. A warning
  does not touch doctor's overall result, and it can be suppressed.
- The core builder's doc no longer describes the spliced design, and
  three test comments name builders that exist.
- explainedAmongInTx says why it does not tolerate a missing table, as
  the file's wisp reads do: the prune runs only on a non-empty read,
  which came from the same core query over all four tables.
- The CHANGELOG entry no longer calls #6601's shape (a sub-epic gated on
  its own children under an exogenously blocked epic) a known limitation
  that keeps the children visible. The prune darkens them, as
  TestParentChildCascade_OwnGateUnderExogenousParent pins.

Left as-is, all non-gating: the refusal hint's "(bd gate)", an API string
pinned by the golden corpus; a parent-child add/remove timing in the
scale bench, whose 1000 issues cannot show the 50k shape (the plan pin
guards it); and hints for the explained-parent subtree walk and the
doctor query. The measured complexity major is runDiagnostics (50/66),
the same at base; the PR adds two lines to it.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(storage): re-pin the classic batch-apply statement count for the parent-child cascade

The B0 statement-count pins predate this PR's parent-child cascade. The
first full run of the storage tiers on the adopted head measured the
40 (classic) shape at 835 statements on both backends, against the
pinned 786 +/- 2.

All 49 extra statements are reads, seven for each of the shape's seven
distinct parent-child dep adds (its eighth repeats a pair and is a
no-op):
- appendSiblingsUnderAncestorsInTx reseeds the parent's other children:
  one ancestorChainInTx walk and four child probes, +5.
- The batched mark/unmark reads its exogeneity set first
  (batchExogeneity): the parent-kind probe and the issue-parent read, +2.

Measured by logging every statement of the classic apply on the CI base
233d151b37 (786) and on b2b93515bf (835), then diffing the two logs by
statement shape. Those seven shapes are the whole difference, each
exactly +7, and no write changed. The 356 and 712 shapes are unchanged
on both backends.

This is the deliberate re-measure and re-pin the pins' doc comment asks
for. The per-change breakdown is in pinnedEmbeddedStatementCounts.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
2026-10-08 08:31:34 +00:00
Steve Yegge
70f9e2f8a3
Beads 1.2 remediation: migration-consent gate — bd never migrates an existing database without explicit consent (#5764)
* feat(schema): never migrate an existing database without explicit consent (bd-qhn8y)

Beads 1.2 release remediation. 1.2.0/1.2.1 auto-applied migrations 54-65 at
store open on ANY command, including reads; older binaries then refuse the
migrated database. MigrateUp now refuses pending main-sequence migrations on
an existing database before any write (dolt_ignore seeding included) unless
the operator consented: bd migrate / bd migrate schema (preview flags
withhold), bd migrate --force, or BD_ALLOW_MIGRATE=1
(BD_ALLOW_REMOTE_MIGRATE honored as an alias). Fresh init and at-latest
databases are unaffected; the #4259 remote gate keeps all its protections.

Also: freeze shipped migrations (hash-pin test, main 0001-0065 + ignored
0001-0024), retract v1.2.0/v1.2.1 in go.mod, agent-executable un-migrate
runbook (docs/recovery/unmigrate-schema-v65.md), and the upgrade smoke +
historical migration harnesses now model the consent contract and assert
the refusal carries guidance.

No version bump here by design: cutting the release stays with the release
owner.

Rebased onto main 800d43eb8d: the go.mod retract hunk is dropped -
main already retracts v1.2.0/v1.2.1 (and v1.1.1) since #5782. The
uow consent grant moves into the TestMain main added in #6441 (a
second TestMain no longer compiles), and the new files are listed
in the BUILD.bazel targets main now carries (gazelle).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(schema): cloned-workspace bootstrap carries creation-consent (bd-qhn8y)

The consent gate treated a workspace `bd init --remote` / `bd bootstrap`
just cloned into existence like a pre-existing database, refusing the
smart gate's provably-safe first-mover migration and stranding init in
the generic open-error path (bypassing the bd-4mpy7 finalize+guidance
ergonomics). Cloning IS creating: those paths now record consent
explicitly, and the #4259 remote-migrate gate still applies to them
unchanged (consent is not its designated-migrator override). Fixes CI
TestEmbeddedInit/remote_behind_schema_gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(migration): consent first in main's wisp lanes too (bd-qhn8y)

The rebase onto 800d43eb8d brought in #6049's wisp-plane lanes, which
postdate this PR's harness changes. Each one makes an ordinary candidate
command the first touch on an old-schema database, and the consent gate
refuses it:

- historical-dolt-upgrade-test.sh run_wisp_plane_upgrade opens with
  `bd list` before any migrate: "candidate could not open the seeded
  wisp workspace" for v1.0.1, v1.1.0, v1.1.2 and v1.2.2 (the
  Migration Test Harness dolt-runtime shard).
- run_dirty_tracked_wisp_scenario (v1.1.2): the consent refusal stops
  the open before MigrateUp reaches the #5816 dirty tables, so the lane
  fails "without naming them".
- upgrade-smoke-test.sh "Wisp plane survives upgrade" upgrades with a
  bare cand_init, so list/ready/blocked/update all hit the refusal
  (4 failures each for v1.3.0 and v1.3.1, both in the PR smoke set).

Use the PR's own pattern: the explicit `bd migrate schema` is the first
candidate touch (cand_upgrade in the smoke script). Base 800d43eb8d
passes these lanes unchanged; with this fixup the full 14-version
historical corpus and all five PR smoke versions pass.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schema): close consent-gate review findings (bd-qhn8y)

Review fixes for the migration-consent gate (adopt-pr review, attempts
1 and 2).

Majors:

- #4566 deadlock rebuilt one check earlier: on a database that is both
  behind and dirty, the consent refusal fired before the dirty-table
  guard, whose remedy (`bd dolt commit`) opens without consent. The
  working-set-reconcile opens now tolerate MigrateConsentError: embedded
  (toleratesConsentRefusal, WSR intent only) and server mode
  (LenientOpen's warnLenientOpenRefusal) print WorkingSetWarning and
  commit at the current schema; strict, read-only and RemoteSync opens
  still refuse. Covered by an embedded test (a refused open writes
  nothing: dolt_status, dolt_ignore and schema_migrations unchanged) and
  a server-mode integration subtest.
- Bare `bd migrate` consented: isMigrateConsentCommand now matches only
  the schema verb, so `bd migrate` (metadata reconcile, including
  --update-repo-id, --inspect and --dry-run) no longer consents to a
  one-way migration; `bd migrate schema` and --force do.
  TestLocalMigrateConsentScope pins the scope through the root pre-run.
  The dirty-table warnings name `bd migrate schema` accordingly.
- Runnable JSON hint: the JSON refusal follows the #4259 AgentDirective
  convention, a non-runnable hint plus a migrate_consent object
  (versions, pending, severity, human_decision_required, options) whose
  commands are retargeted with --global for the shared global database.
  The text refusal prints the same --global hint under `bd --global`.
- Freeze floor: the hash-pinned freeze now covers what v1.3.x shipped,
  main 0001-0066 and ignored 0001-0026; the CHANGELOG puts new
  migrations in 0070+ / 0028+.
- Fresh-bootstrap heal: MigrateUpWithLock with heal authority takes the
  consent-free migrate path, so a retried bd init whose first pass left
  a non-zero cursor converges; other lock callers keep the gate.

Minors: the uow team-server guards are repinned on `bd migrate schema`;
the un-migrate runbook is linked from the recovery index and
cross-linked with the 1.2.1 repair-forward doc; the CHANGELOG BREAKING
entry matches the shipped contract; harness comment grammar.

Docs: the user docs and README named bare `bd migrate` as the step that
applies schema migrations; they now say `bd migrate schema`, which also
exists in the pinned docs release (v1.2.2) and in v1.3.x. The
`bd migrate schema` help no longer claims migrations run automatically
on store open. The Go remedy that sends a remote-gate stop to bare
`bd migrate` for its migrate-or-adopt options is unchanged (pinned by
TestNoticeSharedMigrateRefusalPerDecision).

Upgrade smoke (attempt 2): the consent-gate assertion is graded only when
the old binary left a database. Releases before v1.0 have no
`init --non-interactive`, so prev_init creates nothing there and the
candidate's `bd list` fails with "no beads database found", which has no
consent guidance. Main's Bazel smoke lane (#7331) runs the newest 30
releases, 20 of them pre-1.0, under the required ci-gate; all 20 went red
on the merge. The 10 v1.x releases still take the refusal branch.

Shard manifest (pre-approval CI): the new embedded test
TestEmbeddedMigrateConsent_BehindAndDirtyWorkingSetReconcileRecovers is
listed in the Bazel 15-shard block of embedded-storage-test-shards.txt
(incremental --write; shard 15). Main's ci lane runs
TestStorageEmbeddedShardManifestGeneratorNotStale in //scripts:scripts_test,
which fails on any unlisted top-level embeddeddolt test.

Not addressed (optional): env-only consent docs for embedders, the
unparseable BD_ALLOW_REMOTE_MIGRATE alias hint, and the harness
`|| true` on `cand migrate schema`.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <julianknutsen@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
2026-10-08 08:21:31 +00:00
Bee
2e6289f9fb
fix(hooks): key the push-time suite on the remote ref, never on the local ref's shape (#7352)
Since #7348 the pre-push hook ran scripts/pre-push-suite.sh only for stdin
lines whose local ref matched refs/heads/*. For `git push origin HEAD` and
`HEAD:refs/heads/x` git passes the literal local ref HEAD, so the suite never
ran for those pushes: a Go change pushed that way landed with an empty bazel
log, while the same change pushed as `git push origin br1` was refused.

The hook now skips a line when the remote ref is refs/tags/* or the local sha
is the zero sha (a deletion); everything else is a branch push, HEAD included.

TestPrePushHookSuiteKeysOnRemoteRef feeds the real hook HEAD-sourced ref lines
from a scratch repository with a stub suite that records whether it ran: both
HEAD forms run it on a Go change; a docs-only push, a tag pushed by HEAD or by
name, and a deletion do not. On main's hook the two HEAD cases fail.

TestPrePushHookRunsSuiteOnlyForGoChanges indexed "pre-push-suite.sh", which
the hook's header comment on line 3 also contains, so the ordering assertion
held wherever the call block sat. It now indexes the call line; moving the
suite block after the managed beads section fails it.

Fixes #7351

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 07:49:57 +00:00
Bee
569ad6001c
test(scripts): the retired-lanes BUILD walk follows file symlinks, as a local runfiles tree needs (#7361)
TestBazelRetiredLanesCannotBeNarrowed skipped every ModeSymlink entry in
its two BUILD/.bzl walks. Under `bazel test` on a local executor (the
processwrapper-sandbox an unauthorized fork's PR gets for
//scripts:scripts_test, RBE_INSTANCE being empty) the runfiles tree is a
symlink forest, so the walk read no BUILD file and the pin failed with
`got map[]` on every such run (#7350). Both walks now go through
walkBazelFiles, which decides "file" with the package's isFileOrFileLink:
a regular file or a symlink to one; a symlinked directory is never
followed, a dangling link is skipped. TestBazelFileWalkFollowsFileSymlinks
mirrors the checkout's Bazel files as a symlink forest (plus a directory
link and a dangling link) and requires the same file set as the checkout.

Fixes #7350.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 07:47:37 +00:00
Eddie the Engineer
b32798d1bb fix(httpclient): address #7247 relaunch review iteration 1
Review iteration 1 of the relaunched adoption raised six majors. This
commit fixes all six. Five are the majors the first adoption's
iteration 1 raised. Its fixup, 696803f250 (tag
adopt-pr/bd-f5illd/stranded-fixup-696803f250), fixed them but never
reached the branch. That fix is ported here onto the contributor's
current head, with its reasoning unchanged; the sixth is new.

Creates refused what the CLI sends. Every `bd create` carries
CreatedBy, a workspace whose config.yaml names a prefix also carries
IDPrefix, and the client refused both.
- The server stamps created_by from the actor on all three create
  shapes: batch_create.go and batch_apply.go now match create.go.
- The client sends a CreatedBy that names the actor and refuses any
  other value, which the server's stamp would silently replace.
  createIssue and the apply create item share
  refuseUnwirableIssueMembers.
- IDPrefix refuses only where the role would act on it: an explicit,
  unforced id.
- `bd create --file` stays refused over http. Dropping its Status was
  safe but not enough: the batch create still refuses its Provenance
  and Owner. The markdown.go and batchcreator.go comments that called
  it servable now say so.

keysetFilter ignored AfterPriority. It now compares priority first,
mirroring sqlbuild's KeysetPriorityCreatedAtIDPredicate, and a new
served case resumes a priority keyset position. AfterID or AfterPriority
without AfterCreatedAt no longer reads as a zero-instant position that
discarded every row (the AfterID minor).

Claim with a patch was refused on a false ledger claim. `claim`,
`force_assignee_transfer` and `force_close_policy` now ride updateBody
like `force_notes_overwrite`, so a claim with a patch or a version guard
is one updateIssue call. A claim ALONE sent to a server that predates
the member (#6890) falls back to claimIssue. The three ledger rows are
retired, and the served cases parked on them now run.

OpenReadOnly returns a writable store. It stays writable, as a
documented exception: backends.Backend.OpenReadOnly also serves the
root pre-run's classified reads, and `bd ready --claim` writes through
that store. The exception is named in the backends.Backend contract,
the store_factory invariant and the store doc, and
TestReadOnlyOpenServesTheClaimBdReadyMakes pins it. An OpenNonMutating
backend hook is the follow-up.

Security surface.
- http_target.json is in the doctor gitignore template and its required
  patterns.
- BEADS_HTTP_TOKEN and BEADS_HTTP_TOKEN_COMMAND are host-scoped with
  BEADS_HTTP_CA_FILE's "host[:port]=value" syntax. A value scoped to
  another server falls through to the next rung, and a command scoped
  elsewhere never runs.
- LoadTarget and SaveTarget refuse a url that carries userinfo, and
  Target.String drops userinfo.

The served Tier 3 had no CI home. The new target
//internal/httpclient:httpclient_served_test is an `embedded`-tagged
sh_test. It reruns httpclient_test's binary through
tools/bazel/go_test_variant.sh with the script's
BEADS_TEST_EMBEDDED_DOLT=1 and BEADS_HTTP_TEST_REQUIRED=1, in 24 shards
on the embedded lane's race build. bazel.yml's bazel-embedded job
already runs `bazel test //... --config=embedded`, so no workflow
changes.
- -test.timeout=19m, not the script's 40m: Go's own timeout then fires,
  and dumps every goroutine, before the action's 1200s limit.
- TestBazelRetiredLanesCannotBeNarrowed pins its args and env. The new
  TestConformanceServedTierRunsUnderBazel ties the script's Tier 3
  switches to the rule.
- The new untagged TestServedTierIsLinkedWhenRequired fails a required
  run built without cgo, which drops every served file (the sentinel
  minor).
- A first 12-shard run found two test defects; both are fixed.
  TestServedBatchApplyBoundsTheItemCount's 1000-item apply outlasts the
  client's 60s per-request timeout under -race, so a
  race_detector_{on,off}_test.go pair scales its "at bound" half to 150
  there, as embeddeddolt's TestBatchApplyContract does. And
  TestTheCompositionCatchesAWrongParameter's dropped-filter case leaned
  on rows earlier tests had seeded: run alone, or first in its shard,
  it failed. It now seeds its own row outside the scope.
- conformance.sh, its script test and
  engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md name the target.

Deviations from the review's suggested fixes:
- A bare BEADS_HTTP_TOKEN(_COMMAND) fails closed rather than falling
  back, and the error does not echo the value: a bare token split at an
  "=" would put token text into the pattern half. Dropping the bare
  syntax is safe today: the backend has no Register call site and no
  docs/ page names the variables.
- Target.String drops userinfo instead of using url.URL.Redacted, which
  masks only the password, so a token used as the username would print.
- The pre-#6890 claimIssue fallback leaves RowVersion unset instead of
  borrowing the follow-up read's token. That read is a second snapshot,
  so its token could postdate a write this claim never saw and arm a
  guard that should miss; a guard built from the unset token fails
  closed. Against a current server the claim rides updateIssue, whose
  `revision` is stitched as for any update.

Also fixed: the nit about the completeness failure message, which now
says to regenerate the shell, and the offrole.go tombstone comment that
named a skip list which no longer exists. Left for follow-up (all
latent, since httpclient.Register has no production caller): vocabOnce
latching a failed fetch, snapshot holding s.mu across the handshake
dial, unchunked graph anchors, private tracker IDs in park owners, and
the bounded journal limit above 10000.

Validation:
- git diff --check; gofmt -l; go vet over internal/httpclient/... and
  scripts
- go test -tags gms_pure_go over internal/httpapi/...,
  internal/httpclient/..., internal/storage/backends/... and
  backend/conformance/...; the doctor gitignore tests; the cmd/bd
  events-journal construction guard; ./scripts
- bazel test --config=embedded
  //internal/httpclient:httpclient_served_test on the lane's remote
  executors: 24 shards, 702 top-level tests, 632 pass, 70 skip (69
  ledgered KNOWN DIVERGENCE, plus
  TestCAFileHygieneRefusesGroupWritableFile, which finds no
  supplementary group on the executor), 0 fail, 0 data races; slowest
  shard 283s. tools/bazel/check_testcases.py --bep passes.
- each of the 702 tests run alone (go test -run '^Name$', without
  -race): 0 fail
- the whole tier under go test without -race, with the script's
  switches and -timeout=40m, before the two test fixes: 633 pass, 69
  skip, 0 fail
- removing BEADS_HTTP_TEST_REQUIRED from the rule fails both policy
  tests
- bazel test --config=ci over //internal/httpclient/...,
  //internal/httpapi/..., //scripts/..., //test/docsync/...,
  //internal/storage/backends/..., //backend/conformance/... and
  //cmd/bd/doctor/...: 32 tests pass
- make ci-pr-lint: nogo native plus windows/darwin, passes

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 07:47:17 +00:00
Bee
9b251db072
test(proxied): split TestProxiedServerClose into three parents with legacy rollback pins (#7173)
* test(proxied): split TestProxiedServerClose into Close + Close3 so legacy shard 5 clears the 15m timeout (#7151)

Shard 5 of the legacy 15-shard proxied-server block walled at 927-941s
(go test's 15m timeout) on every measured fork-PR and main run while the
other 14 shards took 337-765s. #7172 pinned the two hash-placed tests
that pushed it over; the parent that dominates the shard remains:
TestProxiedServerClose carries 23 parallel bd-init subtests at 100-130s
apiece under 4-way contention, ~2060 of the shard's ~2712 slot-seconds.

Split it the way TestProxiedServerClose2 was split before: subtests from
close_blocked_refuses_without_force onward (12 of 23) become
TestProxiedServerClose3 with the same header; no subtest body changes.

Manifest: an explicit `15 6 TestProxiedServerClose3` line in the frozen
block (shard 6 is the lightest shard by CI job time that took no #7172
pin; without the line cksum fallback could hash it back onto 5), and
`30 17 TestProxiedServerClose3` beside Close in the Bazel 30-shard block,
which keeps every 30-shard load where it was (heaviest still 372.2s
est.). proxied_test_durations.json apportions the single 174.6s Close
capture by bd-init count (83.5 / 91.1) and says so in its header; the
next sweep replaces both.

Verified: gen_proxied_shard_manifest.py 30 --weights=duration --check
(exit 0); go test ./scripts/ -run 'Shard|Manifest|Coverage'; go vet
./cmd/bd/; BEADS_TEST_PROXIED_SERVER=1 BEADS_TEST_REQUIRE_DOLT_CONTAINER=1
go test ./cmd/bd/ -run '^TestProxiedServerClose3?$' -v: both PASS
(32.24s / 30.53s, 23 subtests).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* ci(proxied): pin TestProxiedServerClose3 onto legacy shard 1, not 6 (#7151)

Shard 6 timed out at 15m on this PR's own run (job 111387202139) with
Close3 added: Close3 passed in 5s but TestProxiedServerClose2's
subtests, sharing the 4 parallel slots and the one proxied server with
Close3's 12 bd-init subtests, were still running at 1m18s-2m31s each
when the alarm fired. On main at b1a0690249 shard 6 runs 9m and shard 1
6m, the lightest of the 15 (15: 8m), so shard 1 has the headroom for
the ~6 min of 4-slot wall the 12 subtests cost.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(proxied): split TestProxiedServerClose3 again into Close3 + Close4 (#7151)

Shard 1 timed out at 15m on this PR's own run (job 111392255572) with
Close3 pinned there: Close3's 12 bd-init subtests finished at 3m33s, but
they held the 4 parallel slots long enough that TestProxiedServerDep2's
subtests only began at 5m35s and were still running when the alarm
fired. Shard 6 did the same before it. Meanwhile shard 5, the shard this
PR exists to relieve, ran 5m with the first 11 subtests.

So 12 inits in one parent is too much for any host shard. Close3 keeps
its first six subtests on shard 1; the last six become
TestProxiedServerClose4, pinned to shard 7 (7m on that run). The
durations file apportions the one 174.6s capture 11/6/6.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* ci(proxied): move the Close3 and Close4 pins off legacy shards 1 and 7 onto 13 and 2 (#7151)

The run at 765248af5 put every Proxied Dolt Cmd shard green, with shard 5
down from the 15m alarm to 5m50s. But shard 1, chosen for Close3 as "the
lightest of the 15 on main", ran 14m48s there, the heaviest of the
fifteen: TestProxiedServerDep2's 29 parallel bd-init subtests alone take
about 10.5 minutes of 4-slot wall on this lane, and Close3's six init
subtests landed on top of them. Shard 7, where Close4 went, ran 13m50s
behind TestProxiedServerReady and TestProxiedServerState.

Pin Close3 onto shard 13 (6m51s on that run) and Close4 onto shard 2
(7m57s). Not shard 5: it is light only because Close left it, and
putting Close3 back would recreate most of the set that timed out. The
30-shard Bazel block is unchanged.

Verified: BEADS_TEST_SHARD_LIST_ONLY=1 .github/scripts/proxied-test-shard.sh
13 15 lists Close3 and 2 15 lists Close4, neither on 1 or 7;
gen_proxied_shard_manifest.py 30 --weights=duration --check exit 0;
go test ./scripts/ -run 'Shard|Manifest|Coverage' ok.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 06:53:13 +00:00
Eddie the Engineer
452def7568 Merge latest main into adopted PR head (adopt-pr gate, merge mode)
Conflicts resolved by the adopt-pr rebase gate's inference repair:

- .github/workflows/conformance.yml: main retired it in #7321 (the
  conformance tiers now run as Bazel targets), so the deletion stands.
  The PR's Tier 3 path filter and 45 -> 100 minute budget had no other
  home.
- scripts/ci_f7c_advisory_test.go: main's rewrite. The PR's only change
  documented the deleted workflow's internal/httpclient/** filter entry.
- scripts/conformance_script_test.go: main's TestConformanceE2ETierRunsUnderBazel
  replaces the retired workflow-budget test; the PR's Tier 3 go call and
  exit-code case are kept.
- scripts/conformance.sh: the PR's Tier 3 and three-tier header on top of
  main's local-entrypoint header, which now says CI runs "the first two
  tiers" rather than "the same two tiers".

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 05:17:18 +00:00
Julian Knutsen
785f10724a
ci: bigger runners for the slowest cached-run jobs (MCP gate 8 vCPU -n16, Bazel test 4 vCPU) (#7381)
* ci: bigger runners for the slowest CI legs (MCP gate, Bazel test, Windows preflight)

Runner-size A/B against merge group 37721729916 (3m49s):

- package-mcp: 4 -> 8 vCPU with pytest -n 16 (BEADS_MCP_PYTEST_WORKERS,
  mode remote only; forks/ubuntu-latest keep package-mcp.sh's default 8).
  pytest was 70s, dominated by CPU-bound per-test `bd init` setup. Local,
  pinned with taskset: -n 8 on 4 CPUs 112s; -n 16 on 8 CPUs 60/57/58s,
  228 passed / 5 skipped in all three runs.
- bazel-test: 2 -> 4 vCPU. `bazel test //... --config=ci` spent ~63 of
  81s in client-side loading/analysis (critical path 7.65s, every action
  a remote cache hit).
- pr-preflight-platforms Windows leg: 4 -> 8 vCPU. main.yml's test-windows
  saver stays on 4 vCPU: the cache keys name runner.os/runner.arch, never
  the label, and actions/cache's version covers only paths + compression,
  so the entries are not size-specific; the other Windows consumers keep
  their exact label. TestWindowsPreflightLegReachesSaverAcrossSizes pins it.

Runner labels live in the shared constants in
scripts/ci_blacksmith_runner_test.go; TestBlacksmithBazelRunnerSizes checks
every bazel.yml lane's runs-on and the MCP worker count per rbe mode with
evalGHExpr. Revert any bump that does not save at least 15s.

Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(pr): keep the Windows preflight leg on 4 vCPU (A/B: 8 vCPU queued 63s)

#7381's PR run put pr-preflight-platforms' Windows leg on
blacksmith-8vcpu-windows-2025. The cache restore from main.yml's 4 vCPU
test-windows saver hit, but the job waited 63s for an 8 vCPU Windows runner
and finished at 144s against the 129s baseline on 4 vCPU, so the leg goes
back to blacksmith-4vcpu-windows-2025. With saver and consumer on one label
again, TestWindowsPreflightLegReachesSaverAcrossSizes and the separate
preflight runner map have nothing left to pin and are dropped; the result
is recorded next to platformsMatrixRunners, in pr.yml and in
CI_REQUIRED_CHECK_TOPOLOGY.md.

Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 05:16:20 +00:00
Julian Knutsen
143c418148
ci(bazel): cut per-lane fixed cost — LLVM .tar.zst slice, release cross-compile lane, no protoc build (#7379)
* ci(bazel): fetch the LLVM toolchain as a 149 MB slice; re-key the runner cache

Port of gascity #7279. @llvm_dist_linux_x86_64 downloaded the 1.94 GB
LLVM-22.1.8-Linux-X64.tar.xz and ran a single-threaded `tar -xJf` over
11 GiB to keep a 688 MB slice. That took 94-147 s in every Bazel lane,
and every cgo, race and stdlib action waits on it.

- llvm_dist.bzl gains `sliced`: download_and_extract of a members-only
  .tar.zst, plus a missing-member check. The xz path stays.
- MODULE.bazel pins the toolchain-llvm-22.1.8-slice-1 asset by sha256
  (de614777...6bec5). The URL is gascity's release asset for now; the
  list form leaves room for a beads mirror of the same bytes.
- tools/cc_toolchain/repack_llvm.sh and llvm_members.txt rebuild the slice
  from the pinned upstream release.
- Runner cache v4, keyed on MODULE.bazel too. MODULE.bazel.lock does not
  record llvm_dist, so the v3 key never changed and kept restoring the
  2.9 GB entry that holds the xz archive.
- bazel_hermetic_cc_test: a sliced llvm_dist may only download from a
  gastownhall/beads or gastownhall/gascity release, and its members must
  equal llvm_members.txt.

Verified locally:
- Every file in the extracted slice (2009 entries: types, modes,
  symlink targets, sha256) is identical to the xz extraction.
- aquery action keys are identical, base vs head, at the same workspace
  path: 7565/7565 over deps(//cmd/bd:bd_for_tests) and 7146/7146 over
  the race deps(//internal/storage/embeddeddolt:embeddeddolt_test).
- Extraction time: 178 s (xz) vs 1.9 s (zstd).

Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the release cross-compile as its own lane

After the LLVM slice, bazel-pure is the longest cached Bazel lane (about
170 s). Its "Cross-compile every release target" step takes 54-58 s of
that (78.9k configured targets, 25k cache-hit lookups) and shares no
analysis with the pure invocations, because release_cross is a split
transition.

The step moves, unchanged, into a new job, bazel-release-cross ("release
cross-compile"). The job uses bazel-pure's runner, `needs: rbe` and skip
rule, so it runs in every mode except skip. It reports job.status like
every lane: the workflow_call output is bazel-release-cross, and pr.yml's
CI Gate requires it as BAZEL_RELEASE_CROSS. bazel-gate.sh skips it only in
mode skip, as it does BAZEL_PURE. Merge groups use the same pr.yml gate.
Push and dispatch runs gate nothing. Existing gate ids are unchanged.

The policy test now pins the step to bazel-release-cross, requires the new
gate id, and rejects any other job that runs the script (no double runs).
The new job is listed as having no CI analytics step. It captured no BEP as
a bazel-pure step either, and the analytics steps are unchanged. Docs:
LINTING.md, TESTING.md, CI_REQUIRED_CHECK_TOPOLOGY.md.

Expected: bazel-pure about 110 s on cached runs; the new lane about 85 s,
in parallel. Cost: one more 2-vCPU Blacksmith runner per run.

Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): stop building protoc for protoc-gen-validate

github.com/envoyproxy/protoc-gen-validate ships validate/validate.pb.go,
but its own validate/BUILD makes the Go library a go_proto_library. So
every lane built protoc from source just to regenerate that file. The
module reaches the graph only through grpc xds and the cloud SDKs.

A gazelle_override now regenerates the module's BUILD files with
`gazelle:proto disable`. It sets build_file_generation = "on" and adds
`gazelle:build_file_name BUILD.bazel`, so gazelle writes BUILD.bazel
(which Bazel prefers) instead of keeping the upstream BUILD. The proto
directive alone changes nothing, because the upstream BUILD files win.
validate is now a plain go_library over validate.pb.go.

Measured locally with aquery:
- //... minus the manual release_cross target: base has 2096 CppCompile,
  612 CppArchive, 314 CppLink and 1 GoProtocGen action; head has 0.
- cquery somepath(//..., @protobuf//:protoc): base goes through
  grpc xds -> envoy core/v3 -> validate_go_proto -> protoc; head is
  empty.
- deps(//cmd/bd:bd_for_tests): 7565 actions become 3077. The race
  deps(//internal/storage/embeddeddolt:embeddeddolt_test): 7146 become
  2658.

MODULE.bazel.lock does not change. The v4 runner cache key includes
MODULE.bazel, so it still moves. Expect one cold compile of the Go
packages above validate on the first run after merge.

Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): bazel mod tidy formatting

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 03:13:34 +00:00
Julian Knutsen
233d151b37
ci(pr): cut the Windows PR preflight leg's fixed costs (slice W) (#7378)
PR preflight process (windows-latest) is a CI Gate input and, after the
Bazel fixed-cost work, the merge-group critical path (ends at +247..+303s).
Step timings from merge group 37710264475 and seven same-repo PR runs:
45-115s setup (setup-go, then module and build cache tar extraction),
33-71s "Exercise the real Bash process boundary", 27-58s ./cmd/bd test
compile+link, 5-9s each for three more go test relinks.

- scripts/pr_preflight_test.go: the ~29 pr-preflight.sh runs (about 1.5s
  each under Git Bash on Windows, 42.8s total in the merge group) now run
  as parallel subtests. Each run already had its own temp dir, call log
  and env; the one test that t.Setenv's stays serial. Linux: 4.8s -> 0.8s.
- pr.yml: the hook-timeout step and the bd build fallback step become one
  go test of ./cmd/bd, saving a package re-hash and test-binary relink
  (5-9s on Windows for a sub-second test). Policy tests and the allowlist
  and topology docs follow the step name.
- warm-non-race-cache.sh: on Windows also compile the three CGO_ENABLED=0
  credential-fixture test packages the Windows-only fixture step runs, so
  main.yml's test-windows seeder covers them.

No step, test or gate input is dropped; the job id, name, matrix and
runner are unchanged.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 02:42:27 +00:00
Julian Knutsen
dff41b4bb8
fix(bazel): re-vendor signed-duration decode fix from gascity (#7374)
gascity commit 9865cc045aee1391dbcc7eac99339c26e3a5ce71 fixed a
production bug: negative protobuf durations in the compact exec log
were decoded as unsigned, so the collector rejected roughly 22% of
artifacts on remote_queue_ms. Re-vendor the extractor and its unit
test byte-identical from that commit and update the sha256 pins in
ci_analytics_extract_test.go accordingly. testdata/ci_analytics/ is
untouched by the fix; its tree-hash pin is re-verified unchanged
against gascity's tree.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-08 01:05:38 +00:00
Julian Knutsen
886ebb375a
ci(rbe): fail remote runs fast on a stale worker-env pin; nightly opens an issue (#7373)
rbe-west's pool workers run gascity main's tools/rbe/worker-env and
advertise its pin; the schedulers match //platforms:rbe_worker's pin
exactly. When gascity re-pins and beads' copy falls behind, every remote
action queues until the lanes time out, and the only check ran nightly
with no notification (review finding S1).

- bazel.yml's rbe job: after decide, only when the lanes execute remotely
  (enabled: modes remote, fork-ro, fork-rw; merge groups are remote), a
  sparse checkout of tools/rbe and platforms and tools/rbe/worker-env-sync
  main. Out of step fails the job, so every lane skips and pr.yml's gate is
  red within a minute, with the diff and the pin to commit in the summary.
  No fallback to mode cache: pr.yml's ci-gate requires a remote run
  (BAZEL_REMOTE), so it would be just as red, only slower. gascity
  unreachable warns and proceeds. The steps set no output; the job's
  outputs stay decide's.
- worker-env-sync: exit 1 only for out of step, exit 2 (warning) when it
  cannot compare (ERR trap; curl timeouts), WORKER_ENV_SYNC_REPORT copies
  the markdown report.
- nightly.yml: top-level permissions contents: read; rbe-worker-env-sync
  gets issues: write and, on exit 1, tools/rbe/worker-env-sync-issue opens
  a ci/infra issue with the report or updates the open one (body + comment
  linking the run). GITHUB_TOKEN is in that one step.
- Policy tests: TestBazelRBEWorkerEnvPreflight (shape + the step's script
  run in step / out of step / unreachable), TestNightlyWorkerEnvSyncOpensIssue
  (permissions, token scope, rc output, issue create/update via a stub gh),
  TestRBEWorkerEnvSync (exit codes, report); rbe job step and checkout
  counts updated.

Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 00:55:02 +00:00
Julian Knutsen
77ab980790
ci(bazel): run the hook-timeout-backends check under Bazel; test-nix stops running tests (#7357)
checks.nix's hook-timeout-backends (the tracked git hook shims against
real GNU/uutils/busybox/toybox timeout implementations, with and without
perl, under dash, bash and busybox ash: 240 cases) was the one test PR
Risk's test-nix ran on a GitHub runner, through `nix flake check`. It is
now //tests/hook_timeout_backends:hook_timeout_backends_test, an sh_test
(pr-core-only) that the bazel-test lane runs remotely on every PR: the
static uutils 0.12.0, busybox 1.35.0 and toybox 0.8.15 binaries come in
sha256-pinned through MODULE.bazel; dash, bash, perl and GNU coreutils
from the executor, failing (not skipping) when one is missing.

checks.nix goes and flake.nix declares no checks, so test-nix's
`nix run .#default` + `nix flake check` is a packaging check (build the
package Nix users install, evaluate every flake output), not a test.
TestHookTimeoutBackendsRunUnderBazel pins the target and that no flake
check comes back.

ga-96smfk.22

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 11:05:39 +00:00
Julian Knutsen
84390b03ab
ci(bazel): run every Windows/macOS job on Blacksmith; retire the Linux preflight leg and the cross-compiled Windows pair (#7355)
- pr.yml's Windows jobs and the macOS/Windows legs of pr-preflight-platforms
  and check-doc-freshness-platforms run on Blacksmith for every PR and merge
  group, forks and Dependabot included: no windows-latest/macos-latest
  fallback (Blacksmith serves this org's fork PRs). Matrix legs name their
  label (runs-on: ${{ matrix.runner }}); the same-repo "matrix marker"
  ternary and its policy plumbing are gone.
- pr-preflight-platforms drops its Linux leg: its tests run under Bazel on
  rbe-west (//scripts:scripts_test, //cmd/bd:bd_test,
  //internal/config:config_test).
- The Linux-cross-compiled Windows path is retired: windows-test-binaries,
  its two -prebuilt twins, main.yml's windows-test-binaries-cache, the
  WINDOWS_PREBUILT_REQUIRED flag and the build script/manifest. The native
  pair (test-windows-liveness, worktree-remove-windows) is the required run.
- main.yml's test-windows and blacksmith-macos-go-build-cache drop their
  GitHub-hosted venue legs (no GitHub-hosted consumer left).

Gate names unchanged. ga-96smfk.22

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 10:41:14 +00:00
Julian Knutsen
9d13485b71
ci(bazel): retire main.yml's Linux jobs, nightly's go test jobs and ci-measurements (#7354)
Push to main and the nightly schedule no longer run go test or repo checks
on GitHub runners (ga-96smfk.22, ga-96smfk.58):

- main.yml keeps only the macOS -race -short suite, the Windows smoke build
  and the push-to-main savers of the GOCACHEs pr.yml's Windows/macOS jobs
  restore. Its Linux jobs (migration hygiene, build artifacts, build tags,
  versions, duplicate migrations, doc flags, package gates, PR policy
  wrapper, gofmt, the Linux and setup-go cache seeders, the Nix flake
  check) are gated on the merge queue's pr.yml/pr-risk.yml run of the
  same commit and re-run as Bazel targets by bazel.yml's push run.
- nightly.yml is the fresh bazel.yml call plus the worker-env check; its
  Full Test Suite, PR Core go-test-json side and measure dispatch are gone,
  as are ci-measurements.yml and bazel.yml's go-test-json-artifact input.
- The non-race embedded batch-apply step becomes
  //internal/storage/embeddeddolt:embeddeddolt_batch_apply_nonrace_test
  (go_test_race_off, BEADS_TEST_EMBEDDED_DOLT=1) in the embedded tier.
- make ci-pr-policy and scripts/ci/pr-policy.sh go: every check it ran is
  a Bazel test (TestPRPolicyChecksRunAsBazelTargets).

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 10:06:59 +00:00
Julian Knutsen
4f924fc161
ci(bazel): retire PR Risk's legacy Dolt tiers, pr.yml's legacy Go jobs and the D2 coverage flags (#7353)
Every pull_request and merge group already took the Bazel lanes for the
retired tiers (BAZEL_RETIRES_LEGACY_* and BAZEL_COVERS_FORKS all "true",
and pr.yml/pr-risk.yml run on no other event), so the legacy jobs those
flags stood down could never run again. Remove them with their decision:

- pr-risk.yml: build-embedded, test-embedded-{storage,conformance,cmd},
  test-proxied-cmd, test-server-storage{,-full}, detect-ci-tier (and its
  ci-embedded-tier.sh / ci-capability-selector.sh scripts) and
  bazel-coverage. PR Risk keeps the Nix packaging check (test-nix) and its
  gate, PR Risk Gate / Required.
- pr.yml: build-artifacts, check-cmd-bd-puregeo-tests, pr-core-wrapper and
  bazel-coverage; the BAZEL_RETIRES_LEGACY_* and BAZEL_COVERS_FORKS flags.
  The three BAZEL_*_RETIRED gate ids become one, BAZEL_REMOTE: the Bazel
  call must have run remotely (remote, fork-ro or fork-rw) on every PR and
  merge group, each lane's own id still required. Gate names unchanged.

Policy tests: the D2 decision matrix becomes
TestPRGateRequiresRemoteBazelLanes (bazel.yml's actual rbe step and
pr.yml's actual gate over every event/variable/secret/fork/actor/mint
fact); the parity tests that read the legacy jobs now pin the Bazel lanes
against those jobs' frozen commands.

ga-96smfk.22

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 09:31:20 +00:00
Julian Knutsen
57e86a2f0d
ci(bazel): add CI analytics extraction to all lanes (S3) (#7356)
tools/bazel/ci_analytics_extract.py turns a lane's BEP file, compact
execution log and JSON profile into a redacted ci-analytics-summary
(timing, per-mnemonic cache classes, cold actions, test results,
repo-fetch time): only validated labels, mnemonics, enums and numbers
leave the runner, never an endpoint, cert/key path, env value, token
or hostname. Every lane's "CI analytics summary" step runs the
extractor and uploads its output as ci-analytics-<lane>-<run_attempt>
(7-day retention); CHECK_RUN_ID uses the real job.check_run_id
context (requires actionlint >= 1.7.8), and both the summary and
upload steps are continue-on-error, the summary capped at
timeout-minutes: 3, and both gated on !cancelled() (not always(),
which also runs after a workflow cancellation).

The extractor, its 62-case unittest file, and testdata/ci_analytics/
(the real/* Bazel 9.2.0 fixtures, poisoned-bep.json, mismatch.bep.jsonl
and regen.sh) are vendored byte-identical from gascity's
ci/analytics-extractor slice (final squashed commit
00161373bc7bbfcef8d6a05ac5ce70409fc7f2ac), which is canonical for this
pair the same way tools/rbe/cache-zstd-probe.sh is a byte copy.
Fixtures are regenerated under env -i with every BEP field stripped
unless it's on an explicit allowlist, and the compact exec log is
committed decompressed (*.binpb) rather than as a zstd frame, so
nothing opaque reaches version control.

scripts/ci_analytics_extract_test.go is the thin Go wrapper:
TestCIAnalyticsExtractorIsPinnedToGascity pins both vendored files'
sha256 so local drift is caught, and TestCIAnalyticsExtractUnitTests
runs gascity's unittest file in place. scripts/ci_analytics_workflow_test.go's
TestCIAnalyticsStepsAreSafeAndBounded mirrors gascity's pin test,
adapted to beads' 8 separately-named lane jobs: continue-on-error,
timeout-minutes bounds, no raw ${{ }} expression in run:, !cancelled()
required in if:, correct per-lane upload artifact name/path, and
CHECK_RUN_ID == ${{ job.check_run_id }}, for every lane.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-07 09:07:24 +00:00
Julian Knutsen
368b225eef
docs(bazel): make bazel test the primary build and test path (ga-96smfk.23) (#7348)
Bazel is beads' build and test system: bazel.yml's lanes gate every PR,
and nogo (lint + vet), gofmt and the repository guards exist only as
Bazel targets. The docs, the Makefile and the hooks now say so.

Makefile: `make test` runs bazel.yml's test lane (`bazel test //...
--config=ci`), `make check` adds the testing.Short policy and the
`make ci-pr-lint` nogo gate, and `make check-docs` runs the docsync and
doc-freshness Bazel targets before the pinned-release CLI flag check.
BAZEL_FLAGS passes --config=fork-cache or --config=remote-exec through.
The plain-Go recipes stay as explicit twins: test-go, check-go,
check-docs-go. No workflow called the primary names, and
TestWorkflowsNameTheirTestEngine keeps it that way.

Hooks: pre-push runs scripts/pre-push-suite.sh for branch pushes whose
commits change Go or Bazel inputs: the test lane with
--config=remote-exec when some rc names an executor, else
--config=fork-cache. Without bazel, or with BD_PREPUSH_SUITE=go, it runs
make test-go under a banner saying that is not what CI gates on.

Docs: AGENTS.md, CONTRIBUTING.md, engdocs/TESTING.md (a per-lane command
table and the fork-cache/remote-exec setup), README, RELEASING, the PR
template and the .bazelrc header. Bazelisk is a required prerequisite;
go test is an inner-loop convenience CI does not enforce.

Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 07:46:02 +00:00
Julian Knutsen
8d7d7fef16
build(rbe): adopt gascity's toolchain-only worker-env pin; check the copy nightly (#7346)
gascity now builds the worker-env manifest from the worker's toolchain,
not the image's patch revisions. The manifest records the arch, the OS
release, Go and dolt, and the upstream releases of the libraries and
tools actions reach (major.minor; git and yq at their major). The Debian
epochs and Ubuntu revisions are gone, so the Blacksmith image's routine
security updates no longer move the hash. That is what stranded every
remote action on 2026-10-07: coreutils 9.4-3ubuntu6.1 -> 6.2 and libc6
2.39-0ubuntu8.6 -> 8.8 forced re-pins (gascity #7266, beads #7338).

- tools/rbe/worker-env.txt: byte-identical copy of gascity's new
  manifest. platforms/BUILD.bazel pins its sha256,
  sha256:6be1381cf08e7fd66684cfce6f26aca57de2561e722c505d1f2f578e9fdeafa7.
  The manifest now also measures python3, curl, openssl, unzip and yq,
  which //scripts:scripts_test runs from the executor.
- scripts/rbe_worker_env_test.go: the manifest must be a rendering of
  gascity's tools/rbe/worker-env (sorted; one arch, dolt, go, os and yq
  line; packages at most major.minor, installed). A copy of an older
  gascity manifest with dpkg revisions, a hand edit or a truncated copy
  fails here.
- tools/rbe/worker-env-sync, run by nightly.yml: compares the manifest
  and the pin with gascity's main byte for byte. If they differ, it
  prints the diff and the pin to commit, and fails.

Merge this together with the gascity change. Pool workers run gascity
main's provisioning, so they serve the new hash only once that change
lands, and beads' actions on any other pin queue. This supersedes #7338
(the old-format re-pin). The new key misses the cache once.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 23:18:24 -07:00
Julian Knutsen
a4509deb2d
ci(bazel): nogo replaces go vet and golangci-lint (ga-96smfk.18) (#7341)
* ci(bazel): nogo replaces go vet and golangci-lint (ga-96smfk.18)

Lint and vet now run as nogo inside the Bazel build, so they gate wherever
the Bazel lanes execute (rbe-west), and the golangci-lint and go vet jobs
leave pr.yml and main.yml.

- //tools/nogo (own go.mod, joined to beads' for go_deps only through
  gazelle.go.work): go test's vet checks (cmd/go's defaultVetFlags, the set
  scripts/ci/go-test-vet.sh ran) plus the eight linters .golangci.yml
  enables, each a wrapper over the library version golangci-lint v2.10.1
  uses (depguard, errcheck, forbidigo, gosec, misspell, sloglint, unconvert,
  unparam).
- .golangci.yml stays the single lint policy: Bazel embeds it and
  tools/nogo/internal/golangci applies it as golangci-lint does (enabled
  set, settings, run.tests: false as library-unit-only analysis, lax
  generated-file detection, path/path-except/text exclusion rules), with
  strict decoding so an unimplemented key fails the build.
- gascity's rules_go //nolint-range patch and x/tools ctrlflow patch,
  byte-identical.
- Windows/darwin: cross-configured, not a golangci leg. The pure lane's
  release cross-compile (//tools/bazel:release_cross, every library and
  binary for all eight release platforms, cgo off) runs nogo on every
  compile, a superset of golangci-lint's windows and darwin legs.
  scripts/pr-lint (make ci-pr-lint, bd preflight) runs `bazel build
  --config=nogo //...` plus release_cross for windows_amd64,darwin_arm64.
  release_cross skips tools/nogo, a separate Go module `go build ./...`
  never compiled.
- pr.yml: scripts-go-checks (vet) and pr-lint-wrapper (x3) are gone, with
  their ci-gate entries; main.yml: pr-lint-wrapper (x6) and go-vet-cache.
  scripts/ci/go-test-vet.sh and install-golangci-lint.sh are deleted.
- pre-commit hook and .pre-commit-config.yaml run `make lint-changed
  LINT_CHANGED_SCOPE=staged` (nogo over the staged packages); Makefile adds
  lint, vet and lint-changed. pr-lint.sh no longer runs gofmt
  (//scripts/repochecks:fmt_test owns it; ga-96smfk.41).
- internal/testutil/integration: five gosec findings in integration-tagged
  library files golangci-lint never built, now linted by the integration
  lanes, carry reasoned //nolint directives.

Parity: with the exclusion rules removed, golangci-lint and nogo report the
same 3,916 findings natively and 3,859 each for windows and darwin (file,
line, linter, text); with .golangci.yml as committed both report none.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): equivalence.py leaves nested Go modules' tests out of go test's set

tools/nogo is its own Go module, so `go test ./...` in the root never runs
its tests; Bazel's unit lane does. Its packages are neither expected nor
extra.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 05:17:22 +00:00
Julian Knutsen
ce661175ff
ci(bazel): retire the docker A/B variants and pr.yml's container jobs (#7340)
The dolt-server lane (--config=doltserver) has been the only pre-merge run
of the Dolt-backed domain, uow, tracker, doctor/fix and protocol suites
since BAZEL_COVERS_FORKS went "true": bazel-coverage reports pr_lanes=true
for every pull_request and merge_group, so pr.yml's test-domain-uow and
contract-corpus never run. An A/B run of every requires-docker target
against its dolt-server twin on the same commit shows identical test sets
and statuses (2683 testcases) except TestDoltServerFingerprint/container,
the container half of the fingerprint, which needs docker by design.

Delete the six requires-docker sh_tests, test:docker in .bazelrc, the
requires-docker tag from the taxonomy and lane filters, bazel.yml's
dolt-lane dispatch input and image pull, and pr.yml's test-domain-uow and
contract-corpus jobs with their ci-gate wiring. The testcontainers backend
in internal/testutil stays: it is plain go test's default. pull-dolt-image.sh
stays for pr-risk.yml's legacy tiers. No required check changes: the
rulesets require only CI Gate / Required and PR Risk Gate / Required.

Refs ga-96smfk.21

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 04:13:24 +00:00
Julian Knutsen
3f1c6038f0
ci: warn-only shard-balance budget for bazel-server-storage (#7339)
rbe-ci-cost-latency-study.md recommendation 4: this tier's last shard
has trailed the rest by 4.3-4.5 min in 2 of 8 runs. tools/bazel/shard_budget.py
reads the job's BEP file and prints ::warning when a sharded target's
slowest shard exceeds max(150s, 2x median); --fail-on-imbalance (not
wired in yet) switches that to ::error and a nonzero exit once the
budget has run clean for a while.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-07 04:05:33 +00:00
Julian Knutsen
5e73670866
ci(bazel): cross-compile every release target with Bazel, retire the go build job (#7332)
pr.yml's check-release-target-cross-compilation ran `CGO_ENABLED=0
GOOS/GOARCH go build -tags gms_pure_go ./...` for the eight release targets
on two GitHub/Blacksmith runners. It is now a step of bazel.yml's pure-Go
lane (BAZEL_PURE, already required by ci-gate):
scripts/ci/bazel-release-cross-compile.sh builds //tools/bazel:release_cross
once, remotely, with --//tools/bazel:release_platforms set from
scripts/ci/release-targets.txt (still the single source of truth, pinned to
.goreleaser.yml). release_cross's split transition configures every
go_library/go_binary for each rules_go platform with cgo off, so all eight
platforms build in parallel in one invocation.

- tools/bazel/release_cross.bzl: the split-transition rule and the
  string-list flag.
- tools/bazel/go_srcs.py (make bazel-sync, checked by bazel-sync-check and
  TestBazelGoSrcsBlocksCurrent) keeps the release_cross target list; the
  script checks with bazel query that every Bazel package holding Go
  targets is reached before it builds.
- //tools/bazel:pure_bd_has_no_cgo_only_deps: Bazel compiles gozstd's pure
  stubs where `go build` rejects them, so the pure bd of every platform
  must not link gozstd.
- //internal/storage/embeddeddolt/cmd:cmd (cgo-only main, skipped by
  `go build ./...` under CGO_ENABLED=0) is tagged cgo-only and incompatible
  with pure builds.
- A failed build is re-run per platform (cache hits) to name each failing
  GOOS/GOARCH.

Same cgo-off coverage boundary as the retired job. Refs ga-96smfk.20.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 03:18:37 +00:00
Julian Knutsen
bb9b9e5020
ci(bazel): run the upgrade smoke tests under Bazel; retire cross-version-smoke.yml (#7331)
cross-version-smoke.yml resolved the newest 5 GitHub releases (30 on a tag
push) with `gh release list`, then built the candidate and downloaded each
release on a GitHub runner for scripts/upgrade-smoke-test.sh.

//tests/upgrade_smoke has one target per release in the newest 30 the
release catalog pins a linux/amd64 asset for (@bd_releases//:versions.bzl):
upgrade_smoke_<release>_test runs the script against the non-race
bd_for_tests with that release pinned as PREV_BIN (sha256-verified when
Bazel fetches it; the script checks it reports the release and downloads
nothing). A release added to the catalog joins the window. Tagged
pr-core-only, so bazel-test (not the integration lane) runs them remotely
on every PR, merge group and
push, cached until the candidate, the script or the release changes.

The tested set is now reviewed rather than resolved at run time: stable
releases from the catalog, where the workflow took GitHub releases
including prereleases. cross-version-smoke.yml and the F7c checks of its
chunk fold, path filter, runners and binary cache are removed (the
path-filter and binary-cache checks had no other workflow left).

Refs: ga-96smfk.19

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 02:49:39 +00:00
Julian Knutsen
7e0adfd353
ci(bazel): run the historical-upgrade corpus under Bazel; retire migration-test.yml (#7330)
* chore(migration-test): extend the release catalog through v1.3.1

The catalog stopped at v1.1.2 (the v1.2 upgrade census). The migration
corpus already upgrades from v1.2.2, and the Bazel targets that replace
migration-test.yml and cross-version-smoke.yml take their pinned release
binaries from the catalog (@bd_releases), so it has to cover the releases
since. Regenerated with `catalog generate` after raising maximumVersion to
v1.3.1: five new stable versions (v1.2.0, v1.2.1, v1.2.2, v1.3.0, v1.3.1;
three with a linux/amd64 asset) and five new proxy prereleases; no existing
entry, drift record or repository-only exclusion changed. The count
constants and the identity digest are updated to match, and the README now
says how to add a release.

Refs: ga-96smfk.19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the historical-upgrade corpus under Bazel; retire migration-test.yml

migration-test.yml ran scripts/migration-test/run.sh --version <release> for
14 reviewed releases in 3 shards on GitHub runners: it downloaded each
release archive, built v0.9.1 from source with Go 1.25 through the module
proxy, installed libicu74 and the dolt 2.1.8 runtime, all over the network.

//tests/migration now has one target per corpus release
(historical_upgrade_<release>_test) plus legacy_bridge_test, all hermetic:
- Published releases come from @bd_releases (the release catalog's
  sha256-verified assets) through a per-target directory
  (HISTORICAL_RELEASE_DIR); lib/binary.sh uses it instead of downloading and
  still checks each binary's self-reported version.
- v0.9.1 is //tests/migration:bd_source_tag, the same `go mod download` +
  `go build -trimpath` with CGO and Go 1.25.0, run offline as a build
  action: the module files come from tests/migration/source_tag_modules.json
  through a file:// GOPROXY tree (tools/bazel/go_module_proxy.bzl), the
  release's own zip is the catalog's source_zip, and go checks every
  dependency against the release's go.sum (SOURCE_TAG_SQLITE_BIN).
- The external Dolt runtime is @dolt_test_runtime_linux_amd64 (dolt.bzl's
  dolt_release gains a version attribute; 2.1.8 is pinned to versions.sh's
  digest), passed as DOLT_BIN to the server-Dolt and wisp-plane releases.
- The workers already provide jq, python3, sqlite3 and libicu74.

The targets are untagged, so bazel-test runs them remotely on every PR,
merge group and push; result caching replaces the path filter. Every
release also passed inside a loopback-only network namespace.
scripts/migration_corpus_bazel_test.go keeps the BUILD lists equal to
versions.sh's corpus, the catalog digests equal to STRICT_RELEASE_SHA256,
the source-tag lock equal to the catalog's source_zip, and the Dolt runtime
pin equal to versions.sh's. migration-test.yml and the policy tests that
pinned its shards, caches, apt step and runner are removed.

Refs: ga-96smfk.19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the historical-upgrade targets in the PR-core lane only

The integration lane runs every untagged target too, and its
check_testcases.py requires Go testcases, which a shell test's test.xml does
not list. A new pr-core-only tag keeps //tests/migration out of
--config=integration (bazelIntegrationExcludedTags adds it), so the corpus
runs once, in bazel-test, against the release-shaped bd_for_tests (no
integration tag). build_source_tag.sh carries the build-tags opt-out: it
builds a historical release with that release's own go.mod, which predates
gms_pure_go.

Refs: ga-96smfk.19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 02:16:11 +00:00
Julian Knutsen
9a91f0629e
ci(bazel): run the differential regression suite under Bazel; retire regression.yml (#7324)
regression.yml ran `go test -tags=regression ./tests/regression/...` on a
GitHub runner: it built the candidate with `go build`, downloaded the v0.49.6
baseline release over the network, and pulled the dolt-sql-server image.
None of that works on rbe-west workers (no network for fork lanes, no docker).

- tools/bazel/bd_releases.bzl: a module extension that reads
  scripts/migration-test/release-catalog.json and declares one repository per
  catalog release with a linux_amd64 asset digest (bd_release_<version>,
  verified against the catalog's sha256), behind the @bd_releases hub
  (@bd_releases//:<version>, plus versions.bzl). Adding a release is a
  catalog change only; nothing downloads until a test declares one.
- //tests/regression:regression_test: the suite's files build under
  `regression || integration`, so gazelle generates the target. It runs the
  injected non-race bd_for_tests (BEADS_TEST_BD_BINARY) against
  @bd_releases//:v0.49.6 (BD_REGRESSION_BASELINE_BIN), both over a hermetic
  dolt sql-server (BEADS_TEST_DOLT_SERVER=local, required), in 8 shards.
  TestMain checks the baseline reports BASELINE_VERSION, so the label and the
  file cannot drift. Tagged dolt-server-cmd: bazel.yml's bazel-cmd-dolt lane,
  which pr.yml's gate requires, runs it remotely; test result caching stands
  in for the old path detector.
- A plain `go test -tags=integration ./...` (nightly) compiles the files but
  skips the suite; only -tags=regression (make test-regression) or Bazel
  runs it.
- regression.yml, main.yml's warm-up compile for it and the policy tests that
  pinned its jobs are removed.

Refs: ga-96smfk.19

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 01:41:12 +00:00
Julian Knutsen
b1fe68e63a
ci(bazel): run pr.yml's repository guards as Bazel tests (#7329)
fast-checks, PR Policy, and the Linux leg of check-doc-freshness-platforms
ran repository guards on GitHub runners. They now run as Bazel tests in
bazel.yml's `test` lane (remote, cached), each over only the files it reads:

- tools/bazel/repo_subset.bzl picks a narrow, path-selected slice of
  //:repo_files at analysis time. scripts/repochecks/run_check.sh copies that
  slice into a throwaway git checkout and runs the unchanged guard there.
- //scripts/repochecks: build_tags, workflow_tags, go_install_guidance,
  winget_portable_alias, migration_hygiene (checks A and B),
  fmt (registered SDK gofmt), doc_freshness, workapi_frontend_boundary,
  types_gen_drift (`make api-check` drift, oapi-codegen from go.mod).
- //scripts:doc_freshness_required_test (integration lane, -required-suite)
  and //scripts/gitattributespolicy:gitattributespolicy_required_host_test
  cover the Linux doc-freshness leg.
- check-versions stays covered by //internal/versioncheck:versioncheck_test.
  check-testing-short stays covered by //scripts:scripts_test.

pr-policy-wrapper is deleted. fast-checks keeps only the two checks that
need the PR base: migration hygiene's delta half (MIGRATION_HYGIENE_SCOPE=
delta) and the .beads diff. check-doc-flags stays, because it builds bd from
the pinned release tag. The example Go modules leave .bazelignore so gofmt
and the grep guards see them. check_testcases.py now accepts --not-go
//pkg:all.

Refs: ga-96smfk.20

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 00:59:18 +00:00
Julian Knutsen
72534f8e57
ci(bazel): check the Dolt server fingerprint on the dolt-server lane; retire pr.yml's fingerprint job (#7328)
pr.yml's every-PR "Test (Dolt server fingerprint)" job installed dolt, pulled
the dolt-sql-server docker image and ran TestDoltServerFingerprint against
both backends. The rbe-west workers have no docker, and the container backend
is now used only by the container-backed legacy jobs, which run only where
bazel-coverage does not cover pr_lanes.

- //internal/testutil:testutil_dolt_test runs testutil_test with the local
  backend selected and required (BEADS_TEST_DOLT_SERVER=local,
  BEADS_TEST_REQUIRE_DOLT_CONTAINER=1) on the dolt-server lane, which
  pr.yml's gate requires: the fingerprint's local half against the pinned
  dolt, plus the TestLocalDoltServer_* tests every other configuration
  skips under BEADS_TEST_SKIP=dolt.
- The container half moves back to test-domain-uow's first test step (where
  it was before D2 step 3), so it runs exactly where containers are used.
  //internal/testutil:testutil_docker_test is its docker-lane A/B control.
  TestPinnedDoltCLIMatchesContainerImage still pins the image tag to the
  dolt release.
- test-dolt-server-fingerprint, its ci-gate need/result and its policy
  tests are removed; TestDoltServerFingerprintRunsWhereEachBackendRuns
  pins the new placement.

Refs: ga-96smfk.19

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 00:59:16 +00:00
Julian Knutsen
c292964571
ci(bazel): run the managed-local proxied lifecycle lane under Bazel; retire proxied-local-smoke.yml (#7326)
proxied-local-smoke.yml built bd and the cmd/bd test binary on a GitHub
runner, installed the pinned dolt, and ran `-test.run ^TestManagedLocalProxied`
under `sudo unshare --net` with only loopback up, to prove the managed-local
lifecycle (bd -> loopback proxy -> dolt sql-server) needs no outbound network.

//cmd/bd:bd_managed_local_test runs the same tests (BEADS_TEST_PROXIED_LOCAL=1,
BEADS_TEST_SKIP=dolt, the non-race bd_for_tests as the bd under test, the
pinned dolt from the test wrapper's PATH) through tools/bazel/go_test_offline.sh,
which enters an unprivileged user + network namespace with only loopback up
and fails if outbound network is reachable: the job's netns, without sudo,
on the rbe-west workers. Tagged dolt-server-proxied, so bazel.yml's
bazel-proxied lane (remote-only, required by pr.yml's gate) runs it; test
result caching replaces the workflow's path filter. The job's "tests are
compiled in" assertion becomes scripts/managed_local_lane_test.go, which
evaluates cmd/bd's build constraints for a linux cgo build and requires each
of the 17 pinned tests.

The target joins the pinned retired-lane targets in
TestBazelRetiredLanesCannotBeNarrowed. proxied-local-smoke.yml and the policy
tests that pinned its job and its dolt version are removed (the Bazel dolt
pin is already enforced against install-dolt.sh).

Refs: ga-96smfk.19

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 00:59:14 +00:00
Julian Knutsen
cb0dcbecf2
ci(bazel): hermetic LLVM C toolchain and pinned rbe_worker exec platform (#7325)
beads' MODULE.bazel registered no C toolchain, so -race and cgo builds used
the autodetected local_config_cc of the machine that ran analysis. From a
client with lld installed (cherry) every remote cgo/-race link failed with
"collect2: fatal error: cannot find 'ld'" (the workers have no lld), and the
host gcc path, lld availability and gcc-version inputs keyed every Go action
to the client machine.

Port gastownhall/gascity's design:
- toolchains_llvm 1.11.0 with a pinned LLVM 22.1.8 slice
  (third_party/cc_toolchain/llvm_dist.bzl) and a sysroot of sha256-pinned
  Ubuntu 24.04 debs (deb_sysroot.bzl), without ICU (ICU policy);
  stock toolchains_llvm LLVM on other hosts.
- .bazelrc: BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 so nothing falls back to the
  host compiler.
- //platforms:rbe_worker, selected unconditionally, whose worker-env exec
  property is the sha256 of tools/rbe/worker-env.txt (a copy of the shared
  rbe-west OSS pool's manifest, owned by gascity), so the worker host is in
  every action key.
- scripts/bazel_hermetic_cc_test.go and scripts/rbe_worker_env_test.go guard
  both.

Refs: ga-96smfk.20

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 00:59:11 +00:00
Bee
bc34f1d77c
ci: upgrade workflows also run on schema-migration changes (#7150)
#7147 gave migration-test, conformance and cross-version-smoke a shared
paths filter: non-test Go files plus build inputs. The schema migrations
are .up.sql files embedded into the binary (go:embed in
internal/storage/schema), so they match no entry. A PR that adds a
migration with only a _test.go beside it skipped all three workflows,
where before #7147 it ran all three. d26230d0d and 63a471a1e have that
shape.

Add internal/storage/schema/migrations/** to the shared base in each
paths block and in advisoryPathFilterBase, which pins the base across
the three files.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 00:40:04 +00:00
Julian Knutsen
8761d2c40e
ci(bazel): run the conformance e2e tier under Bazel; retire conformance.yml and the docs-mintlify docsync job (#7321)
conformance.yml ran two tiers on a GitHub runner. Tier 1 (the embedded-Dolt
TestConformance) already runs as the embedded lane's
embeddeddolt_conformance_{core,audit}_test. Tier 2, the real-binary CLI
corpus (`go test -tags 'gms_pure_go e2e' ./test/conformance`), had only a
go_library under Bazel. Its two test files now build under `e2e ||
integration`, so gazelle generates //test/conformance:conformance_test
(integration-only, against the injected non-race bd_for_tests) and
--config=integration runs it remotely in bazel-integration, which pr.yml's
gate requires. PR Core's go test builds neither tag, so the --config=ci
equivalence check is unchanged; scripts/conformance.sh keeps working with
-tags e2e as the local go test entrypoint. With both tiers covered,
conformance.yml and main.yml's warm-up compiles for it are removed.

docs-mintlify.yml drops its docsync job (`go test ./test/docsync`, run by
bazel-test as //test/docsync:docsync_test on every PR) and keeps only
Mintlify's broken-link check, which needs the network (npx mint) and the
base branch's history; its path filter narrows to what that check reads.

Refs: ga-96smfk.17

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 22:42:25 +00:00
Bee
82872a81a5
fix(ci): rbe pre-warm survives a failed gh run list under bash -e, and clamps an over-long worker count (#7162) (#7163)
Actions runs the dispatch step as bash -e, so a failing gh run list ended the
step at the assignment and the could-not-read warning never printed. The
assignment now tolerates the failure. A RBE_PREWARM_WORKERS too long for the
shell's integer compare slipped past the max-4 clamp and the loop dispatched
without bound; two or more digits is now decided in the case (all zeros stays
the kill switch). The behaviour test runs the script with bash -e, as Actions
does, under a 30s deadline, and pins the clamp.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:35:02 +00:00
Julian Knutsen
0500fc6f40
ci(bazel): run the scripts and source-walking tests remotely; retire their go test legs (#7320)
//scripts:scripts_test and //scripts/check-versions:check-versions_test
drop their host-tools/no-remote-exec/no-remote-cache tags and run on the
rbe-west workers like every other target (the worker image ships bash, git,
python3, jq, yq, curl, openssl, unzip and coreutils; a missing tool fails the
test under Bazel rather than skipping it). Go comes in hermetically as the
registered SDK; check-versions.sh's `go build` is stood in for by the
Bazel-built checker.

The tests that walked the source checkout now get it as data:
tools/bazel/go_srcs.py keeps a managed `repo_files` filegroup in every
package and aggregates them into //:repo_files (exactly the tracked files
Bazel can see; make bazel-sync-check fails on a package without one).
scripts_test and docsync_test take it instead of hand-listed filegroups,
and every TEST_SRCDIR/IsBazel skip and go-test-only block in them is gone.
The release formula under .bazelignore'd .beads/ comes in through a
@beads_formulas local repository; npm-package/ and .claude/ are visible to
Bazel again (only their node_modules/worktrees stay ignored), so the npm
manifest release check and the AGENTS.md skill links run under Bazel too.
TestZZStdioNotLeaked falls back to TestMain's pre-run streams instead of
skipping in a shard without its baseline.

tools/bazel/equivalence_allowlist.txt shrinks from 20 skip entries to the
two cmd/bd tests of plain go test's own bd build fallback, which Bazel never
takes; pr-preflight-platforms now runs them on Linux, macOS and Windows. The
scripts-go-checks scripts-test and allowlisted legs, their scripts
(scripts-go-test.sh, allowlisted-go-tests.sh, run_allowlisted_go_tests.py)
and main.yml's --compile-only warm for them are removed. The vet leg stays
(ga-96smfk.18).

Refs: ga-96smfk.16

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 22:20:16 +00:00
Julian Knutsen
799c18e7db Merge remote-tracking branch 'origin/main' into feat/http-client-store 2026-10-06 21:44:00 +00:00
Julian Knutsen
9f60d71dae
ci(main): push to main runs the Linux test tiers under Bazel only (#7316)
bazel.yml already runs on every push to main and executes the PR Core,
pure-Go, dolt-server, embedded, proxied, server-storage, integration and
cmd/bd Dolt-server tiers on rbe-west in about five minutes. main.yml re-ran
the same tiers as legacy go test jobs (62 jobs, about 380 runner-minutes a
push), and its runs were all failing or cancelled. Remove the duplicated
Linux jobs: the ubuntu -race -short leg with its Codecov upload, PR Core,
the pure-Go compile check, domain+uow, the 6+8 integration shards, the 15
proxied shards, and the embedded build plus its storage, conformance and
cmd shards.

The macOS -race -short suite (rbe-west has no macOS workers) moves to
Blacksmith macOS and restores the macOS saver's caches. The cache savers
that fed the fork path from the removed jobs move to venue matrices: the
macOS saver gains a GitHub-hosted leg, and the Linux saver gains a
GitHub-hosted race leg (the embedded build used to seed it).

Refs ga-96smfk.14

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 20:52:01 +00:00
Julian Knutsen
b91383a41a fix(conformance): pass -tags gms_pure_go to Tier 3's served httpclient run
scripts/conformance.sh:61 ran a bare `go test -timeout=40m ./internal/httpclient/`
for Tier 3 (served HTTP client/role conformance), which `make ci-pr-policy`'s
build-tag policy check refuses ("bare go test without -tags=gms_pure_go").
Pass the same -tags Tier 1 already does, and update
TestConformanceScriptUsesExplicitTimeoutBudgets' pinned argv to match.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-06 09:45:18 +00:00
Julian Knutsen
9ecb33a5f9 feat(ci): add served HTTP conformance as Tier 3 of the conformance gate
The served tier (internal/httpclient's served_*_test.go corpus) never ran
in CI, so HIGH 2-5's regressions (and any future ones) would merge
undetected. Add it as Tier 3 of scripts/conformance.sh, the existing
required entrypoint conformance.yml already runs on every PR/push/merge_group
(not a new workflow, not wired through ci-gate.sh's aggregation -- this
repo's convention for this gate is a single standalone job):

  CGO_ENABLED=1 BEADS_TEST_EMBEDDED_DOLT=1 BEADS_HTTP_TEST_REQUIRED=1 \
    go test -timeout=40m ./internal/httpclient/

Left unsharded: Tier 1 and Tier 2 are themselves single unsharded
invocations, and Tier 3 is one Go package, not a multi-package corpus the
repo's existing package-sharding scripts apply to. No Bazel-lane addition:
bazel-coverage's embedded/dolt_server outputs only ever substitute for
main.yml/pr-risk.yml jobs, never for conformance.yml's own job, so there is
no existing Bazel-lane relationship for this gate to extend.

Bumped the job's timeout-minutes 45 -> 100 (30m + 10m + 40m budgeted
sub-timeouts plus checkout/setup-go overhead) and updated the three policy
tests that pin this script/workflow byte-for-byte
(TestConformanceScriptUsesExplicitTimeoutBudgets,
TestConformanceScriptPropagatesGoTestFailures,
TestConformanceWorkflowHasOuterTimeoutBudget), re-included
internal/httpclient/** in both path filters (push and pull_request, kept
identical) since served_*_test.go files are the suite itself and would
otherwise be excluded by the shared !**_test.go negation, and documented the
new filter entry in ci_f7c_advisory_test.go's own-entries map.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-06 09:45:16 +00:00
Julian Knutsen
e72cd8b315
ci: merge-queue readiness for the required gates; cache test results (fresh nightly) (#7264)
* ci: make the required gates merge-queue ready

The default-branch ruleset requires "CI Gate / Required" and "PR Risk
Gate / Required". Prepare both for a GitHub merge queue (ALLGREEN,
squash), so enabling the ruleset's merge_queue rule is the only step left.

- pr.yml, pr-risk.yml: merge_group pinned to types [checks_requested].
- D2: bazel-coverage covers every merge_group run (same repository, CI
  secrets, actor github-merge-queue[bot], bazel.yml mode remote), so the
  retired legacy tiers stay retired in the queue and ci-gate requires
  their Bazel lanes to have run remotely and passed; a missing secret or
  the farm switch off is red, as on a same-repo PR.
- Diff bases on merge_group: the package gates diff
  merge_group.base_sha..head_sha (not HEAD^, which misses all but the
  last commit of a rebase-method entry), the .beads/issues.jsonl guard
  now runs on merge_group against base_sha (ci-gate no longer excuses
  its skip), and check-doc-flags passes BD_DOCS_DIFF_BASE so CLI-docs
  drift already on main cannot eject every queued PR.
- Test result reuse: on a merge_group run and on attempt 2+ of a PR run,
  the six always-execute Bazel lanes append --config=reuse-test-results
  (--cache_test_results=auto), so tests whose action key is unchanged
  reuse the farm's recorded PASS and only what the merge changed runs.
  Failures are never reused, eviction retries stay 0, and each PR's first
  attempt, push to main and nightly still execute every test.
- scripts/ci_merge_queue_test.go simulates merge_group through the shared
  evalGHExpr evaluator (now with true/false/null literals, hyphenated
  identifiers and case-insensitive ==) and the workflows' own scripts:
  triggers and gate names, concurrency groups, every pull_request field
  read, the coverage and rbe decisions, result reuse, the package-gate,
  .beads and doc-drift diff bases.
- engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md: new "Merge Queue" section (how
  to queue, settings, what runs, result reuse rationale, ejection and
  flakes, rollback).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci: cache Bazel test results except nightly; queue trust and sizing

Review follow-ups to the merge-queue readiness change.

- Test results (S1): --nocache_test_results also stops Bazel writing
  results, so a PR's first run never fed the cache and the merge-queue
  reuse config could only hit on later reuse runs. Drop it from
  test:embedded, test:doltserver-proxied, test:doltserver-integration and
  test:sole-run (which keeps --experimental_remote_cache_eviction_retries=0),
  and drop test:reuse-test-results and BAZEL_REUSE_RESULTS. Results are now
  cached (read and write) on PRs, merge groups and pushes to main.
  nightly.yml passes the new fresh-test-results input, so every `bazel
  test` appends --config=fresh (--nocache_test_results) and the suite
  re-executes nightly. The docker lane keeps --nocache_test_results.
  Measured locally with a disk cache: a default run writes and the next
  reads ("(cached) PASSED"); a fresh run neither reads nor writes.
- Security headers (S2): bazel.yml now states that a merge_group run
  executes the queued commit's workflows, so a queued fork or Dependabot
  PR's CI edits run with every CI secret, the pre-warm app key included
  (merging-level trust), and records the maintainer rule: review fork
  changes to .github/, .bazelrc, tools/bazel, MODULE.bazel, BUILD files
  and CI scripts before queueing. Same note in the topology doc.
- Queue sizing (S3): recommend max_entries_to_build 3 and
  check_response_timeout_minutes 60 (max_entries_to_merge 5, ALLGREEN,
  SQUASH unchanged).
- Nits: the .beads guard fails closed on a missing diff base, and its
  test runs under GitHub's default `bash -e`; evalGHExpr follows GitHub's
  loose equality (true == 'true' is false, false == '' is true, missing
  context keys are null) with a typed-context variant; the
  detect-package-gates comment notes the SQUASH/HEAD^ equivalence and
  path-filter scope; the doc covers re-run reuse, flake re-sampling via
  dispatch, stale queue runs and how to confirm reuse on the farm.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 13:54:15 -07:00
Julian Knutsen
4493eb8d23
ci(pr): run same-repo macOS preflight/doc-freshness legs on Blacksmith macOS (#7231)
pr-preflight-platforms and check-doc-freshness-platforms now resolve their
same-repo-macos marker like the Linux/Windows legs: blacksmith-6vcpu-macos-26
for same-repo PRs and merge_group, GitHub-hosted macos-latest for forks,
Dependabot and every other event. The label is pinned to macos-26 because
GitHub's macos-latest is macos-26-arm64 today, so both paths run the same
OS/arch; 6 vCPU is already 2x macos-latest's 3 vCPU.

Blacksmith cannot see GitHub-saved caches, so main.yml gains
blacksmith-macos-go-build-cache: a push-to-main-only saver on the same label
(same job guard as the other Blacksmith savers) that builds ./cmd/bd and runs
the shared warm-non-race-cache.sh, saving the module cache and a non-race
GOCACHE keyed by go.sum + UTC day. The PR legs stay restore-only. main.yml's
GitHub-hosted macOS full-suite leg is unchanged and keeps seeding the fork
path.

Policy tests: the macOS marker arm is evaluated for same-repo/merge_group/
fork/deleted-head/dependabot/push/schedule and must match the flat macOS
const; the saver joins the PR-guard and per-day-key sweeps and the cache
ownership inventory; TestBlacksmithMacOSSaverMatchesPRLegs pins the
label/path/key/warm-up contract between saver and PR legs.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 13:19:49 -07:00
Julian Knutsen
0773e6334e
docs(agents): contributor-only agent instructions and documented-issue intake (#7223)
Makes the repo's agent and contributor surface describe developing beads, not using bd, and requires every PR to link a documented issue.

- Contributor-only root AGENTS.md with a routing table; cmd/bd/AGENTS.md and engdocs/AGENTS.md seam files; CLAUDE.md symlinks so Claude Code loads nested files.
- Bug and feature issue forms, a PR template that requires Closes #, CONTRIBUTING sections for the triage and pipeline labels, and the opt-in fork-cache during dev setup.
- ROADMAP.md for 1.4: versioned beads (#6132), Graph Preview 2 (#7170), contributor onboarding (#7228).
- docsync guards: root AGENTS.md at or under 20 KB; every AGENTS.md symlinked, routed, and with resolving links, paths, and guard-test names.

Agent-Signature: claude-code-claude-opus-5-5-high on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 08:43:53 +00:00
Julian Knutsen
ec72c8c563
ci(bazel): drop //cmd/bd:bd_test from the integration lane where the cmd/bd Dolt-server tier is required (#7230)
bazel-integration (--config=integration) and bazel-cmd-dolt
(--config=doltserver-cmd, //cmd/bd:bd_dolt_server_test) both run the
integration-build cmd/bd bd_test; the second without BEADS_TEST_SKIP=dolt,
so it is a superset. Once pr.yml requires bazel-cmd-dolt, the integration
lane's bd_test run is a duplicate.

bazel.yml gains a boolean workflow_call input, cmd-dolt-required (default
false). When true, bazel-integration tests //... -//cmd/bd:bd_test. pr.yml
passes it, and a policy test pins it to pr.yml's BAZEL_CMD_DOLT_REQUIRED
(a `with:` cannot read env), so the drop only happens where ci-gate
requires BAZEL_CMD_DOLT. bazel-farm.yml, nightly.yml, push and dispatch
leave it unset and keep bd_test. While the flag is "false" (current main)
nothing changes; with #7226's flip the input flips with it.

Coverage: both lanes share the same `if:` (modes remote, fork-ro, fork-rw,
cache; Dependabot included) and bazel-gate.sh accepts their skips in the
same modes (skip, local). TestBazelIntegrationExcludesBdTestOnlyWhereCmdDoltCovers
pins that, the exact exclusion, the flag/input equality, the gate's
conditional requirement, and that no other caller passes the input.

Rollback: commit BAZEL_CMD_DOLT_REQUIRED "false" and cmd-dolt-required:
false together in pr.yml.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 01:19:47 -07:00
Julian Knutsen
15d74c08ff
ci: require the cmd/bd Dolt-server tier (BAZEL_CMD_DOLT_REQUIRED=true) (#7226)
Advisory since #7185. On the farm it passed on every PR run based on
current main; its only failures (3) were branches predating #7219 and
reproduced a real test bug that #7219 fixed. Flip the committed flag so
CI Gate / Required enforces BAZEL_CMD_DOLT. Rollback: set it back to
"false".

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 01:04:39 -07:00
Julian Knutsen
630eae9b0c
ci(pr): share restored GOCACHE with the native init credential step (#7227)
pr-preflight-platforms' "Exercise native init gateway credential shell"
step had no GOCACHE env, so it fell back to the runner's empty default
cache and cold-compiled all of ./cmd/bd's test binary on every PR, right
after the hook-timeout step had compiled the identical binary (same tags,
same platform-default CGO) into the restored non-race cache. Job logs:
213s of a ~350s macos-latest leg, ~117s on Windows; on Linux the rerun
with a shared cache is a relink only (~8s).

- pr.yml: point that step, and the three Windows-only Go steps, at the
  restored non-race GOCACHE.
- main.yml: the macOS test leg (the only macOS non-race seeder) now runs
  warm-non-race-cache.sh, so the PR macOS legs restore the test packages
  and test-only deps they compile, not just the non-test ./cmd/bd graph.
- Policy tests: every run step in pr-preflight-platforms and
  check-doc-freshness-platforms must use the restored GOCACHE, and the
  macOS seeder must warm before it saves.

No test selection, matrix, or gate wiring changes.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 01:01:23 -07:00
Julian Knutsen
516e0b4bf0
ci: autofix pushes with the gastownhall-autofix App token (no more approve-and-run) (#7215)
* ci(autofix): push bot fixes with a scoped GitHub App token

github-actions[bot] pushes left PR checks action_required instead of
retriggering, and the DOCS_AUTOFIX_TOKEN fallback secret was never
configured. bazel-autofix.yml and docs-autofix.yml now mint a short-lived,
repo-scoped (contents:write) installation token for the gastownhall-autofix
App via the `autofix` environment on same-repo PRs, falling back to the
default workflow token (and its manual re-run note) when the App token
isn't available; fork PRs are unaffected and still only get the apply-recipe
comment.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(ci): job environment parses as a name or a mapping; gofmt

ciWorkflowJob.Environment was a string, so every test that loads all
workflows failed on deploy-pages-redirect.yml's mapping form
(environment: {name, url}).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-04 21:27:06 -07:00
Rongjun GENG
3c65c99c29
ci: trigger pull_request CI on hotfix/** branches, not just release/** (#7148) (#7198)
pr.yml and pr-risk.yml (and the three advisory workflows that already
list release/** beside it -- conformance.yml, cross-version-smoke.yml,
regression.yml) trigger pull_request only on main and release/**. A PR
based on hotfix/1.3.1 (or any sibling hotfix/** line) matched neither
pattern, so it ran almost no CI: #7146, a backport to hotfix/1.3.1,
showed 17 checks -- 15 cross-version historical smokes and triage/label
jobs, 2 skipped -- no Lint, no unit shards, no embedded tier, no CI
Gate / Required, no PR Risk Gate / Required.

pr.yml's own comment states exactly why release/** is there:
release-prep and release-fix PRs are based on the release branch, not
main, and without it they run almost no CI before the tag build. A
hotfix line is in the identical position. Added hotfix/** alongside
release/** in all five files.

New TestPullRequestWorkflowsTriggerOnHotfixBranches pins hotfix/**
presence in each file's pull_request.branches list -- red before this
change (observed [main release/**] in all five), green after. The
existing TestPRRiskBazelCoverageJob equality check (pr.yml and
pr-risk.yml's pull_request triggers must match) still passes, since
both were updated identically. go vet/gofmt clean.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
2026-10-04 20:30:49 -07:00
Julian Knutsen
5e951a2ff3
perf(issueops): batch create/import without per-row round trips; on-demand graph checks (#7205)
* perf(storage): cut batch-create round trips (import, bulk create, apply-batch)

A multi-issue create (import, bulk create, CreateIssuesWithFullOptions) ran
~20 SQL statements per issue, and its dependency pass ran a recursive cycle
CTE per edge over the union of both dependency tables, which Dolt cannot
index — every recursion step rescanned the edge tables, so an import whose
edges form chains went superlinear (400 issues: 73 s of 82 s in that CTE).

issueops batch create, same transaction and same stored outcome:
- createBatchCache: one IN-list read of every explicit id's presence in both
  planes (replacing two COUNT probes per issue), one read of the batch's
  stored labels (each issue's new labels then land in one multi-row INSERT),
  created/label_added events buffered and written in one pass under the ids
  InsertDerivedEvent would mint (same digest, same lowest free ordinal), and
  runs of brand-new rows written with multi-row INSERTs (same ODKU clause)
  then finished in order.
- Dependency pass: wisp routing and target presence from batch reads; the
  cycle and hierarchy checks walk an in-memory copy of the edge tables,
  loaded once and kept in step with every edge the pass writes (a pair it
  already holds is re-read, since a clientFoundRows connection reports a
  matched duplicate as written).
- createBlockedRecomputeIDs seeds all created edges and expands once.

Shared paths:
- Blocked-state recompute splits ids once per call by whether they own a
  dependency row; edgeless ids (every freshly created issue) skip the two
  union UPDATEs for one plain unmark, run after the chunk's union statements
  on the first pass so the unions see what they saw before.
- appendChildrenInTx reads children per IN-list chunk, not per parent.
- ExecuteCreate hands its BatchContext to the batch body instead of having
  it re-read the same config (-6 statements per apply-batch create item).

Not changed: one version-history row per created issue (the version
contract, version_history.go), journal order and content, and the per-row
bodies, which stay the path for singular creates.

Equivalence: internal/storage/createbatchequiv runs an import scenario
(upserts, duplicate ids, wisps, generated ids, cycles, hierarchy conflicts,
dangling/external targets, stale is_blocked, comments, journal and
versioned history on) through the fast and per-row bodies on the embedded
and server engines and compares every written table and the skip reports.

Re-pinned TestLargeBatchApplyStatementCounts*: embedded 7009/14014/846 ->
6396/12790/793, server 7008/14014/846 -> 6396/12791/793 (breakdown in
pinnedEmbeddedStatementCounts).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(storage): batch-create review fixes — lazy edge graph, opt-in recompute probe, insert budget

Review of the batch-create round-trip change (S1-S3 and nits):

- The dependency pass's in-memory graph no longer loads both edge tables.
  It reads a node's outgoing edges through the issue_id index the first
  time a walk reaches it: the batch's own endpoints up front, then one
  IN-list read per BFS level. Memory and reads scale with the subgraph
  the batch's walks touch, not the edge tables, so a two-edge create on a
  large database no longer materialises every edge. A written edge whose
  source was never loaded loads that source (row included).
- The blocked-state no-edge probe runs only when a create asks for it or
  when a chunk holds at least 32 ids. Single-issue recomputes on close,
  reopen, update and dependency edits issue exactly the statements they
  did on main (the reopen sqlmock expectation is back to main's).
- Multi-row issue INSERTs are bounded by an estimated 16 MiB of bound
  text as well as 100 rows. When a multi-row statement fails, its rows are
  replayed one at a time, and the batch continues if every row lands
  instead of returning the original error.
- Nits: document the binary-collation assumption behind Go string
  equality; DisableCreateFastPathsForTest documents that it is
  process-global and non-parallel, and panics on an overlapping caller;
  drop the unreachable skip branch; explain why wisp targets need no
  presence read.

Re-pinned TestLargeBatchApplyStatementCounts* classic 40: 793 -> 786
(the +7 probes on dep-add recomputes are gone; net vs main is -60, all of
it ExecuteCreate's config reuse). 356/712 unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(storage): search the batch dependency graph from both ends

The lazy per-node load (previous commit) still walked forward from the
check's start, so on a large stored graph a single create's cycle check
read every node reachable from its target — about 800 statements per
create on a 2000-issue fixture. The cycle and hierarchy questions a batch
asks almost always end at a row the batch just created, whose only
incoming edges are the batch's own.

depGraph now answers each reachability question with a bidirectional
search: forward from the start along outgoing edges (issue_id index),
backward from the goal along incoming edges (the three target-column
indexes, read as one UNION ALL per table), always expanding the smaller
frontier and loading a level's nodes in that direction once. The batch's
endpoints are loaded in both directions up front, so a create's checks
usually finish with no further read (a chain import, too: each new
edge's goal has no incoming edge yet). Rows are keyed by (table, source,
target), so a row read from both ends counts once. An incoming read
ignores a row whose matching column is not its first set target column,
as the CTE's COALESCE does. A written pair that neither loaded end can
vouch for is re-read instead of assumed new.

On a 50k-issue / 100k-edge database, a create with two blocks edges
now runs 27 statements, against the full-scan or per-node walks before
(numbers in the review-fix summary).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(storage): hold batch create to golden digests from the pre-change code

The batch-create equivalence test compared the fast paths with the same
code's per-row bodies, so changes the fast-path switch does not gate —
the recompute seed merge, the IN-list children read, ExecuteCreate's
context reuse, the prepare/finish split — were checked only by older
contract tests. Its batch was also too small to cross a chunk boundary
and never set RejectStaleUpserts.

createbatchequiv now runs four scenarios, each through the fast and
per-row bodies, and checks both against a per-table row count and
SHA-256 recorded from ebe3b6bcb (before the fast paths existed;
CREATEBATCHEQUIV_GOLDEN_DIR re-records them):
- small: the original scenario;
- import458 and import458-reject-stale: the review's 458-issue batch
  (every chunk boundary, wisps and cross-plane edges, cycles, hierarchy
  conflicts, duplicate ids across deferred runs, generated and
  hierarchical ids, stale upserts rejected and accepted), plus new edges
  whose blocked-state fallout reaches stored rows outside the batch (a
  spawner's waiter, an upserted parent's existing child);
- apply: an apply-batch through ExecuteCreate, dep adds and an update.

The recorded digests match on the embedded and server engines (the
server test now sets the fixture's prefix, so generated ids agree).
Dropping the spawner-waiter seeding is invisible to the fast-vs-per-row
comparison and caught by the golden check.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(storage): large-graph create benchmark

batchbench measures creates against a database already holding a large
dependency graph (batchfixtures.LargeGraphInserts: N issues, ~2N
acyclic edges with wide reachable sets and a deep parent-child forest,
committed as a Dolt version): ExecuteCreate under an existing parent with
an existing blocker, a store create with two blocks edges, and an
apply-batch of creates each with a parent-child and a blocks dep_add.
It logs wall time, statements and Go allocation per shape on the
embedded and server engines. Opt in with BEADS_BENCH_LARGE_GRAPH=<issues>
(and BEADS_BENCH_LARGE_GRAPH_CREATES); it asserts nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(storage): read incoming dependency edges one target column at a time

The bidirectional graph search read a node's incoming edges with one
UNION ALL over the three target columns per table. Dolt plans that as a
scan: 0.7 s per read on a 100k-edge table, where each leg alone is a 1 ms
lookup on its own index (EXPLAIN shows IndexedTableAccess per leg). Issue
the three legs as separate reads into one row set; a row matched through
two columns still counts once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(storage): run the 458-issue equivalence scenarios outside the race lane

The 458-issue golden scenarios took the embedded tier's shard over its
20-minute limit under -race (the in-process engine's own instrumentation;
about 35 s each without it). They move to
TestCreateBatchFastPathsMatchPerRowLarge_Embedded, skipped under -race and
added to nightly.yml's non-race embedded step, following the precedent of
the 712-item statement-count and 1000-item wall-clock cases. The light
scenarios (small, apply) stay in TestCreateBatchFastPathsMatchPerRow_Embedded
on every lane. The server backend (built without -race in its full
suite) runs both: TestCreateBatchFastPathsMatchPerRow_Dolt and, as its own
top-level test so the shard manifest's hash places it on another shard
(about four minutes there), TestCreateBatchFastPathsMatchPerRowLarge_Dolt.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 13:56:12 -07:00
Julian Knutsen
fa3d50eb19
ci(bazel): zstd for the anonymous fork cache when it advertises it (rbe-west E4) (#7202)
* ci(bazel): zstd cache transfers for the anonymous fork cache only, behind RBE_CACHE_ZSTD (E4)

E4 of the rbe-west egress reduction (design A.5), reworked for the
read-only zstd server shape. rbe-west's trusted MAIN and OSS schedulers
(:443 to :50052) and rbe-fork (:8444) advertise no compressor, and Bazel
refuses a remote that does not advertise zstd. So only the anonymous fork
cache (rbe-cache :8443, nativelink-anon, which keeps zstd) may ask for it.

- **`write-bazelrc.sh`:** no `--remote_cache_compression` on the trusted
  remote-exec or rbe-fork paths. In mode cache, with `RBE_CACHE_ZSTD=1`, it
  adds `build:fork-cache --remote_cache_compression`. Empty or 0 adds
  nothing; another value warns and stays off, rather than failing every
  cache lane.
- **`bazel.yml`:** every setup-bazel caller passes
  `RBE_CACHE_ZSTD: ${{ needs.rbe.outputs.mode == 'cache' &&
  vars.RBE_CACHE_ZSTD || '' }}`, so only cache runs see the variable.
  Merging changes nothing, and rollback is the variable, with no revert.
- **`.bazelrc`:** fork-cache no longer sets the flag; a comment points at
  the gate.

Tests:
- `TestBazelCacheModeReachesTheRC` evaluates `vars.` right-hand sides and
  simulates `RBE_CACHE_ZSTD=1`. Mode cache's rc carries exactly the
  fork-cache zstd line; the remote, fork-ro, fork-rw and local modes carry
  no compression at all.
- `TestSetupBazelRCWriter` runs `RBE_CACHE_ZSTD` = 1, 0, empty and a bad
  value in mode cache, and remote-exec with 1 (never compressed).
  `TestSetupBazelRCWriterForkMode` pins rbe-fork's exact lines with
  `RBE_CACHE_ZSTD=1`.
- The bazel.yml setup-env pin includes `RBE_CACHE_ZSTD`.
- `checkBazelrcForkCache` rejects the flag in any `.bazelrc` config, with
  fork-cache and remote-exec fixtures.
- New `TestBazelRemoteCacheCompressionOnlyForkCache` scans `.bazelrc`, the
  workflows and the local action files. The flag must appear once: the
  writer's fork-cache echo.

**Merge order:** after rbe-west's anonymous process advertises zstd. Even
then the flag stays off until `RBE_CACHE_ZSTD=1` is set. If rbe-cache stops
advertising zstd, fork-cache's fallback flags make lanes build locally.

**Action keys:** unchanged; the flag only changes ByteStream resource
names (`compressed-blobs/zstd/`).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): zstd for the fork cache by a GetCapabilities probe, not a repository variable (E4)

Review finding 3: GitHub gives fork pull_request runs no repository
variables (bazel.yml: forks "see no secrets and no vars"), so
vars.RBE_CACHE_ZSTD was always empty exactly where the switch matters. Fork
run 37220594372 logs RBE_VAR_ON: false. The variable could never turn zstd
on for fork-cache clients, and once on it could not follow rbe-cache
rolling zstd back.

- New setup-bazel/cache-zstd-probe.sh, a byte copy of gascity's
  tools/rbe/cache-zstd-probe.sh: one unary REAPI GetCapabilities call
  (instance oss) to rbe-cache :8443 over curl's HTTP/2, decoded with
  python3's standard library. It exits 0 only when
  cache_capabilities.supported_compressors lists ZSTD, the field Bazel
  checks. Any other result exits nonzero: not listed, an HTTP or gRPC
  error, a refused connection, a timeout (connect 3s, total 5s), an
  unparsable answer, or no curl or python3. It writes nothing to stdout
  and one verdict line to stderr. No new action and no new dependency.
- write-bazelrc.sh: in mode cache, `if bash .../cache-zstd-probe.sh` adds
  build:fork-cache --remote_cache_compression. The probe never fails the
  script. RBE_CACHE_ZSTD and its bad-value warning are gone.
- bazel.yml: the ten RBE_CACHE_ZSTD env lines are gone (back to the
  pre-E4 setup env).

Tests:
- New TestCacheZstdProbe runs the real script against a TLS HTTP/2
  stand-in for rbe-cache, which checks the exact request. Passes: zstd
  advertised (packed, and unpacked among others). Fails: rbe-cache's real
  2026-10-04 answer (no compressors), deflate only, zstd for batch updates
  only, zstd outside cache_capabilities, a gRPC error trailer, no
  grpc-status, trailers-only UNIMPLEMENTED, HTTP 502, a compressed or
  truncated message, a truncated field, not gRPC, a timeout, a refused
  connection, an untrusted certificate. It also pins the probe's endpoint
  and instance to .bazelrc's fork-cache.
- TestSetupBazelRCWriter: the fork cache gets exactly the zstd line after
  --config=fork-cache against a zstd stand-in, and none against the live
  answer or a refused port. Remote-exec and rbe-fork, with a zstd
  stand-in, never probe and never carry compression.
- TestBazelCacheModeReachesTheRC again rejects any vars. in setup-bazel's
  env, and no mode's rc carries compression (the probe is refused there).
- The writer harnesses default RBE_CACHE_PROBE_URL to a refused loopback
  port, so no test reaches rbe-cache.

Live check, 2026-10-04: rbe-cache answers without supported_compressors,
so the writer's mode cache rc has no flag until the E1 cutover.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 12:26:54 -07:00
Julian Knutsen
ebe3b6bcb0
test,ci(cmd/bd): run the Dolt-gated cmd/bd suite (advisory Bazel lane), fix in-process runner leaks and rotted tests; init --quiet stays quiet (#7185)
* test(cmd/bd): reset cobra flags between in-process runs; keep them out of parallel tests

runBDInProcess and runBDInProcessAllowError execute the package-level
rootCmd tree again and again in one process, and cobra never resets a
flag between Execute calls: a value one invocation parsed (create's
--title, --labels, list's --status, ...) and its Changed bit stayed on the
FlagSet, so the next invocation saw it, and a slice flag that had been
set appended to the stale value instead of replacing it. With a Dolt test
server (the only setup in which these tests run) that failed about 20
TestCLI_* tests depending on suite order.

TestMain now snapshots every flag of the tree (snapshotCommandFlags,
after all init()s registered them, before any test runs) and both runners
put the tree back before and after each run (resetCommandFlags): the
pointee of each flag.Value is restored (for pflag's scalar types that is
the bound variable; for slice types it also clears the internal "changed"
bit, so the next Set replaces), slice contents via SliceValue.Replace,
and Changed=false. TestResetCommandFlagsRestoresEveryFlag sets every flag
of the real tree and checks the reset. The ad-hoc --title reset in
TestCLI_CreateRejectsEmptyTitle_ProxiedServerMode is no longer needed.

Seven TestCLI_Comment* tests called t.Parallel() around the runners.
inProcessMutex serializes the runners only against each other, so their
chdir, stdio swap, env and config re-initialization raced every other
parallel test (data races on config globals; TestCobraParallelPolicyGuard
failing to read its own source files after a concurrent chdir). They run
serially now, and the guard treats the two runners like rootCmd.Execute.

Also two stale TestCLI_Create* cases: DryRunWithRigPrefix exercised a
create --prefix flag that no longer exists, and the flag-like-title cases
passed "--help" or "--foo-bar" where cobra parses them as flags; they now
pass the title after "--", the only way it reaches create's own guard.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): fix the ADO round-trip and bulk-delete tests that rotted unseen

Both need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd (BEADS_TEST_SKIP=dolt everywhere), so they failed unnoticed:

- TestADORoundTrip* stored ado.pat in the database. ado.pat is a secret
  read only from config.yaml or AZURE_DEVOPS_PAT (config.IsYamlOnlyKey),
  so the tracker found no PAT. The fixture sets AZURE_DEVOPS_PAT.
- TestBulkDeleteNoResurrection expected GetIssue to return (nil, nil) for
  a deleted issue; the Dolt store reports storage.ErrNotFound.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): fix parity, completion and notion tests that only fail with a Dolt server

These are Dolt-gated (integration tag + test server), so no CI lane ran
them and they rotted:

- TestParityCreate*/TestParityReopenErrorExits1Silently: bd create
  auto-routes via routing.DetectUserRole("."), which warns "beads.role
  not configured (GH#2950)" on stderr when the working directory's git
  config has no beads.role. The suite pins the configured case, so the
  parity env now sets beads.role=maintainer in a test-owned global git
  config ($XDG_CONFIG_HOME/git/config; the role lookup scrubs every
  GIT_CONFIG_* variable, so those cannot be used).
- TestIssueIDCompletion: issueIDCompletion resolves a database path
  before looking at the open store and returns nothing when there is
  none; the test now sets dbPath to its store's path.
- TestIssueIDCompletion_UsesWorktreeFallbackWhenStoreNil and
  TestGetNotionConfigReadsDBPathWhenStoreUnset: the code under test
  reopens the store from metadata.json, which names a database but not
  the per-test branch of the shared database that newTestStore uses, so
  the reopened store saw none of the test's rows. They now use their own
  database (newTestStoreIsolatedDB).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): fix stale doctor/e2e/autocommit/readonly Dolt-gated tests

These need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd, so they rotted unseen:

- TestE2E_DoctorFixMetadataRoundtrip, TestE2E_MigrateDoltMetadata,
  TestDoctorCheckHealthReportsVersionMismatchOnRepoLocalPort: bd init now
  defaults to embedded mode, where `bd sql` and `bd dolt start` are
  refused. Init with --server on a repo-local server of their own
  (repoLocalServerEnv: the suite's shared-server port vars cleared, server
  stopped at cleanup) and start it explicitly, since BEADS_TEST_MODE=1 never
  auto-starts one. MigrateDoltMetadata's DELETE also needed `key` quoted.
- TestDoctor{Conventions,Pollution}MaxRows_EnvOnly_Exits2: bdRunRaw drops
  every BEADS_* variable, so the subprocess hit dolt.New's test-database
  firewall (exit 1) before the row cap; pass BEADS_TEST_SERVER=1.
- TestDoltAutoCommit_On_WritesAdvanceHead: the embedded engine keeps its
  data under .beads/embeddeddolt, not .beads/dolt; findDoltRepoDir looks
  in both.
- TestValidateCheck_AllClean: bd doctor validate runs five checks since
  the cross-table duplicates check was added.
- TestConfigValidateReadOnlyIsHermetic: readonly commands take a shared
  workspace gate, whose lock file (.beads/dolt.gate.lock) workspacegate
  creates on first use by design; compare the .beads tree without gate
  lock files.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): undo each in-process execution's env and storage-mode leaks

Every in-process run of the bd command tree (runBDInProcess and the many
tests that call rootCmd.Execute directly) leaves process state behind:
PersistentPreRun exports the selected workspace as BEADS_DIR, loads its
.beads/.env, and records the workspace's mode in serverMode /
proxiedServerMode. With a Dolt test server, later tests in the same shard
then resolved a deleted fixture ("no active beads workspace found",
"migration sidecar is missing"), ran subprocess bd against the wrong
workspace ("recorded as proxied-server in .beads/metadata.json"), or took
the proxied path in RunE-level tests ("proxied-server UOW provider not
initialized", a nil-UOW panic in reopenProxiedResolve that killed a shard).

TestMain now registers a cobra initializer/finalizer pair
(installExecuteIsolation) that snapshots the environment and the two mode
globals when an execution starts and restores them when it ends; the
runners' restoreEnvSnapshot shares the helper. TestDoltConfigEnvironmentOverrides
used os.Setenv + defer os.Unsetenv, which deleted the suite's
BEADS_DOLT_SERVER_PORT (the test server's port) for every later test; it
uses t.Setenv now.

Full integration-tagged cmd/bd suite against a hermetic dolt (16 shards):
18 order-dependent failures and the shard panic gone; the 27 remaining
failures all fail alone too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(init): keep --quiet quiet through the Claude/Codex/Cursor auto-setup

`bd init --quiet` ("Suppress output") still printed the project
integration installers' progress to stdout ("Installing Claude hooks for
this project...", "Claude Code integration installed", ...): the
installers write to os.Stdout unconditionally. Point stdout at the null
device around them while quiet; their failures keep going to stderr as
before. Found by TestInitCommand's quiet case, which asserts no output and
had not run in CI (Dolt-gated).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(init): bring the Dolt-gated init tests up to the current init contract

These tests need a Dolt test server, so no CI lane ran them and they
drifted from the product:

- Embedded is init's default mode, so the server-mode assertions
  (.beads/dolt, opening the database on the test server, dolt_mode
  "server" in metadata.json) now pass --server (TestInitCommand,
  TestInitAlreadyInitialized, TestInitRedirect, TestInitBEADS_DIR,
  TestInitDoltMetadata, TestInitDoltMetadataNoGit, TestInitDatabaseFlag's
  metadata subtests). The in-process ones reset rootCmd's flags around
  themselves so --server cannot leak into later in-process inits.
- Each server init gets its own --database: the suite shares one test
  server, and a "test" database another test created belongs to another
  project (init refuses: PROJECT IDENTITY MISMATCH).
- The marker tests seed a current-era version witness: the legacy-upgrade
  guard refuses a non-empty .beads/dolt without one.
- The redirect test provisions the canonical target with config.yaml: bd
  ignores a redirect to a directory with no project files (#4692).
- Shared-server inits get a free port of their own
  (sharedServerTestEnv): the inherited BEADS_DOLT_SERVER_PORT is the
  suite's test server, which a shared-server start refuses as another
  project's, and the default shared port is 3308, a developer's live
  server.
- --shared-server's config.yaml is now nested YAML (dolt: shared-server:
  true); parse it instead of matching the old flat line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): run the Dolt-gated tests that skipped on a bare dolt.New

setupTestDB (config, kv and Linear round-trip tests), setupGatedTestDB
(mol ready --gated) and the rename-prefix and bd sql tests opened stores
with dolt.New(&dolt.Config{Path: ...}): no server port, no
CreateIfMissing, so against a running test server they got "database ...
not found" and skipped as "Dolt server not available". They now use the
suite's newTestStore helpers (skip only when no test server runs), and
what that exposed is fixed test-side:

- TestCustomStatusConfig: custom_statuses is read back ORDER BY name;
  look statuses up by name / expect name order.
- TestLinearRoundTripCoreFields, TestLinearPullMilestonesCreatesEpicHierarchy:
  linear.api_key is a yaml-only secret (LINEAR_API_KEY), push now needs an
  explicit linear.state_map, and batch push uses TeamLabels and
  issueBatchCreate, which the mock server now answers.
- TestSqlCommand: bd sql refuses in the default embedded mode; set
  serverMode for the server-backed store, and stop discarding RunE errors.
- TestE2E_AutoStartedRepoLocalServerPersistsAcrossCommands skipped on
  this process's embedded mode although every bd it runs is a subprocess
  on a --server workspace; gate on the dolt binary instead and stop the
  auto-started server at cleanup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): no fixed "unused" port, no swallowed JSON, no leaked front-door server

Three more things the Dolt-backed cmd/bd run exposed:

- TestTestServerConnection, TestSharedServerCfgNilHonorsSharedServer and
  the two TestLegacyGuard* e2e tests pointed bd at 127.0.0.1:59999 as a
  port nothing listens on. Any process on the host may hold it; a
  listener that accepts and resets turns the expected "connection
  refused" / "Dolt server unreachable" into another error. They take a
  just-freed port from closedLoopbackPort (testutil.FindFreePort) now.
- TestDoltSetConfigJSONOutput and TestDoltSetConfigWithUpdateConfig
  created .beads 0755, so bd's permissions warning landed in the JSON
  under test and the tests skipped ("output not pure JSON") every run.
  The fixture uses 0700, and non-JSON output fails instead of skipping.
- TestMigrateDoltModeFrontDoor's closing show/list auto-start a
  repo-local server after the reverse migration; nothing stopped it, so
  the suite's leak check failed (when the opt-in
  BEADS_TEST_MIGRATION_FRONTDOOR suite runs). It stops it at cleanup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): advisory cmd/bd Dolt-server tier (bazel-cmd-dolt)

No CI lane ran cmd/bd's Dolt-gated tests (TestCLI_*, the init and
store-backed suites, the doctor/e2e tests): main.yml's integration shards,
nightly's Full Test Suite, and Bazel's --config=prcore/ci and
--config=integration all pass BEADS_TEST_SKIP=dolt, so cmd/bd's TestMain
never starts a test server and those tests skip; the proxied and embedded
tiers run only their manifests' tests. They rotted unseen.

//cmd/bd:bd_dolt_server_test (tag dolt-server-cmd) runs the whole
integration-tagged cmd/bd suite, the integration build's bd_test, against
a hermetic dolt sql-server: BEADS_TEST_DOLT_SERVER=local starts it from
the pinned dolt in the test wrapper's runfiles, and
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 fails the target instead of skipping
into a cached pass. No test selection: the Go binary shards itself over
every top-level test (32 shards), so each runs exactly once, and
check_testcases.py fails a shard that ran none. --config=doltserver-cmd
has exactly --config=integration's build flags, so the two lanes share
bd_test's compile; prcore/ci and integration exclude the tag.

bazel.yml's new bazel-cmd-dolt job runs it exactly where bazel-integration
runs (same if: remotely on the farm for same-repo PRs, fork-ro/fork-rw,
and locally with the read-only cache in mode cache for forks). It is
advisory: job-level continue-on-error keeps a failure out of the call's
aggregate (BAZEL), and pr.yml's ci-gate reads its result (BAZEL_CMD_DOLT)
but requires it only while the committed top-level flag
BAZEL_CMD_DOLT_REQUIRED is "true" (now "false"; flipping it is the one
change needed). bazel-gate.sh accepts its skip in modes skip and local.

Policy tests: a third lane category in TestBazelLaneIsGatedAlongsideLegacy
(bazelFlagGatedLanes: gate env wired, never statically required, added by
the run script exactly when the pinned flag is "true"), the gate
simulation over every mode with the flag off (advisory: nothing it
reports reddens the gate) and on (exactly a gated lane), the cache-mode
and execution-mode checks, the tag taxonomy and prcore/integration
exclusions, and TestBazelCmdDoltJob (job steps, timeouts, .bazelrc
config equal to integration's build with no dolt skip or test selection,
the target's dolt-server rules, and bd_test's env repeated).

Measured locally (all fixes in this series, 32 shards, 16 at a time on a
shared host): 3267 top-level tests, all shards pass, max shard 313s (one
5001-issue export test, 214s, sets it), mean 100s.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(init): hand the project installers a writer instead of swapping os.Stdout

The --quiet fix earlier in this series pointed the process-global
os.Stdout at the null device around the Claude/Codex/Cursor installers.
That works, but it mutates a global from inside a cobra Run: anything else
writing to stdout meanwhile (a goroutine, a future concurrent installer)
would vanish, and a concurrent read of os.Stdout is a data race.

The installers already write through env structs with stdout/stderr
fields. setup.ProjectInstallOutput carries the two writers, and
InstallClaudeProjectTo / InstallCodexProjectTo / InstallCursorProjectTo
use them (the old entry points keep the process streams). init passes
QuietProjectInstallOutput() under --quiet: progress discarded, error lines
still on stderr. The proxied-server init path (domain adapter
InstallClaudeProject now takes quiet) does the same, so its
"Installing Claude hooks..." is quiet too.

Consistent error policy under --quiet: errors still reach stderr. init's
own "Warning: failed to setup <installer>" lines were suppressed under
--quiet while the installers' "Error:" lines were not, and some installer
failures (Cursor rules) print nothing themselves; the warnings now print
whether or not quiet, like the installers' errors.

Coverage in required lanes: TestProjectInstallersReportToTheGivenOutput
(cmd/bd/setup) and TestEmbeddedInitA/basic, which now asserts that
`bd init --quiet` with agents enabled installs the Claude hooks and prints
none of the installers' progress (verified to fail with the quiet output
disabled).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(export): exercise the incremental-export fallback with a small threshold

TestTryIncrementalExport_ThresholdExceededFallsBack seeded
incrementalExportThreshold+1 = 5001 issues to cross the threshold. Batch
create is one transaction but still several statements per issue, so under
-race against the shared test server it took ~214 s, the longest test in
cmd/bd's Dolt-backed suite by far. The behaviour under test is only
"more changed issues than the threshold -> full export fallback".

incrementalExportThreshold becomes a var (same value), and the test, which
is not parallel, sets it to 20 and seeds 21 issues: 214 s -> 0.6 s.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cmd/bd): tighten the flag-reset self-test, guards and canaries

Review follow-ups:
- TestResetCommandFlagsRestoresEveryFlag no longer skips a flag it cannot
  set: a flag type without a sample value, a Set error, or a non-pointer
  Value (which resetCommandFlags could not restore) is an error, so a new
  flag type cannot go untested. resetCommandFlags documents that it also
  resets flag-bound package globals.
- TestCobraParallelPolicyGuard treats .ExecuteC/.ExecuteContext/
  .ExecuteContextC like .Execute (the per-execution isolation writes the
  mode globals; a parallel execution would race).
- TestConfigValidateReadOnlyIsHermetic ignores only the gate a readonly
  command takes (.beads/dolt.gate.lock and its .info sidecar), not any
  "*.gate.lock*" file, so a new unexpected gate still trips the canary.
- TestCLI_CreateRejectsFlagLikeTitles: a bare `bd create --foo-bar` must
  fail and create no issue (bd-2c0's intent), next to the "--" cases.
- TestCustomStatusConfig cites why statuses come back in name order: the
  normalized custom_statuses table (#2961, 8537d943b) has no ordinal and is
  read ORDER BY name, like custom_types; the test predates it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the cmd/bd Dolt-server tier in 16 shards

Each shard pays ~30-40 s of fixed cost (TestMain plus its own dolt
sql-server), so 32 shards spent roughly 40% of the lane's executor time on
setup and added 32 farm actions to every PR touching cmd/bd. With the
5001-issue export test cut to under a second, 16 shards measure (locally,
--nocache_test_results): max 186 s, mean 137 s, 2,187 executor-seconds
(vs ~2,800-3,200 at 32), all 3,267 top-level tests, every shard passing.
TestBazelCmdDoltJob pins the count; bazel.yml's timing comment is
restated (cache mode ~20-30 minutes).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 03:06:54 -07:00