* fix(scripts): update-nix-vendorhash no longer reports a build failure as success
scripts/update-nix-vendorhash.sh treated the absence of a "got:" hash
line as proof the vendorHash was already correct, so a build failure
unrelated to a hash mismatch (e.g. a Go toolchain too old for a
dependency) was misreported as success and the workflow went green on
a broken build (#7496). Fixed by capturing the intentional-failure
build's real exit status instead of discarding it with `|| true`, and
using that exit status (not a "hash mismatch" string grep) to tell a
genuine no-op apart from an unrelated build break.
The step-5 verification build had the same shape of bug: piping
through `tee` without pipefail meant the if-condition tested tee's
exit status, not the build's, so a failing verification build still
printed "Build successful!". Scoped `set -o pipefail` to just that one
line rather than the whole script, since the earlier hash-extraction
step legitimately relies on `grep -o` returning non-zero on no match.
Fixesgastownhall/beads#7498.
Added an independent `nix build .#default` step to
update-vendor-hash.yml so the workflow no longer trusts the script's
exit status alone, even if a future bug in the script reintroduces a
false-success path.
Fixesgastownhall/beads#7499.
Added TestPinnedDoltCLIAtLeastRecommendedMin, asserting the pinned
Dolt CLI version (scripts/ci/install-dolt.sh) is at least
doltversion.RecommendedMin, so the two policies can't silently drift
apart — RecommendedMin's own doc comment already claims this alignment
but nothing enforced it.
Fixesgastownhall/beads#7500.
Verified by reproducing both update-nix-vendorhash.sh failure modes
with a stub `nix` binary (a non-hash build failure, and a step-5
verification build failure) and confirming the intended successful
hash-update path still works end to end. go build/vet clean,
golangci-lint 0 issues, go test ./scripts/... passes except a
pre-existing unrelated failure (TestControlQueryScriptPreservesReadyProbeFailure,
tracked separately as beads#7293 and untouched by this change).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr
* chore: empty commit to retrigger CI -- retrying after a 3-hour quota recovery window
* chore: empty commit to retrigger CI -- isolated single retry after real-bug fixes landed on sibling PRs
* fix(scripts): add missing BUILD.bazel dep, sharpen vendorhash diagnostics
scripts_test now imports internal/doltversion (for
TestPinnedDoltCLIAtLeastRecommendedMin) but scripts/BUILD.bazel had no
matching dep row, which would fail the target's compile under
rules_go once Bazel actually runs against this branch (the Bazel /
jobs failing so far died in bazel.fork-cert, unrelated to this).
Also, per review: scope set -o pipefail to just the step-5 verify
block (turn it back off after, rather than leaving it on for the rest
of the script with a comment claiming it's scoped when it isn't), and
keep the old "hash mismatch" string check as a sub-case in the
ACTUAL_HASH-empty branch, so a hash mismatch whose got: line fails to
parse (e.g. a future Nix output-format change) is diagnosed
differently from a build that failed for an unrelated reason.
Re-verified all four failure/success paths with a stub nix binary
(non-hash failure, hash-mismatch-with-unparseable-got-line,
step-5-verify failure, and the genuine happy path). go build/vet
clean, golangci-lint 0 issues.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr
* fix(scripts): update-nix-vendorhash preserves operator backups, restores on any handled failure
The script backed up default.nix to a fixed name in the repo root,
default.nix.backup, before editing it -- but that is also the
conventional name an operator gives their own manual backup. Running
the script unconditionally overwrote (then later deleted) any such
file. A failed step-5 verification build also left the candidate hash
in default.nix with no backup at all to restore from, since the
backup had already been deleted right after step 4 applied the real
hash.
Moved the backup into a private per-run scratch directory
(mktemp -d, cleaned on every exit via an EXIT trap) instead of a
fixed repo-root name, so it can never collide with an operator's own
default.nix.backup. Copy with cp -p so a restore brings back the
exact original bytes AND mode, not just content. Kept the backup
around through step 5 instead of deleting it right after step 4, and
added a restore on the step-5 verification-failure path, which
previously left the unverified candidate hash in place.
Also: the "no hash mismatch, already correct" exit-0 path previously
inferred correctness purely from the fake-hash probe build's
unexpected success, without ever building the real, restored file.
It now runs a second build against the restored file and only exits 0
if that succeeds too, surfacing a diagnostic log if it doesn't.
Fixesgastownhall/beads#7529.
Verified with a stub nix binary across five scenarios: a pre-existing
operator default.nix.backup (distinct content + mode 600) survives
untouched in every one; a non-hash build failure restores the
original (content + mode 640); the already-correct path now runs two
builds and only succeeds if both do; the core bug repro (hash
extracted, step-5 verify fails) now restores the original instead of
leaving the unverified candidate hash with no backup; the genuine
happy path still updates the hash correctly. No leftover scratch
directories in any case. go build unaffected (shell-only change).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr
* fix(scripts): update-nix-vendorhash restores the original on an interrupted run
A SIGINT/SIGTERM during the probe build (the long step) reaches this
script and its nix child as one process group; the child dies, this
script exits through the EXIT trap with the placeholder hash still in
default.nix. The plain `rm -rf "$SCRATCH_DIR"` trap deleted the only
backup at that point, leaving no way back except `git checkout
default.nix` -- which also discards any uncommitted operator edits.
Worse than every handled failure case this file already restores
from.
The placeholder is never a valid end state, so the EXIT trap now
checks for it and restores first, before dropping the scratch
directory. Every handled exit path has already restored or verified
default.nix by the time it runs, so this is a no-op there -- it only
fires on an interrupted run.
Found by review on PR #7504 (beads#7529's own follow-up): reproduced
and confirmed with a stub nix that signals its own parent process
group, matching a real Ctrl-C mid-build. Confirmed the bug first
(plain trap: placeholder left in place, backup gone, matching the
report exactly), then confirmed the fix (original hash and mode
restored, operator backup untouched, no leftover scratch dir). Also
re-ran the full existing scenario matrix (non-hash failure, hash
mismatch with step-5 verify failure, genuine happy path) to confirm no
regression.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr
* fix(scripts): restore default.nix if vendorhash verification is interrupted
The EXIT trap restored default.nix only while it still held the placeholder
hash. A SIGINT/SIGTERM during the step-5 verification build therefore left
the unverified candidate hash in place and deleted the only backup with the
scratch dir. Track VERIFIED instead: until a build has verified default.nix
as it stands, any exit restores the original. On handled paths this is a
no-op, since they have already restored or verified the file.
Write the build logs under ${TMPDIR:-/tmp} with per-run names. The
verification log had a fixed /tmp name shared by every run and every user.
Add the regression tests the #7498/#7499/#7529 fixes lacked:
- TestUpdateNixVendorHashReportsTheRealBuildResult runs the script with a
stub nix over a non-hash build failure, an unparseable mismatch, a failed
verification, a half-parsed got: hash, a real update, and a probe that
accepts the placeholder (restored file builds, or not). Every failure
exits 1 with default.nix restored byte for byte and mode for mode, the
operator's own default.nix.backup untouched and the named logs kept.
Against main's script it reproduces each false success.
- TestUpdateNixVendorHashInterruptRestoresOriginal signals the script's
process group during the probe build and during verification.
- TestUpdateVendorHashWorkflowBuildsIndependently pins the workflow's
unconditional `nix build .#default` between the script and the
match-report and push steps.
The scenario matrix, operator-backup and file-mode fixture, and workflow
ordering check are adapted from BlueboxC's correction of this PR in their
fork (62e7718e), as described on #7498 and #7529.
Refs #7504, #7496, #7498, #7499, #7500, #7529
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-authored-by: BlueboxC <david@blueboxcomputers.es>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: BlueboxC <david@blueboxcomputers.es>
* test(bazel): shard http_served_test and corpus_served_test (embedded lane critical path)
Both served targets ran as one shard each (235-597s) and were the embedded
lane's longest action in most executing runs; every other embedded shard
ends by ~90-155s.
- http_served_test runs through go_test_pinned_shard.sh like
httpclient_served_test: the 19 TestE2E_* cases >= 3s are pinned onto 6
shards (pin_shards.py from three lane runs' test.xml plus one local run,
<= ~88 measured seconds each), the ~28 fast tests round-robin in a 7th.
- corpus_served_test: TestServedCorpus becomes TestServedCorpusPart1..4.
Shapes are dealt round robin in corpus order; each part starts its own
server and seeds its own pair of workspaces. The shapes are independent:
every check is served-vs-embedded parity after the same commands, each
write owns its target ids, and order preservation keeps each part's reads
ahead of its writes. TestCorpusPartsDealEveryShapeOnce checks the deal
(every shape once, in order, no empty part); the manifest pins part k to
shard k, with a 5th round-robin shard for the engine-free tests, and
scripts TestCorpusPartsArePinnedOnePerShard ties corpusParts, the
manifest and shard_count together. BEADS_CORPUS_REPORT now accumulates
every part the process ran.
- pinnedShardTargets and the retired-lane args pins list both targets;
go_test_sources_test gets the two packages' manifests as data.
Embedded goes from 172 to 182 test actions; the dolt-race/dolt-int union
budgets do not include the embedded tag and are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(corpus): seed the served and embedded sides concurrently in each part
Each TestServedCorpusPartN paid the server start and both seeds serially
(about 100-150s of a 190-220s shard in #7537's CI). The embedded side now
runs bd init and the seed on a goroutine while the test goroutine starts
the server and proxy and seeds the served side.
- prepareSide attaches one side (connect or init), writes its assets and
runs the seed in order, returning failures as errors (execBD, runBD
without the testing.T) so it is safe off the test goroutine; a failing
side cancels the other, and a deferred cancel+Wait keeps the goroutine
from outliving the test or its TempDirs.
- The sides share no directory, env or bd process; shapes, their order and
every assertion are unchanged. Each part logs its setup time.
- bazel.yml's embedded lane uploads the served targets' test.xml on every
run (bazel-embedded-served-testxml), so their pinned shards can be sized
from a sharded run of the lane.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): corpus in 3 parts; re-pin http_served_test from a sharded CI run
Measured on this PR's first embedded-lane run (test.xml from the new
bazel-embedded-served-testxml artifact):
- corpus_served_test with concurrent seeding: parts 1-4 ran 135/140/144/114s,
of which setup (part time minus its shapes) was 76/76/76/62s and the 127
shapes 242s together. The setup floor is still ~70s+, so corpusParts goes
from 4 to 3 (round robin; the measured split is 85/81/76s of shapes) and
the manifest and shard_count follow (4 shards: 3 parts + round-robin).
- http_served_test: #7537's pins were sized from single-shard runs, and the
sharded shards ran 73-184s against ~88s estimated. served_pinned_shards.txt
is regenerated with pin_shards.py from that sharded run's per-test
durations: 119/118/120/128/132/123 measured seconds per pinned shard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): pin http_served_test on the max over two sharded CI runs
The second run of the lane on this PR ran the same tests at 0.2-1x their
first-run times (executor load, not the tests), and its re-pinned shards
came out 36-118s against 118-132s estimated. pin_shards.py now takes each
test's maximum over both runs' test.xml, the same rule the other pinned
targets use: 123-135 measured seconds per pinned shard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(corpus): guard part wiring; seed metadata on e2e-t-ifa (review of #7537)
- testServedCorpus fails (before the engine gate, so in every lane) unless
the test's name is TestServedCorpusPart<part>: a TestServedCorpusPart3
calling testServedCorpus(t, 2) would run part 2 twice and silently drop
part 3's shapes. TestCorpusPartNameMustMatchItsPart covers the check and
that servedCorpusParts lists Part1..N in order. Mutation (Part3 -> 2)
fails //backend/http/corpus:corpus_test with "TestServedCorpusPart3 runs
corpus part 2; only TestServedCorpusPart2 may".
- e2e-t-ifa is seeded with metadata seeded=keep. The parts seed separately,
so the two refused (exit 13) conditional updates no longer always follow
gc-update-if-assignment's k=v; their verify show again holds that a
refusal keeps existing metadata. Mutation (served verify output's
seeded=keep rewritten) fails both stale shapes at
$[0].metadata.seeded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): back to 4 corpus parts (served long pole over test-seconds)
maintainer-city-c3's call on the measured trade-off. With concurrent
seeding, 4 parts ran 114-146s each (run A) and 3 parts ran 170-180s (run D).
3 parts save one ~75s setup, about 75 test-seconds per executing run, but
the longest part is the embedded lane's served long pole. That pole is on
the gate's critical path (forks' above all), and 4 parts cut it by ~34s.
corpusParts=4, TestServedCorpusPart4, manifest part 4 on shard 4,
shard_count=5. The part-name guard is unchanged and covers Part4.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): served-timings upload is telemetry; corpus part bounds and ctx
Review of #7538 (maintainer-city-c3):
- "Upload served test timings" gets continue-on-error: true and
timeout-minutes: 2, like the CI analytics upload. An artifact-service 5xx
or stall no longer fails the required embedded lane or holds it until the
job timeout. TestBazelWorkflowSecretsAndFailureSurface allows that
continue-on-error only on this exact step
(bazelServedTimingsContinueOnErrorPath, which also pins its name,
artifact, path and timeout; TestBazelEmbeddedServedTimingsUploadIsTelemetry).
- corpusPartNameErr also refuses a part outside 1..corpusParts, so a stray
TestServedCorpusPart5 fails instead of running no shapes. Two new cases.
- expand takes ctx: prepareSide's {rev:} lookups run under the seeding
context, so a cancel stops them as well.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): dolt-int gates the server-storage and cmd-dolt tiers in mode remote (item 5 A2)
In mode remote, folds bazel-server-storage (--config=doltserver-integration)
and bazel-cmd-dolt (--config=doltserver-cmd) into one bazel-dolt-int job
(--config=dolt-int), following the dolt-race pattern from #7514. The
members have different gating categories (server-storage plain-gated,
cmd-dolt flag-gated/advisory), so the union carries continue-on-error: true
and a bespoke --label-scoped Shard balance step. All other modes keep the
two lanes running unchanged. Action keys are byte-identical between the
member configs and the union (17 + 46 = 63 TestRunner actions), so cached
results carry over either direction and a revert is a safe rollback.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): dolt-int per-tier results, integration gating, key-parity wording (review)
Coordinator review of #7535 found three must-fixes: integration:"off" was
ignored in remote mode (the union's members already honor it); job-level
continue-on-error made a server-storage-only failure read success on
push/nightly/dispatch, where main stays red; and the PR's key-parity claim
overstated byte-identical actionKeys when only the remote action digest
(not the local actionKey) is unaffected by moving --test_env/--test_arg
onto the targets' own env/args.
Fixes:
- bazel-dolt-int's if now adds `&& inputs.integration != 'off'`, matching
the members (#7491's review); bazelLaneRunModes and
TestBazelDoltIntArmsExactlyOnePerMode cover the "off" case.
- Job-level continue-on-error removed; only the `bazel test` step carries
it, and a new always() "Classify tier results" step
(tools/bazel/classify_tier_results.py, with its own unit tests and a Go
wrapper) computes server_storage_result/cmd_dolt_result from the BEP and
decides job.status per-tier. Confirmed with bazel-gate.sh/ci-gate.sh on
synthetic empty-output inputs that a dead job still fails the gate.
- Corrected the local-actionKey-vs-remote-digest wording in .bazelrc,
bazel.yml's job comment and CI_REQUIRED_CHECK_TOPOLOGY.md.
- Should-fix: test step 23->25 / job 58 minutes with reasoning; --label
coverage added to shard_budget_test.py; stale lane comment fixed in
tests/regression/BUILD.bazel:46; TestBazelDoltIntSoleRunIsInTheUnion's
docstring corrected (members bake sole-run's line in directly, never via
--config=sole-run).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): fail-closed classify, per-tier check_testcases, pinned argv (review)
Final review round for PR #7535 (maintainer-city-c3, bee-ghosttrack,
julianknutsen):
- classify_tier_results.py now fails every tier shut (exit 1) on a
non-PASSED label outside every --tier list, so a newly tagged target
added without updating the step's --tier args cannot silently read as
success (must-fix 1). Its --tier/--check label sets are policy-tied to
the BUILD files' own tag sets by a new
TestBazelDoltIntTierLabelsMatchTheTags.
- A missing or non-integer --rc (a timed-out or cancelled test step)
takes the same fail-everything-shut branch instead of crashing argparse
with no outputs written (bee-ghosttrack's should-fix 2).
- check_testcases.py gained a repeatable --label flag, and bazel.yml's
shared "Every target and shard ran tests" step is split into check-ss
and check-cd, each continue-on-error and folded into its own tier via
classify's new --check NAME=OUTCOME, so a zero-testcase cmd-dolt shard
no longer reddens the job on push/nightly (must-fix 2).
- classify's run: argv is now pinned by exact equality
(bazelDoltIntClassifyRun), and every flag except --tier/--check is
--StoreOnce; a repeated --tier/--check name or an unrecognized flag is
now a hard parser error (must-fix 3).
- "Upload test logs" now also fires on steps.test.outcome == 'failure',
not just failure() (must-fix 4).
- shard_budget.py --label now fails closed (::error, exit 1) on a label
with no executed shard in the BEP at all, instead of silently reporting
"no sharded test target executed two or more shards" (should-fix 3,
carried across both review rounds).
- Updated engdocs/TESTING.md, CI_REQUIRED_CHECK_TOPOLOGY.md, .bazelrc and
scripts/bazel_dolt_int_test.go/ci_workflow_test.go comments for the
nits: stale "Advisory (continue-on-error: true)" wording, the
bazelUnionHasFlagGatedMember comment's stale claim about bazel-dolt-int,
"byte-identical" without specifying remote action digest vs. local
actionKey, and "is unset" vs. ci-gate.sh's actual "was skipped" branch
for an empty union output.
Verified: python3 -I on all four tools/bazel/*_test.py (classify 29,
check_testcases 7, shard_budget 19) green; GOTOOLCHAIN=go1.26.7 go test
./scripts/... green; bazel --nohome_rc test //scripts:scripts_test
--config=ci green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): shard_budget --label fail-closed only on a truly unknown label
The previous commit's fail-closed --label check compared against
read_shard_durations, which only records executed (non-cached) shards, so
a target whose every shard was a cache hit this run had zero entries and
was wrongly treated the same as a --label typo. This was not caught by
the new tests (none exercised an all-cached target) and broke this PR's
own dolt-int CI run: bazel-dolt-int's Shard balance step went red on
//internal/storage/dolt:dolt_server_full_test with "no executed shard for
it", even though that target was correctly, fully cache-hit.
Fix: a new read_tested_labels() reads every label with any testResult
event (cached or executed), and --label's fail-closed check now tests
against that set instead of read_shard_durations. A fully-cached target
still legitimately reports "nothing to judge" through evaluate(); only a
label absent from the BEP altogether is now an error. Added the missing
test row (every shard cached) that would have caught this before push.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(bazel): pin bazel-dolt-int's Upload test logs if-condition
A full 42-mutation re-run for PR #7535's second review round found that
nothing pinned this job's own "Upload test logs" if-condition (must-fix 4):
the other lanes' upload-logs steps are each pinned, but bazel-dolt-int's
(unlike dolt-race, a brand-new job) had no equivalent assertion, so a future
accidental revert to the old failure()-only condition would go uncaught.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): dolt-race gates the dolt-server and proxied tiers in mode remote (item 5 A1b)
In mode remote, bazel-dolt-race (`bazel test //... --config=dolt-race`, the
union of --config=doltserver and --config=doltserver-proxied) runs both tiers
in place of bazel-doltserver and bazel-proxied, and is no longer advisory:
its job.status is both the bazel-doltserver and the bazel-proxied
workflow_call output (union result first, member's otherwise). In the fork
modes, cache and local, the two member jobs run exactly as before: there one
client does both tiers' work without in-flight deduplication, so a union
would cost the sum of the tiers rather than the max. Exactly one arm runs in
any mode, so gate ids, pr.yml, ci-gate.sh and bazel-gate.sh are unchanged.
- bazel.yml: dolt-race if: mode == 'remote' (every event that reaches it);
no continue-on-error; result output and always() recorder; test step
timeout 25 -> 20 (job 55 >= step sum 46 + 5, < 60). Members' if: their old
rule && mode != 'remote'.
- scripts/bazel_dolt_race_test.go (was bazel_dolt_race_shadow_test.go):
union-of-members config, sole-run lines in the union, gating shape, step
and job timeouts, exactly one arm per mode x event with the call outputs
evaluated as GitHub does, and the union's 62 test actions within
--config=remote-exec's --jobs=64.
- Gate simulations evaluate the workflow_call output expressions, and a
gated lane's reporter is the lane or its union lane.
- aquery: all 62 TestRunner actions under --config=dolt-race are
byte-identical (key, argv, env, outputs) to --config=doltserver (with and
without sole-run) and --config=doltserver-proxied on origin/main.
- Docs: CI_REQUIRED_CHECK_TOPOLOGY.md "dolt-race Union Lane", TESTING.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): dolt-race test step 23 min so Bazel logs a 1200 s TIMEOUT first
The proxied tier caps each test at 1200 s; a 20-minute step let GitHub
kill a hung shard before Bazel wrote its TIMEOUT test.log and goroutine
dump. Step 23, job 56 (step sum + 5, under the merge queue's 60).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): say the dolt-race job timeout bounds the job, not the run
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci-policy): close review leftovers from #7485/#7486
Follow-ups from maintainer review on the host-tool exec/shell policy
tests and the bazel.yml integration gate:
- host_tool_shell_policy_test.go: recognize indented function headers
and `function name {` forms when finding a function's closing brace,
so scripts/test.sh's indented cleanup_shared_server() no longer
mis-attributes its trailing lines to the wrong function; classify
same-line `command -v gofmt && gofmt` (and `go`) as the guarded
presence-check idiom instead of flagging it.
- host_tool_exec_policy_test.go: make condPolarity reject mixed-polarity
&& / || compounds (one recognized side, one not) instead of guessing
at the recognized side's polarity, which was unsound in both
directions; add pinning tests documenting the exec-before-gate and
function-value-helper cases as known, safety-conservative limitations
rather than bugs. Fixing this exposed a real ungated fallback in
cmd/bd/scripttest_test.go (a `go build` that could still run under
Bazel on a prebuilt-binary basename mismatch); fixed by symlinking the
prebuilt binary under the expected name instead of rebuilding it.
- bazel.yml: bazel-server-storage now honors the `integration` input
like bazel-integration/bazel-cmd-dolt do, so an explicit "off" skips
it too (latent today: no caller passes "off"). Updated
ci_workflow_test.go's policy tests to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci-policy): close should-fix B (go command -v guard evasion) on #7491
shellCommandVGuard previously excused go the same as gofmt, but under an
exit-127 host-tool stub `command -v` finds the stub and the guarded exec
still runs, and on a worker genuinely lacking go the exec was silently
skipped -- the worker-dependent behavior this policy exists to ban. Drop
go from the guard entirely, require the guarded exec to sit directly after
the guard's own `&&` (no intervening `;`, `||`, `|`, or leading `!`), and
require an exact gofmt match so a same-prefixed sibling like `gofmt-foo`/
`go-junit-report` can't borrow the guard. Add every should-fix B shape as
a fixture that must still be flagged, plus the generalized gofmt-side
cases (leading `!`, exact-name, post-`;` exec).
Also: correct host_tool_exec_policy_test.go's doc claim that a
function-value helper call site is "not a soundness hole" -- mixed with
one gated direct call site, the invisible function-value call site's
ungated-ness is silently lost and the combined verdict flips to gated;
pinned as review M22. And recognize a function closed by `} >&2` as
ended, so abort-style safety computed inside it does not leak into the
rest of the file (no scoped script has this shape today, but a bare `}`
equality check would silently miss it the moment one does).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci: F5 S2 Variant A nogo ownership for the integration configuration group
No full Bazel lane validates the integration / doltserver-integration /
doltserver-cmd configurations anymore (all three now pass
--norun_validations). Instead bazel-integration gains a narrow "Nogo:
integration library files" step that keeps full nogo (vet + golangci-lint)
on the configuration's one non-test library package,
//internal/testutil/integration, via
`bazel build --config=integration --@rules_go//go/config:race --keep_going
--output_groups=nogo_fix -- //internal/testutil/integration:integration`.
This accepts a scoped coverage loss (f5-spec.md §3/§5 S2, Variant A): the 56
integration-tagged _test.go files (listed in
tools/nogo/unvalidated_integration_tests.txt, derived from source by the new
T4 test) no longer get a vet pass in any lane, since golangci-lint already
skips all _test.go files repo-wide (.golangci.yml's run.tests: false) and no
lane builds those test binaries with validations on. Verified directly with
real `bazel build` runs: a planted vet finding in the library package still
fails the new step; the same finding in an integration-tagged _test.go file
no longer fails anything; a planted finding in a plain race-configuration
library still fails bazel-test; and GoCompilePkg action keys for the
integration configuration are identical with and without
--norun_validations (it is a build-request option, not a configuration
flag, so no farm cache invalidation).
scripts/nogo_lint_policy_test.go: the nogoConfigurations table gains a
"integ" row with an ownerIsNogoOnlyStep flag and a dedicated
checkNogoOnlyStepRow, generalizing T1/T2/T5/T6 to cover an owner that is a
narrow build step rather than a full lane; adds T4
(TestNogoIntegrationStepCoversItsConfiguration), which derives the step's
expected target set from source via go/build's own tag matching instead of
a hardcoded list.
Also fixes four should-fix items from julianknutsen's review of #7482:
1. expandBazelrcConfig now follows `common:` lines and --config= tokens
anywhere on a line, not just a first-position `build:`/`test:` one.
2. TestNogoRaceOwnerBuildsEverything now expands every config the owner job
can reach (including shell-variable-gated ones like sole-run), not just
the statically-named "ci".
3. The package-gate check now compares full key-flag fingerprints instead
of substring-matching --@rules_go//go/config:race / --norun_validations.
4. tools/bazel/nogo_ownership.py now fails loudly (exit 2) when both the
owner's and non-owners' action-key sets are empty, instead of reporting
a vacuous pass.
Each fix has new mutation-test coverage in TestNogoPolicyRejectsMutations
(should-fixes 1-3) or nogo_ownership_test.py (should-fix 4).
engdocs/LINTING.md and engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md updated to
remove the stale "integration still validates in all three lanes" claim and
document the Variant A trade-off.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): make T4's source scan work under Bazel, not just plain go test
scripts:scripts_test only declares //:repo_other_files as data (every
tracked file but Go and Markdown source, by design: a Go-only change
should not re-run the CI/workflow policy tests). TestNogoIntegrationStep
CoversItsConfiguration's source scan needs every non-test .go file, so
under `bazel test //scripts:scripts_test` its runfiles tree had none and
the scan always came back empty.
Fixes:
- scripts/BUILD.bazel: add //:repo_go_srcs to scripts_test's data, with a
comment explaining why (this is the one test in the target that needs it).
- integrationOnlyLibraryPackages: use bazelPolicyRoot(t) + repoFiles(t, ...)
(the existing Bazel-safe, runfiles-aware file listing other policy tests
in this package already use) instead of sourceRepoRoot(t) +
filepath.WalkDir, which only sees what //scripts's own go_test target
declares.
Verified with real `bazel test --config=integration --@rules_go//go/config:race`
and `--config=ci --@rules_go//go/config:race` runs of //scripts:scripts_test
(both pass); CI caught the original gap (PR #7486's first push failed
Bazel / integration and Bazel / test with exactly this).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): address independent review + bee-ghosttrack findings on F5 S2 Variant A
Independent review of #7486, must-fixes 1-7: pipefail on the nogo-only
step's tee pipeline; exact source-scan equality check for
tools/nogo/unvalidated_integration_tests.txt; expandBazelrcConfig
follows common:-lines and non-first-position --config= tokens in both
the Go policy test and nogo_ownership.py; mirrored
nogo_ownership.py should-fix guarding against a non-owner action-key
query silently matching nothing.
Cost fix: split T4/T6's whole-repo .go source scan into its own
go_test target (//scripts:nogo_integration_scan_test) so a Go-only
change no longer re-runs all of scripts_test; extracted the shared
bazelInvocations and ciWorkflow helpers into helper-only files so
both targets can depend on them without violating the
one-target-per-test-file partition rule.
bee-ghosttrack review: checkNogoOnlyStepRow's own fingerprint now
excludes test:-sourced .bazelrc lines pulled in via --config=X
(buildOrCommonLines), since the nogo-only step is a `bazel build`
invocation that never reads a test: line; added mutation s2i
reproducing the gap this closes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(nogo): pin integration nogo-only step's target, fingerprint its lane, add behavioral test
Addresses the remaining should-fixes from the independent review of #7486:
1. TestNogoIntegrationStepScriptFailsWithoutMaskingBazelsExitCode
(scripts/nogo_integration_step_script_test.go) extracts the real
"Nogo: integration library files" step script from bazel.yml and runs it
under plain `bash -e` (GitHub's actual run: semantics, no pipefail added)
against a stub bazel, closing the gap a text-only pin can't: it catches
rc=0 before `exit "$rc"`, `|| :`/`|| true` on the tee pipeline, or a
missing/negated `set -o pipefail`, regardless of how the mutation is
spelled.
2. The nogo-only step's target set is now pinned by exact go_library
label, not just package: nogo_integration_scan_test.go derives the
label(s) from source + BUILD.bazel and nogo_lint_policy_test.go pins
them literally (its own go_test target has no //:repo_go_srcs). Both
reject a `-//` exclusion pattern outright.
3. checkNonOwnerCommandLineAddsNoKeyFlag is now also applied to the
integ row's owner's own full lane: under Variant A that lane validates
nothing, so it is effectively a non-owner for this check's purpose, and
must not override a key-affecting flag on its command line either.
4. Latent items: TestNoUnconditionalValidationOff now also scans
.github/actions/setup-bazel/write-bazelrc.sh for run_validations (every
line it emits lacks a :config suffix, so any occurrence would turn
validation off everywhere with no owner lane to catch it).
tools/bazel/nogo_ownership.py's docstring is corrected: it had said S2
was simply "out of scope," which is now stale since S2 has its own
validating lane (the nogo-only step, checked statically); the script's
dynamic aquery-based cross-check remains S1-only because S2's owner has
no non-owner test lane building the same configuration to compare
actions against.
Skipped (not cheap / not reachable):
- bazel-server-storage's `if:` not modeling an "integration" input: no such
input exists to model; the gate is RBE-enabled-only by design, and
resolving this requires understanding pr.yml's broader gating semantics,
not a textual fix.
- defaults.run.shell parsing: bazel.yml has no `defaults:` block today
(confirmed by grep), so there is nothing to parse; the gap is latent only
for a future change, not a live bug.
- race-off scan's future-wrong-fix edge case: the review itself
characterizes this as not reachable with today's file set.
New mutation cases s2j/s2k/s2l in TestNogoPolicyRejectsMutations cover
items 2 and 3. All mutation-verifications performed by hand-editing the
real files, confirming FAIL, and reverting via Edit (never git
stash/checkout).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): one nogo-validating lane per race configuration (F5 S1)
The race configuration (--@rules_go//go/config:race) is built by five
lanes that duplicate the same RunNogo/ValidateNogo work: bazel-test
(--config=ci), bazel-embedded, bazel-doltserver, bazel-proxied, and
the advisory bazel-dolt-race shadow, plus the package-mcp/package-npm
gates' explicit race build. --norun_validations is a build-request
option, not a configuration flag (proven here by aquery: GoCompilePkg/
GoLink/RunNogo/ValidateNogo action keys are byte-identical with and
without it), so passing it on every lane but bazel-test removes the
duplicate work without touching a single action key or dropping any
lint coverage: bazel-test's --config=ci already validates every file
any of the others compiles.
scripts/nogo_lint_policy_test.go replaces the old blanket
"--norun_validations never appears in .bazelrc" check with
TestNogoConfigurationsHaveOneValidatingLane (T1: one owner per
configuration fingerprint), TestNogoOwnersAreRequired (T2: the owner
is a required CI Gate check whose `if:` covers every non-owner's),
TestNogoRaceOwnerBuildsEverything (T3: the owner builds //... whole,
no exclusions) and TestNoUnconditionalValidationOff (T5: nothing else
may disable validation). tools/bazel/nogo_ownership.py (make
nogo-ownership) is a dynamic, advisory aquery cross-check of the same
property, run on demand rather than from inside a Bazel action.
Verified locally: aquery shows identical action keys for the affected
lanes with/without --norun_validations; a planted printf-format vet
finding in internal/idgen/hash.go failed the owner build and passed
the non-owner build (then reverted, never committed);
scripts/bazel_dolt_race_shadow_test.go and the full //scripts suite
stay green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): close S1 nogo-ownership review gaps (allowlist, owner pin, real aquery)
Addresses maintainer-city-c3's review of #7479:
1. T6 TestRunValidationsOnlyOnAllowlistedLines replaces the coverage gap
left when main's blanket run_validations ban was dropped: only the four
race-group non-owners' "test:<cfg> --norun_validations" lines in
.bazelrc, and the two package-gate bazel-build steps in bazel.yml, are
permitted. A later test:integration/doltserver-cmd --norun_validations
(out of T1's table, S2) now fails here instead of landing silently.
Verified by mutation: adding test:integration --norun_validations fails
the new test; reverting passes.
3. TestNogoRaceOwnerBuildsEverything (T3) now also pins that the race
owner's "bazel test //... --config=ci" step has no step-level if: or
continue-on-error, and that its job has none either. Verified by
mutation: adding if: ${{ always() }} to that step fails; reverting
passes.
2. tools/bazel/nogo_ownership.py's aquery comparison was querying the
identical mnemonic("RunNogo", //...) for the owner and every non-owner,
so `missing` could never be non-empty. It now derives each non-owner's
actual built target set the way --build_tests_only + --test_tag_filters
does: the deps closure of tests(//...) filtered to targets carrying one
of the lane's tags, read exactly from `query --output=xml` (not attr()'s
substring regex match, which would wrongly fold "dolt-server-proxied"
and "dolt-server-cmd" into a "dolt-server" filter). Confirmed against a
real (offline, --nohome_rc) bazel run: reports OK at head, and reports
100+ missing targets when the owner's query is artificially narrowed to
a subpackage -- it can now actually fail.
Also: refreshed a stale .bazelrc comment claiming nogo runs beside every
lane's compile, which S1 made untrue for race-group non-owners.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): mode cache reads the remote repo contents cache through rbe-cache
Mode cache (fork and Dependabot PRs while rbe-fork is closed, rbe=cache
dispatches) now reads Bazel's remote repo contents cache: the trees rrc-seed
writes on main pushes, served by rbe-cache's anonymous GetActionResult and
ByteStream.Read (Caddy :8443 -> nativelink-anon -> AC_OSS, read only).
setup-bazel's write-bazelrc.sh adds the trusted readers' two lines to its
fork-cache rc (passed with --bazelrc, so the startup line applies):
startup --experimental_remote_repo_contents_cache
common --loading_phase_threads=64
The committed .bazelrc stays untouched (design R2): the flag is a startup
option and cannot live in build:fork-cache, and developer and macOS
machines never read.
Gated on the new .github/actions/setup-bazel/cache-rrc-probe.sh (a copy of
gascity's tools/rbe/cache-rrc-probe.sh): one GetCapabilities to rbe-cache
(connect 3s, total 5s). Only an answer (gRPC 0, cache_capabilities present)
writes the lines; anything else writes none and logs a ::notice:: with the
reason, and the script never fails on it. Kill switch: fork_rrc_read=on in
that script; set it to off in a one-line PR to stop mode cache reading.
Measured (scratch origin/main, no credentials, 8 CPUs, cold output base,
`build --nobuild //...`): repository rules 665-672 -> 38-41 (the rest are
bd_releases/dolt_release and local or toolchain repos, not reproducible),
analysis 26.5-26.7s -> 12.1-14.8s, wall about 36s -> 25-32s.
Tests: TestSetupBazelRCWriter runs the writer against a stand-in rbe-cache
(answers, answers with zstd, PERMISSION_DENIED, refused): exactly the
reader's lines after --config=fork-cache and any zstd line only when it
answers, a notice otherwise; remote-exec and rbe-fork never get them nor
probe. TestBazelCacheModeReachesTheRC checks no simulated mode carries them
with the probe refused. TestCacheRRCProbe runs the real probe (errors,
malformed, timeout, refused, untrusted cert, kill switch off without a
request).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): mode cache rrc review fixes: reader opt-in, containment, AC probe
Scope. write-bazelrc.sh enabled the remote repo contents cache for every
mode cache job, bazel-release-cross included, whose configs
rrc-lane-commands.txt does not seed. setup-bazel now takes BAZEL_RRC_READ
("true" or empty, anything else fails): only with it does mode cache probe
rbe-cache and add the reader's lines. The 8 trusted reader jobs pass it,
and nothing else. TestBazelRRCOptInMirrorsTheTrustedReaders pins that set
to bazelRRCReadLanes; TestBazelRRCLaneCommandsCoverEveryReader now checks
every job that enables the flag either way (reader step or opt-in) is a
reading lane whose bazel configs are all seeded. TestSetupBazelRCWriter
covers a job without the opt-in (no probe, no lines) and bad values.
Runner. The 5 readers that run in mode cache (test, pure, integration,
doltserver, cmd-dolt) fall back to ubuntu-24.04 instead of ubuntu-latest:
one runs-on line each; mode remote stays Blacksmith, and the remote-only
readers and every other job are unchanged.
Visibility. write-bazelrc.sh writes one step-summary line in mode cache:
whether the job reads, and why (the probe's reason, with rbe-cache's
GetCapabilities TCP connect time, DNS excluded).
Probe. cache-rrc-probe.sh is re-copied from gascity: it also asks
GetActionResult for a fixed key no action has and a ByteStream Read of
that digest as a blob, and passes only if both answer NOT_FOUND (gRPC 5),
proving Caddy, nativelink-anon, the action cache and the CAS answer; it
reports the TCP connect time minus the DNS lookup. serveCapabilities
answers both; TestCacheRRCProbe adds their failure answers.
Containment and limits. The rrc-verify issue body and
CI_REQUIRED_CHECK_TOPOLOGY.md no longer say to set RBE_REPO_CONTENTS_CACHE
off (mode cache never sees it, and off also stops rrc-verify): commit
fork_rrc_read=off in both probe scripts (beads has no fresh-merge, so an
open fork PR keeps its old switch until it is pushed again); for immediate
server-side containment, rbe-cache-gate close on core; and, because the
mode remote readers read the same oss action cache entries, set the
variable to seed on a mismatch (no lane reads, rrc-seed and
rrc-verify keep running), off only to stop seeding too, and leave it as
it is when the run only could not check. The doc also says a
closed or down cache only falls back at probe time (after loading, files
are fetched lazily and losing rbe-cache fails the command, bazel#30218),
and notes the 512 MiB anonymous CAS limit under gascity's R4.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): bazel-dolt-race opts in to the rrc read like the other trusted readers
#7464 added bazel-dolt-race as a trusted reader after this branch was cut, so
TestBazelRRCOptInMirrorsTheTrustedReaders now covers it: pass BAZEL_RRC_READ
(it runs only in mode remote, so this changes nothing today). It never runs in
mode cache, so it keeps the lanes' ubuntu-latest fallback: the cache-reader
runner rule (ubuntu-24.04) excludes it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): add advisory dolt-race shadow lane (item 5 phase 1)
Add --config=dolt-race, the union of --config=doltserver and
--config=doltserver-proxied with one tag filter for both tiers, and an
advisory bazel-dolt-race job that runs it beside bazel-doltserver and
bazel-proxied. The shadow measures one client per group of lanes against
the two lanes it would replace before any cutover (item 5 slice A1a).
The job runs in mode remote on pull_request and merge_group events only.
It has job-level continue-on-error, no job or workflow_call output, and no
gate names it, so it cannot change CI Gate or PR Risk Gate. Its
ci-analytics step is copied verbatim with LANE=dolt-race, and rrc seeds
the new config.
Policy tests pin the config as the exact union of its members' lines (tag
filter aside), the job's run modes and events, its advisory shape, the
analytics lane map and the rrc command list. aquery shows every member
target's TestRunner action (actionKey, argv, env, inputs, outputs) is
unchanged under the union config.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): give the dolt-race shadow timeout headroom; fix gate-wait wording
Review follow-up on the advisory bazel-dolt-race job (maintainer-city-c3,
findings 1, 2 and 5):
- Raise the job's timeout-minutes from 30 to 35. continue-on-error covers
a failed or timed-out step, but not a job GitHub cancels at its own
timeout, which could make the bazel call non-success and turn CI Gate
red. TestBazelDoltRaceShadowIsAdvisory now reads both timeouts from the
workflow and requires job >= test step + 10 (and a step timeout at all).
- Correct the docs and comments: the shadow cannot fail the gate, but
CI Gate still waits for it, because its needs on the bazel call
completes only when every job in it has finished.
- Correct the checks comment: check_testcases covers every target in the
union, including the dolt-server tier its own lane never checked.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): bound every dolt-race shadow step; job timeout covers their sum
Review follow-up (S1): a hang in any bazel-dolt-race step without a
timeout would run into the job's own timeout, and a job GitHub cancels
there is not covered by continue-on-error. Every step now has a
timeout-minutes, and the job's timeout covers their sum:
- checkout 5, rrc read 1, setup-bazel 10, flaky query 5, checks 3 each,
test-log upload 3. The test step (25) and the analytics steps (3, 2)
keep bazel-proxied's values; bazel-proxied sets no timeout on the
others, so these bound them.
- The step timeouts sum to 60 minutes, so the job timeout goes from 35 to
65.
- TestBazelDoltRaceShadowIsAdvisory now requires a timeout on every step
and job >= sum(step timeouts) + 5, read from the workflow, alongside the
existing job >= test step + 10.
Also cite the farm maintainer's confirmation of NativeLink 1.7.1's
in-flight Execute dedupe in the job comment.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): keep dolt-race shadow timeout under the merge queue's
Tighten the shadow's non-test step budgets so the job timeout (step sum
+ 5) is 55 minutes, and pin that it stays below the 60-minute merge
queue check timeout: CI Gate waits for the shadow, so a hung shadow
must not drop a group from the queue.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Port gascity's remote repo contents cache wiring (gascity #7467, #7478) to
bazel.yml, gated on the repository variable RBE_REPO_CONTENTS_CACHE
(unset/off: nothing; seed: rrc-seed on push to main; canary: + bazel-test
reads; on: + every test lane reads). Mode remote only.
- rbe job: an rrc step turns the variable into the rrc output.
- Readers: the eight test lanes append startup
--experimental_remote_repo_contents_cache and common
--loading_phase_threads=64 to .bazelrc.local before Set up Bazel.
- rrc-seed (push to main, id-token: write): a 30-minute
rbe-rrc-writer-beads certificate from rbe-west's mint
(.github/scripts/rrc-writer-credential.sh), then every lane's command
(.github/scripts/rrc-lane-commands.txt) with --nobuild, --remote_cache
only and --remote_upload_local_results (.github/scripts/rrc-run-lanes.sh).
- rrc-verify (schedule/dispatch, issues: write): cold fetch of every lane
and tools/bazel/rrc_verify.py against the cache; a mismatch opens an
rrc-verify issue.
- pr.yml, nightly.yml and bazel-farm.yml grant id-token: write and
issues: write on their bazel.yml call (startup_failure otherwise);
TestBazelWorkflowCallersGrantEveryRequestedPermission guards it.
rrc-writer-credential.sh, rrc_verify.py and rrc_verify_test.py are byte
copies of gascity's, pinned by sha256.
Refs ga-vnycm2.29.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
//internal/storage/embeddeddolt:embeddeddolt_conformance_core_test ran
conformance.RunAll minus Audit as one unsharded action, 108-150s on
rbe-west, the embedded lane's longest after #7443. Every RunAll group builds
its own store from the factory, so the groups share no state and can run in
separate processes; about half the time was two groups,
ReadyCountsPageChunking (~33s) and Portable (~20s).
The new embeddeddolt_conformance_core_slow_test runs exactly those two
groups (-test.run=^TestConformance$/^(ReadyCountsPageChunking|Portable)$),
and the core target skips them along with Audit. The core target skips
only named groups, so any group added to or renamed in RunAll still runs
there: between them the two targets run every core group exactly once
(rbe-west: 57 + 2 groups, no overlap, the same 59 as before). No test
changed. The frozen-partition check now derives both targets' selectors
from the retired job's core selector plus conformanceCoreSlowGroups; the
retired-lane arg pins, bazel.yml's --whole list and the BUILD rules carry
the new target.
rbe-west, `bazel test //... --config=embedded --nocache_test_results`:
conformance core 148s -> 55s + 57s.
Bead: ga-vnycm2.31
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
//cmd/bd:bd_embedded_test was the embedded lane's floor: at Bazel's 50-shard
cap per rule, each shard's wall time was its slowest top-level test, and
~40 tests ran 65-130s on rbe-west.
- 34 slow TestEmbedded* functions are split into 89 smaller top-level
functions. Subtests and table cases are moved verbatim: no assertion,
flag or data changes, the same t.Run set per file, the same skip guard and
t.Parallel. Subtests that read state earlier ones left stay together, in
order; each new function repeats the shared setup it needs.
- The manifest's cmd/bd block is now 100 shards, run by two targets:
bd_embedded_test (shards 1-50) and the new bd_embedded_part2_test
(51-100), through go_test_manifest_shard.sh --shard-offset/--shard-total.
check_shard_coverage.py takes SHARDS@OFFSET/TOTAL and fails unless the
targets' ranges tile the block exactly once, so every test runs in exactly
one shard of one target; the policy tests read both rules' ranges from
cmd/bd/BUILD.bazel.
- Durations re-measured on rbe-west with the split tests and repacked. The
frozen legacy 20-shard block drops the renamed names (no workflow reads
it; new names hash-distribute as before).
Not split: the *Concurrent stress tests (their bd subprocesses queue on the
workspace's single writer lock: TestEmbeddedUpdateConcurrent is ~100
serialized bd commands), TestEmbeddedPurgeWispsPlaneRetention (one stateful
sequence), and the MaxRowsList* tests (setup is 21 sequential creates every
subtest reads).
rbe-west, both targets, --nocache_test_results: slowest shard 138-163s ->
109s (TestEmbeddedUpdateConcurrent alone; every other shard <=86s), median
80-84s -> 40s, shard-seconds 4.1-4.4k -> 4.2k, critical path 213-306s ->
113s.
Bead: ga-vnycm2.17
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Client-side loading and analysis is CPU-bound and gates every action in a
remote lane. Analysis-only measurements on Blacksmith (ga-vnycm2.8, seeded
runner cache, gazelle tools from the repository cache): bazel-test 25.6 s on
4 vCPU vs 16.3 s on 8; the other lanes 17-38 s on 2 vCPU vs 12-22 s on 4.
rbe-prewarm runs no Bazel client and stays on 2 vCPU (owner decision,
ga-vnycm2.12).
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): pin the slowest cmd/bd Dolt-server tests to their own shards
The cmd-dolt lane's wall time when its tests re-run is the bd_test race
compile plus its slowest shard. rules_go shards round-robin by test index,
so whichever shard drew TestSharedServerConcurrent and the TestInit*
family ran 117-170s while the median shard ran ~55s.
tools/bazel/go_test_pinned_shard.sh runs a go_test binary with the tests a
committed manifest pins (measured >= 5s, bin-packed longest-first by
tools/bazel/pin_shards.py) in shards of their own, and every other test in
the binary's own round-robin shards with the pinned set skipped, so each
test still runs exactly once and new tests need no manifest edit.
bd_dolt_server_test goes from 16 to 32 shards (16 pinned + 16 round-robin),
regression_test from 8 to 16 (10 + 6; its go_test becomes the manual
regression_suite binary behind an sh_test of the old name).
Pinning exposed a pre-existing order dependence: TestCLI_Import failed
whenever it was the binary's first test, because runBDInProcess reused the
workspace discovery an earlier in-process execution cached and `bd init` in
a fresh directory resolved the previous one's .beads. The in-process
runners now reset beads/git discovery caches per execution.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): pin cmd/bd Dolt-server shards from two measured runs; regression in 14 shards
The bd manifest now takes each test's slowest of the 16- and 32-shard runs
(pin_shards.py over both), and regression_test drops to 10 pinned + 4
round-robin shards: its unpinned tests total ~128s, so 6 round-robin
shards were mostly fixed cost.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(scripts): place the pinned-shard tests by what they read (#7414 split)
The manifest-name scan reads every cmd/bd and tests/regression _test.go
file, so it moves to go_test_sources_test (with tests/regression's
manifest); the wrapper, generator and wiring checks stay in scripts_test.
Shared helpers live in a helpers-only file both targets compile, and
repoFiles takes #7414's minimum-file count.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): split //scripts:scripts_test by what it reads; partition //:repo_files
//scripts:scripts_test declared //:repo_files (the whole checkout), so every
PR re-ran it, and it took ~181s on rbe-west. Most of that (60-79s) was
TestBazelGatedLanesNeverRetryFlakyTests line-regexping the --run_under
wrapper's pinned dolt binary (tools/bazel/hermetic_bin/dolt, in every
test's runfiles); it now skips binary files, as git grep -I does.
tools/bazel/go_srcs.py's per-package repo_files block now partitions each
package's files into repo_go_srcs, repo_go_test_srcs, repo_doc_files and
repo_other_files, aggregated at the root; //:repo_files is their union
(same 4679 files). The //scripts tests are split into go_test targets by
what they read (traced per test with strace over a runfiles-shaped tree):
scripts_test CI/Bazel policy: //:repo_other_files (+1 Go file)
shell_scripts_test script fixtures: //scripts files, a few root files
go_test_sources_test _test.go scans: //:repo_go_test_srcs + shard files
go_sources_test check-testing-short.sh: every .go file
tracked_files_test every tracked file: //:repo_files
Shared helpers move to helpers-only files; TestScriptsTestTargetsPartition-
TheTests fails on a test file in no target or in two. //test/docsync and
the //scripts/repochecks slices declare partitions or single files instead
of //:repo_files where they can.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): skip only the non-checkout hermetic_bin dir in the flaky-retry walk
Skipping every binary file also stopped scanning tracked binaries (images,
.binpb fixtures). Skip just tools/bazel/hermetic_bin, the --run_under
wrapper's runfiles directory, which is not in the checkout, so the walk's
coverage of tracked files is unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: bigger runners for the slowest CI legs (MCP gate, Bazel test, Windows preflight)
Runner-size A/B against merge group 37721729916 (3m49s):
- package-mcp: 4 -> 8 vCPU with pytest -n 16 (BEADS_MCP_PYTEST_WORKERS,
mode remote only; forks/ubuntu-latest keep package-mcp.sh's default 8).
pytest was 70s, dominated by CPU-bound per-test `bd init` setup. Local,
pinned with taskset: -n 8 on 4 CPUs 112s; -n 16 on 8 CPUs 60/57/58s,
228 passed / 5 skipped in all three runs.
- bazel-test: 2 -> 4 vCPU. `bazel test //... --config=ci` spent ~63 of
81s in client-side loading/analysis (critical path 7.65s, every action
a remote cache hit).
- pr-preflight-platforms Windows leg: 4 -> 8 vCPU. main.yml's test-windows
saver stays on 4 vCPU: the cache keys name runner.os/runner.arch, never
the label, and actions/cache's version covers only paths + compression,
so the entries are not size-specific; the other Windows consumers keep
their exact label. TestWindowsPreflightLegReachesSaverAcrossSizes pins it.
Runner labels live in the shared constants in
scripts/ci_blacksmith_runner_test.go; TestBlacksmithBazelRunnerSizes checks
every bazel.yml lane's runs-on and the MCP worker count per rbe mode with
evalGHExpr. Revert any bump that does not save at least 15s.
Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(pr): keep the Windows preflight leg on 4 vCPU (A/B: 8 vCPU queued 63s)
#7381's PR run put pr-preflight-platforms' Windows leg on
blacksmith-8vcpu-windows-2025. The cache restore from main.yml's 4 vCPU
test-windows saver hit, but the job waited 63s for an 8 vCPU Windows runner
and finished at 144s against the 129s baseline on 4 vCPU, so the leg goes
back to blacksmith-4vcpu-windows-2025. With saver and consumer on one label
again, TestWindowsPreflightLegReachesSaverAcrossSizes and the separate
preflight runner map have nothing left to pin and are dropped; the result
is recorded next to platformsMatrixRunners, in pr.yml and in
CI_REQUIRED_CHECK_TOPOLOGY.md.
Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): fetch the LLVM toolchain as a 149 MB slice; re-key the runner cache
Port of gascity #7279. @llvm_dist_linux_x86_64 downloaded the 1.94 GB
LLVM-22.1.8-Linux-X64.tar.xz and ran a single-threaded `tar -xJf` over
11 GiB to keep a 688 MB slice. That took 94-147 s in every Bazel lane,
and every cgo, race and stdlib action waits on it.
- llvm_dist.bzl gains `sliced`: download_and_extract of a members-only
.tar.zst, plus a missing-member check. The xz path stays.
- MODULE.bazel pins the toolchain-llvm-22.1.8-slice-1 asset by sha256
(de614777...6bec5). The URL is gascity's release asset for now; the
list form leaves room for a beads mirror of the same bytes.
- tools/cc_toolchain/repack_llvm.sh and llvm_members.txt rebuild the slice
from the pinned upstream release.
- Runner cache v4, keyed on MODULE.bazel too. MODULE.bazel.lock does not
record llvm_dist, so the v3 key never changed and kept restoring the
2.9 GB entry that holds the xz archive.
- bazel_hermetic_cc_test: a sliced llvm_dist may only download from a
gastownhall/beads or gastownhall/gascity release, and its members must
equal llvm_members.txt.
Verified locally:
- Every file in the extracted slice (2009 entries: types, modes,
symlink targets, sha256) is identical to the xz extraction.
- aquery action keys are identical, base vs head, at the same workspace
path: 7565/7565 over deps(//cmd/bd:bd_for_tests) and 7146/7146 over
the race deps(//internal/storage/embeddeddolt:embeddeddolt_test).
- Extraction time: 178 s (xz) vs 1.9 s (zstd).
Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the release cross-compile as its own lane
After the LLVM slice, bazel-pure is the longest cached Bazel lane (about
170 s). Its "Cross-compile every release target" step takes 54-58 s of
that (78.9k configured targets, 25k cache-hit lookups) and shares no
analysis with the pure invocations, because release_cross is a split
transition.
The step moves, unchanged, into a new job, bazel-release-cross ("release
cross-compile"). The job uses bazel-pure's runner, `needs: rbe` and skip
rule, so it runs in every mode except skip. It reports job.status like
every lane: the workflow_call output is bazel-release-cross, and pr.yml's
CI Gate requires it as BAZEL_RELEASE_CROSS. bazel-gate.sh skips it only in
mode skip, as it does BAZEL_PURE. Merge groups use the same pr.yml gate.
Push and dispatch runs gate nothing. Existing gate ids are unchanged.
The policy test now pins the step to bazel-release-cross, requires the new
gate id, and rejects any other job that runs the script (no double runs).
The new job is listed as having no CI analytics step. It captured no BEP as
a bazel-pure step either, and the analytics steps are unchanged. Docs:
LINTING.md, TESTING.md, CI_REQUIRED_CHECK_TOPOLOGY.md.
Expected: bazel-pure about 110 s on cached runs; the new lane about 85 s,
in parallel. Cost: one more 2-vCPU Blacksmith runner per run.
Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): stop building protoc for protoc-gen-validate
github.com/envoyproxy/protoc-gen-validate ships validate/validate.pb.go,
but its own validate/BUILD makes the Go library a go_proto_library. So
every lane built protoc from source just to regenerate that file. The
module reaches the graph only through grpc xds and the cloud SDKs.
A gazelle_override now regenerates the module's BUILD files with
`gazelle:proto disable`. It sets build_file_generation = "on" and adds
`gazelle:build_file_name BUILD.bazel`, so gazelle writes BUILD.bazel
(which Bazel prefers) instead of keeping the upstream BUILD. The proto
directive alone changes nothing, because the upstream BUILD files win.
validate is now a plain go_library over validate.pb.go.
Measured locally with aquery:
- //... minus the manual release_cross target: base has 2096 CppCompile,
612 CppArchive, 314 CppLink and 1 GoProtocGen action; head has 0.
- cquery somepath(//..., @protobuf//:protoc): base goes through
grpc xds -> envoy core/v3 -> validate_go_proto -> protoc; head is
empty.
- deps(//cmd/bd:bd_for_tests): 7565 actions become 3077. The race
deps(//internal/storage/embeddeddolt:embeddeddolt_test): 7146 become
2658.
MODULE.bazel.lock does not change. The v4 runner cache key includes
MODULE.bazel, so it still moves. Expect one cold compile of the Go
packages above validate on the first run after merge.
Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): bazel mod tidy formatting
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
PR preflight process (windows-latest) is a CI Gate input and, after the
Bazel fixed-cost work, the merge-group critical path (ends at +247..+303s).
Step timings from merge group 37710264475 and seven same-repo PR runs:
45-115s setup (setup-go, then module and build cache tar extraction),
33-71s "Exercise the real Bash process boundary", 27-58s ./cmd/bd test
compile+link, 5-9s each for three more go test relinks.
- scripts/pr_preflight_test.go: the ~29 pr-preflight.sh runs (about 1.5s
each under Git Bash on Windows, 42.8s total in the merge group) now run
as parallel subtests. Each run already had its own temp dir, call log
and env; the one test that t.Setenv's stays serial. Linux: 4.8s -> 0.8s.
- pr.yml: the hook-timeout step and the bd build fallback step become one
go test of ./cmd/bd, saving a package re-hash and test-binary relink
(5-9s on Windows for a sub-second test). Policy tests and the allowlist
and topology docs follow the step name.
- warm-non-race-cache.sh: on Windows also compile the three CGO_ENABLED=0
credential-fixture test packages the Windows-only fixture step runs, so
main.yml's test-windows seeder covers them.
No step, test or gate input is dropped; the job id, name, matrix and
runner are unchanged.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
rbe-west's pool workers run gascity main's tools/rbe/worker-env and
advertise its pin; the schedulers match //platforms:rbe_worker's pin
exactly. When gascity re-pins and beads' copy falls behind, every remote
action queues until the lanes time out, and the only check ran nightly
with no notification (review finding S1).
- bazel.yml's rbe job: after decide, only when the lanes execute remotely
(enabled: modes remote, fork-ro, fork-rw; merge groups are remote), a
sparse checkout of tools/rbe and platforms and tools/rbe/worker-env-sync
main. Out of step fails the job, so every lane skips and pr.yml's gate is
red within a minute, with the diff and the pin to commit in the summary.
No fallback to mode cache: pr.yml's ci-gate requires a remote run
(BAZEL_REMOTE), so it would be just as red, only slower. gascity
unreachable warns and proceeds. The steps set no output; the job's
outputs stay decide's.
- worker-env-sync: exit 1 only for out of step, exit 2 (warning) when it
cannot compare (ERR trap; curl timeouts), WORKER_ENV_SYNC_REPORT copies
the markdown report.
- nightly.yml: top-level permissions contents: read; rbe-worker-env-sync
gets issues: write and, on exit 1, tools/rbe/worker-env-sync-issue opens
a ci/infra issue with the report or updates the open one (body + comment
linking the run). GITHUB_TOKEN is in that one step.
- Policy tests: TestBazelRBEWorkerEnvPreflight (shape + the step's script
run in step / out of step / unreachable), TestNightlyWorkerEnvSyncOpensIssue
(permissions, token scope, rc output, issue create/update via a stub gh),
TestRBEWorkerEnvSync (exit codes, report); rbe job step and checkout
counts updated.
Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- pr.yml's Windows jobs and the macOS/Windows legs of pr-preflight-platforms
and check-doc-freshness-platforms run on Blacksmith for every PR and merge
group, forks and Dependabot included: no windows-latest/macos-latest
fallback (Blacksmith serves this org's fork PRs). Matrix legs name their
label (runs-on: ${{ matrix.runner }}); the same-repo "matrix marker"
ternary and its policy plumbing are gone.
- pr-preflight-platforms drops its Linux leg: its tests run under Bazel on
rbe-west (//scripts:scripts_test, //cmd/bd:bd_test,
//internal/config:config_test).
- The Linux-cross-compiled Windows path is retired: windows-test-binaries,
its two -prebuilt twins, main.yml's windows-test-binaries-cache, the
WINDOWS_PREBUILT_REQUIRED flag and the build script/manifest. The native
pair (test-windows-liveness, worktree-remove-windows) is the required run.
- main.yml's test-windows and blacksmith-macos-go-build-cache drop their
GitHub-hosted venue legs (no GitHub-hosted consumer left).
Gate names unchanged. ga-96smfk.22
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Push to main and the nightly schedule no longer run go test or repo checks
on GitHub runners (ga-96smfk.22, ga-96smfk.58):
- main.yml keeps only the macOS -race -short suite, the Windows smoke build
and the push-to-main savers of the GOCACHEs pr.yml's Windows/macOS jobs
restore. Its Linux jobs (migration hygiene, build artifacts, build tags,
versions, duplicate migrations, doc flags, package gates, PR policy
wrapper, gofmt, the Linux and setup-go cache seeders, the Nix flake
check) are gated on the merge queue's pr.yml/pr-risk.yml run of the
same commit and re-run as Bazel targets by bazel.yml's push run.
- nightly.yml is the fresh bazel.yml call plus the worker-env check; its
Full Test Suite, PR Core go-test-json side and measure dispatch are gone,
as are ci-measurements.yml and bazel.yml's go-test-json-artifact input.
- The non-race embedded batch-apply step becomes
//internal/storage/embeddeddolt:embeddeddolt_batch_apply_nonrace_test
(go_test_race_off, BEADS_TEST_EMBEDDED_DOLT=1) in the embedded tier.
- make ci-pr-policy and scripts/ci/pr-policy.sh go: every check it ran is
a Bazel test (TestPRPolicyChecksRunAsBazelTargets).
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every pull_request and merge group already took the Bazel lanes for the
retired tiers (BAZEL_RETIRES_LEGACY_* and BAZEL_COVERS_FORKS all "true",
and pr.yml/pr-risk.yml run on no other event), so the legacy jobs those
flags stood down could never run again. Remove them with their decision:
- pr-risk.yml: build-embedded, test-embedded-{storage,conformance,cmd},
test-proxied-cmd, test-server-storage{,-full}, detect-ci-tier (and its
ci-embedded-tier.sh / ci-capability-selector.sh scripts) and
bazel-coverage. PR Risk keeps the Nix packaging check (test-nix) and its
gate, PR Risk Gate / Required.
- pr.yml: build-artifacts, check-cmd-bd-puregeo-tests, pr-core-wrapper and
bazel-coverage; the BAZEL_RETIRES_LEGACY_* and BAZEL_COVERS_FORKS flags.
The three BAZEL_*_RETIRED gate ids become one, BAZEL_REMOTE: the Bazel
call must have run remotely (remote, fork-ro or fork-rw) on every PR and
merge group, each lane's own id still required. Gate names unchanged.
Policy tests: the D2 decision matrix becomes
TestPRGateRequiresRemoteBazelLanes (bazel.yml's actual rbe step and
pr.yml's actual gate over every event/variable/secret/fork/actor/mint
fact); the parity tests that read the legacy jobs now pin the Bazel lanes
against those jobs' frozen commands.
ga-96smfk.22
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
tools/bazel/ci_analytics_extract.py turns a lane's BEP file, compact
execution log and JSON profile into a redacted ci-analytics-summary
(timing, per-mnemonic cache classes, cold actions, test results,
repo-fetch time): only validated labels, mnemonics, enums and numbers
leave the runner, never an endpoint, cert/key path, env value, token
or hostname. Every lane's "CI analytics summary" step runs the
extractor and uploads its output as ci-analytics-<lane>-<run_attempt>
(7-day retention); CHECK_RUN_ID uses the real job.check_run_id
context (requires actionlint >= 1.7.8), and both the summary and
upload steps are continue-on-error, the summary capped at
timeout-minutes: 3, and both gated on !cancelled() (not always(),
which also runs after a workflow cancellation).
The extractor, its 62-case unittest file, and testdata/ci_analytics/
(the real/* Bazel 9.2.0 fixtures, poisoned-bep.json, mismatch.bep.jsonl
and regen.sh) are vendored byte-identical from gascity's
ci/analytics-extractor slice (final squashed commit
00161373bc7bbfcef8d6a05ac5ce70409fc7f2ac), which is canonical for this
pair the same way tools/rbe/cache-zstd-probe.sh is a byte copy.
Fixtures are regenerated under env -i with every BEP field stripped
unless it's on an explicit allowlist, and the compact exec log is
committed decompressed (*.binpb) rather than as a zstd frame, so
nothing opaque reaches version control.
scripts/ci_analytics_extract_test.go is the thin Go wrapper:
TestCIAnalyticsExtractorIsPinnedToGascity pins both vendored files'
sha256 so local drift is caught, and TestCIAnalyticsExtractUnitTests
runs gascity's unittest file in place. scripts/ci_analytics_workflow_test.go's
TestCIAnalyticsStepsAreSafeAndBounded mirrors gascity's pin test,
adapted to beads' 8 separately-named lane jobs: continue-on-error,
timeout-minutes bounds, no raw ${{ }} expression in run:, !cancelled()
required in if:, correct per-lane upload artifact name/path, and
CHECK_RUN_ID == ${{ job.check_run_id }}, for every lane.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): nogo replaces go vet and golangci-lint (ga-96smfk.18)
Lint and vet now run as nogo inside the Bazel build, so they gate wherever
the Bazel lanes execute (rbe-west), and the golangci-lint and go vet jobs
leave pr.yml and main.yml.
- //tools/nogo (own go.mod, joined to beads' for go_deps only through
gazelle.go.work): go test's vet checks (cmd/go's defaultVetFlags, the set
scripts/ci/go-test-vet.sh ran) plus the eight linters .golangci.yml
enables, each a wrapper over the library version golangci-lint v2.10.1
uses (depguard, errcheck, forbidigo, gosec, misspell, sloglint, unconvert,
unparam).
- .golangci.yml stays the single lint policy: Bazel embeds it and
tools/nogo/internal/golangci applies it as golangci-lint does (enabled
set, settings, run.tests: false as library-unit-only analysis, lax
generated-file detection, path/path-except/text exclusion rules), with
strict decoding so an unimplemented key fails the build.
- gascity's rules_go //nolint-range patch and x/tools ctrlflow patch,
byte-identical.
- Windows/darwin: cross-configured, not a golangci leg. The pure lane's
release cross-compile (//tools/bazel:release_cross, every library and
binary for all eight release platforms, cgo off) runs nogo on every
compile, a superset of golangci-lint's windows and darwin legs.
scripts/pr-lint (make ci-pr-lint, bd preflight) runs `bazel build
--config=nogo //...` plus release_cross for windows_amd64,darwin_arm64.
release_cross skips tools/nogo, a separate Go module `go build ./...`
never compiled.
- pr.yml: scripts-go-checks (vet) and pr-lint-wrapper (x3) are gone, with
their ci-gate entries; main.yml: pr-lint-wrapper (x6) and go-vet-cache.
scripts/ci/go-test-vet.sh and install-golangci-lint.sh are deleted.
- pre-commit hook and .pre-commit-config.yaml run `make lint-changed
LINT_CHANGED_SCOPE=staged` (nogo over the staged packages); Makefile adds
lint, vet and lint-changed. pr-lint.sh no longer runs gofmt
(//scripts/repochecks:fmt_test owns it; ga-96smfk.41).
- internal/testutil/integration: five gosec findings in integration-tagged
library files golangci-lint never built, now linted by the integration
lanes, carry reasoned //nolint directives.
Parity: with the exclusion rules removed, golangci-lint and nogo report the
same 3,916 findings natively and 3,859 each for windows and darwin (file,
line, linter, text); with .golangci.yml as committed both report none.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): equivalence.py leaves nested Go modules' tests out of go test's set
tools/nogo is its own Go module, so `go test ./...` in the root never runs
its tests; Bazel's unit lane does. Its packages are neither expected nor
extra.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dolt-server lane (--config=doltserver) has been the only pre-merge run
of the Dolt-backed domain, uow, tracker, doctor/fix and protocol suites
since BAZEL_COVERS_FORKS went "true": bazel-coverage reports pr_lanes=true
for every pull_request and merge_group, so pr.yml's test-domain-uow and
contract-corpus never run. An A/B run of every requires-docker target
against its dolt-server twin on the same commit shows identical test sets
and statuses (2683 testcases) except TestDoltServerFingerprint/container,
the container half of the fingerprint, which needs docker by design.
Delete the six requires-docker sh_tests, test:docker in .bazelrc, the
requires-docker tag from the taxonomy and lane filters, bazel.yml's
dolt-lane dispatch input and image pull, and pr.yml's test-domain-uow and
contract-corpus jobs with their ci-gate wiring. The testcontainers backend
in internal/testutil stays: it is plain go test's default. pull-dolt-image.sh
stays for pr-risk.yml's legacy tiers. No required check changes: the
rulesets require only CI Gate / Required and PR Risk Gate / Required.
Refs ga-96smfk.21
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
rbe-ci-cost-latency-study.md recommendation 4: this tier's last shard
has trailed the rest by 4.3-4.5 min in 2 of 8 runs. tools/bazel/shard_budget.py
reads the job's BEP file and prints ::warning when a sharded target's
slowest shard exceeds max(150s, 2x median); --fail-on-imbalance (not
wired in yet) switches that to ::error and a nonzero exit once the
budget has run clean for a while.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
pr.yml's check-release-target-cross-compilation ran `CGO_ENABLED=0
GOOS/GOARCH go build -tags gms_pure_go ./...` for the eight release targets
on two GitHub/Blacksmith runners. It is now a step of bazel.yml's pure-Go
lane (BAZEL_PURE, already required by ci-gate):
scripts/ci/bazel-release-cross-compile.sh builds //tools/bazel:release_cross
once, remotely, with --//tools/bazel:release_platforms set from
scripts/ci/release-targets.txt (still the single source of truth, pinned to
.goreleaser.yml). release_cross's split transition configures every
go_library/go_binary for each rules_go platform with cgo off, so all eight
platforms build in parallel in one invocation.
- tools/bazel/release_cross.bzl: the split-transition rule and the
string-list flag.
- tools/bazel/go_srcs.py (make bazel-sync, checked by bazel-sync-check and
TestBazelGoSrcsBlocksCurrent) keeps the release_cross target list; the
script checks with bazel query that every Bazel package holding Go
targets is reached before it builds.
- //tools/bazel:pure_bd_has_no_cgo_only_deps: Bazel compiles gozstd's pure
stubs where `go build` rejects them, so the pure bd of every platform
must not link gozstd.
- //internal/storage/embeddeddolt/cmd:cmd (cgo-only main, skipped by
`go build ./...` under CGO_ENABLED=0) is tagged cgo-only and incompatible
with pure builds.
- A failed build is re-run per platform (cache hits) to name each failing
GOOS/GOARCH.
Same cgo-off coverage boundary as the retired job. Refs ga-96smfk.20.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
cross-version-smoke.yml resolved the newest 5 GitHub releases (30 on a tag
push) with `gh release list`, then built the candidate and downloaded each
release on a GitHub runner for scripts/upgrade-smoke-test.sh.
//tests/upgrade_smoke has one target per release in the newest 30 the
release catalog pins a linux/amd64 asset for (@bd_releases//:versions.bzl):
upgrade_smoke_<release>_test runs the script against the non-race
bd_for_tests with that release pinned as PREV_BIN (sha256-verified when
Bazel fetches it; the script checks it reports the release and downloads
nothing). A release added to the catalog joins the window. Tagged
pr-core-only, so bazel-test (not the integration lane) runs them remotely
on every PR, merge group and
push, cached until the candidate, the script or the release changes.
The tested set is now reviewed rather than resolved at run time: stable
releases from the catalog, where the workflow took GitHub releases
including prereleases. cross-version-smoke.yml and the F7c checks of its
chunk fold, path filter, runners and binary cache are removed (the
path-filter and binary-cache checks had no other workflow left).
Refs: ga-96smfk.19
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore(migration-test): extend the release catalog through v1.3.1
The catalog stopped at v1.1.2 (the v1.2 upgrade census). The migration
corpus already upgrades from v1.2.2, and the Bazel targets that replace
migration-test.yml and cross-version-smoke.yml take their pinned release
binaries from the catalog (@bd_releases), so it has to cover the releases
since. Regenerated with `catalog generate` after raising maximumVersion to
v1.3.1: five new stable versions (v1.2.0, v1.2.1, v1.2.2, v1.3.0, v1.3.1;
three with a linux/amd64 asset) and five new proxy prereleases; no existing
entry, drift record or repository-only exclusion changed. The count
constants and the identity digest are updated to match, and the README now
says how to add a release.
Refs: ga-96smfk.19
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the historical-upgrade corpus under Bazel; retire migration-test.yml
migration-test.yml ran scripts/migration-test/run.sh --version <release> for
14 reviewed releases in 3 shards on GitHub runners: it downloaded each
release archive, built v0.9.1 from source with Go 1.25 through the module
proxy, installed libicu74 and the dolt 2.1.8 runtime, all over the network.
//tests/migration now has one target per corpus release
(historical_upgrade_<release>_test) plus legacy_bridge_test, all hermetic:
- Published releases come from @bd_releases (the release catalog's
sha256-verified assets) through a per-target directory
(HISTORICAL_RELEASE_DIR); lib/binary.sh uses it instead of downloading and
still checks each binary's self-reported version.
- v0.9.1 is //tests/migration:bd_source_tag, the same `go mod download` +
`go build -trimpath` with CGO and Go 1.25.0, run offline as a build
action: the module files come from tests/migration/source_tag_modules.json
through a file:// GOPROXY tree (tools/bazel/go_module_proxy.bzl), the
release's own zip is the catalog's source_zip, and go checks every
dependency against the release's go.sum (SOURCE_TAG_SQLITE_BIN).
- The external Dolt runtime is @dolt_test_runtime_linux_amd64 (dolt.bzl's
dolt_release gains a version attribute; 2.1.8 is pinned to versions.sh's
digest), passed as DOLT_BIN to the server-Dolt and wisp-plane releases.
- The workers already provide jq, python3, sqlite3 and libicu74.
The targets are untagged, so bazel-test runs them remotely on every PR,
merge group and push; result caching replaces the path filter. Every
release also passed inside a loopback-only network namespace.
scripts/migration_corpus_bazel_test.go keeps the BUILD lists equal to
versions.sh's corpus, the catalog digests equal to STRICT_RELEASE_SHA256,
the source-tag lock equal to the catalog's source_zip, and the Dolt runtime
pin equal to versions.sh's. migration-test.yml and the policy tests that
pinned its shards, caches, apt step and runner are removed.
Refs: ga-96smfk.19
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the historical-upgrade targets in the PR-core lane only
The integration lane runs every untagged target too, and its
check_testcases.py requires Go testcases, which a shell test's test.xml does
not list. A new pr-core-only tag keeps //tests/migration out of
--config=integration (bazelIntegrationExcludedTags adds it), so the corpus
runs once, in bazel-test, against the release-shaped bd_for_tests (no
integration tag). build_source_tag.sh carries the build-tags opt-out: it
builds a historical release with that release's own go.mod, which predates
gms_pure_go.
Refs: ga-96smfk.19
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
regression.yml ran `go test -tags=regression ./tests/regression/...` on a
GitHub runner: it built the candidate with `go build`, downloaded the v0.49.6
baseline release over the network, and pulled the dolt-sql-server image.
None of that works on rbe-west workers (no network for fork lanes, no docker).
- tools/bazel/bd_releases.bzl: a module extension that reads
scripts/migration-test/release-catalog.json and declares one repository per
catalog release with a linux_amd64 asset digest (bd_release_<version>,
verified against the catalog's sha256), behind the @bd_releases hub
(@bd_releases//:<version>, plus versions.bzl). Adding a release is a
catalog change only; nothing downloads until a test declares one.
- //tests/regression:regression_test: the suite's files build under
`regression || integration`, so gazelle generates the target. It runs the
injected non-race bd_for_tests (BEADS_TEST_BD_BINARY) against
@bd_releases//:v0.49.6 (BD_REGRESSION_BASELINE_BIN), both over a hermetic
dolt sql-server (BEADS_TEST_DOLT_SERVER=local, required), in 8 shards.
TestMain checks the baseline reports BASELINE_VERSION, so the label and the
file cannot drift. Tagged dolt-server-cmd: bazel.yml's bazel-cmd-dolt lane,
which pr.yml's gate requires, runs it remotely; test result caching stands
in for the old path detector.
- A plain `go test -tags=integration ./...` (nightly) compiles the files but
skips the suite; only -tags=regression (make test-regression) or Bazel
runs it.
- regression.yml, main.yml's warm-up compile for it and the policy tests that
pinned its jobs are removed.
Refs: ga-96smfk.19
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
fast-checks, PR Policy, and the Linux leg of check-doc-freshness-platforms
ran repository guards on GitHub runners. They now run as Bazel tests in
bazel.yml's `test` lane (remote, cached), each over only the files it reads:
- tools/bazel/repo_subset.bzl picks a narrow, path-selected slice of
//:repo_files at analysis time. scripts/repochecks/run_check.sh copies that
slice into a throwaway git checkout and runs the unchanged guard there.
- //scripts/repochecks: build_tags, workflow_tags, go_install_guidance,
winget_portable_alias, migration_hygiene (checks A and B),
fmt (registered SDK gofmt), doc_freshness, workapi_frontend_boundary,
types_gen_drift (`make api-check` drift, oapi-codegen from go.mod).
- //scripts:doc_freshness_required_test (integration lane, -required-suite)
and //scripts/gitattributespolicy:gitattributespolicy_required_host_test
cover the Linux doc-freshness leg.
- check-versions stays covered by //internal/versioncheck:versioncheck_test.
check-testing-short stays covered by //scripts:scripts_test.
pr-policy-wrapper is deleted. fast-checks keeps only the two checks that
need the PR base: migration hygiene's delta half (MIGRATION_HYGIENE_SCOPE=
delta) and the .beads diff. check-doc-flags stays, because it builds bd from
the pinned release tag. The example Go modules leave .bazelignore so gofmt
and the grep guards see them. check_testcases.py now accepts --not-go
//pkg:all.
Refs: ga-96smfk.20
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pr.yml's every-PR "Test (Dolt server fingerprint)" job installed dolt, pulled
the dolt-sql-server docker image and ran TestDoltServerFingerprint against
both backends. The rbe-west workers have no docker, and the container backend
is now used only by the container-backed legacy jobs, which run only where
bazel-coverage does not cover pr_lanes.
- //internal/testutil:testutil_dolt_test runs testutil_test with the local
backend selected and required (BEADS_TEST_DOLT_SERVER=local,
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1) on the dolt-server lane, which
pr.yml's gate requires: the fingerprint's local half against the pinned
dolt, plus the TestLocalDoltServer_* tests every other configuration
skips under BEADS_TEST_SKIP=dolt.
- The container half moves back to test-domain-uow's first test step (where
it was before D2 step 3), so it runs exactly where containers are used.
//internal/testutil:testutil_docker_test is its docker-lane A/B control.
TestPinnedDoltCLIMatchesContainerImage still pins the image tag to the
dolt release.
- test-dolt-server-fingerprint, its ci-gate need/result and its policy
tests are removed; TestDoltServerFingerprintRunsWhereEachBackendRuns
pins the new placement.
Refs: ga-96smfk.19
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
proxied-local-smoke.yml built bd and the cmd/bd test binary on a GitHub
runner, installed the pinned dolt, and ran `-test.run ^TestManagedLocalProxied`
under `sudo unshare --net` with only loopback up, to prove the managed-local
lifecycle (bd -> loopback proxy -> dolt sql-server) needs no outbound network.
//cmd/bd:bd_managed_local_test runs the same tests (BEADS_TEST_PROXIED_LOCAL=1,
BEADS_TEST_SKIP=dolt, the non-race bd_for_tests as the bd under test, the
pinned dolt from the test wrapper's PATH) through tools/bazel/go_test_offline.sh,
which enters an unprivileged user + network namespace with only loopback up
and fails if outbound network is reachable: the job's netns, without sudo,
on the rbe-west workers. Tagged dolt-server-proxied, so bazel.yml's
bazel-proxied lane (remote-only, required by pr.yml's gate) runs it; test
result caching replaces the workflow's path filter. The job's "tests are
compiled in" assertion becomes scripts/managed_local_lane_test.go, which
evaluates cmd/bd's build constraints for a linux cgo build and requires each
of the 17 pinned tests.
The target joins the pinned retired-lane targets in
TestBazelRetiredLanesCannotBeNarrowed. proxied-local-smoke.yml and the policy
tests that pinned its job and its dolt version are removed (the Bazel dolt
pin is already enforced against install-dolt.sh).
Refs: ga-96smfk.19
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
conformance.yml ran two tiers on a GitHub runner. Tier 1 (the embedded-Dolt
TestConformance) already runs as the embedded lane's
embeddeddolt_conformance_{core,audit}_test. Tier 2, the real-binary CLI
corpus (`go test -tags 'gms_pure_go e2e' ./test/conformance`), had only a
go_library under Bazel. Its two test files now build under `e2e ||
integration`, so gazelle generates //test/conformance:conformance_test
(integration-only, against the injected non-race bd_for_tests) and
--config=integration runs it remotely in bazel-integration, which pr.yml's
gate requires. PR Core's go test builds neither tag, so the --config=ci
equivalence check is unchanged; scripts/conformance.sh keeps working with
-tags e2e as the local go test entrypoint. With both tiers covered,
conformance.yml and main.yml's warm-up compiles for it are removed.
docs-mintlify.yml drops its docsync job (`go test ./test/docsync`, run by
bazel-test as //test/docsync:docsync_test on every PR) and keeps only
Mintlify's broken-link check, which needs the network (npx mint) and the
base branch's history; its path filter narrows to what that check reads.
Refs: ga-96smfk.17
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
//scripts:scripts_test and //scripts/check-versions:check-versions_test
drop their host-tools/no-remote-exec/no-remote-cache tags and run on the
rbe-west workers like every other target (the worker image ships bash, git,
python3, jq, yq, curl, openssl, unzip and coreutils; a missing tool fails the
test under Bazel rather than skipping it). Go comes in hermetically as the
registered SDK; check-versions.sh's `go build` is stood in for by the
Bazel-built checker.
The tests that walked the source checkout now get it as data:
tools/bazel/go_srcs.py keeps a managed `repo_files` filegroup in every
package and aggregates them into //:repo_files (exactly the tracked files
Bazel can see; make bazel-sync-check fails on a package without one).
scripts_test and docsync_test take it instead of hand-listed filegroups,
and every TEST_SRCDIR/IsBazel skip and go-test-only block in them is gone.
The release formula under .bazelignore'd .beads/ comes in through a
@beads_formulas local repository; npm-package/ and .claude/ are visible to
Bazel again (only their node_modules/worktrees stay ignored), so the npm
manifest release check and the AGENTS.md skill links run under Bazel too.
TestZZStdioNotLeaked falls back to TestMain's pre-run streams instead of
skipping in a shard without its baseline.
tools/bazel/equivalence_allowlist.txt shrinks from 20 skip entries to the
two cmd/bd tests of plain go test's own bd build fallback, which Bazel never
takes; pr-preflight-platforms now runs them on Linux, macOS and Windows. The
scripts-go-checks scripts-test and allowlisted legs, their scripts
(scripts-go-test.sh, allowlisted-go-tests.sh, run_allowlisted_go_tests.py)
and main.yml's --compile-only warm for them are removed. The vet leg stays
(ga-96smfk.18).
Refs: ga-96smfk.16
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bazel.yml already runs on every push to main and executes the PR Core,
pure-Go, dolt-server, embedded, proxied, server-storage, integration and
cmd/bd Dolt-server tiers on rbe-west in about five minutes. main.yml re-ran
the same tiers as legacy go test jobs (62 jobs, about 380 runner-minutes a
push), and its runs were all failing or cancelled. Remove the duplicated
Linux jobs: the ubuntu -race -short leg with its Codecov upload, PR Core,
the pure-Go compile check, domain+uow, the 6+8 integration shards, the 15
proxied shards, and the embedded build plus its storage, conformance and
cmd shards.
The macOS -race -short suite (rbe-west has no macOS workers) moves to
Blacksmith macOS and restores the macOS saver's caches. The cache savers
that fed the fork path from the removed jobs move to venue matrices: the
macOS saver gains a GitHub-hosted leg, and the Linux saver gains a
GitHub-hosted race leg (the embedded build used to seed it).
Refs ga-96smfk.14
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: make the required gates merge-queue ready
The default-branch ruleset requires "CI Gate / Required" and "PR Risk
Gate / Required". Prepare both for a GitHub merge queue (ALLGREEN,
squash), so enabling the ruleset's merge_queue rule is the only step left.
- pr.yml, pr-risk.yml: merge_group pinned to types [checks_requested].
- D2: bazel-coverage covers every merge_group run (same repository, CI
secrets, actor github-merge-queue[bot], bazel.yml mode remote), so the
retired legacy tiers stay retired in the queue and ci-gate requires
their Bazel lanes to have run remotely and passed; a missing secret or
the farm switch off is red, as on a same-repo PR.
- Diff bases on merge_group: the package gates diff
merge_group.base_sha..head_sha (not HEAD^, which misses all but the
last commit of a rebase-method entry), the .beads/issues.jsonl guard
now runs on merge_group against base_sha (ci-gate no longer excuses
its skip), and check-doc-flags passes BD_DOCS_DIFF_BASE so CLI-docs
drift already on main cannot eject every queued PR.
- Test result reuse: on a merge_group run and on attempt 2+ of a PR run,
the six always-execute Bazel lanes append --config=reuse-test-results
(--cache_test_results=auto), so tests whose action key is unchanged
reuse the farm's recorded PASS and only what the merge changed runs.
Failures are never reused, eviction retries stay 0, and each PR's first
attempt, push to main and nightly still execute every test.
- scripts/ci_merge_queue_test.go simulates merge_group through the shared
evalGHExpr evaluator (now with true/false/null literals, hyphenated
identifiers and case-insensitive ==) and the workflows' own scripts:
triggers and gate names, concurrency groups, every pull_request field
read, the coverage and rbe decisions, result reuse, the package-gate,
.beads and doc-drift diff bases.
- engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md: new "Merge Queue" section (how
to queue, settings, what runs, result reuse rationale, ejection and
flakes, rollback).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: cache Bazel test results except nightly; queue trust and sizing
Review follow-ups to the merge-queue readiness change.
- Test results (S1): --nocache_test_results also stops Bazel writing
results, so a PR's first run never fed the cache and the merge-queue
reuse config could only hit on later reuse runs. Drop it from
test:embedded, test:doltserver-proxied, test:doltserver-integration and
test:sole-run (which keeps --experimental_remote_cache_eviction_retries=0),
and drop test:reuse-test-results and BAZEL_REUSE_RESULTS. Results are now
cached (read and write) on PRs, merge groups and pushes to main.
nightly.yml passes the new fresh-test-results input, so every `bazel
test` appends --config=fresh (--nocache_test_results) and the suite
re-executes nightly. The docker lane keeps --nocache_test_results.
Measured locally with a disk cache: a default run writes and the next
reads ("(cached) PASSED"); a fresh run neither reads nor writes.
- Security headers (S2): bazel.yml now states that a merge_group run
executes the queued commit's workflows, so a queued fork or Dependabot
PR's CI edits run with every CI secret, the pre-warm app key included
(merging-level trust), and records the maintainer rule: review fork
changes to .github/, .bazelrc, tools/bazel, MODULE.bazel, BUILD files
and CI scripts before queueing. Same note in the topology doc.
- Queue sizing (S3): recommend max_entries_to_build 3 and
check_response_timeout_minutes 60 (max_entries_to_merge 5, ALLGREEN,
SQUASH unchanged).
- Nits: the .beads guard fails closed on a missing diff base, and its
test runs under GitHub's default `bash -e`; evalGHExpr follows GitHub's
loose equality (true == 'true' is false, false == '' is true, missing
context keys are null) with a typed-context variant; the
detect-package-gates comment notes the SQUASH/HEAD^ equivalence and
path-filter scope; the doc covers re-run reuse, flake re-sampling via
dispatch, stale queue runs and how to confirm reuse on the farm.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pr-preflight-platforms and check-doc-freshness-platforms now resolve their
same-repo-macos marker like the Linux/Windows legs: blacksmith-6vcpu-macos-26
for same-repo PRs and merge_group, GitHub-hosted macos-latest for forks,
Dependabot and every other event. The label is pinned to macos-26 because
GitHub's macos-latest is macos-26-arm64 today, so both paths run the same
OS/arch; 6 vCPU is already 2x macos-latest's 3 vCPU.
Blacksmith cannot see GitHub-saved caches, so main.yml gains
blacksmith-macos-go-build-cache: a push-to-main-only saver on the same label
(same job guard as the other Blacksmith savers) that builds ./cmd/bd and runs
the shared warm-non-race-cache.sh, saving the module cache and a non-race
GOCACHE keyed by go.sum + UTC day. The PR legs stay restore-only. main.yml's
GitHub-hosted macOS full-suite leg is unchanged and keeps seeding the fork
path.
Policy tests: the macOS marker arm is evaluated for same-repo/merge_group/
fork/deleted-head/dependabot/push/schedule and must match the flat macOS
const; the saver joins the PR-guard and per-day-key sweeps and the cache
ownership inventory; TestBlacksmithMacOSSaverMatchesPRLegs pins the
label/path/key/warm-up contract between saver and PR legs.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bazel-integration (--config=integration) and bazel-cmd-dolt
(--config=doltserver-cmd, //cmd/bd:bd_dolt_server_test) both run the
integration-build cmd/bd bd_test; the second without BEADS_TEST_SKIP=dolt,
so it is a superset. Once pr.yml requires bazel-cmd-dolt, the integration
lane's bd_test run is a duplicate.
bazel.yml gains a boolean workflow_call input, cmd-dolt-required (default
false). When true, bazel-integration tests //... -//cmd/bd:bd_test. pr.yml
passes it, and a policy test pins it to pr.yml's BAZEL_CMD_DOLT_REQUIRED
(a `with:` cannot read env), so the drop only happens where ci-gate
requires BAZEL_CMD_DOLT. bazel-farm.yml, nightly.yml, push and dispatch
leave it unset and keep bd_test. While the flag is "false" (current main)
nothing changes; with #7226's flip the input flips with it.
Coverage: both lanes share the same `if:` (modes remote, fork-ro, fork-rw,
cache; Dependabot included) and bazel-gate.sh accepts their skips in the
same modes (skip, local). TestBazelIntegrationExcludesBdTestOnlyWhereCmdDoltCovers
pins that, the exact exclusion, the flag/input equality, the gate's
conditional requirement, and that no other caller passes the input.
Rollback: commit BAZEL_CMD_DOLT_REQUIRED "false" and cmd-dolt-required:
false together in pr.yml.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Advisory since #7185. On the farm it passed on every PR run based on
current main; its only failures (3) were branches predating #7219 and
reproduced a real test bug that #7219 fixed. Flip the committed flag so
CI Gate / Required enforces BAZEL_CMD_DOLT. Rollback: set it back to
"false".
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pr-preflight-platforms' "Exercise native init gateway credential shell"
step had no GOCACHE env, so it fell back to the runner's empty default
cache and cold-compiled all of ./cmd/bd's test binary on every PR, right
after the hook-timeout step had compiled the identical binary (same tags,
same platform-default CGO) into the restored non-race cache. Job logs:
213s of a ~350s macos-latest leg, ~117s on Windows; on Linux the rerun
with a shared cache is a relink only (~8s).
- pr.yml: point that step, and the three Windows-only Go steps, at the
restored non-race GOCACHE.
- main.yml: the macOS test leg (the only macOS non-race seeder) now runs
warm-non-race-cache.sh, so the PR macOS legs restore the test packages
and test-only deps they compile, not just the non-test ./cmd/bd graph.
- Policy tests: every run step in pr-preflight-platforms and
check-doc-freshness-platforms must use the restored GOCACHE, and the
macOS seeder must warm before it saves.
No test selection, matrix, or gate wiring changes.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(autofix): push bot fixes with a scoped GitHub App token
github-actions[bot] pushes left PR checks action_required instead of
retriggering, and the DOCS_AUTOFIX_TOKEN fallback secret was never
configured. bazel-autofix.yml and docs-autofix.yml now mint a short-lived,
repo-scoped (contents:write) installation token for the gastownhall-autofix
App via the `autofix` environment on same-repo PRs, falling back to the
default workflow token (and its manual re-run note) when the App token
isn't available; fork PRs are unaffected and still only get the apply-recipe
comment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): job environment parses as a name or a mapping; gofmt
ciWorkflowJob.Environment was a string, so every test that loads all
workflows failed on deploy-pages-redirect.yml's mapping form
(environment: {name, url}).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
pr.yml and pr-risk.yml (and the three advisory workflows that already
list release/** beside it -- conformance.yml, cross-version-smoke.yml,
regression.yml) trigger pull_request only on main and release/**. A PR
based on hotfix/1.3.1 (or any sibling hotfix/** line) matched neither
pattern, so it ran almost no CI: #7146, a backport to hotfix/1.3.1,
showed 17 checks -- 15 cross-version historical smokes and triage/label
jobs, 2 skipped -- no Lint, no unit shards, no embedded tier, no CI
Gate / Required, no PR Risk Gate / Required.
pr.yml's own comment states exactly why release/** is there:
release-prep and release-fix PRs are based on the release branch, not
main, and without it they run almost no CI before the tag build. A
hotfix line is in the identical position. Added hotfix/** alongside
release/** in all five files.
New TestPullRequestWorkflowsTriggerOnHotfixBranches pins hotfix/**
presence in each file's pull_request.branches list -- red before this
change (observed [main release/**] in all five), green after. The
existing TestPRRiskBazelCoverageJob equality check (pr.yml and
pr-risk.yml's pull_request triggers must match) still passes, since
both were updated identically. go vet/gofmt clean.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
* perf(storage): cut batch-create round trips (import, bulk create, apply-batch)
A multi-issue create (import, bulk create, CreateIssuesWithFullOptions) ran
~20 SQL statements per issue, and its dependency pass ran a recursive cycle
CTE per edge over the union of both dependency tables, which Dolt cannot
index — every recursion step rescanned the edge tables, so an import whose
edges form chains went superlinear (400 issues: 73 s of 82 s in that CTE).
issueops batch create, same transaction and same stored outcome:
- createBatchCache: one IN-list read of every explicit id's presence in both
planes (replacing two COUNT probes per issue), one read of the batch's
stored labels (each issue's new labels then land in one multi-row INSERT),
created/label_added events buffered and written in one pass under the ids
InsertDerivedEvent would mint (same digest, same lowest free ordinal), and
runs of brand-new rows written with multi-row INSERTs (same ODKU clause)
then finished in order.
- Dependency pass: wisp routing and target presence from batch reads; the
cycle and hierarchy checks walk an in-memory copy of the edge tables,
loaded once and kept in step with every edge the pass writes (a pair it
already holds is re-read, since a clientFoundRows connection reports a
matched duplicate as written).
- createBlockedRecomputeIDs seeds all created edges and expands once.
Shared paths:
- Blocked-state recompute splits ids once per call by whether they own a
dependency row; edgeless ids (every freshly created issue) skip the two
union UPDATEs for one plain unmark, run after the chunk's union statements
on the first pass so the unions see what they saw before.
- appendChildrenInTx reads children per IN-list chunk, not per parent.
- ExecuteCreate hands its BatchContext to the batch body instead of having
it re-read the same config (-6 statements per apply-batch create item).
Not changed: one version-history row per created issue (the version
contract, version_history.go), journal order and content, and the per-row
bodies, which stay the path for singular creates.
Equivalence: internal/storage/createbatchequiv runs an import scenario
(upserts, duplicate ids, wisps, generated ids, cycles, hierarchy conflicts,
dangling/external targets, stale is_blocked, comments, journal and
versioned history on) through the fast and per-row bodies on the embedded
and server engines and compares every written table and the skip reports.
Re-pinned TestLargeBatchApplyStatementCounts*: embedded 7009/14014/846 ->
6396/12790/793, server 7008/14014/846 -> 6396/12791/793 (breakdown in
pinnedEmbeddedStatementCounts).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): batch-create review fixes — lazy edge graph, opt-in recompute probe, insert budget
Review of the batch-create round-trip change (S1-S3 and nits):
- The dependency pass's in-memory graph no longer loads both edge tables.
It reads a node's outgoing edges through the issue_id index the first
time a walk reaches it: the batch's own endpoints up front, then one
IN-list read per BFS level. Memory and reads scale with the subgraph
the batch's walks touch, not the edge tables, so a two-edge create on a
large database no longer materialises every edge. A written edge whose
source was never loaded loads that source (row included).
- The blocked-state no-edge probe runs only when a create asks for it or
when a chunk holds at least 32 ids. Single-issue recomputes on close,
reopen, update and dependency edits issue exactly the statements they
did on main (the reopen sqlmock expectation is back to main's).
- Multi-row issue INSERTs are bounded by an estimated 16 MiB of bound
text as well as 100 rows. When a multi-row statement fails, its rows are
replayed one at a time, and the batch continues if every row lands
instead of returning the original error.
- Nits: document the binary-collation assumption behind Go string
equality; DisableCreateFastPathsForTest documents that it is
process-global and non-parallel, and panics on an overlapping caller;
drop the unreachable skip branch; explain why wisp targets need no
presence read.
Re-pinned TestLargeBatchApplyStatementCounts* classic 40: 793 -> 786
(the +7 probes on dep-add recomputes are gone; net vs main is -60, all of
it ExecuteCreate's config reuse). 356/712 unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): search the batch dependency graph from both ends
The lazy per-node load (previous commit) still walked forward from the
check's start, so on a large stored graph a single create's cycle check
read every node reachable from its target — about 800 statements per
create on a 2000-issue fixture. The cycle and hierarchy questions a batch
asks almost always end at a row the batch just created, whose only
incoming edges are the batch's own.
depGraph now answers each reachability question with a bidirectional
search: forward from the start along outgoing edges (issue_id index),
backward from the goal along incoming edges (the three target-column
indexes, read as one UNION ALL per table), always expanding the smaller
frontier and loading a level's nodes in that direction once. The batch's
endpoints are loaded in both directions up front, so a create's checks
usually finish with no further read (a chain import, too: each new
edge's goal has no incoming edge yet). Rows are keyed by (table, source,
target), so a row read from both ends counts once. An incoming read
ignores a row whose matching column is not its first set target column,
as the CTE's COALESCE does. A written pair that neither loaded end can
vouch for is re-read instead of assumed new.
On a 50k-issue / 100k-edge database, a create with two blocks edges
now runs 27 statements, against the full-scan or per-node walks before
(numbers in the review-fix summary).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): hold batch create to golden digests from the pre-change code
The batch-create equivalence test compared the fast paths with the same
code's per-row bodies, so changes the fast-path switch does not gate —
the recompute seed merge, the IN-list children read, ExecuteCreate's
context reuse, the prepare/finish split — were checked only by older
contract tests. Its batch was also too small to cross a chunk boundary
and never set RejectStaleUpserts.
createbatchequiv now runs four scenarios, each through the fast and
per-row bodies, and checks both against a per-table row count and
SHA-256 recorded from ebe3b6bcb (before the fast paths existed;
CREATEBATCHEQUIV_GOLDEN_DIR re-records them):
- small: the original scenario;
- import458 and import458-reject-stale: the review's 458-issue batch
(every chunk boundary, wisps and cross-plane edges, cycles, hierarchy
conflicts, duplicate ids across deferred runs, generated and
hierarchical ids, stale upserts rejected and accepted), plus new edges
whose blocked-state fallout reaches stored rows outside the batch (a
spawner's waiter, an upserted parent's existing child);
- apply: an apply-batch through ExecuteCreate, dep adds and an update.
The recorded digests match on the embedded and server engines (the
server test now sets the fixture's prefix, so generated ids agree).
Dropping the spawner-waiter seeding is invisible to the fast-vs-per-row
comparison and caught by the golden check.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): large-graph create benchmark
batchbench measures creates against a database already holding a large
dependency graph (batchfixtures.LargeGraphInserts: N issues, ~2N
acyclic edges with wide reachable sets and a deep parent-child forest,
committed as a Dolt version): ExecuteCreate under an existing parent with
an existing blocker, a store create with two blocks edges, and an
apply-batch of creates each with a parent-child and a blocks dep_add.
It logs wall time, statements and Go allocation per shape on the
embedded and server engines. Opt in with BEADS_BENCH_LARGE_GRAPH=<issues>
(and BEADS_BENCH_LARGE_GRAPH_CREATES); it asserts nothing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): read incoming dependency edges one target column at a time
The bidirectional graph search read a node's incoming edges with one
UNION ALL over the three target columns per table. Dolt plans that as a
scan: 0.7 s per read on a 100k-edge table, where each leg alone is a 1 ms
lookup on its own index (EXPLAIN shows IndexedTableAccess per leg). Issue
the three legs as separate reads into one row set; a row matched through
two columns still counts once.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): run the 458-issue equivalence scenarios outside the race lane
The 458-issue golden scenarios took the embedded tier's shard over its
20-minute limit under -race (the in-process engine's own instrumentation;
about 35 s each without it). They move to
TestCreateBatchFastPathsMatchPerRowLarge_Embedded, skipped under -race and
added to nightly.yml's non-race embedded step, following the precedent of
the 712-item statement-count and 1000-item wall-clock cases. The light
scenarios (small, apply) stay in TestCreateBatchFastPathsMatchPerRow_Embedded
on every lane. The server backend (built without -race in its full
suite) runs both: TestCreateBatchFastPathsMatchPerRow_Dolt and, as its own
top-level test so the shard manifest's hash places it on another shard
(about four minutes there), TestCreateBatchFastPathsMatchPerRowLarge_Dolt.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): zstd cache transfers for the anonymous fork cache only, behind RBE_CACHE_ZSTD (E4)
E4 of the rbe-west egress reduction (design A.5), reworked for the
read-only zstd server shape. rbe-west's trusted MAIN and OSS schedulers
(:443 to :50052) and rbe-fork (:8444) advertise no compressor, and Bazel
refuses a remote that does not advertise zstd. So only the anonymous fork
cache (rbe-cache :8443, nativelink-anon, which keeps zstd) may ask for it.
- **`write-bazelrc.sh`:** no `--remote_cache_compression` on the trusted
remote-exec or rbe-fork paths. In mode cache, with `RBE_CACHE_ZSTD=1`, it
adds `build:fork-cache --remote_cache_compression`. Empty or 0 adds
nothing; another value warns and stays off, rather than failing every
cache lane.
- **`bazel.yml`:** every setup-bazel caller passes
`RBE_CACHE_ZSTD: ${{ needs.rbe.outputs.mode == 'cache' &&
vars.RBE_CACHE_ZSTD || '' }}`, so only cache runs see the variable.
Merging changes nothing, and rollback is the variable, with no revert.
- **`.bazelrc`:** fork-cache no longer sets the flag; a comment points at
the gate.
Tests:
- `TestBazelCacheModeReachesTheRC` evaluates `vars.` right-hand sides and
simulates `RBE_CACHE_ZSTD=1`. Mode cache's rc carries exactly the
fork-cache zstd line; the remote, fork-ro, fork-rw and local modes carry
no compression at all.
- `TestSetupBazelRCWriter` runs `RBE_CACHE_ZSTD` = 1, 0, empty and a bad
value in mode cache, and remote-exec with 1 (never compressed).
`TestSetupBazelRCWriterForkMode` pins rbe-fork's exact lines with
`RBE_CACHE_ZSTD=1`.
- The bazel.yml setup-env pin includes `RBE_CACHE_ZSTD`.
- `checkBazelrcForkCache` rejects the flag in any `.bazelrc` config, with
fork-cache and remote-exec fixtures.
- New `TestBazelRemoteCacheCompressionOnlyForkCache` scans `.bazelrc`, the
workflows and the local action files. The flag must appear once: the
writer's fork-cache echo.
**Merge order:** after rbe-west's anonymous process advertises zstd. Even
then the flag stays off until `RBE_CACHE_ZSTD=1` is set. If rbe-cache stops
advertising zstd, fork-cache's fallback flags make lanes build locally.
**Action keys:** unchanged; the flag only changes ByteStream resource
names (`compressed-blobs/zstd/`).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): zstd for the fork cache by a GetCapabilities probe, not a repository variable (E4)
Review finding 3: GitHub gives fork pull_request runs no repository
variables (bazel.yml: forks "see no secrets and no vars"), so
vars.RBE_CACHE_ZSTD was always empty exactly where the switch matters. Fork
run 37220594372 logs RBE_VAR_ON: false. The variable could never turn zstd
on for fork-cache clients, and once on it could not follow rbe-cache
rolling zstd back.
- New setup-bazel/cache-zstd-probe.sh, a byte copy of gascity's
tools/rbe/cache-zstd-probe.sh: one unary REAPI GetCapabilities call
(instance oss) to rbe-cache :8443 over curl's HTTP/2, decoded with
python3's standard library. It exits 0 only when
cache_capabilities.supported_compressors lists ZSTD, the field Bazel
checks. Any other result exits nonzero: not listed, an HTTP or gRPC
error, a refused connection, a timeout (connect 3s, total 5s), an
unparsable answer, or no curl or python3. It writes nothing to stdout
and one verdict line to stderr. No new action and no new dependency.
- write-bazelrc.sh: in mode cache, `if bash .../cache-zstd-probe.sh` adds
build:fork-cache --remote_cache_compression. The probe never fails the
script. RBE_CACHE_ZSTD and its bad-value warning are gone.
- bazel.yml: the ten RBE_CACHE_ZSTD env lines are gone (back to the
pre-E4 setup env).
Tests:
- New TestCacheZstdProbe runs the real script against a TLS HTTP/2
stand-in for rbe-cache, which checks the exact request. Passes: zstd
advertised (packed, and unpacked among others). Fails: rbe-cache's real
2026-10-04 answer (no compressors), deflate only, zstd for batch updates
only, zstd outside cache_capabilities, a gRPC error trailer, no
grpc-status, trailers-only UNIMPLEMENTED, HTTP 502, a compressed or
truncated message, a truncated field, not gRPC, a timeout, a refused
connection, an untrusted certificate. It also pins the probe's endpoint
and instance to .bazelrc's fork-cache.
- TestSetupBazelRCWriter: the fork cache gets exactly the zstd line after
--config=fork-cache against a zstd stand-in, and none against the live
answer or a refused port. Remote-exec and rbe-fork, with a zstd
stand-in, never probe and never carry compression.
- TestBazelCacheModeReachesTheRC again rejects any vars. in setup-bazel's
env, and no mode's rc carries compression (the probe is refused there).
- The writer harnesses default RBE_CACHE_PROBE_URL to a refused loopback
port, so no test reaches rbe-cache.
Live check, 2026-10-04: rbe-cache answers without supported_compressors,
so the writer's mode cache rc has no flag until the E1 cutover.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): reset cobra flags between in-process runs; keep them out of parallel tests
runBDInProcess and runBDInProcessAllowError execute the package-level
rootCmd tree again and again in one process, and cobra never resets a
flag between Execute calls: a value one invocation parsed (create's
--title, --labels, list's --status, ...) and its Changed bit stayed on the
FlagSet, so the next invocation saw it, and a slice flag that had been
set appended to the stale value instead of replacing it. With a Dolt test
server (the only setup in which these tests run) that failed about 20
TestCLI_* tests depending on suite order.
TestMain now snapshots every flag of the tree (snapshotCommandFlags,
after all init()s registered them, before any test runs) and both runners
put the tree back before and after each run (resetCommandFlags): the
pointee of each flag.Value is restored (for pflag's scalar types that is
the bound variable; for slice types it also clears the internal "changed"
bit, so the next Set replaces), slice contents via SliceValue.Replace,
and Changed=false. TestResetCommandFlagsRestoresEveryFlag sets every flag
of the real tree and checks the reset. The ad-hoc --title reset in
TestCLI_CreateRejectsEmptyTitle_ProxiedServerMode is no longer needed.
Seven TestCLI_Comment* tests called t.Parallel() around the runners.
inProcessMutex serializes the runners only against each other, so their
chdir, stdio swap, env and config re-initialization raced every other
parallel test (data races on config globals; TestCobraParallelPolicyGuard
failing to read its own source files after a concurrent chdir). They run
serially now, and the guard treats the two runners like rootCmd.Execute.
Also two stale TestCLI_Create* cases: DryRunWithRigPrefix exercised a
create --prefix flag that no longer exists, and the flag-like-title cases
passed "--help" or "--foo-bar" where cobra parses them as flags; they now
pass the title after "--", the only way it reaches create's own guard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix the ADO round-trip and bulk-delete tests that rotted unseen
Both need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd (BEADS_TEST_SKIP=dolt everywhere), so they failed unnoticed:
- TestADORoundTrip* stored ado.pat in the database. ado.pat is a secret
read only from config.yaml or AZURE_DEVOPS_PAT (config.IsYamlOnlyKey),
so the tracker found no PAT. The fixture sets AZURE_DEVOPS_PAT.
- TestBulkDeleteNoResurrection expected GetIssue to return (nil, nil) for
a deleted issue; the Dolt store reports storage.ErrNotFound.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix parity, completion and notion tests that only fail with a Dolt server
These are Dolt-gated (integration tag + test server), so no CI lane ran
them and they rotted:
- TestParityCreate*/TestParityReopenErrorExits1Silently: bd create
auto-routes via routing.DetectUserRole("."), which warns "beads.role
not configured (GH#2950)" on stderr when the working directory's git
config has no beads.role. The suite pins the configured case, so the
parity env now sets beads.role=maintainer in a test-owned global git
config ($XDG_CONFIG_HOME/git/config; the role lookup scrubs every
GIT_CONFIG_* variable, so those cannot be used).
- TestIssueIDCompletion: issueIDCompletion resolves a database path
before looking at the open store and returns nothing when there is
none; the test now sets dbPath to its store's path.
- TestIssueIDCompletion_UsesWorktreeFallbackWhenStoreNil and
TestGetNotionConfigReadsDBPathWhenStoreUnset: the code under test
reopens the store from metadata.json, which names a database but not
the per-test branch of the shared database that newTestStore uses, so
the reopened store saw none of the test's rows. They now use their own
database (newTestStoreIsolatedDB).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix stale doctor/e2e/autocommit/readonly Dolt-gated tests
These need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd, so they rotted unseen:
- TestE2E_DoctorFixMetadataRoundtrip, TestE2E_MigrateDoltMetadata,
TestDoctorCheckHealthReportsVersionMismatchOnRepoLocalPort: bd init now
defaults to embedded mode, where `bd sql` and `bd dolt start` are
refused. Init with --server on a repo-local server of their own
(repoLocalServerEnv: the suite's shared-server port vars cleared, server
stopped at cleanup) and start it explicitly, since BEADS_TEST_MODE=1 never
auto-starts one. MigrateDoltMetadata's DELETE also needed `key` quoted.
- TestDoctor{Conventions,Pollution}MaxRows_EnvOnly_Exits2: bdRunRaw drops
every BEADS_* variable, so the subprocess hit dolt.New's test-database
firewall (exit 1) before the row cap; pass BEADS_TEST_SERVER=1.
- TestDoltAutoCommit_On_WritesAdvanceHead: the embedded engine keeps its
data under .beads/embeddeddolt, not .beads/dolt; findDoltRepoDir looks
in both.
- TestValidateCheck_AllClean: bd doctor validate runs five checks since
the cross-table duplicates check was added.
- TestConfigValidateReadOnlyIsHermetic: readonly commands take a shared
workspace gate, whose lock file (.beads/dolt.gate.lock) workspacegate
creates on first use by design; compare the .beads tree without gate
lock files.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): undo each in-process execution's env and storage-mode leaks
Every in-process run of the bd command tree (runBDInProcess and the many
tests that call rootCmd.Execute directly) leaves process state behind:
PersistentPreRun exports the selected workspace as BEADS_DIR, loads its
.beads/.env, and records the workspace's mode in serverMode /
proxiedServerMode. With a Dolt test server, later tests in the same shard
then resolved a deleted fixture ("no active beads workspace found",
"migration sidecar is missing"), ran subprocess bd against the wrong
workspace ("recorded as proxied-server in .beads/metadata.json"), or took
the proxied path in RunE-level tests ("proxied-server UOW provider not
initialized", a nil-UOW panic in reopenProxiedResolve that killed a shard).
TestMain now registers a cobra initializer/finalizer pair
(installExecuteIsolation) that snapshots the environment and the two mode
globals when an execution starts and restores them when it ends; the
runners' restoreEnvSnapshot shares the helper. TestDoltConfigEnvironmentOverrides
used os.Setenv + defer os.Unsetenv, which deleted the suite's
BEADS_DOLT_SERVER_PORT (the test server's port) for every later test; it
uses t.Setenv now.
Full integration-tagged cmd/bd suite against a hermetic dolt (16 shards):
18 order-dependent failures and the shard panic gone; the 27 remaining
failures all fail alone too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(init): keep --quiet quiet through the Claude/Codex/Cursor auto-setup
`bd init --quiet` ("Suppress output") still printed the project
integration installers' progress to stdout ("Installing Claude hooks for
this project...", "Claude Code integration installed", ...): the
installers write to os.Stdout unconditionally. Point stdout at the null
device around them while quiet; their failures keep going to stderr as
before. Found by TestInitCommand's quiet case, which asserts no output and
had not run in CI (Dolt-gated).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(init): bring the Dolt-gated init tests up to the current init contract
These tests need a Dolt test server, so no CI lane ran them and they
drifted from the product:
- Embedded is init's default mode, so the server-mode assertions
(.beads/dolt, opening the database on the test server, dolt_mode
"server" in metadata.json) now pass --server (TestInitCommand,
TestInitAlreadyInitialized, TestInitRedirect, TestInitBEADS_DIR,
TestInitDoltMetadata, TestInitDoltMetadataNoGit, TestInitDatabaseFlag's
metadata subtests). The in-process ones reset rootCmd's flags around
themselves so --server cannot leak into later in-process inits.
- Each server init gets its own --database: the suite shares one test
server, and a "test" database another test created belongs to another
project (init refuses: PROJECT IDENTITY MISMATCH).
- The marker tests seed a current-era version witness: the legacy-upgrade
guard refuses a non-empty .beads/dolt without one.
- The redirect test provisions the canonical target with config.yaml: bd
ignores a redirect to a directory with no project files (#4692).
- Shared-server inits get a free port of their own
(sharedServerTestEnv): the inherited BEADS_DOLT_SERVER_PORT is the
suite's test server, which a shared-server start refuses as another
project's, and the default shared port is 3308, a developer's live
server.
- --shared-server's config.yaml is now nested YAML (dolt: shared-server:
true); parse it instead of matching the old flat line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): run the Dolt-gated tests that skipped on a bare dolt.New
setupTestDB (config, kv and Linear round-trip tests), setupGatedTestDB
(mol ready --gated) and the rename-prefix and bd sql tests opened stores
with dolt.New(&dolt.Config{Path: ...}): no server port, no
CreateIfMissing, so against a running test server they got "database ...
not found" and skipped as "Dolt server not available". They now use the
suite's newTestStore helpers (skip only when no test server runs), and
what that exposed is fixed test-side:
- TestCustomStatusConfig: custom_statuses is read back ORDER BY name;
look statuses up by name / expect name order.
- TestLinearRoundTripCoreFields, TestLinearPullMilestonesCreatesEpicHierarchy:
linear.api_key is a yaml-only secret (LINEAR_API_KEY), push now needs an
explicit linear.state_map, and batch push uses TeamLabels and
issueBatchCreate, which the mock server now answers.
- TestSqlCommand: bd sql refuses in the default embedded mode; set
serverMode for the server-backed store, and stop discarding RunE errors.
- TestE2E_AutoStartedRepoLocalServerPersistsAcrossCommands skipped on
this process's embedded mode although every bd it runs is a subprocess
on a --server workspace; gate on the dolt binary instead and stop the
auto-started server at cleanup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): no fixed "unused" port, no swallowed JSON, no leaked front-door server
Three more things the Dolt-backed cmd/bd run exposed:
- TestTestServerConnection, TestSharedServerCfgNilHonorsSharedServer and
the two TestLegacyGuard* e2e tests pointed bd at 127.0.0.1:59999 as a
port nothing listens on. Any process on the host may hold it; a
listener that accepts and resets turns the expected "connection
refused" / "Dolt server unreachable" into another error. They take a
just-freed port from closedLoopbackPort (testutil.FindFreePort) now.
- TestDoltSetConfigJSONOutput and TestDoltSetConfigWithUpdateConfig
created .beads 0755, so bd's permissions warning landed in the JSON
under test and the tests skipped ("output not pure JSON") every run.
The fixture uses 0700, and non-JSON output fails instead of skipping.
- TestMigrateDoltModeFrontDoor's closing show/list auto-start a
repo-local server after the reverse migration; nothing stopped it, so
the suite's leak check failed (when the opt-in
BEADS_TEST_MIGRATION_FRONTDOOR suite runs). It stops it at cleanup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): advisory cmd/bd Dolt-server tier (bazel-cmd-dolt)
No CI lane ran cmd/bd's Dolt-gated tests (TestCLI_*, the init and
store-backed suites, the doctor/e2e tests): main.yml's integration shards,
nightly's Full Test Suite, and Bazel's --config=prcore/ci and
--config=integration all pass BEADS_TEST_SKIP=dolt, so cmd/bd's TestMain
never starts a test server and those tests skip; the proxied and embedded
tiers run only their manifests' tests. They rotted unseen.
//cmd/bd:bd_dolt_server_test (tag dolt-server-cmd) runs the whole
integration-tagged cmd/bd suite, the integration build's bd_test, against
a hermetic dolt sql-server: BEADS_TEST_DOLT_SERVER=local starts it from
the pinned dolt in the test wrapper's runfiles, and
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 fails the target instead of skipping
into a cached pass. No test selection: the Go binary shards itself over
every top-level test (32 shards), so each runs exactly once, and
check_testcases.py fails a shard that ran none. --config=doltserver-cmd
has exactly --config=integration's build flags, so the two lanes share
bd_test's compile; prcore/ci and integration exclude the tag.
bazel.yml's new bazel-cmd-dolt job runs it exactly where bazel-integration
runs (same if: remotely on the farm for same-repo PRs, fork-ro/fork-rw,
and locally with the read-only cache in mode cache for forks). It is
advisory: job-level continue-on-error keeps a failure out of the call's
aggregate (BAZEL), and pr.yml's ci-gate reads its result (BAZEL_CMD_DOLT)
but requires it only while the committed top-level flag
BAZEL_CMD_DOLT_REQUIRED is "true" (now "false"; flipping it is the one
change needed). bazel-gate.sh accepts its skip in modes skip and local.
Policy tests: a third lane category in TestBazelLaneIsGatedAlongsideLegacy
(bazelFlagGatedLanes: gate env wired, never statically required, added by
the run script exactly when the pinned flag is "true"), the gate
simulation over every mode with the flag off (advisory: nothing it
reports reddens the gate) and on (exactly a gated lane), the cache-mode
and execution-mode checks, the tag taxonomy and prcore/integration
exclusions, and TestBazelCmdDoltJob (job steps, timeouts, .bazelrc
config equal to integration's build with no dolt skip or test selection,
the target's dolt-server rules, and bd_test's env repeated).
Measured locally (all fixes in this series, 32 shards, 16 at a time on a
shared host): 3267 top-level tests, all shards pass, max shard 313s (one
5001-issue export test, 214s, sets it), mean 100s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(init): hand the project installers a writer instead of swapping os.Stdout
The --quiet fix earlier in this series pointed the process-global
os.Stdout at the null device around the Claude/Codex/Cursor installers.
That works, but it mutates a global from inside a cobra Run: anything else
writing to stdout meanwhile (a goroutine, a future concurrent installer)
would vanish, and a concurrent read of os.Stdout is a data race.
The installers already write through env structs with stdout/stderr
fields. setup.ProjectInstallOutput carries the two writers, and
InstallClaudeProjectTo / InstallCodexProjectTo / InstallCursorProjectTo
use them (the old entry points keep the process streams). init passes
QuietProjectInstallOutput() under --quiet: progress discarded, error lines
still on stderr. The proxied-server init path (domain adapter
InstallClaudeProject now takes quiet) does the same, so its
"Installing Claude hooks..." is quiet too.
Consistent error policy under --quiet: errors still reach stderr. init's
own "Warning: failed to setup <installer>" lines were suppressed under
--quiet while the installers' "Error:" lines were not, and some installer
failures (Cursor rules) print nothing themselves; the warnings now print
whether or not quiet, like the installers' errors.
Coverage in required lanes: TestProjectInstallersReportToTheGivenOutput
(cmd/bd/setup) and TestEmbeddedInitA/basic, which now asserts that
`bd init --quiet` with agents enabled installs the Claude hooks and prints
none of the installers' progress (verified to fail with the quiet output
disabled).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(export): exercise the incremental-export fallback with a small threshold
TestTryIncrementalExport_ThresholdExceededFallsBack seeded
incrementalExportThreshold+1 = 5001 issues to cross the threshold. Batch
create is one transaction but still several statements per issue, so under
-race against the shared test server it took ~214 s, the longest test in
cmd/bd's Dolt-backed suite by far. The behaviour under test is only
"more changed issues than the threshold -> full export fallback".
incrementalExportThreshold becomes a var (same value), and the test, which
is not parallel, sets it to 20 and seeds 21 issues: 214 s -> 0.6 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): tighten the flag-reset self-test, guards and canaries
Review follow-ups:
- TestResetCommandFlagsRestoresEveryFlag no longer skips a flag it cannot
set: a flag type without a sample value, a Set error, or a non-pointer
Value (which resetCommandFlags could not restore) is an error, so a new
flag type cannot go untested. resetCommandFlags documents that it also
resets flag-bound package globals.
- TestCobraParallelPolicyGuard treats .ExecuteC/.ExecuteContext/
.ExecuteContextC like .Execute (the per-execution isolation writes the
mode globals; a parallel execution would race).
- TestConfigValidateReadOnlyIsHermetic ignores only the gate a readonly
command takes (.beads/dolt.gate.lock and its .info sidecar), not any
"*.gate.lock*" file, so a new unexpected gate still trips the canary.
- TestCLI_CreateRejectsFlagLikeTitles: a bare `bd create --foo-bar` must
fail and create no issue (bd-2c0's intent), next to the "--" cases.
- TestCustomStatusConfig cites why statuses come back in name order: the
normalized custom_statuses table (#2961, 8537d943b) has no ordinal and is
read ORDER BY name, like custom_types; the test predates it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the cmd/bd Dolt-server tier in 16 shards
Each shard pays ~30-40 s of fixed cost (TestMain plus its own dolt
sql-server), so 32 shards spent roughly 40% of the lane's executor time on
setup and added 32 farm actions to every PR touching cmd/bd. With the
5001-issue export test cut to under a second, 16 shards measure (locally,
--nocache_test_results): max 186 s, mean 137 s, 2,187 executor-seconds
(vs ~2,800-3,200 at 32), all 3,267 top-level tests, every shard passing.
TestBazelCmdDoltJob pins the count; bazel.yml's timing comment is
restated (cache mode ~20-30 minutes).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(git): canonicalize PinNoRepositoryUnderForTesting root so the pin matches on macOS
On macOS t.TempDir()/os.TempDir() live under /var/folders/..., but /var is
a symlink to /private/var and getcwd(2) reports /private/var/folders/....
underPinnedRootForTesting compared the raw pin against os.Getwd() with a
string prefix test, so the pin never matched there and
TestPinNoRepositoryUnderForTestingSurvivesResetCaches failed on every Main
macOS run (GetRepoRoot() returned the /private/var repo instead of "").
Store the pin both as given (absolute) and symlink-resolved, and match the
working directory in both raw and resolved form. Canonicalization can only
widen the fence, never narrow it (e.g. Bazel runfiles symlink trees keep
matching exactly as before).
TestPinNoRepositoryUnderForTestingMatchesThroughSymlinks reproduces the
macOS shape on any platform with a symlinked repo root; it fails without
this change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): make run-go-test-binary.sh and install-golangci-lint.sh parse under bash 3.2
The Main macOS job runs the scripts package tests with macOS's /bin/bash
3.2, where two constructs abort under `set -u`:
- run-go-test-binary.sh expanded the usually-empty PASSTHROUGH array bare;
bash < 4.4 treats that as an unbound variable ("PASSTHROUGH[*]: unbound
variable"), failing TestTestScriptPrebuiltTestBinaryContract and
TestRunGoTestBinaryDefaultsTimeoutWhenOmitted. Use the ${arr[@]+...}
guard.
- install-golangci-lint.sh used `declare -A`; bash 3.2 parses it as an
indexed array and `[amd64]=` dies as an arithmetic lookup ("amd64:
unbound variable"), failing
TestInstallGolangciLintFailsClosedOnChecksumMismatch. Replace it with a
case function and update TestGolangciLintInstallScriptPinned's pin check.
Reproduced and verified against a bash 3.2.57 build: all four tests fail
on origin/main and pass with this change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
actions/create-github-app-token (v3.2.0, pinned) deprecates its app-id
input in favor of client-id. rbe-prewarm's mint step now passes the
bazel-allocator App's public Client ID (Iv23ligrqVEhlamZnoPU, from
GET /apps/bazel-allocator) as a reviewed literal: it is not a credential,
and a literal keeps it policy-tested with the rest of the step instead of
in an unversioned repository variable. The private key stays a secret.
RBE_POOL_APP_ID is now unused, so its workflow_call declaration in
bazel.yml and its pass-through in pr.yml and nightly.yml are removed,
and a policy test rejects any workflow mentioning it again. The
repository secret can be deleted.
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md's "rbe-west Pre-warm" section now
documents the client-id, the single remaining secret, and
RBE_PREWARM_WORKERS (default 1, clamped to 4, 0 = kill switch, repo
variable currently 2). It also no longer cites a maintainer's local
planning-notes path; a new tracked-file policy test fails if one
reappears.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The prebuilt twins of the Windows liveness and worktree-remove jobs passed
alongside the native ones on every PR since F4 (#7135): #7139, #7145,
#7147, #7155, #7156, #7159, #7160 — 1-1.5 min vs 3-8 min native. Flip the
committed flag so ci-gate enforces the prebuilt pair; the native pair keeps
running as advisory and is the rollback (set the flag back to "false").
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): best-effort pre-warm rbe-west's OSS worker pool on PR open
bazel.yml's Bazel lanes each separately wait ~120s at "N actions, 0
running" while gastownhall/gascity's rbe-west NativeLink worker pool
scales up from zero. Add an advisory rbe-prewarm job (never a `needs`
of any lane, never in bazel-gate.sh's vocabulary, job-level
continue-on-error so it can never fail bazel.yml's own run) that
mints a short-lived installation token for a "bazel-allocator" GitHub
App (installed on gastownhall/gascity only, Actions read/write +
Metadata read) and dispatches gastownhall/gascity's
rbe-worker-pool.yml ahead of time, only when this run's lanes will
actually target rbe-west's "oss" instance (modes remote and fork-rw;
fork-ro targets a separate pool this doesn't serve).
Adds RBE_POOL_APP_ID / RBE_POOL_APP_PRIVATE_KEY as new optional
workflow_call secrets, threaded through every caller (pr.yml,
bazel-farm.yml, nightly.yml); either secret missing (the app not yet
provisioned) or repo var RBE_PREWARM_WORKERS=0 disables pre-warming
cleanly. Policy tests (scripts/bazel_rbe_prewarm_test.go, plus updates
to the existing bazel.yml structural tests) pin the mode gate, the
credential scope, the dispatch target, and that the job can never gate
or fail a run; each invariant was hand mutation-tested. Docs added to
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md ("rbe-west Pre-warm").
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): close rbe-prewarm token-theft path, fix stale secret-count docs (B1)
Security review of bdef342d5 found a BLOCKER (B1): the rbe-prewarm job
checked out inputs.checkout-sha (the fork head, reachable via
bazel-farm.yml's pull_request_target path) and then ran
.github/scripts/rbe-prewarm.sh with the minted gastownhall/gascity
dispatch token present. Script-file edits don't require workflow-approval
the way workflow-file edits do, so a VIP fork or same-repo branch pusher
could edit that script to steal the shared bazel-allocator key.
Fix: delete the checkout step and the .sh file; inline the dispatch logic
into the step's own `run:` block, whose bytes always come from the
trusted (approval-gated) workflow definition even under
pull_request_target. Mint the token in the immediately preceding step;
nothing after the mint step touches a repo-relative path. Stop forwarding
RBE_POOL_APP_ID/RBE_POOL_APP_PRIVATE_KEY from bazel-farm.yml (that fork
path now gets a cold start instead of pre-warming) and narrow the job's
`if:` to mode == 'remote' only (fork-rw never carried secrets anyway, so
it was dead weight - S1).
Also addresses the review's should-fix items:
- S2: document the widened trust boundary (the bazel-allocator key is
shared with gascity's own rbe-west scaler; leak blast radius; rotation
breaks the scaler, so the kill switch is unset-secret /
RBE_PREWARM_WORKERS=0, not rotation) as an accepted risk in bazel.yml's
header and engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md.
- S3: fix stale "exactly four secrets" claims (pr.yml/nightly.yml pass 6;
bazel-farm.yml now correctly passes 4 again) and the false "never a
long-lived credential" claim.
- S4: flag the continue-on-error-preserves-caller-success assumption as
reasoned-through but not live-validated in this sandbox.
- N1: drop the unused RBE_PREWARM_IDLE_MINUTES knob entirely.
- N4: strengthen the dispatch-target test to check exact gh command
strings and ban hardcoded repo/workflow literals outside $POOL_REPO/
$POOL_WORKFLOW.
Test changes (scripts/bazel_rbe_prewarm_test.go rewritten; ci_workflow_test.go,
bazel_farm_workflow_test.go, bazel_cache_mode_test.go updated for the new
job shape): new TestRBEPrewarmNoCheckoutStep,
TestRBEPrewarmNothingAfterMintTouchesARepoPath,
TestRBEPrewarmTokenOnlyInDispatchStep, and
TestRBEPrewarmAppSecretsOnlyExpectedCallers directly target the B1
mutation class; TestRBEPrewarmDispatchScriptBehavior keeps the fake-gh
behavioral coverage for the now-inlined dispatch script. Verified by hand
that both of the reviewer's mutations (widen the `if:` back to include
fork-rw; add a step after mint running a checked-out script) are now
caught.
Verification: go test ./scripts/..., go vet, gofmt, actionlint on the
four touched workflow files, make ci-pr-policy, make bazel-sync-check,
and `bazel test //scripts:scripts_test` all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>