* ci: retire PR Risk's legacy proxied and server-Dolt tiers where the gated Bazel lanes run them (D2 step 2)
On same-repo, non-Dependabot pull_requests, PR Risk's test-proxied-cmd (15),
test-server-storage and test-server-storage-full (16) jobs stand down while
the committed BAZEL_RETIRES_LEGACY_DOLT_SERVER_TIERS flag is "true" (in both
pr.yml and pr-risk.yml, policy-tested equal); pr.yml's gated Bazel
proxied-server and server-Dolt storage lanes are then the tiers' only
pre-merge run. build-embedded, whose embedded-test-binaries artifact feeds
exactly the six retired legacy test jobs, stands down where both D2 steps
apply.
Generalizes step 1 instead of copying it:
- bazel-embedded-coverage becomes bazel-coverage (identical in both
workflows): one decision step, outputs embedded and dolt_server, each
flag && pull_request && !fork && actor != dependabot[bot]; gate id
BAZEL_COVERAGE.
- PR Risk's gate accepts the retired jobs' skips only when their output is
true, and BUILD_EMBEDDED's only when both are.
- pr.yml's gate adds BAZEL_DOLT_SERVER_RETIRED: red when dolt_server is true
and the mode is not remote or BAZEL_PROXIED / BAZEL_SERVER_STORAGE is not
success (shared helper with BAZEL_EMBEDDED_RETIRED).
- Lane hardening as for embedded: --nocache_test_results for
doltserver-proxied and doltserver-integration, the flaky-target query,
check_shard_coverage.py (proxied 15 shards; server full suite 16 shards
plus --whole conformance), pinned test steps, rc lines and target
args/env; the shard listings now run concurrently (the server script
forks per test).
- Policy tests generalized over the retired tiers: decision equivalence
with bazel.yml's mode across two runs, both gates simulated, flag
equality, artifact consumers, real shard scripts in list-only mode.
Revert: commit BAZEL_RETIRES_LEGACY_DOLT_SERVER_TIERS: "false" in both
workflows.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record the retired legacy proxied and server-Dolt tiers in the check topology
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): no whole-invocation retry on remote cache eviction in the retired tiers' lanes
Review F1 (D2 step 2): Bazel 9 re-runs a whole `bazel test` invocation
(--experimental_remote_cache_eviction_retries, default 5) when an input was
evicted from the remote cache, and that exit code outranks a test
failure. With --nocache_test_results every test runs again and only the
retry is reported, so a flaky failure in the embedded, proxied or
server-storage lane (each now its tier's only pre-merge run) could turn
green. Set the retries to 0 in test:embedded, test:doltserver-proxied and
test:doltserver-integration (not remote-exec, which must stay key-neutral),
pin the lines, forbid any other line or the setup-bazel action from
setting the count, and correct the topology doc, which claimed the retry
never re-runs a failed test.
Reproduced against a local HTTP cache emptied mid-build: with the default
the build hit "Lost inputs", retried and exited 0; with 0 it failed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): pin how the retired lanes' shard scripts run their tests
Review F2 (D2 step 2): the narrowing scan covered tools/bazel and the
setup-bazel action but not the four .github/scripts/*-test-shard.sh
scripts the sharded targets execute, so appending -test.short to an exec
line or exporting BEADS_TEST_SKIP there passed the policy tests (and
check_shard_coverage.py only rejects a shard that is entirely skipped).
Pin each script's prebuilt-binary and go test fallback commands exactly
and its RUN_REGEX construction, and forbid anything else in the scripts
from running tests or selecting, skipping or switching them. The scripts
themselves are unchanged (the legacy jobs run them too); a self-test
checks that six narrowing edits are rejected.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): simulate a non-true coverage decision per retired tier
Review F3 (D2 step 2): the "decision succeeded but its output is not
exactly 'true'" gate scenarios set both bazel-coverage outputs bad at
once, so the embedded ids reddened the gate whatever the dolt_server rule
said, and weakening PR Risk's gate to `!= "false"` for one tier passed.
Run them per tier, with the other output a valid "false" and only that
tier's job skipped, and isolate build-embedded's both-true rule the same
way. Failed or cancelled decisions keep the all-skipped scenarios.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): list Bazel tier coverage and the proxied/server-Dolt checks as not directly required
Review F4 (D2 step 2): the topology doc still named the renamed
`Bazel embedded coverage` job, and its list of check names not to require
directly lacked the proxied and server-Dolt test jobs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): skip the real shard-script listing test on bash < 4
The CI shard scripts use associative arrays and only run on Linux runners;
macOS's /bin/bash 3.2 can't execute them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(pr-risk): stand down legacy embedded tier where the gated Bazel lane runs it
D2 step 1. A new bazel-embedded-coverage job decides, from the same
expressions as bazel.yml's rbe job, whether pr.yml's Bazel call takes
execution mode remote on a same-repo pull_request (RBE_WEST_WORKERS on,
executor secret present, not a fork). Only then do test-embedded-storage,
test-embedded-conformance and test-embedded-cmd (27 jobs) skip, and PR
Risk's gate accepts those skips only then; a failed or missing decision is
red. Forks, Dependabot, secret-less runs, the farm switch unset and
merge_group keep the legacy tier. build-embedded keeps running: its
artifact feeds the proxied and server Dolt jobs.
The decision job reads no repository code and the secret only as an
emptiness test. scripts/pr_risk_bazel_coverage_test.go runs both actual
decision scripts over every event/var/secret/fork combination and requires
covered <=> (pull_request && Bazel mode remote), pins the shared env
expressions, checks pr.yml still requires BAZEL_EMBEDDED and shares PR
Risk's triggers, and simulates PR Risk's gate over every tier/decision.
Revert: unset RBE_WEST_WORKERS.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record the retired legacy embedded tier in the check topology
Who keeps the legacy tier, how the decision and gate work, the drift
guard, the residual cross-run race, the conformance.yml duplicate, and
how to revert (unset RBE_WEST_WORKERS).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(pr-risk): retire the embedded tier by committed flag; pr.yml gate fails closed
Review F1. PR Risk's coverage decision read the mutable RBE_WEST_WORKERS
variable in a different run from pr.yml's rbe job, so unsetting it and
re-running one workflow could leave both required checks green with
neither the legacy tier nor the Bazel lane having run the tier.
- Both workflows now commit BAZEL_RETIRES_LEGACY_EMBEDDED (policy-tested
equal) and run the identical bazel-embedded-coverage job:
covered = flag && pull_request && !fork && executor secret present.
RBE_WEST_WORKERS is no longer read by PR Risk at all.
- pr.yml's ci-gate requires the job's result (BAZEL_EMBEDDED_COVERAGE)
and BAZEL_EMBEDDED_RETIRED, which is red when covered but the Bazel
embedded lane did not run remotely and pass (mode skip included).
- Revert is a commit of the flag to "false" in both workflows; in-flight
PRs pick it up on their next push. The kill-switch runbook now flips
the flag first.
Tests run both decision steps, bazel.yml's rbe step and pr.yml's real
gate step over every event/var/secret/fork/flag combination, with the
variable differing between the two runs, and require no double-green
without the tier.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): never cache embedded-tier test results
Review F2. Since D2 step 1 the Bazel embedded lane is the tier's only
pre-merge run on same-repo PRs, so a cached PASS (stale, or a poisoned
entry in the shared farm action cache) or a cached run that never
re-exercises an intermittent -race finding would stand in for the legacy
jobs' -test.count=1 run. test:embedded now sets --nocache_test_results,
and the policy test requires it and forbids any other rc line that turns
test result caching back on.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): forbid flaky-test retries in the gated Bazel lanes
Review F3. Adding --flaky_test_attempts to a lane's config or
flaky = True to a lane target passed every policy test, and would hide
a flaky failure the legacy jobs (which ran each test once) would have
reported; since D2 step 1 nothing else runs the embedded tier on
same-repo PRs. TestBazelGatedLanesNeverRetryFlakyTests rejects
flaky_test_attempts and runs_per_test_detects_flakes in .bazelrc,
.github (workflows, setup-bazel's generated rc) and tools/bazel, and
flaky = True in every BUILD file and .bzl macro.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): pin the Bazel embedded lane's command line and config
Review F4. An extra --test_arg=-test.short/-test.skip on bazel.yml's
`bazel test //... --config=embedded` line, or a test:embedded
--test_filter in .bazelrc, narrowed the tier while every policy test
passed (check_testcases only rejects a shard with zero tests). Since D2
step 1 this lane is the tier's only pre-merge run on same-repo PRs, so:
- bazel-embedded's test step must be exactly the pinned script;
- the --config=embedded lines must be exactly the pinned list, in order;
- no unconfigured common/build/test line, and nothing setup-bazel's
generated rc writes, may select or narrow tests (test_filter,
test_arg, -test.short/run/skip, tag/size/lang/timeout filters).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): add the D2 policy tests to scripts_test
scripts/pr_risk_bazel_coverage_test.go was missing from
//scripts:scripts_test's srcs, so `bazel test //scripts:scripts_test`
(and the BUILD-sync check) did not see it. The repository-walking
flaky-retry test skips under Bazel, whose runfiles hold no other
package's BUILD files.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): check every listed embedded test ran in its Bazel shard
Review F5. The shard scripts discover tests from source with grep, but
each Bazel test binary holds only its target's srcs, so a discovered test
the binary lacks matches nothing in the shard's -test.run selector and
passes silently (check_testcases.py only rejects a shard with no tests).
tools/bazel/check_shard_coverage.py runs each shard script in list-only
mode (BEADS_TEST_SHARD_LIST_ONLY=1) and requires every Bazel shard's
test.xml to hold exactly the listed top-level tests: missing, extra,
wrong-shard, shard-count mismatch, missing test.xml and a failing script
are errors. bazel-embedded runs it after the tier for
//cmd/bd:bd_embedded_test (20) and embeddeddolt_embedded_test (5); the
policy test pins the suites to pr-risk.yml's jobs and shard counts, and
a fixture test exercises the script.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): correct the D2 enforcement, re-run and merge-queue claims
Review F6.
- The beads-only ruleset enforces both gates on main only; release/**
PRs run them but merging does not wait for them, before and after D2.
- There is no merge queue and strict is off: merge_group runs are not a
pre-merge safety net for the retired tier.
- Spell out "Re-run all/failed jobs" semantics under the committed flag,
the remaining secret-deletion window, and that the revert needs a new
push to in-flight PRs.
- Record the lane hardening the retirement relies on (no result caching,
no retries, pinned selection, per-shard coverage check).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): drop TestMain from the embedded shard-coverage check
Re-review G1 (blocking). embedded-storage-test-shard.sh discovers tests
with `grep '^func Test'`, which also lists TestMain(m *testing.M)
(internal/storage/embeddeddolt/test_fixture_test.go). It is never a
<testcase>, so check_shard_coverage.py reported it missing on every real
remote run, turning bazel-embedded and CI Gate red on every same-repo PR.
The checker now drops NOT_TESTS = {TestMain}. The shard scripts are left
unchanged so the legacy jobs' -test.run selection stays byte-for-byte the
same (dropping TestMain there would be harmless to shard assignment,
which hashes per name, but would touch the legacy CI scripts this step
does not otherwise change).
TestEmbeddedShardScriptsListOnlyRealTests runs both real shard scripts
in list-only mode for every shard and requires each listed name to be a
`func Name(t *testing.T)` in the package, or in NOT_TESTS (which may hold
only TestMain, and only if it is not a test). The fixture shard script
now lists TestMain too, so the filter itself is exercised.
Verified on a real remote run (`bazel test //... --config=embedded
--config=remote-exec`, 4/4 targets, 27 test.xml, 582 top-level tests,
348 s uncached): check_testcases.py and check_shard_coverage.py both
pass (199 cmd + 381 storage tests); the previous checker failed on
TestMain.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(pr-risk): decide embedded coverage by actor, not by the executor secret
Re-review G2. covered still read secrets.RBE_WEST_EXECUTOR, which an admin
can delete between PR Risk's run and a re-run of pr.yml: PR Risk stayed
green with the legacy tier skipped, while the re-run computed covered =
false, took mode local and passed with no embedded lane.
Both workflows' bazel-embedded-coverage now decide
covered = flag && pull_request && !fork && github.actor != dependabot[bot]
and read no secret or variable at all (PR Risk now reads no secret).
Dependabot (no Actions secrets) keeps the legacy tier; github.actor,
unlike triggering_actor, survives a re-run by someone else. A same-repo
PR without the secret is covered, so pr.yml's gate goes red (mode local)
instead of passing.
The two-run test matrix now varies the secret as well as RBE_WEST_WORKERS
between PR Risk's and pr.yml's runs, adds the actor to the facts, and
names the missing-secret case (red). Docs updated: the residual is gone,
the cost of failing closed is recorded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): close the remaining ways to narrow the Bazel embedded lane
Re-review G3. Skip-inducing edits outside the pinned --config=embedded
lines and command line still passed every policy test (reviewer
survivors m1, m2, m5, m6, m9, m10, m11), and the runtime checks counted
skipped testcases as run.
TestBazelEmbeddedLaneCannotBeNarrowed now rejects:
- any .bazelrc line of a config the lane uses (unconfigured, embedded,
remote-exec from setup-bazel's rc, and any config they reference) that
filters, narrows, re-runs or re-routes tests (test_filter, test_arg,
-test.*, *_filters, test_env=BEADS_TEST*/GO_TEST*/TESTBRIDGE*,
--config=, cache_test_results, run_under other than test_env, flaky,
runs_per_test, sharding strategy);
- any import other than the two gitignored try-imports, and any
committed rc file besides .bazelrc (try-import would load it in CI);
- -test.short/run/skip/list, BEADS_TEST_SKIP, BEADS_TEST_EMBEDDED_DOLT,
TESTBRIDGE_TEST_ONLY and similar in tools/bazel/*.sh (test_env.sh,
go_test_manifest_shard.sh, ...) and anywhere in the setup-bazel action,
and any --config= there but remote-exec;
- any change to the set of embedded-tagged targets or their args/env.
At run time check_shard_coverage.py fails a shard whose top-level tests
were all skipped, and its new --whole option applies the same check to
the unsharded conformance core/audit targets (added to bazel-embedded's
step). The real remote run still passes (no shard is all-skipped today;
one cmd shard has 1 of 9 tests skipped).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): ask Bazel for flaky test targets before the embedded tier
Re-review G4. The F3 grep wanted a literal `flaky = True`, so
`flaky = _RETRY` (or a macro default) slipped through (reviewer m3).
- bazel-embedded runs `bazel query 'attr(flaky, 1, tests(//...))'` after
setup-bazel and before the tier, and fails on any result (loading only;
about 0.2 s warm). Verified locally: with `_RETRY = True` /
`flaky = _RETRY` on embeddeddolt_embedded_test the query lists it.
- The policy grep now rejects any `flaky =` whose value is not a literal
False or 0, and TestBazelEmbeddedQueriesFlakyTargets pins the step.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): cover the zero-testcase shard in check_shard_coverage
Re-review G5. Skipping the "listed but did not run" report when a shard's
test.xml has no top-level tests survived the fixture test (reviewer m7);
check_testcases.py catches that shard at run time, but the coverage
check should not depend on it. Two fixture cases: a shard with no
testcases, and one with only a subtest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record the G1-G5 embedded-lane hardening in the topology doc
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): red — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)
* feat(ci): green — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)
Add test-server-storage-full as a sibling job to test-server-storage in
pr-risk.yml. build-embedded already compiles every top-level test in
internal/storage/dolt into one binary (embedded-test-binaries); the
existing job only ever runs -test.run '^TestConformance$' against it.
The new job reuses that same artifact and live-server setup with no new
build step, and runs everything else via -test.skip '^TestConformance$'
so previously-stranded tests (TestCreateGuard_*, federation/concurrent
suites, and diff-owned tests such as TestBenchDBPurgeDoesNotLeak on PR
#5792) get a real PASS/FAIL instead of a silent SKIP. Sets
BEADS_TEST_ENV_RUN_DOLT=1 so federation_test.go's own live-server check
fails loudly rather than skipping if this job's setup ever regresses.
Wired into ci-gate: added to needs, CI_GATE_REQUIRED, and the
tier-gated skipped_ok fallback (mirrors test-server-storage's existing
treatment in each).
Also updates two pre-existing meta-tests over pr-risk.yml that pinned
exact snapshots of the file this job necessarily changes:
scripts/ci_capability_selector_test.go's ci-gate authority snapshot,
and scripts/pull_dolt_image_test.go's retry-helper call count
(pr-risk.yml: 2 -> 3, the new job's own Dolt-image-pull step).
* chore: release gate PASS for be-pp7e (PR-CI lane for TestBenchDBPurgeDoesNotLeak)
Deploy gate record for be-pp7e, gating be-aiy5's PR-CI lane addition
(runs TestBenchDBPurgeDoesNotLeak against a live Dolt server instead of
letting checkDolt SKIP it). Review: be-w90n, verdict pass. All 7 gate
criteria PASS; two pre-existing/unrelated failures encountered during
evaluation (be-jy56, be-vf95) were attributed and excluded from the
verdict.
* test(ci): red — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)
TestPRRiskGateReachesFullServerDoltStorageSuite now asserts
pr-risk.yml's test-server-storage-full job is a fail-fast:false matrix
over shard 1..16, with the Test step invoking
server-storage-test-shard.sh, instead of pinning the old single-job
unsharded command. Fails against the current workflow, which still
runs all 1126 tests unsharded in one 15m job (be-extn).
* feat(ci): green — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)
Adds .github/scripts/server-storage-test-shard.sh (mirrors
embedded-storage-test-shard.sh: discovers internal/storage/dolt/*_test.go,
excludes TestConformance, manifest-plus-hash-fallback shard assignment) and
its shard manifest .github/scripts/server-storage-test-shards.txt, pinning
TestCloudAuthCLIRouting (~9.5m) alone on shard 1 and spreading the other
known 10-20s-heavy tests across the rest.
pr-risk.yml's test-server-storage-full job becomes a fail-fast:false matrix
over shard 1..16, Test step now runs server-storage-test-shard.sh instead of
the old unsharded single-job command that died at 256/1126 tests after
15m28s with zero failures (job 95616282508).
Documents the new lane in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md alongside the
existing embedded matrix description.
TestPRRiskGateReachesFullServerDoltStorageSuite now passes. Offline shard
partition verified disjoint and complete across all 1125 discovered tests
(BEADS_TEST_SHARD_LIST_ONLY=1 for i=1..16). (be-extn)
* test(feat): red — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)
* feat: green — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)
* chore: release gate PASS for be-23f3
Extends deploy/be-pp7e-gate (PR gastownhall/beads#5836) with be-gepv's
fix rather than cutting a fresh isolated branch: be-gepv's own bead
text and be-extn's precedent both establish this fix belongs on the
already-open PR. All 7 gate criteria PASS; two repo-wide lint/policy
failures attributed to already-tracked pre-existing issues (be-vf95,
be-jy56), confirmed untouched by this PR lineage.
be-gepv, be-6yo1, be-23f3
* fix(dolt): federation test isolation + panic (be-3c78s)
TestFederationDatabaseIsolation and TestFederationVersionControlAPIs are
federation-prototype tests that predate the suite's shared-server,
branch-per-test isolation architecture (setupTestStore/StartTestBranch)
and were never updated to account for it.
- setupFederationStore hardcoded Database: "beads" for both towns. New()
always connects in server mode against one shared Dolt server for the
whole test binary (TestMain), so Path alone doesn't isolate two stores
— Database does. Both towns silently shared one database, so Alpha
could read Beta's issues and vice versa. Give each town its own
database (beads_test_federation_<prefix>, matching the beads_test AD-01
cleanup-firewall prefix).
- TestFederationDatabaseIsolation's not-found assertions assumed GetIssue
returns (nil, nil) on a miss. It never does — issueops.GetIssueInTx
always returns a wrapped storage.ErrNotFound. Switched to
errors.Is(err, storage.ErrNotFound).
- TestFederationVersionControlAPIs hardcoded Checkout(ctx, "main") to
return to its "starting" branch, but setupTestStore already checks out
a private branch (test-<name>-<hex>) — "main" is a branch shared by
every parallel test in the binary and never has vc-001. That mismatch
is what produced the nil-pointer panic (GetIssue's error was discarded
via `_`, and the resulting nil issue was dereferenced). Capture the
real starting branch via CurrentBranch() right after setupTestStore()
and check both previously-discarded GetIssue errors before
dereferencing.
Neither bug is a production security issue: New(), Checkout(), and
GetIssue() all behave exactly as specified. Both bugs are confined to
this prototype test file's own setup/assertions, which hadn't caught up
to the isolation model the rest of the suite already uses correctly.
Verified via `go build`/`go vet`/`go test -c` (all clean) and running the
compiled binary, which now reaches and cleanly SKIPs both tests (no
Docker locally) instead of the previous isolation failure / panic
surfacing at compile-adjacent runtime. Full GREEN confirmation is via
CI, which has a working Dolt/Docker environment.
Refs be-3c78s
* chore: release gate PASS for be-42tlj
Record fail-closed candidate lane selection without changing which PR Risk jobs execute. This creates replayable evidence for capability-owned admission while branch protection and historical validation are still pending.
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot
Co-authored-by: CI Bot <ci@beads.test>