* ci: make the required gates merge-queue ready
The default-branch ruleset requires "CI Gate / Required" and "PR Risk
Gate / Required". Prepare both for a GitHub merge queue (ALLGREEN,
squash), so enabling the ruleset's merge_queue rule is the only step left.
- pr.yml, pr-risk.yml: merge_group pinned to types [checks_requested].
- D2: bazel-coverage covers every merge_group run (same repository, CI
secrets, actor github-merge-queue[bot], bazel.yml mode remote), so the
retired legacy tiers stay retired in the queue and ci-gate requires
their Bazel lanes to have run remotely and passed; a missing secret or
the farm switch off is red, as on a same-repo PR.
- Diff bases on merge_group: the package gates diff
merge_group.base_sha..head_sha (not HEAD^, which misses all but the
last commit of a rebase-method entry), the .beads/issues.jsonl guard
now runs on merge_group against base_sha (ci-gate no longer excuses
its skip), and check-doc-flags passes BD_DOCS_DIFF_BASE so CLI-docs
drift already on main cannot eject every queued PR.
- Test result reuse: on a merge_group run and on attempt 2+ of a PR run,
the six always-execute Bazel lanes append --config=reuse-test-results
(--cache_test_results=auto), so tests whose action key is unchanged
reuse the farm's recorded PASS and only what the merge changed runs.
Failures are never reused, eviction retries stay 0, and each PR's first
attempt, push to main and nightly still execute every test.
- scripts/ci_merge_queue_test.go simulates merge_group through the shared
evalGHExpr evaluator (now with true/false/null literals, hyphenated
identifiers and case-insensitive ==) and the workflows' own scripts:
triggers and gate names, concurrency groups, every pull_request field
read, the coverage and rbe decisions, result reuse, the package-gate,
.beads and doc-drift diff bases.
- engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md: new "Merge Queue" section (how
to queue, settings, what runs, result reuse rationale, ejection and
flakes, rollback).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: cache Bazel test results except nightly; queue trust and sizing
Review follow-ups to the merge-queue readiness change.
- Test results (S1): --nocache_test_results also stops Bazel writing
results, so a PR's first run never fed the cache and the merge-queue
reuse config could only hit on later reuse runs. Drop it from
test:embedded, test:doltserver-proxied, test:doltserver-integration and
test:sole-run (which keeps --experimental_remote_cache_eviction_retries=0),
and drop test:reuse-test-results and BAZEL_REUSE_RESULTS. Results are now
cached (read and write) on PRs, merge groups and pushes to main.
nightly.yml passes the new fresh-test-results input, so every `bazel
test` appends --config=fresh (--nocache_test_results) and the suite
re-executes nightly. The docker lane keeps --nocache_test_results.
Measured locally with a disk cache: a default run writes and the next
reads ("(cached) PASSED"); a fresh run neither reads nor writes.
- Security headers (S2): bazel.yml now states that a merge_group run
executes the queued commit's workflows, so a queued fork or Dependabot
PR's CI edits run with every CI secret, the pre-warm app key included
(merging-level trust), and records the maintainer rule: review fork
changes to .github/, .bazelrc, tools/bazel, MODULE.bazel, BUILD files
and CI scripts before queueing. Same note in the topology doc.
- Queue sizing (S3): recommend max_entries_to_build 3 and
check_response_timeout_minutes 60 (max_entries_to_merge 5, ALLGREEN,
SQUASH unchanged).
- Nits: the .beads guard fails closed on a missing diff base, and its
test runs under GitHub's default `bash -e`; evalGHExpr follows GitHub's
loose equality (true == 'true' is false, false == '' is true, missing
context keys are null) with a typed-context variant; the
detect-package-gates comment notes the SQUASH/HEAD^ equivalence and
path-filter scope; the doc covers re-run reuse, flake re-sampling via
dispatch, stale queue runs and how to confirm reuse on the farm.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pr-preflight-platforms and check-doc-freshness-platforms now resolve their
same-repo-macos marker like the Linux/Windows legs: blacksmith-6vcpu-macos-26
for same-repo PRs and merge_group, GitHub-hosted macos-latest for forks,
Dependabot and every other event. The label is pinned to macos-26 because
GitHub's macos-latest is macos-26-arm64 today, so both paths run the same
OS/arch; 6 vCPU is already 2x macos-latest's 3 vCPU.
Blacksmith cannot see GitHub-saved caches, so main.yml gains
blacksmith-macos-go-build-cache: a push-to-main-only saver on the same label
(same job guard as the other Blacksmith savers) that builds ./cmd/bd and runs
the shared warm-non-race-cache.sh, saving the module cache and a non-race
GOCACHE keyed by go.sum + UTC day. The PR legs stay restore-only. main.yml's
GitHub-hosted macOS full-suite leg is unchanged and keeps seeding the fork
path.
Policy tests: the macOS marker arm is evaluated for same-repo/merge_group/
fork/deleted-head/dependabot/push/schedule and must match the flat macOS
const; the saver joins the PR-guard and per-day-key sweeps and the cache
ownership inventory; TestBlacksmithMacOSSaverMatchesPRLegs pins the
label/path/key/warm-up contract between saver and PR legs.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Makes the repo's agent and contributor surface describe developing beads, not using bd, and requires every PR to link a documented issue.
- Contributor-only root AGENTS.md with a routing table; cmd/bd/AGENTS.md and engdocs/AGENTS.md seam files; CLAUDE.md symlinks so Claude Code loads nested files.
- Bug and feature issue forms, a PR template that requires Closes #, CONTRIBUTING sections for the triage and pipeline labels, and the opt-in fork-cache during dev setup.
- ROADMAP.md for 1.4: versioned beads (#6132), Graph Preview 2 (#7170), contributor onboarding (#7228).
- docsync guards: root AGENTS.md at or under 20 KB; every AGENTS.md symlinked, routed, and with resolving links, paths, and guard-test names.
Agent-Signature: claude-code-claude-opus-5-5-high on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bazel-integration (--config=integration) and bazel-cmd-dolt
(--config=doltserver-cmd, //cmd/bd:bd_dolt_server_test) both run the
integration-build cmd/bd bd_test; the second without BEADS_TEST_SKIP=dolt,
so it is a superset. Once pr.yml requires bazel-cmd-dolt, the integration
lane's bd_test run is a duplicate.
bazel.yml gains a boolean workflow_call input, cmd-dolt-required (default
false). When true, bazel-integration tests //... -//cmd/bd:bd_test. pr.yml
passes it, and a policy test pins it to pr.yml's BAZEL_CMD_DOLT_REQUIRED
(a `with:` cannot read env), so the drop only happens where ci-gate
requires BAZEL_CMD_DOLT. bazel-farm.yml, nightly.yml, push and dispatch
leave it unset and keep bd_test. While the flag is "false" (current main)
nothing changes; with #7226's flip the input flips with it.
Coverage: both lanes share the same `if:` (modes remote, fork-ro, fork-rw,
cache; Dependabot included) and bazel-gate.sh accepts their skips in the
same modes (skip, local). TestBazelIntegrationExcludesBdTestOnlyWhereCmdDoltCovers
pins that, the exact exclusion, the flag/input equality, the gate's
conditional requirement, and that no other caller passes the input.
Rollback: commit BAZEL_CMD_DOLT_REQUIRED "false" and cmd-dolt-required:
false together in pr.yml.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Advisory since #7185. On the farm it passed on every PR run based on
current main; its only failures (3) were branches predating #7219 and
reproduced a real test bug that #7219 fixed. Flip the committed flag so
CI Gate / Required enforces BAZEL_CMD_DOLT. Rollback: set it back to
"false".
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pr-preflight-platforms' "Exercise native init gateway credential shell"
step had no GOCACHE env, so it fell back to the runner's empty default
cache and cold-compiled all of ./cmd/bd's test binary on every PR, right
after the hook-timeout step had compiled the identical binary (same tags,
same platform-default CGO) into the restored non-race cache. Job logs:
213s of a ~350s macos-latest leg, ~117s on Windows; on Linux the rerun
with a shared cache is a relink only (~8s).
- pr.yml: point that step, and the three Windows-only Go steps, at the
restored non-race GOCACHE.
- main.yml: the macOS test leg (the only macOS non-race seeder) now runs
warm-non-race-cache.sh, so the PR macOS legs restore the test packages
and test-only deps they compile, not just the non-test ./cmd/bd graph.
- Policy tests: every run step in pr-preflight-platforms and
check-doc-freshness-platforms must use the restored GOCACHE, and the
macOS seeder must warm before it saves.
No test selection, matrix, or gate wiring changes.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(autofix): push bot fixes with a scoped GitHub App token
github-actions[bot] pushes left PR checks action_required instead of
retriggering, and the DOCS_AUTOFIX_TOKEN fallback secret was never
configured. bazel-autofix.yml and docs-autofix.yml now mint a short-lived,
repo-scoped (contents:write) installation token for the gastownhall-autofix
App via the `autofix` environment on same-repo PRs, falling back to the
default workflow token (and its manual re-run note) when the App token
isn't available; fork PRs are unaffected and still only get the apply-recipe
comment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): job environment parses as a name or a mapping; gofmt
ciWorkflowJob.Environment was a string, so every test that loads all
workflows failed on deploy-pages-redirect.yml's mapping form
(environment: {name, url}).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
pr.yml and pr-risk.yml (and the three advisory workflows that already
list release/** beside it -- conformance.yml, cross-version-smoke.yml,
regression.yml) trigger pull_request only on main and release/**. A PR
based on hotfix/1.3.1 (or any sibling hotfix/** line) matched neither
pattern, so it ran almost no CI: #7146, a backport to hotfix/1.3.1,
showed 17 checks -- 15 cross-version historical smokes and triage/label
jobs, 2 skipped -- no Lint, no unit shards, no embedded tier, no CI
Gate / Required, no PR Risk Gate / Required.
pr.yml's own comment states exactly why release/** is there:
release-prep and release-fix PRs are based on the release branch, not
main, and without it they run almost no CI before the tag build. A
hotfix line is in the identical position. Added hotfix/** alongside
release/** in all five files.
New TestPullRequestWorkflowsTriggerOnHotfixBranches pins hotfix/**
presence in each file's pull_request.branches list -- red before this
change (observed [main release/**] in all five), green after. The
existing TestPRRiskBazelCoverageJob equality check (pr.yml and
pr-risk.yml's pull_request triggers must match) still passes, since
both were updated identically. go vet/gofmt clean.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
* perf(storage): cut batch-create round trips (import, bulk create, apply-batch)
A multi-issue create (import, bulk create, CreateIssuesWithFullOptions) ran
~20 SQL statements per issue, and its dependency pass ran a recursive cycle
CTE per edge over the union of both dependency tables, which Dolt cannot
index — every recursion step rescanned the edge tables, so an import whose
edges form chains went superlinear (400 issues: 73 s of 82 s in that CTE).
issueops batch create, same transaction and same stored outcome:
- createBatchCache: one IN-list read of every explicit id's presence in both
planes (replacing two COUNT probes per issue), one read of the batch's
stored labels (each issue's new labels then land in one multi-row INSERT),
created/label_added events buffered and written in one pass under the ids
InsertDerivedEvent would mint (same digest, same lowest free ordinal), and
runs of brand-new rows written with multi-row INSERTs (same ODKU clause)
then finished in order.
- Dependency pass: wisp routing and target presence from batch reads; the
cycle and hierarchy checks walk an in-memory copy of the edge tables,
loaded once and kept in step with every edge the pass writes (a pair it
already holds is re-read, since a clientFoundRows connection reports a
matched duplicate as written).
- createBlockedRecomputeIDs seeds all created edges and expands once.
Shared paths:
- Blocked-state recompute splits ids once per call by whether they own a
dependency row; edgeless ids (every freshly created issue) skip the two
union UPDATEs for one plain unmark, run after the chunk's union statements
on the first pass so the unions see what they saw before.
- appendChildrenInTx reads children per IN-list chunk, not per parent.
- ExecuteCreate hands its BatchContext to the batch body instead of having
it re-read the same config (-6 statements per apply-batch create item).
Not changed: one version-history row per created issue (the version
contract, version_history.go), journal order and content, and the per-row
bodies, which stay the path for singular creates.
Equivalence: internal/storage/createbatchequiv runs an import scenario
(upserts, duplicate ids, wisps, generated ids, cycles, hierarchy conflicts,
dangling/external targets, stale is_blocked, comments, journal and
versioned history on) through the fast and per-row bodies on the embedded
and server engines and compares every written table and the skip reports.
Re-pinned TestLargeBatchApplyStatementCounts*: embedded 7009/14014/846 ->
6396/12790/793, server 7008/14014/846 -> 6396/12791/793 (breakdown in
pinnedEmbeddedStatementCounts).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): batch-create review fixes — lazy edge graph, opt-in recompute probe, insert budget
Review of the batch-create round-trip change (S1-S3 and nits):
- The dependency pass's in-memory graph no longer loads both edge tables.
It reads a node's outgoing edges through the issue_id index the first
time a walk reaches it: the batch's own endpoints up front, then one
IN-list read per BFS level. Memory and reads scale with the subgraph
the batch's walks touch, not the edge tables, so a two-edge create on a
large database no longer materialises every edge. A written edge whose
source was never loaded loads that source (row included).
- The blocked-state no-edge probe runs only when a create asks for it or
when a chunk holds at least 32 ids. Single-issue recomputes on close,
reopen, update and dependency edits issue exactly the statements they
did on main (the reopen sqlmock expectation is back to main's).
- Multi-row issue INSERTs are bounded by an estimated 16 MiB of bound
text as well as 100 rows. When a multi-row statement fails, its rows are
replayed one at a time, and the batch continues if every row lands
instead of returning the original error.
- Nits: document the binary-collation assumption behind Go string
equality; DisableCreateFastPathsForTest documents that it is
process-global and non-parallel, and panics on an overlapping caller;
drop the unreachable skip branch; explain why wisp targets need no
presence read.
Re-pinned TestLargeBatchApplyStatementCounts* classic 40: 793 -> 786
(the +7 probes on dep-add recomputes are gone; net vs main is -60, all of
it ExecuteCreate's config reuse). 356/712 unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): search the batch dependency graph from both ends
The lazy per-node load (previous commit) still walked forward from the
check's start, so on a large stored graph a single create's cycle check
read every node reachable from its target — about 800 statements per
create on a 2000-issue fixture. The cycle and hierarchy questions a batch
asks almost always end at a row the batch just created, whose only
incoming edges are the batch's own.
depGraph now answers each reachability question with a bidirectional
search: forward from the start along outgoing edges (issue_id index),
backward from the goal along incoming edges (the three target-column
indexes, read as one UNION ALL per table), always expanding the smaller
frontier and loading a level's nodes in that direction once. The batch's
endpoints are loaded in both directions up front, so a create's checks
usually finish with no further read (a chain import, too: each new
edge's goal has no incoming edge yet). Rows are keyed by (table, source,
target), so a row read from both ends counts once. An incoming read
ignores a row whose matching column is not its first set target column,
as the CTE's COALESCE does. A written pair that neither loaded end can
vouch for is re-read instead of assumed new.
On a 50k-issue / 100k-edge database, a create with two blocks edges
now runs 27 statements, against the full-scan or per-node walks before
(numbers in the review-fix summary).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): hold batch create to golden digests from the pre-change code
The batch-create equivalence test compared the fast paths with the same
code's per-row bodies, so changes the fast-path switch does not gate —
the recompute seed merge, the IN-list children read, ExecuteCreate's
context reuse, the prepare/finish split — were checked only by older
contract tests. Its batch was also too small to cross a chunk boundary
and never set RejectStaleUpserts.
createbatchequiv now runs four scenarios, each through the fast and
per-row bodies, and checks both against a per-table row count and
SHA-256 recorded from ebe3b6bcb (before the fast paths existed;
CREATEBATCHEQUIV_GOLDEN_DIR re-records them):
- small: the original scenario;
- import458 and import458-reject-stale: the review's 458-issue batch
(every chunk boundary, wisps and cross-plane edges, cycles, hierarchy
conflicts, duplicate ids across deferred runs, generated and
hierarchical ids, stale upserts rejected and accepted), plus new edges
whose blocked-state fallout reaches stored rows outside the batch (a
spawner's waiter, an upserted parent's existing child);
- apply: an apply-batch through ExecuteCreate, dep adds and an update.
The recorded digests match on the embedded and server engines (the
server test now sets the fixture's prefix, so generated ids agree).
Dropping the spawner-waiter seeding is invisible to the fast-vs-per-row
comparison and caught by the golden check.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): large-graph create benchmark
batchbench measures creates against a database already holding a large
dependency graph (batchfixtures.LargeGraphInserts: N issues, ~2N
acyclic edges with wide reachable sets and a deep parent-child forest,
committed as a Dolt version): ExecuteCreate under an existing parent with
an existing blocker, a store create with two blocks edges, and an
apply-batch of creates each with a parent-child and a blocks dep_add.
It logs wall time, statements and Go allocation per shape on the
embedded and server engines. Opt in with BEADS_BENCH_LARGE_GRAPH=<issues>
(and BEADS_BENCH_LARGE_GRAPH_CREATES); it asserts nothing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): read incoming dependency edges one target column at a time
The bidirectional graph search read a node's incoming edges with one
UNION ALL over the three target columns per table. Dolt plans that as a
scan: 0.7 s per read on a 100k-edge table, where each leg alone is a 1 ms
lookup on its own index (EXPLAIN shows IndexedTableAccess per leg). Issue
the three legs as separate reads into one row set; a row matched through
two columns still counts once.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): run the 458-issue equivalence scenarios outside the race lane
The 458-issue golden scenarios took the embedded tier's shard over its
20-minute limit under -race (the in-process engine's own instrumentation;
about 35 s each without it). They move to
TestCreateBatchFastPathsMatchPerRowLarge_Embedded, skipped under -race and
added to nightly.yml's non-race embedded step, following the precedent of
the 712-item statement-count and 1000-item wall-clock cases. The light
scenarios (small, apply) stay in TestCreateBatchFastPathsMatchPerRow_Embedded
on every lane. The server backend (built without -race in its full
suite) runs both: TestCreateBatchFastPathsMatchPerRow_Dolt and, as its own
top-level test so the shard manifest's hash places it on another shard
(about four minutes there), TestCreateBatchFastPathsMatchPerRowLarge_Dolt.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): reset cobra flags between in-process runs; keep them out of parallel tests
runBDInProcess and runBDInProcessAllowError execute the package-level
rootCmd tree again and again in one process, and cobra never resets a
flag between Execute calls: a value one invocation parsed (create's
--title, --labels, list's --status, ...) and its Changed bit stayed on the
FlagSet, so the next invocation saw it, and a slice flag that had been
set appended to the stale value instead of replacing it. With a Dolt test
server (the only setup in which these tests run) that failed about 20
TestCLI_* tests depending on suite order.
TestMain now snapshots every flag of the tree (snapshotCommandFlags,
after all init()s registered them, before any test runs) and both runners
put the tree back before and after each run (resetCommandFlags): the
pointee of each flag.Value is restored (for pflag's scalar types that is
the bound variable; for slice types it also clears the internal "changed"
bit, so the next Set replaces), slice contents via SliceValue.Replace,
and Changed=false. TestResetCommandFlagsRestoresEveryFlag sets every flag
of the real tree and checks the reset. The ad-hoc --title reset in
TestCLI_CreateRejectsEmptyTitle_ProxiedServerMode is no longer needed.
Seven TestCLI_Comment* tests called t.Parallel() around the runners.
inProcessMutex serializes the runners only against each other, so their
chdir, stdio swap, env and config re-initialization raced every other
parallel test (data races on config globals; TestCobraParallelPolicyGuard
failing to read its own source files after a concurrent chdir). They run
serially now, and the guard treats the two runners like rootCmd.Execute.
Also two stale TestCLI_Create* cases: DryRunWithRigPrefix exercised a
create --prefix flag that no longer exists, and the flag-like-title cases
passed "--help" or "--foo-bar" where cobra parses them as flags; they now
pass the title after "--", the only way it reaches create's own guard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix the ADO round-trip and bulk-delete tests that rotted unseen
Both need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd (BEADS_TEST_SKIP=dolt everywhere), so they failed unnoticed:
- TestADORoundTrip* stored ado.pat in the database. ado.pat is a secret
read only from config.yaml or AZURE_DEVOPS_PAT (config.IsYamlOnlyKey),
so the tracker found no PAT. The fixture sets AZURE_DEVOPS_PAT.
- TestBulkDeleteNoResurrection expected GetIssue to return (nil, nil) for
a deleted issue; the Dolt store reports storage.ErrNotFound.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix parity, completion and notion tests that only fail with a Dolt server
These are Dolt-gated (integration tag + test server), so no CI lane ran
them and they rotted:
- TestParityCreate*/TestParityReopenErrorExits1Silently: bd create
auto-routes via routing.DetectUserRole("."), which warns "beads.role
not configured (GH#2950)" on stderr when the working directory's git
config has no beads.role. The suite pins the configured case, so the
parity env now sets beads.role=maintainer in a test-owned global git
config ($XDG_CONFIG_HOME/git/config; the role lookup scrubs every
GIT_CONFIG_* variable, so those cannot be used).
- TestIssueIDCompletion: issueIDCompletion resolves a database path
before looking at the open store and returns nothing when there is
none; the test now sets dbPath to its store's path.
- TestIssueIDCompletion_UsesWorktreeFallbackWhenStoreNil and
TestGetNotionConfigReadsDBPathWhenStoreUnset: the code under test
reopens the store from metadata.json, which names a database but not
the per-test branch of the shared database that newTestStore uses, so
the reopened store saw none of the test's rows. They now use their own
database (newTestStoreIsolatedDB).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix stale doctor/e2e/autocommit/readonly Dolt-gated tests
These need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd, so they rotted unseen:
- TestE2E_DoctorFixMetadataRoundtrip, TestE2E_MigrateDoltMetadata,
TestDoctorCheckHealthReportsVersionMismatchOnRepoLocalPort: bd init now
defaults to embedded mode, where `bd sql` and `bd dolt start` are
refused. Init with --server on a repo-local server of their own
(repoLocalServerEnv: the suite's shared-server port vars cleared, server
stopped at cleanup) and start it explicitly, since BEADS_TEST_MODE=1 never
auto-starts one. MigrateDoltMetadata's DELETE also needed `key` quoted.
- TestDoctor{Conventions,Pollution}MaxRows_EnvOnly_Exits2: bdRunRaw drops
every BEADS_* variable, so the subprocess hit dolt.New's test-database
firewall (exit 1) before the row cap; pass BEADS_TEST_SERVER=1.
- TestDoltAutoCommit_On_WritesAdvanceHead: the embedded engine keeps its
data under .beads/embeddeddolt, not .beads/dolt; findDoltRepoDir looks
in both.
- TestValidateCheck_AllClean: bd doctor validate runs five checks since
the cross-table duplicates check was added.
- TestConfigValidateReadOnlyIsHermetic: readonly commands take a shared
workspace gate, whose lock file (.beads/dolt.gate.lock) workspacegate
creates on first use by design; compare the .beads tree without gate
lock files.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): undo each in-process execution's env and storage-mode leaks
Every in-process run of the bd command tree (runBDInProcess and the many
tests that call rootCmd.Execute directly) leaves process state behind:
PersistentPreRun exports the selected workspace as BEADS_DIR, loads its
.beads/.env, and records the workspace's mode in serverMode /
proxiedServerMode. With a Dolt test server, later tests in the same shard
then resolved a deleted fixture ("no active beads workspace found",
"migration sidecar is missing"), ran subprocess bd against the wrong
workspace ("recorded as proxied-server in .beads/metadata.json"), or took
the proxied path in RunE-level tests ("proxied-server UOW provider not
initialized", a nil-UOW panic in reopenProxiedResolve that killed a shard).
TestMain now registers a cobra initializer/finalizer pair
(installExecuteIsolation) that snapshots the environment and the two mode
globals when an execution starts and restores them when it ends; the
runners' restoreEnvSnapshot shares the helper. TestDoltConfigEnvironmentOverrides
used os.Setenv + defer os.Unsetenv, which deleted the suite's
BEADS_DOLT_SERVER_PORT (the test server's port) for every later test; it
uses t.Setenv now.
Full integration-tagged cmd/bd suite against a hermetic dolt (16 shards):
18 order-dependent failures and the shard panic gone; the 27 remaining
failures all fail alone too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(init): keep --quiet quiet through the Claude/Codex/Cursor auto-setup
`bd init --quiet` ("Suppress output") still printed the project
integration installers' progress to stdout ("Installing Claude hooks for
this project...", "Claude Code integration installed", ...): the
installers write to os.Stdout unconditionally. Point stdout at the null
device around them while quiet; their failures keep going to stderr as
before. Found by TestInitCommand's quiet case, which asserts no output and
had not run in CI (Dolt-gated).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(init): bring the Dolt-gated init tests up to the current init contract
These tests need a Dolt test server, so no CI lane ran them and they
drifted from the product:
- Embedded is init's default mode, so the server-mode assertions
(.beads/dolt, opening the database on the test server, dolt_mode
"server" in metadata.json) now pass --server (TestInitCommand,
TestInitAlreadyInitialized, TestInitRedirect, TestInitBEADS_DIR,
TestInitDoltMetadata, TestInitDoltMetadataNoGit, TestInitDatabaseFlag's
metadata subtests). The in-process ones reset rootCmd's flags around
themselves so --server cannot leak into later in-process inits.
- Each server init gets its own --database: the suite shares one test
server, and a "test" database another test created belongs to another
project (init refuses: PROJECT IDENTITY MISMATCH).
- The marker tests seed a current-era version witness: the legacy-upgrade
guard refuses a non-empty .beads/dolt without one.
- The redirect test provisions the canonical target with config.yaml: bd
ignores a redirect to a directory with no project files (#4692).
- Shared-server inits get a free port of their own
(sharedServerTestEnv): the inherited BEADS_DOLT_SERVER_PORT is the
suite's test server, which a shared-server start refuses as another
project's, and the default shared port is 3308, a developer's live
server.
- --shared-server's config.yaml is now nested YAML (dolt: shared-server:
true); parse it instead of matching the old flat line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): run the Dolt-gated tests that skipped on a bare dolt.New
setupTestDB (config, kv and Linear round-trip tests), setupGatedTestDB
(mol ready --gated) and the rename-prefix and bd sql tests opened stores
with dolt.New(&dolt.Config{Path: ...}): no server port, no
CreateIfMissing, so against a running test server they got "database ...
not found" and skipped as "Dolt server not available". They now use the
suite's newTestStore helpers (skip only when no test server runs), and
what that exposed is fixed test-side:
- TestCustomStatusConfig: custom_statuses is read back ORDER BY name;
look statuses up by name / expect name order.
- TestLinearRoundTripCoreFields, TestLinearPullMilestonesCreatesEpicHierarchy:
linear.api_key is a yaml-only secret (LINEAR_API_KEY), push now needs an
explicit linear.state_map, and batch push uses TeamLabels and
issueBatchCreate, which the mock server now answers.
- TestSqlCommand: bd sql refuses in the default embedded mode; set
serverMode for the server-backed store, and stop discarding RunE errors.
- TestE2E_AutoStartedRepoLocalServerPersistsAcrossCommands skipped on
this process's embedded mode although every bd it runs is a subprocess
on a --server workspace; gate on the dolt binary instead and stop the
auto-started server at cleanup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): no fixed "unused" port, no swallowed JSON, no leaked front-door server
Three more things the Dolt-backed cmd/bd run exposed:
- TestTestServerConnection, TestSharedServerCfgNilHonorsSharedServer and
the two TestLegacyGuard* e2e tests pointed bd at 127.0.0.1:59999 as a
port nothing listens on. Any process on the host may hold it; a
listener that accepts and resets turns the expected "connection
refused" / "Dolt server unreachable" into another error. They take a
just-freed port from closedLoopbackPort (testutil.FindFreePort) now.
- TestDoltSetConfigJSONOutput and TestDoltSetConfigWithUpdateConfig
created .beads 0755, so bd's permissions warning landed in the JSON
under test and the tests skipped ("output not pure JSON") every run.
The fixture uses 0700, and non-JSON output fails instead of skipping.
- TestMigrateDoltModeFrontDoor's closing show/list auto-start a
repo-local server after the reverse migration; nothing stopped it, so
the suite's leak check failed (when the opt-in
BEADS_TEST_MIGRATION_FRONTDOOR suite runs). It stops it at cleanup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): advisory cmd/bd Dolt-server tier (bazel-cmd-dolt)
No CI lane ran cmd/bd's Dolt-gated tests (TestCLI_*, the init and
store-backed suites, the doctor/e2e tests): main.yml's integration shards,
nightly's Full Test Suite, and Bazel's --config=prcore/ci and
--config=integration all pass BEADS_TEST_SKIP=dolt, so cmd/bd's TestMain
never starts a test server and those tests skip; the proxied and embedded
tiers run only their manifests' tests. They rotted unseen.
//cmd/bd:bd_dolt_server_test (tag dolt-server-cmd) runs the whole
integration-tagged cmd/bd suite, the integration build's bd_test, against
a hermetic dolt sql-server: BEADS_TEST_DOLT_SERVER=local starts it from
the pinned dolt in the test wrapper's runfiles, and
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 fails the target instead of skipping
into a cached pass. No test selection: the Go binary shards itself over
every top-level test (32 shards), so each runs exactly once, and
check_testcases.py fails a shard that ran none. --config=doltserver-cmd
has exactly --config=integration's build flags, so the two lanes share
bd_test's compile; prcore/ci and integration exclude the tag.
bazel.yml's new bazel-cmd-dolt job runs it exactly where bazel-integration
runs (same if: remotely on the farm for same-repo PRs, fork-ro/fork-rw,
and locally with the read-only cache in mode cache for forks). It is
advisory: job-level continue-on-error keeps a failure out of the call's
aggregate (BAZEL), and pr.yml's ci-gate reads its result (BAZEL_CMD_DOLT)
but requires it only while the committed top-level flag
BAZEL_CMD_DOLT_REQUIRED is "true" (now "false"; flipping it is the one
change needed). bazel-gate.sh accepts its skip in modes skip and local.
Policy tests: a third lane category in TestBazelLaneIsGatedAlongsideLegacy
(bazelFlagGatedLanes: gate env wired, never statically required, added by
the run script exactly when the pinned flag is "true"), the gate
simulation over every mode with the flag off (advisory: nothing it
reports reddens the gate) and on (exactly a gated lane), the cache-mode
and execution-mode checks, the tag taxonomy and prcore/integration
exclusions, and TestBazelCmdDoltJob (job steps, timeouts, .bazelrc
config equal to integration's build with no dolt skip or test selection,
the target's dolt-server rules, and bd_test's env repeated).
Measured locally (all fixes in this series, 32 shards, 16 at a time on a
shared host): 3267 top-level tests, all shards pass, max shard 313s (one
5001-issue export test, 214s, sets it), mean 100s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(init): hand the project installers a writer instead of swapping os.Stdout
The --quiet fix earlier in this series pointed the process-global
os.Stdout at the null device around the Claude/Codex/Cursor installers.
That works, but it mutates a global from inside a cobra Run: anything else
writing to stdout meanwhile (a goroutine, a future concurrent installer)
would vanish, and a concurrent read of os.Stdout is a data race.
The installers already write through env structs with stdout/stderr
fields. setup.ProjectInstallOutput carries the two writers, and
InstallClaudeProjectTo / InstallCodexProjectTo / InstallCursorProjectTo
use them (the old entry points keep the process streams). init passes
QuietProjectInstallOutput() under --quiet: progress discarded, error lines
still on stderr. The proxied-server init path (domain adapter
InstallClaudeProject now takes quiet) does the same, so its
"Installing Claude hooks..." is quiet too.
Consistent error policy under --quiet: errors still reach stderr. init's
own "Warning: failed to setup <installer>" lines were suppressed under
--quiet while the installers' "Error:" lines were not, and some installer
failures (Cursor rules) print nothing themselves; the warnings now print
whether or not quiet, like the installers' errors.
Coverage in required lanes: TestProjectInstallersReportToTheGivenOutput
(cmd/bd/setup) and TestEmbeddedInitA/basic, which now asserts that
`bd init --quiet` with agents enabled installs the Claude hooks and prints
none of the installers' progress (verified to fail with the quiet output
disabled).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(export): exercise the incremental-export fallback with a small threshold
TestTryIncrementalExport_ThresholdExceededFallsBack seeded
incrementalExportThreshold+1 = 5001 issues to cross the threshold. Batch
create is one transaction but still several statements per issue, so under
-race against the shared test server it took ~214 s, the longest test in
cmd/bd's Dolt-backed suite by far. The behaviour under test is only
"more changed issues than the threshold -> full export fallback".
incrementalExportThreshold becomes a var (same value), and the test, which
is not parallel, sets it to 20 and seeds 21 issues: 214 s -> 0.6 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): tighten the flag-reset self-test, guards and canaries
Review follow-ups:
- TestResetCommandFlagsRestoresEveryFlag no longer skips a flag it cannot
set: a flag type without a sample value, a Set error, or a non-pointer
Value (which resetCommandFlags could not restore) is an error, so a new
flag type cannot go untested. resetCommandFlags documents that it also
resets flag-bound package globals.
- TestCobraParallelPolicyGuard treats .ExecuteC/.ExecuteContext/
.ExecuteContextC like .Execute (the per-execution isolation writes the
mode globals; a parallel execution would race).
- TestConfigValidateReadOnlyIsHermetic ignores only the gate a readonly
command takes (.beads/dolt.gate.lock and its .info sidecar), not any
"*.gate.lock*" file, so a new unexpected gate still trips the canary.
- TestCLI_CreateRejectsFlagLikeTitles: a bare `bd create --foo-bar` must
fail and create no issue (bd-2c0's intent), next to the "--" cases.
- TestCustomStatusConfig cites why statuses come back in name order: the
normalized custom_statuses table (#2961, 8537d943b) has no ordinal and is
read ORDER BY name, like custom_types; the test predates it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the cmd/bd Dolt-server tier in 16 shards
Each shard pays ~30-40 s of fixed cost (TestMain plus its own dolt
sql-server), so 32 shards spent roughly 40% of the lane's executor time on
setup and added 32 farm actions to every PR touching cmd/bd. With the
5001-issue export test cut to under a second, 16 shards measure (locally,
--nocache_test_results): max 186 s, mean 137 s, 2,187 executor-seconds
(vs ~2,800-3,200 at 32), all 3,267 top-level tests, every shard passing.
TestBazelCmdDoltJob pins the count; bazel.yml's timing comment is
restated (cache mode ~20-30 minutes).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
actions/create-github-app-token (v3.2.0, pinned) deprecates its app-id
input in favor of client-id. rbe-prewarm's mint step now passes the
bazel-allocator App's public Client ID (Iv23ligrqVEhlamZnoPU, from
GET /apps/bazel-allocator) as a reviewed literal: it is not a credential,
and a literal keeps it policy-tested with the rest of the step instead of
in an unversioned repository variable. The private key stays a secret.
RBE_POOL_APP_ID is now unused, so its workflow_call declaration in
bazel.yml and its pass-through in pr.yml and nightly.yml are removed,
and a policy test rejects any workflow mentioning it again. The
repository secret can be deleted.
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md's "rbe-west Pre-warm" section now
documents the client-id, the single remaining secret, and
RBE_PREWARM_WORKERS (default 1, clamped to 4, 0 = kill switch, repo
variable currently 2). It also no longer cites a maintainer's local
planning-notes path; a new tracked-file policy test fails if one
reappears.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The prebuilt twins of the Windows liveness and worktree-remove jobs passed
alongside the native ones on every PR since F4 (#7135): #7139, #7145,
#7147, #7155, #7156, #7159, #7160 — 1-1.5 min vs 3-8 min native. Flip the
committed flag so ci-gate enforces the prebuilt pair; the native pair keeps
running as advisory and is the rollback (set the flag back to "false").
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): best-effort pre-warm rbe-west's OSS worker pool on PR open
bazel.yml's Bazel lanes each separately wait ~120s at "N actions, 0
running" while gastownhall/gascity's rbe-west NativeLink worker pool
scales up from zero. Add an advisory rbe-prewarm job (never a `needs`
of any lane, never in bazel-gate.sh's vocabulary, job-level
continue-on-error so it can never fail bazel.yml's own run) that
mints a short-lived installation token for a "bazel-allocator" GitHub
App (installed on gastownhall/gascity only, Actions read/write +
Metadata read) and dispatches gastownhall/gascity's
rbe-worker-pool.yml ahead of time, only when this run's lanes will
actually target rbe-west's "oss" instance (modes remote and fork-rw;
fork-ro targets a separate pool this doesn't serve).
Adds RBE_POOL_APP_ID / RBE_POOL_APP_PRIVATE_KEY as new optional
workflow_call secrets, threaded through every caller (pr.yml,
bazel-farm.yml, nightly.yml); either secret missing (the app not yet
provisioned) or repo var RBE_PREWARM_WORKERS=0 disables pre-warming
cleanly. Policy tests (scripts/bazel_rbe_prewarm_test.go, plus updates
to the existing bazel.yml structural tests) pin the mode gate, the
credential scope, the dispatch target, and that the job can never gate
or fail a run; each invariant was hand mutation-tested. Docs added to
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md ("rbe-west Pre-warm").
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): close rbe-prewarm token-theft path, fix stale secret-count docs (B1)
Security review of bdef342d5 found a BLOCKER (B1): the rbe-prewarm job
checked out inputs.checkout-sha (the fork head, reachable via
bazel-farm.yml's pull_request_target path) and then ran
.github/scripts/rbe-prewarm.sh with the minted gastownhall/gascity
dispatch token present. Script-file edits don't require workflow-approval
the way workflow-file edits do, so a VIP fork or same-repo branch pusher
could edit that script to steal the shared bazel-allocator key.
Fix: delete the checkout step and the .sh file; inline the dispatch logic
into the step's own `run:` block, whose bytes always come from the
trusted (approval-gated) workflow definition even under
pull_request_target. Mint the token in the immediately preceding step;
nothing after the mint step touches a repo-relative path. Stop forwarding
RBE_POOL_APP_ID/RBE_POOL_APP_PRIVATE_KEY from bazel-farm.yml (that fork
path now gets a cold start instead of pre-warming) and narrow the job's
`if:` to mode == 'remote' only (fork-rw never carried secrets anyway, so
it was dead weight - S1).
Also addresses the review's should-fix items:
- S2: document the widened trust boundary (the bazel-allocator key is
shared with gascity's own rbe-west scaler; leak blast radius; rotation
breaks the scaler, so the kill switch is unset-secret /
RBE_PREWARM_WORKERS=0, not rotation) as an accepted risk in bazel.yml's
header and engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md.
- S3: fix stale "exactly four secrets" claims (pr.yml/nightly.yml pass 6;
bazel-farm.yml now correctly passes 4 again) and the false "never a
long-lived credential" claim.
- S4: flag the continue-on-error-preserves-caller-success assumption as
reasoned-through but not live-validated in this sandbox.
- N1: drop the unused RBE_PREWARM_IDLE_MINUTES knob entirely.
- N4: strengthen the dispatch-target test to check exact gh command
strings and ban hardcoded repo/workflow literals outside $POOL_REPO/
$POOL_WORKFLOW.
Test changes (scripts/bazel_rbe_prewarm_test.go rewritten; ci_workflow_test.go,
bazel_farm_workflow_test.go, bazel_cache_mode_test.go updated for the new
job shape): new TestRBEPrewarmNoCheckoutStep,
TestRBEPrewarmNothingAfterMintTouchesARepoPath,
TestRBEPrewarmTokenOnlyInDispatchStep, and
TestRBEPrewarmAppSecretsOnlyExpectedCallers directly target the B1
mutation class; TestRBEPrewarmDispatchScriptBehavior keeps the fake-gh
behavioral coverage for the now-inlined dispatch script. Verified by hand
that both of the reviewer's mutations (widen the `if:` back to include
fork-rw; add a step after mint running a checked-out script) are now
caught.
Verification: go test ./scripts/..., go vet, gofmt, actionlint on the
four touched workflow files, make ci-pr-policy, make bazel-sync-check,
and `bazel test //scripts:scripts_test` all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): embedded-Dolt tier 20/5 -> 40/14 duration-balanced shards (F1)
Give the Bazel-only bazel-embedded job its own duration-balanced manifest
blocks for //cmd/bd:bd_embedded_test (20 -> 40 shards) and
//internal/storage/embeddeddolt:embeddeddolt_embedded_test (5 -> 14
shards), decoupled from PR Risk's/main.yml's legacy frozen 20/5-shard
fork/push jobs, mirroring F2's bd_proxied_test precedent.
- Measured every TestEmbedded* / embeddeddolt test's wall time in
isolation; split 21 cmd-tier and 1 storage-tier single top-level test
over 180s into disjoint sub-tests (3 irreducible exceptions left:
TestEmbeddedUpdateConcurrent 475s, TestEmbeddedCloseConcurrent 308s,
TestLargeBatchApplyStatementCounts712_Embedded 261s).
- New scripts/ci/gen_embedded_{cmd,storage}_shard_manifest.py +
_embedded_shard_manifest_lib.py generate/check duration-balanced
manifest blocks (LPT packing, incremental --write/--repack, exact-once
--check), chosen at N=40/M=14 where the heaviest shard reaches the
irreducible-test floor.
- Fixed 18 stale legacy 20-shard cmd manifest entries (and 1 storage
entry) left over from earlier test splits, which would have hard-failed
every shard of PR Risk's real test-embedded-cmd jobs on their next run.
- New scripts/embedded_shard_count_test.go policy tests tie the
Bazel-only block's shard count to bazel.yml's check_shard_coverage.py
argument and catch generator staleness; generalized
TestBazelRetiredLanesCheckListedTestsRan's live-shard-count dispatch
from bd_proxied_test-only to a map covering both embedded targets;
removed TestBazelEmbeddedJobMirrorsEmbeddedTier's now-invalid
BUILD.bazel-shard-count-equals-PR-Risk-matrix assertion.
- Verified: go test ./scripts/..., bazel test //scripts:scripts_test,
actionlint, make ci-pr-policy, make bazel-sync-check all pass; full
local bazel test of both targets at 40/14 shards passes 2/2; mutation
testing confirms each new/changed policy test fails exactly when it
should.
See ~/beads-bazel-plan/f1/impl-report.md for full detail.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(embedded-dolt): skip 712-item statement-count assertion under -race (F1)
TestLargeBatchApplyStatementCounts712_Embedded genuinely costs ~260-300s
under -race (confirmed by solo re-measurement: 298.60s) -- 14014 real SQL
statement round-trips through the race-instrumented in-process Dolt engine,
not an artifact and not splittable (one indivisible ApplyBatchInTx
transaction). It was the storage tier's single-test floor for CI shard
packing.
Apply the exact precedent TestLargeBatchApplyWallClock_Embedded already set
for its 1000-item shape: skip this assertion under raceEnabled (the cost is
race-instrumentation overhead on the Dolt engine's own internal
goroutine/lock machinery, not this test's own logic) and extend nightly.yml's
"Embedded Dolt batch-apply suite (non-race)" step's -run regex so the full
pinned 14014-statement baseline still runs, non-race, every night. Verified
directly: under -race the assertion now skips in 0.00s; without -race it
still runs and asserts Total=14015 (within the existing +/-2 tolerance) in
36.76s.
Updates scripts/ci_workflow_test.go's
TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite to match the extended
regex.
Storage tier's new single-test ceiling under the per-PR -race lane drops
from ~260.7s to 152.38s (TestBatchApplyContract).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): correct cmd-tier long-pole durations, re-pack embedded manifolds (F1)
Investigated coordinator item 2 (single-test long poles): TestEmbeddedUpdateConcurrent
(recorded 475.39s) and TestEmbeddedCloseConcurrent (recorded 308.06s), plus
the two other >180s cmd-tier entries from the same capture
(TestEmbeddedInitA 196.94s, TestEmbeddedEpicCloseEligibleReason 192.47s).
Root cause: NOT host contention, NOT backoff/retry sleeps (bdRunWithFlockRetry
contributes negligibly), NOT race-instrumented-subprocess overhead alone.
These numbers were captured via `bazel --nohome_rc test --config=embedded`
with the then-current 20-shard legacy manifest (~10-11 tests/shard) and
`-test.parallel=4`. Go's t.Parallel() starts each parallel test's elapsed
clock when its function is invoked, before the Parallel() call blocks it
waiting for a free one-of-4 slot -- so a test queued behind several other
heavy shard-mates has that queueing wait baked into its own recorded
`--- PASS: TestX (Ys)` time. This is shard-composition-dependent scheduling
delay, not a cost intrinsic to the test.
Confirmed two ways in an isolated clone (not this worktree, per the
no-plain-go-test-from-a-beads-checkout rule):
- Solo (no rivals at all): 151.83s / 106.48s / 63.58s / 66.08s.
- All four together with the real -race -test.parallel=4 flags and exactly
4 tests requested (so nobody queues -- parallel slots == test count):
141.34s / 113.80s / 69.79s / 74.63s.
Both agree closely and both are far below the capture-time numbers.
F1's LPT packer already isolates the heaviest tests into shards with few or
no other heavy rivals (TestEmbeddedUpdateConcurrent lands alone in its own
40-shard-block shard), so under the new manifest these tests see little to
no queueing delay in real CI. None of the four needed splitting, parallel
subtests, or raceEnabled-scaled N: once correctly measured, none exceeds the
180s single-test guideline (new corrected weights: 155.0 / 115.0 / 72.0 /
77.0s). Corrected scripts/ci/embedded_cmd_test_durations.json accordingly
(with a doc-comment explaining the methodology and both confirmations) and
re-packed the 40-shard Bazel-only block (--repack --write); --check passes
(exact-once coverage of all 241 discovered tests). The frozen legacy 20-shard
block is untouched and independently re-verified exact-once by simulating
embedded-test-shard.sh across all 20 shards (241/241 tests, no
duplicates/misses/errors).
New cmd-tier single-test ceiling: 179.47s (TestEmbeddedCloseAlreadyClosed,
likely itself somewhat inflated by the same mechanism but already under the
180s line as recorded, so no action needed).
Also updates scripts/ci/embedded_storage_test_durations.json: the prior
commit's raceEnabled-skip for TestLargeBatchApplyStatementCounts712_Embedded
means it now costs ~0s in the per-PR -race lane bazel-embedded actually
packs for (not its real ~260-300s non-race cost, which nightly.yml still
exercises) -- weight set to 1.0 with a doc comment tying it to the skip in
large_batch_apply_measure_test.go. Re-packed the 14-shard Bazel-only block
accordingly; --check passes (exact-once coverage of all 387 discovered
tests, excluding TestMain/TestConformance per that file's existing
convention). The frozen legacy 5-shard block is untouched and independently
re-verified exact-once via embedded-storage-test-shard.sh simulation
(388/388 tests, no duplicates/misses/errors).
Verification: go test ./scripts/... (includes the nightly.yml policy test),
make ci-pr-policy, make bazel-sync-check, actionlint all pass. Real local
`bazel --nohome_rc test --config=embedded //cmd/bd:bd_embedded_test
//internal/storage/embeddeddolt:embeddeddolt_embedded_test`
(--local_test_jobs=8, both targets together, on a shared/noisy sandbox host)
passes both targets; per-shard wall time from each shard's test.xml: cmd/bd
max 218.9s (shard 23/40, min 20.6s), storage max 251.1s (shard 5/14, min
0.98s) -- down from the prior oversubscribed sweep's 783.41s / 351.94s and
well under the single-test-floor-based ~475s projection this slice started
from. Projected tier lane time (max of the two targets, real dedicated CI
runners should do at least this well): ~251s, under the 5-minute target.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(embedded): un-vacuous TestEmbeddedDepB edges, split epic-close test, trim concurrency stress N (F1 review)
Coordinator review follow-ups on the F1 embedded-Dolt resharding slice:
- TestEmbeddedDepB (one of TestEmbeddedDep's two CI-speed split halves) was
asserting its tree_* subtests against an edgeless graph, passing
vacuously; restores the same parent-child/blocks edges TestEmbeddedDepA
builds so those subtests exercise real multi-node traversal (review S2).
- TestEmbeddedEpicCloseEligibleReason split into
TestEmbeddedEpicCloseEligibleReasonFlag/...Output for CI shard balance,
matching the project's established single-concern split pattern.
- TestEmbeddedUpdateConcurrent/TestEmbeddedCloseConcurrent's
issuesPerWorker reduced 5->2: under real concurrent-shard-load
measurement these were genuine (not queueing-artifact) serialized-write
costs; 2 preserves every invariant the test checks (duplicate-ID
detection across all workers, one non-decreasing-list-count check per
worker) at a fraction of the serialized work.
- large712ShapeName: the 712-shape's race skip and its zero-match guard now
share one named constant instead of two independent literal shape-name
strings, so a future rename of the shape can't silently skip the pinned
statement-count assertion under race (review N2).
- Comment-only fixes (review N5) in files referencing test names since
renamed/split by this slice's earlier work: close_proxied_integration_test.go,
graph_apply_test.go, storage_class_proxied_integration_test.go,
tests/regression/discovery_test.go, plus doc-comment path cleanups (N1:
drop unverified ~/beads-bazel-plan paths/bead ids in favor of the
committed duration JSON files and engdocs/TESTING.md).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): re-derive embedded-Dolt durations under concurrent load; cap cmd shards at Bazel's 50-shard limit (F1 round 2)
Coordinator review required re-deriving every per-test duration weight from
a single, consistent, CI-realistic measurement (all shards running
concurrently, -test.parallel=4, race-enabled) instead of the prior pass's
mix of crowded-shard captures and solo/4-together spot-corrections, and
required measuring per-shard wall time via BEP testAttemptDuration rather
than a shard's outer test.xml <testsuites time> (which can include a
different, already-finished shard-mate's setup/teardown).
- scripts/ci/embedded_cmd_test_durations.json (242 tests) and
scripts/ci/embedded_storage_test_durations.json (387 tests) rewritten in
full from one consistent sweep. Every entry is now a fresh measurement;
none is a carried-over capture-time value or an unverified estimate. Both
files' _comment blocks retract the earlier "queueing delay baked into a
test's own reported elapsed time" theory (testing.go's (*T).Parallel does
not add wait time to a blocked test's own clock) and document the real
methodology.
- Only one cmd-tier test remains over 200s (TestEmbeddedListFiltering,
239.70s), already under the ~240s ceiling; no further top-level splits
were needed this round.
- New finding: Bazel hard-caps shard_count at 50 per sh_test-like rule
("Having more than 50 shards is indicative of poor test organization";
a native analysis-time limit, not configurable). The previous 88-shard
plan for //cmd/bd:bd_embedded_test (chosen as the LPT-packed floor before
this limit was discovered) is reverted to 50, Bazel's real ceiling.
Repacking the fresh weights at 50 shards gives a heaviest shard of
409.6s -- above the single-test floor of 239.7s, because 50 bins is not
enough headroom over the suite's ~20145 measured test-seconds to flatten
every bin to that floor. This makes the <=5 min lane target unreachable
for the cmd tier via resharding alone within Bazel's native per-rule
limit; see ~/beads-bazel-plan/f1/impl-report.md section 3c for the full
writeup and the two remaining levers (suite-wide speedups or a
multi-Bazel-rule split), neither attempted in this slice.
- internal/storage/embeddeddolt gets 15 Bazel-only shards (unaffected by
the 50-shard cap): LPT-packed heaviest shard 225.3s, within target.
- cmd/bd/BUILD.bazel, internal/storage/embeddeddolt/BUILD.bazel,
.github/workflows/bazel.yml, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md,
scripts/embedded_shard_count_test.go, scripts/pr_risk_bazel_coverage_test.go,
scripts/ci/gen_embedded_cmd_shard_manifest.py updated to the corrected
50/15 shard counts (and, for the generator, N3: --write without an
explicit total_shards now errors instead of silently regenerating the
frozen legacy 20-shard block).
- .github/scripts/embedded-{cmd,storage}-test-shards.txt: re-packed
Bazel-only blocks at 50/15; frozen legacy 20/5 blocks untouched except for
N5 fixes to stale entries left by this slice's earlier test splits
(TestShardScriptsListOnlyRealTests catches this class of bug; both
legacy blocks reconfirmed exact-once).
- .github/workflows/nightly.yml / scripts/ci_workflow_test.go (N4): the
embedded-dolt batch-apply nightly step now runs with `if: ${{
!cancelled() }}` and its own explicit `-timeout 20m`, independent of the
job-level timeout, with both halves now asserted by
TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite.
Verified: gofmt -l clean, go vet ./scripts/... clean, go test ./scripts/...
green, both generators' --check pass at 50/15 (and legacy 20/5 remain
exact-once via the frozen-block invariant).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): rename embedded manifest staleness tests out of the ^TestEmbedded skip
The allowlisted go-test leg runs with -skip ^TestEmbedded, so tests named
TestEmbedded*ShardManifestGeneratorNotStale never ran there and the leg
failed with "matched no test that ran".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci: move same-repo PR Linux/Windows/macOS legs to Blacksmith, shard preflight (F7b)
PR-path jobs for same-repo PRs/merge_group now run on Blacksmith instead
of GitHub-hosted runners, following F4/F7a's ternary-expression and
restore-only-cache patterns; forks/Dependabot are unaffected:
- scripts-go-checks, pr-lint-wrapper: same-repo Blacksmith 4vcpu Linux.
- check-doc-freshness-platforms, pr-preflight-platforms: restructured
from a top-level matrix.os list to an include:-only "matrix marker"
shape (runner: same-repo-linux/-windows/-macos) so each host can
resolve an independent same-repo Blacksmith selection via one chained
ternary runs-on expression.
- Six standalone Windows jobs (test-windows-liveness,
worktree-remove-windows, test-windows-small,
test-windows-liveness-prebuilt, worktree-remove-windows-prebuilt,
windows-make-shell) move to same-repo Blacksmith Windows
(blacksmith-{2,4}vcpu-windows-2025, public beta). CGO/mingw-w64 and
Chocolatey preinstall are unconfirmed on that image; both risks are
flagged with one-line rollback instructions at the call sites.
- pr-preflight-platforms' Windows leg is split into two shards (common
steps on shard 1, the three Windows-only heavy steps on shard 2) to
shorten its critical path; shard 2 downloads Go modules directly
instead of sharing shard 1's runner/cache.
main.yml gains cache savers for the newly same-repo-Blacksmith PR
consumers, since Blacksmith and GitHub-hosted runners cannot see each
other's actions/cache entries even for identical keys:
- blacksmith-go-build-cache warms the Blacksmith-selection race/non-race
GOCACHEs scripts-go-checks and pr-preflight-platforms/check-doc-
freshness-platforms' ubuntu legs need, push-to-main only. Its non-race
leg shares scripts/ci/warm-non-race-cache.sh with build-artifacts, and
compiles tools/bazel/run_allowlisted_go_tests.py's allowlisted
packages via its new --compile-only flag, so the two paths and the
warm-up can never drift from what they actually run.
- pr-lint-wrapper, go-vet-cache, test-windows gain a venue: [blacksmith,
github] matrix leg to seed both runner selections pr.yml's
same-repo-Blacksmith-or-fork-hosted consumers can land on.
Updates the policy-test suite (ci_workflow_test.go,
ci_blacksmith_runner_test.go, pr_lanes_bazel_coverage_test.go) to cover
the new runner selections, matrix-marker shape, shard gating, and cache
topology, including a dedicated inventory/writer check for
blacksmith-go-build-cache.
Deferred follow-up (not done here): sharding check-doc-freshness-
platforms and the standalone Windows jobs, and moving/sharding macOS
legs beyond what check-doc-freshness-platforms'/pr-preflight-platforms'
matrix-marker form already covers.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): F7b review response -- msys2/Blacksmith-saver guards, macOS stays hosted (B1/B2/S3)
Addresses ~/beads-bazel-plan/f7b/review.md's two cache-safety blockers and
one follow-up gap found after F7b moved same-repo PR Linux/Windows/macOS
legs onto Blacksmith and added push-to-main Blacksmith cache seeders:
- B1: windows-make-shell's msys2/setup-msys2 step caches its installed
packages by default, which implicitly SAVED a GitHub Actions cache entry
from a same-repo PR the moment it ran on a Blacksmith runner (confirmed
via a real cache-list entry from a refs/pull/.../merge ref). Disabled with
a fail-closed `cache: ${{ runner.environment == 'github-hosted' }}`, and
generalized the sweep (TestBlacksmithReachablePRJobsDisableDefaultCachingActions)
to cover any default-caching setup-* action on a Blacksmith-reachable job,
not just this one.
- B2: blacksmith-go-build-cache, pr-lint-wrapper, go-vet-cache and
test-windows (main.yml) had no (or an incomplete) same-repo push-to-main
guard, so a same-repo fork with push access to its own main could run --
and save a cache entry from -- a paid Blacksmith job. Added the exact
push-to-main/same-repo `if:` F7c's own blacksmith-setup-go-cache already
uses, pinned by TestBlacksmithSaverJobsGuardedAgainstPullRequest.
- S3: the macOS leg of pr-preflight-platforms/check-doc-freshness-platforms
resolved to Blacksmith for same-repo PRs; Blacksmith bills macOS at
roughly 20x its Linux/Windows rate and no push-to-main Blacksmith-macOS
saver exists, so that leg would pay the premium for a guaranteed cold
cache. macOS now stays unconditionally on GitHub-hosted macos-latest.
Also pins S7 (blacksmith-go-build-cache/go-vet-cache/test-windows's cache
keys are bounded by go.sum+UTC-day, not github.sha) and S6/S8 (their
20-minute timeouts), both already in place but previously unpinned, and
replaces run_allowlisted_go_tests.py's fragile GO_TEST_FLAGS[-2:] slice with
an explicit BUILD_FLAGS constant (N1).
Re-ran the reviewer's 15 mutations (~/beads-bazel-plan/f7b/review/mutate.py):
all now killed or structurally moot (the Windows preflight shard split this
review asked to revert-unless-helpful was reverted, in an earlier commit on
this branch, removing the vulnerability surface 6 of them targeted).
scripts/BUILD.bazel: add the new ci_f7b_review_test.go to scripts_test's
srcs -- bazel test caught it missing (file existed on disk but was not
compiled into the Bazel-built test binary).
Verification: go vet/test/build, gofmt, bazel --nohome_rc test
//scripts:scripts_test, actionlint -shellcheck, shellcheck, make
ci-pr-policy, make bazel-sync-check all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): drop local review-doc path from comment
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(advisory): move Conformance/Regression/Migration/Smoke/Docs/Proxied-Local to Blacksmith, fold matrices, path-filter, drop redundant nix-build PR trigger (F7c)
Reduces per-PR GitHub-hosted job load for advisory (non-required) workflows
per the "PR CI -> 5 min" plan, slice F7c. None of these workflows gate the
required CI Gate / PR Risk Gate checks.
- conformance.yml: same-repo PR/merge_group runs move to Blacksmith
(4vcpu); PR trigger gets the shared "upgrade-relevant code" path filter
(any non-test Go change or build input); also now runs on push to main.
- regression.yml: fold detect-regression-need into regression's first step
(gated via `if:`), eliminating a whole job; move to Blacksmith.
- migration-test.yml: fold the 14-version x cache-identity matrix into 3
shards (src/pre-dolt/dolt-runtime), looping every version in a shard via
a new "Verify explicit historical upgrades" step so a bad version does
not hide the rest; gains the shared path filter; move to Blacksmith.
- cross-version-smoke.yml: fold the 1-job-per-version matrix into chunks of
5 (delegated to upgrade-smoke-test.sh's native SMOKE_VERSIONS support);
gains the shared path filter; move to Blacksmith.
- docs-mintlify.yml, proxied-local-smoke.yml: move to Blacksmith
(2vcpu/4vcpu); add missing timeout-minutes where absent.
- nix-build.yml: drop the pull_request trigger — pr-risk.yml's required
test-nix job (nix run + nix flake check) is a strict superset of
`nix build .#default`, pinned by a new regression test.
- main.yml: add blacksmith-setup-go-cache, a push-to-main-only job that
warms setup-go's implicit cache with cache: true, since Blacksmith
runners cannot see GitHub-saved actions/cache entries that the six
workflows above now rely on via setup-go's own caching. Carves a scoped,
documented exception into assertPinnedGoCacheActions for this one job;
every other setup-go step in main.yml/pr.yml/pr-risk.yml still must pin
cache: false.
Forks and Dependabot PRs are unaffected (still ubuntu-latest, no secrets),
via the same conditional runner expression used elsewhere in the repo.
New policy tests in scripts/ci_f7c_advisory_test.go pin: path-filter
identity across the three filtered workflows, full version-set coverage
of the migration-harness fold (no scenario dropped), the per-shard/per-
chunk retry-all-versions-even-on-failure loop, fork/dependabot runner
selection semantics, no Blacksmith job reads secrets, and that the new
cache-seed job exists and that its six consumers don't disable the cache
it warms. All new tests and their underlying behaviors were mutation-
tested (each fails when the behavior it pins is reverted).
Caveat for rollout: the Blacksmith cache-seed job assumes Blacksmith
labels have branch-protected cache isolation from fork/untrusted PRs at
the org level (spec-f7.md Group B precondition) — this needs confirming
with an org admin before merge, since the seed job's cache becomes shared
infrastructure the six advisory workflows implicitly trust.
Replayed the last 60 merged PRs' changed-file lists against the new path
filter: 20/60 would now skip Conformance/Migration/Smoke (matches the
spec's estimate), and none of the 60 touched non-test Go code while being
classified as skip.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,advisory): F7c review B1/B2 - pin migration fallback, restore-only Blacksmith caches
B1: migration-test.yml's non-Blacksmith historical-upgrades fallback was
`ubuntu-latest`, which actions/runner-images#14748 schedules to move from
24.04 to 26.04 starting 2026-10-19. 26.04 drops libicu74, which the v0.55.4
fixture's ICU ABI depends on (and which "Install test dependencies" installs
explicitly). Pin the fallback to the literal `ubuntu-24.04` label instead, and
add the matching sameRepoBlacksmith4vcpuNoble constant so the pin itself is
tested. Confirmed via `git show <pre-F7c>` that none of the other five
workflows F7c touched ever had a pinned-OS fallback, so this regression was
scoped to migration-test.yml only.
B2: Blacksmith cannot see GitHub-hosted cache entries, and a same-repo PR
writing a cache entry another PR or main later reads back unverified is a
poisoning vector. Make every advisory consumer (conformance, regression,
migration-test, cross-version-smoke, docs-mintlify, proxied-local-smoke)
disable setup-go's own implicit cache on self-hosted runners and restore-only
from a self-defined `blacksmith-sg-v1-...` key namespace (not setup-go's own
implicit key, so there is no dependency on Blacksmith's $ImageOS matching
GitHub-hosted resolution). Only main.yml's new blacksmith-setup-go-cache job
(push-to-main only, always trusted) saves: it always restores, unconditionally
re-runs every warm-up command (no longer gated on a cache hit, so a stale or
partial restore self-heals), and always saves under the same key. Delete the
now-unneeded goCacheImplicitCacheExceptions carve-out in ci_workflow_test.go.
Also convert cross-version-smoke.yml's smoke-binaries-* and regression.yml's
regression-baseline-* caches from a monolithic actions/cache (auto-saves on
any miss) into an explicit restore (always, modulo the job's own gate) / save
(gated off pull_request) pair, closing the same poisoning vector for
per-release binaries. migration-test.yml's historical-dolt-* cache is left
unchanged: it is already checksum-verified on every read by
scripts/migration-test/lib/binary.sh.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci,advisory): F7c review SF - behavioral tests, path filters, F7a naming coordination
Replace substring-matching tests with behavioral ones that execute the real
embedded bash/jq against synthetic input, closing the reviewer's 12 surviving
mutations:
- TestMigrationHarnessLoopExecutesBehaviorally runs the real "Verify explicit
historical upgrades" step against a stub run.sh: every version in a shard
still runs after an earlier one fails (closes `exit 0`/`break` mutations),
failures are ::error:: annotated and written to $GITHUB_STEP_SUMMARY, a bad
or empty SHARD_VERSIONS fails closed without ever invoking run.sh, and a
hung version is bounded by `timeout --kill-after` without hiding the version
queued after it. The step's env now exports SHARD so its run: body has no
literal ${{ }} GH-expression syntax left for a test harness to choke on.
- TestMigrationHarnessCacheHashMatchesDerivation pins each shard's cache-hash
against sha256(join(",", <shard's cache-identity strings>))[:16], recovered
from the pre-fold 14-entry table (git show 362eecc6bf), closing a stale-
cache-hash mutation.
- TestMigrationHarnessDoltRuntimeStepGatedByShard pins the exact gate
expression for "Install Dolt test runtime".
- TestCrossVersionSmokeChunksEveryResolvedVersion now executes the real jq
chunking and join programs (via the new ghExtractJQProgram helper) against
synthetic version lists of several sizes, and asserts the smoke step's run:
command is an exact match - closing jq-slice-off-by-one and
dropped-SMOKE_VERSIONS mutations a substring check could not catch.
- TestRegressionStepsAfterDetectAreGated pins that every step after
regression.yml's detect step stays gated on
steps.detect.outputs.run_regression == 'true', closing an
ungated-regression-step mutation.
- TestBlacksmithSetupGoSeedExistsForAdvisoryConsumers,
TestAdvisoryBlacksmithConsumersAreCacheRestoreOnly (replacing
TestAdvisoryBlacksmithConsumersKeepImplicitSetupGoCache),
TestBlacksmithSetupGoCacheKeysMatchAcrossSeederAndConsumers, and
TestAdvisoryBinaryCachesAreRestoreAlwaysSavePRGated cover the B2
restore-only/seeder-saves cache topology end to end, including a
seeder/consumer key-match check and scoping the "no save in a consumer job"
check to the blacksmith-sg-v1- namespace specifically so migration-test.yml's
legitimate, checksum-verified historical-dolt-* cache is not flagged.
Path filters: add scripts/migrate-legacy-to-current.sh to migration-test.yml
(the harness invokes it directly but it lives outside scripts/migration-test/)
and internal/storage/embeddeddolt/conformance_test.go plus backend/conformance/**
to conformance.yml, with a new TestConformancePushPathsMatchPullRequestPaths
pinning push.paths and pull_request.paths stay identical.
Coordination with F7a (ci/f7a-blacksmith-fold, lands before F7c): F7a
independently defines its own sameRepoBlacksmith4vcpu/2vcpu constants and a
TestSameRepoBlacksmithExpressionSemantics, and is moving a real GH-Actions
expression evaluator into a shared scripts test helper file. Per the
coordinator, rename this branch's same-named test to
TestF7cAdvisorySameRepoBlacksmithExpressionSemantics (deliberately left as the
same simple literal mirror it always was, not a competing evaluator) so it
does not collide with F7a's helper on rebase; at rebase time this test should
be reduced or replaced in favor of F7a's shared evaluator.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(ci): update stale nix-build PR-trigger and renamed check-name references (F7c review S6)
F7c (bb81136203) dropped nix-build.yml's pull_request trigger as redundant
with PR Risk's test-nix job, and folded cross-version-smoke.yml's one-job-
per-version matrix into one job per 5-version chunk. Update the docs that
still described the old topology:
- CONTRIBUTING.md: nix build .#default no longer runs on PRs; point at PR
Risk's required test-nix job instead.
- update-vendor-hash.yml: same correction in its header comment and the
GITHUB_TOKEN-retrigger caveat.
- engdocs/CI_TEST_SURFACE_AUDIT.md: migration-test.yml now also runs on PRs
(path-filtered, advisory) across 3 shards instead of 14 per-version legs;
nix-build.yml's row now reflects push-to-main-only.
- engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md: replace `Upgrade smoke
(<version> -> candidate)` with the current `Upgrade smoke (chunk N)` job
name in both the "do not require directly" list and the Cross-Version
Smoke section; rewrite the Nix Build section to describe nix-build.yml as
push-to-main-only (no longer a PR check to warn about requiring directly)
and remove the now-unreachable `nix build .#default` bullet from the
"do not require" list.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(advisory): rebase onto F7a shared Blacksmith helper, drop duplicate const/evaluator
F7a (PR #7139, ci/f7a-blacksmith-fold) landed its own sameRepoBlacksmith4vcpu
const and TestSameRepoBlacksmithExpressionSemantics, plus a real GitHub
Actions expression evaluator (evalGHExpr/mustEvalGHRunsOn) in a new shared
scripts/ci_blacksmith_runner_test.go. This branch rebased onto
origin/ci/f7a-blacksmith-fold and reconciles the two:
- Removed this branch's duplicate sameRepoBlacksmith4vcpu definition in
ci_workflow_test.go (rebase conflict); it's now served solely by F7a's
shared const. sameRepoBlacksmith4vcpuNoble (F7c's B1 fix, fallback pinned
to ubuntu-24.04 instead of ubuntu-latest) is kept as its own const, as
instructed, since it is F7c-specific and not shared.
- Replaced TestF7cAdvisorySameRepoBlacksmithExpressionSemantics's
hand-written Go mirror of the same-repo Blacksmith boolean logic with real
evaluation of each F7c advisory job's actual, as-parsed runs-on expression
text via F7a's evalGHExpr/mustEvalGHRunsOn, across the same event-shape
truth table F7a's own test uses (same-repo/fork/dependabot/deleted-fork-
head/merge_group/push/pull_request_target/schedule/workflow_dispatch).
This covers all 8 F7c advisory Blacksmith jobs, including
migration-test.yml's historical-upgrades job falling back to the literal
ubuntu-24.04 label rather than ubuntu-latest. Confirmed via a cp-backed
mutation (flipping the Noble fallback back to ubuntu-latest) that this new
test, unlike the deleted hand-written mirror, actually catches a
regression in the real expression text.
- Merged two more rebase conflicts in ci_workflow_test.go: the
assertNoUnmanagedGoCacheSteps main.yml allowlist (F7a's
windows-test-binaries-cache plus F7c's blacksmith-setup-go-cache) and a
subsequent non-conflicting-but-incomplete 3-way merge of
TestSameRepoBlacksmithRunners' main.yml entry (added
windows-test-binaries-cache alongside blacksmith-setup-go-cache so the "no
other Blacksmith label leaks" sweep covers both).
Did not merge TestBlacksmithAdvisoryJobsReadNoSecrets into F7a's
TestBlacksmithJobsReadNoSecrets (ci_workflow_test.go) even though the shared
helper file's doc comment suggests it; out of scope for this round per the
coordinator's explicit instructions, left as a follow-up.
Re-ran full verification after rebase + reconciliation: go build/vet/test
./scripts/..., bazel --nohome_rc test //scripts:scripts_test, actionlint
-shellcheck=shellcheck across all touched workflows, make ci-pr-policy, make
bazel-sync-check - all green. gofmt -l clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,advisory): close F7c re-review blocker X1 and S1-S5/N1 gaps
Re-review (~/beads-bazel-plan/f7c/rereview.md) found one blocker and five
should-fix gaps against ci/f7c-advisory-workflows:
X1 (blocker): migration-test.yml's historical-dolt-* cache was still a bare
actions/cache that auto-saves on any key miss on Blacksmith for same-repo
PRs. tar -P extraction plus lib/binary.sh's cp -f following a planted
symlink can redirect a write before sha256 verification ever runs, so the
"checksum-verified" exemption was unsound. Converted to the same
restore-always/save-off-PR split as smoke-binaries-*/regression-baseline-*,
dropped the exemption comments, and added it to advisoryBinaryCaches.
S1: TestRegressionDetectStepBehavioral executes regression.yml's detect
step's actual bash against a scratch git repo, closing mutation M13 (detect
step hard-wired to emit run_regression=false).
S2: TestBlacksmithReachableAdvisoryJobsNeverSaveACache sweeps every job in
every F7c advisory workflow (plus main.yml) whose runs-on resolves to a
Blacksmith label under a same-repo pull_request/merge_group context, and
fails on any bare actions/cache, any un-gated actions/cache/save, or any
setup-go/setup-node/setup-python step that would still cache there. Also
added a job-level `if: github.event_name == 'push' && github.ref ==
'refs/heads/main' && github.repository == 'gastownhall/beads'` guard to
main.yml's seeder job (TestBlacksmithSeederGuardedAgainstPullRequest) so a
future pull_request/merge_group trigger on main.yml can't turn the seeder
into a same-repo-PR-reachable cache writer.
S3: advisoryBinaryCaches' save-gate check is now exact-equality, not
strings.Contains, so a `... || true` gate can no longer slip past it.
S4: conformance.yml's path filters now include
internal/storage/embeddeddolt/test_fixture_test.go, which holds TestMain
for the whole package (moved out of conformance_test.go) and was previously
excluded by the blanket !**_test.go negation.
S5: main.yml's blacksmith-setup-go-cache seeder keyed its save on
github.sha, so every push to main - regardless of whether go.sum changed -
wrote a new multi-GB ~/go/pkg/mod + ~/.cache/go-build entry, risking LRU
eviction of F7a's own Blacksmith-seeded caches. Added a "Compute cache
date" step and keyed on the UTC calendar day instead, capping writes to at
most once per day; the same step/key was added to all 6 consumers so their
restore key stays byte-identical to the seeder's save key
(TestBlacksmithSetupGoCacheKeysMatchAcrossSeederAndConsumers).
N1: flipped `cache: ${{ runner.environment != 'self-hosted' }}` to
`cache: ${{ runner.environment == 'github-hosted' }}` on setup-go's own
implicit cache across all 7 call sites, so an empty/unknown
runner.environment value fails closed (caching off) instead of open.
Verification: go build/vet/test ./scripts/..., bazel --nohome_rc test
//scripts:scripts_test, actionlint -shellcheck=shellcheck
.github/workflows/*.yml, make ci-pr-policy, make bazel-sync-check all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,advisory): close merge_group gap in binary-cache save gates
Running TestBlacksmithReachableAdvisoryJobsNeverSaveACache against every
Blacksmith trust context it is reachable under (instead of stopping at the
first match found, which made the check's coverage depend on Go's
randomized map iteration order) surfaced a real gap the S2 sweep was meant
to close: smoke-binaries-*, regression-baseline-* and historical-dolt-*'s
save gates were all `if: github.event_name != 'pull_request'`, a deny-list
that also admits a hypothetical future merge_group event. All three jobs'
runs-on expressions already resolve to a Blacksmith label under
merge_group, so a merge_group run would have been able to write those
caches too - the same same-repo-writer risk X1 closed for pull_request,
just not currently reachable since none of these three workflows has an
`on: merge_group:` trigger today.
Changed all three save gates to an exact allow-list of the triggers they
actually intend to write from (push, workflow_dispatch) rather than a
deny-list of the one trigger they don't. Updated
TestAdvisoryBinaryCachesAreRestoreAlwaysSavePRGated's wantSaveIf strings to
match, and made TestBlacksmithReachableAdvisoryJobsNeverSaveACache check
every blacksmithTrustContexts entry per job (sorted for determinism)
instead of breaking on the first Blacksmith-reachable context, so this
class of context-dependent gate gap can't hide behind map iteration order
again.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci,advisory): directly assert main.yml has no PR-reachable trigger
TestBlacksmithSeederGuardedAgainstPullRequest's job-level guard defuses the
vulnerability structurally, but a mutation that only edits main.yml's on:
block (adding pull_request:) without touching the job-level if: was not
caught by any existing test (confirmed: mutate.py's N6 probe survived).
Add TestMainWorkflowHasNoSameRepoPRReachableTrigger, the direct half of
F7c review fix S2's "pin the seeder's trust boundary" ask: assert main.yml's
on: has none of pull_request/pull_request_target/merge_group/workflow_run,
and still has push. Confirmed this kills N6 (manually applied and reverted
via a cp backup).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37159233834
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* ci: F7a same-repo Blacksmith moves and job folds (pr.yml, pr-risk.yml)
Move cache-free same-repo pr.yml/pr-risk.yml jobs onto the org's Blacksmith
runner pool (forks/Dependabot stay ubuntu-latest), and fold several
independent single-purpose jobs into fewer multi-step jobs, to cut same-repo
PR queue time without changing either required gate's name or its token
contract:
- fast-checks: folds check-build-tags, check-version-consistency,
check-migration-hygiene, check-no-beads-changes and fmt-check into one
Blacksmith job with five independent steps; ci-gate reads each step's
outcome via job outputs into the same CI_GATE_REQUIRED token the standalone
job used, plus a FAST_CHECKS backstop token for a lost job.
- check-release-target-cross-compilation: 8 per-target matrix legs collapse
to 2 (unix/desktop), driven by the new scripts/ci/release-targets.txt
manifest (kept in sync with .goreleaser.yml) and
scripts/ci/check-release-cross-compile.sh.
- advisory-reports: folds build-examples and complexity-report (both
already advisory) into one job, keeping their pre-fold
step-level continue-on-error/timeout design.
- windows-make-shell: 3-way host matrix folds into one job with an
aggregator step reproducing the old native/MSYS2/Cygwin pass/fail rules;
test-windows-doltversion and test-windows-dbproxy-server fold into
test-windows-small (neither was gated).
- Runner moves: fast-checks, advisory-reports,
check-release-target-cross-compilation, check-doc-flags,
pr-policy-wrapper, test-dolt-server-fingerprint (pr.yml) and test-nix
(pr-risk.yml) move to the same-repo Blacksmith expression at their
measured vCPU size.
Policy tests: extend TestSameRepoBlacksmithRunners for the moved/added jobs,
add TestSameRepoBlacksmithExpressionSemantics, TestBlacksmithJobsReadNoSecrets
and TestPRCIGateFastChecksTokens, retarget the complexity/build-examples and
dolt-server-fingerprint tests, rewrite
TestReleaseTargetCrossCompilationMatrixMatchesGoreleaser against the new
manifest, and teach the ci-gate simulation helper to resolve
needs.fast-checks.outputs.*. Document the moves/folds and the Blacksmith-off
revert set in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): windows-make-shell/test-windows-small fail natively, raise advisory-reports headroom (review SF-1, SF-2, N-2, N-3)
- windows-make-shell: drop continue-on-error from install/exercise steps
(kept only on the Cygwin setup step), delete the hand-written "Fail if a
required Windows Make shell leg failed" aggregator, and delete the unused
native/msys2/cygwin job outputs. A failing leg now reds the job via native
GitHub Actions propagation instead of relying on an aggregator step that a
mutation showed could silently undershoot (SF-1, N-3).
- test-windows-small: same fix - drop continue-on-error from doltversion/
dbproxy-server steps and delete their aggregator step.
- Add timeout-minutes: 10 to the MSYS2 and Cygwin "Set up ... GNU Make"
steps, so a hung installer download can no longer take out the whole job
(SF-2).
- advisory-reports: raise timeout-minutes from 45 to 60 for more headroom
above the sum of its step timeouts (N-2).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): real GH-expression evaluator shared with F7c, behavioral xcompile test, secret/run-command coverage (review SF-3, SF-5, SF-6, N-1, N-4, N-6)
- New scripts/ci_blacksmith_runner_test.go: a real GitHub Actions expression
evaluator (tokenizer + recursive-descent parser for ==, !=, &&, ||, !,
parens, string literals, dotted lookups, with GitHub's short-circuit-
returns-operand && / || semantics) that evaluates the REAL pinned
sameRepoBlacksmith{2,4,8}vcpu expression strings from pr.yml/pr-risk.yml,
replacing the old TestSameRepoBlacksmithExpressionSemantics' tautological
hand-written Go mirror (SF-3). This file is the shared helper F7c's own
TestSameRepoBlacksmithExpressionSemantics should rebase onto - see its
doc comment for the "Public surface" (the three consts, evalGHExpr, and
mustEvalGHRunsOn) (SF-5).
- ci_workflow_test.go: drop the old consts/test now living in the shared
file; extend TestBlacksmithJobsReadNoSecrets to also scan job.If and
step.If for secret references, not just env/with/run (N-4; container/
services credentials and matrix-marker resolution stay deferred to F7b);
pin fast-checks' build-tags step invoking all 3 scripts and the
migration-hygiene BASE_SHA env, and add
TestPRWindowsSmallDbproxyServerIsCGODisabled (N-6); add
TestPRWindowsMakeShellLegsFailTheJob and
TestPRWindowsMakeShellSetupStepsHaveTimeouts pinning the SF-1/SF-2 fixes.
- New scripts/check_release_cross_compile_test.go: behavioral test for
check-release-cross-compile.sh using a fake `go` on PATH - asserts a
failing target propagates as non-zero, every target in the group is still
attempted, every invocation passes CGO_ENABLED=0 and the gms_pure_go tag,
and an unknown group fails loudly with zero invocations (SF-6).
- check-release-cross-compile.sh: document the deliberate sequential-build
(vs. the spec's 2-wide suggestion) choice and its revisit trigger (N-1).
- scripts/BUILD.bazel: add the two new test files to scripts_test's explicit
srcs list (not a glob); verified against `make bazel-sync-check`.
All new/changed tests mutation-tested: each production-code mutation was
caught by the relevant test, then reverted via `cp` from a /tmp backup and
diff-verified byte-identical (never git stash / git checkout --).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(ci): fix stale check names and aggregator description in CI_REQUIRED_CHECK_TOPOLOGY.md (review N-7)
- Replace the stale "Check build-tag policy", "Check version consistency",
"Check for .beads changes" and "Check formatting" entries under "Do not
require these existing check names" with the actual post-fold name,
"Fast checks (build tags, versions, migrations, beads diff, fmt)", and add
the cross-compilation and windows-make-shell fold's actual check names.
- Rewrite the windows-make-shell fold description: it no longer has a
hand-written aggregator step (deleted per SF-1 in the prior commit), and
"the separate cygwin job" is now described correctly as a pre-fold matrix
leg, not a job.
- Update the advisory-reports timeout mention from 45 to 60 minutes (N-2).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(ci): drop local design-doc path from topology doc
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): fork and Dependabot PRs execute remotely on rbe-fork when rbe-west's mint is open (F9)
Fork and Dependabot pull_request runs get no secrets, no OIDC token and no
repository variables, so rbe-west decides for them. bazel.yml's rbe job asks
rbe-fork-mint's /v1/status (rbe-mint.ops.gascity.com:8444) once per run:
- open, tier ro: mode fork-ro (instance oss-fork, the fork pool, nothing
cached for anyone);
- open, tier rw (allowlisted author's own fork, allowlisted trigger): mode
fork-rw (instance oss, the OSS pool);
- closed, refusing, unreachable or anything else: mode cache, exactly as
today (local execution plus the anonymous read-only cache).
In the fork modes every lane runs, remotely, from a GitHub-hosted runner:
setup-bazel makes an EC P-256 key and a CSR, uploads the CSR as an artifact
of this run (the mint's proof), gets a ~2 h certificate
(fork-credential.sh, a byte copy of gascity's tools/rbe/fork-credential.sh
from F6) and writes an rc for grpcs://rbe-fork.ops.gascity.com:8444 with
the mint's instance. A lane whose certificate names another tier than the
rbe job fails ("re-run all jobs"). Fork lanes add --config=sole-run like
mode remote.
BAZEL_COVERS_FORKS ships "false" in pr.yml and pr-risk.yml: forks and
Dependabot keep their legacy tiers, and the Bazel lanes run beside them
(remotely while rbe-fork is open). F13 commits "true"; then a covered fork
run in mode cache is red, like a covered same-repo PR without the secret.
Safe while rbe-fork is dark: with the gate closed the mint's port resets
connections and the status probe falls back to mode cache in seconds; with
the mint up but disabled, canaried or rw off it answers open false or 403,
also mode cache.
Not in this change (later slices): bazel-farm.yml and its inputs stay (F12),
bazel.yml keeps its push trigger and pr.yml/pr-risk.yml gain none (F14).
keep their go build fallback on GitHub-hosted runners.
Tests: the rbe decision against a stubbed mint (ro, rw, closed, rw closed,
canary, busy, garbage, unknown tier, unreachable); the coverage and gate
matrices learn the fork modes, BAZEL_COVERS_FORKS and mint answers that
change between runs; TestBazelCacheModeReachesTheRC drives setup-bazel's
fork steps against a stand-in mint and checks each lane's rc;
fork-credential.sh's client contract and write-bazelrc.sh's fork mode and
exclusivity; the endpoint pin is allowed as exactly its one line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): fork-credential.sh retries the mint's 429 and never sleeps after the last try
A 429 (per-minute rate, live-certificate quota) may pass on its own like a
502, so it is retried with the same 10/20/30 s backoff (one minute, the
rate window); the loop no longer sleeps after the fourth failed attempt.
The cert request and the rbe job's /v1/status probe give up connecting
after 5 s, so a closed gate (dropped SYNs) costs seconds, not minutes.
Byte-identical to gascity's tools/rbe/fork-credential.sh.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: gascity-agent <agent@gascity.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(F4): advisory Linux-cross-built Windows test binaries, macOS cache restore
Side-by-side rollout slice of the PR CI -> 5 min plan (F4):
- Add pr.yml `windows-test-binaries`: a Linux job that cross-compiles
bd.exe and two go-test-c binaries (cgo and nocgo) for windows/amd64
via mingw (CC/CXX=x86_64-w64-mingw32-*, matching release.yml's
goreleaser setup), driven by a committed manifest
(scripts/ci/windows-test-binaries.txt) and build script
(scripts/ci/build-windows-test-binaries.sh). Artifacts are uploaded
and consumed by two new advisory "-prebuilt" twins
(test-windows-liveness-prebuilt, worktree-remove-windows-prebuilt)
that run the identical test selection/flags/env (including
BEADS_TEST_REPO_ROOT) against the prebuilt binaries via
scripts/ci/run-go-test-binary.sh and scripts/test.sh's new
BEADS_TEST_PREBUILT_TEST_BINARY short-circuit.
- The original native Windows jobs are untouched and remain required;
the new jobs are advisory only (not in CI_GATE_REQUIRED) for this
rollout phase. Flipping to the cross-built path later is a one-line
CI_GATE_REQUIRED change.
- Add main.yml `windows-test-binaries-cache`: seeds the Go build cache
for the cross-compile toolchain/packages on a schedule, mirroring
the existing module/build cache warmers; reorder build-artifacts'
warm-cache step to precede the module-cache save for a cleaner
build -> warm -> save grouping.
- Restore macOS's native build-cache save step in main.yml (lost
during an earlier job consolidation).
- Remove the non-required "Test (macos-latest)" full-suite job from
PRs; the native macOS gate jobs still run, so required coverage is
unaffected. This does drop a non-required pre-merge macOS race
signal; see docs/notes for the accepted tradeoff.
- Add scripts/windows_test_binaries_manifest_test.go: policy tests
pinning the manifest's three advisory-phase entries, pinning the
mingw toolchain to release.yml's, checking consumer jobs only
reference manifest-listed binaries, checking upload/download
artifact names and `needs` match, checking the prebuilt jobs select
exactly the same tests (-run literals) as their native counterparts,
and checking none of the new jobs reference secrets (fork safety).
All mutation-tested.
- Add two scripts/ci_workflow_apt_get_timeout_test.go budget entries
for the new "Install mingw cross-compiler" steps.
- Fix two latent Bazel-only scripts_test failures surfaced while
wiring the new test file into scripts/BUILD.bazel: materialize
cmd/bd's sources via the existing //cmd/bd:go_srcs filegroup so
sourceRepoRoot(t)-based package-directory checks work under Bazel's
runfiles root, and strip Bazel's own TEST_TIMEOUT env var before
exercising scripts/test.sh's hardcoded default in
test_script_test.go (name collision with test.sh's own override
knob).
Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint; shellcheck; make ci-pr-policy;
make bazel-sync-check; a from-scratch /tmp clone cross-build using
the exact job commands (cold 130.2s, warm/cache-hit 30.3s, PE32+
x86-64 validity via file/objdump, byte-identical SHA-256 across
cold/warm/traced runs); wine -test.list confirms
TestPrimeBinaryPortfolio/TestIsServerProbablyRunning* present in the
cgo binary and the full TestWorktreeRemove* family (including
Windows-specific variants) present in the nocgo binary.
Scoping note: only 2 of the spec's 4 consumer jobs (liveness,
worktree-remove) get prebuilt twins, and only 3 of 8 spec-listed
manifest binaries are built this slice; preflight, doc-freshness and
credential-fixture consumers are deferred as documented follow-up.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): address F4 review findings (SF-1..SF-6 + nits)
Addresses the F4 review (~/beads-bazel-plan/f4/review.md): no blockers, six
substantive findings plus several nits.
- SF-1: main.yml's windows-test-binaries-cache job now seeds the Windows
cross-compile cache from both runner selections pr.yml's consumer job can
land on (Blacksmith for same-repo PRs/merge_group, ubuntu-latest for
forks/Dependabot) via a 2-leg matrix with identical keys/paths, instead of
a single ubuntu-latest leg Blacksmith runners can never see.
- SF-2: removed the two consumer-less "Warm non-race Go build cache
(test-only packages)" steps from main.yml (build-artifacts, test macOS
leg); they had no real consumer and cost ~270s on main's macOS leg for
nothing.
- SF-3: scripts/ci/run-go-test-binary.sh now injects the same
-test.timeout=10m default (when -timeout is omitted) and unconditional
-test.paniconexit0 flag `go test` itself injects; added
timeout-minutes to test-windows-liveness-prebuilt; added a direct
regression test (TestRunGoTestBinaryDefaultsTimeoutWhenOmitted) for the
omitted-timeout branch, which no path through scripts/test.sh can reach
(test.sh always supplies its own -timeout default first).
- SF-4: scripts/test.sh's BEADS_TEST_PREBUILT_TEST_BINARY branch now runs
run-go-test-binary.sh as a child process (capturing and re-exiting its
status) instead of `exec`-ing it, so the EXIT trap (beads_test_env_cleanup)
still runs and the per-run BEADS_TEST_ENV_ROOT temp dir (and, with
BEADS_TEST_SHARED_SERVER=1, the shared dolt sql-server) is cleaned up
instead of leaked/orphaned.
- SF-5: closed the five surviving mutations (cgo/nocgo swap between the
Windows prebuilt twins, dropped -count=1, an unguarded twin in
ci-gate.needs, a goreleaser compiler change, a PKGDIR change) with
dedicated policy tests, and replaced the three-edits-per-job required/
advisory toggle with a single committed WINDOWS_PREBUILT_REQUIRED flag
(pr.yml top-level env) in the style of the existing BAZEL_RETIRES_LEGACY_*
flags: both Windows pairs are now unconditionally in ci-gate.needs and
always evaluated, and only which pair lands in CI_GATE_REQUIRED vs.
CI_GATE_SKIPPED_OK is decided by the flag, in one place.
- SF-6: dropped the //cmd/bd:go_srcs data dep from scripts_test; the
prebuilt-binary contract test now exercises ./scripts/ci (already a data
dep of this target) instead of ./cmd/bd, since it never builds or runs the
real package.
- Nits: removed dangling ~/beads-bazel-plan references and a nonexistent
test name from committed comments; log the resolved mingw-w64 version
when cross-compiling; removed a no-op shellcheck disable comment;
documented why run-go-test-binary.sh's BEADS_TEST_REPO_ROOT re-export
doesn't race beads_test_env_enter's unset; rewrote the pr.yml tombstone
comment for the removed macOS PR job to state the actual, narrower
tradeoff instead of an overclaiming "main.yml covers this" statement.
Verified: bazel --nohome_rc test //scripts:scripts_test passes; actionlint
clean on pr.yml/main.yml; shellcheck clean on all edited scripts (only
pre-existing, unrelated warnings remain elsewhere); make bazel-sync-check
and make ci-pr-policy both pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(scripts): isolate prebuilt-binary contract test from ambient hermetic env
TestTestScriptPrebuiltTestBinaryContract's "cleans up BEADS_TEST_ENV_ROOT
after the binary exits" subtest (added for F4 review finding SF-4) passed
locally but failed in CI's scripts-go-checks/scripts-test leg.
scripts/ci/scripts-go-test.sh (the CI driver for this package's `go test
./scripts/...` run) calls beads_test_env_enter in its own shell before
invoking go test, so BEADS_TEST_ENV_ACTIVE=1 and an outer
BEADS_TEST_ENV_ROOT are already present in this test binary's own
os.Environ(). runTestScriptPrebuiltBinary built the scripts/test.sh
subprocess's env from os.Environ() (minus TEST_TIMEOUT), so that ambient
state leaked through. Inside the subprocess, scripts/test.sh's own
beads_test_env_enter call (test.sh:17) saw the already-active guard and
no-opped: no new root, no new EXIT trap armed in that process - so the
fake binary observed the outer, still-alive root, and the subtest's
"directory is gone after test.sh exits" assertion failed, even though the
SF-4 fix (run the prebuilt binary as a child instead of exec'ing it) is
correct.
Every real production caller of scripts/test.sh's prebuilt path (pr.yml's
Windows liveness/worktree-remove prebuilt steps) runs it as the first
command of a fresh GitHub Actions `run:` step, never nested under an
already-entered hermetic env, so this nesting is purely an artifact of
this test binary's own ambient environment under scripts-go-test.sh, not
a real CI behavior gap.
Strip BEADS_TEST_ENV_ACTIVE/_ROOT/_DISABLE/_KEEP (alongside the existing
TEST_TIMEOUT filter) from the child env in runTestScriptPrebuiltBinary so
the subtest deterministically exercises scripts/test.sh's real top-level,
non-nested invocation regardless of what ambient hermetic-env state this
test binary itself happens to be running under.
Verified: reproduced the CI failure locally by exporting
BEADS_TEST_ENV_ACTIVE=1/BEADS_TEST_ENV_ROOT before `go test ./scripts/...`;
confirmed the fix resolves it; re-ran bazel test //scripts:scripts_test
and scripts/ci/scripts-go-test.sh directly, both pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(lint,scripts): fold duplicate Lint job; matrix PR Lint and scripts-go-checks (F5)
Lint (golangci-lint-action, only-new-issues) was functionally identical to
pr-lint-wrapper's native leg (H7): same pinned binary, config, flags, and
diff scope. Fold it into the wrapper in both pr.yml and main.yml, and drop
LINT from pr.yml's ci-gate.
pr-lint-wrapper becomes a 3-leg matrix (native/windows/darwin, fail-fast:
false) under the same PR_LINT_WRAPPER gate id, installing golangci-lint
v2.10.1 from a pinned, sha256-verified release binary
(scripts/ci/install-golangci-lint.sh) instead of go install/the Action, with
a per-leg golangci-lint cache (pr.yml restores only; main.yml's matrix is
the sole saver). scripts-go-checks becomes a 3-leg matrix
(scripts-test/vet/allowlisted, fail-fast: false) under the same
SCRIPTS_GO_CHECKS gate id, giving go vet its own build cache instead of
sharing the race build cache. main.yml gains a go-vet-cache job to warm that
cache on push; build-artifacts remains the Go module cache's sole saver.
Splitting each job's previously-sequential passes into parallel matrix legs,
combined with removing the duplicate Lint job outright, takes wall-clock
time off the PR gate's critical path.
Updates scripts/ci_workflow_test.go and scripts/pr_lanes_bazel_coverage_test.go
for the new matrices, cache topology, and the removed LINT id; adds
TestGolangciLintInstallScriptPinned. Updates engdocs/LINTING.md and the live
prose in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md (its disclaimed "historical
snapshot" gate-wiring block is left as-is).
Verified: go test ./scripts/..., bazel --nohome_rc test //scripts:scripts_test
(after adding engdocs/LINTING.md and .githooks/pre-commit as scripts_test
data), actionlint, shellcheck, make ci-pr-policy, make bazel-sync-check.
Confirmed lint-leg parity against current main with the pinned binary (0
issues PR-scoped; the same two pre-existing unparam findings full-tree,
unchanged) and scripts-go-checks leg parity standalone. Mutation-tested the
new/updated policy assertions with six deliberate breaks (dropped matrix
leg, reintroduced LINT, corrupted the pinned sha256, removed a cache-save
step, dropped the vet leg, reintroduced the lint job); each was caught with
an on-target failure.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): review-fix pass on F5's lint/scripts-go-checks matrix (SF-1..SF-5, N-1..N-8)
Addresses all SHOULD-FIX and NIT findings from the review of 7e858e2714
(no BLOCKERs):
- SF-1: lint/vet cache saves now run on !cancelled() instead of bare
cache-miss, so a red lint/vet run (main is currently lint-red on two
pre-existing, unrelated unparam findings) still warms the cache for
PR legs.
- SF-2: ci-measurements.yml now exports golangci-lint's install dir onto
PATH in the same step, since $GITHUB_PATH only takes effect in later
steps.
- SF-3: adds a behavioral test (fake curl/tar) proving the install
script's sha256 verification fails closed.
- SF-4: drops the commit sha from the golangci-lint cache key (saves
only on config/dependency change, not every push) and trims the
windows/darwin legs to cache only the small golangci-lint result
cache, not their large GOCACHE.
- SF-5: removes the vet cache's dead non-race-build-cache fallback
restore key (different path list, can never hit) in both pr.yml and
main.yml.
- N-1..N-8: doc/comment accuracy fixes, restores the "moved code reads
as new" caveat, adds a module-cache restore step and timeout-minutes
to pr-lint-wrapper/go-vet-cache, pins fetch-depth/GOCACHE via new
policy-test assertions, and flags a superseded doc snapshot.
New commit on top of 7e858e2714; existing commit trailers are left
untouched per explicit instruction. Not pushed, no PR opened.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37105481015
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* ci(bazel): raise proxied-server tier from 15 to 30 shards (F2)
bazel.yml's bazel-proxied lane (//cmd/bd:bd_proxied_test, run only with
remote execution) was sharing PR Risk's legacy 15-shard, frozen
bd-init-cost-proxy manifest block, so its longest shard was the long
pole of the Bazel-only lanes. Give it its own 30-shard block in
.github/scripts/proxied-cmd-test-shards.txt, bin-packed
longest-processing-time-first by measured wall-time
(scripts/ci/proxied_test_durations.json, captured from two bazel-proxied
job failures' testlogs) instead of the bd-init count proxy, via
scripts/ci/gen_proxied_shard_manifest.py's new --weights=duration mode.
Coverage is proven equal to the full discovered test set by two
independent methods (sorted-name diff and an actual 30-shard invocation
of proxied-test-shard.sh in list-only mode), both exact matches with no
duplicates.
cmd/bd:bd_proxied_test's shard_count moves to 30; PR Risk's and
main.yml's own test-proxied-cmd jobs keep their 15-shard matrix and
frozen manifest block untouched, per D2's retirement design (the Bazel
lane only needs to run the same tests, not the same split). Updated the
two coupled policy tests (scripts/ci_workflow_test.go's
TestBazelDoltServerTiersMirrorPRRisk, scripts/pr_risk_bazel_coverage_test.go's
TestBazelRetiredLanesCheckListedTestsRan) to allow the Bazel target's
shard_count to differ from its legacy job's own matrix size, and
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md / engdocs/TESTING.md to document
the split.
Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint on bazel.yml; make bazel-sync-check;
make ci-pr-policy; 3 representative shards (1, 15, 30 of 30) run
directly against Bazel-built race bd_test/bd_for_tests binaries and the
pinned dolt 2.2.0 release, all green, 55-81s wall time each (the
duration data's absolute seconds are from a contention-era sample and
run far faster here; the LPT balance itself, not the absolute SLA, is
the property that matters).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): close fork-PR gap in proxied 30-shard manifest validation (review B1/S1)
TestShardScriptsListOnlyRealTests only ran proxied-test-shard.sh at PR
Risk's legacy 15-shard matrix, never at the Bazel-only bazel-proxied
lane's 30-shard manifest block. bazel-proxied only runs in mode=remote
(skipped on fork PRs), so a fork PR could rename, duplicate, or corrupt
a TestProxiedServer*/TestServerMode* manifest entry in the 30-block and
merge green via go test ./scripts/... (which runs on every PR), only
breaking the required lane afterward on main and every same-repo PR.
Fix: the test now computes every shard total it must validate per
script (this job's own PR Risk matrix size, every distinct total found
in that script's own committed manifest, and - for proxied-test-shard.sh
- cmd/bd:bd_proxied_test's live shard_count from BUILD.bazel) and runs
the full per-shard validation loop at each one, relying on the script's
own duplicate/stale-name guards. It also sums each total's "manifest: N"
counts across all shards and fails if they sum to zero, catching a
silently-deleted block that would otherwise fall back to 100% hashing
undetected.
Also fold in S1: replace the independently hard-coded `30` overrides in
ci_workflow_test.go and pr_risk_bazel_coverage_test.go with
bazelProxiedShardCount(t), a new shared helper that reads
cmd/bd:bd_proxied_test's shard_count live from cmd/bd/BUILD.bazel. This
turns the check_shard_coverage.py argument comparison in
pr_risk_bazel_coverage_test.go into a genuine cross-file pin (BUILD.bazel
vs bazel.yml) instead of two literals that could drift together
unnoticed. ci_workflow_test.go's tiers loop no longer asserts
BUILD.bazel's shard_count against a value read from the same file
(tautological); it instead asserts pr-risk.yml's and main.yml's legacy
test-proxied-cmd matrices equal each other, restoring the pin S1 found
removed on this branch relative to base.
Verified against the reviewer's mutations (duplicate entry, bogus
entry, whole-block deletion, stale rename, matrix divergence, BUILD
shard_count drift, bazel.yml arg drift): each now fails go test
./scripts/..., where it previously passed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): add generator --write/--check and freeze legacy manifest block (review S3/N1/N4/N5)
gen_proxied_shard_manifest.py:
- Add --write (rewrite only the target total_shards block in place,
leaving every other block's header/body text untouched) and --check
(regenerate in memory and diff against the committed block, exiting
1 with the exact --write command needed if stale). Both operate via
new split_blocks()/write_block()/check_block() helpers that round-trip
the real manifest file losslessly.
- N1: fix the duration-weights header's false claim about the
fallback ratio.
- N5: warn on stderr when proxied_test_durations.json has duration
captures for tests that no longer exist in cmd/bd/*_test.go (renamed,
split, or deleted since the last capture).
proxied-cmd-test-shards.txt:
- N4: mark the legacy 15-shard block's header as frozen for
pr-risk.yml's/main.yml's legacy fork/push jobs: it is deliberately
not kept in sync with newly-added tests (those hash-fallback
instead), so regenerating it is not routine maintenance. Only the
header text changed; all 149 explicit assignment lines and the
"known blind spots" callouts are untouched (verified: a fresh
--write would pack all 164 currently-discovered tests explicitly,
silently changing which shard runs which test for this block, which
is exactly what "frozen" means to avoid).
- Regenerate the Bazel-only 30-shard block's header via
`gen_proxied_shard_manifest.py 30 --weights=duration --write` to
describe --write/--check instead of the old manual-copy-paste
instructions. proxied_test_durations.json has not changed yet (S2
is still in progress), so this run only touched header text; the 164
assignment lines are unchanged.
pr_risk_bazel_coverage_test.go:
- Add TestProxiedShardManifestGeneratorNotStale: runs
gen_proxied_shard_manifest.py 30 --weights=duration --check under
plain `go test` (skipped under `bazel test`, whose runfiles hold
neither the generator's sources nor cmd/bd) so a stale 30-block
(e.g. committed by hand, or left behind after adding tests without
regenerating) fails scripts-go-checks on every PR, including forks.
tools/bazel/equivalence_allowlist.txt: allowlist the new test's guarded
Bazel skip.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): rename 14 undiscovered proxied-gated tests; N2/N3/N6 doc fixes
cmd/bd/*_test.go: 14 tests call requireProxiedServerEnv/
requireSharedProxiedServer but their names did not match the shard
scripts' discovery regex ^Test(ProxiedServer|ServerMode), so no CI lane
ever set BEADS_TEST_PROXIED_SERVER=1 for them and they silently skipped
everywhere (pre-existing gap, not caused by F2; list was in
~/beads-bazel-plan/f2/review/undiscovered_gated.txt).
Fixed by renaming rather than widening discovery: 162 of the 164
already-discovered tests call one of the two gate functions in the first
few lines of their body, so name and gate already correlate almost
perfectly, but the 14 tests' names are each styled differently with no
single regex term that would cover all of them without a hand-maintained
list. Renaming lets the existing discovery mechanism, hash-fallback
behavior, and both shard-coverage policy tests pick them up with zero
other changes. Checked each of the 14 names for cross-references first;
each appeared only in its own file plus at most one doc-comment in a
sibling file, both updated alongside the rename. No collisions with
existing test names.
Discovery went from 164 to 178 tests. Regenerated the Bazel-only 30-shard
block (gen_proxied_shard_manifest.py 30 --weights=duration --write) to
pack the 14 new tests; this is the first --write this round to actually
change assignments, not just header text. The frozen legacy 15-shard
block (N4) was deliberately left alone: the 14 renamed tests now
hash-fallback into it, exactly like every other test added since it was
last generated.
cmd/bd/BUILD.bazel (N6): restore the bd_proxied_test comment sentence
bazel.yml's mirror comment kept but BUILD.bazel's had dropped: PR Risk
builds its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd
twin uses build-artifacts' non-race bd-cmd-test, so this lane is the
stricter of the two.
.github/workflows/bazel.yml, .bazelrc (N2/N3): remove stale timing
language ("30 shards of ~5-10 minutes" / ".bazelrc's leftover "15 shards
of ~10 minutes") never reconciled with this task's own local
representative-shard measurements (55-81s/shard); replaced with the
concurrency argument (30 simultaneous dolt sql-servers don't fit a
GitHub-hosted runner regardless of any single shard's wall time). Also
removes the "within 5s of each other" balance claim (S2): the underlying
duration weights are contaminated by -test.parallel overlap in the
top-level <testsuite time> figures they were derived from; a corrected
re-measurement is in progress (separate commit once it lands).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied duration weights without overlap double-counting (review S2)
scripts/ci/proxied_test_durations.json's 178 entries are now each a
clean, single-job (no concurrent heavy bazel/dolt job on the box),
-test.parallel=1 measurement: a fresh manifest-aligned 30-shard sweep's
own <testsuite time> per top-level test where errors="0" (including
failing-but-complete suites, whose elapsed time is still trustworthy),
plus, for the handful too slow to finish inside Bazel's default
per-target timeout, a `bazel run` leaf-subtest-only sum that never adds
a parallel group's own PASS/FAIL line on top of its already-overlapping
children's lines. An earlier pass at this data summed every PASS/FAIL
line indiscriminately, double-counting parallel subtest groups.
A first attempt at re-measuring the slow stragglers overlapped in
wall-clock time with a second, unrelated heavy sweep and came out 2-3x
inflated versus the same tests measured cleanly; that contaminated data
was discarded in favor of isolated re-runs. TestProxiedServerDep2
initially looked hung during its isolated re-run (15+ minutes with no
PASS/FAIL output and no CPU growth in the top-level test binary) but
was not: each of its subtests does real `bd init --proxied-server`
work in a short-lived child process, and Go buffers a parallel
subtest group's output until the group's last member finishes. Letting
it run to completion (2357.72s) rather than guessing avoided reporting
bad data.
seconds_per_init_fallback is recomputed as the weighted average
seconds-per-init across all 178 measured tests (sum of durations / sum
of inits), for use only when a brand-new test has no entry yet.
Re-packing the 30-shard manifest with this corrected data changes the
shard-balance picture: TestProxiedServerDep2 and TestProxiedServerClose
are each large enough, alone, to dominate their own shard (~2358s and
~1105s locally) while every other shard clusters around 741-876s. The
"within 5s" claim this replaces was never earned by the underlying
data; .github/workflows/bazel.yml now documents the real, uneven
spread and flags that the two outlier shards' local numbers already
exceed the job's 25m step timeout on their own merits (unverified
against RBE's actual per-shard throughput, since this job has only
ever run remote).
Verified: go vet ./scripts/..., go test ./scripts/... (includes the
manifest/coverage policy tests), actionlint, make bazel-sync-check,
make ci-pr-policy all pass. Exact-once coverage re-confirmed: 178
manifest assignments for 178 discovered tests, zero duplicates, zero
gaps.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): apply re-review candidate fixes; revert 2 unready renames (R-B1)
Re-review found the prior round's 14 renames would turn the required
Bazel + legacy 15-shard lanes red once discovered:
- TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix/external-unix and
TestProxiedServerOutageReconnectAcceptanceMatrix/external-unix-socket
built a dolt.sock path under t.TempDir(), which nests Bazel's hermetic
tmp-root plus the (64-byte-truncated) test name deep enough to exceed
sockaddr_un's 108-byte sun_path limit. Both now use a new shortSocketPath
helper (a short os.MkdirTemp("", "bds") directory) instead.
- runProxiedDeadline raced a deadline-watcher goroutine against
exec.Cmd.Start()/Run()'s write to cmd.Process, caught by -race. Replaced
with cmd.Cancel, which exec.CommandContext only invokes after Start, so
the read is ordered after Start's write.
With those two fixes applied, TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix
passes outright. TestProxiedServerOutageReconnectAcceptanceMatrix's race is
fixed, but both its subtests still fail a real "exceeded 3s bound" assertion
against a stopped external upstream -- a genuine latency issue, not a test
bug -- so it is reverted back to its original name
(TestProxiedOutageReconnectAcceptanceMatrix) with a comment explaining why,
pending a bead against the underlying latency.
TestServerModeSharedServeGlobalReportsTheServedDatabase also reverted back
to TestSharedServerModeServeGlobalReportsTheServedDatabase: `bd serve
--global` refuses to auto-apply 4 pending schema migrations (v65->v69) to a
shared/proxied server database and exits 1 with a consent-required usage
message instead of starting, per the #5920 migration-consent guard. Also a
genuine product/test-fixture question, not fixed here; reverted with a
comment, pending a bead.
The other 12 of the original 14 renames are kept; all 12 verified passing
in both lanes (see commits that follow for the manifest update and full
verification).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied weights from parallel=4 CI-identical sweep (R-S1)
The prior round's duration data (committed in 6be9605d9f) was captured at
-test.parallel=1 (serial). That fixed a real double-counting bug, but
serial timings don't model this lane, which always runs -test.parallel=4
(--config=doltserver-proxied's own setting). Re-review measured near-zero
correlation (r=0.08) between the serial-derived predicted shard load and
actual parallel=4 wall time -- e.g. TestProxiedServerRecomputeBlocked was
weighted 812s serially but measured 33s actual at parallel=4.
Replaced with a full fresh capture: `bazel --nohome_rc test
--config=doltserver-proxied //cmd/bd:bd_proxied_test` (local execution
only, the exact config bazel.yml's proxied-server tier uses), 30/30 shards
passing, 0 failures, 0 races. Every one of the 176 discovered tests'
top-level <testsuite time> was captured directly from this run's
bazel-testlogs test.xml files -- no leaf-summation workaround was needed
this time, since none of the 176 hit Bazel's per-target default timeout
running alongside the rest of its shard.
Re-ran `gen_proxied_shard_manifest.py 30 --weights=duration --repack
--write` with these corrected weights: a deliberate full reweight, not an
incremental add, since every weight changed. The new packing clusters all
30 shards within a few seconds of ~355-360s each (176 tests summing to
10712s / 30), a dramatic improvement over the serial data's wrongly
implied two-outlier-shard spread (TestProxiedServerDep2 alone at ~2358s,
TestProxiedServerClose alone at ~1105s -- both artifacts of serial
timing, not real per-shard cost). Updated bazel.yml's paragraph to report
this honestly instead of the prior, false "exceeds 25-minute timeout"
claim, which was itself downstream of the serial/parallel mismatch.
Verified: the 12 tests kept renamed in the previous commit, plus
TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix's socket-path fix,
all pass under the new 30-shard Bazel manifest (this capture run) and
under the legacy 15-shard hash-fallback script (ran the 7 shards their
hash lands in: 2, 3, 4, 5, 8, 10, 11 -- all pass).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): incremental --write, --repack, name-coverage --check (R-S2)
The prior --check compared the committed 30-shard block byte-for-byte
against a fresh from-scratch LPT pack, and --write always did a full
repack. That meant any weight drift (even from proxied_test_durations.json
alone, with no test added/renamed/removed) failed --check, and two
unrelated PRs each adding one proxied test would both force a full
repack and conflict on unrelated shard lines -- re-review demonstrated
one new test moving 73 assignment lines under the old full-repack
behavior.
Redesigned:
- --check now verifies exact-once NAME coverage of the currently
discovered TestProxiedServer*/TestServerMode* set against the
committed block -- missing, stale, or duplicated names fail with an
actionable message -- but does NOT compare shard assignments, so a
weights-only change no longer forces --check to fail.
- --write is now incremental by default: every already-assigned test
keeps its current shard, stale (no longer discovered) names are
dropped, and only newly-discovered names are LPT-placed onto whichever
shard is currently lightest. A full from-scratch LPT pack now requires
the explicit --repack flag (used in the preceding commit for the
deliberate full reweight; ordinary new-test additions should use the
incremental default).
Mutation-tested directly (not just reviewed): verified --check correctly
flags a stale name as "stale/unknown" and a newly added test as "missing"
with the exact --write command to fix it; verified incremental --write
touches only the lines for the added/removed names; verified --repack
--write produces a materially different (full rebalance) diff from the
incremental default.
Updated TestProxiedShardManifestGeneratorNotStale's comment in
pr_risk_bazel_coverage_test.go to describe the new name-coverage-only
--check semantics (it previously described the old byte-identical-packing
behavior; the test's own assertion -- exit code only -- was already
correct and unaffected).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(lint): drop unparam findings in primeGitCmd and expandStep
primeGitCmd's error return was always nil (GH#4927 made every path
infallible but left the signature); drop it and update both call sites.
expandStep's depth parameter was always called with 0 from every caller;
recursion depth is tracked internally by expandStepWith via its own
depth+1 calls on children, so the parameter was dead. Hardcode the 0
start inside expandStep and drop the parameter.
Fixes golangci-lint unparam findings:
cmd/bd/prime.go:306:67: primeGitCmd - result 1 (error) is always nil
internal/formula/expand.go:197:49: expandStep - depth always receives 0
make ci-pr-lint is clean (native + windows/amd64 + darwin/arm64 non-CGO).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(dolt): give each TestBatchApplyContract subtest its own context
TestBatchApplyContract's subtests all shared one 90s context built once
in newDoltBatchApplyFixture. BoundsTheItemCount legitimately needs most
of that budget against the server-Dolt tier (ApplyBatchInTx's create
path calls ExecuteCreate once per item, ~5-8 round trips each, for all
1000 items at the MaxApplyBatchItems bound raised by #7051). Once
earlier subtests had spent part of the shared budget, BoundsTheItemCount
would blow the deadline and every subtest that ran after it in the same
parent would fail too with "context deadline exceeded", masking the
real failure behind unrelated red subtests (nightly run 37089445362,
shard 2/16: BoundsTheItemCount at 78.9s, deadline exceeded).
Give each t.Run its own fresh testContext(t) instead of sharing one,
mirroring openStoreWithOwnBudget's established rationale for the same
bug class (be-gvnsq). The store and its copy-on-write branch stay
shared across subtests, and subtests still run in sequence; only the
per-subtest deadline is decoupled. No production code changed: the
underlying per-item round-trip cost is genuine, documented, deliberate
behavior (see applyCreate's doc comment), and batching it safely would
be a larger, riskier change than this CI flake warrants.
Verified via bazel --nohome_rc test --config=doltserver-integration
//internal/storage/dolt:dolt_server_full_test --test_filter=TestBatchApplyContract
-t- --test_output=all: shard 2/16 (same shard as the original failure)
now passes, TestBatchApplyContract in 77.68s with BoundsTheItemCount at
66.08s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(embeddeddolt): skip redundant 1000-item wall-clock case under -race
TestLargeBatchApplyWallClock_Embedded's 1000-item shape was taking
597-651s under --config=embedded (-race), inflated by race-instrumenting
the in-process Dolt engine's own internal goroutine/lock machinery on
every statement issueops.ApplyBatchInTx issues. The Dolt server (TCP)
tier doesn't pay this cost: its sql-server runs in a separate,
non-raced subprocess (dolt_race_off), but the embedded backend runs
Dolt entirely inside this test binary, so -race instruments all of it.
This pushed the embedded-storage CI lane's shard 5 over its 19-minute
per-shard test timeout (panic: test timed out after 19m0s).
The 1000-item case carries no pass/fail assertion on duration or count
(the test is explicitly a logging-only measurement, not a regression
gate), so skipping it under race removes no coverage. Real 1000-item
(issueops.MaxApplyBatchItems) correctness coverage is unaffected: it
still runs and is asserted by TestBatchApplyContract/BoundsTheItemCount
in this same package, under the same race lane.
Adds the established raceEnabled build-tag pair (see
internal/workapi and cmd/bd for the sibling idiom) to this package so
the skip can be conditioned on -race.
Verified via Bazel (--config=embedded, --nohome_rc):
TestLargeBatchApplyWallClock_Embedded: 651.23s -> 86.79s
(1000 now SKIPped; 356 still runs and passes)
Shard 5 of embeddeddolt_embedded_test: FAIL -> PASS
Not fixed in this commit: TestBatchApplyContract/BoundsTheItemCount's
embedded instantiation still takes ~903-945s under -race and leaves
shard 4 of embeddeddolt_embedded_test over its 19-minute budget. Unlike
the wall-clock test, that subtest carries real correctness assertions
via the shared backend/conformance.RunBatchApplyBoundsTheItemCount, and
shrinking it would require a cross-backend conformance-test design
decision (e.g. a per-fixture item-count override) that trades away real
MaxApplyBatchItems boundary coverage for one backend - judged out of
scope for an isolated test-only fix here. Flagging for a deliberate
follow-up rather than weakening the assertion to make it fast.
TestLargeBatchApplyStatementCounts_Embedded and the dolt-tier
TestLargeBatchApplyStatementCounts_Dolt were checked and left
untouched: neither includes the 1000-item shape, neither showed any
slowness or flakiness, and both pin exact statement-count regression
baselines that should not be disturbed without cause.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cmd/bd): fence test-binary beads/git discovery against ancestor .beads
cmd/bd's test binary ambiently discovers its beads workspace the same way
production bd does: via internal/beads.FindBeadsDir's worktree-fallback and
ancestor-walk logic. When this package's checkout is itself a linked git
worktree (e.g. a sibling checkout like beads-bazel-y next to beads), the
first cached git-context call made anywhere in the test binary - before any
individual test has had a chance to chdir into its own isolated fixture -
permanently answers isWorktree=true/a real --git-common-dir for the rest of
the process. beads.FindBeadsDir then treats that as license to read (and
tests that don't isolate properly could write) the main checkout's shared
.beads database.
Confirmed via direct reproduction in a disposable scratch main+worktree pair
under /data/tmp (never /data/projects/beads): running a correctly-built `bd
--version` from the worktree checkout's cmd/bd directory resolves and warns
about the fake main checkout's .beads permissions, demonstrating the
fallback is reachable from plain ambient discovery, not just from a
contrived unit test.
internal/git.PinNoRepositoryForTesting seeds the process-wide git context
cache with a "not a git repository" sentinel as the first statement of
testMainInner, making the worktree-fallback vector inert by default. Tests
that need real git behavior for a fixture they control already call
git.ResetCaches/beads.ResetCaches after chdir'ing there (the existing
cmd/bd/git_test_helpers.go idiom), which overrides the pin for their own
scope - this is unchanged.
Separately, testMainInner now appends (never overwrites) a
BEADS_CEILING_DIRECTORIES entry for the package's own repo root, bounding
any literal ancestor-directory walk that starts at or under this checkout -
closing a second vector (an ancestor walk escaping past a nested worktree
root into a directory-with-.beads further up) that the pin alone doesn't
address. The existing pollution guard is extended to also watch the
worktree-fallback .beads directory (resolved via a direct, cache-independent
git shell-out) as a defense-in-depth detection backstop.
Verified via two full before/after `go test -tags "cgo integration" -v .`
runs of cmd/bd in a disposable linked-worktree scratch clone: identical
pass/fail sets before and after (no regressions), and zero filesystem
writes observed (inotify-monitored) in the scratch main checkout's .beads
in either run - the existing subprocess test helpers already discipline
themselves to set an explicit cmd.Dir/BEADS_DIR, so no currently-live test
was caught actively polluting the fallback directory in this repro. This
change is defense-in-depth against the mechanism itself (and against
regressions reintroducing it), rather than a fix for an empirically
isolated failing test in the current suite.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): make embedded-dolt and go:embed tests discoverable by their CI shards
embedded-test-shard.sh discovers cmd/bd's embedded-dolt tier by grepping
`^func TestEmbedded` out of cmd/bd/*_embedded_test.go — a filename-glob
match, not a package-wide one. 11 TestEmbedded* functions lived in files
that didn't match that glob (auto_import_upgrade_test.go,
dep_list_cross_db_test.go, and four files mixing TestEmbedded* with
unrelated TestProxiedServer*/TestServerMode*/TestWithFetchOneExtra_*
siblings: max_rows_test.go, store_factory_test.go,
purge_wisps_plane_integration_test.go, types_create_agreement_test.go).
They compiled and would run under a plain `go test`, but were invisible
to every CI lane that drives this tier off that discovery (the 20-shard
embedded cmd matrix), so they never ran in CI at all.
Fixed by relocating them into *_embedded_test.go files: a straight
`git mv` for the two files that were purely TestEmbedded* content, and a
function-level split for the four mixed files (moving just the
TestEmbedded* function itself; shared helpers/types stay put since
Go doesn't care which file in a package defines a symbol). The newly
discoverable tests aren't in embedded-cmd-test-shards.txt — consistent
with the ~55 pre-existing discovered-but-unlisted tests already in this
suite, they fall back to the manifest's documented hash(name)%shards
assignment. Verified via `BEADS_TEST_SHARD_LIST_ONLY=1` across all 20
shards: all 212 tests (201 + 11) are now listed exactly once, no shard
comes up empty. Also verified for real: built the cmd/bd test binary
and ran all 11 against a prebuilt bd binary (BEADS_TEST_BD_BINARY) from
the repo root, matching how CI invokes the prebuilt CMD_BINARY — all 11
pass.
Separately, 5 functions had a `TestEmbedded` prefix for an unrelated
reason — go:embed'd content (agents_test.go's AGENTS.md template tests,
schema_test.go's migration-source duplicate-version check) — and so were
being silently caught and skipped by the repo-wide `-skip '^TestEmbedded'`
pattern PR Core/main CI lanes use to exclude the Dolt-embedded tier from
./... runs. Renamed to a TestGoEmbed* prefix so they run where every
other unit test in their packages does; no other references to the old
names exist in the repo. Confirmed with `-skip '^TestEmbedded'` locally:
all 5 now run (previously would have been silently skipped).
Repo-wide grep for stray `^func TestEmbedded` outside cmd/bd,
internal/storage/embeddeddolt and internal/storage/dolt now returns only
internal/tracker's TestEmbeddedDoltExternalRefChangedAfterUsesHistoryFastPath,
which is genuinely about the Dolt-embedded backend and correctly named.
cmd/bd/BUILD.bazel regenerated via `bazel run //:gazelle` to reflect the
renames/new files; it also picked up internal/ceiling as a go_test dep,
a sync gap left over from test_repo_beads_guard_test.go's import in the
prior commit on this branch.
Verified: go build/vet (default, -tags cgo) clean; gofmt clean; go test
./scripts/...; bazel --nohome_rc test //scripts:scripts_test; bazel
--nohome_rc build //cmd/bd:bd_embedded_test; bazel --nohome_rc run
//:gazelle (cmd/bd/BUILD.bazel diff only); python3 tools/bazel/go_srcs.py
--check (make bazel-sync-check); make ci-pr-policy; make ci-pr-lint
(gofmt + golangci-lint native/windows/darwin, 0 issues). cmd/bd test
binary runs were built from cmd/bd (not the module root) and executed
with BEADS_TEST_BD_BINARY set to a binary built the same way, to avoid
buildEmbeddedBD's `go build -o X .` resolving "." against the wrong
package when invoked outside `go test`'s own working directory — not a
bug in this change, reproduced identically against an untouched,
already-correctly-named test (TestEmbeddedVersion). /data/projects/beads/.beads
checked before and after: no change beyond the pre-existing 166 untracked
.beads/formulas/*.toml files already present at session start.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(embeddeddolt): shrink BoundsTheItemCount's real apply under -race
TestBatchApplyContract/BoundsTheItemCount's embedded instantiation
measured 903-945s under -race (shard 4's single biggest long pole),
because the in-process cgo Dolt engine's own goroutine/lock machinery
runs inside the race-instrumented binary for a full 1000-item apply.
Every backend's ApplyBatch validates via the same storage.PlanApplyBatch
before opening any transaction, so the refusal half of this subtest is
already O(1) regardless of count, and the exact 1000/1001 boundary is
already pinned, for real and sub-millisecond, by
TestPlanApplyBatchAcceptsTheBoundary in internal/storage/batch_apply_test.go.
That leaves only "does this backend's own write body durably land N
accepted rows" as the expensive, backend-specific thing this subtest's
"at bound" half needs N real items for - and N does not need to be 1000
to prove that.
Split RunBatchApplyBoundsTheItemCount into a thin wrapper (unchanged,
still used by internal/storage/dolt and internal/storage/uow at the
full MaxApplyBatchItems) plus a new RunBatchApplyBoundsTheItemCountAtScale
taking an explicit atBoundItems. Embedded's subtest now applies 150 items
under -race (65.33s locally, down from 749.22s unmodified on the same
host) and the full 1000 otherwise, via the existing raceEnabled idiom.
internal/storage/dolt's non-race BoundsTheItemCount still covers the
shared inner write body at the full 1000-item count every PR. No CI lane
covers embedded's own post-commit wrapper at 1000 items anymore after
this change (the only non-race embeddeddolt build is compile-only and
never executed) - a known, reported, deliberate gap, documented in both
the updated large_batch_apply_measure_test.go comment and the task's
impl-report.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(git): make test-repository pin survive ResetCaches
PinNoRepositoryForTesting wrote its "no repository" sentinel into the
same gitCtx variable ResetCaches() unconditionally clears, so any test
fixture that calls ResetCaches() after the pin is set (runInDir's
cleanup, resetRepoCachesForTest, 25+ call sites) silently un-pins the
whole test binary for every later test. A reviewer's probe test proved
the bypass: GetRepoRoot() returned the real checkout path after a single
runInDir round-trip.
Replace it with PinNoRepositoryUnderForTesting(root), a separate,
directory-scoped pin checked live against os.Getwd() on every
getGitContext() call, before the gitCtxOnce/gitCtx one-shot cache is
touched at all. The pin survives ResetCaches() for any working directory
at-or-under the pinned root, while fixtures that chdir OUTSIDE that root
(the normal t.TempDir() pattern) still get real git detection.
Add TestPinNoRepositoryUnderForTestingSurvivesResetCaches, reproducing
the reviewer's exact repro and proving the fix: pins a real repo root,
confirms unpinned after chdir+ResetCaches outside the root, and confirms
STILL pinned after chdir+ResetCaches back inside the root (mimicking
runInDir's cleanup).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): widen guard fallback coverage, document -C/doctor gap
Two review follow-ups on the test-repo .beads guard:
1. Honest scope documentation: FindBeadsDirFrom/ResolveBeadsDirForRepo's
-C/doctor paths shell out to git directly with the ceiling env var
stripped, so PinNoRepositoryUnderForTesting (see previous commit) does
not fence them. Document this residual gap explicitly rather than
implying full coverage; covering it would need those call sites to
consult the pin too, which is out of scope here.
2. The guard's fallback-.beads watch list only watched metadata.json and
skipped watching entirely when the main .beads directory was absent —
exactly the layout where the worktree fallback is actually in use.
Restructure to check the main and fallback directories independently,
add config.yaml/routes.jsonl to the flat watch list, and add a
recursive snapshot of dolt/ (snapshotDoltTree) so a real pollution
write anywhere under the fallback's Dolt-backed state is caught.
diffSnapshots is generalized to also report newly-appeared keys, since
a recursive tree snapshot can legitimately gain entries between
before/after in ways the old fixed-name flat list never could.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): pin MaxRows embedded tests off their shared race shard
TestEmbeddedMaxRowsNonListPaths and TestEmbeddedMaxRowsList both hashed
to embedded cmd shard 16, which measured 746s wall under -race
(605s + 214s) against a 19-20 minute budget on a 192-core host with
effectively unbounded test.parallel; CI's 4-way parallelism would be
worse. Neither test had ever run under race in CI before this branch
surfaced them to their own shards.
Both tests spawn many independent bd subprocesses per subtest
(bdInit + exec.Command), so their cost is dominated by subtest/subprocess
count rather than a single large-N parameter; buildEmbeddedBD also never
passes -race to the spawned bd binary, so there is no raceEnabled-style
"shrink N" lever analogous to BoundsTheItemCount's that would apply here
without just deleting coverage. Pin them to shards 11 and 13 instead,
the two lightest shards by current manifest+hash-fallback test count
(verified via BEADS_TEST_SHARD_LIST_ONLY=1, which lists shard
composition without executing any tests).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(embeddeddolt): correct coverage comments, range-check, document build tags
- large_batch_apply_measure_test.go's doc comment above
TestLargeBatchApplyWallClock_Embedded had three factual errors: the
150-item sibling subtest does not always run at 150 "race or not" (it's
1000 without -race); the 356-item shape is not "non-race only" (it runs
under race too, as the same comment's next line said); and the
server-Dolt full suite does not run on "every PR" (full_embedded in
.github/scripts/ci-embedded-tier.sh is unconditional on merge_group/push
but conditional on PRs, per pr-risk.yml's gating). Rewrote the comment to
be self-contained (no out-of-repo citation) and accurate, and reworded
the related t.Skip message. Fixed the same "every PR" overstatement in
batch_apply_contract_test.go's BoundsTheItemCount race-branch comment.
- Added a range-check to
conformance.RunBatchApplyBoundsTheItemCountAtScale: atBoundItems must be
1..publicops.MaxApplyBatchItems, since that's the production bound this
contract pins.
- Documented why race_detector_on_test.go/race_detector_off_test.go use
`cgo && race`/`cgo && !race` instead of the plain `race`/`!race` pair
used by internal/workapi and cmd/bd: every consumer of raceEnabled here
is itself cgo-gated, so the matching cgo tag keeps this file out of
non-cgo builds too, where it would otherwise compile with no consumer
and trip an unused-lint finding.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(dolt): give BoundsTheItemCount 3x timeout budget
The server-Dolt tier's BoundsTheItemCount subtest measured 66s against
the package's shared 90s testTimeout — a thin margin on a loaded CI
runner. Give it a dedicated 3x (270s) context.WithTimeout instead of
raising the shared testTimeout constant, since that constant also scales
every other subtest's 5*testTimeout cleanup bound package-wide; a local
override keeps the blast radius to the one expensive subtest.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(ci): run embedded-Dolt batch-apply suite non-race in nightly
The nightly full-test job's main suite (go test -race ... ./...) never
sets BEADS_TEST_EMBEDDED_DOLT, so every skipUnlessEmbeddedDolt-gated test
in internal/storage/embeddeddolt — including TestBatchApplyContract and
the 1000-item case of TestLargeBatchApplyWallClock_Embedded — is skipped
on every CI lane, not just this one. Add a dedicated non-race step so a
real 1000-item apply through the embedded backend's own
version-commit-published-after-the-tx wrapper gets exercised somewhere:
non-race because the embedded engine's own goroutine/lock machinery runs
inside this test binary and the race detector dramatically inflates its
wall-clock (see race_detector_on_test.go), which would defeat the wall-
clock test's purpose and reintroduce the kind of shard-timeout pressure
the MaxRows shard-pin commit just worked around elsewhere.
Verified locally from a fresh /tmp clone (outside any .beads ancestor,
per this task's safety rules):
BEADS_TEST_EMBEDDED_DOLT=1 go test -tags gms_pure_go -run '^(TestBatchApplyContract|TestLargeBatchApplyWallClock_Embedded)$' ./internal/storage/embeddeddolt
PASS, 178.3s total (BoundsTheItemCount 81.06s, WallClock/1000 63.14s),
comfortably inside the job's 45-minute timeout.
Add TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite to
scripts/ci_workflow_test.go, pinning the step's exact run command, its
BEADS_TEST_EMBEDDED_DOLT=1 env, that it runs after the main suite step,
and that it stays non-race.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): move package-mcp/package-npm gates into bazel.yml (F3)
Package gates now build bd via Bazel (no --config=ci) when the rbe job
enables it and fall back to go build otherwise, so forks/Dependabot/rbe-off
runs are unaffected. pr.yml's CI Gate keeps PACKAGE_MCP/PACKAGE_NPM required
and never-skippable, now sourced from needs.bazel.outputs.*; the retired
detect-package-gates/package-mcp/package-npm pr.yml jobs and
DETECT_PACKAGE_GATES are deleted. MCP pytest pins -n 8 explicitly with
pytest-xdist locked via uv lock. rbe, bazel-coverage, and both required gate
jobs move to Blacksmith for same-repo PRs (gate names unchanged).
Also closes a gap in detect-package-gates.sh (missing bazel.yml in the
force-both-gates path list, since the job definitions moved there) and wires
integrations/beads-mcp's manifest/lock into scripts_test's Bazel data so the
new pytest-xdist pin check runs under `bazel test` too.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): review fix-up for F3 package gates (SF-1, SF-2, nits)
Review of 65f378d241 (~/beads-bazel-plan/f3/review.md, no blockers) found:
- SF-1: the package gates' `bazel build //cmd/bd:bd_for_tests` resolved to
a different output configuration (k8-fastbuild) than `bazel test
--config=ci`'s bd_for_tests (k8-fastbuild-ST-..., via test:ci ->
test:prcore's --@rules_go//go/config:race), so it got no action-cache
reuse from the test job. Add the matching race flag to both jobs' build
step; verified with cquery that both invocations now cquery to the
identical bazel-out/k8-fastbuild-ST-.../bin/cmd/bd/bd_for_tests/bd path,
and with a real build that it's an action-cache hit once bazel test
--config=ci has run. Update the pinned Run string in
pr_lanes_bazel_coverage_test.go and the impl report's cache-reuse claim.
- SF-2: drop stale comments/doc text saying the package gates download
bazel-ci-build-artifacts; they build their own bd now (bazel.yml,
pr.yml, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md).
Nits: pin both required gates' display names in TestSameRepoBlacksmithRunners
(nothing else reads job.Name, so a rename would otherwise go uncaught); pin
the package jobs' Python/uv/Node setup against main.yml's copies; add
timeout-minutes to both gate jobs and detect-ci-tier (same-repo PRs now run
on paid Blacksmith minutes, not a free runner's 360-minute default); shut
down Bazel and remove the RBE client key right after each package job
packages its Bazel-built bd (nothing downstream needs either); add
scripts/ci/package-bazel-bd.sh to detect-package-gates.sh's both-gates
trigger arm and pin it in TestDetectPackageGatesCoversBazelWorkflow.
Every fix mutation-tested (mutate -> test fails with a specific message ->
restore from a /tmp backup, diff-confirmed identical -> test passes again).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): retry Go module fetches and scale download timeouts in every mode
A fork run failed Bazel / test on one proxy.golang.org TLS handshake
timeout. gazelle's go_repository fetches modules with the go command
(fetch_repo), which never retries, and Bazel's downloader retry flags do
not reach it. setup-bazel's generated rc now sets
--repo_env=GOPROXY=https://proxy.golang.org|https://proxy.golang.org|direct:
with "|" the go command moves to the next entry on any error, so the proxy
is tried twice and then the module's origin. fetch_repo checks every
module against its go.sum hash whatever the source. Verified with Bazel
9.2.0: an unreachable first entry still fetches all 438 go_deps repos,
and without a fallback the fetch fails.
--http_timeout_scaling=2.0 doubles the timeouts of Bazel's own
downloader (the Go SDK, http_archive), which already retries.
Both flags are key neutral (repository fetching only) and apply in every
mode, because repository fetching always runs on the runner.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the integration lane for fork and Dependabot PRs in cache mode
bazel-integration was remote-only, so fork and Dependabot PRs (mode
cache) had no PR-time integration run at all: main.yml's integration jobs
run only after merge. With rbe-west's anonymous read-only cache, every
action trusted CI already ran is a hit, so the GitHub-hosted runner builds
and tests only what the PR changed.
Simulated in an ubuntu:24.04 container with 4 CPUs and only the generated
rc (no user rc). This setup's action keys match CI's: the ci lane
reproduced the fork run's 6550 processes with 4847 cache hits.
- Main HEAD: 143 s (4656 hits, 12 local, 113 of 115 tests cached).
- A cmd/bd change: 170-290 s.
- Cache closed (cold, all local): about 30 minutes.
- bazel.yml: the lane's if is now mode remote or cache, never local
(rbe=off). The step timeout goes from 45 to 75 minutes (job 85), so a
closed cache (the farm's kill switch) still leaves fork PRs green.
- bazel-gate.sh: mode cache no longer allows BAZEL_INTEGRATION to skip.
Mode local still does.
- New TestBazelCacheModeReachesTheRC runs, end to end, the real decision
step for 22 starts:
- fork, Dependabot and same-repo PRs, push, rbe=cache/off dispatches
and bazel-farm, each with the variable on and unset;
- same-repo runs with and without the secret;
- a fork PR that "has" the secrets, and one passing fork-farm.
For each start it evaluates every lane's if and setup-bazel env, runs
write-bazelrc.sh, and checks bazel-gate.sh's skips. A cache-mode rc
carries --config=fork-cache and the fetch hardening, and no executor,
TLS file, instance, upload, header, BES or masked secret.
- ci_workflow_test.go: the integration if, the run modes, and a
closed-cache floor for the step timeout.
- Comments in bazel.yml, pr.yml and bazel-farm.yml, and the topology
engdoc. The engdoc also covers:
- fork-cache's slow-blob behaviour: 15 s read deadlines trip the
breaker, then "Lost inputs" and one build restart, still green;
- why the endpoint's FindMissingBlobs denial never affects cache-only
clients.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: retire pr.yml's legacy PR Core, build, pure-Go, domain+uow and contract jobs where the gated Bazel lanes run them (D2 step 3)
On same-repo, non-Dependabot pull_requests while the committed
BAZEL_RETIRES_LEGACY_PR_LANES flag is "true" (bazel-coverage's new
pr_lanes output; the job stays identical in pr.yml and pr-risk.yml),
pr.yml's build-artifacts, pr-core-wrapper, check-cmd-bd-puregeo-tests,
test-domain-uow and contract-corpus stand down. ci-gate accepts those
skips only then, and BAZEL_PR_LANES_RETIRED requires the same run's
Bazel test, pure-Go/js-wasm and dolt-server lanes to have run remotely
and passed. Revert: commit the flag "false" in both workflows.
What the retired jobs did besides their tests keeps running on every PR:
- the package gates take bazel-ci-build-artifacts (bazel-test's bd, same
layout) where build-artifacts stands down;
- the Dolt server fingerprint moves to its own required job;
- the Go tests Bazel does not run or skips (equivalence allowlist) run
under go test in pr-preflight-platforms' Linux leg, each required to
pass (tools/bazel/run_allowlisted_go_tests.py).
In mode remote the three lanes' bazel test invocations add
--config=sole-run (no cached test results, no whole-invocation retry on
remote cache eviction), as steps 1 and 2 do for their lanes.
Policy tests: decision matrix and named cases extended to the third
flag, gate simulation for the pr.yml tier, artifact consumers, pinned
lane steps and rc lines, a repo-wide narrowing scan of BUILD rules and
.bzl files, and the allowlisted-test runner.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record the retired pr.yml PR Core, build, pure-Go, domain+uow and contract jobs in the check topology
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): only pr.yml's workflow env sets a BAZEL_RETIRES_* flag, and only the decision step reads one
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(pr): keep go test ./scripts/..., go test's vet and the Bazel-skipped tests on every PR (D2 step 3 review F1, F1b, F3, L2)
Review of D2 step 3: retiring PR Core removed the only required pre-merge
`go test` of the scripts packages, whose policy tests (the D2 guards
among them) check part or all of their rules under go test only, and
go test's implicit vet for every package (rules_go's go_test runs none).
New required job scripts-go-checks (SCRIPTS_GO_CHECKS), on every PR, in
PR Core's environment (dolt, git and dolt identity, test-env):
- scripts/ci/scripts-go-test.sh: `go test ./scripts/...` with PR Core's
flags;
- scripts/ci/go-test-vet.sh: `go vet` with cmd/go's defaultVetFlags over
./..., policy-tested equal to the running toolchain's list;
- scripts/ci/allowlisted-go-tests.sh, moved here from
pr-preflight-platforms.
TestBazelOnlySkipsAreAllowlisted also requires every test that runs part
of its checks under go test only (a Bazel-guarded return/continue, or a
go-test-only block that reports failures) to live under ./scripts, the
only packages go test still runs on every PR; and step 3's
TestRunAllowlistedGoTestsScript gets its allowlist entry.
Package gates on a covered PR whose Bazel call did not run remotely (the
farm switch off) now fail in their own step naming
BAZEL_PR_LANES_RETIRED instead of on a missing artifact (L2).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record scripts-go-checks, the skip-allowlist check and the re-run-safe uploads for D2 step 3
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): allowlist the scripts tests that skip under Bazel, and check new ones before merge
nightly.yml's skip-parity check (equivalence.py --go-test-json) went red
on 2026-10-02 (run 36955306415): TestBazelGatedLanesNeverRetryFlakyTests
and TestCheckShardCoverageScript pass under go test and deliberately
t.Skip under Bazel (their inputs are not in //scripts:scripts_test's
runfiles), but had no `skip` allowlist entry. Step 2's
TestShardScriptsListOnlyRealTests and TestShardScriptNarrowingCheck, now
on main, have the same shape and would have turned the next nightly red.
Add the four entries, and TestBazelOnlySkipsAreAllowlisted: it parses
every _test.go file of the module and requires each top-level test whose
t.Skip is guarded by TEST_SRCDIR or bazeltest.IsBazel() (directly or
through a helper) to have an entry, so a new one fails under go test
before the merge instead of in the next nightly. It skips under Bazel
itself and is allowlisted too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(test): keep cmd/bd's integration init out of the credential-command helper process
nightly.yml's Full Test Suite (go test -tags=integration ./...) failed
TestApplyInitGatewayCredentialAdoptsToken and ...FailsClosed on
2026-10-02 (run 36955306415): credentialcmd re-execs the test binary as
the credential helper, and cli_fast_test.go's integration-tagged init()
runs in that child before TestMain can dispatch it, tries `go build .`
from the helper's working directory and panics, so the helper exits 2.
Add credentialcmd.IsHelperProcess (Dispatch's two gates) and return from
that init early, as it already does for the worktree-remove helper.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): overwrite fixed-name artifacts so "Re-run failed jobs" does not 409
A job re-run is the same workflow run, and upload-artifact v4+ refuses a
second artifact of the same name there (409 Conflict). bazel-test uploads
the bd package even after its tests fail, so re-running a failed Bazel /
test lane, the documented recovery for an eviction or a flake, failed on
the upload even when every test passed. Set overwrite: true on all eight
uploads in bazel.yml (the bd package, BUILD sync patch, every lane's test
logs) and policy-test it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(test): keep skip-scan fixture functions out of line-based test discovery
The fixture source in bazel_skip_allowlist_test.go had top-level
`func TestXxx(t *testing.T)` lines, which equivalence.py (and the CI shard
scripts) count as tests of the scripts package; write them with a FUNC
placeholder and substitute at write time.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci: retire PR Risk's legacy proxied and server-Dolt tiers where the gated Bazel lanes run them (D2 step 2)
On same-repo, non-Dependabot pull_requests, PR Risk's test-proxied-cmd (15),
test-server-storage and test-server-storage-full (16) jobs stand down while
the committed BAZEL_RETIRES_LEGACY_DOLT_SERVER_TIERS flag is "true" (in both
pr.yml and pr-risk.yml, policy-tested equal); pr.yml's gated Bazel
proxied-server and server-Dolt storage lanes are then the tiers' only
pre-merge run. build-embedded, whose embedded-test-binaries artifact feeds
exactly the six retired legacy test jobs, stands down where both D2 steps
apply.
Generalizes step 1 instead of copying it:
- bazel-embedded-coverage becomes bazel-coverage (identical in both
workflows): one decision step, outputs embedded and dolt_server, each
flag && pull_request && !fork && actor != dependabot[bot]; gate id
BAZEL_COVERAGE.
- PR Risk's gate accepts the retired jobs' skips only when their output is
true, and BUILD_EMBEDDED's only when both are.
- pr.yml's gate adds BAZEL_DOLT_SERVER_RETIRED: red when dolt_server is true
and the mode is not remote or BAZEL_PROXIED / BAZEL_SERVER_STORAGE is not
success (shared helper with BAZEL_EMBEDDED_RETIRED).
- Lane hardening as for embedded: --nocache_test_results for
doltserver-proxied and doltserver-integration, the flaky-target query,
check_shard_coverage.py (proxied 15 shards; server full suite 16 shards
plus --whole conformance), pinned test steps, rc lines and target
args/env; the shard listings now run concurrently (the server script
forks per test).
- Policy tests generalized over the retired tiers: decision equivalence
with bazel.yml's mode across two runs, both gates simulated, flag
equality, artifact consumers, real shard scripts in list-only mode.
Revert: commit BAZEL_RETIRES_LEGACY_DOLT_SERVER_TIERS: "false" in both
workflows.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record the retired legacy proxied and server-Dolt tiers in the check topology
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): no whole-invocation retry on remote cache eviction in the retired tiers' lanes
Review F1 (D2 step 2): Bazel 9 re-runs a whole `bazel test` invocation
(--experimental_remote_cache_eviction_retries, default 5) when an input was
evicted from the remote cache, and that exit code outranks a test
failure. With --nocache_test_results every test runs again and only the
retry is reported, so a flaky failure in the embedded, proxied or
server-storage lane (each now its tier's only pre-merge run) could turn
green. Set the retries to 0 in test:embedded, test:doltserver-proxied and
test:doltserver-integration (not remote-exec, which must stay key-neutral),
pin the lines, forbid any other line or the setup-bazel action from
setting the count, and correct the topology doc, which claimed the retry
never re-runs a failed test.
Reproduced against a local HTTP cache emptied mid-build: with the default
the build hit "Lost inputs", retried and exited 0; with 0 it failed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): pin how the retired lanes' shard scripts run their tests
Review F2 (D2 step 2): the narrowing scan covered tools/bazel and the
setup-bazel action but not the four .github/scripts/*-test-shard.sh
scripts the sharded targets execute, so appending -test.short to an exec
line or exporting BEADS_TEST_SKIP there passed the policy tests (and
check_shard_coverage.py only rejects a shard that is entirely skipped).
Pin each script's prebuilt-binary and go test fallback commands exactly
and its RUN_REGEX construction, and forbid anything else in the scripts
from running tests or selecting, skipping or switching them. The scripts
themselves are unchanged (the legacy jobs run them too); a self-test
checks that six narrowing edits are rejected.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): simulate a non-true coverage decision per retired tier
Review F3 (D2 step 2): the "decision succeeded but its output is not
exactly 'true'" gate scenarios set both bazel-coverage outputs bad at
once, so the embedded ids reddened the gate whatever the dolt_server rule
said, and weakening PR Risk's gate to `!= "false"` for one tier passed.
Run them per tier, with the other output a valid "false" and only that
tier's job skipped, and isolate build-embedded's both-true rule the same
way. Failed or cancelled decisions keep the all-skipped scenarios.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): list Bazel tier coverage and the proxied/server-Dolt checks as not directly required
Review F4 (D2 step 2): the topology doc still named the renamed
`Bazel embedded coverage` job, and its list of check names not to require
directly lacked the proxied and server-Dolt test jobs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): skip the real shard-script listing test on bash < 4
The CI shard scripts use associative arrays and only run on Linux runners;
macOS's /bin/bash 3.2 can't execute them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): measure ApplyBatchInTx SQL statement counts (B0)
Adds a driver.Connector-level counting/latency-injecting wrapper
(internal/storage/sqlcount) plus shared plan-request fixtures
(internal/storage/batchfixtures: mol-normalized 356/712-item shapes and a
40-item classic shape) and backend-specific pinning tests + benchmarks
(internal/storage/embeddeddolt, internal/storage/dolt) that measure the
ACTUAL number of SQL statements issueops.ApplyBatchInTx issues for each
shape, on both the embedded and Dolt (TCP) backends.
Measured totals (Prepare+Exec+Query+StmtExec+StmtQuery inside one
transaction): 356 items ~7008-7009, 712 items 14014-14015, 40 items 846,
identical on both backends (expected: statement issuance is driven by Go
application logic, not backend-specific SQL planning). Pinned with a small
(+/-2) tolerance to absorb an observed, host-contention-driven +/-1 Query
drift (Exec is deterministic on every run). This is the regression baseline
slice B2 (a later, lighter fast path) must demonstrably lower.
batchfixtures.UncappedPlan bypasses issueops.MaxApplyBatchItems so this
measurement does not depend on slice B1's cap raise.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(httpapi,issueops): raise batchApply envelope to 1000 items with bounded large-apply handling
POST /v0/beads/issues:batchApply now accepts up to 1000 items (up from 100);
issueops.MaxApplyBatchItems moves accordingly and the request body cap rises
to 16 MiB. The whole plan still applies in exactly ONE transaction. A request
over 100 items runs under a separate, flat, operator-configured whole-run
budget (`bd serve --large-apply-ceiling`, 5 minutes by default) instead of
the ordinary 60s request deadline; a request at or under 100 items is
unaffected and keeps exactly the 60s deadline and 15s commit-retry ceiling it
always had.
Concurrency and shutdown are bounded, not just documented:
- Oversized requests serialize through a dedicated one-wide semaphore, and at
most one caller may wait for that semaphore at a time (a CAS-guarded single
waiter) — a burst of oversized requests is shed with 503 rather than
queuing and starving ordinary traffic behind the general semaphore.
- The run deadline is stamped only once the slot is actually acquired, not
at the time the request arrived.
- Graceful shutdown closes new admissions before computing its drain budget,
then waits out an in-flight large apply up to its own remaining deadline
(plus a few seconds' grace for the response to write); any large apply
still queued for the semaphore when shutdown begins is refused rather than
extending the drain further. A queued caller that disconnects gets a clean
503, never a misclassified context error. `bd serve`'s orchestrator stop
grace must be at least `--large-apply-ceiling` — see
engdocs/SERVE_RUNBOOK.md.
- ApplyBatch's commit-retry budget only extends past the ordinary 15s ceiling
on this explicit large-apply path (via a context marker), with headroom
reserved so the final retry attempt can't be killed mid-commit by the
context's own expiry.
A new `issues.batchApplyLarge` capability token on `GET /v0/beads/context`
lets a client learn whether a given server accepts the raised envelope.
Wall-clock measurement (embedded Dolt, whole-transaction build+commit) backs
the 5-minute ceiling default: 356 items ~12.6s (23.8x headroom), 1000 items
~57.5s (5.2x headroom).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi,serve): wire the large-apply retry budget; repair stale pins
Review fixes for the large batchApply envelope, on top of the test
repairs the PR needed against main.
The extended commit-retry budget was never switched on.
internal/storage/uow's retryTxBudget lets a large apply keep retrying a
commit-time conflict past the ordinary 15s ceiling only when its context
carries issueops.WithExtendedRetryBudget's marker, and the docs on both
sides say admitLargeApply sets it, but nothing did: the marker's only
callers were the uow tests, which mark their contexts by hand. A
1000-item apply that hit a conflict still gave up after 15s with most of
its 5-minute ceiling unused. admitLargeApply now marks the run context
it builds, and TestApplyBatchMarksOnlyALargeApplyForTheExtendedRetryBudget
pins the seam over HTTP: marked at 101 and 1000 items, unmarked at 100.
Removing the mark fails the 101 and 1000 cases; marking every request in
handleApplyBatch fails the 100 case.
Smaller fixes from the same review:
- `bd serve --large-apply-ceiling` repeated the 5m default as a literal
of its own, and since bd serve always passes a positive ceiling, that
literal, not httpapi's, was the default every deployment ran. Export
httpapi.DefaultLargeApplyCeiling, default the flag to it, and pin the
flag's 5m0s default in TestServeFlags.
- admitLargeApply's release cancels the run context before clearing
s.largeApplyDeadline, the order acquireLargeApply's doc states; it
cleared the deadline first.
- Drop the batchfixtures test's dead t.Skipf, which waited for the cap
to reach 1000, and correct a graph-counter comment that still cited
the 100-item cap and the TestServeFlags doc line that called the
no-argument server unchanged.
The PR raises issueops.MaxApplyBatchItems from 100 to 1000 and adds
`bd serve --large-apply-ceiling`, but two base-owned tests still pinned
the old surface and went red on every lane that runs them (Bazel / test,
Test (macos-latest)):
- TestServeFlags pins the exact `bd serve` flag list; add
large-apply-ceiling.
- TestPlanApplyBatchRefusesAnUnusableRequest/too_many_items built 101
items and expected "at most 100 items", a request the 1000 cap now
plans successfully. Size the over-cap request and its message from
issueops.MaxApplyBatchItems, as TestPlanApplyBatchAcceptsTheBoundary
already does for the other side of the cap; the cap's value stays
pinned by TestCapBatchApplyLargeTiesAllThreeLimits.
The PR's own TestLargeApplyBurstNeverStarvesSmallRequestsForSemTimeout
also went red in PR Core (-race, -p 4 on a 4-vCPU runner): all eight
small requests took 160-165ms against a 100ms bound. The stall it
guards against lasts semTimeout (5s), while a small request can still
queue briefly behind the refused large requests, each holding a general
sem slot while its body decodes. Bound it at a second, the "want well
under a second" its sibling TestLargeApplyQueueDoesNotStarveOrdinaryRequests
already uses. Pinned to four cores beside 24 busy loops, small requests
measured 183ms mean and 338ms max, and the 1s bound passed 80/80 runs;
with the one-waiter gate in acquireLargeApply disabled the test still
fails 3/3, on its 2s never-completed check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(uow): stop a doomed large-apply retry before ctx cuts it off
retryTxBudget sizes maxElapsed once at ApplyBatch entry, but says nothing
about how long an individual attempt takes. A slow first attempt (e.g.
157s of a large apply's commit) left little of the budget for a retry,
which then got killed mid-transaction by ctx's own deadline instead of
finishing and reporting its own outcome -- surfacing as
context.DeadlineExceeded (isUnavailable excludes it -> a generic 500)
instead of the exhausted-write-conflict result this loop already reports
correctly (-> 503 + Retry-After).
RunTxResultWithin now tracks the previous attempt's duration and, before
starting a new one, stops early with the last real serialization error
when time left on ctx is less than that duration plus a fixed headroom --
but only when publicops.HasExtendedRetryBudget(ctx) is set (the same
marker retryTxBudget itself checks). Applying the look-ahead
unconditionally regressed TestApplyBatchHonorsAnExtendedRetryBudgetFromContext's
unmarked case: ordinary requests have cheap attempts and are meant to run
until their own ctx deadline ends the loop, and the large-apply-sized 5s
headroom was tripping that for any short-ctx caller with a near-instant
failing attempt.
Adds TestRunTxResultWithin_SlowAttemptStopsBeforeADoomedRetry (a 1.2s
commit against a 2s marked ctx must stop after one attempt with its own
serialization error, never DeadlineExceeded, and never touch a second
unit of work) and its control, TestRunTxResultWithin_ShortAttemptStillRetries
(a 50ms attempt against the same shape must still retry and succeed).
Both mutation-checked against the look-ahead condition and its gate.
Also updates the stale "items accepts at most 100 entries" prose on the
applyBatch endpoint description (openapi.v0.yaml) to the large-envelope
wording already used elsewhere in the spec: 1000-item cap behind the
issues.batchApplyLarge capability, large-apply-ceiling deadline extension,
<=100 unaffected. dependencies:count's own "at most 100" lines are
unrelated and untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,serve): fit the 1000-item cap's tests in the storage shard budget
Review fixes for the relaunched #7051, iteration 1.
Raising issueops.MaxApplyBatchItems from 100 to 1000 made the base-owned
TestBatchApplyContract take 604s under -race, against 97s at the old cap
(main run 35862663577). It hashed onto the legacy test-embedded-storage
shard 2/5, beside ~5m of other tests, and that shard hit its 15m budget
twice (PR Risk run 36807191489). PR Risk no longer runs the legacy tier
on same-repo PRs (#7071), but Main's push run and fork PRs still do, so
the merge would have carried the timeout to Main.
Raising the jobs' timeout-minutes alone would not fix it: the shard
script passes the binary its own -test.timeout=15m. So:
- embedded-storage-test-shard.sh passes -test.timeout=20m (and its go
test fallback -timeout 20m), as embedded-test-shard.sh does for
cmd/bd.
- test-embedded-storage in main.yml and pr-risk.yml: timeout-minutes 15
-> 25, the script's 20m plus setup.
- embeddeddolt_embedded_test appends -test.timeout=19m, as
bd_embedded_test does, so Go's timeout, which dumps every goroutine,
fires before rbe-west's 1200s action kill, which does not.
TestBazelEmbeddedJobMirrorsEmbeddedTier now checks the override on
both sharded targets instead of only cmd/bd's, and the PR Risk
coverage test pins the storage target's new args.
- embedded-storage-test-shards.txt places TestBatchApplyContract on
shard 4, which has ~3m of other tests, and the PR's two large-batch
tests, TestLargeBatchApplyStatementCounts_Embedded (247s) and
TestLargeBatchApplyWallClock_Embedded (261s), on shards 2 and 5. From
the PR run's per-test times, the heaviest shard projects to ~13m, down
from shard 2's 15m-plus.
Also correct the stop-grace guidance. For an in-flight large apply,
Serve's drain waits out the apply's remaining deadline plus drainGrace
(5s), so the orchestrator's stop grace must be at least
--large-apply-ceiling plus 5s, as Serve's own doc says. The runbook, the
flag help, the LargeApplyCeiling field doc and the CHANGELOG entry asked
for the ceiling alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix: make release-version checks shell-free (#5533)
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
* fix(release): preserve checker status and explain gate failures (saa-cgh.16)
Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell
* docs: retain release-pinned CLI references (saa-cgh.16)
Preserve corrected live preflight help while generated references continue to match docs/cli-docs.pin v1.2.2. Fix the exact two-line hosted docs-policy failure without changing pin authority.
Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell
* fix(versioncheck): carry base's release gates into the Go checker
The base branch gained three release-metadata gates in scripts/check-versions.sh
after this PR's merge point (release: prep v1.3.0, b3ef65c854): the Copilot
plugin manifest, the four winres.json Windows PE version fields, and
manifest.xml's assemblyIdentity version. This PR replaces that shell body with
the Go authority, so adopting it as-written would have silently dropped all
three — each one guards a file whose bump is a sed anchored on the old value,
so a drifted copy no-ops and ships advertising the previous version.
Port them into internal/versioncheck so the shell-free checker enforces the
same set:
- source gains an optional expected() so a surface can derive its required
value; the Windows PE numeric fields must be purely numeric and so carry the
prerelease-stripped form, and manifest.xml carries the four-part form.
- readWinresVersionField reads the nested RT_VERSION resource path through the
existing duplicate-key-checked JSON helpers.
- readManifestAssemblyVersion anchors on line start, since the XML declaration
also carries a version attribute.
Also regenerate the Bazel BUILD files (make bazel-sync). cmd/bd/BUILD.bazel
enumerates srcs explicitly, so the PR's new preflight_versions.go was missing
from it and //cmd/bd:bd_lib failed to compile with `undefined:
runBeadsVersionSyncCheck`; gazelle adds that src, the //internal/versioncheck
dep edge, and BUILD files for the two new packages.
Known follow-up for review: the PR's TestRepositoryReleaseVersionsMatch and
TestPrePushHookOrdinaryPushWorksWithSystemBash read repository files directly,
so they need `data` runfiles to pass under `bazel test` (advisory lane only).
* fix(versioncheck): stop the release gate over-claiming docs coverage; distinguish --expect mismatch
Maintainer review fixup for iteration 1 (both findings minor, non-gating;
scorecard 915/850 approve with Required Changes: None).
Contract & Interface Fidelity (claude + codex): the success line claimed
"Version files and released-docs policy pass" while the checker reads no
documentation surface, and it replaced the shell predecessor's explicit
disclaimer that the docs release line and docs/cli-docs.pin are manual
release steps. Restore the base wording ("Version files in sync at: <v>")
plus the disclaimer, and update the three pins in main_test.go.
This also repairs a consumer neither review lane censused: the tracked
release runbook .beads/formulas/beads-release.formula.toml:475 instructs the
operator that the gate "must print `✓ Version files in sync at: <version>`
before you continue" — the renamed string had silently broken that
acceptance check. Grepping the new spelling finds nothing; the breakage only
shows up when grepping the old one.
Error Handling & Resilience (codex): --expect mismatch reused exit 1, so
.githooks/pre-push answered a tag/checkout mismatch with the version-drift
remedy ("Fix with: scripts/update-versions.sh <version>") — advice that
would rewrite every release surface to the wrong version. Give that case
status 3 and add the matching hook branch pointing at CONTRIBUTING's actual
rule. Only two --expect call sites exist: release.yml:45 (bare non-zero
check, unaffected) and this hook.
Verified at this head: check-versions.sh rc=0 (24 green rows, all three
ported base gates); exit taxonomy 0/2/3 as documented; go test
./internal/versioncheck/ ./scripts/check-versions/ PASS=65 FAIL=0 SKIP=0;
cmd/bd TestTrackedManagedHookSectionsMatchGenerator ok (hook edit sits above
the generator-managed markers); gofmt/vet clean. Both fixes proven
non-vacuous by mutating each back and requiring red.
Declined as out of cleanup scope, recorded for follow-up: making
TestRepositoryReleaseVersionsMatch fixture-based (redesigns the PR's own
test architecture and interacts with the deliberate untracked-discovery pin
at check_test.go:174; harm measured local-only) and declaring bazel data
runfiles (gate impact none — no BAZEL key in either CI_GATE_REQUIRED map at
this base; the remedy needs root filegroups that touch
scripts/bazel_policy_test.go invariants).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(versioncheck): declare the release-gate tests' repo inputs as Bazel data
Base commit e1b2e79484 (#6941) promoted the seven BAZEL* lanes into pr.yml's
CI_GATE_REQUIRED after this branch's merge point, so the two packages this PR
adds are now graded by a required check. Both reach real repository files
through a hand-rolled filepath.Join("..", ".."), which resolves outside the
runfiles tree, and `bazel test //...` failed on them:
TestRepositoryReleaseVersionsMatch
open ../../cmd/bd/version.go: no such file or directory
TestPrePushHookOrdinaryPushWorksWithSystemBash
runfiles .githooks/pre-push: No such file or directory
Resolve the root through internal/testutil/bazeltest.RepoRoot, the base's
helper for exactly this, which reduces to the previous behavior under plain
`go test`; and declare the inputs as data through a //:release_metadata_files
filegroup aggregating the surfaces versioncheck.releaseSources gates, the
canonical version it compares them against, the uv.lock pin and the tracked
hooks, exported from the packages that own them. The filegroup is hand-listed
because releaseSources is itself a closed list: a newly gated surface must be
added there too, and until it is the test fails under Bazel.
The shell entrypoint's exit-status subtests build the checker from source at
the repository root, which cannot work against a runfiles tree that has no
go.mod, so they skip under Bazel and keep running under `go test`. The guard
sits inside each subtest, not above the loop, so the ordinary-push assertion
still reports as run rather than hiding behind a skip.
Validation: `go test ./internal/versioncheck/ ./scripts/check-versions/` is
unchanged at 65 pass / 0 skip, with all three entrypoint subtests still
executing; both targets pass `bazel test --config=ci`; //scripts:scripts_test
(the tag-taxonomy and Bazel policy tests) passes; `bazel build //...` is clean
across 321 targets; `make bazel-sync` reports no drift and is idempotent.
* test(versioncheck): stop declaring the bazelignored npm manifest as Bazel data
The required `Bazel / test` lane failed on the refreshed head in fork-PR CI:
TestRepositoryReleaseVersionsMatch
npm package.json: open .../versioncheck_test.runfiles/_main/
npm-package/package.json: no such file or directory (expected 1.3.0)
npm-package/ is in .bazelignore (#6798), so Bazel never links it into the
execroot, and a local sandbox cannot stage npm-package/package.json even
though //:release_metadata_files named it and analysis accepted the label.
It resolved only under remote execution, where the client uploads the file
from the workspace. That is how the previous fixup's `bazel test
--config=ci` passed on a host whose bazelrc defaults to remote execution.
Fork PRs get no remote-execution secrets, so CI runs the test in
processwrapper-sandbox, where it fails.
Drop the entry from the filegroup. Under Bazel only, let
TestRepositoryReleaseVersionsMatch tolerate exactly that surface's absence:
the file must be missing, and every other gated surface, the uv.lock pin and
the hook markers must pass. The test logs the gap. This mirrors the base's
skipReleaseFormulaUnderBazel for the bazelignored .beads/. `go test`, which
sees the real checkout, still checks the npm manifest.
Validation: forcing local execution the way fork CI runs (`bazel test
--config=ci --nocache_test_results --remote_executor=
--strategy=TestRunner=processwrapper-sandbox`) reproduced the failure before
the change. After it, //internal/versioncheck:versioncheck_test,
//scripts/check-versions:check-versions_test and //scripts:scripts_test pass
(3 of 3 executed), and both release-gate targets also pass under remote
execution. Drifting a declared surface still fails the sandboxed run. The
tolerance was probed through TEST_SRCDIR and BEADS_TEST_REPO_ROOT: each of
these still fails:
- another drifted surface
- a present but mismatched npm manifest
- an unreadable canonical version
- plain `go test` with the manifest missing
`go test ./internal/versioncheck/ ./scripts/check-versions/`, golangci-lint
and `make bazel-sync-check` are clean.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* ci(bazel): fork PRs use rbe-west's anonymous read-only cache (mode cache)
Fork PRs (unless bazel-farm.yml authorized them), rbe=cache dispatches and
secret-less same-repo runs (Dependabot) now run in mode "cache": local
execution on ubuntu-latest as before, plus .bazelrc's new fork-cache config,
rbe-west's credential-free rbe-cache endpoint (instance oss, reads only).
rbe=off keeps the old local mode without the cache.
- .bazelrc: build:fork-cache. Only key-neutral flags (--jobs, --remote_*), so
fork actions hash like trusted ones. --noremote_upload_local_results, and
--remote_local_fallback plus
--incompatible_remote_local_fallback_for_remote_cache: without both, a
closed endpoint (farm kill switch) fails every action in GetCapabilities.
- setup-bazel: BAZEL_FORK_CACHE=true appends --config=fork-cache; it is
mutually exclusive with the RBE secrets; new output cache.
- bazel.yml: RBE_INPUT_CACHE, mode cache, dispatch option cache; every
setup-bazel step gets BAZEL_FORK_CACHE. Remote and fork-farm paths
unchanged.
- bazel-gate.sh: cache/false is valid and skips what local skips.
- Policy tests: the two public endpoint lines are allowlisted in .bazelrc
only; TestBazelForkCacheConfig pins the fork-cache contents and checks both
fork-cache and remote-exec stay key neutral; decision, gate and rc-writer
tests cover cache mode.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): fork cache gives up on a slow endpoint and uploads no BEP files
Review fixes for the fork read-only cache (mode cache).
- .bazelrc fork-cache: --remote_timeout 60 -> 15 and
--experimental_circuit_breaker_strategy=failure, so a slow (not closed)
endpoint stops being called once too many lookups fail instead of
costing every lookup 60s x retries; --experimental_build_event_upload_strategy=local,
because bazel.yml's --build_event_json_file and --profile otherwise make
Bazel upload test logs and the profile over ByteStream. equivalence.py
and check_testcases.py read test.xml from the local testlogs dir, not
from BEP URIs, so they are unaffected.
- Policy test: parseBazelrcOptions now scans plain command lines too
(lane checks still ignore them), so `build|common --config=fork-cache`
is caught. Outside fork-cache and remote-exec, no line (test: lines
included) may turn uploads on, turn either fallback off, or set a
non-local BEP upload strategy. fork-cache must end with the breaker,
the local BEP strategy and a --remote_timeout of 1-15s; keyNeutralFlag
accepts the breaker and BEP-strategy flags.
- engdoc: the closed-endpoint fallback sentence now also describes the
slow-endpoint path (timeout, breaker). bazel-farm.yml and pr.yml
comments say fork PRs run in mode cache, not plain local.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): reconcile fork cache (mode cache) with the legacy embedded retirement
After the rebase onto D2 step 1 (#7071), two policy tests failed against
mode cache.
- evalRBEExpr learns `inputs.rbe == 'cache'` (bazel.yml's RBE_INPUT_CACHE).
The decision itself is unchanged and right for mode cache: covered never
depends on the mode, forks and Dependabot (now mode cache) keep the legacy
tier, and a covered same-repo PR without the secret (now mode cache, not
local) still turns pr.yml's gate red. The named cases now pin the mode
too, and the "mode alone makes it red" check includes cache.
- TestBazelEmbeddedLaneCannotBeNarrowed treats fork-cache like remote-exec:
a config setup-bazel's rc enables for every command. Its lines (and any
config they reference) are now in use for the lane, so the rc narrowing
regex applies to them, including a forbidden nested --config=. The
setup-bazel scan accepts exactly --config=remote-exec and
--config=fork-cache.
- Comments in bazel.yml, pr.yml, pr-risk.yml and the engdoc that said
missing-secret, fork or Dependabot runs take mode local now say cache.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(pr-risk): stand down legacy embedded tier where the gated Bazel lane runs it
D2 step 1. A new bazel-embedded-coverage job decides, from the same
expressions as bazel.yml's rbe job, whether pr.yml's Bazel call takes
execution mode remote on a same-repo pull_request (RBE_WEST_WORKERS on,
executor secret present, not a fork). Only then do test-embedded-storage,
test-embedded-conformance and test-embedded-cmd (27 jobs) skip, and PR
Risk's gate accepts those skips only then; a failed or missing decision is
red. Forks, Dependabot, secret-less runs, the farm switch unset and
merge_group keep the legacy tier. build-embedded keeps running: its
artifact feeds the proxied and server Dolt jobs.
The decision job reads no repository code and the secret only as an
emptiness test. scripts/pr_risk_bazel_coverage_test.go runs both actual
decision scripts over every event/var/secret/fork combination and requires
covered <=> (pull_request && Bazel mode remote), pins the shared env
expressions, checks pr.yml still requires BAZEL_EMBEDDED and shares PR
Risk's triggers, and simulates PR Risk's gate over every tier/decision.
Revert: unset RBE_WEST_WORKERS.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record the retired legacy embedded tier in the check topology
Who keeps the legacy tier, how the decision and gate work, the drift
guard, the residual cross-run race, the conformance.yml duplicate, and
how to revert (unset RBE_WEST_WORKERS).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(pr-risk): retire the embedded tier by committed flag; pr.yml gate fails closed
Review F1. PR Risk's coverage decision read the mutable RBE_WEST_WORKERS
variable in a different run from pr.yml's rbe job, so unsetting it and
re-running one workflow could leave both required checks green with
neither the legacy tier nor the Bazel lane having run the tier.
- Both workflows now commit BAZEL_RETIRES_LEGACY_EMBEDDED (policy-tested
equal) and run the identical bazel-embedded-coverage job:
covered = flag && pull_request && !fork && executor secret present.
RBE_WEST_WORKERS is no longer read by PR Risk at all.
- pr.yml's ci-gate requires the job's result (BAZEL_EMBEDDED_COVERAGE)
and BAZEL_EMBEDDED_RETIRED, which is red when covered but the Bazel
embedded lane did not run remotely and pass (mode skip included).
- Revert is a commit of the flag to "false" in both workflows; in-flight
PRs pick it up on their next push. The kill-switch runbook now flips
the flag first.
Tests run both decision steps, bazel.yml's rbe step and pr.yml's real
gate step over every event/var/secret/fork/flag combination, with the
variable differing between the two runs, and require no double-green
without the tier.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): never cache embedded-tier test results
Review F2. Since D2 step 1 the Bazel embedded lane is the tier's only
pre-merge run on same-repo PRs, so a cached PASS (stale, or a poisoned
entry in the shared farm action cache) or a cached run that never
re-exercises an intermittent -race finding would stand in for the legacy
jobs' -test.count=1 run. test:embedded now sets --nocache_test_results,
and the policy test requires it and forbids any other rc line that turns
test result caching back on.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): forbid flaky-test retries in the gated Bazel lanes
Review F3. Adding --flaky_test_attempts to a lane's config or
flaky = True to a lane target passed every policy test, and would hide
a flaky failure the legacy jobs (which ran each test once) would have
reported; since D2 step 1 nothing else runs the embedded tier on
same-repo PRs. TestBazelGatedLanesNeverRetryFlakyTests rejects
flaky_test_attempts and runs_per_test_detects_flakes in .bazelrc,
.github (workflows, setup-bazel's generated rc) and tools/bazel, and
flaky = True in every BUILD file and .bzl macro.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): pin the Bazel embedded lane's command line and config
Review F4. An extra --test_arg=-test.short/-test.skip on bazel.yml's
`bazel test //... --config=embedded` line, or a test:embedded
--test_filter in .bazelrc, narrowed the tier while every policy test
passed (check_testcases only rejects a shard with zero tests). Since D2
step 1 this lane is the tier's only pre-merge run on same-repo PRs, so:
- bazel-embedded's test step must be exactly the pinned script;
- the --config=embedded lines must be exactly the pinned list, in order;
- no unconfigured common/build/test line, and nothing setup-bazel's
generated rc writes, may select or narrow tests (test_filter,
test_arg, -test.short/run/skip, tag/size/lang/timeout filters).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): add the D2 policy tests to scripts_test
scripts/pr_risk_bazel_coverage_test.go was missing from
//scripts:scripts_test's srcs, so `bazel test //scripts:scripts_test`
(and the BUILD-sync check) did not see it. The repository-walking
flaky-retry test skips under Bazel, whose runfiles hold no other
package's BUILD files.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): check every listed embedded test ran in its Bazel shard
Review F5. The shard scripts discover tests from source with grep, but
each Bazel test binary holds only its target's srcs, so a discovered test
the binary lacks matches nothing in the shard's -test.run selector and
passes silently (check_testcases.py only rejects a shard with no tests).
tools/bazel/check_shard_coverage.py runs each shard script in list-only
mode (BEADS_TEST_SHARD_LIST_ONLY=1) and requires every Bazel shard's
test.xml to hold exactly the listed top-level tests: missing, extra,
wrong-shard, shard-count mismatch, missing test.xml and a failing script
are errors. bazel-embedded runs it after the tier for
//cmd/bd:bd_embedded_test (20) and embeddeddolt_embedded_test (5); the
policy test pins the suites to pr-risk.yml's jobs and shard counts, and
a fixture test exercises the script.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): correct the D2 enforcement, re-run and merge-queue claims
Review F6.
- The beads-only ruleset enforces both gates on main only; release/**
PRs run them but merging does not wait for them, before and after D2.
- There is no merge queue and strict is off: merge_group runs are not a
pre-merge safety net for the retired tier.
- Spell out "Re-run all/failed jobs" semantics under the committed flag,
the remaining secret-deletion window, and that the revert needs a new
push to in-flight PRs.
- Record the lane hardening the retirement relies on (no result caching,
no retries, pinned selection, per-shard coverage check).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): drop TestMain from the embedded shard-coverage check
Re-review G1 (blocking). embedded-storage-test-shard.sh discovers tests
with `grep '^func Test'`, which also lists TestMain(m *testing.M)
(internal/storage/embeddeddolt/test_fixture_test.go). It is never a
<testcase>, so check_shard_coverage.py reported it missing on every real
remote run, turning bazel-embedded and CI Gate red on every same-repo PR.
The checker now drops NOT_TESTS = {TestMain}. The shard scripts are left
unchanged so the legacy jobs' -test.run selection stays byte-for-byte the
same (dropping TestMain there would be harmless to shard assignment,
which hashes per name, but would touch the legacy CI scripts this step
does not otherwise change).
TestEmbeddedShardScriptsListOnlyRealTests runs both real shard scripts
in list-only mode for every shard and requires each listed name to be a
`func Name(t *testing.T)` in the package, or in NOT_TESTS (which may hold
only TestMain, and only if it is not a test). The fixture shard script
now lists TestMain too, so the filter itself is exercised.
Verified on a real remote run (`bazel test //... --config=embedded
--config=remote-exec`, 4/4 targets, 27 test.xml, 582 top-level tests,
348 s uncached): check_testcases.py and check_shard_coverage.py both
pass (199 cmd + 381 storage tests); the previous checker failed on
TestMain.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(pr-risk): decide embedded coverage by actor, not by the executor secret
Re-review G2. covered still read secrets.RBE_WEST_EXECUTOR, which an admin
can delete between PR Risk's run and a re-run of pr.yml: PR Risk stayed
green with the legacy tier skipped, while the re-run computed covered =
false, took mode local and passed with no embedded lane.
Both workflows' bazel-embedded-coverage now decide
covered = flag && pull_request && !fork && github.actor != dependabot[bot]
and read no secret or variable at all (PR Risk now reads no secret).
Dependabot (no Actions secrets) keeps the legacy tier; github.actor,
unlike triggering_actor, survives a re-run by someone else. A same-repo
PR without the secret is covered, so pr.yml's gate goes red (mode local)
instead of passing.
The two-run test matrix now varies the secret as well as RBE_WEST_WORKERS
between PR Risk's and pr.yml's runs, adds the actor to the facts, and
names the missing-secret case (red). Docs updated: the residual is gone,
the cost of failing closed is recorded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): close the remaining ways to narrow the Bazel embedded lane
Re-review G3. Skip-inducing edits outside the pinned --config=embedded
lines and command line still passed every policy test (reviewer
survivors m1, m2, m5, m6, m9, m10, m11), and the runtime checks counted
skipped testcases as run.
TestBazelEmbeddedLaneCannotBeNarrowed now rejects:
- any .bazelrc line of a config the lane uses (unconfigured, embedded,
remote-exec from setup-bazel's rc, and any config they reference) that
filters, narrows, re-runs or re-routes tests (test_filter, test_arg,
-test.*, *_filters, test_env=BEADS_TEST*/GO_TEST*/TESTBRIDGE*,
--config=, cache_test_results, run_under other than test_env, flaky,
runs_per_test, sharding strategy);
- any import other than the two gitignored try-imports, and any
committed rc file besides .bazelrc (try-import would load it in CI);
- -test.short/run/skip/list, BEADS_TEST_SKIP, BEADS_TEST_EMBEDDED_DOLT,
TESTBRIDGE_TEST_ONLY and similar in tools/bazel/*.sh (test_env.sh,
go_test_manifest_shard.sh, ...) and anywhere in the setup-bazel action,
and any --config= there but remote-exec;
- any change to the set of embedded-tagged targets or their args/env.
At run time check_shard_coverage.py fails a shard whose top-level tests
were all skipped, and its new --whole option applies the same check to
the unsharded conformance core/audit targets (added to bazel-embedded's
step). The real remote run still passes (no shard is all-skipped today;
one cmd shard has 1 of 9 tests skipped).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): ask Bazel for flaky test targets before the embedded tier
Re-review G4. The F3 grep wanted a literal `flaky = True`, so
`flaky = _RETRY` (or a macro default) slipped through (reviewer m3).
- bazel-embedded runs `bazel query 'attr(flaky, 1, tests(//...))'` after
setup-bazel and before the tier, and fails on any result (loading only;
about 0.2 s warm). Verified locally: with `_RETRY = True` /
`flaky = _RETRY` on embeddeddolt_embedded_test the query lists it.
- The policy grep now rejects any `flaky =` whose value is not a literal
False or 0, and TestBazelEmbeddedQueriesFlakyTargets pins the step.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): cover the zero-testcase shard in check_shard_coverage
Re-review G5. Skipping the "listed but did not run" report when a shard's
test.xml has no top-level tests survived the fixture test (reviewer m7);
check_testcases.py catches that shard at run time, but the coverage
check should not depend on it. Two fixture cases: a shard with no
testcases, and one with only a subtest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record the G1-G5 embedded-lane hardening in the topology doc
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): red — pin TESTCONTAINERS_RYUK_DISABLED on Dolt testcontainer steps (refs be-2on)
* feat: green — pin TESTCONTAINERS_RYUK_DISABLED on Dolt testcontainer steps (refs be-2on)
Sets TESTCONTAINERS_RYUK_DISABLED=true on the 5 CI step-instances that start
a Dolt testcontainer from a package TestMain: pr.yml and main.yml's
test-domain-uow job ("Test domain + uow + tracker" and "Test doctor/fix"
steps), plus main.yml's test job "Test (with coverage + JUnit XML)" step,
whose unfiltered ./... sweep runs internal/storage/dolt, internal/molecules,
internal/tracker, and cmd/bd as concurrent Dolt-container-starting binaries.
testcontainers-go shares one Ryuk reaper per host; when a step runs multiple
such packages as concurrent test binaries, they race to attach to that
shared reaper, and a failed handshake from one process reaps a sibling's
live container mid-suite. GitHub Actions runners are destroyed after the
job, so the reaper buys nothing there and only costs this race.
* docs(release-gates): record be-9b8 deploy gate PASS
* chore: remove leaked internal release-gate doc from PR branch
release-gates/be-9b8-dolt-ryuk-reaper-gate.md
Tracked our internal gc-management/beads-rig release-gate checklist and
should never have been committed to this upstream branch -- it's process
residue from our deploy tooling, not part of the change itself.
Per maintainer bee-ghosttrack's request during review of PR #6571. The
root cause (our deploy tooling auto-committing this file) is already fixed
as of gm-qatxl1; this branch predates that fix, so this commit is cleanup
only -- no other content on this branch is touched.
Ref: be-lk7o3
* fix(ci): drop the inert main.yml test-job Ryuk pin; narrow the guard's universal
Maintainer review fixup for PR #5770 (adopt-pr review iterations 1 and 2).
Iteration 1, Finding 1 (major, correctness) — the main.yml `test` job's
`./...` sweep cannot start a Dolt testcontainer, so the pin there was inert
and its justification comment was false. `checkDolt` (internal/testutil/
testdoltserver.go) reaches `doltReady` only via `isDoltImageCached`
(`docker image inspect`, no auto-pull), and the image's only CI source is
scripts/ci/pull-dolt-image.sh, which in main.yml runs in `test-domain-uow`
and `test-proxied-cmd` — not in `test` (no services: block either). The
base already documents this in pr.yml's contract-corpus header ("every
broad ./... job skips them for want of a Dolt store"). Took the synthesis's
option (a): removed the env pin + its 7-line comment and dropped the fifth
guard row, rather than keeping a dead pin. This also moots the Finding 3
nit, whose four-package list was a sample presented as a census.
Iteration 1, Finding 2 part 1 (major, maintainability) — the guard's doc
comment claimed it pins "every CI step that starts a real Dolt
testcontainer" while the implementation is an allow-list of literal
triples. Narrowed the claim to the enumerated steps, stated the scope as
the `test-domain-uow` job (noting the single-package doctor/fix step is
pinned for in-job consistency, not because it has an in-step sibling), and
recorded the container-capable jobs left unpinned: pr.yml/contract-corpus,
main.yml/test-proxied-cmd, pr-risk.yml/test-proxied-cmd,
pr-risk.yml/test-server-storage(-full), regression.yml/regression, and
bazel.yml's --config=docker lane — the last needing --test_env since bazel
does not forward ambient environment. Part 2 (class detection) is left as
follow-up.
Iteration 2 (two minors, both taken):
- The doctor/fix step's comment in main.yml and pr.yml read "Same
shared-Ryuk-reaper race as the step above", contradicting the guard's
own scope note: that step runs a single package, so no sibling test
binary races it for the reaper. Both copies now say the pin is kept for
consistency with the step above.
- The guard's census named bazel.yml/bazel-docker, a job current main
replaced (#6898) with bazel-doltserver, which defaults to
--config=doltserver and pre-pulls the Dolt image only on a
dolt-lane=docker dispatch. The census now names the lane by its
--config=docker selector, which is accurate on this base (the
bazel-docker job) and on current main (the dispatch-only A/B control),
and the --test_env remark is scoped to that docker lane.
Validation: git diff --check clean; ./scripts/ package green;
TestDoltTestcontainerStepsDisableRyuk runs 4 subtests, matching the 4 pins
exactly (main.yml x2, pr.yml x2); flipping main.yml's doctor/fix pin to
"false" reds only that subtest with a step-specific message, and the file
restores byte-identical. `git merge-tree --write-tree origin/main HEAD` is
clean and ./scripts/ is green on the merged tree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ci): require direct workflow build tags
Parse workflow run scalars structurally and require recognizable direct Go commands to declare the literal gms_pure_go tag. Keep shell and Make checks lightweight and preserve their existing conventions.
Fixes#5527
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>
* fix(ci): recognize double-dash Go build tag flags
Address PR #5679 review 5191180453. Preserve last-tag precedence and test argument boundaries, including --args.
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell
* fix(ci): bazel-sync the new checker, restore its opt-out and the Bazel census
Maintainer review fixes for PR #5679, adoption iterations 1 and 2. Iteration
1's synthesis is `request_changes` on one major plus three minors; all four
are addressed. Iteration 2 approves with one minor, also addressed.
Major (Release Safety): the PR adds two production Go packages with no
BUILD.bazel, so `bazel build //...` could not see them and the repo's own
"BUILD files in sync" step fails on the resulting gazelle drift. Added the two
generated files via `make bazel-sync`; yaml.v3 is already a direct go.mod
requirement, so no module change is owed.
Minor (Release Safety): the new checker runs in the required pr-policy lane with
no way out, while the shell checker it partly replaces honors
`# build-tags: allow-bare` in a file's first five lines (check-build-tags.sh, and
engdocs/ICU-POLICY.md documents that window). CheckDir now honors the same
marker in the same window -- one convention, not two -- pinned by a table that
includes the fifth line and the sixth, so the two checkers cannot drift apart on
where the window ends.
Minor (Change Impact): dropping the workflow glob from `candidates` moved
workflows to the Go checker but also, silently, out of the `bazel_users` census --
and the Go checker does no Bazel handling at all, so .bazelrc could have lost its
tag with this gate still green. Workflows now feed the census through their own
`git ls-files` list and a separate loop, deliberately not rejoining the `go`-verb
scan. This is not hypothetical: `.github/workflows/bazel.yml` matches the Bazel
invocation regex, so it is exactly the file the census was losing.
Minor (Test Evidence): `Line` is the operator-facing half of a violation -- it is
what the error message points at -- and nothing asserted it, so reporting every
violation at line 1 would have been invisible. Added a `wantLine` column, asserted
on the multi-line "source is not workflow authority" case (line 2), and asserted
`Path`/`Job` where CheckDir fills them in.
One deviation, recorded: the finding also asked for `Line` on the existing
"continuation" case. That case is tagged, so it reports no violation and has no
Line to assert. Added a continuing-but-violating case instead, which makes the
same attribution observable -- a logical line spanning three physical ones is
reported at the first.
Not included: `make bazel-sync` also wants `//internal/debug` in
internal/utils/BUILD.bazel. That is base drift, not PR content -- absent at this
branch's rebase base but already present on origin/main, 24 commits ahead -- so it
heals on the next base refresh, and carrying it would duplicate main's own fix on
a file this PR never touches.
Minor (Test Evidence, iteration 2): the census half of the Change Impact fix
was right in code but pinned by nothing -- turning the workflow `bazel_users`
append into a no-op, or feeding the census-only list into the `go`-verb loop,
left every check_build_tags_test.go case green. Two cases on the existing
fixture harness now pin both directions: a workflow that runs `bazel build`
with an untagged .bazelrc fails and is named as the Bazel user, and a workflow
`run: go test ./...` passes here because scripts/checkworkflowtags owns it.
Each fails under its mutant, as does one more apiece (re-adding the workflow
glob to `candidates`; dropping `*.yml` from the census list), while the three
existing cases stay green under all four.
Validation: ./scripts/workflowtags passes (all four CheckDir tests and the full
CheckRun table), `go vet ./scripts/...` clean, `bash -n` clean, and both checkers
run green against the repository itself -- check-build-tags.sh reports 90 files
scanned all clear, and checkworkflowtags reports all direct commands clear.
For iteration 2, `go test ./scripts/` passes, and on a merge with current main
`bazel test //scripts:scripts_test` passes too; on this branch's own base that
target cannot build, because of the internal/utils drift noted above.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(ci): pin main.yml's bd-cmd-test build to the literal gms_pure_go tag
#6940 pinned main.yml build-artifacts' non-race bd-cmd-test build as
`go test -tags "$BEADS_BUILD_TAGS" -c -o artifacts/bd-cmd-test ./cmd/bd`.
This PR requires a literal gms_pure_go tag on direct workflow Go commands
and rewrites that line to `go test -tags gms_pure_go -c ...`, so on
current main TestBazelDoltServerTiersMirrorPRRisk fails in both
`go test ./scripts` and //scripts:scripts_test. Restoring the pinned
spelling is not an option: checkworkflowtags rejects it ("direct `go test`
must declare literal gms_pure_go in -tags").
Update the pin to the literal spelling. It still pins what #6940 cares
about, a non-race bd-cmd-test built from ./cmd/bd: adding -race to that
line still fails the test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* test(ci): make doc-freshness selection fail closed
Require the protected doc-freshness workflow to select its exact compiled top-level inventory plus the guard that exercises real zero- and partial-selection refusals. Ordinary focused scripts tests remain unchanged unless required-suite mode is explicit.
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
* test(ci): fail suite self-test on inventory drift (#5726)
Fail the contract's compiled-suite subtest when an unregistered or missing
member would otherwise silently suppress its TestMain selector falsifiers.
Keep the integration-free build usable with an explicit subtest skip and
point doc-freshness test authors to the maintained registry.
The untagged TestMain fronts the entire scripts_test package, including
ordinary tagged and untagged go test ./scripts invocations. Without the
required-suite flag it delegates to m.Run; runtime failures and intentional
skips remain Go's responsibility.
Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell
* chore(bazel): enumerate required_suite_test.go in the scripts go_test srcs
gazelle-generated, scoped to this PR only. The file is untagged so the
go_test target owes its enumeration; the in-package census showed the
gap was exactly this file with zero pre-existing untagged gaps.
Note make bazel-sync also wants internal/storage/dolt/BUILD.bazel for
version_staging_completeness_test.go, but that is a BASE-owned gap from
the just-merged #6844 and is deliberately left out of this diff.
* test(ci): register the 11th doc-freshness test and pin real flag reads
Two maintainer review fixes on top of the contributor branch.
Blocker: requiredSuiteDefinitions pinned 10 TestDocFreshness* names while 11
are compiled at this head. The 11th,
TestDocFreshnessRejectsMultipleLastReviewedMarkers, arrived on the base in
0bf13494f0 (#6482) during the 164-commit merge-base advance, so the
contributor's list was correct when written. Left unregistered, the required
check-doc-freshness-platforms step this PR itself edits fails on all three
OSes (pr.yml:476, required via pr.yml:1378 and :1408), and the same drift
independently reds TestRequiredSuiteContract/compiled_suite in every
integration-tagged whole-package lane. Registering the name is the whole fix.
Major (test evidence): the flag table injects synthetic requiredSuiteFlags
literals, so it proved only that the predicates reject narrowing flags, never
that real flag values reach them. currentRequiredSuiteFlags is the sole
production reader of real test flags and went unexercised, so a wrong lookup
key silently reopened the fail-open hole this PR exists to close. skip, cpu,
list and short all compare against zero values and so fail open; run and count
fail closed. The new "real flag reads" arms hand a child real -test.skip,
-test.cpu, -test.list and -test.short alongside a valid -test.run and require
the matching diagnostic rather than the generic prefix, because a wrong key
reads the zero value and falls through to a different failure that the prefix
would accept. Mutating each lookup key in turn reds exactly its own arm and
leaves the other three green, tagged and untagged.
Those arms are the first to hand a child a valid selection, which necessarily
selects this test too, so a child that got past a broken guard re-entered here
and spawned without bound (measured: a monotonically growing child count that
never terminates). BEADS_TEST_REQUIRED_SUITE_CHILD stops that at depth 1, so
the regression surfaces as a failed assertion in seconds instead of a hang.
The arms sit outside "compiled suite" because the flag guards run ahead of the
inventory check, so they also hold in the untagged fast and bazel lanes where
compiled suite self-skips.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
pr.yml's bazel.yml call now passes integration: "on", and its ci-gate
requires BAZEL_INTEGRATION (bazel-integration's job.status output, a
missing result reading as skipped). The lane stays remote-only:
bazel-gate.sh allows it to skip in modes local (fork and Dependabot PRs,
until the farm has a read-only cache for them) and skip, never in remote.
bazel-farm.yml's advisory call runs it too.
The integration test step gets 45 minutes (job 55) for cold PR compiles;
the policy test keeps it at or above a 20-minute cold compile plus
.bazelrc's test:integration --test_timeout (the 1200s per-action cap).
The integration input (default "on") stays for callers that want the lane
off; pr.yml's call inputs are pinned, and the gate simulation now covers
the lane skipped, failed and cancelled in each mode.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(winget): require PortableCommandAlias for GasTownHall.Beads
GasTownHall.Beads 1.1.0 on winget-pkgs only sets Commands: (search
metadata) and never creates WinGet\Links\bd.exe, so already-running
shells never see bd after install (GH#4908).
- Add canonical GasTownHall.Beads.installer.yaml with PortableCommandAlias
- update-winget.sh regenerates that package id for new releases
- Document both package ids and the Links requirement in winget/README
- Add check-winget-portable-alias.sh guard
Fixes#4908
* fix(winget): address maphew's PR #5134 review (grep bug, guard, CI wiring)
Maintainer commits requested by the PR author on the reviewer's behalf,
addressing maphew's 2026-07-29 review of PR #5134:
- scripts/update-winget.sh: `grep windows` matched both amd64 and arm64
checksum lines, corrupting InstallerSha256 into an unparseable
two-line value. Narrow to `grep windows_amd64` (mirrors the existing
arm64 grep on the next block).
- scripts/update-winget.sh: quote the arm64 InstallerSha256 placeholder
so 64 bare zero digits don't parse as a YAML integer, matching the
quoting already used in the checked-in SteveYegge.beads.installer.yaml.
- scripts/check-winget-portable-alias.sh: replace the bare
`grep -q PortableCommandAlias:` substring check with a yq-based parse,
so a corrupted manifest (like the one above) fails the guard instead
of reporting OK; also fail closed on an empty winget/ glob instead of
silently exiting 0.
- .github/workflows/pr.yml, main.yml: run the guard in the existing
check-build-tags job so it executes on every PR instead of only when
someone remembers to invoke it by hand.
Locally verified: guard passes against the current checked-in
manifests, fails against a reproduction of the exact grep-bug
corruption, fails on an empty winget/ dir, and the generator produces
a yq-parseable manifest against a synthetic checksums.txt. shellcheck
clean; both workflow files parse as valid YAML.
* fix(winget): harden the PortableCommandAlias guard and its release docs
Maintainer review fixups for the adopt-pr review of PR #5134 (iteration 1,
synthesis approve / 0 blockers and 0 majors; all seven items were non-gating).
Scorecard 961/1000, Required Changes: None. Every edit site below is a line
this PR itself added.
Finding 1 (minor, Error Handling) — `yq -e ... >/dev/null 2>&1` collapsed three
distinct failures (yq absent at exit 127, unparseable manifest, alias genuinely
missing) into "missing valid PortableCommandAlias", defeating the script's own
stated rationale and pointing a debugger at manifests that are already correct.
The guard now names a missing yq up front and no longer discards stderr, so
yq's own diagnostic reaches the log of a required-gate job. yq is CI's first and
only yq consumer, running in a bare actions/checkout job, and sits outside the
host-tools inventory declared in scripts/BUILD.bazel; that is now stated where
the dependency lives.
Finding 4 (minor, Change Impact) — the glob is winget/*.installer.yaml but the
assertion was unconditional on installer type, so a future msi/exe/msix
manifest (no NestedInstallerFiles at all) would hard-fail a gate-feeding job
with no opt-out. Non-portable manifests are now classified out with a SKIP line.
The parse check deliberately precedes the class check: without that ordering an
unparseable manifest fails the NestedInstallerType test and is silently skipped
as "not portable", which is strictly worse than the behaviour being fixed.
Finding 2 (minor, Error Handling) — amd64 and arm64 had opposite failure
policies in one script: a missing amd64 checksum hard-fails, while a missing
arm64 checksum substituted 64 zeros and continued to success, disclosed only in
a source comment. The echoed "Next steps" block now ends in a loud
"DO NOT PUBLISH" warning naming the placeholder, emitted only when the fallback
actually fires.
The finding's second arm — "teach the guard to reject an all-zero
InstallerSha256" — is implemented as a WARNING rather than a failure, and
that is a deliberate downgrade. winget/SteveYegge.beads.installer.yaml:15
already carries exactly that placeholder for its arm64 entry; that file is
base-owned and is not in this PR's six-file diff, so a fatal tripwire would
turn a required check red over pre-existing content the PR never touched.
The legacy heredoc in update-winget.sh emits no arm64 entry at all, so the
placeholder is stale hand-authored state that the next generator run removes.
Verified: the guard still exits 0 on the real tree and reports the warning.
Finding 3 (minor, Contract & Interface Fidelity) — a winget-pkgs version
directory needs a version + defaultLocale + installer set, but the repo carries
only GasTownHall.Beads.installer.yaml, so README step 2 as written produced a
set winget-pkgs validation rejects. Took the documentation arm: GasTownHall is
now steered to `wingetcreate update` (which supplies the other two manifests
from the published set) and the hand-copy path is marked as applying only to
the legacy id, which does have a complete three-file set. Generating two more
manifests would have changed the contribution's deliverable scope.
Finding 5 (nit, Architectural Consistency) — the first update-winget.sh run
overwrote the checked-in manifest with a three-line header, deleting this PR's
own GH#4908 rationale from the artifact whose purpose is to carry it. The
heredoc now emits the same header, plus a fresh ReleaseDate. Verified by
generating into a temp tree: the generated header is byte-identical to the
checked-in file's first ten lines.
Finding 6 (nit, Documentation Accuracy) — the README froze SteveYegge.beads at
the "0.30.x era" while the script rewrites all three of its manifests to the
new version. Note that the section already disclosed that both ids are
refreshed, so the residual gap was narrower than the finding stated: what was
missing is that the legacy id's version is bumped out of the era the table
advertises. That clause is now explicit.
Finding 7 (nit, Test Evidence Quality) — the guard's fail-closed behaviour was
pinned by nothing in-repo. Adds scripts/check_winget_portable_alias_test.go
with ten cases: alias present, alias missing, wrong alias, empty winget dir,
unparseable manifest (asserting it is neither skipped nor misreported and that
yq's stderr survives), missing yq, a skipped non-portable manifest, a violating
portable sibling of a skipped manifest, and the advisory placeholder warning.
Deviates from the prescribed `requireHostTool(t, "yq")` on purpose.
requireHostTool fails rather than skips under Bazel because
//scripts:scripts_test is tagged host-tools, and that tag's declared
inventory is "git, python3, awk, sed, find, grep and timeout" — yq is not in
it, so the prescription would red the shared target on hosts that keep the
promise the tag actually makes. A skip-only gate is used and the reason is
recorded at the helper. Fixtures are synthetic so the target needs no new
data dependency; the real manifests are covered by the check-build-tags job,
which runs this script against the actual checkout.
Also fixed, found while editing and not raised by either lane: winget/README.md
told readers to run `grep windows` over checksums.txt, which is the exact
two-line ambiguity this PR fixed in the script (`grep windows` ->
`grep windows_amd64`). Before this PR the script and doc agreed; after it they
diverged, so the PR created the inconsistency. The doc now uses `grep windows_`
and says which line feeds which architecture.
Validation. `git diff --check` clean. The guard runs green on the real tree
(both manifests OK, rc=0) and on a freshly generated tree. go test ./scripts/
ok 21.7s (the package whose tests pin workflow content). Each new pin was
proven non-vacuous by reverting the corresponding fix and confirming the test
reds: dropping the class check reds the non-portable case; dropping the
placeholder tripwire reds the warning case; dropping the parse-before-classify
check reds the unparseable case; dropping the yq preflight reds the missing-yq
case; restoring `2>&1` reds the stderr assertion. All five arms red, and the
guard was restored byte-identical (md5 verified) after each. The arm64 warning
was exercised both ways against a stubbed curl: present arm64 checksum emits no
warning, absent one emits DO NOT PUBLISH.
New yq expressions stay inside the vocabulary shared by mikefarah yq v4 (what
ubuntu-latest ships) and python-yq/jq — identity, `==`, `[]`, `select` and
`-e`'s exit status — because only the pre-existing expression had been proven
green on mikefarah in CI, and this host has python-yq. Builtins the two flavours
spell differently (jq's `any(cond)` vs mikefarah's `any_c`) are avoided, and the
placeholder tripwire uses grep precisely because it looks for a known-bad
literal and so can only miss a bad value, never report OK on a manifest it
failed to parse.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Bee <bee@ghosttrack.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
* test(init): make gateway credential fixture portable (#5715)
Replace shell-specific fixture commands with an opt-in test helper, exercise exact credential semantics under a restricted shell PATH without copied Windows PE fixtures, and require the focused suite in native Windows PR CI.
Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell
* test(init): keep the original shell across native credential fixtures
Use a symlink to the original Unix shell so macOS AMFI does not reject a
copied system executable. Keep Windows cmd.exe and the test image in place,
internalize helper environment names and assert the typed helper exit code.
Run the six fixture cases through one Bash3.2-compatible driver in the
existing required three-host preflight matrix. Preserve the liveness check
name and optional macOS suite policy; reject missing or skipped execution.
Review: https://github.com/gastownhall/beads/pull/5725#pullrequestreview-5096996756
Bead: saa-cgh.47
Agent-Signature: unknown-model/unknown-reasoning
* test(ci): sync-check the native fixture list and stop forcing a second build graph
Maintainer review fixes for PR #5725, adoption iteration 1. The synthesis is
`approve` with no required changes; all findings are minor or nit. The rebase
lane had to repair this branch (`repaired_clean`, and the original head still
conflicts with main), so there is no accepted-as-is path and finalize owes a
push either way — these cost nothing extra.
Test evidence (minor): the `expected` array is, by its own comment, the single
list owning both test selection and required execution evidence -- but nothing
tied it back to the source file. A seventh TestApplyInitGatewayCredential*
fixture added later would fall out of the `-run` selector, lose native
macOS/Windows execution entirely, and still leave the driver exiting 0 with the
required gate green: precisely the "green because nothing ran" mode the per-name
PASS counting exists to prevent. The driver now compares the array against
`grep -Eo '^func TestApplyInitGatewayCredential[A-Za-z0-9_]*'` on
cmd/bd/init_gateway_test.go, by name and by count, before running anything, and
exits non-zero naming the offending fixture. That file is the whole population --
no same-prefix fixture lives in any other cmd/bd test file.
Release safety (minor): the driver sourced .buildflags and then forced
CGO_ENABLED=0, compiling a package graph disjoint from the ./cmd/bd build the
same job performs two steps earlier under the default. Re-measured under the
tags the steps use: `go list -deps -tags=gms_pure_go ./cmd/bd` is 1505 packages
with cgo and 742 without, and the differing cgo users include `net` and
`os/user`, whose build IDs propagate upward -- so nothing was reused and this
blocking, `cache: false` lane paid a second cold compile on macOS and Windows.
The export is dropped. Two consequences beyond the time: the step becomes
incremental, and the natively executed fixtures now exercise the same build of
cmd/bd that actually ships on those platforms rather than a nocgo one. The
comment now explains the build configuration, which is what the old one did not
do -- it explained why no database is needed, a separate question.
Debuggability (nit): the base comment above test-windows-liveness carried three
checkable mechanism clauses (the test is //go:build windows; the Ubuntu job
selects it by name but compiles it out; main.yml's Windows job runs only on
pushes to main). The replacement stated a conclusion a future maintainer cannot
verify, which is how a redundant-looking CI job gets deleted. The mechanisms are
restored and the new pr-preflight-platforms pointer is appended rather than
substituted.
The remaining nit asked for no change in this PR (the credential cache is keyed
by command string, which is harmless as shipped since neither the marker nor the
failure arm populates the cache) and none was made.
Validation: the driver runs green on Linux under the inherited default CGO --
6 of 6 fixtures PASS, exit 0 -- and the sync check was proved by appending an
unwired seventh fixture, which reds it with exit 1 naming that fixture, before
any test runs. `bash -n` clean, no shellcheck SC2086, and ./scripts/ stays green.
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* test(credentials): make command fixtures Windows-portable (#5748)
Replace POSIX-only credential test commands with a shared shell-portable protocol, exercise their intended failure and non-invocation semantics on native Windows, and add the focused required CI selector.
Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell
* test(ci): require each Windows credential fixture to pass
Include the standalone credentialcmd protocol test in the existing Windows
step. One package/test map drives both selection and exact terminal checks,
so missing, skipped, failed, or duplicate tests cannot report success.
Resolve native Go once and reuse it for the test invocation.
Bead: saa-cgh.66
Agent-Signature: unknown-model/unknown-reasoning
* test(ci): select the first Go application on PATH
Hosted Windows images can expose more than one Go application. Preserve
setup-go's PATH precedence, select one absolute application and reuse it.
Bead: saa-cgh.66
Agent-Signature: unknown-model/unknown-reasoning
* chore(bazel): sync BUILD.bazel srcs for the new credentialcmd test helper
Base landed the Bazel PR gates (#6881/#6887) after this PR was authored, so
the PR's new internal/testutil/credentialcmd package and internal/creds
testmain_test.go were never enumerated in any BUILD.bazel. Two already
enumerated test srcs (internal/creds/command_test.go and
internal/storage/dolt/gateway_credential_test.go) import the new package, so
`bazel build //internal/creds:all` fails GoCompilePkg without this.
Generated by `make bazel-sync` (gazelle); a second sync is a no-op. The
internal/storage/dolt version_staging_completeness_test.go srcs row is
pre-existing base drift that gazelle corrects in the same pass, not PR content.
* test(credentialcmd): pin the Windows gate's assertions and stop claiming foreign helpers
Maintainer review fixes for PR #5749, adoption iteration 1. The synthesis is
`approve` with no required changes; all four findings are minor or nit. A
maintainer bazel commit already exists on this branch, so the push was owed
regardless and none of these cost anything extra. No production code changes.
F1 — the new required Windows gate pinned its own wiring but almost none of its
assertions. The guard checked three package paths and one test name, so it
stayed green if someone dropped six of the nine test names from $expected,
deleted the `Action -in 'fail', 'skip'` throw, or deleted the exactly-one-PASS
loop -- leaving a required gate whose only real guarantee was "go test exited
0", which a fully skipped suite also satisfies. The pinned slice now carries all
nine test names plus both load-bearing guard tokens, keeping the file's existing
substring idiom.
F2 — the property the fixture exists for was unassertable end to end. Emit is
written to produce byte-exact stdout across sh and the Windows .cmd trampoline,
but parseCredential runs bytes.TrimSpace before every comparison, so every
existing assertion is blind to a trailing newline: a trampoline changed to
`echo %OUT%` would keep the whole suite green. TestCommandSourceRealShell now
wraps the existing credRunner package seam to capture the helper's stdout as the
shell produced it, and asserts it byte-exact. The real shell stays in the loop,
so this remains end-to-end rather than becoming a second in-process test.
F3 — Dispatch claimed any process where BEADS_TEST_CREDENTIAL_COMMAND_HELPER=1,
on the env alone and before inspecting argv, exiting 97 when the sentinel was
absent. command() arms that variable in the *parent* process and BEADS_-prefixed
variables reach every re-exec'd child through the integration allowlist, so the
first composition with a package that re-execs its own test binary -- which
internal/storage/dolt already does -- would have died with a wrong-subsystem
"missing protocol sentinel". Both gates are now required, and an env-armed
process without the sentinel falls through to its own TestMain.
The second half of that finding's prescription is deliberately NOT applied.
Adding BEADS_TEST_CREDENTIAL_COMMAND_HELPER to helperSubprocessSentinels reds
TestHelperSubprocessSentinelsAreComplete: that guard has a reverse-direction
control, and its regex looks for a literal os.Getenv("BEADS_..._HELPER") in
internal/storage/dolt's own test sources. This sentinel is read in
internal/testutil/credentialcmd, so the entry registers as a stale one. Measured,
not assumed -- the failure is "helperSubprocessSentinels lists
BEADS_TEST_CREDENTIAL_COMMAND_HELPER, but no test source reads it". With the
Dispatch gate fixed the entry also buys nothing: a dolt helper child already
carries its own sentinel, so isHelperSubprocess is true for it either way.
F4 — the exported constructors documented themselves as "returns a unique shell
command" while replacing PATH for the rest of the calling test, setting env vars
via t.Setenv (which bans t.Parallel), and silently rejecting an empty payload.
The shared contract is now stated once above the three of them, and Emit fails
fast with an explanation instead of letting the shell drop the empty payload
word and the helper exit 97. No caller passes an empty payload today.
Each fix was checked by reverting it and confirming its own pin reddens: the
dropped test name reds the guard by name; the trailing-newline wrong
implementation reds the new raw-stdout assertion while the sibling end-to-end
test stays green, which is the blindness the finding described; and reverting
the Dispatch gate reproduces `(97, true)`. `make bazel-sync` is a no-op -- every
edit landed in a file already enumerated in srcs.
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* ci: cross-compile release targets on PRs
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Atirna
* ci(pr): pin the release-target cross-compilation gate and disclose its CGO boundary
Maintainer review fixup for #5684.
Disclose the coverage boundary on the new job. All eight legs build
CGO_ENABLED=0, but five of them stand for release binaries built at
CGO_ENABLED=1: bd-linux-amd64, bd-linux-arm64 and bd-windows-amd64 in
.goreleaser.yml, plus the darwin pair built natively by release.yml's
goreleaser-macos job. The cgo half of the tree (beads_cgo.go ->
internal/storage/embeddeddolt) is never compiled by this check, so a
break confined to it still passes green. Only android/arm64,
windows/arm64 and freebsd/amd64 replicate their release configuration
exactly. The comment also records that the ./... build scope is
deliberate extra coverage rather than an oversight, since every
.goreleaser.yml build id declares main: ./cmd/bd.
Pin the gate wiring. Nothing asserted the new job: deleting
CHECK_RELEASE_TARGET_CROSS_COMPILATION from CI_GATE_REQUIRED left
go test ./scripts/ fully green, against a repo convention of pinning
each load-bearing gate token by name.
Guard matrix/.goreleaser.yml drift in both directions, so a future
release target cannot go uncovered behind a green "release target
cross-compilation" check -- a failure mode worse than having no check,
because the check's existence implies the coverage it has lost.
Restore the Go module cache, matching the file's other multi-leg matrix
jobs. This is the largest matrix in the file and the only one building
the entire module.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* test(dolt): red — defer auto-wake must run for classified-read-only server-mode opens (be-vbhpf)
TestServerModeDeferAutoWake exercises the same defer/wake contract as the
embedded and proxied-server variants, against a real dolt sql-server. Today
the sweep (DoltStore.wakeExpiredDefers) unconditionally skips whenever the
store is opened read-only — including when that read-only-ness comes purely
from command classification (bd ready/bd list are read-only for the CURRENT
project), which is indistinguishable, before the fix, from a strict
--readonly or foreign-project open that must never sweep.
RED: expired_dated_defer_wakes_on_ready and expired_dated_defer_wakes_on_list_ready
fail (issue never returns to open). dateless_defer_never_wakes and
future_dated_defer_stays_hidden pass today too, trivially (no sweep runs at
all pre-fix) — kept as regression guards for the GREEN step, which is where a
unit test proving classifiedRead=false still does NOT sweep belongs, since
that distinction doesn't exist yet at RED.
* feat: green — defer auto-wake runs for classified-read-only server-mode opens (refs be-vbhpf)
bd ready/bd list on a dolt-sql-server-backed store open with the store
classified read-only for command purposes; the lazy defer-wake sweep
silently early-returned before ever running, so expired defers never
woke unless something else opened the store writable first.
Threads a new ClassifiedRead flag through dolt.Config -> DoltStore,
set at the call site to exactly "read-only because of command
classification, not strict --readonly/preview/foreign-project"
(policy.readOnly && !readonlyMode && !previewMode in main.go).
wakeExpiredDefers's guard now delegates to a small extracted
predicate, deferWakeSweepEligible(readOnly, classifiedRead), so the
truth table can be unit-tested without a live store.
* chore: release gate PASS for be-6iglh-gate
* docs(release-gates): record the be-5fjqh rebase onto current main
origin/main advanced 6 commits past this branch's base and PR #6082 went
CONFLICTING; hourly pr-audit filed be-5fjqh. The branch is now rebased onto
0efe0adf5.
The only conflict was cmd/bd/main.go: this branch and e05000d64 (wy-sbgucn)
each append a field to the same dolt.Config literal at the same anchor line.
Resolved keep-both -- PoolReadTimeoutFallback and ClassifiedRead drive
disjoint code paths.
The addendum records the new base and SHAs, patch-id equivalence for the five
files that did not conflict, what was re-verified at the new base (build plus
the diff-owned tests, container-backed), and what was NOT re-run (the
full-suite and ci-pr-policy lanes still refer to the old base).
Agent-Signature: claude-code-claude-opus-5[1m]-unknown-reasoning on behalf of Jim Wordelman
* docs(release-gates): record the be-qjern re-propose rebase and full re-gate
PR #6082 was closed and its branch deleted 2026-09-01 by an unknown actor
under the shared quad341 identity. Operator ruling 2026-09-08: accidental,
not a rejection — re-proposing. This addendum records a second self-rebase
onto current origin/main (zero conflicts this time), the full-suite and
ci-pr-policy re-gate the 09-01 addendum left outstanding (0 FAIL, policy
FAIL attributed to pre-existing be-mrl6j), and re-verification of both
diff-owned container-backed tests at the new HEAD.
refs be-qjern
* test(feat): red — add strict --readonly server-mode subtest to TestServerModeDeferAutoWake (refs be-acsyx)
* docs(release-gates): green — reword container-backed evidence claims, add nit-3 comment (refs be-acsyx)
bee-ghosttrack nit 2: TestDeferWakeSweepEligibleHonorsClassifiedRead is a
pure-function table test, not container-backed like TestServerModeDeferAutoWake
— reworded release-gates/be-6iglh-gate.md criteria 2(4) and 3 accordingly, and
attributed the preview/foreign-project claim to code structure at
cmd/bd/main.go:1485 rather than to that test. PR #6424 body reworded to match
(not part of this commit; edited via gh pr edit).
bee-ghosttrack nit 3 (optional): one-line comment on the unreachable
"classifiedRead without readOnly" table case explaining it documents totality,
not a reachable state.
Full suite: 0 FAIL. Targeted re-verification (TESTCONTAINERS_RYUK_DISABLED=true
to dodge host-load reaper flakiness): TestServerModeDeferAutoWake 5/5 subtests
PASS (12.427s), TestDeferWakeSweepEligibleHonorsClassifiedRead 4/4 subtests
PASS (5.957s).
* chore: remove leaked internal release-gate doc from PR branch
release-gates/be-6iglh-gate.md
Tracked our internal gc-management/beads-rig release-gate checklist and
should never have been committed to this upstream branch -- it's process
residue from our deploy tooling, not part of the change itself.
Per maintainer bee-ghosttrack's request during review of PR #6571. The
root cause (our deploy tooling auto-committing this file) is already fixed
as of gm-qatxl1; this branch predates that fix, so this commit is cleanup
only -- no other content on this branch is touched.
Ref: be-lk7o3
* ci(proxied-shard): discover TestServerMode* entry points
Review finding (major, Test Evidence Quality): TestServerModeDeferAutoWake —
this PR's only end-to-end guard for the classified-read defer-wake fix — was
selected by no CI lane. The two lanes that set BEADS_TEST_PROXIED_SERVER=1
both shard through proxied-test-shard.sh, whose discovery grep matched only
`^func TestProxiedServer`, so the wiring lines at cmd/bd/main.go:1675 and
internal/storage/dolt/store.go:2136 had zero CI-executed coverage: the one
CI-reachable new test pins the pure predicate and cannot catch a mis-wired
flag.
Widen discovery to `^func Test(ProxiedServer|ServerMode)` in the shard script
and in scripts/ci/gen_proxied_shard_manifest.py so the two discovery patterns
stay in step, and widen the shard script's manifest-name validation to match.
That last part is load-bearing, not cosmetic: without it a regenerated
manifest containing a TestServerMode* row is rejected outright ("Invalid
shard manifest line") and the whole lane exits 1.
This also rescues the two pre-existing TestServerModeServe* suites, which
share the identical requireSharedProxiedServer gate and the same shared dolt
sql-server harness and were equally invisible — the unclosed class this PR
was following.
Verified: discovery goes 151 -> 154; a 15-shard BEADS_TEST_SHARD_LIST_ONLY=1
dry run assigns each of the 154 to exactly one shard with no pre-existing
test moved (the three land on shards 4/13/15 via hash fallback); the
pre-change script selects none of the three.
Also clarifies the reviewers' nit: the "foreign-project" exclusion named in
the new comments is not tested at the guard or in the policy expression —
those opens build their own dolt.Config and leave ClassifiedRead false, so
the guarantee is fail-closed by default.
Iteration-2 review (two minors, both non-gating): the fixup's own new prose
asserted two universals that measurement falsifies — the same class as the
nit above, reintroduced in the CI files this fixup added.
* proxied-test-shard.sh claimed "Both families gate on
requireSharedProxiedServer", contradicted two lines earlier by its own
mention of the TestProxiedServerPathHelpers unit test. Transitive gate
census over the 154 discovered entry points: 143 reach
requireSharedProxiedServer, 10 reach the different helper
requireProxiedServerEnv, and TestProxiedServerPathHelpers
(cmd/bd/proxied_server_test.go, bare `package main`, three assert.Equal
calls on path helpers) has no gate and no build tag, so it also runs in
every ordinary `go test ./cmd/bd/` lane. Now scoped to the
TestServerMode* family, which does gate uniformly (3/3), and the
TestProxiedServer* side is described as mixed rather than generalized
over.
* main.yml claimed TestServerMode* discovery means "neither family can be
silently left out of CI". Discovery is a non-recursive cmd/bd/*_test.go
glob: 154 top-level matches versus 156 recursive, the difference being
cmd/bd/doctor/database_test.go:60 and cmd/bd/doctor/dolt_test.go:126.
Now scoped to the top-level entry points, and the comment names the
residual hazard (a container-gated suite added under cmd/bd/<subpkg>/ is
still never selected here) instead of implying it cannot happen.
Both edits are comment-only and behaviorally inert: non-comment lines are
byte-identical in both files, and the 15-shard BEADS_TEST_SHARD_LIST_ONLY=1
sweep is byte-identical before and after (154 assignments, 0 duplicates,
assigned set == discovered set, the three TestServerMode* still at 4/13/15).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* ci(bazel): add defer_wake_server_test.go to cmd/bd go_test srcs
Maintainer CI repair, generated by `make bazel-sync` (gazelle).
`cmd/bd/BUILD.bazel`'s `go_test` srcs is an explicit list, not a glob, so the
new `cmd/bd/defer_wake_server_test.go` was never compiled or run by the Bazel
lane. `Bazel / test` failed two required steps on the pushed head
`0a3f5e0047`:
Go test equivalence:
go test: 105 packages, 8988 top-level tests
bazel: 107 go_test targets, 106 tested; 8987 top-level tests seen
tests go test runs that Bazel did not (1):
cmd/bd TestServerModeDeferAutoWake
EQUIVALENCE: FAIL
BUILD files in sync (gazelle, go_srcs, MODULE.bazel): out of sync
`BAZEL_TEST` is listed in `CI_GATE_REQUIRED` for `CI Gate / Required`
(.github/workflows/pr.yml), so this is a required-lane red, not advisory.
That the missing test is this PR's own headline guard makes the repair squarely
in scope: the PR exists to close exactly this "no CI lane runs it" gap, and its
review fixup closed it for the proxied shard lane. The Bazel lane had the same
gap for a different reason.
Attribution: main's Bazel run at this PR's exact base 43a526b828 concluded
success, so the red is this branch's, not inherited.
Validation:
- `make bazel-sync` produces exactly this one line; a second run is a no-op
(gazelle idempotent, md5 7d1d0549fcd466159451c02181661c78).
- Byte-identical to CI's own `bazel-sync-patch` artifact for this head
(blob pair efec974..6dd76e3).
- srcs-only change: gazelle added no deps entry, so every import the new file
needs was already declared.
- `bazel build //cmd/bd:bd_test` succeeds, and the generated
`bd_test_/testmain.go` now enumerates TestServerModeDeferAutoWake, which is
the row the equivalence check reported missing.
- `tools/bazel/go_srcs.py --check` was already clean, and `bazel mod tidy`
produced no MODULE.bazel drift, so this single line is the whole repair.
No contributor commit is modified and no Go source changes.
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(make): preserve atomic Windows replacement
Stage and rename bd.exe before enumerating legacy extensionless aliases. This avoids MSYS .exe fallback, preserves the working install when staging fails, and handles case variants and redirected install directories.
Extend the native Windows Make proof with failed-replacement preservation, uppercase-alias cleanup, and a real directory junction.
Fixes#5626
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
* fix(make): preserve directories during Windows alias cleanup
Exclude directories named bd or BD from extensionless alias removal and
document the existing MSYS find PATH dependency. Extend the native install
proof to preserve a nonempty BD directory and exact replacement contents.
Fixes#6196
* docs(make): clarify Windows alias traversal (saa-cgh.81)
Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell
* Clarify find predicate without assuming junction classification (saa-cgh.81)
Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell
* ci(bazel): remote Bazel lanes for allowlisted fork authors (advisory)
Add bazel-farm.yml, a pull_request_target caller of bazel.yml. It gives the
RBE farm to fork PRs to main whose author and triggering user (sender: the
pusher on synchronize) are both on .github/bazel-farm-allowlist.txt, which
is read from the base commit. The list has julianknutsen, quad341, sjarmak
and csells, the same as gascity's blacksmith allowlist.
- authorize job: sparse checkout of github.sha that holds only the
allowlist and .github/scripts/bazel-farm-authorize.sh; no token on disk,
no secrets, event facts through env only. The script also checks fork vs
base repo, the default branch, the action, and a 40-hex head SHA.
- farm job: calls bazel.yml with contents: read, exactly the four RBE
secrets, checkout-sha pinned to the event's head SHA, and
fork-farm: authorized.
- bazel.yml: new checkout-sha and fork-farm inputs (no-ops for pr.yml and
nightly.yml). Every lane checks out inputs.checkout-sha with
persist-credentials: false. The rbe job runs a fork remotely only for an
authorized pull_request_target call, and then remote or skip, never
local. PR concurrency also cancels on pull_request_target. The
bazel-sync-patch upload is skipped on pull_request_target.
Advisory only: pr.yml is unchanged, so every fork PR still runs the local
Bazel lanes inside the enforced CI Gate / Required.
Policy tests pin the trigger and types, both jobs' conditions, the
allowlist source and contents, the pinned checkout, read-only permissions,
where secrets appear, no expressions in run scripts, per-PR concurrency,
the timeout, and that no workflow_run consumer can pick up a farm run.
TestBazelLaneIsGatedAlongsideLegacy now allows pull_request_target on a
bazel.yml caller for bazel-farm.yml only. 26 mutations were each caught
by these tests (sender check dropped, allowlist read from the PR head,
write permission added, event data interpolated into run, SHA pin dropped,
and others).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): trusted-author fork PR farm design and threat model
Document bazel-farm.yml in the required-check topology:
- the design, and why it is advisory rather than part of the enforced
gate;
- the gate options considered;
- the threat model: allowlist spoofing, sender vs. author, TOCTOU with the
pinned SHA, workflow-file edits, script, label and comment injection,
token scope, artifact poisoning, and farm and Actions cache poisoning;
- what can only be verified on GitHub.
The one capability a listed author gets beyond a same-repo PR is called
out as an accepted risk, with mitigations: pull_request_target runs use
the default branch's Actions cache scope.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): opt the farm's lane checkouts into fork-head checkout (F1)
actions/checkout@3d3c42e5 is v7.0.1, not v6, and v7 refuses to check out
a fork PR's head SHA on pull_request_target unless
allow-unsafe-pr-checkout is true. Every lane of an authorized farm run
therefore failed at checkout.
- Each bazel.yml lane checkout now sets allow-unsafe-pr-checkout to
inputs.fork-farm == 'authorized' && github.event_name ==
'pull_request_target'. That is false for every other caller and event.
- TestBazelWorkflowForkFarmInputs pins that exact expression. It also
fails if the key appears anywhere else in the workflows (so never a
literal true, and never on the authorize checkout).
- Relabel the checkout pin as # v7.0.1 in bazel.yml and bazel-farm.yml.
- Add the opt-in to the engdoc's design section and to the
verify-on-GitHub list.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): verify everything restored from caches in Bazel lanes (F3)
A farm run executes PR code in the default branch's cache scope. GitHub's
own cache is read-only for pull_request_target: GitHub issues a read-only
token, and cache-mode defaults to read. Whether Blacksmith's colocated
cache enforces the same is unverified. So trusted runs no longer execute
anything they restore without verifying it first:
- setup-bazel: Bazelisk's home moves out of the runner cache to a fresh
$RUNNER_TEMP/bazelisk-home, so the Bazel binary is never restored from
a cache. BAZELISK_VERIFY_SHA256 is set to the release sha256 pinned for
the .bazelversion version on each arch (9.2.0 amd64 and arm64). The
wrapper step then re-checks every Bazel binary in that home.
- write-bazelrc.sh: add common --repo_contents_cache=. The repo contents
cache holds extracted repos that Bazel never re-verifies. The
content-addressable repository cache stays, since Bazel re-hashes it on
every hit.
- Runner cache key bumped to bazel-repo-v3 (the content changed).
- bazel-test runs `go mod verify` right after the Go module cache restore.
- Policy tests:
- cache-mode may only be read or none in bazel-farm.yml, bazel.yml and
setup-bazel;
- bazel.yml permissions are exactly contents: read;
- the Bazel sha pin matches .bazelversion, and Bazelisk's home is
outside the cache;
- the binary is verified after download;
- go mod verify follows every module cache restore;
- the rc disables the repo contents cache.
- Engdoc and bazel-farm.yml header: corrected about GitHub's read-only
cache, noted Blacksmith as unknown, and listed the full blast radius
(release.yml hardening tracked separately). The canary design is in the
rollout notes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): farm runs only the author's own fork, on opened/synchronize (F4)
"The sender is listed" did not prove that a listed user chose the head
commit:
- A listed author could open a PR whose head is a third party's fork, and
that fork's owner could push between the author's review and "Create".
- A collaborator on a listed author's fork could push while the PR was
closed or a draft (that push is skipped), and the author's later
"Reopen" or "Ready for review" click would run it with secrets.
Changes:
- The authorize script requires the head repository owner's numeric id to
equal the PR author's (HEAD_OWNER_ID, PR_AUTHOR_ID, passed via env).
- Drop the reopened and ready_for_review triggers, and the script now
refuses those actions too. Drafts already run on opened and
synchronize.
- Tests pin the trigger types and the new env, and add script cases:
- cross-fork head, missing or non-numeric owner id;
- reopened and ready_for_review refused;
- unanchored action (xopenedx);
- base-repo prefix (gastownhall/beads-evil). This covers two mutations
the reviewer found surviving.
- Engdoc: the new checks, and the residual that stays the listed author's
responsibility (commits pushed by fork collaborators).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): match the farm allowlist on numeric user ids (F5)
Logins are mutable: after a rename, anyone can register the old login,
and before this change they would then be listed as both author and
sender.
- The allowlist now holds one numeric user id per line, with the login as
a comment. The ids were looked up with gh api users/<login> --jq .id.
- The authorize script matches PR_AUTHOR_ID and SENDER_ID
(github.event.pull_request.user.id, github.event.sender.id). Logins are
now used only in the log. An allowlist entry that is not a numeric id
fails the job.
- Tests pin the id-to-login table and the SENDER_ID env. Script cases:
- a renamed login re-registered with a new id (author and sender);
- id prefix, suffix and leading zero;
- an id that appears only in a comment;
- listed ids with unlisted logins;
- a PR from another listed user's fork;
- broken allowlists (a login entry, a signed id).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): farm re-run semantics and the add/remove-user procedure (F6)
A re-run replays the original event. It checks out the same pinned head
SHA, which is fine. It also reads the allowlist at the original base
commit, so a user removed from the list still gets the RBE secrets when
one of their old runs is re-run. Only users with write access can
re-run.
- Engdoc: how to add a user.
- Engdoc: how to remove a user. Merge the removal, then cancel and delete
their farm runs (a gh api query by head-repository owner id). For
cause, rotate RBE_TLS_CERT/KEY and have the farm revoke the old
certificate; rotating also closes the re-run window.
- bazel-farm.yml header: points to the procedure.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): name the farm call "Bazel Farm", not "Bazel" (F7)
The farm's check runs were named "Bazel / <lane>", the same as pr.yml's.
Farm results are PR-controlled: PR code runs setup-bazel and can make a
lane exit 0. If a ruleset ever required a "Bazel / ..." check, a farm run
could satisfy it. The farm's checks are now "Bazel Farm / <lane>", and a
test pins the name and requires it to differ from pr.yml's bazel job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): run the farm authorize script in the C locale (F8)
Bash bracket ranges and ${v,,} depend on the locale. In en_US.UTF-8,
[0-9], [0-9a-f] and [A-Za-z] match some non-ASCII characters, and the
Kelvin sign lowercases to "k". This is not exploitable (GitHub ids,
logins and SHAs are ASCII, and the allowlist is maintainer-controlled),
but the script now exports LC_ALL=C.
The script tests now run under en_US.UTF-8 and add cases with non-ASCII
digits in ids and a non-ASCII character in the SHA. Without the export,
the SHA case fails, so the tests catch its removal.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): farm authorize script runs on bash 3.2 (macOS)
${v,,} is bash >= 4; macOS runners' /bin/bash is 3.2, so the script's
tests failed there. Lowercase with tr under LC_ALL=C (ASCII only, as before).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(prime): use native config root in global fixture
The global PRIME override fixture staged Linux and macOS paths while the Windows child resolved os.UserConfigDir from inherited APPDATA. Build one isolated profile, ask the host for its native config directory, and stage the override only at that path. Production behavior is unchanged.
Related to #4638.
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
* test(prime): create advertised profiles and exercise Windows fixture
Create each isolated profile directory before exporting its path. Run the
global override through the shared test runner in the existing required
Windows job, rejecting absent, skipped or failed selected coverage.
Agent-Signature: unknown-model/unknown-reasoning
---------
Co-authored-by: Bee <bee@ghosttrack.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* ci: retry the module fetch in pr-policy-wrapper, not the build
PR Policy (wrapper timing) is a required check, and it is the one job in
pr.yml that can be failed by someone else's network. setup-go runs with
cache: false, so every run pulls the entire module graph from
proxy.golang.org, and nothing retries. A single proxy blip fails the check
outright.
Observed on a docs-only PR (one markdown file, +18/-0): the job died inside
a ~12s proxy.golang.org outage with
stream error: stream ID NNN; INTERNAL_ERROR; received from peer
while 40 other jobs on the same commit, triggered at the same instant, built
the same module graph and passed. The split was by clock, not by content --
nothing about the change could have caused it.
Fetch the modules in their own retried step before the wrapper runs. With
the cache warm, the go build inside scripts/ci/pr-policy.sh compiles without
touching the network and stays unwrapped: a compile error still fails on the
first attempt rather than taking three, and an exhausted retry reports the
fetch as the cause instead of presenting a module outage as a build failure.
The loop mirrors the bounded apt-get steps in migration-test.yml and
release.yml: three attempts, a per-attempt timeout, a fixed backoff with none
after the last, and a step-level timeout-minutes above the resulting worst
case.
* ci: retry the module fetch in main.yml's pr-policy-wrapper too
pr.yml and main.yml carry byte-identical `pr-policy-wrapper` jobs. The
retry added in the previous commit covered only the pr.yml copy, so the
same unretried cold fetch stayed on the main-branch trigger — the job
this PR names is fixed in one workflow and not the other.
Mirrors the pr.yml step exactly: retries the fetch, never the build, and
fails load-bearing while naming the fetch so a module outage is not
reported as a build failure.
This is not a cache step, so the enumerated cache topology pinned by
TestGoCacheOwnershipTopology (scripts/ci_workflow_test.go) is untouched;
`go test ./scripts/` and actionlint are clean.
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
setup-go's default cache in release.yml is keyed predictably and falls back
to the default branch's Go module and build caches, which are restored
without re-verification before binaries are built, signed and attested.
Turn it off in every release job and pin that with a policy test.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): gate the Bazel lane alongside the legacy jobs (D1)
pr.yml now calls bazel.yml (workflow_call) as its `bazel` job, and its
CI Gate / Required requires the call's aggregate (BAZEL) plus each gated
lane's result (BAZEL_TEST, BAZEL_PURE, BAZEL_EMBEDDED, BAZEL_DOLTSERVER).
No legacy job is removed: PR Core, build-artifacts, the pure-Go/js-wasm
check, domain+uow, contract corpus and pr-risk.yml's embedded tier stay
required (D2 is separate).
- bazel.yml drops its own pull_request/merge_group triggers, so a PR runs
the lane once; push to main, workflow_dispatch and workflow_call stay.
- Every lane records job.status in an always() last step and exports it
as a workflow_call output named after the job, next to the rbe job's
rbe-mode / rbe-enabled.
- .github/scripts/bazel-gate.sh reads the call's rbe-mode/rbe-enabled and
never re-derives the variable/fork/secret condition: mode skip allows
every Bazel id to skip, local only the remote-only BAZEL_EMBEDDED,
remote none; a missing or inconsistent mode fails the gate.
- bazel-integration runs in the PR call but stays advisory (its legacy
counterparts in main.yml run only on push to main): no gate id, and the
gate script excuses the aggregate failure only when that lane reported
failure or cancelled in remote mode; the gated lanes are still checked
on their own ids.
- Callers (pr.yml, nightly.yml) pass exactly the four RBE secrets that
bazel.yml now declares, never `secrets: inherit`.
- Inside pr.yml's run the Bazel bd artifact is bazel-ci-build-artifacts,
so it cannot collide with build-artifacts' ci-build-artifacts upload.
- bazel-autofix.yml now triggers on workflow "PR": the bazel-sync-patch
artifact is uploaded inside the PR run, and "Bazel" runs are no longer
PR events. Its security model is unchanged (base-branch checkout only,
artifact read from the triggering run id, data-only patch).
- Policy tests (review D1 F3/F4/F7):
- TestBazelLaneIsGatedAlongsideLegacy: every bazel.yml job is the rbe
job, a gated lane (bazelLaneGateIDs) or an advisory lane with a
reason; gate ids, env expressions and outputs are derived from that
map; no re-derivation in the gate step or script.
- TestBazelGateSimulation runs pr.yml's actual gate step over mode in
{remote, local, skip} (pull_request and merge_group): the script's
skips equal the ids whose job `if:` is false, and for each lane that
should run, it alone skipped, failed or cancelled turns the gate red
whatever the aggregate says; advisory, aggregate and invalid-mode
cases too.
- TestBazelArtifactNamesUniqueInCallerRuns: no artifact name collides
within a pr.yml or nightly.yml run, following local workflow calls.
- Autofix: workflow_run on "PR", which must upload bazel-sync-patch
itself or through an unconditional local call.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): keep integration out of the PR call; pin the call's inputs (D1 review v2)
- N1: bazel.yml gains a string workflow_call input `integration`
(default "on"); bazel-integration runs only when remote and
`inputs.integration != 'off'`. A string, not a boolean: on push and
dispatch the input is null, and null == false would silently stop the
lane on main. pr.yml passes integration: "off" (main.yml's integration
jobs, its legacy counterpart, run only on push to main). The gate no
longer reads bazel-integration and bazel-gate.sh no longer excuses an
aggregate failure: BAZEL is the call's result, red on a missing or
inconsistent rbe mode.
- N3: pr.yml's call passes exactly build-artifact-name and
integration: "off" (no rbe or other override).
- N4: bazel.yml's concurrency group never uses github.workflow (the
caller's name in a call) and differs from each caller's group for every
event, so the call cannot deadlock against pr.yml.
- N5: no caller of bazel.yml (pr.yml, nightly.yml) may have a
pull_request_target or workflow_run trigger.
- Simulation: lanes' run modes are evaluated with pr.yml's call inputs;
advisory lanes must not run in the PR call, gated lanes must run when
remote.
- engdocs: runbook for main BUILD drift, a farm outage or the RBE cert
expiry (~2027-09-27): fix main / renew, or unset RBE_WEST_WORKERS.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record the enforced required-gates ruleset and PR Risk's renamed gate
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): gate the proxied-server and server-Dolt storage lanes
bazel-proxied and bazel-server-storage (remote-only, #6940) join the
gated lanes like bazel-embedded: pr.yml's ci-gate requires
BAZEL_PROXIED and BAZEL_SERVER_STORAGE, each read from the lane's
job.status output (always() recorder step, workflow_call output named
after the job). Both run in pr.yml's call (no integration switch).
bazel-gate.sh allows them to skip in mode local (forks, Dependabot:
pr-risk.yml's legacy tiers cover those PRs) and in mode skip, never in
mode remote. The gate simulation derives this from the jobs' ifs, so
each lane skipped, failed or cancelled in remote mode turns the gate
red. Jobs are renamed "proxied-server tier" / "server-Dolt storage tier"
(shown as "Bazel / ..." under pr.yml's call).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(dbproxy): retire the proxy when its Dolt backend exits cleanly
DoltServer supervised `dolt sql-server` with cmd.Wait inside an
errgroup, and Running read the errgroup context. errgroup cancels that
context only for a non-nil result, so a backend that shut down
gracefully (dolt exits 0 on SIGTERM) was reported as running forever.
The proxy's backend health watcher never fired: the proxy stayed up and
adoptable in front of a dead store. With --proxied-server-idle-timeout 0
(the setting Gas City uses) it never went away; `bd dolt status` said
"running" and every command, `bd ping` included, failed until
`bd dolt stop`.
The supervising goroutine now reports a clean exit as errBackendExited,
so Running flips on any backend exit and Stop treats it as benign. This
follows the existing proxied lifecycle contract (child death retires
the owned proxy; the next bd invocation starts a fresh proxy/backend
pair) rather than adding in-proxy restarts: spawn, adoption, stop
epochs and identity already live in the on-demand start path, and a
second restart loop inside the proxy would duplicate that and could
mask a crash-looping backend.
`bd dolt status` no longer headlines a proxy whose managed backend is
gone as "running"; it says the proxy is up but not serving and points
at `bd dolt stop` (for proxies from an older bd). The JSON payload is
unchanged. `bd ping` already verifies by running SELECT 1 through the
proxy, so after this fix a successful ping reflects a live pair.
Tests: DoltServer Running goes false after SIGTERM (clean exit) and
SIGKILL, and Stop succeeds; managed-local lane test that SIGTERMs the
backend with idle timeout 0 and requires the proxy to retire and
`bd ping --json` to succeed on a fresh pair; status render case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(dolt status): report running=false for a proxy over a dead managed backend
Review follow-up: the JSON running field now means "serving" (a live proxy
and, on a managed topology, a live dolt behind it), so scripts that key off
running alone are not fooled by a stranded proxy left by an older bd.
proxy_pid/proxy_port and backend_* are unchanged. Adds the PR link to the
CHANGELOG entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): resolve merge markers left by the origin/main merge
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* test(integration): let SubprocessRunner reuse a prebuilt test binary
BEADS_TEST_SUBPROCESS_BINARY hands SubprocessRunner an already-built test
binary of the package instead of running `go test -c`, which no Bazel
action can do offline (it needs the module source tree and module cache).
A Bazel target passes the running go_test itself: the helper tests
(TestHelperSchemaInit, TestHelperMultiStore) live in the same package.
Unset, behavior is unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(dolt): keep a lane's REQUIRE switch out of the leak-probe child
TestSuiteFixtureLeakFailsOwningSuite re-runs the suite with
BEADS_TEST_SKIP=dolt on purpose. When the parent runs under
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 (a fail-closed server lane), the child
inherited it and TestMain exited FATAL before the leak sweep the test
checks. Clear it in the child like the other suite-scoped variables.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(bazel): proxied-server and server-Dolt storage tiers (D6d, D6e)
PR Risk's "Test (Proxied Dolt Cmd N/15)" (also main.yml's) and "Test
(Server Dolt Conformance)" / "Test (Server Dolt Full Suite N/16)" as
Bazel variants against local dolt sql-servers (internal/testutil local
backend, pinned dolt from the test wrapper), fail-closed like the
dolt-server lane:
- //cmd/bd:bd_proxied_test: 15 shards; shard k runs the job's
proxied-test-shard.sh k+1 15 through go_test_manifest_shard.sh over the
race bd_test, non-race bd_for_tests as the subprocess bd,
BEADS_TEST_PROXIED_SERVER=1, GOMAXPROCS=4. Tag dolt-server-proxied,
--config=doltserver-proxied (the doltserver lane's build).
- //internal/storage/dolt:dolt_server_{conformance,full}_test over
dolt_race_off (go_test_race_off of dolt_test: the jobs' binary is not
race); the full suite runs server-storage-test-shard.sh k+1 16 with
BEADS_TEST_ENV_RUN_DOLT=1 and hands SubprocessRunner its own binary.
Tag dolt-server-integration, --config=doltserver-integration, whose
build flags are exactly --config=integration's, so the tier shares that
lane's compile (race-off deps land where bd_for_tests already is)
instead of compiling the tagged graph a third time.
- dolt_test gains //internal/storage/schema:migrations as data (the
PR4107/migration replay tests read ../schema/migrations).
- test_env.sh: BEADS_TEST_GIT_IDENTITY=1 writes the jobs' global git
identity ("Configure Git"); the git-remote tests commit in fresh repos.
- Both tags join the taxonomy and bazelPRCoreExcludedTags, so prcore/ci
and integration exclude them.
Per-shard names and statuses equal a replay of the jobs' commands
(container backend) for all 32 jobs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the proxied and server storage tiers remotely (advisory)
bazel-doltserver gains a second, remote-only step, `bazel test //...
--config=doltserver-proxied` (15 shards of ~5-10 min with a busy dolt
sql-server each do not fit a GitHub-hosted runner; forks and rbe=off keep
the local lane unchanged). bazel-integration gains `bazel test //...
--config=doltserver-integration`, which shares its build flags, so the
server storage tier compiles nothing the integration step has not. Both
steps write their own BEP, run check_testcases.py on it, run even when the
job's first test step failed, and upload logs on failure.
Policy (TestBazelDoltServerTiersMirrorPRRisk): each variant runs its jobs'
shard script with their shard total and has script + manifest in data;
the conformance variant passes the job's flags; the jobs' race/non-race
binaries and switches (BEADS_TEST_PROXIED_SERVER, BEADS_TEST_ENV_RUN_DOLT)
match; doltserver-integration's tags and race equal --config=integration's;
the steps are pinned to their jobs. checkDoltServerRules now holds all
three dolt-server lane tags to the local-backend, fail-closed,
remote-capable rules.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the proxied and server storage tiers in jobs of their own
Review of D6d/e: as steps of bazel-doltserver and bazel-integration the
tiers inherited those jobs' gating (a caller switching the integration
lane off would drop PR Risk's PR-time server tier; a gate requiring the
dolt-server lane would silently require the proxied tier too) and
lengthened their critical paths.
bazel-proxied (--config=doltserver-proxied, 15 shards) and
bazel-server-storage (--config=doltserver-integration: conformance + 16
shards) are remote-only jobs that need only rbe, each with its own BEP,
check_testcases.py step and log upload, 25-minute test step inside a
30-minute job. bazel-doltserver and bazel-integration are back to their
previous content, including the dolt-server step's 80-minute timeout for
local runs. The server tier shares bazel-integration's compiles through
the remote cache (same build flags).
The proxied tier mirrors PR Risk's race bd-cmd-test; main.yml's twin
job runs build-artifacts' non-race one, so Bazel is the stricter of the
two. Comments now say so, and the policy test pins main.yml's build so a
change there is noticed. Policy: the new jobs are pinned (command,
config, remote-only if, BEP + testcase steps, timeouts, log upload), and
no tier step may run in bazel-integration or bazel-doltserver.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pr.yml and pr-risk.yml both reported a check run named "CI Gate / Required".
GitHub resolves required checks (and pr-preflight its rollup) latest-per-name,
so a red gate in one workflow could be masked by a later green one in the
other. Rename PR Risk's aggregate to "PR Risk Gate / Required" so both can be
required independently.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): decide the RBE execution mode once, in an rbe job
Dependabot PRs are same-repo (fork=false) but receive no Actions secrets,
so the per-job vars/fork conditions sent them down the remote path with
no executor: every lane ran locally on the 2-vCPU Blacksmith runner and
the remote-only embedded tier timed out (review D1 F1). The conditions
were also repeated in every job and re-read at different times (F5/F6).
A new first job, rbe, decides the mode from four booleans (the
RBE_WEST_WORKERS var, rbe input, fork, and whether RBE_WEST_EXECUTOR is
non-empty) and outputs mode=remote|local|skip plus enabled. Every lane
needs it and takes runs-on, if and the setup-bazel env from its outputs
alone; remote-only bazel-embedded skips unless remote, local-capable
lanes run on ubuntu-latest for forks, rbe=off and secret-less runs.
Same-repo runs still skip while the var is unset. The outputs are also
workflow_call outputs (rbe-enabled, rbe-mode) for a future gate.
Policy tests: every lane needs rbe and uses the pinned expressions; no
value outside the decision step mentions the var, inputs.rbe or fork;
the one extra secret reference is pinned to the emptiness test in the
decision step's env; and the step script is run for same-repo, push,
Dependabot, fork, rbe=off, var-unset and mixed-case scenarios.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): add the advisory bazel-integration job
Runs `bazel test //... --config=integration` (#6916, main.yml's
integration jobs) as its own job in bazel.yml: remote-only like
bazel-embedded (skipped unless the rbe job chose remote), 30-minute test
step inside a 40-minute job, BEP plus check_testcases.py, test logs on
failure. A separate job keeps ci's and integration's shared
bazel-testlogs apart (review-integration F3). No equivalence step:
equivalence.py's default mode is the --config=ci selection, and its job
mode needs the integration jobs' go test -json, which no run here has.
check_testcases.py gains --not-go LABEL for a tested non-Go target
(//tools/bazel:dolt_version_test, a plain sh_test the integration tag
filter keeps); an exemption for a target the BEP does not list fails.
Policy test pins the job's command (no extra filter, selector or remote
config), the count step, the log upload, its remote-only if, and that no
other job runs --config=integration (nor this one --config=ci or
equivalence.py); the .bazelrc tag filter stays pinned by
checkBazelrcIntegrationLane.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>