Commit graph

49 commits

Author SHA1 Message Date
Julian Knutsen
0500fc6f40
ci(bazel): run the scripts and source-walking tests remotely; retire their go test legs (#7320)
//scripts:scripts_test and //scripts/check-versions:check-versions_test
drop their host-tools/no-remote-exec/no-remote-cache tags and run on the
rbe-west workers like every other target (the worker image ships bash, git,
python3, jq, yq, curl, openssl, unzip and coreutils; a missing tool fails the
test under Bazel rather than skipping it). Go comes in hermetically as the
registered SDK; check-versions.sh's `go build` is stood in for by the
Bazel-built checker.

The tests that walked the source checkout now get it as data:
tools/bazel/go_srcs.py keeps a managed `repo_files` filegroup in every
package and aggregates them into //:repo_files (exactly the tracked files
Bazel can see; make bazel-sync-check fails on a package without one).
scripts_test and docsync_test take it instead of hand-listed filegroups,
and every TEST_SRCDIR/IsBazel skip and go-test-only block in them is gone.
The release formula under .bazelignore'd .beads/ comes in through a
@beads_formulas local repository; npm-package/ and .claude/ are visible to
Bazel again (only their node_modules/worktrees stay ignored), so the npm
manifest release check and the AGENTS.md skill links run under Bazel too.
TestZZStdioNotLeaked falls back to TestMain's pre-run streams instead of
skipping in a shard without its baseline.

tools/bazel/equivalence_allowlist.txt shrinks from 20 skip entries to the
two cmd/bd tests of plain go test's own bd build fallback, which Bazel never
takes; pr-preflight-platforms now runs them on Linux, macOS and Windows. The
scripts-go-checks scripts-test and allowlisted legs, their scripts
(scripts-go-test.sh, allowlisted-go-tests.sh, run_allowlisted_go_tests.py)
and main.yml's --compile-only warm for them are removed. The vet leg stays
(ga-96smfk.18).

Refs: ga-96smfk.16

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 22:20:16 +00:00
Julian Knutsen
9f60d71dae
ci(main): push to main runs the Linux test tiers under Bazel only (#7316)
bazel.yml already runs on every push to main and executes the PR Core,
pure-Go, dolt-server, embedded, proxied, server-storage, integration and
cmd/bd Dolt-server tiers on rbe-west in about five minutes. main.yml re-ran
the same tiers as legacy go test jobs (62 jobs, about 380 runner-minutes a
push), and its runs were all failing or cancelled. Remove the duplicated
Linux jobs: the ubuntu -race -short leg with its Codecov upload, PR Core,
the pure-Go compile check, domain+uow, the 6+8 integration shards, the 15
proxied shards, and the embedded build plus its storage, conformance and
cmd shards.

The macOS -race -short suite (rbe-west has no macOS workers) moves to
Blacksmith macOS and restores the macOS saver's caches. The cache savers
that fed the fork path from the removed jobs move to venue matrices: the
macOS saver gains a GitHub-hosted leg, and the Linux saver gains a
GitHub-hosted race leg (the embedded build used to seed it).

Refs ga-96smfk.14

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 20:52:01 +00:00
Julian Knutsen
e72cd8b315
ci: merge-queue readiness for the required gates; cache test results (fresh nightly) (#7264)
* ci: make the required gates merge-queue ready

The default-branch ruleset requires "CI Gate / Required" and "PR Risk
Gate / Required". Prepare both for a GitHub merge queue (ALLGREEN,
squash), so enabling the ruleset's merge_queue rule is the only step left.

- pr.yml, pr-risk.yml: merge_group pinned to types [checks_requested].
- D2: bazel-coverage covers every merge_group run (same repository, CI
  secrets, actor github-merge-queue[bot], bazel.yml mode remote), so the
  retired legacy tiers stay retired in the queue and ci-gate requires
  their Bazel lanes to have run remotely and passed; a missing secret or
  the farm switch off is red, as on a same-repo PR.
- Diff bases on merge_group: the package gates diff
  merge_group.base_sha..head_sha (not HEAD^, which misses all but the
  last commit of a rebase-method entry), the .beads/issues.jsonl guard
  now runs on merge_group against base_sha (ci-gate no longer excuses
  its skip), and check-doc-flags passes BD_DOCS_DIFF_BASE so CLI-docs
  drift already on main cannot eject every queued PR.
- Test result reuse: on a merge_group run and on attempt 2+ of a PR run,
  the six always-execute Bazel lanes append --config=reuse-test-results
  (--cache_test_results=auto), so tests whose action key is unchanged
  reuse the farm's recorded PASS and only what the merge changed runs.
  Failures are never reused, eviction retries stay 0, and each PR's first
  attempt, push to main and nightly still execute every test.
- scripts/ci_merge_queue_test.go simulates merge_group through the shared
  evalGHExpr evaluator (now with true/false/null literals, hyphenated
  identifiers and case-insensitive ==) and the workflows' own scripts:
  triggers and gate names, concurrency groups, every pull_request field
  read, the coverage and rbe decisions, result reuse, the package-gate,
  .beads and doc-drift diff bases.
- engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md: new "Merge Queue" section (how
  to queue, settings, what runs, result reuse rationale, ejection and
  flakes, rollback).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci: cache Bazel test results except nightly; queue trust and sizing

Review follow-ups to the merge-queue readiness change.

- Test results (S1): --nocache_test_results also stops Bazel writing
  results, so a PR's first run never fed the cache and the merge-queue
  reuse config could only hit on later reuse runs. Drop it from
  test:embedded, test:doltserver-proxied, test:doltserver-integration and
  test:sole-run (which keeps --experimental_remote_cache_eviction_retries=0),
  and drop test:reuse-test-results and BAZEL_REUSE_RESULTS. Results are now
  cached (read and write) on PRs, merge groups and pushes to main.
  nightly.yml passes the new fresh-test-results input, so every `bazel
  test` appends --config=fresh (--nocache_test_results) and the suite
  re-executes nightly. The docker lane keeps --nocache_test_results.
  Measured locally with a disk cache: a default run writes and the next
  reads ("(cached) PASSED"); a fresh run neither reads nor writes.
- Security headers (S2): bazel.yml now states that a merge_group run
  executes the queued commit's workflows, so a queued fork or Dependabot
  PR's CI edits run with every CI secret, the pre-warm app key included
  (merging-level trust), and records the maintainer rule: review fork
  changes to .github/, .bazelrc, tools/bazel, MODULE.bazel, BUILD files
  and CI scripts before queueing. Same note in the topology doc.
- Queue sizing (S3): recommend max_entries_to_build 3 and
  check_response_timeout_minutes 60 (max_entries_to_merge 5, ALLGREEN,
  SQUASH unchanged).
- Nits: the .beads guard fails closed on a missing diff base, and its
  test runs under GitHub's default `bash -e`; evalGHExpr follows GitHub's
  loose equality (true == 'true' is false, false == '' is true, missing
  context keys are null) with a typed-context variant; the
  detect-package-gates comment notes the SQUASH/HEAD^ equivalence and
  path-filter scope; the doc covers re-run reuse, flake re-sampling via
  dispatch, stale queue runs and how to confirm reuse on the farm.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 13:54:15 -07:00
Julian Knutsen
4493eb8d23
ci(pr): run same-repo macOS preflight/doc-freshness legs on Blacksmith macOS (#7231)
pr-preflight-platforms and check-doc-freshness-platforms now resolve their
same-repo-macos marker like the Linux/Windows legs: blacksmith-6vcpu-macos-26
for same-repo PRs and merge_group, GitHub-hosted macos-latest for forks,
Dependabot and every other event. The label is pinned to macos-26 because
GitHub's macos-latest is macos-26-arm64 today, so both paths run the same
OS/arch; 6 vCPU is already 2x macos-latest's 3 vCPU.

Blacksmith cannot see GitHub-saved caches, so main.yml gains
blacksmith-macos-go-build-cache: a push-to-main-only saver on the same label
(same job guard as the other Blacksmith savers) that builds ./cmd/bd and runs
the shared warm-non-race-cache.sh, saving the module cache and a non-race
GOCACHE keyed by go.sum + UTC day. The PR legs stay restore-only. main.yml's
GitHub-hosted macOS full-suite leg is unchanged and keeps seeding the fork
path.

Policy tests: the macOS marker arm is evaluated for same-repo/merge_group/
fork/deleted-head/dependabot/push/schedule and must match the flat macOS
const; the saver joins the PR-guard and per-day-key sweeps and the cache
ownership inventory; TestBlacksmithMacOSSaverMatchesPRLegs pins the
label/path/key/warm-up contract between saver and PR legs.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 13:19:49 -07:00
Julian Knutsen
630eae9b0c
ci(pr): share restored GOCACHE with the native init credential step (#7227)
pr-preflight-platforms' "Exercise native init gateway credential shell"
step had no GOCACHE env, so it fell back to the runner's empty default
cache and cold-compiled all of ./cmd/bd's test binary on every PR, right
after the hook-timeout step had compiled the identical binary (same tags,
same platform-default CGO) into the restored non-race cache. Job logs:
213s of a ~350s macos-latest leg, ~117s on Windows; on Linux the rerun
with a shared cache is a relink only (~8s).

- pr.yml: point that step, and the three Windows-only Go steps, at the
  restored non-race GOCACHE.
- main.yml: the macOS test leg (the only macOS non-race seeder) now runs
  warm-non-race-cache.sh, so the PR macOS legs restore the test packages
  and test-only deps they compile, not just the non-test ./cmd/bd graph.
- Policy tests: every run step in pr-preflight-platforms and
  check-doc-freshness-platforms must use the restored GOCACHE, and the
  macOS seeder must warm before it saves.

No test selection, matrix, or gate wiring changes.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 01:01:23 -07:00
Julian Knutsen
683993660f
fix(proxy): fail fast with a clear MySQL error when an external upstream is unreachable (#7181)
* fix(proxy): fail fast with a MySQL error when the external upstream is unreachable

A proxied-server command against a stopped external Dolt upstream stalled
~20-30s before failing with a bare "invalid connection". The local db proxy
stays up, so on a refused backend dial (or a front such as stunnel/socat
that accepts and closes because its own upstream is gone) it simply closed
the client connection. The MySQL driver reports that as invalid
connection / unexpected EOF, which the uow bootstrap ping (#6003) classifies
as a transient drop and retries with exponential backoff for its whole 30s
budget. Retrying cannot help: the proxy already proved the upstream at its
own startup, and the outage is the upstream's.

The proxy now answers such a connection itself with a MySQL ERR packet
(2003, HY000) in place of the server greeting, naming the upstream and the
cause, exactly as a real server refuses a connection before the handshake.
The driver surfaces it as a *mysql.MySQLError, which the bootstrap retry
already treats as permanent, so the command fails in ~0.1s with an
actionable message. Only shapes that prove the upstream is not serving
qualify: ECONNREFUSED/ENOENT/EHOSTUNREACH/ENETUNREACH on dial, or a backend
that reaches EOF having sent zero bytes (a MySQL server always speaks
first). A dial timeout or a drop after the greeting keeps the plain close
and the client's transient retry.

External backends only: a managed (local sidecar) backend is owned by the
proxy and its dial can be refused transiently while bd drives it (the
full proxied lane caught TestProxiedServerInitExplicitBeadsDirIsAuthoritative
hitting exactly that mid-init when this applied to every backend), so
managed backends keep the bare close and the client's bootstrap retry.
TestExternalProxyOutageAndRestoreReDialsTCPAndUnix asserted the old
silent close during an outage; it now asserts the ERR packet.

With the latency fixed, TestProxiedOutageReconnectAcceptanceMatrix is
renamed into the proxied discovery pattern as
TestProxiedServerOutageReconnectAcceptanceMatrix and placed in the Bazel
30-shard block by gen_proxied_shard_manifest.py's incremental --write
(shard 8). The frozen legacy 15-shard block is untouched; the test
hash-distributes there like every other test added since it was frozen.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(proxy,uow): retry the proxy's upstream-outage report for ~1s; hedge the zero-byte case

Review follow-up to the fail-fast change.

A zero-byte close from the upstream is not proof it is down: Dolt closes
queued connections without a byte when max_connections/back_log are full,
when a queued connection times out, and while it shuts down for a
restart; an L4 front does the same while its target is replaced. Its
report now says "closed the connection before the MySQL greeting (down,
restarting, or at its connection limit)" instead of "unreachable", and
the comment no longer claims proof.

Refusals and endpoint churn also lost #6003's retry entirely, so an
external Dolt restart or a Service with no ready endpoints failed a
command at once. The client now retries the proxy's report
(proxy.IsUpstreamOutageError: 2003 with the "beads db proxy: " prefix)
for upstreamOutageRetryWindow (1s) from the first time it sees it, with
the backoff clamped so the last attempt lands at the window's end; other
transient errors keep the 30s budget. A real outage still fails in about
a second, inside the acceptance test's 3s bound.

Also: the dial report drops Go call-site wrapping ("dial tcp H:P:
connection refused"), truncation keeps UTF-8 intact, the comment states
how the packet differs from a real server's pre-handshake ERR (keeps the
'#'+SQLSTATE marker for vitess; 2003 is a client code chosen on purpose),
a Stats counter for reported outages, and the isTransientPingError
paragraph is reflowed and corrected.

Tests: pingWithRetry retries the report inside the window and gives up
at its end without using the full budget; openDB against a front that
plays the proxy (its real packet) and comes back 300ms after the first
connection succeeds in one call, and fails in about the window when it
never does; NewProxyServer enables the report for external backends
only; the dial message carries no Go type names; rune-safe truncation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(proxied): fail, not skip, socat-bridged subtests in the Bazel lane

The external-unix front-door matrix subtest and both outage/reconnect
matrix subtests bridge an external endpoint with socat and skipped when
it was missing, so they could stop running without anyone noticing (the
parent test still passes). requireSocat now fails them when
BEADS_TEST_REQUIRE_SOCAT=1, which //cmd/bd:bd_proxied_test sets (its
remote workers have socat; a recent main run's external-unix subtest
passed there). The legacy GitHub 15-shard jobs have no socat and leave
it unset, so the subtests keep skipping there. The check now runs before
the upstream Dolt server is started, so a skip no longer pays for one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(proxied): measured weight for the outage matrix; pin it to legacy shard 1

Give TestProxiedServerOutageReconnectAcceptanceMatrix a duration entry
(80s: local runs measured 55-79s alone, 60-70s in its shard, and review
68-109s under load) and re-place it with the generator's incremental
--write: it moves from 30/8 to 30/11, one line, nothing else moved. The
heaviest estimated shard is now 435s against a ~357s median; a repack
would even that out but moves most assignments, so it is left to a
separate CI-only change that refreshes all durations together.

Legacy 15-shard block: the test would hash onto shard 14, already ~13.5
of the 15 min timeout on main. It is pinned to shard 1 (~6.5 min in
recent main runs) under the block's reviewed-pin rule; it skips there
today (no socat), and the pin keeps shard 14 safe if that changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 02:37:58 -07:00
Julian Knutsen
b1a0690249
ci: pin four hash-placed proxied tests off overloaded legacy shards 5 and 3 (#7172)
Main's "Test (proxied-server cmd 5/15)" has hit the 15m -test.timeout in
6 of the 9 Main runs since #7129 where the shard ran to completion.
That PR renamed 12 proxied tests into TestProxiedServer* discovery.
They are not in the frozen 15-shard legacy block, so they hash-place,
and two landed on shard 5:
TestProxiedServerStorageClass (nine parallel inits, the heaviest of the
12) and TestProxiedServerServeRefusesAStaleCheckpointWithGone. Shard 5
went from 430-620s of job time to 800-950s. In those runs the timeout
fired while TestProxiedServerMolReadyGated's parallel subtests were still
queued behind the rest of the shard. That test is the victim, not a
pathological one. Shard 3 (also hash-loaded) runs 810-870s, within about
a minute of the same limit.

Measured locally (4 pinned CPUs, GOMAXPROCS=4), shard 5 took 897s with
its current test set and 524s with the pre-#7129 set.

Pin those two tests, plus shard 3's two heaviest hash-placed tests
(TestProxiedServerListComments and
TestProxiedServerServeRefusesAStreamOnADisabledJournal), onto the legacy
shards with the lowest measured job time: 14, 7, 4 and 1. Every test
already listed in the block stays where it is, so shard 5 is back to its
pre-#7129 test set. pr-risk.yml's fork jobs read the same block, so they
get the same relief. The script's 15m timeout, the job and step
timeouts, and the Bazel 30-shard block are unchanged. The block header
and the generator's legacy header now say a reviewed pin is allowed.

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 00:15:08 -07:00
Julian Knutsen
4b2852ce21
fix: macOS main job — /private/var symlink in test repo pin; bash 3.2-safe CI scripts (#7165)
* fix(git): canonicalize PinNoRepositoryUnderForTesting root so the pin matches on macOS

On macOS t.TempDir()/os.TempDir() live under /var/folders/..., but /var is
a symlink to /private/var and getcwd(2) reports /private/var/folders/....
underPinnedRootForTesting compared the raw pin against os.Getwd() with a
string prefix test, so the pin never matched there and
TestPinNoRepositoryUnderForTestingSurvivesResetCaches failed on every Main
macOS run (GetRepoRoot() returned the /private/var repo instead of "").

Store the pin both as given (absolute) and symlink-resolved, and match the
working directory in both raw and resolved form. Canonicalization can only
widen the fence, never narrow it (e.g. Bazel runfiles symlink trees keep
matching exactly as before).

TestPinNoRepositoryUnderForTestingMatchesThroughSymlinks reproduces the
macOS shape on any platform with a symlinked repo root; it fails without
this change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): make run-go-test-binary.sh and install-golangci-lint.sh parse under bash 3.2

The Main macOS job runs the scripts package tests with macOS's /bin/bash
3.2, where two constructs abort under `set -u`:

- run-go-test-binary.sh expanded the usually-empty PASSTHROUGH array bare;
  bash < 4.4 treats that as an unbound variable ("PASSTHROUGH[*]: unbound
  variable"), failing TestTestScriptPrebuiltTestBinaryContract and
  TestRunGoTestBinaryDefaultsTimeoutWhenOmitted. Use the ${arr[@]+...}
  guard.
- install-golangci-lint.sh used `declare -A`; bash 3.2 parses it as an
  indexed array and `[amd64]=` dies as an arithmetic lookup ("amd64:
  unbound variable"), failing
  TestInstallGolangciLintFailsClosedOnChecksumMismatch. Replace it with a
  case function and update TestGolangciLintInstallScriptPinned's pin check.

Reproduced and verified against a bash 3.2.57 build: all four tests fail
on origin/main and pass with this change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 23:08:51 -07:00
Julian Knutsen
1137aeb70d
ci(bazel): embedded-Dolt tier 20/5 → 50/15 duration-balanced Bazel shards (F1) (#7156)
* ci(bazel): embedded-Dolt tier 20/5 -> 40/14 duration-balanced shards (F1)

Give the Bazel-only bazel-embedded job its own duration-balanced manifest
blocks for //cmd/bd:bd_embedded_test (20 -> 40 shards) and
//internal/storage/embeddeddolt:embeddeddolt_embedded_test (5 -> 14
shards), decoupled from PR Risk's/main.yml's legacy frozen 20/5-shard
fork/push jobs, mirroring F2's bd_proxied_test precedent.

- Measured every TestEmbedded* / embeddeddolt test's wall time in
  isolation; split 21 cmd-tier and 1 storage-tier single top-level test
  over 180s into disjoint sub-tests (3 irreducible exceptions left:
  TestEmbeddedUpdateConcurrent 475s, TestEmbeddedCloseConcurrent 308s,
  TestLargeBatchApplyStatementCounts712_Embedded 261s).
- New scripts/ci/gen_embedded_{cmd,storage}_shard_manifest.py +
  _embedded_shard_manifest_lib.py generate/check duration-balanced
  manifest blocks (LPT packing, incremental --write/--repack, exact-once
  --check), chosen at N=40/M=14 where the heaviest shard reaches the
  irreducible-test floor.
- Fixed 18 stale legacy 20-shard cmd manifest entries (and 1 storage
  entry) left over from earlier test splits, which would have hard-failed
  every shard of PR Risk's real test-embedded-cmd jobs on their next run.
- New scripts/embedded_shard_count_test.go policy tests tie the
  Bazel-only block's shard count to bazel.yml's check_shard_coverage.py
  argument and catch generator staleness; generalized
  TestBazelRetiredLanesCheckListedTestsRan's live-shard-count dispatch
  from bd_proxied_test-only to a map covering both embedded targets;
  removed TestBazelEmbeddedJobMirrorsEmbeddedTier's now-invalid
  BUILD.bazel-shard-count-equals-PR-Risk-matrix assertion.
- Verified: go test ./scripts/..., bazel test //scripts:scripts_test,
  actionlint, make ci-pr-policy, make bazel-sync-check all pass; full
  local bazel test of both targets at 40/14 shards passes 2/2; mutation
  testing confirms each new/changed policy test fails exactly when it
  should.

See ~/beads-bazel-plan/f1/impl-report.md for full detail.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(embedded-dolt): skip 712-item statement-count assertion under -race (F1)

TestLargeBatchApplyStatementCounts712_Embedded genuinely costs ~260-300s
under -race (confirmed by solo re-measurement: 298.60s) -- 14014 real SQL
statement round-trips through the race-instrumented in-process Dolt engine,
not an artifact and not splittable (one indivisible ApplyBatchInTx
transaction). It was the storage tier's single-test floor for CI shard
packing.

Apply the exact precedent TestLargeBatchApplyWallClock_Embedded already set
for its 1000-item shape: skip this assertion under raceEnabled (the cost is
race-instrumentation overhead on the Dolt engine's own internal
goroutine/lock machinery, not this test's own logic) and extend nightly.yml's
"Embedded Dolt batch-apply suite (non-race)" step's -run regex so the full
pinned 14014-statement baseline still runs, non-race, every night. Verified
directly: under -race the assertion now skips in 0.00s; without -race it
still runs and asserts Total=14015 (within the existing +/-2 tolerance) in
36.76s.

Updates scripts/ci_workflow_test.go's
TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite to match the extended
regex.

Storage tier's new single-test ceiling under the per-PR -race lane drops
from ~260.7s to 152.38s (TestBatchApplyContract).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* ci(bazel): correct cmd-tier long-pole durations, re-pack embedded manifolds (F1)

Investigated coordinator item 2 (single-test long poles): TestEmbeddedUpdateConcurrent
(recorded 475.39s) and TestEmbeddedCloseConcurrent (recorded 308.06s), plus
the two other >180s cmd-tier entries from the same capture
(TestEmbeddedInitA 196.94s, TestEmbeddedEpicCloseEligibleReason 192.47s).

Root cause: NOT host contention, NOT backoff/retry sleeps (bdRunWithFlockRetry
contributes negligibly), NOT race-instrumented-subprocess overhead alone.
These numbers were captured via `bazel --nohome_rc test --config=embedded`
with the then-current 20-shard legacy manifest (~10-11 tests/shard) and
`-test.parallel=4`. Go's t.Parallel() starts each parallel test's elapsed
clock when its function is invoked, before the Parallel() call blocks it
waiting for a free one-of-4 slot -- so a test queued behind several other
heavy shard-mates has that queueing wait baked into its own recorded
`--- PASS: TestX (Ys)` time. This is shard-composition-dependent scheduling
delay, not a cost intrinsic to the test.

Confirmed two ways in an isolated clone (not this worktree, per the
no-plain-go-test-from-a-beads-checkout rule):
- Solo (no rivals at all): 151.83s / 106.48s / 63.58s / 66.08s.
- All four together with the real -race -test.parallel=4 flags and exactly
  4 tests requested (so nobody queues -- parallel slots == test count):
  141.34s / 113.80s / 69.79s / 74.63s.
Both agree closely and both are far below the capture-time numbers.

F1's LPT packer already isolates the heaviest tests into shards with few or
no other heavy rivals (TestEmbeddedUpdateConcurrent lands alone in its own
40-shard-block shard), so under the new manifest these tests see little to
no queueing delay in real CI. None of the four needed splitting, parallel
subtests, or raceEnabled-scaled N: once correctly measured, none exceeds the
180s single-test guideline (new corrected weights: 155.0 / 115.0 / 72.0 /
77.0s). Corrected scripts/ci/embedded_cmd_test_durations.json accordingly
(with a doc-comment explaining the methodology and both confirmations) and
re-packed the 40-shard Bazel-only block (--repack --write); --check passes
(exact-once coverage of all 241 discovered tests). The frozen legacy 20-shard
block is untouched and independently re-verified exact-once by simulating
embedded-test-shard.sh across all 20 shards (241/241 tests, no
duplicates/misses/errors).

New cmd-tier single-test ceiling: 179.47s (TestEmbeddedCloseAlreadyClosed,
likely itself somewhat inflated by the same mechanism but already under the
180s line as recorded, so no action needed).

Also updates scripts/ci/embedded_storage_test_durations.json: the prior
commit's raceEnabled-skip for TestLargeBatchApplyStatementCounts712_Embedded
means it now costs ~0s in the per-PR -race lane bazel-embedded actually
packs for (not its real ~260-300s non-race cost, which nightly.yml still
exercises) -- weight set to 1.0 with a doc comment tying it to the skip in
large_batch_apply_measure_test.go. Re-packed the 14-shard Bazel-only block
accordingly; --check passes (exact-once coverage of all 387 discovered
tests, excluding TestMain/TestConformance per that file's existing
convention). The frozen legacy 5-shard block is untouched and independently
re-verified exact-once via embedded-storage-test-shard.sh simulation
(388/388 tests, no duplicates/misses/errors).

Verification: go test ./scripts/... (includes the nightly.yml policy test),
make ci-pr-policy, make bazel-sync-check, actionlint all pass. Real local
`bazel --nohome_rc test --config=embedded //cmd/bd:bd_embedded_test
//internal/storage/embeddeddolt:embeddeddolt_embedded_test`
(--local_test_jobs=8, both targets together, on a shared/noisy sandbox host)
passes both targets; per-shard wall time from each shard's test.xml: cmd/bd
max 218.9s (shard 23/40, min 20.6s), storage max 251.1s (shard 5/14, min
0.98s) -- down from the prior oversubscribed sweep's 783.41s / 351.94s and
well under the single-test-floor-based ~475s projection this slice started
from. Projected tier lane time (max of the two targets, real dedicated CI
runners should do at least this well): ~251s, under the 5-minute target.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(embedded): un-vacuous TestEmbeddedDepB edges, split epic-close test, trim concurrency stress N (F1 review)

Coordinator review follow-ups on the F1 embedded-Dolt resharding slice:

- TestEmbeddedDepB (one of TestEmbeddedDep's two CI-speed split halves) was
  asserting its tree_* subtests against an edgeless graph, passing
  vacuously; restores the same parent-child/blocks edges TestEmbeddedDepA
  builds so those subtests exercise real multi-node traversal (review S2).
- TestEmbeddedEpicCloseEligibleReason split into
  TestEmbeddedEpicCloseEligibleReasonFlag/...Output for CI shard balance,
  matching the project's established single-concern split pattern.
- TestEmbeddedUpdateConcurrent/TestEmbeddedCloseConcurrent's
  issuesPerWorker reduced 5->2: under real concurrent-shard-load
  measurement these were genuine (not queueing-artifact) serialized-write
  costs; 2 preserves every invariant the test checks (duplicate-ID
  detection across all workers, one non-decreasing-list-count check per
  worker) at a fraction of the serialized work.
- large712ShapeName: the 712-shape's race skip and its zero-match guard now
  share one named constant instead of two independent literal shape-name
  strings, so a future rename of the shape can't silently skip the pinned
  statement-count assertion under race (review N2).
- Comment-only fixes (review N5) in files referencing test names since
  renamed/split by this slice's earlier work: close_proxied_integration_test.go,
  graph_apply_test.go, storage_class_proxied_integration_test.go,
  tests/regression/discovery_test.go, plus doc-comment path cleanups (N1:
  drop unverified ~/beads-bazel-plan paths/bead ids in favor of the
  committed duration JSON files and engdocs/TESTING.md).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* ci(bazel): re-derive embedded-Dolt durations under concurrent load; cap cmd shards at Bazel's 50-shard limit (F1 round 2)

Coordinator review required re-deriving every per-test duration weight from
a single, consistent, CI-realistic measurement (all shards running
concurrently, -test.parallel=4, race-enabled) instead of the prior pass's
mix of crowded-shard captures and solo/4-together spot-corrections, and
required measuring per-shard wall time via BEP testAttemptDuration rather
than a shard's outer test.xml <testsuites time> (which can include a
different, already-finished shard-mate's setup/teardown).

- scripts/ci/embedded_cmd_test_durations.json (242 tests) and
  scripts/ci/embedded_storage_test_durations.json (387 tests) rewritten in
  full from one consistent sweep. Every entry is now a fresh measurement;
  none is a carried-over capture-time value or an unverified estimate. Both
  files' _comment blocks retract the earlier "queueing delay baked into a
  test's own reported elapsed time" theory (testing.go's (*T).Parallel does
  not add wait time to a blocked test's own clock) and document the real
  methodology.
- Only one cmd-tier test remains over 200s (TestEmbeddedListFiltering,
  239.70s), already under the ~240s ceiling; no further top-level splits
  were needed this round.
- New finding: Bazel hard-caps shard_count at 50 per sh_test-like rule
  ("Having more than 50 shards is indicative of poor test organization";
  a native analysis-time limit, not configurable). The previous 88-shard
  plan for //cmd/bd:bd_embedded_test (chosen as the LPT-packed floor before
  this limit was discovered) is reverted to 50, Bazel's real ceiling.
  Repacking the fresh weights at 50 shards gives a heaviest shard of
  409.6s -- above the single-test floor of 239.7s, because 50 bins is not
  enough headroom over the suite's ~20145 measured test-seconds to flatten
  every bin to that floor. This makes the <=5 min lane target unreachable
  for the cmd tier via resharding alone within Bazel's native per-rule
  limit; see ~/beads-bazel-plan/f1/impl-report.md section 3c for the full
  writeup and the two remaining levers (suite-wide speedups or a
  multi-Bazel-rule split), neither attempted in this slice.
- internal/storage/embeddeddolt gets 15 Bazel-only shards (unaffected by
  the 50-shard cap): LPT-packed heaviest shard 225.3s, within target.
- cmd/bd/BUILD.bazel, internal/storage/embeddeddolt/BUILD.bazel,
  .github/workflows/bazel.yml, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md,
  scripts/embedded_shard_count_test.go, scripts/pr_risk_bazel_coverage_test.go,
  scripts/ci/gen_embedded_cmd_shard_manifest.py updated to the corrected
  50/15 shard counts (and, for the generator, N3: --write without an
  explicit total_shards now errors instead of silently regenerating the
  frozen legacy 20-shard block).
- .github/scripts/embedded-{cmd,storage}-test-shards.txt: re-packed
  Bazel-only blocks at 50/15; frozen legacy 20/5 blocks untouched except for
  N5 fixes to stale entries left by this slice's earlier test splits
  (TestShardScriptsListOnlyRealTests catches this class of bug; both
  legacy blocks reconfirmed exact-once).
- .github/workflows/nightly.yml / scripts/ci_workflow_test.go (N4): the
  embedded-dolt batch-apply nightly step now runs with `if: ${{
  !cancelled() }}` and its own explicit `-timeout 20m`, independent of the
  job-level timeout, with both halves now asserted by
  TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite.

Verified: gofmt -l clean, go vet ./scripts/... clean, go test ./scripts/...
green, both generators' --check pass at 50/15 (and legacy 20/5 remain
exact-once via the frozen-block invariant).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(ci): rename embedded manifest staleness tests out of the ^TestEmbedded skip

The allowlisted go-test leg runs with -skip ^TestEmbedded, so tests named
TestEmbedded*ShardManifestGeneratorNotStale never ran there and the leg
failed with "matched no test that ran".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-03 21:30:03 -07:00
Julian Knutsen
84ce244857
ci: same-repo Linux/Windows PR jobs on Blacksmith with main-seeded warm build caches (F7b) (#7155)
* ci: move same-repo PR Linux/Windows/macOS legs to Blacksmith, shard preflight (F7b)

PR-path jobs for same-repo PRs/merge_group now run on Blacksmith instead
of GitHub-hosted runners, following F4/F7a's ternary-expression and
restore-only-cache patterns; forks/Dependabot are unaffected:

- scripts-go-checks, pr-lint-wrapper: same-repo Blacksmith 4vcpu Linux.
- check-doc-freshness-platforms, pr-preflight-platforms: restructured
  from a top-level matrix.os list to an include:-only "matrix marker"
  shape (runner: same-repo-linux/-windows/-macos) so each host can
  resolve an independent same-repo Blacksmith selection via one chained
  ternary runs-on expression.
- Six standalone Windows jobs (test-windows-liveness,
  worktree-remove-windows, test-windows-small,
  test-windows-liveness-prebuilt, worktree-remove-windows-prebuilt,
  windows-make-shell) move to same-repo Blacksmith Windows
  (blacksmith-{2,4}vcpu-windows-2025, public beta). CGO/mingw-w64 and
  Chocolatey preinstall are unconfirmed on that image; both risks are
  flagged with one-line rollback instructions at the call sites.
- pr-preflight-platforms' Windows leg is split into two shards (common
  steps on shard 1, the three Windows-only heavy steps on shard 2) to
  shorten its critical path; shard 2 downloads Go modules directly
  instead of sharing shard 1's runner/cache.

main.yml gains cache savers for the newly same-repo-Blacksmith PR
consumers, since Blacksmith and GitHub-hosted runners cannot see each
other's actions/cache entries even for identical keys:

- blacksmith-go-build-cache warms the Blacksmith-selection race/non-race
  GOCACHEs scripts-go-checks and pr-preflight-platforms/check-doc-
  freshness-platforms' ubuntu legs need, push-to-main only. Its non-race
  leg shares scripts/ci/warm-non-race-cache.sh with build-artifacts, and
  compiles tools/bazel/run_allowlisted_go_tests.py's allowlisted
  packages via its new --compile-only flag, so the two paths and the
  warm-up can never drift from what they actually run.
- pr-lint-wrapper, go-vet-cache, test-windows gain a venue: [blacksmith,
  github] matrix leg to seed both runner selections pr.yml's
  same-repo-Blacksmith-or-fork-hosted consumers can land on.

Updates the policy-test suite (ci_workflow_test.go,
ci_blacksmith_runner_test.go, pr_lanes_bazel_coverage_test.go) to cover
the new runner selections, matrix-marker shape, shard gating, and cache
topology, including a dedicated inventory/writer check for
blacksmith-go-build-cache.

Deferred follow-up (not done here): sharding check-doc-freshness-
platforms and the standalone Windows jobs, and moving/sharding macOS
legs beyond what check-doc-freshness-platforms'/pr-preflight-platforms'
matrix-marker form already covers.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci): F7b review response -- msys2/Blacksmith-saver guards, macOS stays hosted (B1/B2/S3)

Addresses ~/beads-bazel-plan/f7b/review.md's two cache-safety blockers and
one follow-up gap found after F7b moved same-repo PR Linux/Windows/macOS
legs onto Blacksmith and added push-to-main Blacksmith cache seeders:

- B1: windows-make-shell's msys2/setup-msys2 step caches its installed
  packages by default, which implicitly SAVED a GitHub Actions cache entry
  from a same-repo PR the moment it ran on a Blacksmith runner (confirmed
  via a real cache-list entry from a refs/pull/.../merge ref). Disabled with
  a fail-closed `cache: ${{ runner.environment == 'github-hosted' }}`, and
  generalized the sweep (TestBlacksmithReachablePRJobsDisableDefaultCachingActions)
  to cover any default-caching setup-* action on a Blacksmith-reachable job,
  not just this one.

- B2: blacksmith-go-build-cache, pr-lint-wrapper, go-vet-cache and
  test-windows (main.yml) had no (or an incomplete) same-repo push-to-main
  guard, so a same-repo fork with push access to its own main could run --
  and save a cache entry from -- a paid Blacksmith job. Added the exact
  push-to-main/same-repo `if:` F7c's own blacksmith-setup-go-cache already
  uses, pinned by TestBlacksmithSaverJobsGuardedAgainstPullRequest.

- S3: the macOS leg of pr-preflight-platforms/check-doc-freshness-platforms
  resolved to Blacksmith for same-repo PRs; Blacksmith bills macOS at
  roughly 20x its Linux/Windows rate and no push-to-main Blacksmith-macOS
  saver exists, so that leg would pay the premium for a guaranteed cold
  cache. macOS now stays unconditionally on GitHub-hosted macos-latest.

Also pins S7 (blacksmith-go-build-cache/go-vet-cache/test-windows's cache
keys are bounded by go.sum+UTC-day, not github.sha) and S6/S8 (their
20-minute timeouts), both already in place but previously unpinned, and
replaces run_allowlisted_go_tests.py's fragile GO_TEST_FLAGS[-2:] slice with
an explicit BUILD_FLAGS constant (N1).

Re-ran the reviewer's 15 mutations (~/beads-bazel-plan/f7b/review/mutate.py):
all now killed or structurally moot (the Windows preflight shard split this
review asked to revert-unless-helpful was reverted, in an earlier commit on
this branch, removing the vulnerability surface 6 of them targeted).

scripts/BUILD.bazel: add the new ci_f7b_review_test.go to scripts_test's
srcs -- bazel test caught it missing (file existed on disk but was not
compiled into the Bazel-built test binary).

Verification: go vet/test/build, gofmt, bazel --nohome_rc test
//scripts:scripts_test, actionlint -shellcheck, shellcheck, make
ci-pr-policy, make bazel-sync-check all pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(ci): drop local review-doc path from comment

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-03 17:58:31 -07:00
Julian Knutsen
fac9a1e32c
ci: same-repo Linux PR jobs on Blacksmith; fold tiny checks and cross-compile legs (F7a) (#7139)
* ci: F7a same-repo Blacksmith moves and job folds (pr.yml, pr-risk.yml)

Move cache-free same-repo pr.yml/pr-risk.yml jobs onto the org's Blacksmith
runner pool (forks/Dependabot stay ubuntu-latest), and fold several
independent single-purpose jobs into fewer multi-step jobs, to cut same-repo
PR queue time without changing either required gate's name or its token
contract:

- fast-checks: folds check-build-tags, check-version-consistency,
  check-migration-hygiene, check-no-beads-changes and fmt-check into one
  Blacksmith job with five independent steps; ci-gate reads each step's
  outcome via job outputs into the same CI_GATE_REQUIRED token the standalone
  job used, plus a FAST_CHECKS backstop token for a lost job.
- check-release-target-cross-compilation: 8 per-target matrix legs collapse
  to 2 (unix/desktop), driven by the new scripts/ci/release-targets.txt
  manifest (kept in sync with .goreleaser.yml) and
  scripts/ci/check-release-cross-compile.sh.
- advisory-reports: folds build-examples and complexity-report (both
  already advisory) into one job, keeping their pre-fold
  step-level continue-on-error/timeout design.
- windows-make-shell: 3-way host matrix folds into one job with an
  aggregator step reproducing the old native/MSYS2/Cygwin pass/fail rules;
  test-windows-doltversion and test-windows-dbproxy-server fold into
  test-windows-small (neither was gated).
- Runner moves: fast-checks, advisory-reports,
  check-release-target-cross-compilation, check-doc-flags,
  pr-policy-wrapper, test-dolt-server-fingerprint (pr.yml) and test-nix
  (pr-risk.yml) move to the same-repo Blacksmith expression at their
  measured vCPU size.

Policy tests: extend TestSameRepoBlacksmithRunners for the moved/added jobs,
add TestSameRepoBlacksmithExpressionSemantics, TestBlacksmithJobsReadNoSecrets
and TestPRCIGateFastChecksTokens, retarget the complexity/build-examples and
dolt-server-fingerprint tests, rewrite
TestReleaseTargetCrossCompilationMatrixMatchesGoreleaser against the new
manifest, and teach the ci-gate simulation helper to resolve
needs.fast-checks.outputs.*. Document the moves/folds and the Blacksmith-off
revert set in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci): windows-make-shell/test-windows-small fail natively, raise advisory-reports headroom (review SF-1, SF-2, N-2, N-3)

- windows-make-shell: drop continue-on-error from install/exercise steps
  (kept only on the Cygwin setup step), delete the hand-written "Fail if a
  required Windows Make shell leg failed" aggregator, and delete the unused
  native/msys2/cygwin job outputs. A failing leg now reds the job via native
  GitHub Actions propagation instead of relying on an aggregator step that a
  mutation showed could silently undershoot (SF-1, N-3).
- test-windows-small: same fix - drop continue-on-error from doltversion/
  dbproxy-server steps and delete their aggregator step.
- Add timeout-minutes: 10 to the MSYS2 and Cygwin "Set up ... GNU Make"
  steps, so a hung installer download can no longer take out the whole job
  (SF-2).
- advisory-reports: raise timeout-minutes from 45 to 60 for more headroom
  above the sum of its step timeouts (N-2).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(ci): real GH-expression evaluator shared with F7c, behavioral xcompile test, secret/run-command coverage (review SF-3, SF-5, SF-6, N-1, N-4, N-6)

- New scripts/ci_blacksmith_runner_test.go: a real GitHub Actions expression
  evaluator (tokenizer + recursive-descent parser for ==, !=, &&, ||, !,
  parens, string literals, dotted lookups, with GitHub's short-circuit-
  returns-operand && / || semantics) that evaluates the REAL pinned
  sameRepoBlacksmith{2,4,8}vcpu expression strings from pr.yml/pr-risk.yml,
  replacing the old TestSameRepoBlacksmithExpressionSemantics' tautological
  hand-written Go mirror (SF-3). This file is the shared helper F7c's own
  TestSameRepoBlacksmithExpressionSemantics should rebase onto - see its
  doc comment for the "Public surface" (the three consts, evalGHExpr, and
  mustEvalGHRunsOn) (SF-5).
- ci_workflow_test.go: drop the old consts/test now living in the shared
  file; extend TestBlacksmithJobsReadNoSecrets to also scan job.If and
  step.If for secret references, not just env/with/run (N-4; container/
  services credentials and matrix-marker resolution stay deferred to F7b);
  pin fast-checks' build-tags step invoking all 3 scripts and the
  migration-hygiene BASE_SHA env, and add
  TestPRWindowsSmallDbproxyServerIsCGODisabled (N-6); add
  TestPRWindowsMakeShellLegsFailTheJob and
  TestPRWindowsMakeShellSetupStepsHaveTimeouts pinning the SF-1/SF-2 fixes.
- New scripts/check_release_cross_compile_test.go: behavioral test for
  check-release-cross-compile.sh using a fake `go` on PATH - asserts a
  failing target propagates as non-zero, every target in the group is still
  attempted, every invocation passes CGO_ENABLED=0 and the gms_pure_go tag,
  and an unknown group fails loudly with zero invocations (SF-6).
- check-release-cross-compile.sh: document the deliberate sequential-build
  (vs. the spec's 2-wide suggestion) choice and its revisit trigger (N-1).
- scripts/BUILD.bazel: add the two new test files to scripts_test's explicit
  srcs list (not a glob); verified against `make bazel-sync-check`.

All new/changed tests mutation-tested: each production-code mutation was
caught by the relevant test, then reverted via `cp` from a /tmp backup and
diff-verified byte-identical (never git stash / git checkout --).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(ci): fix stale check names and aggregator description in CI_REQUIRED_CHECK_TOPOLOGY.md (review N-7)

- Replace the stale "Check build-tag policy", "Check version consistency",
  "Check for .beads changes" and "Check formatting" entries under "Do not
  require these existing check names" with the actual post-fold name,
  "Fast checks (build tags, versions, migrations, beads diff, fmt)", and add
  the cross-compilation and windows-make-shell fold's actual check names.
- Rewrite the windows-make-shell fold description: it no longer has a
  hand-written aggregator step (deleted per SF-1 in the prior commit), and
  "the separate cygwin job" is now described correctly as a pre-fold matrix
  leg, not a job.
- Update the advisory-reports timeout mention from 45 to 60 minutes (N-2).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(ci): drop local design-doc path from topology doc

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-03 13:13:33 -07:00
Julian Knutsen
c1fc671fcf
ci: cross-build Windows test binaries on Linux (advisory side-by-side), drop PR macOS full suite (F4) (#7135)
* ci(F4): advisory Linux-cross-built Windows test binaries, macOS cache restore

Side-by-side rollout slice of the PR CI -> 5 min plan (F4):

- Add pr.yml `windows-test-binaries`: a Linux job that cross-compiles
  bd.exe and two go-test-c binaries (cgo and nocgo) for windows/amd64
  via mingw (CC/CXX=x86_64-w64-mingw32-*, matching release.yml's
  goreleaser setup), driven by a committed manifest
  (scripts/ci/windows-test-binaries.txt) and build script
  (scripts/ci/build-windows-test-binaries.sh). Artifacts are uploaded
  and consumed by two new advisory "-prebuilt" twins
  (test-windows-liveness-prebuilt, worktree-remove-windows-prebuilt)
  that run the identical test selection/flags/env (including
  BEADS_TEST_REPO_ROOT) against the prebuilt binaries via
  scripts/ci/run-go-test-binary.sh and scripts/test.sh's new
  BEADS_TEST_PREBUILT_TEST_BINARY short-circuit.
- The original native Windows jobs are untouched and remain required;
  the new jobs are advisory only (not in CI_GATE_REQUIRED) for this
  rollout phase. Flipping to the cross-built path later is a one-line
  CI_GATE_REQUIRED change.
- Add main.yml `windows-test-binaries-cache`: seeds the Go build cache
  for the cross-compile toolchain/packages on a schedule, mirroring
  the existing module/build cache warmers; reorder build-artifacts'
  warm-cache step to precede the module-cache save for a cleaner
  build -> warm -> save grouping.
- Restore macOS's native build-cache save step in main.yml (lost
  during an earlier job consolidation).
- Remove the non-required "Test (macos-latest)" full-suite job from
  PRs; the native macOS gate jobs still run, so required coverage is
  unaffected. This does drop a non-required pre-merge macOS race
  signal; see docs/notes for the accepted tradeoff.
- Add scripts/windows_test_binaries_manifest_test.go: policy tests
  pinning the manifest's three advisory-phase entries, pinning the
  mingw toolchain to release.yml's, checking consumer jobs only
  reference manifest-listed binaries, checking upload/download
  artifact names and `needs` match, checking the prebuilt jobs select
  exactly the same tests (-run literals) as their native counterparts,
  and checking none of the new jobs reference secrets (fork safety).
  All mutation-tested.
- Add two scripts/ci_workflow_apt_get_timeout_test.go budget entries
  for the new "Install mingw cross-compiler" steps.
- Fix two latent Bazel-only scripts_test failures surfaced while
  wiring the new test file into scripts/BUILD.bazel: materialize
  cmd/bd's sources via the existing //cmd/bd:go_srcs filegroup so
  sourceRepoRoot(t)-based package-directory checks work under Bazel's
  runfiles root, and strip Bazel's own TEST_TIMEOUT env var before
  exercising scripts/test.sh's hardcoded default in
  test_script_test.go (name collision with test.sh's own override
  knob).

Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint; shellcheck; make ci-pr-policy;
make bazel-sync-check; a from-scratch /tmp clone cross-build using
the exact job commands (cold 130.2s, warm/cache-hit 30.3s, PE32+
x86-64 validity via file/objdump, byte-identical SHA-256 across
cold/warm/traced runs); wine -test.list confirms
TestPrimeBinaryPortfolio/TestIsServerProbablyRunning* present in the
cgo binary and the full TestWorktreeRemove* family (including
Windows-specific variants) present in the nocgo binary.

Scoping note: only 2 of the spec's 4 consumer jobs (liveness,
worktree-remove) get prebuilt twins, and only 3 of 8 spec-listed
manifest binaries are built this slice; preflight, doc-freshness and
credential-fixture consumers are deferred as documented follow-up.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci): address F4 review findings (SF-1..SF-6 + nits)

Addresses the F4 review (~/beads-bazel-plan/f4/review.md): no blockers, six
substantive findings plus several nits.

- SF-1: main.yml's windows-test-binaries-cache job now seeds the Windows
  cross-compile cache from both runner selections pr.yml's consumer job can
  land on (Blacksmith for same-repo PRs/merge_group, ubuntu-latest for
  forks/Dependabot) via a 2-leg matrix with identical keys/paths, instead of
  a single ubuntu-latest leg Blacksmith runners can never see.
- SF-2: removed the two consumer-less "Warm non-race Go build cache
  (test-only packages)" steps from main.yml (build-artifacts, test macOS
  leg); they had no real consumer and cost ~270s on main's macOS leg for
  nothing.
- SF-3: scripts/ci/run-go-test-binary.sh now injects the same
  -test.timeout=10m default (when -timeout is omitted) and unconditional
  -test.paniconexit0 flag `go test` itself injects; added
  timeout-minutes to test-windows-liveness-prebuilt; added a direct
  regression test (TestRunGoTestBinaryDefaultsTimeoutWhenOmitted) for the
  omitted-timeout branch, which no path through scripts/test.sh can reach
  (test.sh always supplies its own -timeout default first).
- SF-4: scripts/test.sh's BEADS_TEST_PREBUILT_TEST_BINARY branch now runs
  run-go-test-binary.sh as a child process (capturing and re-exiting its
  status) instead of `exec`-ing it, so the EXIT trap (beads_test_env_cleanup)
  still runs and the per-run BEADS_TEST_ENV_ROOT temp dir (and, with
  BEADS_TEST_SHARED_SERVER=1, the shared dolt sql-server) is cleaned up
  instead of leaked/orphaned.
- SF-5: closed the five surviving mutations (cgo/nocgo swap between the
  Windows prebuilt twins, dropped -count=1, an unguarded twin in
  ci-gate.needs, a goreleaser compiler change, a PKGDIR change) with
  dedicated policy tests, and replaced the three-edits-per-job required/
  advisory toggle with a single committed WINDOWS_PREBUILT_REQUIRED flag
  (pr.yml top-level env) in the style of the existing BAZEL_RETIRES_LEGACY_*
  flags: both Windows pairs are now unconditionally in ci-gate.needs and
  always evaluated, and only which pair lands in CI_GATE_REQUIRED vs.
  CI_GATE_SKIPPED_OK is decided by the flag, in one place.
- SF-6: dropped the //cmd/bd:go_srcs data dep from scripts_test; the
  prebuilt-binary contract test now exercises ./scripts/ci (already a data
  dep of this target) instead of ./cmd/bd, since it never builds or runs the
  real package.
- Nits: removed dangling ~/beads-bazel-plan references and a nonexistent
  test name from committed comments; log the resolved mingw-w64 version
  when cross-compiling; removed a no-op shellcheck disable comment;
  documented why run-go-test-binary.sh's BEADS_TEST_REPO_ROOT re-export
  doesn't race beads_test_env_enter's unset; rewrote the pr.yml tombstone
  comment for the removed macOS PR job to state the actual, narrower
  tradeoff instead of an overclaiming "main.yml covers this" statement.

Verified: bazel --nohome_rc test //scripts:scripts_test passes; actionlint
clean on pr.yml/main.yml; shellcheck clean on all edited scripts (only
pre-existing, unrelated warnings remain elsewhere); make bazel-sync-check
and make ci-pr-policy both pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(scripts): isolate prebuilt-binary contract test from ambient hermetic env

TestTestScriptPrebuiltTestBinaryContract's "cleans up BEADS_TEST_ENV_ROOT
after the binary exits" subtest (added for F4 review finding SF-4) passed
locally but failed in CI's scripts-go-checks/scripts-test leg.

scripts/ci/scripts-go-test.sh (the CI driver for this package's `go test
./scripts/...` run) calls beads_test_env_enter in its own shell before
invoking go test, so BEADS_TEST_ENV_ACTIVE=1 and an outer
BEADS_TEST_ENV_ROOT are already present in this test binary's own
os.Environ(). runTestScriptPrebuiltBinary built the scripts/test.sh
subprocess's env from os.Environ() (minus TEST_TIMEOUT), so that ambient
state leaked through. Inside the subprocess, scripts/test.sh's own
beads_test_env_enter call (test.sh:17) saw the already-active guard and
no-opped: no new root, no new EXIT trap armed in that process - so the
fake binary observed the outer, still-alive root, and the subtest's
"directory is gone after test.sh exits" assertion failed, even though the
SF-4 fix (run the prebuilt binary as a child instead of exec'ing it) is
correct.

Every real production caller of scripts/test.sh's prebuilt path (pr.yml's
Windows liveness/worktree-remove prebuilt steps) runs it as the first
command of a fresh GitHub Actions `run:` step, never nested under an
already-entered hermetic env, so this nesting is purely an artifact of
this test binary's own ambient environment under scripts-go-test.sh, not
a real CI behavior gap.

Strip BEADS_TEST_ENV_ACTIVE/_ROOT/_DISABLE/_KEEP (alongside the existing
TEST_TIMEOUT filter) from the child env in runTestScriptPrebuiltBinary so
the subtest deterministically exercises scripts/test.sh's real top-level,
non-nested invocation regardless of what ambient hermetic-env state this
test binary itself happens to be running under.

Verified: reproduced the CI failure locally by exporting
BEADS_TEST_ENV_ACTIVE=1/BEADS_TEST_ENV_ROOT before `go test ./scripts/...`;
confirmed the fix resolves it; re-ran bazel test //scripts:scripts_test
and scripts/ci/scripts-go-test.sh directly, both pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-03 11:49:56 -07:00
Julian Knutsen
bc19f6e478
ci: parallel PR Lint matrix with pinned golangci-lint, fold duplicate Lint, split scripts-go-checks (F5) (#7127)
* ci(lint,scripts): fold duplicate Lint job; matrix PR Lint and scripts-go-checks (F5)

Lint (golangci-lint-action, only-new-issues) was functionally identical to
pr-lint-wrapper's native leg (H7): same pinned binary, config, flags, and
diff scope. Fold it into the wrapper in both pr.yml and main.yml, and drop
LINT from pr.yml's ci-gate.

pr-lint-wrapper becomes a 3-leg matrix (native/windows/darwin, fail-fast:
false) under the same PR_LINT_WRAPPER gate id, installing golangci-lint
v2.10.1 from a pinned, sha256-verified release binary
(scripts/ci/install-golangci-lint.sh) instead of go install/the Action, with
a per-leg golangci-lint cache (pr.yml restores only; main.yml's matrix is
the sole saver). scripts-go-checks becomes a 3-leg matrix
(scripts-test/vet/allowlisted, fail-fast: false) under the same
SCRIPTS_GO_CHECKS gate id, giving go vet its own build cache instead of
sharing the race build cache. main.yml gains a go-vet-cache job to warm that
cache on push; build-artifacts remains the Go module cache's sole saver.

Splitting each job's previously-sequential passes into parallel matrix legs,
combined with removing the duplicate Lint job outright, takes wall-clock
time off the PR gate's critical path.

Updates scripts/ci_workflow_test.go and scripts/pr_lanes_bazel_coverage_test.go
for the new matrices, cache topology, and the removed LINT id; adds
TestGolangciLintInstallScriptPinned. Updates engdocs/LINTING.md and the live
prose in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md (its disclaimed "historical
snapshot" gate-wiring block is left as-is).

Verified: go test ./scripts/..., bazel --nohome_rc test //scripts:scripts_test
(after adding engdocs/LINTING.md and .githooks/pre-commit as scripts_test
data), actionlint, shellcheck, make ci-pr-policy, make bazel-sync-check.
Confirmed lint-leg parity against current main with the pinned binary (0
issues PR-scoped; the same two pre-existing unparam findings full-tree,
unchanged) and scripts-go-checks leg parity standalone. Mutation-tested the
new/updated policy assertions with six deliberate breaks (dropped matrix
leg, reintroduced LINT, corrupted the pinned sha256, removed a cache-save
step, dropped the vet leg, reintroduced the lint job); each was caught with
an on-target failure.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): review-fix pass on F5's lint/scripts-go-checks matrix (SF-1..SF-5, N-1..N-8)

Addresses all SHOULD-FIX and NIT findings from the review of 7e858e2714
(no BLOCKERs):

- SF-1: lint/vet cache saves now run on !cancelled() instead of bare
  cache-miss, so a red lint/vet run (main is currently lint-red on two
  pre-existing, unrelated unparam findings) still warms the cache for
  PR legs.
- SF-2: ci-measurements.yml now exports golangci-lint's install dir onto
  PATH in the same step, since $GITHUB_PATH only takes effect in later
  steps.
- SF-3: adds a behavioral test (fake curl/tar) proving the install
  script's sha256 verification fails closed.
- SF-4: drops the commit sha from the golangci-lint cache key (saves
  only on config/dependency change, not every push) and trims the
  windows/darwin legs to cache only the small golangci-lint result
  cache, not their large GOCACHE.
- SF-5: removes the vet cache's dead non-race-build-cache fallback
  restore key (different path list, can never hit) in both pr.yml and
  main.yml.
- N-1..N-8: doc/comment accuracy fixes, restores the "moved code reads
  as new" caveat, adds a module-cache restore step and timeout-minutes
  to pr-lint-wrapper/go-vet-cache, pins fetch-depth/GOCACHE via new
  policy-test assertions, and flags a superseded doc snapshot.

New commit on top of 7e858e2714; existing commit trailers are left
untouched per explicit instruction. Not pushed, no PR opened.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): auto-sync BUILD files

Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37105481015
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-03 09:36:20 -07:00
Julian Knutsen
6e9a15507e
ci(bazel): proxied-server tier 15 → 30 duration-balanced shards; un-hide 12 proxied tests (F2) (#7129)
* ci(bazel): raise proxied-server tier from 15 to 30 shards (F2)

bazel.yml's bazel-proxied lane (//cmd/bd:bd_proxied_test, run only with
remote execution) was sharing PR Risk's legacy 15-shard, frozen
bd-init-cost-proxy manifest block, so its longest shard was the long
pole of the Bazel-only lanes. Give it its own 30-shard block in
.github/scripts/proxied-cmd-test-shards.txt, bin-packed
longest-processing-time-first by measured wall-time
(scripts/ci/proxied_test_durations.json, captured from two bazel-proxied
job failures' testlogs) instead of the bd-init count proxy, via
scripts/ci/gen_proxied_shard_manifest.py's new --weights=duration mode.
Coverage is proven equal to the full discovered test set by two
independent methods (sorted-name diff and an actual 30-shard invocation
of proxied-test-shard.sh in list-only mode), both exact matches with no
duplicates.

cmd/bd:bd_proxied_test's shard_count moves to 30; PR Risk's and
main.yml's own test-proxied-cmd jobs keep their 15-shard matrix and
frozen manifest block untouched, per D2's retirement design (the Bazel
lane only needs to run the same tests, not the same split). Updated the
two coupled policy tests (scripts/ci_workflow_test.go's
TestBazelDoltServerTiersMirrorPRRisk, scripts/pr_risk_bazel_coverage_test.go's
TestBazelRetiredLanesCheckListedTestsRan) to allow the Bazel target's
shard_count to differ from its legacy job's own matrix size, and
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md / engdocs/TESTING.md to document
the split.

Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint on bazel.yml; make bazel-sync-check;
make ci-pr-policy; 3 representative shards (1, 15, 30 of 30) run
directly against Bazel-built race bd_test/bd_for_tests binaries and the
pinned dolt 2.2.0 release, all green, 55-81s wall time each (the
duration data's absolute seconds are from a contention-era sample and
run far faster here; the LPT balance itself, not the absolute SLA, is
the property that matters).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci,bazel): close fork-PR gap in proxied 30-shard manifest validation (review B1/S1)

TestShardScriptsListOnlyRealTests only ran proxied-test-shard.sh at PR
Risk's legacy 15-shard matrix, never at the Bazel-only bazel-proxied
lane's 30-shard manifest block. bazel-proxied only runs in mode=remote
(skipped on fork PRs), so a fork PR could rename, duplicate, or corrupt
a TestProxiedServer*/TestServerMode* manifest entry in the 30-block and
merge green via go test ./scripts/... (which runs on every PR), only
breaking the required lane afterward on main and every same-repo PR.

Fix: the test now computes every shard total it must validate per
script (this job's own PR Risk matrix size, every distinct total found
in that script's own committed manifest, and - for proxied-test-shard.sh
- cmd/bd:bd_proxied_test's live shard_count from BUILD.bazel) and runs
the full per-shard validation loop at each one, relying on the script's
own duplicate/stale-name guards. It also sums each total's "manifest: N"
counts across all shards and fails if they sum to zero, catching a
silently-deleted block that would otherwise fall back to 100% hashing
undetected.

Also fold in S1: replace the independently hard-coded `30` overrides in
ci_workflow_test.go and pr_risk_bazel_coverage_test.go with
bazelProxiedShardCount(t), a new shared helper that reads
cmd/bd:bd_proxied_test's shard_count live from cmd/bd/BUILD.bazel. This
turns the check_shard_coverage.py argument comparison in
pr_risk_bazel_coverage_test.go into a genuine cross-file pin (BUILD.bazel
vs bazel.yml) instead of two literals that could drift together
unnoticed. ci_workflow_test.go's tiers loop no longer asserts
BUILD.bazel's shard_count against a value read from the same file
(tautological); it instead asserts pr-risk.yml's and main.yml's legacy
test-proxied-cmd matrices equal each other, restoring the pin S1 found
removed on this branch relative to base.

Verified against the reviewer's mutations (duplicate entry, bogus
entry, whole-block deletion, stale rename, matrix divergence, BUILD
shard_count drift, bazel.yml arg drift): each now fails go test
./scripts/..., where it previously passed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci,bazel): add generator --write/--check and freeze legacy manifest block (review S3/N1/N4/N5)

gen_proxied_shard_manifest.py:
- Add --write (rewrite only the target total_shards block in place,
  leaving every other block's header/body text untouched) and --check
  (regenerate in memory and diff against the committed block, exiting
  1 with the exact --write command needed if stale). Both operate via
  new split_blocks()/write_block()/check_block() helpers that round-trip
  the real manifest file losslessly.
- N1: fix the duration-weights header's false claim about the
  fallback ratio.
- N5: warn on stderr when proxied_test_durations.json has duration
  captures for tests that no longer exist in cmd/bd/*_test.go (renamed,
  split, or deleted since the last capture).

proxied-cmd-test-shards.txt:
- N4: mark the legacy 15-shard block's header as frozen for
  pr-risk.yml's/main.yml's legacy fork/push jobs: it is deliberately
  not kept in sync with newly-added tests (those hash-fallback
  instead), so regenerating it is not routine maintenance. Only the
  header text changed; all 149 explicit assignment lines and the
  "known blind spots" callouts are untouched (verified: a fresh
  --write would pack all 164 currently-discovered tests explicitly,
  silently changing which shard runs which test for this block, which
  is exactly what "frozen" means to avoid).
- Regenerate the Bazel-only 30-shard block's header via
  `gen_proxied_shard_manifest.py 30 --weights=duration --write` to
  describe --write/--check instead of the old manual-copy-paste
  instructions. proxied_test_durations.json has not changed yet (S2
  is still in progress), so this run only touched header text; the 164
  assignment lines are unchanged.

pr_risk_bazel_coverage_test.go:
- Add TestProxiedShardManifestGeneratorNotStale: runs
  gen_proxied_shard_manifest.py 30 --weights=duration --check under
  plain `go test` (skipped under `bazel test`, whose runfiles hold
  neither the generator's sources nor cmd/bd) so a stale 30-block
  (e.g. committed by hand, or left behind after adding tests without
  regenerating) fails scripts-go-checks on every PR, including forks.

tools/bazel/equivalence_allowlist.txt: allowlist the new test's guarded
Bazel skip.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci,bazel): rename 14 undiscovered proxied-gated tests; N2/N3/N6 doc fixes

cmd/bd/*_test.go: 14 tests call requireProxiedServerEnv/
requireSharedProxiedServer but their names did not match the shard
scripts' discovery regex ^Test(ProxiedServer|ServerMode), so no CI lane
ever set BEADS_TEST_PROXIED_SERVER=1 for them and they silently skipped
everywhere (pre-existing gap, not caused by F2; list was in
~/beads-bazel-plan/f2/review/undiscovered_gated.txt).

Fixed by renaming rather than widening discovery: 162 of the 164
already-discovered tests call one of the two gate functions in the first
few lines of their body, so name and gate already correlate almost
perfectly, but the 14 tests' names are each styled differently with no
single regex term that would cover all of them without a hand-maintained
list. Renaming lets the existing discovery mechanism, hash-fallback
behavior, and both shard-coverage policy tests pick them up with zero
other changes. Checked each of the 14 names for cross-references first;
each appeared only in its own file plus at most one doc-comment in a
sibling file, both updated alongside the rename. No collisions with
existing test names.

Discovery went from 164 to 178 tests. Regenerated the Bazel-only 30-shard
block (gen_proxied_shard_manifest.py 30 --weights=duration --write) to
pack the 14 new tests; this is the first --write this round to actually
change assignments, not just header text. The frozen legacy 15-shard
block (N4) was deliberately left alone: the 14 renamed tests now
hash-fallback into it, exactly like every other test added since it was
last generated.

cmd/bd/BUILD.bazel (N6): restore the bd_proxied_test comment sentence
bazel.yml's mirror comment kept but BUILD.bazel's had dropped: PR Risk
builds its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd
twin uses build-artifacts' non-race bd-cmd-test, so this lane is the
stricter of the two.

.github/workflows/bazel.yml, .bazelrc (N2/N3): remove stale timing
language ("30 shards of ~5-10 minutes" / ".bazelrc's leftover "15 shards
of ~10 minutes") never reconciled with this task's own local
representative-shard measurements (55-81s/shard); replaced with the
concurrency argument (30 simultaneous dolt sql-servers don't fit a
GitHub-hosted runner regardless of any single shard's wall time). Also
removes the "within 5s of each other" balance claim (S2): the underlying
duration weights are contaminated by -test.parallel overlap in the
top-level <testsuite time> figures they were derived from; a corrected
re-measurement is in progress (separate commit once it lands).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci,bazel): re-derive proxied duration weights without overlap double-counting (review S2)

scripts/ci/proxied_test_durations.json's 178 entries are now each a
clean, single-job (no concurrent heavy bazel/dolt job on the box),
-test.parallel=1 measurement: a fresh manifest-aligned 30-shard sweep's
own <testsuite time> per top-level test where errors="0" (including
failing-but-complete suites, whose elapsed time is still trustworthy),
plus, for the handful too slow to finish inside Bazel's default
per-target timeout, a `bazel run` leaf-subtest-only sum that never adds
a parallel group's own PASS/FAIL line on top of its already-overlapping
children's lines. An earlier pass at this data summed every PASS/FAIL
line indiscriminately, double-counting parallel subtest groups.

A first attempt at re-measuring the slow stragglers overlapped in
wall-clock time with a second, unrelated heavy sweep and came out 2-3x
inflated versus the same tests measured cleanly; that contaminated data
was discarded in favor of isolated re-runs. TestProxiedServerDep2
initially looked hung during its isolated re-run (15+ minutes with no
PASS/FAIL output and no CPU growth in the top-level test binary) but
was not: each of its subtests does real `bd init --proxied-server`
work in a short-lived child process, and Go buffers a parallel
subtest group's output until the group's last member finishes. Letting
it run to completion (2357.72s) rather than guessing avoided reporting
bad data.

seconds_per_init_fallback is recomputed as the weighted average
seconds-per-init across all 178 measured tests (sum of durations / sum
of inits), for use only when a brand-new test has no entry yet.

Re-packing the 30-shard manifest with this corrected data changes the
shard-balance picture: TestProxiedServerDep2 and TestProxiedServerClose
are each large enough, alone, to dominate their own shard (~2358s and
~1105s locally) while every other shard clusters around 741-876s. The
"within 5s" claim this replaces was never earned by the underlying
data; .github/workflows/bazel.yml now documents the real, uneven
spread and flags that the two outlier shards' local numbers already
exceed the job's 25m step timeout on their own merits (unverified
against RBE's actual per-shard throughput, since this job has only
ever run remote).

Verified: go vet ./scripts/..., go test ./scripts/... (includes the
manifest/coverage policy tests), actionlint, make bazel-sync-check,
make ci-pr-policy all pass. Exact-once coverage re-confirmed: 178
manifest assignments for 178 discovered tests, zero duplicates, zero
gaps.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(cmd/bd): apply re-review candidate fixes; revert 2 unready renames (R-B1)

Re-review found the prior round's 14 renames would turn the required
Bazel + legacy 15-shard lanes red once discovered:

- TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix/external-unix and
  TestProxiedServerOutageReconnectAcceptanceMatrix/external-unix-socket
  built a dolt.sock path under t.TempDir(), which nests Bazel's hermetic
  tmp-root plus the (64-byte-truncated) test name deep enough to exceed
  sockaddr_un's 108-byte sun_path limit. Both now use a new shortSocketPath
  helper (a short os.MkdirTemp("", "bds") directory) instead.

- runProxiedDeadline raced a deadline-watcher goroutine against
  exec.Cmd.Start()/Run()'s write to cmd.Process, caught by -race. Replaced
  with cmd.Cancel, which exec.CommandContext only invokes after Start, so
  the read is ordered after Start's write.

With those two fixes applied, TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix
passes outright. TestProxiedServerOutageReconnectAcceptanceMatrix's race is
fixed, but both its subtests still fail a real "exceeded 3s bound" assertion
against a stopped external upstream -- a genuine latency issue, not a test
bug -- so it is reverted back to its original name
(TestProxiedOutageReconnectAcceptanceMatrix) with a comment explaining why,
pending a bead against the underlying latency.

TestServerModeSharedServeGlobalReportsTheServedDatabase also reverted back
to TestSharedServerModeServeGlobalReportsTheServedDatabase: `bd serve
--global` refuses to auto-apply 4 pending schema migrations (v65->v69) to a
shared/proxied server database and exits 1 with a consent-required usage
message instead of starting, per the #5920 migration-consent guard. Also a
genuine product/test-fixture question, not fixed here; reverted with a
comment, pending a bead.

The other 12 of the original 14 renames are kept; all 12 verified passing
in both lanes (see commits that follow for the manifest update and full
verification).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci,bazel): re-derive proxied weights from parallel=4 CI-identical sweep (R-S1)

The prior round's duration data (committed in 6be9605d9f) was captured at
-test.parallel=1 (serial). That fixed a real double-counting bug, but
serial timings don't model this lane, which always runs -test.parallel=4
(--config=doltserver-proxied's own setting). Re-review measured near-zero
correlation (r=0.08) between the serial-derived predicted shard load and
actual parallel=4 wall time -- e.g. TestProxiedServerRecomputeBlocked was
weighted 812s serially but measured 33s actual at parallel=4.

Replaced with a full fresh capture: `bazel --nohome_rc test
--config=doltserver-proxied //cmd/bd:bd_proxied_test` (local execution
only, the exact config bazel.yml's proxied-server tier uses), 30/30 shards
passing, 0 failures, 0 races. Every one of the 176 discovered tests'
top-level <testsuite time> was captured directly from this run's
bazel-testlogs test.xml files -- no leaf-summation workaround was needed
this time, since none of the 176 hit Bazel's per-target default timeout
running alongside the rest of its shard.

Re-ran `gen_proxied_shard_manifest.py 30 --weights=duration --repack
--write` with these corrected weights: a deliberate full reweight, not an
incremental add, since every weight changed. The new packing clusters all
30 shards within a few seconds of ~355-360s each (176 tests summing to
10712s / 30), a dramatic improvement over the serial data's wrongly
implied two-outlier-shard spread (TestProxiedServerDep2 alone at ~2358s,
TestProxiedServerClose alone at ~1105s -- both artifacts of serial
timing, not real per-shard cost). Updated bazel.yml's paragraph to report
this honestly instead of the prior, false "exceeds 25-minute timeout"
claim, which was itself downstream of the serial/parallel mismatch.

Verified: the 12 tests kept renamed in the previous commit, plus
TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix's socket-path fix,
all pass under the new 30-shard Bazel manifest (this capture run) and
under the legacy 15-shard hash-fallback script (ran the 7 shards their
hash lands in: 2, 3, 4, 5, 8, 10, 11 -- all pass).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ci,bazel): incremental --write, --repack, name-coverage --check (R-S2)

The prior --check compared the committed 30-shard block byte-for-byte
against a fresh from-scratch LPT pack, and --write always did a full
repack. That meant any weight drift (even from proxied_test_durations.json
alone, with no test added/renamed/removed) failed --check, and two
unrelated PRs each adding one proxied test would both force a full
repack and conflict on unrelated shard lines -- re-review demonstrated
one new test moving 73 assignment lines under the old full-repack
behavior.

Redesigned:

- --check now verifies exact-once NAME coverage of the currently
  discovered TestProxiedServer*/TestServerMode* set against the
  committed block -- missing, stale, or duplicated names fail with an
  actionable message -- but does NOT compare shard assignments, so a
  weights-only change no longer forces --check to fail.
- --write is now incremental by default: every already-assigned test
  keeps its current shard, stale (no longer discovered) names are
  dropped, and only newly-discovered names are LPT-placed onto whichever
  shard is currently lightest. A full from-scratch LPT pack now requires
  the explicit --repack flag (used in the preceding commit for the
  deliberate full reweight; ordinary new-test additions should use the
  incremental default).

Mutation-tested directly (not just reviewed): verified --check correctly
flags a stale name as "stale/unknown" and a newly added test as "missing"
with the exact --write command to fix it; verified incremental --write
touches only the lines for the added/removed names; verified --repack
--write produces a materially different (full rebalance) diff from the
incremental default.

Updated TestProxiedShardManifestGeneratorNotStale's comment in
pr_risk_bazel_coverage_test.go to describe the new name-coverage-only
--check semantics (it previously described the old byte-identical-packing
behavior; the test's own assertion -- exit code only -- was already
correct and unaffected).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-03 09:36:05 -07:00
Julian Knutsen
aaae7e6aff
ci: package gates start after rbe, MCP pytest -n 8, gate jobs on Blacksmith (F3) (#7126)
* ci(bazel): move package-mcp/package-npm gates into bazel.yml (F3)

Package gates now build bd via Bazel (no --config=ci) when the rbe job
enables it and fall back to go build otherwise, so forks/Dependabot/rbe-off
runs are unaffected. pr.yml's CI Gate keeps PACKAGE_MCP/PACKAGE_NPM required
and never-skippable, now sourced from needs.bazel.outputs.*; the retired
detect-package-gates/package-mcp/package-npm pr.yml jobs and
DETECT_PACKAGE_GATES are deleted. MCP pytest pins -n 8 explicitly with
pytest-xdist locked via uv lock. rbe, bazel-coverage, and both required gate
jobs move to Blacksmith for same-repo PRs (gate names unchanged).

Also closes a gap in detect-package-gates.sh (missing bazel.yml in the
force-both-gates path list, since the job definitions moved there) and wires
integrations/beads-mcp's manifest/lock into scripts_test's Bazel data so the
new pytest-xdist pin check runs under `bazel test` too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(bazel): review fix-up for F3 package gates (SF-1, SF-2, nits)

Review of 65f378d241 (~/beads-bazel-plan/f3/review.md, no blockers) found:

- SF-1: the package gates' `bazel build //cmd/bd:bd_for_tests` resolved to
  a different output configuration (k8-fastbuild) than `bazel test
  --config=ci`'s bd_for_tests (k8-fastbuild-ST-..., via test:ci ->
  test:prcore's --@rules_go//go/config:race), so it got no action-cache
  reuse from the test job. Add the matching race flag to both jobs' build
  step; verified with cquery that both invocations now cquery to the
  identical bazel-out/k8-fastbuild-ST-.../bin/cmd/bd/bd_for_tests/bd path,
  and with a real build that it's an action-cache hit once bazel test
  --config=ci has run. Update the pinned Run string in
  pr_lanes_bazel_coverage_test.go and the impl report's cache-reuse claim.
- SF-2: drop stale comments/doc text saying the package gates download
  bazel-ci-build-artifacts; they build their own bd now (bazel.yml,
  pr.yml, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md).

Nits: pin both required gates' display names in TestSameRepoBlacksmithRunners
(nothing else reads job.Name, so a rename would otherwise go uncaught); pin
the package jobs' Python/uv/Node setup against main.yml's copies; add
timeout-minutes to both gate jobs and detect-ci-tier (same-repo PRs now run
on paid Blacksmith minutes, not a free runner's 360-minute default); shut
down Bazel and remove the RBE client key right after each package job
packages its Bazel-built bd (nothing downstream needs either); add
scripts/ci/package-bazel-bd.sh to detect-package-gates.sh's both-gates
trigger arm and pin it in TestDetectPackageGatesCoversBazelWorkflow.

Every fix mutation-tested (mutate -> test fails with a specific message ->
restore from a /tmp backup, diff-confirmed identical -> test passes again).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-03 00:03:43 -07:00
Julian Knutsen
0588af25d1
ci: retire pr.yml's legacy PR Core, build, pure-Go, domain+uow and contract jobs where the gated Bazel lanes run them (D2 step 3) (#7097)
* ci: retire pr.yml's legacy PR Core, build, pure-Go, domain+uow and contract jobs where the gated Bazel lanes run them (D2 step 3)

On same-repo, non-Dependabot pull_requests while the committed
BAZEL_RETIRES_LEGACY_PR_LANES flag is "true" (bazel-coverage's new
pr_lanes output; the job stays identical in pr.yml and pr-risk.yml),
pr.yml's build-artifacts, pr-core-wrapper, check-cmd-bd-puregeo-tests,
test-domain-uow and contract-corpus stand down. ci-gate accepts those
skips only then, and BAZEL_PR_LANES_RETIRED requires the same run's
Bazel test, pure-Go/js-wasm and dolt-server lanes to have run remotely
and passed. Revert: commit the flag "false" in both workflows.

What the retired jobs did besides their tests keeps running on every PR:
- the package gates take bazel-ci-build-artifacts (bazel-test's bd, same
  layout) where build-artifacts stands down;
- the Dolt server fingerprint moves to its own required job;
- the Go tests Bazel does not run or skips (equivalence allowlist) run
  under go test in pr-preflight-platforms' Linux leg, each required to
  pass (tools/bazel/run_allowlisted_go_tests.py).

In mode remote the three lanes' bazel test invocations add
--config=sole-run (no cached test results, no whole-invocation retry on
remote cache eviction), as steps 1 and 2 do for their lanes.

Policy tests: decision matrix and named cases extended to the third
flag, gate simulation for the pr.yml tier, artifact consumers, pinned
lane steps and rc lines, a repo-wide narrowing scan of BUILD rules and
.bzl files, and the allowlisted-test runner.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(ci): record the retired pr.yml PR Core, build, pure-Go, domain+uow and contract jobs in the check topology

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(ci): only pr.yml's workflow env sets a BAZEL_RETIRES_* flag, and only the decision step reads one

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(pr): keep go test ./scripts/..., go test's vet and the Bazel-skipped tests on every PR (D2 step 3 review F1, F1b, F3, L2)

Review of D2 step 3: retiring PR Core removed the only required pre-merge
`go test` of the scripts packages, whose policy tests (the D2 guards
among them) check part or all of their rules under go test only, and
go test's implicit vet for every package (rules_go's go_test runs none).

New required job scripts-go-checks (SCRIPTS_GO_CHECKS), on every PR, in
PR Core's environment (dolt, git and dolt identity, test-env):
- scripts/ci/scripts-go-test.sh: `go test ./scripts/...` with PR Core's
  flags;
- scripts/ci/go-test-vet.sh: `go vet` with cmd/go's defaultVetFlags over
  ./..., policy-tested equal to the running toolchain's list;
- scripts/ci/allowlisted-go-tests.sh, moved here from
  pr-preflight-platforms.

TestBazelOnlySkipsAreAllowlisted also requires every test that runs part
of its checks under go test only (a Bazel-guarded return/continue, or a
go-test-only block that reports failures) to live under ./scripts, the
only packages go test still runs on every PR; and step 3's
TestRunAllowlistedGoTestsScript gets its allowlist entry.

Package gates on a covered PR whose Bazel call did not run remotely (the
farm switch off) now fail in their own step naming
BAZEL_PR_LANES_RETIRED instead of on a missing artifact (L2).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(ci): record scripts-go-checks, the skip-allowlist check and the re-run-safe uploads for D2 step 3

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 06:08:52 -07:00
Julian Knutsen
e87aceedf9
test(init): port gateway credential fixtures onto shared credentialcmd (#7056)
#5725 gave the cmd/bd init gateway credential fixtures a private helper
protocol. While it was queued, #5749 landed internal/testutil/credentialcmd
for the same production shell boundary, leaving two parallel
implementations: the same emit/exit-23/marker operations, the same exit-97
malformed sentinel and the same Windows .cmd trampoline. They had already
drifted. The private copy claims a process on an env var alone, with no
argv sentinel, and its command text is stable rather than unique per call,
although the production credential cache is keyed on that text.

Port cmd/bd onto the shared fixture, as internal/creds and
internal/storage/dolt already use it:

- TestMain claims helper processes with credentialcmd.Dispatch() before
  any suite setup, and calls credentialcmd.Cleanup() after the suite.
- AdoptsToken, SkipsEmbeddedMode, FailsClosed and PresetWins build their
  commands with credentialcmd.Emit/Exit23/Marker and check non-invocation
  with credentialcmd.AssertMarkerAbsent. Their assertions are unchanged.
- The private protocol, its constants and
  TestApplyInitGatewayCredentialHelperProtocol, which tested only that
  protocol, are removed. credentialcmd's own TestProtocol covers the
  shared one and already runs natively on Windows in pr.yml.
- The native driver's fixture list drops the removed test, and bd_test
  gains the //internal/testutil/credentialcmd dep (gazelle agrees).

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR

Co-authored-by: Eddie the Engineer <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 08:03:19 -07:00
ecuthiell
46f182aa26
fix(ci): require direct workflow build tags (#5679)
* fix(ci): require direct workflow build tags

Parse workflow run scalars structurally and require recognizable direct Go commands to declare the literal gms_pure_go tag. Keep shell and Make checks lightweight and preserve their existing conventions.

Fixes #5527

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>

* fix(ci): recognize double-dash Go build tag flags

Address PR #5679 review 5191180453. Preserve last-tag precedence and test argument boundaries, including --args.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* fix(ci): bazel-sync the new checker, restore its opt-out and the Bazel census

Maintainer review fixes for PR #5679, adoption iterations 1 and 2. Iteration
1's synthesis is `request_changes` on one major plus three minors; all four
are addressed. Iteration 2 approves with one minor, also addressed.

Major (Release Safety): the PR adds two production Go packages with no
BUILD.bazel, so `bazel build //...` could not see them and the repo's own
"BUILD files in sync" step fails on the resulting gazelle drift. Added the two
generated files via `make bazel-sync`; yaml.v3 is already a direct go.mod
requirement, so no module change is owed.

Minor (Release Safety): the new checker runs in the required pr-policy lane with
no way out, while the shell checker it partly replaces honors
`# build-tags: allow-bare` in a file's first five lines (check-build-tags.sh, and
engdocs/ICU-POLICY.md documents that window). CheckDir now honors the same
marker in the same window -- one convention, not two -- pinned by a table that
includes the fifth line and the sixth, so the two checkers cannot drift apart on
where the window ends.

Minor (Change Impact): dropping the workflow glob from `candidates` moved
workflows to the Go checker but also, silently, out of the `bazel_users` census --
and the Go checker does no Bazel handling at all, so .bazelrc could have lost its
tag with this gate still green. Workflows now feed the census through their own
`git ls-files` list and a separate loop, deliberately not rejoining the `go`-verb
scan. This is not hypothetical: `.github/workflows/bazel.yml` matches the Bazel
invocation regex, so it is exactly the file the census was losing.

Minor (Test Evidence): `Line` is the operator-facing half of a violation -- it is
what the error message points at -- and nothing asserted it, so reporting every
violation at line 1 would have been invisible. Added a `wantLine` column, asserted
on the multi-line "source is not workflow authority" case (line 2), and asserted
`Path`/`Job` where CheckDir fills them in.

One deviation, recorded: the finding also asked for `Line` on the existing
"continuation" case. That case is tagged, so it reports no violation and has no
Line to assert. Added a continuing-but-violating case instead, which makes the
same attribution observable -- a logical line spanning three physical ones is
reported at the first.

Not included: `make bazel-sync` also wants `//internal/debug` in
internal/utils/BUILD.bazel. That is base drift, not PR content -- absent at this
branch's rebase base but already present on origin/main, 24 commits ahead -- so it
heals on the next base refresh, and carrying it would duplicate main's own fix on
a file this PR never touches.

Minor (Test Evidence, iteration 2): the census half of the Change Impact fix
was right in code but pinned by nothing -- turning the workflow `bazel_users`
append into a no-op, or feeding the census-only list into the `go`-verb loop,
left every check_build_tags_test.go case green. Two cases on the existing
fixture harness now pin both directions: a workflow that runs `bazel build`
with an untagged .bazelrc fails and is named as the Bazel user, and a workflow
`run: go test ./...` passes here because scripts/checkworkflowtags owns it.
Each fails under its mutant, as does one more apiece (re-adding the workflow
glob to `candidates`; dropping `*.yml` from the census list), while the three
existing cases stay green under all four.

Validation: ./scripts/workflowtags passes (all four CheckDir tests and the full
CheckRun table), `go vet ./scripts/...` clean, `bash -n` clean, and both checkers
run green against the repository itself -- check-build-tags.sh reports 90 files
scanned all clear, and checkworkflowtags reports all direct commands clear.
For iteration 2, `go test ./scripts/` passes, and on a merge with current main
`bazel test //scripts:scripts_test` passes too; on this branch's own base that
target cannot build, because of the internal/utils drift noted above.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ci): pin main.yml's bd-cmd-test build to the literal gms_pure_go tag

#6940 pinned main.yml build-artifacts' non-race bd-cmd-test build as
`go test -tags "$BEADS_BUILD_TAGS" -c -o artifacts/bd-cmd-test ./cmd/bd`.
This PR requires a literal gms_pure_go tag on direct workflow Go commands
and rewrites that line to `go test -tags gms_pure_go -c ...`, so on
current main TestBazelDoltServerTiersMirrorPRRisk fails in both
`go test ./scripts` and //scripts:scripts_test. Restoring the pinned
spelling is not an option: checkworkflowtags rejects it ("direct `go test`
must declare literal gms_pure_go in -tags").

Update the pin to the literal spelling. It still pins what #6940 cares
about, a non-race bd-cmd-test built from ./cmd/bd: adding -race to that
line still fails the test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 01:24:03 -07:00
ecuthiell
6899813df6
fix(preflight): share checkout lint authority (#5780)
* fix(preflight): share checkout lint authority

Move the Beads repository's native and Windows non-CGO lint policy into a checkout-owned Go driver. The supported Make/Bash wrapper and bd preflight now execute that same authority, while generic Go repositories retain their direct lint contract and formatting is rooted at the detected project.

Fixes #5300.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell

* docs(lint): clarify shared driver scope and timeout limits

Address PR #5780 review 5191123668. Document existing inherited-tag semantics and distinguish the wrapper/preflight driver from standalone workflow and pre-commit callers.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* fix(lint): prefer the selected Windows Go toolchain (#6865)

Keep per-pass toolchain requests and deadlines while avoiding the extra auto-selection launcher under the linter. Nonstandard SDKs retain their original PATH.

Agent-Signature: codex-tui-gpt-6-astra-max on behalf of Ewen Cuthiell

* fix(lint): retain PATH for SDK directories containing separators (#6865)

Agent-Signature: codex-tui-gpt-6-astra-max on behalf of Ewen Cuthiell

* test(preflight): declare lint fixture executable for Bazel (#5300)

Agent-Signature: codex-tui-gpt-6-astra-max on behalf of Ewen Cuthiell

* fix(bazel): keep cmd/bd/testdata out of gazelle's package scan

PR #5780 adds the compilable fixture cmd/bd/testdata/preflight-go.go. Left to
itself gazelle generates a BUILD.bazel for cmd/bd/testdata, which makes it its
own package and then (a) empties cmd/bd's data = glob(["testdata/**"]) under the
default allow_empty = False and (b) puts testdata/preflight-go.go across a
package boundary from the preflight_go_fixture target that lists it.

bazel.yml's "BUILD files in sync" step runs `make bazel-sync` and fails on any
resulting drift, and fork PRs always run that job, so this was a live red whose
prescribed remedy ("commit the result") would have broken the build further.
Exclude the directory instead, and take gazelle's ordering for the data list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(preflight): bound the lint deadline and report each check's scope

Review iteration 1 for PR #5780. Synthesis and scorecard both approve (975/1000,
no blocker or major), so this addresses non-gating findings only.

Finding 4 (Resource Lifecycle) - the 13-minute deadline could not bound the call
it guards. CombinedOutput collects through a pipe every descendant inherits, so
Wait kept blocking on a surviving golangci-lint after the context killed the
direct `go run` child. Measured on the same process shape (direct child killed at
1s, descendant holding the pipe for 8s): without WaitDelay the call returned in
8.00s, with WaitDelay in 1.50s. Sets cmd.WaitDelay so the deadline is a real
bound and the "lint check exceeded" line can actually print.

Also gates that line on err != nil, which retires the claude lane's nit (a): a
child that exits cleanly as the deadline fires is a pass, not a timeout. Measured
at the crossover, 68 of 300 passing runs would have carried a false "exceeded"
line; 0 after the gate.

Finding 1 (Change Impact) and nit 7 (Debuggability), one remedy - the PR re-roots
lint and fmt at preflightProjectRoot() but leaves runTestCheck in the caller's
cwd, so a --check run from a subdirectory mixes a repo-wide verdict with a
subtree-only one "with nothing in the output distinguishing them", and the
reported command is only reproducible from the root. CheckResult now carries the
directory each check resolved to, and the report prints it.

Took the synthesis's second arm (state the scope split) rather than re-rooting
runTestCheck, because re-rooting is a behaviour change neither lane priced: it
widens a subdirectory user's test run from 5 packages to 128, including every
Dolt-dependent package, on a repo whose full test gate the PR body already
discloses as red. The stated hazard is that the scopes are indistinguishable,
which reporting the directory discharges directly.

Finding 2 (Architectural Consistency), second clause - adds
TestBuildFlagsContractMatchesShellSource, which parses .buildflags and pins the
driver constants to it. The existing mirror test asserts today's values from the
Go side only, so the files could drift silently; buildEnvironment overrides
rather than merges -tags, so a tag added to .buildflags alone would be dropped
with every test still green. Each of the three legs was mutation-proved to red.

Declined, with the measurements that decided it:

- Finding 2's first clause (route the key comparisons through
  internal/execenv.KeyEqualForOS). The two are provably equivalent today -
  execenv's keyIdentityForWindows is strings.ToLower for the same documented
  reason - so this is de-duplication, not a defect, and the hazard it closes
  (os/exec key-identity semantics changing) is far less live than the
  .buildflags drift the second clause pins. The faithful swap means threading
  goos through five driver functions and ~27 test call sites in this PR's
  centerpiece file, and it would put a non-stdlib dependency into pr-lint_lib,
  which the binary currently does not have. Worth doing on its own footing, not
  inside a non-gating cleanup.

- Nit 6 (parse GOFLAGS into fields instead of a substring test). .buildflags:22
  guards its own append with `!= *gms_pure_go*`, a substring test, and the Go
  code is a deliberate mirror of it. Making only the Go side stricter would
  manufacture exactly the drift finding 2 warns about: on
  GOFLAGS=-tags=gms_pure_go_x the shell would skip its append while the driver
  performed one. The fix belongs to .buildflags and the driver together, and
  .buildflags is outside this PR. The new mirror test now pins that agreement.

- Nit 5 (probe golangci-lint as well as `go`). The claude lane examined the
  LookPath target change and recorded it as deliberate, with the driver's own
  missing-linter error as the mitigation. The suggested alternative - have the
  driver emit preflight's `--skip-lint` hint - puts a bd-specific flag into a
  driver that CI also invokes, where that advice is wrong.

- Finding 3 (two false PR-body claims). Correcting the body is outward-facing
  and belongs to finalize; stamped for it. Swept the in-tree copies: the PR's own
  LINTING.md sentence "Toolchain requests, formatting, and lint deadlines are
  unchanged" is scoped to the Windows SDK-preference paragraph it sits in and is
  true there, so it is graded rather than repaired. CHANGELOG.md contains no
  pr-lint entry, so the remedy's CHANGELOG half has no target in this PR.

Validation: go build, go vet and gofmt clean on the changed packages;
./scripts/pr-lint/ 12 pass / 2 Windows-only skip; ./scripts/prlintmake/ pass;
cmd/bd preflight selector pass; scripts/check-doc-freshness.sh PASSED;
make bazel-sync idempotent and bazel test //scripts/pr-lint:pr-lint_test PASSED.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(prlintmake): inject the pr-lint wrapper's gofmt shim through GOFMT

The merge with #6294 (be-gx8) left runPRLintWrapper relying on a
code path #6294 made deliberately loud. The helper put its gofmt shim
on PATH. Since #6294, fmt-check.sh resolves gofmt through
scripts/ci/gofmt-bin.sh, which ignores a PATH gofmt by design.

Both TestPRLintWrapper* tests still passed, but only via the
resolver's fallback. The go shim answers `go env GOVERSION` and
`go env GOROOT` with nothing, so gofmt-bin.sh warns "could not resolve
a gofmt from go.mod's go1.26.7 toolchain" and falls back to whatever
gofmt is first on PATH, which happened to be the shim. With GOFMT set
in the environment, which is #6294's documented override, the
resolver uses that value instead. Both tests then fail: exit status
127 from fmt-check.sh, and a missing aggregate failure diagnostic
because pr-lint.sh stops before `go run`. At the pre-merge head
(7f37ba5165) the same ambient GOFMT=/nonexistent run passed all 5
tests.

Pass the shim through GOFMT, as main's runFmtCheck does, and keep it
off PATH so only the go shim is found there. gofmt-bin.sh now returns
the override immediately, and the go shim sees only `version <shim>`
and `run ... ./scripts/pr-lint`. Both wrapper assertions, on the
driver arguments and on the failure diagnostic, are unchanged.

go test ./scripts/prlintmake/: 9/9 pass by default and 9/9 pass with
GOFMT=/nonexistent in the environment. Before this change the second
run was 7/9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 22:41:55 -07:00
ecuthiell
61865bb0fe
test(init): make gateway credential fixtures portable across native shells (#5715) (#5725)
* test(init): make gateway credential fixture portable (#5715)

Replace shell-specific fixture commands with an opt-in test helper, exercise exact credential semantics under a restricted shell PATH without copied Windows PE fixtures, and require the focused suite in native Windows PR CI.

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* test(init): keep the original shell across native credential fixtures

Use a symlink to the original Unix shell so macOS AMFI does not reject a
copied system executable. Keep Windows cmd.exe and the test image in place,
internalize helper environment names and assert the typed helper exit code.

Run the six fixture cases through one Bash3.2-compatible driver in the
existing required three-host preflight matrix. Preserve the liveness check
name and optional macOS suite policy; reject missing or skipped execution.

Review: https://github.com/gastownhall/beads/pull/5725#pullrequestreview-5096996756
Bead: saa-cgh.47
Agent-Signature: unknown-model/unknown-reasoning

* test(ci): sync-check the native fixture list and stop forcing a second build graph

Maintainer review fixes for PR #5725, adoption iteration 1. The synthesis is
`approve` with no required changes; all findings are minor or nit. The rebase
lane had to repair this branch (`repaired_clean`, and the original head still
conflicts with main), so there is no accepted-as-is path and finalize owes a
push either way — these cost nothing extra.

Test evidence (minor): the `expected` array is, by its own comment, the single
list owning both test selection and required execution evidence -- but nothing
tied it back to the source file. A seventh TestApplyInitGatewayCredential*
fixture added later would fall out of the `-run` selector, lose native
macOS/Windows execution entirely, and still leave the driver exiting 0 with the
required gate green: precisely the "green because nothing ran" mode the per-name
PASS counting exists to prevent. The driver now compares the array against
`grep -Eo '^func TestApplyInitGatewayCredential[A-Za-z0-9_]*'` on
cmd/bd/init_gateway_test.go, by name and by count, before running anything, and
exits non-zero naming the offending fixture. That file is the whole population --
no same-prefix fixture lives in any other cmd/bd test file.

Release safety (minor): the driver sourced .buildflags and then forced
CGO_ENABLED=0, compiling a package graph disjoint from the ./cmd/bd build the
same job performs two steps earlier under the default. Re-measured under the
tags the steps use: `go list -deps -tags=gms_pure_go ./cmd/bd` is 1505 packages
with cgo and 742 without, and the differing cgo users include `net` and
`os/user`, whose build IDs propagate upward -- so nothing was reused and this
blocking, `cache: false` lane paid a second cold compile on macOS and Windows.
The export is dropped. Two consequences beyond the time: the step becomes
incremental, and the natively executed fixtures now exercise the same build of
cmd/bd that actually ships on those platforms rather than a nocgo one. The
comment now explains the build configuration, which is what the old one did not
do -- it explained why no database is needed, a separate question.

Debuggability (nit): the base comment above test-windows-liveness carried three
checkable mechanism clauses (the test is //go:build windows; the Ubuntu job
selects it by name but compiles it out; main.yml's Windows job runs only on
pushes to main). The replacement stated a conclusion a future maintainer cannot
verify, which is how a redundant-looking CI job gets deleted. The mechanisms are
restored and the new pr-preflight-platforms pointer is appended rather than
substituted.

The remaining nit asked for no change in this PR (the credential cache is keyed
by command string, which is harmless as shipped since neither the marker nor the
failure arm populates the cache) and none was made.

Validation: the driver runs green on Linux under the inherited default CGO --
6 of 6 fixtures PASS, exit 0 -- and the sync check was proved by appending an
unwired seventh fixture, which reds it with exit 1 naming that fixture, before
any test runs. `bash -n` clean, no shellcheck SC2086, and ./scripts/ stays green.

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 20:10:22 -07:00
mtccressall
1f7a5f1eaf
fix(ci): resolve gofmt from go.mod's toolchain, not PATH (be-gx8) (#6294)
* fix(ci): resolve gofmt from go.mod's toolchain, not PATH (be-gx8)

gofmt's output is not stable across Go releases, so whichever binary runs
decides the formatting verdict. Three sites reached for a bare `gofmt` on
PATH -- scripts/ci/fmt-check.sh, the Makefile fmt target, and
.githooks/pre-commit -- and PATH belongs to whatever Go is installed
locally. CI has no such freedom: actions/setup-go installs exactly the
go.mod version via go-version-file.

GOTOOLCHAIN=auto does not correct for this, which is why `go env GOROOT`
or `go fmt` would not have been a fix either. The toolchain switch only
ever moves UP to satisfy the go directive, never down, so a host running
a newer Go keeps formatting with the newer gofmt. Measured against
go.mod's go 1.26.5 on a go1.27.0 host: 1.27 flags three
role_fixture_kit_test.go files that 1.26.5 calls clean, and no branch
touches them.

The failure ran in the dangerous direction. The required lint gate went
red on untouched code, which reads as "main's lint is broken" and poisons
the pre-existing-on-main judgement every PR author uses to decide whether
a red gate is theirs. Its own advice -- "Run 'make fmt' to fix
formatting" -- then rewrote those files into a form CI's pinned gofmt
rejects. The pre-commit hook did that same rewrite and re-staged it
silently on every Go commit, with nobody choosing it.

Route all three through one resolver rather than patch three sites, so
the next gofmt call site cannot drift. scripts/ci/gofmt-bin.sh names
go.mod's toolchain explicitly and reads GOROOT from it, short-circuiting
to a plain `go env GOROOT` when the host go already is the pinned version
(the CI case, so nothing is fetched where nothing needs fetching). A
fallback to the PATH binary stays available but warns loudly on stderr:
silently falling back is indistinguishable from resolving correctly, and
reintroduces exactly this skew. fmt-check.sh now prints the gofmt version
it used, so a skew presents as a skew instead of as a repo defect.

The new tests pin the resolution, not the reporting, because the
resolution is where the bug lived: every output assertion around the bare
gofmt stayed green. Reverting gofmt-bin.sh to `command -v gofmt` fails
all three. The pre-existing fmt-check tests move their shim from PATH to
GOFMT -- a PATH shim is now correctly ignored, so leaving them there
would have made them grade the real toolchain instead of the case they
name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PoN7f9TMmGTCLRpDPzSH4y

Executed-By: beadsrig/polecats/nux

* fix(ci): resolve gofmt from go.mod's toolchain directive, and declare it to Bazel

Maintainer review fixes for #6294 (adoption iterations 1 and 2, one squashed
commit). Sections 1-6 address all six findings in the attempt-1 synthesis; the
two majors are the substantive ones. Sections 7-8 address the two minors the
attempt-2 synthesis left (it decided approve, 1000/1000, no blocker or major).

1. Contract & Interface Fidelity (major, the scorecard's only Required Change).
   gofmt-bin.sh read the `go` directive (1.26.5) while go.mod also carries
   `toolchain go1.26.7`, and toolchain-first is the repo's own declared
   precedence -- Makefile:85-90 exports GOTOOLCHAIN from the same pair for
   every make target, and scripts/bazel_policy_test.go's goModToolchainVersion
   is the same rule in Go. The consequence was not cosmetic: `go env GOVERSION`
   at the repo root is go1.26.7 (the directive auto-switches), so the fast-path
   test compared go1.26.7 against go1.26.5 and was false by construction on
   every route, sending each required fmt-check run down the elif arm to fetch
   a second, older toolchain. Parse `toolchain go<version>` first, fall back to
   the `go` directive. Measured after the change: pinned=1.26.7, the fast path
   fires (`go env GOVERSION` then `go env GOROOT`, no GOTOOLCHAIN= prefix and
   nothing fetched), and `make fmt-check` now reports
   `Using gofmt go1.26.7 (.../toolchain@v0.0.1-go1.26.7.../bin/gofmt)`.
   .githooks/pre-commit:24's `go`-directive read is deliberately left alone --
   its comment explains it is a golangci-lint pin for a different reason.

   gofmtbin_test.go's helper now implements the same precedence, and
   TestGofmtBinMatchesGoModDirective is renamed TestGofmtBinMatchesGoModToolchain
   so the pinned contract is legible from the test name.

   Deviation from the prescription, recorded deliberately: the review asked for
   a helper "name" change as well. The name goModToolchainVersion already
   described toolchain-first semantics; it was the implementation that did not.
   Correcting the semantics is what removes the collision with
   scripts/bazel_policy_test.go's identically-named, opposite-behaving helper,
   so the name is kept and the doc comment now states the precedence and cites
   both in-repo sources of it.

2. Release Safety (major). fmt-check.sh now execs a sibling script
   unconditionally under `set -euo pipefail`. Live origin/main gives
   //scripts/prlintmake a `data` list containing exactly ci/fmt-check.sh, so on
   merge all three TestFmtCheck* cases would have died at rc 127 in the Bazel
   runfiles sandbox -- the GOFMT short-circuit cannot rescue them because it
   lives inside the missing script. Export ci/gofmt-bin.sh to
   //scripts/prlintmake and put it in the target's `data`, and add
   gofmtbin_test.go to `srcs` (it carries no build tag, so its omission was not
   explained away).

   Only ci/gofmt-bin.sh is exported here, not ci/fmt-check.sh: main already
   exports that one for this same package, and a second exports_files call
   naming the same file in one package is an error. This base predates both of
   main's additions, which is also why editing the three-line go_test rule
   conflicts with main's independent edit of it. Verified by materialising the
   merge: every other file auto-merges and the resolution is the union of the
   two attribute sets (srcs = both test files; data = both scripts plus
   //:go.mod per section 7; keep main's deps on //internal/testutil/bazeltest).
   That deps edge is load-bearing at main even though it was stale at this base:
   main rewrote sourceRepoRoot to bazeltest.CallerDir, so the union must keep it.

3. Test Evidence Quality (minor). environment() starts from os.Environ(), so
   deleting GOFMT from the override map did not unset it. Export it empty
   instead -- gofmt-bin.sh reads `[[ -n "${GOFMT:-}" ]]`, so empty already is
   its "not set" case, and the override test stays distinguishable by setting a
   non-empty value.

4. Architectural Consistency (minor). "Every gofmt invocation in this repo goes
   through here" was false at authoring: cmd/bd/preflight.go's doctor check is
   a fourth front door on bare PATH gofmt. Scope the claim to the repo's
   shell-side call sites (re-censused: fmt-check.sh, the Makefile fmt target,
   and .githooks/pre-commit are the complete set; the two workflow "Check
   gofmt" steps and scripts/ci/pr-lint.sh reach them through make fmt-check),
   and name the in-binary exception with the reason it is one -- bd runs
   against whatever project the user points it at, so it has no scripts/ci/ to
   call. preflight.go is byte-identical at base, at HEAD and at origin/main, so
   nothing in this PR widens it; converting it is a separate change.

5. Test Evidence Quality (minor). describe_gofmt's version-bearing arm had zero
   coverage -- all three existing cases inject a bash script, for which
   `go version` errors and only the bare-path arm can run. Add
   TestFmtCheckReportsGofmtVersion against a real Go-built no-op binary, which
   is the only shim shape `go version` can report on.

6. Error Handling & Resilience (nit). Warn (without refusing) when GOFMT names
   something non-executable, so a typo is reported by the resolver that
   accepted it rather than as a "not found" from whichever call site ran it;
   and guard the fields index in the resolver test.

Mutation proofs, each red isolated to the intended test with the other six
green (7/7 pass unmutated, `go test ./scripts/prlintmake/`):

  - revert the toolchain-first parse -> resolver returns the 1.26.5 gofmt and
    TestGofmtBinMatchesGoModToolchain alone fails.
  - delete describe_gofmt's `if command -v go` block (codex's own mutation,
    previously all-green) -> TestFmtCheckReportsGofmtVersion alone fails, the
    three incumbents stay green. That asymmetry is both the coverage-hole proof
    and the proof the new case is not a duplicate.
  - ambient GOFMT, two values x two arms: with the fix both /nonexistent and a
    correctly-pinned gofmt leave all three resolver tests passing; with the
    delete restored, /nonexistent fails TestGofmtBinMatchesGoModToolchain with a
    message blaming the script, and a correctly-pinned value turns all three
    green vacuously without the resolver ever running.

7. Release Safety (minor, iteration 2). The target now reads go.mod at run
   time -- the resolver opens "$REPO_ROOT/go.mod" and the test helper opens
   filepath.Join(sourceRepoRoot(), "go.mod"), where the base test read it not at
   all -- but go.mod appeared in no `data` list, so the rule declared strictly
   less than it uses. Add //:go.mod to //scripts/prlintmake's `data`. No new
   export is needed: //:BUILD.bazel already exports go.mod publicly, and
   //internal/storage/dolt and //internal/testutil/integration already consume
   it as "//:go.mod" in `data`, so this is the sanctioned shape rather than a
   new one.

8. Contract & Interface Fidelity (minor, iteration 2). `toolchain default` is a
   legal go.mod directive value, and the parse matched on the field name alone,
   so sub(/^go/, "", "default") no-opped and left pinned=default. Because that
   is non-empty, neither the `go`-directive fallback nor the "no toolchain or go
   directive" guard could fire, and the script degraded to the bare PATH gofmt
   it exists to eliminate while a usable `go 1.26.5` sat two lines above. The
   mangled "godefault" in the warning was the tell. Require the literal `go`
   prefix, which is exactly what the repo's two other readers of this pair
   already do (Makefile:85's `s/^toolchain go//p` and bazel_policy_test.go's
   `^toolchain\s+go(\S+)\s*$`), so `default` falls through to the `go`
   directive.

   The same guard is added to gofmtbin_test.go's goModToolchainVersion, and that
   is not redundant: strings.TrimPrefix("default", "go") returns "default"
   unchanged and withPatchComponent leaves a dotless string alone, so the helper
   mirrored the bug exactly and the test could never have caught it. Guarding
   only the script would have left the test agreeing with a buggy resolver.

   Measured on a synthetic tree, five arms (the script resolves REPO_ROOT from
   its own location, so each arm is a copied script beside a written go.mod):

     - `toolchain default` + `go 1.26.5`: before, /usr/local/go/bin/gofmt with
       the "godefault" warning; after, the go1.26.5 toolchain gofmt -- i.e. the
       `go` directive, identical to the arm where no toolchain line exists.
     - `toolchain go1.26.7`: unchanged before and after (no regression).
     - `go 1.26.5` alone: unchanged before and after (no regression).
     - `toolchain default` with no `go` directive: now reaches the previously
       bypassed "no toolchain or go directive" guard and falls back loudly, with
       the correct message rather than the mangled one.
     - `toolchain default` written before `go`: also resolves 1.26.5, so the
       fallthrough is order-independent.

   The Go helper's parse was checked over the same six inputs and now agrees
   with the shell on every one.

   No regression test is added for this case. Doing so would need a new
   synthetic-go.mod fixture -- every existing case runs the real script against
   the real repo go.mod through sourceRepoRoot() -- which is a new test
   instrument rather than the prescribed remedy, on a finding whose gate impact
   is none (go.mod carries `toolchain go1.26.7`, so nothing can fire today).

Iteration 2 validation: go test ./scripts/prlintmake/ -count=1 -v 7/7 PASS
(same roster as iteration 1, no test added or removed), go vet rc 0,
go test ./scripts/ -run 'Bazel|BUILD' 11/11 PASS (the BUILD-policy gate, run
because this touches a BUILD file), python3 tools/bazel/go_srcs.py --check
rc 0 (`srcs` is unchanged; only `data` moved), make fmt-check rc 0 still
reporting go1.26.7, shellcheck clean, git diff --check clean.

Also run: go vet ./scripts/prlintmake, go test ./scripts/... (all packages ok),
make fmt-check (rc 0), python3 tools/bazel/go_srcs.py --check (rc 0),
shellcheck on both changed scripts.

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
2026-09-30 18:38:46 -07:00
Jim Wordelman
9d646fc887
fix(dolt): server-mode bd ready never wakes expired defers (re-proposes #6082) (#6424)
* test(dolt): red — defer auto-wake must run for classified-read-only server-mode opens (be-vbhpf)

TestServerModeDeferAutoWake exercises the same defer/wake contract as the
embedded and proxied-server variants, against a real dolt sql-server. Today
the sweep (DoltStore.wakeExpiredDefers) unconditionally skips whenever the
store is opened read-only — including when that read-only-ness comes purely
from command classification (bd ready/bd list are read-only for the CURRENT
project), which is indistinguishable, before the fix, from a strict
--readonly or foreign-project open that must never sweep.

RED: expired_dated_defer_wakes_on_ready and expired_dated_defer_wakes_on_list_ready
fail (issue never returns to open). dateless_defer_never_wakes and
future_dated_defer_stays_hidden pass today too, trivially (no sweep runs at
all pre-fix) — kept as regression guards for the GREEN step, which is where a
unit test proving classifiedRead=false still does NOT sweep belongs, since
that distinction doesn't exist yet at RED.

* feat: green — defer auto-wake runs for classified-read-only server-mode opens (refs be-vbhpf)

bd ready/bd list on a dolt-sql-server-backed store open with the store
classified read-only for command purposes; the lazy defer-wake sweep
silently early-returned before ever running, so expired defers never
woke unless something else opened the store writable first.

Threads a new ClassifiedRead flag through dolt.Config -> DoltStore,
set at the call site to exactly "read-only because of command
classification, not strict --readonly/preview/foreign-project"
(policy.readOnly && !readonlyMode && !previewMode in main.go).
wakeExpiredDefers's guard now delegates to a small extracted
predicate, deferWakeSweepEligible(readOnly, classifiedRead), so the
truth table can be unit-tested without a live store.

* chore: release gate PASS for be-6iglh-gate

* docs(release-gates): record the be-5fjqh rebase onto current main

origin/main advanced 6 commits past this branch's base and PR #6082 went
CONFLICTING; hourly pr-audit filed be-5fjqh. The branch is now rebased onto
0efe0adf5.

The only conflict was cmd/bd/main.go: this branch and e05000d64 (wy-sbgucn)
each append a field to the same dolt.Config literal at the same anchor line.
Resolved keep-both -- PoolReadTimeoutFallback and ClassifiedRead drive
disjoint code paths.

The addendum records the new base and SHAs, patch-id equivalence for the five
files that did not conflict, what was re-verified at the new base (build plus
the diff-owned tests, container-backed), and what was NOT re-run (the
full-suite and ci-pr-policy lanes still refer to the old base).

Agent-Signature: claude-code-claude-opus-5[1m]-unknown-reasoning on behalf of Jim Wordelman

* docs(release-gates): record the be-qjern re-propose rebase and full re-gate

PR #6082 was closed and its branch deleted 2026-09-01 by an unknown actor
under the shared quad341 identity. Operator ruling 2026-09-08: accidental,
not a rejection — re-proposing. This addendum records a second self-rebase
onto current origin/main (zero conflicts this time), the full-suite and
ci-pr-policy re-gate the 09-01 addendum left outstanding (0 FAIL, policy
FAIL attributed to pre-existing be-mrl6j), and re-verification of both
diff-owned container-backed tests at the new HEAD.

refs be-qjern

* test(feat): red — add strict --readonly server-mode subtest to TestServerModeDeferAutoWake (refs be-acsyx)

* docs(release-gates): green — reword container-backed evidence claims, add nit-3 comment (refs be-acsyx)

bee-ghosttrack nit 2: TestDeferWakeSweepEligibleHonorsClassifiedRead is a
pure-function table test, not container-backed like TestServerModeDeferAutoWake
— reworded release-gates/be-6iglh-gate.md criteria 2(4) and 3 accordingly, and
attributed the preview/foreign-project claim to code structure at
cmd/bd/main.go:1485 rather than to that test. PR #6424 body reworded to match
(not part of this commit; edited via gh pr edit).

bee-ghosttrack nit 3 (optional): one-line comment on the unreachable
"classifiedRead without readOnly" table case explaining it documents totality,
not a reachable state.

Full suite: 0 FAIL. Targeted re-verification (TESTCONTAINERS_RYUK_DISABLED=true
to dodge host-load reaper flakiness): TestServerModeDeferAutoWake 5/5 subtests
PASS (12.427s), TestDeferWakeSweepEligibleHonorsClassifiedRead 4/4 subtests
PASS (5.957s).

* chore: remove leaked internal release-gate doc from PR branch

  release-gates/be-6iglh-gate.md

Tracked our internal gc-management/beads-rig release-gate checklist and
should never have been committed to this upstream branch -- it's process
residue from our deploy tooling, not part of the change itself.

Per maintainer bee-ghosttrack's request during review of PR #6571. The
root cause (our deploy tooling auto-committing this file) is already fixed
as of gm-qatxl1; this branch predates that fix, so this commit is cleanup
only -- no other content on this branch is touched.

Ref: be-lk7o3

* ci(proxied-shard): discover TestServerMode* entry points

Review finding (major, Test Evidence Quality): TestServerModeDeferAutoWake —
this PR's only end-to-end guard for the classified-read defer-wake fix — was
selected by no CI lane. The two lanes that set BEADS_TEST_PROXIED_SERVER=1
both shard through proxied-test-shard.sh, whose discovery grep matched only
`^func TestProxiedServer`, so the wiring lines at cmd/bd/main.go:1675 and
internal/storage/dolt/store.go:2136 had zero CI-executed coverage: the one
CI-reachable new test pins the pure predicate and cannot catch a mis-wired
flag.

Widen discovery to `^func Test(ProxiedServer|ServerMode)` in the shard script
and in scripts/ci/gen_proxied_shard_manifest.py so the two discovery patterns
stay in step, and widen the shard script's manifest-name validation to match.
That last part is load-bearing, not cosmetic: without it a regenerated
manifest containing a TestServerMode* row is rejected outright ("Invalid
shard manifest line") and the whole lane exits 1.

This also rescues the two pre-existing TestServerModeServe* suites, which
share the identical requireSharedProxiedServer gate and the same shared dolt
sql-server harness and were equally invisible — the unclosed class this PR
was following.

Verified: discovery goes 151 -> 154; a 15-shard BEADS_TEST_SHARD_LIST_ONLY=1
dry run assigns each of the 154 to exactly one shard with no pre-existing
test moved (the three land on shards 4/13/15 via hash fallback); the
pre-change script selects none of the three.

Also clarifies the reviewers' nit: the "foreign-project" exclusion named in
the new comments is not tested at the guard or in the policy expression —
those opens build their own dolt.Config and leave ClassifiedRead false, so
the guarantee is fail-closed by default.

Iteration-2 review (two minors, both non-gating): the fixup's own new prose
asserted two universals that measurement falsifies — the same class as the
nit above, reintroduced in the CI files this fixup added.

  * proxied-test-shard.sh claimed "Both families gate on
    requireSharedProxiedServer", contradicted two lines earlier by its own
    mention of the TestProxiedServerPathHelpers unit test. Transitive gate
    census over the 154 discovered entry points: 143 reach
    requireSharedProxiedServer, 10 reach the different helper
    requireProxiedServerEnv, and TestProxiedServerPathHelpers
    (cmd/bd/proxied_server_test.go, bare `package main`, three assert.Equal
    calls on path helpers) has no gate and no build tag, so it also runs in
    every ordinary `go test ./cmd/bd/` lane. Now scoped to the
    TestServerMode* family, which does gate uniformly (3/3), and the
    TestProxiedServer* side is described as mixed rather than generalized
    over.

  * main.yml claimed TestServerMode* discovery means "neither family can be
    silently left out of CI". Discovery is a non-recursive cmd/bd/*_test.go
    glob: 154 top-level matches versus 156 recursive, the difference being
    cmd/bd/doctor/database_test.go:60 and cmd/bd/doctor/dolt_test.go:126.
    Now scoped to the top-level entry points, and the comment names the
    residual hazard (a container-gated suite added under cmd/bd/<subpkg>/ is
    still never selected here) instead of implying it cannot happen.

Both edits are comment-only and behaviorally inert: non-comment lines are
byte-identical in both files, and the 15-shard BEADS_TEST_SHARD_LIST_ONLY=1
sweep is byte-identical before and after (154 assignments, 0 duplicates,
assigned set == discovered set, the three TestServerMode* still at 4/13/15).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* ci(bazel): add defer_wake_server_test.go to cmd/bd go_test srcs

Maintainer CI repair, generated by `make bazel-sync` (gazelle).

`cmd/bd/BUILD.bazel`'s `go_test` srcs is an explicit list, not a glob, so the
new `cmd/bd/defer_wake_server_test.go` was never compiled or run by the Bazel
lane. `Bazel / test` failed two required steps on the pushed head
`0a3f5e0047`:

  Go test equivalence:
    go test:  105 packages, 8988 top-level tests
    bazel:    107 go_test targets, 106 tested; 8987 top-level tests seen
    tests go test runs that Bazel did not (1):
      cmd/bd  TestServerModeDeferAutoWake
    EQUIVALENCE: FAIL

  BUILD files in sync (gazelle, go_srcs, MODULE.bazel): out of sync

`BAZEL_TEST` is listed in `CI_GATE_REQUIRED` for `CI Gate / Required`
(.github/workflows/pr.yml), so this is a required-lane red, not advisory.

That the missing test is this PR's own headline guard makes the repair squarely
in scope: the PR exists to close exactly this "no CI lane runs it" gap, and its
review fixup closed it for the proxied shard lane. The Bazel lane had the same
gap for a different reason.

Attribution: main's Bazel run at this PR's exact base 43a526b828 concluded
success, so the red is this branch's, not inherited.

Validation:
- `make bazel-sync` produces exactly this one line; a second run is a no-op
  (gazelle idempotent, md5 7d1d0549fcd466159451c02181661c78).
- Byte-identical to CI's own `bazel-sync-patch` artifact for this head
  (blob pair efec974..6dd76e3).
- srcs-only change: gazelle added no deps entry, so every import the new file
  needs was already declared.
- `bazel build //cmd/bd:bd_test` succeeds, and the generated
  `bd_test_/testmain.go` now enumerates TestServerModeDeferAutoWake, which is
  the row the equivalence check reported missing.
- `tools/bazel/go_srcs.py --check` was already clean, and `bazel mod tidy`
  produced no MODULE.bazel drift, so this single line is the whole repair.

No contributor commit is modified and no Go source changes.

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 15:53:22 -07:00
Julian Knutsen
90b84ea150
ci: auto-push Bazel BUILD sync fixes to PRs; harden docs autofix (#6907)
* ci(bazel): auto-push `make bazel-sync` fixes to PR branches

bazel.yml's BUILD sync step now writes the BUILD.bazel / MODULE.bazel /
MODULE.bazel.lock part of its drift as the bazel-sync-patch artifact
(scripts/ci/bazel-sync-patch.sh) and still fails. The new bazel-autofix.yml
workflow_run job (on "Bazel" and "PR") applies it with
scripts/bazel-autofix-push.sh: a sync commit on same-repo PR branches, an
idempotent apply-recipe comment on fork PRs.

Same security model as docs-autofix: the write-permission side never checks
out or runs PR code, extracts only the two expected regular files, and
refuses any patch naming a path outside an anchored allowlist (no hidden or
`..` segments, nothing under third_party/) or carrying a mode change,
symlink, rename or binary hunk; `git apply --index` and a post-apply check
back that up. Reuses the DOCS_AUTOFIX_TOKEN secret for the push.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): add bazel_autofix_push_test.go to scripts_test

`make bazel-sync` output, produced by scripts/ci/bazel-sync-patch.sh and
applied with scripts/bazel-autofix-push.sh as an end-to-end exercise of the
new autofix path.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): harden the bazel and docs autofix push scripts

Addresses the security review of the autofix pipeline, identically in
scripts/bazel-autofix-push.sh and scripts/docs-autofix-push.sh (the
security-critical functions are now shared verbatim):

- validate_patch refuses ANY "rename "/"copy " header (the legacy
  "rename old/new" form slipped past and, on docs-autofix, pushed a
  scripts.sh -> docs/cli-reference/x.md rename), checks every "index "
  line whatever its ids, takes creations/deletions/renames from
  `git apply --summary`, and reads names from `--numstat -z` (which
  names only the NEW path of a rename; the old comment was wrong).
- docs-autofix gains the post-apply staged re-check and a --check mode.
- The patch is applied to an index in a bare clone (read-tree +
  apply --cached + commit-tree): the PR tree never touches the disk.
  All git calls run with core.hooksPath=/dev/null, GIT_LFS_SKIP_SMUDGE=1.
- Push is leased: --force-with-lease=refs/heads/$HEAD_BRANCH:$HEAD_SHA.
- The PR must target this repo, and the head branch must be neither
  protected nor under a ruleset per the GitHub API (fail closed), on top
  of the name list.
- Only a marker comment written by github-actions[bot] is ever edited.
- bazel-autofix only pushes files in packages the PR changed
  (merge-base..head, computed from fetched refs), MODULE files only when
  go.mod/go.sum/MODULE changed; base-branch drift is left alone and
  listed, and a patch with nothing attributable gets a comment only.

Tests cover each reviewer attack for both scripts (--check and the full
flow against a local bare remote and fake gh), the staged check alone
with the validator disabled, spoofed comments, protection/ruleset/API
errors, the lease, main drift, and symlinked PR trees.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): trigger bazel-autofix on Bazel only; share job-level concurrency

- bazel-autofix.yml triggers only on "Bazel": pr.yml never calls
  bazel.yml, so "PR" runs never carry the artifact, and their completions
  cancelled real fixes through the workflow-level group. The policy test
  now requires the triggering workflow to run on pull_request and upload
  the consumed artifact instead of asserting the false claim.
- Both autofix workflows use one job-level concurrency group per head
  branch (autofix-<head repo>-<head branch>) without cancel-in-progress:
  skipped runs take no part, and the two bots queue instead of racing.
- Base checkouts use persist-credentials: false; docs-autofix extracts
  only cli-docs-freshness.patch, as a regular file, outside the workspace.
- Policy tests for the trigger, concurrency scope, checkout, and the
  scripts' shared guards (rename/index checks, comment author, lease,
  protection lookup, index-only apply, hook-free git).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): tighten autofix attribution, circuit breaker and PR selection

Re-review follow-ups for the bazel and docs autofix push scripts:

- Root package: paths outside any Bazel package (docs/, .github/,
  README.md, ...) no longer count as changing //:. Only root-level Go and
  build inputs (*.go, *.s, *.c, *.h, *.bzl, go.mod, go.sum, BUILD.bazel,
  MODULE.bazel*) make root BUILD.bazel drift the PR's, so main's root
  drift is not pushed onto a docs-only PR.
- Circuit breaker: the head subject is read from the fetched commit (no
  API), a failed read counts as non-convergent, and the check runs only
  once the PR's own staged fix is non-empty. A bot head whose remaining
  drift is main's now gets the "files this PR did not change" comment
  instead of a wrong "did not converge" one. Same in docs-autofix.
- PR selection: several open PRs can share one head branch. bazel picks
  the one whose number matches the artifact metadata (among PRs already
  matching head repo/branch/base repo; head sha still verified); without
  a unique match both scripts skip instead of taking .[0].
- The privileged clone fetches only the head (and, for bazel, base)
  branch into an init'ed bare partial repo instead of cloning every
  branch.
- docs-autofix job gets timeout-minutes: 10 (it shares a non-cancelling
  concurrency group with bazel-autofix).

Tests: docs-only/README/workflow/Makefile PRs with root drift get no push,
a root Go change gets root BUILD.bazel; breaker cases (bot head with own
vs main drift, unreadable subject) for both scripts; PR selection by
metadata and ambiguity; HEAD_SHA 40-hex guard; staged-check mode arm; a
--summary-only validator copy; both workflows' download steps run as
written against a fake gh (run-scoped artifact listing, only the named
files extracted, symlinked entries refused); timeout asserted for both
jobs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): bazel-autofix requires bash 4; skip its tests on macOS bash 3.2

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(ci): skip every autofix script test on bash < 4

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): sync internal/utils BUILD with main

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 17:23:27 -07:00
Bùi Phan Viết Cường
5cc896c8f0
fix(lint): suppress two darwin-only gosec findings and cross-lint the darwin build in CI (#6243)
* fix(lint): suppress two darwin-only gosec findings and cross-lint the darwin build in CI

scripts/ci/pr-lint.sh lints natively on the ubuntu runner plus one
GOOS=windows CGO_ENABLED=0 cross-lint, so files behind //go:build darwin
(and the !windows && !linux fallbacks) were never linted in CI. Linting the
darwin build reports two gosec findings that are clean on linux:

- internal/config/permissions_open_nonlinux.go: G304 on os.Open(path). The
  file comment already explains why it is safe (fixBeadsDirPermissions
  re-checks descriptor identity before chmod); it just had no directive.
- internal/utils/path_case_darwin.go: G103 on unsafe.Pointer. A directive
  existed but sat on the first line of a multi-line unix.Syscall call,
  while gosec reports the argument line four lines down, so it never
  covered the finding. It now sits on its own line above the statement,
  the placement cmd/bd/nocow_linux.go already uses.

Add a GOOS=darwin GOARCH=arm64 CGO_ENABLED=0 cross-lint block to
pr-lint.sh mirroring the windows block (same flags, same lane scoping,
skipped when the native target already matches), and update the three
places that described the wrapper as two passes: engdocs/LINTING.md,
engdocs/CI_CLEANUP_PLAN.md and the comment in main.yml.

The darwin pass is compile-only on the Linux runner: no macOS runner, no
change to the pr-lint platform column. Like the windows pass it is
CGO_ENABLED=0, so darwin+cgo code stays outside its reach.

* docs(linting): bump the Last reviewed marker alongside the pr-lint change
2026-09-28 15:44:52 -07:00
ecuthiell
4dc96f6f21
fix(config): identify native config environment in path errors (#6325)
* fix(config): name the native config environment source in errors

Identify APPDATA on Windows, HOME on Darwin/iOS, home on Plan9, and the
selected XDG_CONFIG_HOME or HOME source on other Unix hosts. Only error
labels change; the standard-library resolver and path precedence remain
the same, and the original wrapped error is retained.

Fixes #6314
Reported-by: bee-ghosttrack
Bead: saa-cgh.95
Agent-Signature: unknown-model/unknown-reasoning

* test(config): exercise native source diagnostics on the required platforms

Select the diagnostic parent and its two XDG states through one Bash3.2
driver in the existing three-host preflight matrix. Bind the declared OS
to Bash and the setup-selected Go host/target, retain child process status,
and reject missing, duplicate, unexpected, failed or skipped execution.

Preserve existing platform steps, job identity and required gate mapping.
Use the existing YAML workflow helpers to pin the new placement.

Follow-up: https://github.com/gastownhall/beads/issues/6314
Bead: saa-cgh.95
Agent-Signature: unknown-model/unknown-reasoning

* fix(config): keep native diagnostics with candidate builder (saa-cgh.95)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell
2026-09-28 12:17:20 -07:00
Julian Knutsen
fc4d7d3958
ci(bazel): PR-gate parity lanes (docker, pure-Go, js/wasm) + bd artifact (#6887)
* test(bazel): require exclude read-permission coverage in test:prcore

pr.yml's PR Core step sets BEADS_TEST_REQUIRE_EXCLUDE_PERMISSION=1 so
TestAddExcludePatternsRefusesReadErrors fails, rather than skips, on a root
or non-POSIX executor. test:prcore now sets it too, and a policy test keeps
the two in step.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(bazel): compare Bazel runs with one CI job's go test results

equivalence.py --job takes a job's `go test -json` as the complete expected
set, so a Bazel selection can be shown to replace that job: every top-level
test the job ran must run under Bazel, no other test in those packages may,
and nothing the job passed may be skipped. --bep is repeatable (a job can map
to --config=ci plus --config=docker), and go_test_variant.sh sh_tests count,
since their go_test binary writes the same test.xml. In job mode a test seen
in several targets keeps its best status: the prcore run skips what the
docker variant exists to run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): docker-lane variant for internal/tracker

pr.yml's "Test (storage domain + uow)" job runs ./internal/tracker/... with
-race and without -short or BEADS_TEST_SKIP=dolt, so tracker's TestMain
starts the shared dolt container and all 112 top-level tests run. Under
--config=prcore they skip. tracker_docker_test reruns tracker_test that way
in the docker lane, and tracker's TestMain now fails instead of skipping when
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 and the container does not start.

The job's other packages (storage/domain, domain/fs, domain/git,
tracker/conformance) consult neither testing.Short nor the dolt skip, and
pass identically under --config=ci.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): publish the Bazel-built bd as a ci-build-artifacts artifact

pr.yml's build-artifacts job uploads ci-build-artifacts (bd-linux-gms-pure,
SHA256SUMS, build-manifest.txt), which PR Core, the domain+uow job and both
package gates download and check with `sha256sum -c`. The Bazel lane now
uploads the same name and layout from //cmd/bd:bd_for_tests, the non-race
gms_pure_go cgo bd its own tests exec (//cmd/bd:bd is race-built under
--config=ci); test:ci downloads it under --remote_download_minimal. No
consumer is repointed yet.

A rules_go binary has no vcs.* or CGO_ENABLED build settings: `bd version`
shows no commit, and scripts/verify-cgo.sh would pass it vacuously. No
consumer of this artifact reads either; the manifest records the commit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): pure-Go and js/wasm lane for pr.yml's boundary job

pr.yml's "Check pure-Go and js/wasm boundaries (CGO_ENABLED=0)" job builds
bd and three test binaries with CGO off, runs a 16-name -short cmd/bd subset,
and runs the js/wasm hook refusal test under node with an exact count guard.
bazel.yml's new bazel-pure job does the same with Bazel binaries:

- --config=pure (--@rules_go//go/config:pure, -test.short) builds //cmd/bd:bd,
  bd_test, embeddeddolt_test and tracker_test, and runs bd_test unsharded
  with the job's -run selector (policy-tested to stay equal to pr.yml's).
- --config=js-wasm builds //internal/hooks:hooks_test for js/wasm
  (--use_target_platform_for_tests satisfies the test toolchain; nothing is
  tested by Bazel under it); the Go SDK's go_js_wasm_exec runs it under node
  with pr.yml's selector and count guard, verbatim.

rules_go compiles every declared dep, and gazelle's deps ignore the cgo
constraint of beads' embedded-Dolt files, so a pure build compiled dolt's
store/nbs and failed in github.com/dolthub/gozstd (cgo only; go build never
imports it with CGO_ENABLED=0). third_party/patches/gozstd_nocgo.patch adds
panicking !cgo stubs for the API dolt uses.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): run prcore tests with -test.parallel=4 like pr-core.sh

pr-core.sh runs go test -parallel 4. test:prcore passed no -test.parallel,
so Go defaulted to GOMAXPROCS: the executor's core count (192 on a farm
worker) instead of a runner's 4, which let dozens of t.Parallel bd
subprocesses run at once. A policy test keeps the value equal to
pr-core.sh's GO_TEST_PARALLEL default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): -test.parallel=4 in the pure lane

pr.yml's pure-Go job passes no -parallel, so go test uses GOMAXPROCS, 4 on
ubuntu-latest; under Bazel the default would be the executor's core count.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): run the contract corpus non-race in the docker lane

pr.yml's Contract corpus job runs go test ./cmd/bd/protocol without -race,
but --config=docker races every target. protocol_docker_test now runs
:protocol_race_off, protocol_test built through a race=false transition
(tools/bazel/race.bzl), so no second go_test rule has to track gazelle's
srcs. Tagged manual so other lanes do not build it; the extra non-race work
is protocol_test's link plus ~100 test-only libraries.

The docker lane also mirrors the jobs' concurrency now: they pass no -p or
-parallel, so go test uses GOMAXPROCS, 4 on ubuntu-latest, for both; the
lane runs at most 4 test processes (so at most 4 dolt containers start at
once) with -test.parallel=4. It writes -test.v test.xml like test:ci, for
equivalence.py.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the docker lane in bazel.yml

New advisory job bazel-docker runs `bazel test //... --config=docker` on
ubuntu-latest after pre-pulling the dolt image like pr.yml's container jobs.
The requires-docker variants (domain/db, uow, tracker, doctor/fix,
protocol) run on the runner against its docker daemon (no-remote-exec);
their builds use remote execution when setup-bazel configured it. With
bazel-test's --config=ci this covers "Test (storage domain + uow)" and
"Contract corpus". The lane name is one env var, so a hermetic dolt-server
lane can replace it.

Policy tests now cover every bazel.yml job: an exact job list, timeouts, no
continue-on-error, pinned actions (setup-node added), one setup-bazel step
per job as the only secret reader, cache saves only in bazel-test, plus
job-specific mirrors of pr.yml (the docker job's runner, image pull and
lane; a requires-docker target in each container-backed package).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(bazel): build the non-race contract corpus in bd_for_tests' configuration

go_test_race_off only set race=false, while rules_go's go_transition (as
used by //cmd/bd:bd_for_tests, race = "off") also records the overridden
value in original_race. The two non-race configurations differed, so the
docker lane (and --config=ci, which builds protocol_docker_test through
//...) compiled the whole dolt graph twice without race. Record
original_race the same way: aquery GoCompilePkg for
deps(protocol_docker_test) under race drops from ~2630 actions in two
configurations to 1359 in one.

Fix the BUILD comment: manual does not keep protocol_race_off out of other
lanes' builds; protocol_docker_test pulls it in.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(bazel): job-mode equivalence fails on any target's failure or missing test.xml

In --job mode a test seen in several targets keeps its best status, so a
failure in one variant was hidden when another target passed the test, and
a go_test_variant.sh sh_test without a test.xml was skipped silently. List
every failed test per target and every target in the job's packages whose
test.xml is missing, and fail on either.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(bazel): gate pure-lane contamination, never cache docker-lane results

- gozstd_nocgo.patch: the stubs' package init panics, so a pure binary
  that links gozstd (which CGO_ENABLED=0 go build rejects at compile time)
  fails at startup. bazel-pure starts every pure artifact (bd version, and
  embeddeddolt_test/tracker_test with -test.run=^$; bd_test runs its
  subset) so contamination fails the lane.
- test:docker --nocache_test_results: the requires-docker variants depend on
  host state no action key covers, like the container jobs' -count=1.
  build:remote-exec --noremote_upload_local_results in the committed rc, so
  local runs cannot seed the shared cache with host-dependent results.
- The artifact packaging moves to scripts/ci/package-bazel-bd.sh, which
  sources .buildflags itself; sourcing it in bazel.yml exempted the whole
  workflow from the build-tag scan.
- bazel-pure and bazel-docker use bazel-test's runs-on (Blacksmith when
  remote execution is possible); the policy test pins it for every job.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): put the pure and docker lanes behind the rbe-west gate

Apply bazel-test's rbe-west setup-bazel env (RBE_WEST_EXECUTOR, instance
oss, RBE_WEST_WORKERS) and its job if to bazel-pure and bazel-docker, so
same-repo runs of all three lanes skip together until the variable is set
and forks and rbe=off dispatches still run locally. The policy test pins
runs-on, if and the setup-bazel env for every bazel.yml job.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): run the js/wasm hook test with a minimal environment

wasm_exec.js caps argv+env size, and the Blacksmith runner's environment
exceeds it ("total length of command line and environment variables
exceeds limit").

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 02:03:51 -07:00
Julian Knutsen
6da2a43477
ci(bazel): advisory Bazel workflow + go test/Bazel equivalence check (#6881)
* ci(bazel): add advisory Bazel workflow and setup-bazel action

Add .github/workflows/bazel.yml, a side-by-side lane that runs
`bazel test //... --config=ci` (PR Core parity) on pull_request,
merge_group, push to main and workflow_dispatch. It is advisory: not in
ci-gate's CI_GATE_REQUIRED.

.github/actions/setup-bazel installs a sha256-pinned Bazelisk (Bazel from
.bazelversion), restores the runner cache (--disk_cache and
--repository_cache only), and writes a CI bazelrc outside the workspace
and outside the cached directory. Remote execution is enabled only when
BAZEL_REMOTE_EXECUTOR, RBE_TLS_CERT and RBE_TLS_KEY are all set (base64
PEM cert/key; optional RBE_TLS_CA, RBE_INSTANCE); none set (fork PRs, or
workflow_dispatch rbe=off) runs locally; a partial set fails. The endpoint
and its bare host are masked. The rc logic lives in write-bazelrc.sh so it
is testable offline.

The runner cache is saved only by push-to-main runs under a run-unique key
and restored by prefix, so it never freezes at its first save and PRs do
not churn the quota. The job also reports the tier (T1/T2/local) and
wall time in the step summary, runs tools/bazel/critpath.py against the
profile, and checks BUILD drift with make bazel-sync-check plus
make bazel-sync and a clean git status.

tools/bazel/critpath.py reports the critical path as the sum of the
chained "critical path component" events rather than the longest one (a
cold race build: 395s over 75 components, where the single-component
version printed 69s and a false scheduling gap), lists the longest
members and Bazel's processes: cache line, and checks a report-only
budget (warning annotation, exit 0) unless --enforce is given.

scripts/ci_workflow_test.go covers the new lane: not wired into any gate,
every action SHA-pinned to the repo's released SHAs, no monolithic
actions/cache, no unbounded apt-get, the cache topology, secrets only
reaching setup-bazel, --profile after `test`, and the rc writer's
none/partial/full/invalid-PEM/secret-dir-in-workspace behaviour.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): check go test / Bazel test equivalence in the Bazel lane

tools/bazel/equivalence.py compares the Go tests `bazel test //...
--config=ci` ran with the ones pr.yml's go test lanes run, so a tag
filter, a stray manual tag, a package without a go_test target, or a
shard that runs zero tests cannot shrink the Bazel lane while it stays
green:

- package set: every package `go list -race -tags gms_pure_go` reports
  with test files has a go_test target (bazel query);
- test-name set: every top-level TestXxx in the files go list selects
  (minus ^TestEmbedded, as PR Core skips) appears as a testcase in the
  test.xml of a go_test target this invocation tested (from the BEP);
  runtime skips count as run. Tests Bazel ran that go test would not
  are divergences too.

Expected differences go in tools/bazel/equivalence_allowlist.txt, one
justified line each (empty today); unused entries are reported.
Results go to the step summary; the full listing with --details.

.bazelrc test:ci now sets GO_TEST_WRAP_TESTV=1 (rules_go lists passing
tests in test.xml only under -test.v) and downloads test.xml despite
--remote_download_minimal.

The Bazel workflow runs the check after the critical-path report, and
nightly.yml calls the workflow (suite "bazel", and on schedule) with the
full listing.

Verified locally against a remote --config=ci run: 103 packages and
8843 tests on both sides. Tagging //internal/gitignore:gitignore_test
manual keeps `bazel test` green but fails the check, listing its two
tests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(fs): run RepoContextAndRole from its own git repo

The test relied on its working directory being inside a git checkout to
make BEADS_DIR external to the CWD repo. Under Bazel the working
directory is the runfiles tree, not a git repository, so RepoContext
failed with "not a git repository" and the test skipped itself.
Create a second temporary repo and chdir into it instead; go test keeps
passing and Bazel now runs the assertions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(bazel): fail equivalence on tests Bazel skips but go test passes

The equivalence docstring claimed runtime skips count as run because go
test skips them the same way. It does not: against PR Core's go test
-json on the same commit, 11 tests passed under go test and skipped
under Bazel while the check printed ok with an empty allowlist.

- equivalence.py --go-test-json compares per-test status with the
  Bazel test.xml: passed under go test and skipped under Bazel is a
  divergence; other status differences are listed, not failed. The
  in-job (PR) run cannot see skip divergences and says so.
- The allowlist gains a `skip` kind (`<pkg> <test> skip  # why`, the
  justification still mandatory) and lists the 10 intentional skips:
  cmd/bd TestGOMODCACHENotUnderTestHome and TestZZStdioNotLeaked, and
  the scripts policy tests that walk the source checkout. The eleventh,
  fs TestContextRepository_RepoContextAndRole, is fixed in the test.
- scripts/ci/pr-core.sh writes `go test -json` to
  BEADS_PR_CORE_GO_TEST_JSON when set (same flags otherwise).
- nightly.yml runs that in a new job, uploads the JSON, and calls
  bazel.yml with go-test-json-artifact, which downloads it and passes
  --go-test-json to the equivalence step.

Verified on a remote `bazel test //... --config=ci` run: ok with the
allowlist; a TEST_SRCDIR-only t.Skip injected into
internal/gitignore TestAppendLines fails the check (and passes the
name-only in-job check, as documented).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): freeze the runner cache per lock, mask key material, tighten policy

Runner cache: every push to main saved ~2.3 GB (repository cache, disk
cache, Bazelisk) under a run-unique key, on a repo already over its
Actions cache quota. The cache now holds only the repository cache and
Bazelisk, whose content is fixed by .bazelversion and MODULE.bazel.lock:
exact key bazel-repo-v2-<os>-<hash>, restore falls back to the previous
lock's entry, and only a push to main that missed the key saves. No
--disk_cache in either mode (remote runs have the remote cache; local
runs never used it). About 2 GB per lock change.

Credentials:
- write-bazelrc.sh masks every non-armor line of each decoded PEM
  (cert, key, CA). GitHub only masks the base64 value as stored. The CA
  used to be decoded inside the rc redirect, so its mask commands would
  have gone into the rc; it is decoded first now.
- The CI rc sets build:remote-exec --noremote_upload_local_results, so
  CI never writes locally executed (PR-code) results into the shared
  action cache. Developer rcs keep Bazel's default.
- RBE_INSTANCE stays optional; an empty one is a ::warning.
- bazel.yml's header states the accepted risk: same-repo PR code runs
  with the RBE client certificate (as in gascity); forks never get it;
  what is in place and what is pending a human decision.

Policy tests (each proved with a scratch mutation that now fails):
triggers must be exactly pull_request, merge_group, push,
workflow_dispatch and workflow_call (no pull_request_target); secrets
may appear only in the setup-bazel step's env, not in workflow or job
env, run, with or if; no continue-on-error anywhere in bazel.yml or
setup-bazel; the cache-save if is matched exactly; the critical-path
and equivalence steps must end with the bare script call and the test
step with exit "$rc", with no exit 0, || true, || :, ; true, set +e or
trap.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(ci): record the same-repo RBE decision in bazel.yml

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): bazel wrapper version check; keep go-test prose out of the build-tag scan

The wrapper prepends --bazelrc, so 'bazel --version' reached the server
as an unknown startup option; ask bazelisk directly. check-build-tags
matched 'go test' in workflow prose and step names.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): run the RBE lane on Blacksmith runners; mask the executor's IPs

GitHub-hosted runners are refused by the remote-execution edge; the
Blacksmith pool (as in gascity) is admitted and does not count against
GitHub's concurrent-job cap. Forks and rbe=off keep ubuntu-latest.
Bazel prints the resolved executor address in connection errors, so mask
every address the endpoint host resolves to.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): target rbe-west (instance oss) behind RBE_WEST_WORKERS

The original edge does not admit CI runners. Mirror gascity: remote
execution uses secrets.RBE_WEST_EXECUTOR and instance oss when the repo
variable RBE_WEST_WORKERS is true; same-repo runs are skipped until then
instead of failing, while forks and rbe=off still run locally.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 23:05:21 -07:00
ecuthiell
cefcabb1e3
fix(gitenv): retain explicit null config suppression (#6461)
* fix(exec): match Windows environment key identity (#5618)

Centralize os/exec-compatible environment-key comparison and use it for credential, metrics, Git-hook, trace, and repro subprocess filters. Preserve exact Unix semantics and Unicode near-collisions.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>

* fix(worktree): scrub inherited Git routing state

Establish CWD as the worktree repository boundary before discovery can consume inherited Git routing. Fence the startup config probe, clear routing in-process before root pre-run discovery, and retain command-owned subprocess scrubbing as defense in depth without changing PID or signal semantics.

Fixes #5531

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell

* refactor(worktree): explain owned filtering and remove an unused shim

Document why filtering the fresh routing slice in place is safe and call
the shared routing-key classifier directly from tests. Retain the two
forwarders used by production worktree subprocess/removal paths.

Review: https://github.com/gastownhall/beads/pull/5623#pullrequestreview-5098421671
Bead: saa-cgh.23
Agent-Signature: unknown-model/unknown-reasoning

* fix(config): isolate identity and tracked-file Git probes from routing

Reuse the worktree routing owner for GetIdentity and isGitTracked so
explicit cwd/path selection is not displaced by inherited repository,
index or injected config state. Preserve identity precedence and the
tracked-secret refusal contract, with real target/decoy regressions.

Follow-up: https://github.com/gastownhall/beads/issues/6222
Dependency: https://github.com/gastownhall/beads/pull/5623
Reported-by: bee-ghosttrack
Bead: saa-cgh.93
Agent-Signature: unknown-model/unknown-reasoning

* test(execenv): exercise every environment operation under both host rules

Thread the host choice through narrow private helpers behind the existing
exported functions. Keep one normalization owner and retain native wrapper
tests; the additional table covers both host branches for equality, prefix
matching, last-wins lookup, filtering and input preservation.

Review: https://github.com/gastownhall/beads/pull/5672#pullrequestreview-5096948965
Bead: saa-cgh.78
Agent-Signature: unknown-model/unknown-reasoning

* docs(worktree): explain startup Git environment scope (saa-cgh.23)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(execenv): exercise exported host policy wiring (saa-cgh.78)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(config): share subprocess key identity for Git routing (saa-cgh.93)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(config): isolate role commands from Git routing (saa-cgh.99)

Follow-up to bee-ghosttrack's review of gastownhall/beads#6322; refs #6357.

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(env): exercise shared key helpers on native Windows (saa-cgh.78)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* refactor(gitenv): share key parsing and normalize routing membership once (saa-cgh.93)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* docs(config): explain role command routing policy (saa-cgh.99)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(config): isolate role readers from inherited Git routing (saa-cgh.109.1)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(doctor): isolate role lookups from inherited Git routing (saa-cgh.109.2)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(git): isolate domain role configuration from inherited routing (saa-cgh.109.5)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(doctor): isolate explicit-path worktree fallback from Git routing (saa-cgh.109.6)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(init): probe the selected repository before configuring roles (saa-cgh.109.8)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(beads): cover common-directory role routing (saa-cgh.109.1)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(git): explain system config routing flags (saa-cgh.109.6)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* docs(init): explain role probe discovery ceiling behavior (saa-cgh.109.8)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(beads): scope stable worktree probes to selected path (saa-cgh.109.13)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(init): bind proxied tail Git operations to selected directory (saa-cgh.109.14)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* Clarify selected discovery probes and cover separate routing vectors (saa-cgh.120)

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* Keep init role operations on the captured selected Git context (saa-cgh.120)

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* Observe inline Git configuration directly in linked-worktree fixture (saa-cgh.120)

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* Align selected-tail fixtures with captured role authority (saa-cgh.120)

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* fix(beads): select explicit directory discovery context (GH#6384) (saa-cgh.120.6)

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* fix(gitenv): retain explicit Git config suppression (saa-cgh.120.7)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(config): pin identity under null config suppression (saa-cgh.120.7)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(gitenv): keep role authority off inherited config suppression

Maintainer review fixes for PR #6461 (adopt-pr iterations 1-2). Addresses the
three major findings raised against the contributor branch, plus the major,
three minors and nit that iteration 2 raised against the first fixup; each fix
is pinned by a test that was mutation-proved against the pre-fix source.

F1 (major, security) — the leaf exempts GIT_CONFIG_GLOBAL=/dev/null,
GIT_CONFIG_SYSTEM=/dev/null and GIT_CONFIG_NOSYSTEM from ScrubRouting. Null
suppression cannot redirect a read, but it can blind one, and DetectUserRole
answers a missed read with detectFromURL, whose no-remote and SSH arms both
return Maintainer. Reproduced on this branch: a global beads.role=contributor
with GIT_CONFIG_GLOBAL=/dev/null resolved to maintainer, where the leaf's
parent fb7a93ad4 scrubbed the entry and read contributor.

Taken the synthesis's alternative arm (a): keep the exemption where the value
has no authority, drop it where it does. New ScrubRoutingAndSuppression
applies at every beads.role reader — internal/routing, doctor/role.go,
config_show.go, RepoContext.Role, bd config get/set/unset/set-many, and the
domain adapter's key-specific arms (see A1: iteration 1 claimed that last site
and did not actually reach it). Arm (b), making roleFromGitConfig's miss
non-permissive, was declined: "no beads.role and no remote implies maintainer"
is documented base behavior for local projects, so closing it there would
reclassify every such user rather than the blinded slice this PR introduced.

F2 (major, blast radius) — the stack hardened every role reader and left four
writer/reader sites fully inherited, a divergence that did not exist at base:
an inherited GIT_DIR sends setBeadsRole's write to another repository while
the scrubbed readers look at the one the user is standing in. All four now
share the same boundary (init.go's getBeadsRole/setBeadsRole,
init_contributor.go, hooks.go's uninstall --get/--unset pair).

The required fix also asked for cmd.Dir = repoRoot on the init.go pair. Still
not taken, but not on the grounds iteration 1 recorded (see A2). What holds is
the measurement against the vector the fix targets: with GIT_DIR, GIT_WORK_TREE
and GIT_COMMON_DIR gone, no production path chdirs away from the invocation
directory — `grep -rn 'os\.Chdir' --include='*.go'` over production cmd/bd
returns nothing and `bd init` takes no path argument — so threading a repoRoot
through both signatures and their eight call sites buys no behavior change
against a redirect.

F3 (major, security) — this PR converted internal/config's isGitTracked probe
from inherited to scrub-only. The guard is fail-open by construction, so in a
legitimately routed checkout (bare repo.git, GIT_DIR/GIT_WORK_TREE exported,
no in-tree .git) the probe stopped answering and `bd config set
linear.api_key` would have written the credential into a tracked file.
Restored the two-probe shape sibling #6464 applied to the structural twin:
scrubbed probe first, inherited second, either "tracked" refuses.

F5 (minor) — disclose that scrubbing GIT_CEILING_DIRECTORIES widens upward
discovery rather than bounding it, in the worktree comment and the docs
paragraph that asserted only the narrowing effect. F2's fix extends that
trade-off to two more sites, so the disclosure is now load-bearing.

F7 (minor) — renamed isConfigSuppression to isConfigSuppressionControl and
documented why the NOSYSTEM arm is deliberately value-blind. Took the
rename/document arm rather than restricting the arm to git-true values, which
would have contradicted the PR's own pinned table.

F10 (nit) — pinned both suppression tables in the windows-latest required
list; verified each name matches the gate's grep exactly once.

--- Iteration 2 ---

The first fixup's F1 census named a site it did not reach, and its cmd.Dir
rationale named a mechanism that does not hold. Both are corrected here.

A1 (major, security) — the domain adapter's key-specific arm was guarded by
`key == "beads.role" && r.env == nil`, which is dead on the only production
role path: runInitProxiedServerTail rebinds the use case to
NewInitGitRepository whenever t.workDir != "" (its sole construction site
passes cwd), and that constructor captures ScrubRouting(os.Environ()) —
non-nil, suppression retained. The nil-env constructor has no
BeadsRole/SetBeadsRole caller, so the arm could only fire from tests, while
`bd init` under an inherited GIT_CONFIG_GLOBAL=/dev/null still missed a global
role, took hasRole=false, and persisted beads.role=maintainer into
repository-local config, where it outranks the global value for every later
read in a clean environment. Iteration 1 therefore left F1's escalation not
just alive but durable, on the one path the fix was supposed to close.

Fixed by keying the strict scrub on the config key alone and applying it to
whichever environment the repository already carries (roleAuthorityEnv). That
preserves capture semantics — NewInitGitRepository keeps its captured HOME,
the property git_test.go's captured_home case pins — and removes only the
suppression entries. Codex's alternative of rebuilding the constructor's whole
env was declined: it moves every key for no gain on this finding.

A2 (minor, contract fidelity) — iteration 1 justified declining cmd.Dir with
"repository selection is left to the working directory — the one input the
scrub keeps". That is false, and false in the file a future reader consults
before deciding whether to bound the pair: GIT_CEILING_DIRECTORIES is a routing
key too, so the scrub removes a bound rather than installing one. Measured from
a non-repository directory nested under a repository: `git config beads.role
<value>` fails "not in a git directory" while a ceiling below that repository is
inherited, and writes that repository's local config once the ceiling is
scrubbed. The decline itself stands, since the measurement behind it was made
against the GIT_DIR vector; getBeadsRole now records selection as the working
directory *or a containing ancestor*, which is also what makes
clearWorktreeGitRoutingEnv's existing cross-reference to "init's role probe"
true.

A3 (minor, consistency) — `bd config unset beads.role` and `bd config set-many
beads.role=` were the two spellings of that mutation still on ScrubRouting, in
the same file as the two that moved. Both measured inert: `git config <key>
<value>` and `--unset <key>` target the repository-local file, which the
suppression variables cannot select. Moved for parity rather than annotated,
because a divergence inside one file is how the next one starts.

Test evidence (minor) — the writer pair's two subtests both pass on the
redirect alone, so downgrading getBeadsRole back to ScrubRouting left them
green even though suppression is the load-bearing half at those sites
(init.go treats a getBeadsRole miss as permission to write a default role).
Added a global_null blinding subtest with the same unblinded fixture guard.

A4 (nit) — doctor/role.go's isGitRepo was the last probe in that file on the
suppression-retaining scrub, and it gates the "beads.role not configured"
warning. rev-parse is config-sensitive through safe.directory, which lives in
global config, so a retained GIT_CONFIG_GLOBAL=/dev/null can downgrade that
warning to "N/A (not a git repository)". Moved onto the same boundary as its
two siblings. As the finding records, the safe.directory precondition was not
reproduced end to end; the change is uniformity within the file.

Tests. Every pin reds only on its own defect:
- routing: reverting to ScrubRouting reds only the blinded-read subtest; the
  unblinded fixture guard and the PR's inject-vector table stay green.
- gitenv: keeping suppression reds 10 of 16 new subtests, none of the PR's.
- cmd/bd: restoring the inherited env reds both redirect subtests and names
  both halves ("the write did not land where bd is standing" / "an inherited
  GIT_DIR captured the write"), while the PR's own role roundtrip stays green
  — it followed the same redirect on both sides, so it never saw this.
- config: the two arms are proved separately. Scrub-only reds only the
  legitimate-routing subtest; inherited-only reds only the poisoned-routing
  subtest, so the new fallback cannot be read as a way around the scrub.
- domain/git (A1): restoring the r.env == nil provenance guard reds
  global_null only; unblinded stays green, and no other test in the package
  moves — which is the same measurement that confirms the arm had no coverage
  before this iteration.

Deferred to follow-up, recorded for the approval summary: F4 (delete
isRoleGitRepo and resolve the dead else-if), F6 (user.name degrading to
hostname, which needs the PR's own subtest re-pointed), F8 (execenv fold
divergence), F9 and F11 (GetIdentity hostname degradation under inherited
suppression). F9's mechanism is confirmed — identity supplied only via
GIT_CONFIG_COUNT is lost when GIT_CONFIG_* is stripped, git reports "Please
tell me who you are", while GIT_AUTHOR_*/GIT_COMMITTER_* survive — but its
"leave artifact commits on the inherited environment" arm would re-open the
GIT_DIR redirect this PR's tail swap closed, and the non-regressing arm
(re-injecting inline identity pre-scrub) is a design change with no observed
victim in this repo.

Validation (iteration 2): internal/{gitenv,routing,storage/domain/git} full
packages green; cmd/bd/doctor full package run; targeted cmd/bd run over the
role, config and init blast radius green at 54 tests; go build ./... clean; go
vet plus CGO_ENABLED=0 and GOOS=windows CGO_ENABLED=0 vet clean on every
changed package. Remaining local reds are base-parity, each matching the
recorded cause: cmd/bd/doctor's seven (the launcher's global
beads.role=maintainer short-circuits CheckBeadsRole before isGitRepo is
reached, plus Dolt server availability) and
TestInitContributorSetsBeadsRoleContributor ("--contributor requires
interactive prompts", this session's stdin is not a TTY).

--- Iteration 3 ---

Iteration 3 reviewed 0220eed32 and approved it: claude filed no findings,
codex's six fix validations all came back correct, synthesis recorded
Decision: approve, and the scorecard scored 881/1000 with Required Changes:
None. The residual was two minors and a nit, none of them merge gates. All
three are taken here because they are comment-and-constant scope, so nothing
in the approved behavior moves.

B1 (minor, blast radius; codex filed it as major, demoted at synthesis) — the
isGitRepo() gate that decides whether the role pair runs is still fully
inherited while the pair it guards is now strict, so from a non-repository
directory with an inherited GIT_DIR the gate opens, the scrubbed read misses,
and the safety-net write fails "not in a git directory". Recorded at the
GH#2950 safety net rather than restructured, for the three reasons the
demotion rests on. The achievable outcome is base parity: sync_git.go is
byte-identical at base and the pair was inherited there too, so a GIT_DIR
holder already reached Maintainer before this PR — more directly, by pointing
at a repository whose beads.role is maintainer. The failure is not silent:
init prints "Warning: failed to set beads.role" unless --quiet, and every
later detection prints "beads.role not configured (GH#2950)" with two Fix
lines. And the prescribed roleRepoAvailable() gate provably does not close it:
all five gate sites are bare `if isGitRepo()` with no else, so a strict gate
answers false in exactly the described band, nothing is written, the role
stays unset, and detection still falls to detectFromURL. The optional
hardening is therefore declined rather than deferred — it would delete a
warning and change no role — and the comment says so, so the next reader does
not mistake a gate change for closing the fail-open path.

B2 (minor, contract fidelity) — the getBeadsRole rationale iteration 2 wrote
to close A2 introduced a new overstatement in its closing sentence: "bd init
takes no path argument — the residual is upward discovery from an unintended
cwd, not a redirect". `bd -C <dir>` is a redirect and takes no path argument.
Verified in tree rather than from the finding: applyChangeDirSelection
(main.go) resolves the target's .beads dir and sets BEADS_DIR with no chdir,
it runs from the root PersistentPreRunE ahead of every init-related early
return in that hook, and init honors BEADS_DIR for workspace selection — so
`bd -C other init` acts on other/.beads while the pair reads and writes
beads.role in the invocation repository. init.go's own "PersistentPreRun
doesn't run for init command" note is about the later skipsStoreInit return,
not about -C, which is what made this vector easy to miss twice. Reachability
is bounded: -C requires an existing beads project at the target, so the
pairing arises on re-init rather than first init. Comment narrowed to name the
residual; threading init's resolved project root through the pair is the
stronger arm and stays out of scope as a behavior change.

B3 (nit, architectural consistency) — the adapter selected the role-authority
boundary on a bare "beads.role" literal while the use case passed the
unexported beadsRoleConfigKey, two spellings of one contract the compiler
could not couple. Exported it as domain.BeadsRoleConfigKey and compared
against it at both adapter arms; no new import edge, since package git already
imports domain. Mutation-proved rather than asserted: replacing the
comparison with the case-variant "Beads.Role" the finding names reds
TestInitGitRepositoryRoleIgnoresInheritedConfigSuppression/global_null alone
while unblinded stays green, so the drift the nit describes is caught by an
existing pin. The comparison is still exact, not case-folded: git config names
are case-insensitive, so a caller hand-writing a variant spelling would bypass
roleAuthorityEnv, but folding widens which keys take the strict environment
and no production caller passes anything but the const — a behavior change the
nit did not ask for.

Validation (iteration 3): go build ./... clean; go vet plus CGO_ENABLED=0 and
GOOS=windows CGO_ENABLED=0 vet clean on cmd/bd, cmd/bd/doctor,
internal/storage/domain, internal/storage/domain/git and internal/gitenv;
internal/storage/domain/... full packages green (domain, db, fs, git); targeted
cmd/bd role run green at 19 subtests with no skips
(TestBeadsRoleCommandsIgnoreInheritedGitRouting and
TestBeadsRoleWriterIgnoresInheritedGitRouting, including the suppression
subtest added in iteration 2); gofmt and git diff --check clean. Full cmd/bd is
still not run locally: it times out at base as well. The iteration-1 deferrals
are unchanged and still owed follow-up bead ids at finalize (F4, F6/F11, F8,
F9).

---------

Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>
Co-authored-by: Bee <bee@ghosttrack.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-27 15:59:08 -07:00
Julian Knutsen
469462d56a
test(bazel): run every Bazel test against a hermetic, pinned dolt CLI (#6853)
* test(bazel): run every test against a hermetic, pinned dolt CLI (slice B4)

Tests that shell out to dolt ran whatever /usr/local/bin/dolt the executor
had, a version outside the action key, and skipped (a vacuous, cacheable
PASS) on an executor without one.

- tools/bazel/dolt.bzl: dolt_release repository rule fetching the upstream
  release pinned to scripts/ci/install-dolt.sh (2.2.0), sha256 for
  linux/darwin amd64/arm64; MODULE.bazel declares one repo per platform and
  //tools/bazel:dolt selects the one for the execution platform.
- //tools/bazel:test_env (the --run_under wrapper) carries the pinned dolt
  as data. Bazel merges the run_under runfiles into every test's runfiles, so
  every test action declares dolt as an input; test_env.sh puts it first on
  PATH, exports BEADS_TEST_DOLT_BINARY, and fails the test if it is missing.
  DOLT_ROOT_PATH identity seeding stays in test_env.sh.
- testutil.requireDoltBinaryPresent fails instead of skipping under Bazel.
- TestPinnedDoltCLIMatchesContainerImage also checks tools/bazel/dolt.bzl.
  scripts_test declares the pin files and resolves the repo root through
  bazeltest, so the three dolt pin tests now pass under bazel test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): declare the Go source dolt_test's scope guards scan

TestEveryRawTxJournalScopeIsScopedOrExempt, TestEveryRawTxVersionScopeIsScopedOrExempt,
TestTxMintingWrappersScopeVersionedHistory and TestHelperSubprocessSentinelsAreComplete
parse internal/storage/dolt and internal/storage/issueops at test time; under
bazel test that source was not in the runfiles, so all four failed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(bazel): tie hermetic dolt download to its version; skip on unpinned platforms

- Key DOLT_SHA256 by version and pass canonical_id=url, so bumping
  DOLT_VERSION without new digests fails instead of silently reusing the
  previous release from the sha256-keyed repository cache.
- //tools/bazel:dolt_version_test runs the hermetic dolt and checks it
  reports DOLT_VERSION.
- Unpinned platforms (e.g. windows) resolve to an incompatible target, so
  bazel build //... skips the wrapper there instead of failing analysis.
- test_env.sh also unsets BEADS_DOLT_BIN so it cannot bypass the pin.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 14:29:49 -07:00
Julian Knutsen
f66375459f
test(bazel): runfiles helpers + declared inputs for scan-guard tests (#6828)
* test(bazel): add internal/testutil/bazeltest runfiles helpers

Repository-path resolution for tests that works identically under plain
go test (package cwd, full checkout), bazel test locally, and remote
execution (declared runfiles only):

- IsBazel, RepoRoot (BEADS_TEST_REPO_ROOT, then the runfiles workspace
  root, then a go.mod walk), OverrideRoot (drop-in condition for existing
  helpers; empty under plain go test), CallerDir
- Runfile for $(rlocationpath) values, GOROOT for type-checking tests
- ShardFreeEnv for helpers that re-exec a test binary

Every helper reduces to the pre-Bazel behavior under go test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): declare source/data reads for 14 scan-guard targets

Source-scanning and repo-file tests read files their Bazel targets never
declared, so they failed under `bazel test` (and could not run on remote
workers at all). Declare narrow inputs as `data` and resolve paths through
bazeltest where the test located the tree via runtime.Caller:

- tools/bazel/go_srcs.py (run by `make bazel-sync` after gazelle) keeps a
  managed `go_srcs` filegroup in the packages whose Go source tests scan,
  plus `//internal/storage:tree_go_srcs` for the storage-tree walk. Bazel
  globs stop at package boundaries, so the aggregate needs one filegroup
  per package; the script keeps it in sync and gazelle-stable.
- Hand-written narrow inputs: //internal/storage/schema:migrations,
  //:docsync_files, //plugins/beads:plugin_files, and exports of
  .gitattributes, .claude-plugin/marketplace.json, cmd/bd list.go and
  ready.go, testutil/testdoltcommon.go, scripts README.md and the release
  catalog, each visible only to its consumer.
- runtime.Caller-based roots in backend, backend/conformance,
  internal/storage, internal/formula and test/docsync now go through
  bazeltest.CallerDir / OverrideRoot; relative-path tests need data only.
- integrations/ leaves .bazelignore (it is still gazelle-excluded) so the
  docsync link check can declare integrations/beads-mcp/PYPI.md.

Under --config=prcore every target reports the same PASS/SKIP/FAIL test
set as `go test -race -short` for its package, locally and remotely.
Plain `go test` behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): guard generated go_srcs and harden bazeltest root override

- tools/bazel/go_srcs.py: add --check (print diff, exit 1 on stale blocks);
  new `make bazel-sync-check` target runs it.
- scripts/bazel_policy_test.go: under go test, assert every tree_go_srcs
  (//internal/storage) lists exactly the BUILD.bazel packages below its
  root, run go_srcs.py --check, and forbid BUILD files under docs/ and
  engdocs/ (they would cut subtrees out of //:docsync_files).
- bazeltest: honor BEADS_TEST_REPO_ROOT only when it holds go.mod
  (RepoRoot fails loudly otherwise); drop the unused GOROOT() helper,
  whose compiled-in branch could never fire under rules_go.
- scripts/ci/lib/test-env.sh: unset BEADS_TEST_REPO_ROOT.
- Document the hand-listed out-of-tree link targets in //:docsync_files.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): re-sync BUILD files with main

Regenerated with make bazel-sync after commits that landed while the
scaffold PR was in CI (new internal/execenv, gitenv, gitignore packages
and new sources/deps).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): declare init_safety_help.go for docsync

TestInitSafetyCLIHelpCoversReCloneGotchas (#6431) reads
cmd/bd/init_safety_help.go; export it to //test/docsync only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 08:24:08 -07:00
ecuthiell
79631656c0
fix(config): pin role writes to a fresh common-config context (#6454)
* fix(exec): match Windows environment key identity (#5618)

Centralize os/exec-compatible environment-key comparison and use it for credential, metrics, Git-hook, trace, and repro subprocess filters. Preserve exact Unix semantics and Unicode near-collisions.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>

* fix(worktree): scrub inherited Git routing state

Establish CWD as the worktree repository boundary before discovery can consume inherited Git routing. Fence the startup config probe, clear routing in-process before root pre-run discovery, and retain command-owned subprocess scrubbing as defense in depth without changing PID or signal semantics.

Fixes #5531

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell

* refactor(worktree): explain owned filtering and remove an unused shim

Document why filtering the fresh routing slice in place is safe and call
the shared routing-key classifier directly from tests. Retain the two
forwarders used by production worktree subprocess/removal paths.

Review: https://github.com/gastownhall/beads/pull/5623#pullrequestreview-5098421671
Bead: saa-cgh.23
Agent-Signature: unknown-model/unknown-reasoning

* fix(config): isolate identity and tracked-file Git probes from routing

Reuse the worktree routing owner for GetIdentity and isGitTracked so
explicit cwd/path selection is not displaced by inherited repository,
index or injected config state. Preserve identity precedence and the
tracked-secret refusal contract, with real target/decoy regressions.

Follow-up: https://github.com/gastownhall/beads/issues/6222
Dependency: https://github.com/gastownhall/beads/pull/5623
Reported-by: bee-ghosttrack
Bead: saa-cgh.93
Agent-Signature: unknown-model/unknown-reasoning

* test(execenv): exercise every environment operation under both host rules

Thread the host choice through narrow private helpers behind the existing
exported functions. Keep one normalization owner and retain native wrapper
tests; the additional table covers both host branches for equality, prefix
matching, last-wins lookup, filtering and input preservation.

Review: https://github.com/gastownhall/beads/pull/5672#pullrequestreview-5096948965
Bead: saa-cgh.78
Agent-Signature: unknown-model/unknown-reasoning

* docs(worktree): explain startup Git environment scope (saa-cgh.23)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(execenv): exercise exported host policy wiring (saa-cgh.78)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(config): share subprocess key identity for Git routing (saa-cgh.93)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(hooks): isolate tracked-file guard from Git routing (saa-cgh.98)

Follow-up to bee-ghosttrack's review of gastownhall/beads#6322; refs #6356.

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(config): isolate role commands from Git routing (saa-cgh.99)

Follow-up to bee-ghosttrack's review of gastownhall/beads#6322; refs #6357.

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(env): exercise shared key helpers on native Windows (saa-cgh.78)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* refactor(gitenv): share key parsing and normalize routing membership once (saa-cgh.93)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* docs(config): explain role command routing policy (saa-cgh.99)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(hooks): retain legitimate inherited tracking context (saa-cgh.98)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(hooks): isolate trusted global configuration fixture (saa-cgh.98)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(hooks): isolate role cleanup in the selected repository (saa-cgh.109.4)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(hooks): reset only common local role configuration (saa-cgh.109.10)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(hooks): scope hooks-path cleanup to selected common config (saa-cgh.109.11)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(hooks): make worktree role preservation independent (saa-cgh.120)

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* Report selected config read failures during hook reset (saa-cgh.120)

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* fix(hooks): keep successful config stdout separate from diagnostics (saa-cgh.120)

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* fix(config): pin role writes to fresh common context (GH#6384) (saa-cgh.120.3)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(config): retain Git diagnostics from role writes (saa-cgh.120.3)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(config): assert the actual Git lock diagnostic (saa-cgh.120.3)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* fix(config,hooks): restore the inherited-tracking fallback in the secret guard

Maintainer review fixes for PR #6454 (adopt-pr iteration 1). The scorecard
approved at 913/1000 with no blocker or major surviving synthesis; these are
the non-gating findings that were cheap to close while the fixup was open.

Finding 1 (minor, security) is the only in-range regression in the report. The
range switched internal/config.isGitTracked from the inherited environment to
gitenv.ScrubRouting only, so a config.yaml tracked solely through legitimate
routing -- a bare repository whose work tree is named by GIT_DIR/GIT_WORK_TREE
-- reported untracked, and a secret-key write that upstream-base refused was
silently allowed. Restores the scrubbed-then-inherited union the same stack
already gave cmd/bd.isGitTrackedFile.

Finding 4 (nit) composes with it rather than conflicting, so both land as one
change: the inherited probe now runs only when the scrubbed probe failed for a
configuration reason. git's exit codes discriminate exactly -- 0 tracked, 1
repository reached but path untracked (final), 128 no reachable repository --
so the ordinary untracked-hook miss no longer spawns a second `git ls-files`
while the bare-work-tree fallback is preserved.

Finding 2 (minor). scrubWorktreeRemovalGitEnvForOS keyed its stricter
no-optional-locks policy on strings.ToUpper while the rest of the PR uses
execenv key identity. The two disagree in both directions on Windows:
GIT_OPTIONAL_LOCKS with a dotted capital I (U+0130) survived the scrub although
Git honors it, and GIT_OPTIONAL_LOCK with a long s (U+017F) was dropped
although os/exec keeps it distinct. normalizeWorktreeGitEnvKey is deleted in
favour of execenv.KeyEqualForOS. Base applied ToUpper on all platforms, so this
is a consistency fix in newly rewritten code, not a regression.

Finding 5 (nit). The comment above resetHooksPathIfBeadsManaged's probe
asserted that repoRoot and commonDir share the cached Git context. Probed
false: with cwd A and GIT_DIR=B/.git, `git rev-parse --git-common-dir
--show-toplevel` prints B/.git and A. Restated as intent.

Finding 3 (scrubGitHookEnv is a fifth, strictly weaker routing scrubber) is
deliberately not taken here: cmd/bd/export_auto.go is untouched by this range,
so the hazard is byte-identical at base, and the synthesis scoped it to a
follow-up bead rather than growing this stack.

Each fix is mutation-proved. Reverting isGitTracked reds only the new
TestSecretGitTrackingHonorsInheritedBareWorkTree -- including "refusal = <nil>",
the actual security consequence -- while the incumbent decoy test stays green,
so the pin adds coverage rather than duplicating it. Restoring ToUpper reds
only the new Windows row, showing both directions at once.
TestGuardHookWritePathHonorsInheritedRepository gains a fixture guard asserting
its scrubbed probe fails for a configuration reason rather than exit 1, so a
future fixture change cannot silently make the fallback unreachable.
scrubWorktreeRemovalGitEnvForOS had no test at all before this.

Green locally: ./internal/config, ./internal/execenv, ./internal/gitenv,
./internal/githooksenv, ./internal/gittraceenv, ./internal/metrics, ./scripts,
./internal/beads (the GIT_CONFIG_GLOBAL blast radius neither reviewer ran), and
a blast-radius cmd/bd run over the hooks, worktree, role and uninstall suites.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>
Co-authored-by: Bee <bee@ghosttrack.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-27 02:00:16 -07:00
Jim Wordelman
6e36c3f19c
ci: raise proxied Dolt cmd test job timeout, add step timeout, refresh shard manifest (#6689)
* ci(proxied-cmd): green — give the proxied-shard job its own step timeout, refresh manifest (refs be-bruke)

The job-level timeout-minutes was the only backstop for the "Test
proxied-server cmd shard" step, and it counts the "Download binaries"
step too — so a slow download alone could exhaust the job budget and
kill an otherwise-healthy shard's test run with no way to tell the two
failure modes apart. Give the test step its own 16m timeout inside the
existing 30m job budget, and regenerate the bin-packed shard manifest
via scripts/ci/gen_proxied_shard_manifest.py so it stays in sync with
the current TestProxiedServer* set (0 drift, all 15 shards fallback:0).

* ci(proxied-cmd): mirror the shard timeouts onto the main lane, regen the manifest at the adoption base

Adoption review of #6689, iteration 1 — three minor findings, applied here
rather than deferred.

main.yml's `test-proxied-cmd` is the same job as pr-risk.yml's twin — same
proxied-test-shard.sh, same manifest, same total 15 — behind a
download/verify step inside a 20m budget with no step timeout, so the exact
ambiguous-timeout failure mode this PR removes from the PR lane survived on
the post-merge lane, which is where triage happens with nobody watching.
Mirror both edits (job 20m -> 30m, step 16m) and record the ordering at the
step, since it is not obvious from the site that 16m is deliberately below
the script's own -test.timeout=15m plus setup.

The manifest's only record of the cost proxy's two known blind spots lived
in a hand-written comment block the regeneration deleted:
TestProxiedServerListSortRetiresTheWalk seeds 1400 rows and walks the
listing three ways off a single init, and TestProxiedServerServeReadParity
is the CLI/HTTP read-parity oracle whose presence in this gate-required lane
is what makes the anti-drift claim enforced. Re-adding prose to a generated
file would just be deleted again, so it moves into the generator's emitted
header.

Base drift since authoring left the refreshed manifest one test stale —
#6584 added TestProxiedServerBackupRefusedOnExternalTopology, which
hash-distributed to shard 14 — so "0 drift, all 15 shards fallback:0" was
already false on the merge base. Benign (unlisted tests hash-distribute and
still run, and drift removed no name, so the strict parser's stale-entry
hard-fail was unreachable), but the stated deliverable did not hold.
Regenerated at the adoption base: 149 of 149 discovered tests listed,
fallback:0 on every shard, peak load unchanged at 39 estimated inits. That
figure is a point-in-time measurement, not an invariant: nothing in CI diffs
the committed manifest against a regeneration, so it re-drifts the moment
the next TestProxiedServer* lands. A freshness step
(`gen_proxied_shard_manifest.py 15 | diff - <manifest>`) on a lane already
triggered by .github/scripts/* would make it one; that is follow-up scope.

Verified: the generator reproduces the committed manifest byte-identically;
BEADS_TEST_SHARD_LIST_ONLY=1 across all 15 shards assigns all 149 discovered
TestProxiedServer* functions exactly once with no duplicates or orphans;
actionlint clean on both workflows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-26 07:17:03 -07:00
Julian Knutsen
b3d17caa6d Merge release/1.3.0 into main (post-tag forward-port)
Forward-ports the 38 commits that shipped on release/1.3.0 (tagged v1.3.0 at
f45b249ce) into main, as RELEASING.md mandates after a tag. None of them had
ever been forward-ported. Recorded as a merge so main becomes a superset of
the release branch and the lineage is in the history.

Migration ordinal collision (the reason this needed a human-shaped merge):
both branches independently claimed clone-local ignored slot 0026 under
different filenames, so git merged both in with no conflict and left the
series with two migrations at ordinal 26 — a single-integer cursor cannot
address that, and migrationSource.list() panics on it.

Main's ignored/0026_add_wisps_current_revision is renumbered to 0027.
Release's 0026_dep_rekey_dedup_marker is in the shipped v1.3.0 tag, so field
stores have already recorded it at 26 and it can never move; main's twin has
never shipped in any release, so it is the one that is free to move. 0027 was
unused in the ignored series (both sides topped out at 26).

This trips check-migration-hygiene.sh Check C (frozen migrations), which has
no exemption mechanism. That is expected and accepted: breaking the collision
requires a rename, and the ignored series' cursor table is itself dolt-ignored,
so its content hashes are clone-local and never participate in the cross-clone
ContentHashSkew comparison that Check C exists to protect. Main's 0067 body is
deliberately NOT edited for the same reason in reverse — it is main-plane and
its hash IS compared across clones.

Conflicts resolved:

  CHANGELOG.md                       release's finalized [1.3.0] section kept
                                     byte-identical; main's two unreleased
                                     entries (#6023, #5859) moved up under
                                     [Unreleased] where they belong
  cmd/bd/init_safety.go              union — release's added paragraph is
  cmd/bd/init_safety_help.go         accurate against the merged code (three
                                     exit-12 sources, incl. init.go:3147)
  internal/storage/uow/dolt_sql_provider.go
                                     kept main's newProvider refactor AND
                                     release's readOnly field/assertion;
                                     newProvider gained readOnly: opts.readOnly
                                     so release's read-through-refused-migration
                                     behavior survives main's refactor
  .github/workflows/main.yml         single -timeout=30m per release's #6091
  scripts/ci_workflow_test.go        design (flag on the invocation, not in
                                     matrix.test-flags); the shadowed 25m in
                                     matrix.test-flags removed
  scripts/migration-test/lib/versions.sh
                                     union — main's darwin/arm64 corpus plus
                                     release's v1.2.2 linux/amd64 entry
  integrations/beads-mcp/uv.lock     ruff 0.16.5 (release, newer); block is
                                     internally consistent, uv lock --check ok
  default.nix                        vendorHash left at main's value and is
                                     KNOWN STALE — see below

Version stamp lands at 1.3.0 (main had been sitting at the previously-released
1.2.2, this project's between-releases pattern). check-versions.sh: 24/24.

Known follow-up: default.nix vendorHash. The merged go.mod/go.sum is a genuine
union (release's kin-openapi/oapi-codegen bumps + main's newer Dolt stack) and
matches neither side, so neither recorded hash is correct. It must be
regenerated with scripts/update-nix-vendorhash.sh. The repo's automatic fixer
(update-vendor-hash.yml) is dependabot-only and will not run on this PR.
2026-09-15 22:09:13 +00:00
Julian Knutsen
2acf75edc7
ci(pr-core): give the -race lane an explicit 30m go-test timeout (#6091)
cmd/bd has outgrown Go's implicit 10m per-package default. Measured
through this wrapper it takes 572s, leaving under 5% margin, so runner
variance pushes it over: it failed at 600.343s and 600.728s on two
consecutive runs of #6090 while #6038 landed just under.

No single test is responsible, so the timeout panic names whichever test
was in flight and reads like a hang rather than the package running out
of clock.

30m matches every other -race lane in the repo for this same code:
main.yml's integration package and cmd/bd shards, ci-measurements.yml,
and nightly.yml. Splitting or de-slowing cmd/bd is the real fix and
belongs on main; this only stops the clock from being the failure.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-31 00:34:10 -07:00
Julian Knutsen
d530cddfa6
ci: add advisory cyclomatic complexity reporting (#6081)
* ci: add advisory cyclomatic complexity reporting (bd-7g21)

Add a pinned gocyclo report, checked-in baseline, stable-key PR diff, and opt-in guard for shipped Go code. Run the report on every PR as a non-required artifact while maintainers measure signal quality; preserve existing complexity as grandfathered debt.

Agent-Signature: unknown-model/unknown-reasoning

* test(protocol): refresh dependency removal corpus
2026-08-30 19:05:17 -07:00
Julian Knutsen
e7894732e0
fix(ci): make a failed pinned-dolt download actually fail the job (#5935)
* fix(ci): make a failed pinned-dolt download actually fail the job

scripts/ci/install-dolt.sh retried the download with

    if curl -fsSL -o "$workdir/$asset" "$url"; then
      break
    fi
    status=$?

`status=$?` there reads the status of the *if compound*, not of curl. A
compound whose condition fails and which has no else branch is itself
successful, so `status` was 0 on every failed attempt, and the
retries-exhausted branch ran `exit 0`. A download that never succeeded
reported success.

That defeats the entire point of the script. The step goes green, the
version-drift guard at the bottom never runs, and the job proceeds on
whatever `dolt` the runner image already had on PATH -- the unpinned
binary the pin exists to keep out, and the one that made
TestFreshBootstrapHealIncarnation flake when releases/latest moved to
2.3.x.

Capture curl's status directly instead, and include it in the
retries-exhausted message so the log says which failure it was.

Guarded by TestInstallDoltPropagatesDownloadFailure, which runs the real
script against stub curl/sleep/tar/sudo/dolt/uname on PATH. Against the
unfixed script exactly one subtest reddens ("exhausted retries": exit 0,
want 42); "immediate success", "eventual success" and "installed version
drifts" stay green, so the harness demonstrably observes success and the
new signal is specific to this bug rather than to everything failing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(ci): pin what install-dolt.sh actually downloads and installs

The review council mutated the script and found two changes the new test
waved through: swapping the pinned release URL for releases/latest, and
installing mode 0644 into /opt/dolt instead of 0755 into /usr/local/bin.
Both are exactly the drift this script exists to prevent, and both left
all four subtests green because the test only counted calls. The sibling
pull_dolt_image_test.go already asserts its docker argv; this one now
does the same for curl and sudo.

The council also flagged the drift guard itself: `[[ $installed !=
*"$version"* ]]` accepts any version containing the pin as a substring,
so 12.2.0 and 2.2.0-rc1 both pass. Compare the version token exactly.

Red before green, by real on-disk swaps (a -overlay cannot reach a file
the test execs):

  releases/latest URL          all 5 subtests red (was: all green)
  install -m 0644 /opt/dolt    4 of 5 red -- exhausted_retries stays
                               green because it never reaches sudo
  substring drift guard        only installed_version_merely_contains
                               _the_pin reddens

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-21 16:58:46 -07:00
Julian Knutsen
825cd4c937
fix(ci): pin the Dolt CLI — 2.3.0 broke DOLT_RESET on fresh databases (ga-m0dlk) (#5894)
* fix(dolt): git-remote tests need a sql-server that shares their filesystem

The four git-remote tests that push through the store API fail on every run
of the new test-server-storage-full lane (#5836), which is what has the
merge pipeline stalled: shards 1, 9, 10 and 15 all die with

  the remote: origin 'git+file:///tmp/embedded-git-remote-test-*/remote.git'
  could not be accessed; fatal: ... does not appear to be a git repository

setupEmbeddedGitRemote dates from when this package had an in-process
embedded engine. store.go now hardcodes serverMode:true, so New() connects
to the suite's shared Dolt server — which TestMain runs in a Docker
container whose only mount is the image's own /var/lib/dolt. CALL DOLT_PUSH
executes inside that container, so the host bare repo the test just created
does not exist for it, and neither does the store's own Path (which is why
TestGitRemotePushSkipsUserPrePushHook cannot find its cache mirror and
TestGitRemoteSyncRoundTrip's clone comes up empty). The dolt credential
hints in the log are generic advice, not a credential problem — nothing
here needs a credential.

Give these tests their own local dolt sql-server rooted at the store's data
directory, the arrangement TestGitRemoteExternalServerRouting,
TestSQLRemotePersistsAcrossExternalServerRestart and
TestCredentialCLIRoutingE2E already use. The server is spawned with
doltserver.ServerSpawnEnv(), the same environment bd gives the server it
starts: a `dolt` CLI command run against a directory a sql-server already
serves is proxied to that server, so the git subprocess is spawned by the
server and the CLI-side guards (git tracing, core.hooksPath) only take
effect if the server carries them too.

Two consequences fall out and are fixed here:

  - The setup now takes a genesis CommitAll. Commit() deliberately skips
    config (GH#2455), so issue_prefix stayed dirty forever: Pull() refuses
    to auto-commit a dirty internal config key, and a peer cloning the
    database got no prefix at all. The CLI sibling setupGitRemote has
    always done this with DOLT_COMMIT('-Am', 'Genesis: schema and config').

  - The gate is testutil.RequireDoltBinary rather than skipIfNoDolt: these
    tests no longer need the shared container, only dolt and git, so
    requiring the container would be a skip for no reason.

Setup asserts <Path>/<db>/.dolt exists on this filesystem, so if the store
ever drifts back onto a containerized server these tests fail loudly
instead of quietly testing nothing.

Verified against the real shard script on a live Dolt server: shard 1 goes
from 1 failure to 0, shard 9 from 1 to 0.

* test(dolt): pin what TestCreateIssueAfterPull's Pull actually delivered

Shard 1 still fails in CI after the topology fix, now at the last assertion:
'GetIssue(ai-clone-001) failed: not found'. Deterministic across two CI runs,
not reproducible locally on dolt 2.1.8 or 2.3.1 (the whole shard is green
here). The end-of-test assertion cannot distinguish 'Pull() reported success
but merged nothing' from 'Pull() delivered the row and the post-pull
CreateIssue dropped it', which are different bugs in different places.

Assert immediately after Pull, before the post-pull write, so the CI failure
names which one it is.

* fix(ci): pin the Dolt CLI — 2.3.0 broke DOLT_RESET on fresh databases

Test (Server Dolt Full Suite 12/16) has been failing on a coin flip with

  TestFreshBootstrapHealIncarnation/exact_creator_heals_its_interrupted_bootstrap
  creator-authorized migration: schema migration: pending schema migrations
  alter pre-existing dirty tables: issues; run 'bd dolt commit' ... (#4566)
      schema: fresh-bootstrap reset: Error 1105 (HY000): context canceled

That second line is the whole story, and it is not a beads bug. The heal was
authorized and its capability consumed; what failed is the
CALL DOLT_RESET('--hard') in schema.MigrateUpWithLock. Dolt 2.3.0 regressed
that procedure: roughly one freshly created database in twenty comes up with
it permanently unusable, answering "Error 1105 (HY000): context canceled" to
every session, from any connection, for the life of the server process. It is
not transient and not load-related — a beads-free probe that only does
CREATE DATABASE followed immediately by the procedure reproduces it:

  dolt 2.1.8   0/40 databases broken
  dolt 2.2.0   0/60
  dolt 2.3.0   3/60
  dolt 2.3.1   3/100

Every workflow installed the CLI by piping dolthub/dolt's releases/latest
install.sh, so the binary under test changed the moment 2.3.x shipped —
while internal/testutil/testdoltcommon.go had the sql-server container pinned
at 2.2.0 the whole time. Server-mode tests run both halves at once against the
same databases, so the suite was exercising a Dolt pair no release ever
shipped. Pin the CLI to the container's release and keep the two locked
together:

- scripts/ci/install-dolt.sh installs a pinned Dolt, verifies `dolt version`
  on PATH actually resolves to it, and records the per-version measurements
  and the criterion for raising the pin.
- All 25 install sites across six workflows route through it.
- TestWorkflowsInstallPinnedDolt rejects any reintroduction of
  releases/latest; TestPinnedDoltCLIMatchesContainerImage fails when the CLI
  pin, DoltDockerImage and pull-dolt-image.sh disagree.

The test itself is unchanged in what it asserts. It gains one preflight probe:
before exercising the heal it checks that this server can hard-reset the
database it just created, so a broken Dolt is named as such instead of
surfacing as the generic #4566 refusal — which reads like a beads defect and
sends the reader to "run 'bd dolt commit'", advice that cannot work when the
reset the heal needs is the thing that is broken. On a pristine database the
probe is a no-op, and over 60 fresh databases a successful first reset never
broke a later one.

Verification: 48 runs of the test (6 workers x 8) and two full shard-12 runs
on the pinned 2.2.0, zero failures. Same binary on 2.3.1: 3 of 24 runs still
fail, now in 0.04s with the explicit diagnosis instead of 1s in with the
misleading one.

Beyond this test: bd dolt compact, bd dolt flatten and mergesettle's rollback
all issue DOLT_RESET('--hard'), so they are unreliable for real users on any
Dolt 2.3.x server. That needs an upstream report; pinning CI does not fix it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 20:59:27 -07:00
ecuthiell
1b67d0b1fb
fix(ci): remove recursive Make from lint formatting (#5302) (#5320)
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 21:43:59 -07:00
Julian Knutsen
cff62bd631
Make the pre-commit lint gate trustworthy, and stop one main-side lint word from reddening every PR (#5538)
* fix(githooks): build golangci-lint with the repo's Go, not golangci-lint's

The pre-commit hook ran `go run golangci-lint@v2.10.1`. For a pkg@version
build the go command honours the TARGET module's toolchain directive, so
GOTOOLCHAIN=auto downgraded to go1.25.12 — and golangci-lint built by
go1.25 refuses this repo's `go 1.26.5` at config load:

    go: github.com/golangci/golangci-lint/v2@v2.10.1 requires go >= 1.25.0; switching to go1.25.12
    Error: can't load config: the Go language version (go1.25) used to build golangci-lint is lower than the targeted Go version (1.26.5)

Every Go commit failed there, so the hook taught six-plus contributors to
reach for --no-verify — the one habit a lint gate must never train.

CI never hit it because actions/setup-go installs the go.mod version and
exports GOTOOLCHAIN=local, so its `go install ...@v2.10.1` compiles the
linter with go1.26.5. Pinning GOTOOLCHAIN to the go directive reproduces
that resolution on any host, including one whose base toolchain is older
than the repo's target (where a bare GOTOOLCHAIN=local would pin the
linter to that older toolchain instead).

Verified on a real staged Go change: red before with the message above,
`0 issues.` after, and still exit 1 on a planted errcheck violation.

Bead: ga-2ltro.13

* ci: lint PRs for what they introduce, sweep the whole tree on main

Both PR lint gates linted the whole tree: the `lint` job left
only-new-issues at its false default, and pr-lint-wrapper ran
`make ci-pr-lint` over ./... . So a single lint word landing on main —
twice this wave, once a prose misspell — turned EVERY open PR red for a
change it did not make, and each author had to either fix someone else's
line or wait.

The tradeoff, recorded beside the job: a pre-existing issue in untouched
code no longer blocks a PR, and because "new" is measured off the diff, a
violation carried in by a MOVE or a rename does. What it buys is that a
main-side word now reds main's own run, where it belongs. main.yml
already runs both jobs unscoped on every push, so whole-tree coverage —
including the GOOS=windows cross-lint — is unchanged; only its blast
radius moves.

The wrapper takes its scope from BD_LINT_NEW_FROM_MERGE_BASE, set only
by pr.yml, so `make ci-pr-lint` run by hand still sweeps the tree. That
makes the local wrapper STRICTER than the PR gate rather than equal to
it, which is what LINTING.md and CI_CLEANUP_PLAN.md said before and no
longer would: both now state the two-lane contract and say the local
run's extra strictness is deliberate. Their freshness markers move with
the workflow and wrapper files they are pinned to.

Bead: ga-uwix0.2

---------

Co-authored-by: Julian Knutsen <ci@beads.test>
2026-08-10 04:55:55 -07:00
Julian Knutsen
06b34deba0
ci: retry Dolt image pulls (bd-m0o1z) (#5228) 2026-08-01 08:18:41 -07:00
CI Bot
f373508dfd merge(bd-ilb): OpenAPI v0 spec, types-only codegen and drift gates 2026-07-31 18:34:34 +00:00
CI Bot
e988fd002d feat(httpapi): OpenAPI v0 spec, types-only codegen, and the drift gates (bd-ilb)
Lands the bd serve wire contract and the machinery that keeps it honest. No
runtime behavior: nothing routes, nothing serves, and no CLI surface changes.

internal/httpapi/spec/openapi.v0.yaml is hand-written and is the source of
truth; internal/httpapi/apigen is generated from it with oapi-codegen
types-only (no server stubs, no router), via a go.mod tool directive so
contributors need no extra install.

The decisions this slice fixes, because they are one-way doors:

- No second wire struct. Issue, IssueWithCounts, IssueDetails and
  IssueWithDependencyMetadata are pinned to the canonical Go types with
  x-go-type, so the generator emits type aliases and the CLI's --json, the
  JSONL interchange and the HTTP bodies all marshal one struct. A hand-written
  Go-to-Go mapping layer would recreate the drift disease this epic exists to
  cure. Guarded by a two-way JSON-tag bijection test, not a round-trip fixture:
  a new omitempty field absent from a fixture round-trips cleanly in both
  directions and proves nothing.
- limit=0 means unlimited, matching the CLI exactly, with defaults 50 (list)
  and 100 (ready) documented as the shared constants. The only exception --
  refusal under --allow-non-loopback -- is in the limit description.
- ready returns IssueWithCounts, not bare Issue.
- Unknown query parameters are rejected with 400 invalid_argument naming the
  parameter in a machine-readable extension member: with types-only codegen the
  decoders are hand-written, and Go's silent-ignore default would WIDEN a
  filtered result set under version skew.
- /v0/beads/issues forces (created_at DESC, id ASC) on every request, and the
  spec documents the deliberate divergence from bd list's priority ordering.
- One RFC 9457 problem+json envelope everywhere, with a frozen code vocabulary
  mapped from the sentinels via errors.Is. 5xx bodies carry a static per-code
  detail; the real error goes only to the log, because driver and dial errors
  routinely embed the DSN.
- Only the status codes the six operations need are documented. Every
  documented status+code pair is permanent wire surface.

Two traps found while building this, both now guarded:

- oapi-codegen v2.6.0 silently drops x-go-type from a component that also
  carries allOf. An allOf-composed IssueWithCounts generated
  "type IssueWithCounts = types.Issue" -- the ready endpoint would have shipped
  with the count fields gone. The pinned schemas therefore repeat their
  property lists, and internal/httpapi/pinning.go turns a regression into a
  build failure.
- context.DeadlineExceeded satisfies net.Error, so a tripped request deadline
  would have mapped to 503 db_unavailable instead of a generic 500.

make api-check is the three-part gate (regenerate, diff-or-fail, spec tests)
and runs from scripts/ci/pr-policy.sh, i.e. in a required PR job -- the Main
workflow is perpetually concurrency-cancelled, so a gate living only there
would be unenforced.

The go.mod diff is the tool directive and its closure. The protocol corpus is
untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 08:37:34 +00:00
CI Bot
e2d82a54d7 refactor(list): route both bd list paths through internal/workapi and delete list_filter.go (bd-fv4)
Second half of the extraction. cmd/bd/list_filter.go is gone; the direct
path (list.go), the proxied path (list_proxied_server.go), and the four
other consumers the file had grown (search, search_proxied_server, count,
count_proxied_server, graph_apply) call workapi.BuildListFilter /
LoadStoreListConfig / LoadUOWListConfig / ApplyStatusFilter /
ValidStatusList instead.

listInput now embeds workapi.ListParams rather than copying into it. A
50-field flags -> params mapping would be a silent-swap hazard the compiler
cannot see; embedding makes cmd/bd fill the shared struct directly, so
there is one definition of a list query and nothing to keep in sync. The
CLI-only fields (output format, pager, tree/deps rendering, effectiveLimit,
--repo) stay in listInput, which is why ListParams is a subset and not the
whole thing.

applyStatusFilter and validStatusList are exported rather than kept
unexported as the design sketch had them: search.go, search_proxied_server.go
and graph_apply.go already called them, and leaving them behind in cmd/bd
would have forked the status-parsing rules.

The --limit flag now registers workapi.DefaultListLimit. internal/config's
list.limit viper default keeps its own literal 50 - internal/config cannot
import workapi, because workapi imports internal/config for
GetCustomTypesFromYAML.

Boundary enforcement lands with the package, per the design: a
workapi-frontend-boundary depguard rule denying cobra and net/http under
internal/workapi, and a pr-policy.sh grep for the symbol-level half depguard
cannot express (config.GetDirectoryLabels, os.Getenv). Both were verified to
fail on a deliberate violation and then reverted.

Gates: protocol corpus unchanged (whole cmd/bd/protocol package green with
BEADS_PROTOCOL_REQUIRE_DOLT=1, no testdata diff); TestBuildListFilterGolden
35/35 against the filters recorded from the old builder; TestEmbeddedList
52 subtests and TestProxiedServerList 65 subtests green; count, search and
graph-apply suites green; golangci-lint clean on cmd/bd and internal/workapi
apart from the two pre-existing main.go gosec baselines.

Agent-Signature: claude-code-opus-5-unknown-reasoning on behalf of CI Bot
2026-07-31 06:01:48 +00:00
matt wilkie
46417b5837
ci(lint): cross-lint Windows-tagged Go, fix fmt-check exit status, pin golangci-lint (core of #5075) (#5083)
Closes #4991; extracted from PR #5075 per its review (split-merge). The
full PR bundled this fix with substantial CI toolchain-identity/host-binding
machinery; this commit keeps only the reviewed core:

- scripts/ci/pr-lint.sh: after the native golangci-lint pass, add a second
  pass with GOOS=windows GOARCH=amd64 CGO_ENABLED=0 GOWORK=off (skipped if
  the native target already matches), so files guarded by
  //go:build windows && !cgo get linted.
- Makefile fmt-check: stop swallowing gofmt's own exit status - a gofmt
  failure (not just "found unformatted files") now fails the check.
- Pin golangci-lint-action from version: latest to v2.10.1 in
  .github/workflows/pr.yml and .github/workflows/main.yml, and harden the
  lint invocations with --config=.golangci.yml,
  --modules-download-mode=readonly, and GOWORK=off.

.github/workflows/ci-measurements.yml already pinned golangci-lint to
v2.10.1 via `go install ...@v2.10.1`, so no change was needed there.


Agent-Signature: claude-fable-5-medium on behalf of maphew

Co-authored-by: Ewen Cuthiell <ewencuthiell@gmail.com>
2026-07-26 12:41:18 -07:00
coffeegoddd☕️✨
6389546c56 test(proxied): split the six heaviest suites so shards balance under the timeout
The proxied cmd lane is init-bound: each subtest's bd init runs the full
migration chain, so a shard's wall-time tracks its total init count. The six
heaviest suites (Update 56 inits, Show 47, Create/Close 41, Dep 36, Ready 34)
were each a single top-level function, and a function is the atomic unit of
sharding — so no shard split could pull the worst shard below Update's ~169s
(local), which is what kept brushing the 15m budget on CI.

Split each of the six into 2-3 top-level functions (Update/Show ->3, the rest
->2), balanced by init count, by inserting function boundaries between the
already-independent per-subtest blocks — no test logic moves, every t.Run set
is byte-identical to before. Regenerate the 15-shard manifest by greedily
bin-packing every TestProxiedServer* function by init count
(scripts/ci/gen_proxied_shard_manifest.py); the heaviest shard drops from ~56
to ~31 estimated inits, roughly halving the long pole.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 15:55:17 -07:00
Chris Sells
52aea3f7e3 docs: go all-in on Mintlify — delete the Docusaurus site and its machinery
Per the decision record: no parallel run and no cutover PR. Removes
website/ (including the four versioned doc snapshots — current-only
versioning; snapshots remain in git history), deploy-docs.yml,
generate-llms-full.sh (Mintlify serves llms.txt natively), the ci-website
package gate (Makefile target, scripts/ci/website.sh, the package-website
jobs and their aggregate-gate entries in pr.yml/main.yml/release.yml, the
ci-measurements suite, detect-package-gates wiring), and the release-time
docs snapshot flow (snapshot-release-docs.sh, check-docs-version.sh, their
call sites in update-versions.sh/check-versions.sh, RELEASING.md section
rewritten for the Mintlify reality).

Drift machinery now covers exactly the Mintlify artifacts:
check-cli-docs-drift.sh pathspecs/fingerprint, docs-autofix-push.sh
allowlist, and check-doc-flags.sh scans drop the website/llms paths. The
beads-docs skill's generated-content and verification sections updated to
the bd-generic + docsmint pipeline.

NOTE for merge time: if branch protection requires the 'Package Gate
(website)' check by name (rather than the aggregate gate), drop it from
the required checks or PRs will wait forever.

Gates: docsync PASS, generate-cli-docs.sh --check PASS, freshness PASS.
2026-07-12 10:37:17 -07:00
Chris Sells
3198395a32 docs(ci): add Mintlify preview, docsync guard, and broken-links workflow
- mint.sh + make docs-dev: local preview of the docs/ Mintlify site
  (Node 22 handling modeled on the Gas City wrapper).
- test/docsync: Go guard keeping docs/ and docs.json in exact
  correspondence — every nav entry has a file, every page is in the nav
  (tiny stub allowlist for paths released bd binaries print), published
  pages use root-relative extensionless links that resolve, and engdocs/
  plus curated root markdown keep exact GitHub-style paths. Wired into
  make check-docs.
- .github/workflows/docs-mintlify.yml: docsync + baseline-aware
  mint broken-links check on PRs touching docs/**.
- Retire scripts/sync-website-docs.sh and its mirror-drift gate: the six
  mirrored files now live once in the Mintlify site; the committed
  Docusaurus mirrors stay frozen so the parallel-run site keeps building.
- Repoint inbound links to moved docs across README, guides, examples,
  npm-package, plugin resources, and Go comments; add docs/SYNC_CONCEPTS.md
  pointer stub (bd's agent templates print that path). Runtime strings in
  cmd/bd (prime.go, store_factory_nocgo.go, init_git_hooks.go) still print
  old paths — covered by stubs, flagged in the PR.
2026-07-12 10:37:07 -07:00
matt wilkie
cea892d43f
docs: publish community tools on website (#4299)
* docs: publish community tools on website

Mirror docs/COMMUNITY_TOOLS.md into Docusaurus so the Pages build publishes it for the default released docs and unreleased docs without relying on manual copy steps.

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie

* docs: publish core support docs on website

Mirror Sync Concepts, Labels, Metadata, Uninstalling, and Antivirus docs into Docusaurus and include them in the default released snapshot and LLM bundle.

Clarify that agent execution and verification metadata are examples of metadata as a schema extension point, not native issue fields.

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie

* docs: clarify metadata examples

Replace local verification queue metadata with a tracker round-trip example so public metadata docs show upstream-aligned schema extension patterns.

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie
---------
2026-06-03 13:44:47 -07:00
Julian Knutsen
9e8d411f71
Add PR CI wrapper command surface (#4211)
* ci: add PR wrapper command surface

* ci: collect wrapper timing in PR checks

* docs: record initial CI wrapper measurements

* ci: add manual measurement workflow

* ci: allow branch measurement dispatch

* ci: keep measuring independent commands after failures

* docs: record branch-dispatched CI measurement

* ci: keep measuring package suites after failures

* docs: record CI measurement batch

* fix: repair CI-measured test failures

* test: make local Go gates hermetic

* ci: add sharded integration measurement lane

* test: fix internal beads integration TestMain

* docs: record fixed integration shard measurement

* ci: add hybrid integration sharding measurement

* ci: speed up cmd/bd test shard discovery

* ci: add 16-way integration sharding measurement

* ci: measure prebuilt cmd bd sharding

* docs: record prebuilt sharding measurement

* docs: design CI build artifact stage

* docs: record repeat prebuilt integration measurement

* ci: add build artifact stage

* ci: reuse build artifact in linux test lane

* ci: promote main linux integration shards

* ci: gate legacy platform tests to main

* ci: satisfy workflow shell lint

* ci: ignore archived migrations in duplicate check

* ci: add package gate wrappers

* test: clean up testing short boundaries

* ci: allow unreleased version docs lag

* ci: gate releases on package checks

* ci: split workflow by tier

* ci: add aggregate PR gates
2026-05-29 18:52:53 -07:00