beads/release-gates/be-23f3-gate.md
Jim Wordelman 30f30ca95b
ci: PR-CI lane runs the full internal/storage/dolt suite against a live Dolt server (#5836)
* test(ci): red — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)

* feat(ci): green — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)

Add test-server-storage-full as a sibling job to test-server-storage in
pr-risk.yml. build-embedded already compiles every top-level test in
internal/storage/dolt into one binary (embedded-test-binaries); the
existing job only ever runs -test.run '^TestConformance$' against it.
The new job reuses that same artifact and live-server setup with no new
build step, and runs everything else via -test.skip '^TestConformance$'
so previously-stranded tests (TestCreateGuard_*, federation/concurrent
suites, and diff-owned tests such as TestBenchDBPurgeDoesNotLeak on PR
#5792) get a real PASS/FAIL instead of a silent SKIP. Sets
BEADS_TEST_ENV_RUN_DOLT=1 so federation_test.go's own live-server check
fails loudly rather than skipping if this job's setup ever regresses.
Wired into ci-gate: added to needs, CI_GATE_REQUIRED, and the
tier-gated skipped_ok fallback (mirrors test-server-storage's existing
treatment in each).

Also updates two pre-existing meta-tests over pr-risk.yml that pinned
exact snapshots of the file this job necessarily changes:
scripts/ci_capability_selector_test.go's ci-gate authority snapshot,
and scripts/pull_dolt_image_test.go's retry-helper call count
(pr-risk.yml: 2 -> 3, the new job's own Dolt-image-pull step).

* chore: release gate PASS for be-pp7e (PR-CI lane for TestBenchDBPurgeDoesNotLeak)

Deploy gate record for be-pp7e, gating be-aiy5's PR-CI lane addition
(runs TestBenchDBPurgeDoesNotLeak against a live Dolt server instead of
letting checkDolt SKIP it). Review: be-w90n, verdict pass. All 7 gate
criteria PASS; two pre-existing/unrelated failures encountered during
evaluation (be-jy56, be-vf95) were attributed and excluded from the
verdict.

* test(ci): red — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)

TestPRRiskGateReachesFullServerDoltStorageSuite now asserts
pr-risk.yml's test-server-storage-full job is a fail-fast:false matrix
over shard 1..16, with the Test step invoking
server-storage-test-shard.sh, instead of pinning the old single-job
unsharded command. Fails against the current workflow, which still
runs all 1126 tests unsharded in one 15m job (be-extn).

* feat(ci): green — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)

Adds .github/scripts/server-storage-test-shard.sh (mirrors
embedded-storage-test-shard.sh: discovers internal/storage/dolt/*_test.go,
excludes TestConformance, manifest-plus-hash-fallback shard assignment) and
its shard manifest .github/scripts/server-storage-test-shards.txt, pinning
TestCloudAuthCLIRouting (~9.5m) alone on shard 1 and spreading the other
known 10-20s-heavy tests across the rest.

pr-risk.yml's test-server-storage-full job becomes a fail-fast:false matrix
over shard 1..16, Test step now runs server-storage-test-shard.sh instead of
the old unsharded single-job command that died at 256/1126 tests after
15m28s with zero failures (job 95616282508).

Documents the new lane in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md alongside the
existing embedded matrix description.

TestPRRiskGateReachesFullServerDoltStorageSuite now passes. Offline shard
partition verified disjoint and complete across all 1125 discovered tests
(BEADS_TEST_SHARD_LIST_ONLY=1 for i=1..16). (be-extn)

* test(feat): red — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)

* feat: green — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)

* chore: release gate PASS for be-23f3

Extends deploy/be-pp7e-gate (PR gastownhall/beads#5836) with be-gepv's
fix rather than cutting a fresh isolated branch: be-gepv's own bead
text and be-extn's precedent both establish this fix belongs on the
already-open PR. All 7 gate criteria PASS; two repo-wide lint/policy
failures attributed to already-tracked pre-existing issues (be-vf95,
be-jy56), confirmed untouched by this PR lineage.

be-gepv, be-6yo1, be-23f3

* fix(dolt): federation test isolation + panic (be-3c78s)

TestFederationDatabaseIsolation and TestFederationVersionControlAPIs are
federation-prototype tests that predate the suite's shared-server,
branch-per-test isolation architecture (setupTestStore/StartTestBranch)
and were never updated to account for it.

- setupFederationStore hardcoded Database: "beads" for both towns. New()
  always connects in server mode against one shared Dolt server for the
  whole test binary (TestMain), so Path alone doesn't isolate two stores
  — Database does. Both towns silently shared one database, so Alpha
  could read Beta's issues and vice versa. Give each town its own
  database (beads_test_federation_<prefix>, matching the beads_test AD-01
  cleanup-firewall prefix).

- TestFederationDatabaseIsolation's not-found assertions assumed GetIssue
  returns (nil, nil) on a miss. It never does — issueops.GetIssueInTx
  always returns a wrapped storage.ErrNotFound. Switched to
  errors.Is(err, storage.ErrNotFound).

- TestFederationVersionControlAPIs hardcoded Checkout(ctx, "main") to
  return to its "starting" branch, but setupTestStore already checks out
  a private branch (test-<name>-<hex>) — "main" is a branch shared by
  every parallel test in the binary and never has vc-001. That mismatch
  is what produced the nil-pointer panic (GetIssue's error was discarded
  via `_`, and the resulting nil issue was dereferenced). Capture the
  real starting branch via CurrentBranch() right after setupTestStore()
  and check both previously-discarded GetIssue errors before
  dereferencing.

Neither bug is a production security issue: New(), Checkout(), and
GetIssue() all behave exactly as specified. Both bugs are confined to
this prototype test file's own setup/assertions, which hadn't caught up
to the isolation model the rest of the suite already uses correctly.

Verified via `go build`/`go vet`/`go test -c` (all clean) and running the
compiled binary, which now reaches and cleanly SKIPs both tests (no
Docker locally) instead of the previous isolation failure / panic
surfacing at compile-adjacent runtime. Full GREEN confirmation is via
CI, which has a working Dolt/Docker environment.

Refs be-3c78s

* chore: release gate PASS for be-42tlj
2026-08-19 20:39:38 -07:00

7.1 KiB

Release gate — shard test-server-storage-full so prebuilt-binary runs land in the right package dir (be-gepv)

  • Builder bead: be-gepv — fix .github/scripts/server-storage-test-shard.sh so the prebuilt-binary branch cds into internal/storage/dolt before exec, matching the go-test fallback branch's working directory.
  • Deploy bead: be-23f3
  • Review bead: be-6yo1 — verdict PASS, recorded on commit 15c6d0a7d08b8fa03e4f2a827adf23f81e55de4d
  • Commits: 2cb3d8f2845b023741d65571827267a0a5323ce4 (tdd_red) then 15c6d0a7d08b8fa03e4f2a827adf23f81e55de4d (tdd_green), 2 commits over the prior deploy tip 9caf412234fa71dc1564e8f25df6fc3be52d9e4b
  • Branch: builder/be-gepv (provenance only, not a push target)
  • Evaluated: 2026-08-18 by beads/deployer

Scope deviation: extends PR #5836 instead of a fresh isolated branch

be-gepv's own bead text is explicit that this fix targets an already-open PR: "This lands on PR #5836, branch deploy/be-pp7e-gate. ... So this fix belongs ON that PR, not on main." This matches established precedent — be-extn's prior fix (sharding test-server-storage-full's 1126-test job to stop timing out) was already pushed directly onto the same deploy/be-pp7e-gate branch rather than spawning a new isolated branch+PR. be-23f3's boilerplate description text ("cut a fresh ISOLATED deploy branch... open the PR") is the generic single-bead template; it is superseded here by the bead's own first-party provenance note and the directly-applicable precedent.

Accordingly, this gate evaluates be-gepv's own incremental diff against the existing deploy/be-pp7e-gate tip, not a full rebuild of PR #5836's 7-commit history (already separately gated: be-pp7e/be-aiy5/be-extn covered the prior commits). Ancestry scope formally validated via assert_deploy_ancestry_scope:

assert_deploy_ancestry_scope 9caf412234fa71dc1564e8f25df6fc3be52d9e4b \
  15c6d0a7d08b8fa03e4f2a827adf23f81e55de4d be-gepv be-6yo1        # rc=0
assert_deploy_ancestry_scope origin/main \
  15c6d0a7d08b8fa03e4f2a827adf23f81e55de4d \
  be-gepv be-yy25 be-pp7e be-aiy5 be-extn be-6yo1 be-w90n          # rc=0
assert_safe_push_target "deploy/be-pp7e-gate"                      # rc=0

Scope

be-gepv's own increment, confirmed via git diff --stat 9caf412234f..15c6d0a7d (2 files, 86 insertions, 0 deletions, no .claude/** paths):

  • .github/scripts/server-storage-test-shard.sh — 4-line fix: adds a comment and cd internal/storage/dolt immediately before the exec "$STORAGE_BINARY" call, on the prebuilt-binary branch only.
  • scripts/ci_workflow_test.go — new test TestServerStorageShardScriptRunsPrebuiltBinaryFromPackageDir (82 lines) asserting the fix's exact line-ordering invariants: discovery-grep before cd, cd between the prebuilt-binary guard and its exec, cd absent from the go-test fallback branch.

Gate criteria

# Criterion Verdict Evidence
1 Review PASS present PASS be-6yo1 records verdict PASS on commit 15c6d0a7d08b8fa03e4f2a827adf23f81e55de4d — exact match to this deploy's commit.
2 Acceptance criteria met PASS be-6yo1's spec_findings.tests_green: true, confirmed via TEST_COVER=1 ./scripts/test.sh; be-gepv's Done-when checklist (4 items) independently re-confirmed below.
3 Tests pass PASS Diff-owned package re-run by deployer, real PASS (see below), matching reviewer's own record.
3a Pre-existing-failure attribution N/A — attributed make ci-pr-lint repo-wide fails solely on 3x gosec G602 in backend/conformance/{cycle_detector_contract.go:494,importer_contract.go:381,relations_contract.go:672} — tracked by already-open be-vf95, confirmed byte-identical to origin/main (git log origin/main..HEAD -- backend/conformance/ empty — this PR lineage never touches that directory). make ci-pr-policy fails solely on the .githooks/commit-msg version-marker check — tracked by already-open be-jy56, likewise confirmed untouched by this lineage (git log origin/main..HEAD -- .githooks/commit-msg empty).
3b Policy/lint lane PASS Diff-scoped golangci-lint run ./scripts/... reports 0 issues. gofmt clean (via make ci-pr-lint's own Go-scoped check — "All Go files are properly formatted"; a stray manual gofmt -l invocation against the shell script itself is a self-inflicted false alarm, gofmt cannot parse non-Go source). go vet ./scripts/... clean. bash -n on the shell script clean.
4 No unresolved HIGH findings PASS be-6yo1: style_findings: none, security_findings: none — full OWASP Top-10 walk, minimal diff surface, no injection/auth/data-exposure concerns.
5 Clean working tree PASS git status --porcelain on the evaluated commit shows no staged/unstaged changes — only the pre-existing, unrelated untracked scratch files already present in this worktree (release-gates/be-hi97-*.md, release-gates/be-k9js-*.md, release-gates/be-uoat-*.md, scripts/rebase-resolve-lib.sh), never staged.
6 Clean divergence from origin/main PASS git merge-base --is-ancestor origin/main HEAD succeeds — clean fast-forward relationship. assert_deploy_ancestry_scope over the full 7-commit lineage returns rc=0 against the accepted bead-id set (see scope-deviation section above).
7 Single feature theme PASS Both files serve the one fix: the shell-script correction plus the one test file whose new case pins the corrected behavior. No unrelated changes riding along.

Tests run on release branch (independent re-verification)

Diff-owned package, run via the canonical test runner:

./scripts/test.sh ./scripts/...

Result: package github.com/steveyegge/beads/scripts — PASS in 2.439s, no skips. Includes TestServerStorageShardScriptRunsPrebuiltBinaryFromPackageDir passing for real (not skipped), confirming the line-ordering invariants the fix establishes.

Static checks, independently re-run:

Check Result
go build ./... clean
go vet ./scripts/... clean
bash -n .github/scripts/server-storage-test-shard.sh clean
golangci-lint run ./scripts/... (diff-scoped) clean, 0 issues
make ci-pr-lint (repo-wide) FAIL — solely be-vf95 (pre-existing; see criterion 3a)
make ci-pr-policy (repo-wide) FAIL — solely be-jy56 (pre-existing; see criterion 3a)

Findings from reviews (no action required)

From be-6yo1: no style or security findings. gofmt/bash -n/go vet all clean; minimal diff surface (4-line shell fix + 1 new test); no injection/auth/data-exposure/etc. concerns identified across the full OWASP Top-10 walk.

Verdict

PASS — all 7 criteria (plus 3a/3b) clear. This fix extends the already gated-and-open deploy/be-pp7e-gate branch / PR #5836 rather than cutting a new isolated branch (see scope-deviation section). Per this repo's contributor-only status, the job ends at the updated PR — no merge-request will be routed to mayor and no deploy-clearance status will be posted; merge authority belongs to the upstream maintainers.