* docs(agents): bazel test is authoritative; agents never run go test
Agents run tests only with `bazel test`, also while iterating on one
behavior: AGENTS.md shows the focused form
(`bazel test //cmd/bd:bd_test --config=ci --test_filter='^TestName$'`), the
target naming, that one --test_filter covers every target, that a cached
result for unchanged inputs is the same evidence, and that a new test file
needs `make bazel-sync` before Bazel runs it. `go test` (`./scripts/test.sh`,
`make test-go`) stays a convenience for humans working offline. TESTING.md's
focused red/green row and its plain-go-test section say the same.
Why: on the maintainer-city factory host, seats' own uncached `go test`
reruns of the same packages are the largest tool cost in adopt-pr, while
`bazel test` there executes on rbe-west and returns an unchanged target's
cached result in seconds.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(agents): check that a focused test ran; exempt golden regeneration
Review fixes for #7695 (iterations 1 and 2).
The focused form exits 0 with "1 test passes" when the test never ran: a
filter that matches nothing, an integration-tagged file --config=ci does not
compile, the TestEmbedded* name skip, a -short skip, and a parent test whose
every subtest skipped (Go still prints "--- PASS" for the parent). It now
adds --test_output=all, and a run counts only when its log shows
"--- PASS: <TestName>" and no "--- SKIP" for the test or a <TestName>/...
subtest; a subtest that skips by design is left out with a subtest filter.
engdocs/TESTING.md gains "Check That the Test Ran"; AGENTS.md links there
instead of restating it. The section names the tests --config=ci does not
run and the way into each:
- Dolt-backed: the three --test_env flags, or the test's Dolt lane.
- -short-gated: --test_arg=-test.short=false.
- integration-tagged: --config=integration; the Dolt-backed ones of cmd/bd,
tests/regression and internal/storage/dolt run in their Dolt-server lane
target, whose env the suite needs. No lane runs those of other packages
with Dolt, so a failure there is compared with the base commit first.
- TestEmbedded*: --test_arg=-test.skip= lifts the name skip; a test that
then still skips for BEADS_TEST_EMBEDDED_DOLT=1 runs in its package's
embedded-tagged target under --config=embedded (the cmd/bd and
internal/storage/embeddeddolt targets are named).
- files BUILD.bazel does not list: make bazel-sync.
A lane's sh_test can hand the test binary its own -test.run, which
overrides --test_filter, so a trailing --test_arg=-test.run with sharding
disabled selects the test there. In a sharded target the other shards print
"testing: warning: no tests to run". The section also gives the
target-naming rule.
Golden regeneration (make corpus-regen, make githooks-regen,
BD_UPDATE_GOLDEN=1 ... go test, -update-ledger-doc) writes the source tree,
which a Bazel test cannot do: it runs from its runfiles tree, and on
rbe-west update mode fails with "permission denied". AGENTS.md and the new
"Regenerating Golden Files" section exempt these go test runs; where a host
refuses go test even for them, a maintainer regenerates. The factory seat
shim (gastownhall/workflows#102, scripts/seat-bin/go) still refuses update
mode and needs the same carve-out there.
Also: the plain-go-test opener scopes "humans working offline" to the
runner and its go test targets, so the Dolt-backend, fixture and
testing.Short() rules still bind agents; make ci-pr-core is marked plain go
test; the beads-docs skill's docsync gate is the Bazel target; RELEASING.md
and engdocs/RELEASE-STABILITY-GATE.md name the regression suite's Bazel
target (//tests/regression:regression_test under --config=doltserver-cmd)
in place of make test-regression, which is plain go test.
Measured on rbe-west (exit 0 unless noted). Iteration 1, at c4d5409895:
- a mistyped filter, and //internal/compact TestNew (integration-tagged)
under --config=ci: "testing: warning: no tests to run"
- TestNew under --config=integration: "--- SKIP" (BEADS_TEST_SKIP=dolt);
with the Dolt flags added it runs (and fails, exit 3, on the local
backend: its store finds no "beads" database; outside this PR)
- TestConcurrentHeartbeatReclaimClose with the Dolt flags: "--- SKIP"
(short mode); plus --test_arg=-test.short=false: "--- PASS"
- //cmd/bd:bd_embedded_test, sharding disabled: --test_filter alone ran
TestEmbeddedUpdateConcurrent (manifest shard 1's test); the trailing
-test.run ran "--- PASS: TestEmbeddedShowBasicsAndJSON"
- //scripts:go_test_sources_test
TestEmbeddedCmdShardManifestBareWriteIsIdempotent: --config=ci "no tests
to run"; plus --test_arg=-test.skip=: "--- PASS"
Iteration 2, at b1d25543aa (the same Go inputs):
- //internal/storage/dolt:dolt_test TestMoveIssuePersistenceInTxTransitions
under --config=ci: "--- PASS" for the parent and "--- SKIP" for all 6
subtests (BEADS_TEST_SKIP=dolt), "1 test passes"; with the Dolt flags and
--test_filter='^TestMoveIssuePersistenceInTxTransitions$/^persistent-to-ephemeral$'
"--- PASS" for the parent and that subtest only, no "--- SKIP"
- //internal/storage/embeddeddolt:embeddeddolt_test
TestEmbeddedRemoteMigrateGate_BlocksReopen under --config=ci plus
--test_arg=-test.skip=: "--- SKIP" (BEADS_TEST_EMBEDDED_DOLT unset);
//internal/storage/embeddeddolt:embeddeddolt_embedded_test under
--config=embedded, sharding disabled, trailing -test.run: "--- PASS"
(the shard script listed 433 tests; only the named one ran)
- --config=ci plus --test_arg=-test.skip=: "--- PASS" for //tools/nogo
TestEmbeddedConfigIsTheRepositoryConfig and //internal/tracker
TestEmbeddedDoltExternalRefChangedAfterUsesHistoryFastPath
- sharding disabled, trailing -test.run: "--- PASS" for
//internal/storage/dolt:dolt_server_full_test
TestWispDeleteCascade_TransactionDeleteIssue under
--config=doltserver-integration, and for //cmd/bd:bd_dolt_server_test
TestCLI_ReopenClearsDeferUntil under --config=doltserver-cmd
- make check-docs: exit 0 (docsync: 10 test cases ran, none skipped)
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2.7 KiB
2.7 KiB
Release Stability Gate
A release MUST NOT ship until the stability gate passes. This gate protects users from silent upgrade regressions like mode flips, missing config, and data-path changes.
Upgrade Matrix
Every release candidate must pass upgrade smoke tests from these starting points:
| From Version | Mode | Scenario |
|---|---|---|
| Previous release (N-1) | Embedded (maintainer) | Init → create issues → upgrade → verify data + role |
| Previous release (N-1) | Shared-server (maintainer) | Init → create issues → upgrade → verify routing + data |
| Previous release (N-1) | Contributor | Init --contributor → upgrade → verify role preserved |
| Two releases back (N-2) | Embedded (maintainer) | Init → upgrade → verify schema migration |
What each scenario verifies
- Data preservation: issues created before upgrade are visible after upgrade
- Mode preservation:
embeddedstaysembedded,shared-serverstaysshared-server - Role preservation:
beads.rolegit config is not cleared or changed - Config continuity:
bd doctor quickpasses after upgrade - No silent errors: upgrade path produces no unexpected warnings or errors
Running the Gate
# Run all upgrade smoke tests
make test-upgrade
# Or directly:
./scripts/upgrade-smoke-test.sh
The script:
- Downloads the previous release binary (cached in
~/.cache/beads-regression/) - Creates isolated workspaces for each scenario
- Initialises with the old binary, creates test data
- Runs
bd initwith the candidate binary (simulating upgrade) - Verifies data, role, and mode are preserved
- Reports pass/fail for each scenario
Release Checklist Integration
Before cutting a release, the release process (see RELEASING.md) requires:
- Upgrade smoke tests pass (
make test-upgrade) - Breaking changes documented in CHANGELOG.md with migration steps
- If config/schema migration is involved: recovery steps documented
- Regression tests pass (
bazel test //tests/regression:regression_test --config=doltserver-cmd)
Sign-off
The person cutting the release is responsible for verifying the gate passes. If any scenario fails, the release is blocked until the failure is resolved.
Gate failures block the release — there is no override. Fix the bug or document the breaking change with explicit migration steps before shipping.