beads/engdocs
Julian Knutsen e29b5eadb4
Write down how to retire a test against a contract, and ship the tool for it (#5425)
Follow-up to the four-slice deduplication pass (#5398, #5399, #5405, #5406).
No behaviour change.

That pass retired ~700 lines net, and its reviews found about a dozen coverage
losses and six tests that could not fail — every one of them green under
go build, go vet, go test and golangci-lint. The reasoning that caught them
lived in review transcripts and a scratch directory. Written down nowhere, the
next pass rediscovers it by shipping the loss first.

The engdoc section states each rule as the check that would have caught the
failure earlier: a mutation verdict is only true of the body you mutated; a
wrapper is a composition the contract structurally cannot see; write a residue
from the decision surface rather than the mutants you happened to run, since
half of what one such residue missed were POSITIVE assertions; ask what the
fixture makes unobservable, because every fixture-defect case had correct
assertions; never let a role-answer assertion replace a raw-row one; cite by
symbol, not line.

scripts/mutation-equivalence.sh is the tool. It ran the entire pass from a
scratch directory, which means it would not have survived it.

A fact-check review then found more wrong with this change than with anything it
describes. The section cited `runTransaction`, which does not exist in the dolt
package — while teaching symbol citation. It claimed no contract case reads
is_blocked; there are 45 references across ten files, several added by the pass
it describes. Three of its counts did not reconcile with the merged record, and
one was curated to make a sentence read true. All corrected, with the counts now
qualified and the pull requests named as the record.

The same review demonstrated three false-REDUNDANT routes in the script — the
one wrong answer that costs coverage — plus a guard that refused the repo only
by accident, and would have hard-reset the caller's own checkout on a slightly
different setup. Build failures and timeouts are now reported UNUSABLE rather
than read as catches; checkout and reset failures are fatal rather than
measuring a different commit; off-file mutations are fatal; A == B is refused;
and the worktree must be a detached scratch worktree of the repo.

A seventh guard came from using it: it measures HEAD, not the working tree, so
uncommitted work is invisible and the run reads INCONCLUSIVE. It now warns.
2026-08-07 18:58:48 -07:00
..
adr Add append-only provenance_events log and bd provenance CLI (#4461) 2026-08-07 09:43:20 -07:00
decisions docs(pipeline): pin the CLI docs to the released bd; repair main's broken CLI reference (#4876) 2026-07-17 18:03:28 -07:00
design docs: re-review and bump the three 90-day-stale freshness markers (#5394) 2026-08-07 02:35:11 -07:00
staged-for-removal docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
ADDING_AN_ISSUEOPS_ROLE.md Write down how to retire a test against a contract, and ship the tool for it (#5425) 2026-08-07 18:58:48 -07:00
AGENT_SIGNING.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
ATTRIBUTION.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
CI_CLEANUP_PLAN.md docs(lint): align contributor guidance with CI (saa-90w) (#5317) 2026-08-07 05:37:12 -07:00
CI_REQUIRED_CHECK_TOPOLOGY.md ci: shard Embedded Dolt Storage into 5 manifest-driven shards (#5117) 2026-07-28 22:16:48 -07:00
CI_TEST_SURFACE_AUDIT.md ci: shard Embedded Dolt Storage into 5 manifest-driven shards (#5117) 2026-07-28 22:16:48 -07:00
CLAUDE.md docs(test): make TESTING.md canonical (bd-mrs20) (#5226) 2026-08-01 02:30:21 -07:00
COLLISION_MATH.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
CONTRIBUTOR_NAMESPACE_ISOLATION.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
DOC_INVENTORY.md Add append-only provenance_events log and bd provenance CLI (#4461) 2026-08-07 09:43:20 -07:00
ERROR_HANDLING.md docs: port user docs into the Mintlify site tree under docs/ 2026-07-12 10:37:07 -07:00
EXCLUSIVE_LOCK.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
EXTENDING.md feat(storage): opt-in lite SELECT shape (Issue body columns deferred) (#3906) 2026-07-26 15:29:08 -07:00
ICU-POLICY.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
INTEGRATION_CHARTER.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
INTERNALS.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
LINTING.md docs(lint): align contributor guidance with CI (saa-90w) (#5317) 2026-08-07 05:37:12 -07:00
messaging.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
PERFORMANCE_TESTING.md docs(engdocs): point storage test and bench references at the Dolt packages 2026-07-18 02:43:57 +00:00
PROJECT_CHARTER.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
README_TESTING.md docs(test): make TESTING.md canonical (bd-mrs20) (#5226) 2026-08-01 02:30:21 -07:00
RELEASE-STABILITY-GATE.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
RELEASING.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
REPO_CONTEXT.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
RULES_AUDIT_DESIGN.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
SECURITY-DEPENDENCY-EXCEPTIONS.md Bump Go to 1.26.5 (#4942) 2026-07-23 18:54:32 -07:00
SERVE_RUNBOOK.md Serve a registered backend from the store bd already opened (#5352) 2026-08-05 00:15:11 -07:00
TESTING.md docs(test): make TESTING.md canonical (bd-mrs20) (#5226) 2026-08-01 02:30:21 -07:00
TESTING_PHILOSOPHY.md docs(test): make TESTING.md canonical (bd-mrs20) (#5226) 2026-08-01 02:30:21 -07:00
UI_PHILOSOPHY.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00