beads/backend
Julian Knutsen 225ed7f7bb
test(backend/http/corpus): served MC command corpus against the reference server profile (S23) (#7483)
* test(backend/http/corpus): served command corpus against the reference server profile

Add the S23 suite: every distinct bd command shape gc and its packs and
formulas issue against a work store (124 shapes in testdata/corpus.yaml),
run by the real bd binary against an in-process bd serve and compared
with the same command on an embedded workspace.

The server is bound in non-loopback mode with bearer auth. A reference
proxy masks it to a deployed bd 1.3.0 server's wire surface
(testdata/reference_server.json): exactly that server's capabilities and
handshake members (no wire_revision), a 404 for operations it does not
publish, and a 400 unknown_parameter for query parameters and request
members it does not declare. Every bd run sets BEADS_ACTOR and
CLAUDE_SESSION_ID.

A shape passes when the served run exits like the embedded run, carries
no refusal, and matches its output (json/ids/count/text) and its verify
reads. Known gaps are marked `pending` with the slice that fixes them.
A pending shape still runs, and it fails once it passes, so removing the
mark is how a slice proves itself. Permanent refusals are listed in
testdata/allowlist.yaml with reasons.

A limited listing that cuts through a tie set (same priority, created_at in
one second) lands on different rows on two workspaces seeded seconds apart;
such a shape names its unlimited form in `prefix_of`, and each side's
limited rows must be an ordered prefix of that side's own unlimited listing.

//backend/http/corpus:corpus_test takes //cmd/bd:bd_for_tests (size large,
timeout long), so under bazel the suite runs instead of skipping on a
missing BEADS_TEST_BD_BINARY.

Stacked on S6b/c/d, whose marks this file no longer carries: besides the
S6b/S6c/S6d marks, seven list shapes once marked S6c+S15 (their --limit 0
list walks the keyset cursor without S15) and the S6c+S16 last-step close
(an ordinary epic root never auto-closes) already pass on it.

On S6b/c/d (bazel, BEADS_TEST_EMBEDDED_DOLT=1): 97 pass, 25 pending,
3 allowlisted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(backend/http/corpus): hand the corpus server the LeaseReclaimer role

main gained the LeaseReclaimer role (issues.reclaim) after S23 was written,
and httpapi.Config needs it when built from roles rather than a Provider, so on the restacked
S6bcd TestServedCorpus could not bind its in-process server ("no database
source"). The harness now passes s.LeaseReclaimer() with the rest, as
backend/http's e2e harness does. The reference proxy still masks the
issues.reclaim token, so gc-reclaim stays pending.

On the restacked S6bcd (bazel, BEADS_TEST_EMBEDDED_DOLT=1): 97 pass,
25 pending, 3 allowed, unchanged from before the restack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(backend/http/corpus): run the corpus in CI, hold allowlisted shapes

Review fixes for #7483 (review attempt 1): both majors, and four of the
seven minors and nits.

No CI lane ran TestServedCorpus. corpus_test sets neither
BEADS_TEST_EMBEDDED_DOLT nor BEADS_HTTP_TEST_REQUIRED, so the suite
skipped everywhere. corpus_served_test, an embedded-tagged sh_test over
go_test_variant.sh like http_served_test, runs it with both set and with
Bazel's prebuilt bd, and pr_risk_bazel_coverage_test pins its args and
env. Every other file in the package builds only under cgo, so a new
cgo-free test fails a BEADS_HTTP_TEST_REQUIRED=1 run that would otherwise
pass having run nothing. README.md and engdocs/TESTING.md say which lane
runs the suite.

The allowlist branch returned before the embedded exit check, so an
allowlisted shape was never held to what it says on the embedded
backend. The embedded run is now checked against the shape's expect_exit
and expect_stderr before the allowlist branch, and an embedded mismatch
fails as a corpus bug whatever the shape's pending mark or allowlist
entry. TestCorpusFileIsWellFormed requires an allowlisted shape to state
expect_exit and to carry nothing the allowlist would skip (compare: exit,
no verify). The three allowlisted shapes now pin their embedded refusals,
and the bd sql reasons say the command needs a sql-server workspace
instead of calling it local-only.

Minors and nits taken:
- two stale-guard shapes (--if-assignee nobody, --if-status open) expect
  exit 13 on both sides; each kills a client that drops that guard;
- defer_until and due_at are compared as given instead of scrubbed to
  <ts>, which kills a client that shifts --defer;
- gc-reclaim's pending mark names SRV-1 alone: the reference profile
  does not advertise issues.reclaim, so the client refuses at its
  capability preflight;
- refusalMarkers matches "does not advertise capability" unquoted, which
  also catches the JSON-escaped preflight error.

Declined:
- a failure on empty-vs-empty comparisons, which needs an allow_empty
  opt-in. Review attempt 2 seeded rows for two of the three ready shapes
  that compared two empty lists (below); gc-ready-graph-anchor still
  does;
- the context route's query check in the reference proxy: the server
  already answers an unknown query parameter there with a 400
  unknown_parameter, which the proxy passes through, so the served run
  still fails; only the proxy's own refusal note is missing;
- a shared server constructor for the two e2e harnesses, which the review
  marks optional.

Review attempt 2: four of its six minors and nits.
- gc-ready-ephemeral-assignee-unlimited and gc-ready-assigned-tier
  compared two empty lists, so a client that sent the wrong assignee or
  dropped --include-ephemeral passed both. Two seed rows assigned to the
  worker, an open task (P2) and an open wisp (P1), give them
  [e2e-wisp-wr e2e-wr] and [e2e-wisp-wr]. The priorities differ, so
  --limit=1 cuts no tie. A client that rewrites the assignee, or drops
  include_ephemeral, now fails both shapes; without the rows it passed
  them.
- templatePattern looped forever on a profile path with an unclosed "{",
  growing its pattern without bound. It now returns an error, which
  loadReferenceProfile reports with the operation and path, and
  TestCorpusFileIsWellFormed loads the profile, so corpus_test catches a
  bad edit without the embedded engine.
- TestCorpusFileIsWellFormed compiles every expect_stderr and every
  allowlist refusal, and runShape compiles expect_stderr once and fails
  the shape as a corpus bug instead of panicking on a bad pattern.
- README.md documents the stdin, setup, ordered and ignore_fields
  fields.

Not in this commit:
- the PR description (shape counts, allowlist reasons, gc-reclaim's
  SRV-1 mark, the Bazel run line) is brought up to date when this commit
  is pushed;
- hoisting runBD, minimalBDEnv, buildBD and the skip gate into
  internal/testutil, or cross-reference comments in their three copies,
  which the review marks optional and which reaches outside this
  package.

The full corpus gives 127 shapes: 99 pass, 25 pending, 3 allowed, with
no shape changing status from the attempt 1 fixup, on this tree and
merged with main at 0a163ca285.

Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 05:34:51 +00:00
..
conformance feat(http): list/show counts/dep tree/ping over http, the advertised external-dependency policy, one-lookup not-found (S6b/S6c/S6d) (#7471) 2026-10-10 18:11:56 +00:00
http test(backend/http/corpus): served MC command corpus against the reference server profile (S23) (#7483) 2026-10-11 05:34:51 +00:00
backend.go feat(backends): registry Remote + OpenWith options; beads.OpenBestAvailableWith for embedders (#7188) 2026-10-04 09:34:36 -07:00
BUILD.bazel ci(bazel): split //scripts:scripts_test by what it reads; partition //:repo_files (#7414) 2026-10-09 05:13:07 +00:00
completeness_test.go test(bazel): runfiles helpers + declared inputs for scan-guard tests (#6828) 2026-09-27 08:24:08 -07:00
errors.go fix(dolt): preserve transaction outcome boundaries (#5341) 2026-08-06 04:34:46 -07:00
errors_test.go fix(dolt): preserve transaction outcome boundaries (#5341) 2026-08-06 04:34:46 -07:00
types.go show/list: render a derived GATED decoration when an open gate holds the bead (#6606) 2026-10-08 10:37:51 +00:00