* feat(http): bd blocked over bd serve (blocked.list) through issueops.BlockedLister
`bd blocked` refused against an HTTP workspace: it called the legacy
GetBlockedIssues, which the HTTP backend stubs as unsupported. It now runs
one role on every route, and bd serve publishes that role.
- Role: issueops.BlockedLister.ListBlocked(BlockedRequest{ParentID, Labels,
LabelsAny, ExcludeLabels}) -> BlockedResult{Items}. One body,
internal/storage/issueops.ExecuteListBlockedInTx over the unchanged
GetBlockedIssuesInTx; labels normalized in the role (BlockedFilter);
Items never nil and in a total order (priority, created_at desc, id).
- Legs: DoltStore (read tx), EmbeddedDoltStore (read conn), uow (read unit
of work through the use case's GetBlockedIssues, so the
external-dependency uow decorator applies). Census: storage.DoltStorage
gains BlockedLister(); HookFiringStore passes it through (a read),
telemetry wraps it, uow.notifyingProvider, externaldeps.uowProvider and
httpapi.timedProvider build it.
- externaldeps.Store.BlockedLister is built over its own GetBlockedIssues,
so local `bd blocked` output is unchanged. Over a remote store it takes
the server's listing when the server advertises
policy.external_dependencies, else decorates the server's listing here
through EdgeReader and BatchGetter (decorateBlocked, factored out of
GetBlockedIssues).
- Wire: GET /v0/beads/blocked (listBlockedIssues; parent, label,
label_any, exclude_label) answering BlockedPage {items, has_more} with
the new x-go-type-pinned BlockedIssue schema, behind the new
blocked.list token. apigen and the wire-shape golden regenerated; the
golden diff is additive only, so wire_revision stays 2.
Config.BlockedLister is a required role, wired in bd serve.
- Client: httpclient.Store.BlockedLister dials through Store.dispatch, so
an older server is refused BEFORE the dial with *wire.CapabilityError
naming blocked.list. The legacy Store.GetBlockedIssues rides the role
(page and row cap applied to its answer) and leaves the unsupported
shell. Divergence ledger row F-blocked retired.
- CLI: `bd blocked` takes BlockedLister() from the store on the direct
routes and from the provider's BlockedListerSource on the proxied route;
runBlockedProxiedServer is deleted. --json output is the same rows in
the same order, [] when empty.
Tests: backend/conformance RunBlockedLister (10 cases: direct blocker,
child attributed to its blocked parent, closed blocker, wisp plane, pinned
status, parent filter, label filters with normalization, order, empty
slice, request not mutated) on dolt, embedded, uow and the served HTTP
client; httpapi handler tests (mapping, unknown/repeated params,
projection, capability); httpclient stub tests (pre-dial refusal naming
the token, encoding, projection, legacy door paging); skew and vocabulary
counts; TestE2E_BlockedOverHTTP (served `bd blocked --json` equals the
local command's after mapping ids and timestamps, member order included,
external blocker named, older-server refusal with no dial); corpus
gc-blocked re-marked and gc-blocked-parent added, both pending until the
reference server advertises blocked.list.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(blocked): pin server-side external decoration, refuse has_more, serve the legacy door over http
Review fixes for the blocked.list slice.
- Server-side external decoration is now pinned on the compositions bd
serve answers from. TestWrapUOWProviderBlockedListerNamesExternalBlockers
lists through the wrapped unit-of-work provider's own BlockedLister
accessor (the bd serve provider path) and requires an external-only row
and a mixed row to carry their external: refs; building that lister over
the inner provider fails it. TestServedBlockedListerCarriesExternalRefs-
WhenTheServerEnforces serves the store-shaped composition
(externaldeps.Store.BlockedLister, policy advertised) and requires the
operation's own answer, through the undecorated client role, to carry the
refs, and the client's decorated chain to return it unchanged; serving
the inner store's lister fails it.
- The client refuses an answer reporting has_more with the new typed
httpclient.TruncatedListingError instead of returning a short listing:
the operation has no cursor, and a caller reads a missing row as
unblocked. The legacy door inherits it.
- externaldeps.Store.GetBlockedIssues over an http store reads the extra
sources through the served roles (EdgeReader, BatchGetter) instead of the
raw GetIssuesByIDs / GetDependencyRecordsForIssues the http backend
refuses, and the decoration skips the source read when there is nothing
to load. The legacy door on the CLI chain (bd mol stale, the wisp
commands) now answers over http in both policy modes;
TestServedLegacyGetBlockedIssuesThroughTheDecorator pins it, paging
included. bd ready --explain still refuses on its other raw reads; the
CHANGELOG says so.
- Docs: the spec and the retired F-blocked ledger row state that the
listing is unpaged, bounded by the client's 32 MiB response limit
(refused, never read in part), and that has_more is refused.
- Contract: new cases for waits-for and conditional-blocks blockers, no
type exclusions (gate, epic), and a stale flag with no nameable blocker
(on the legs that can write the column: dolt, embedded; skipped
elsewhere with the reason). Order seeds explicit created_at values with
five rows tied on priority and created_at, two of them wisps, so only the
id tiebreak orders them; dropping the tiebreak now fails every run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(blocked): bd blocked --flags, the blocked flag per id from stdin
`bd blocked --flags [--json]` reads ids from stdin, one per line (blank
lines and repeats ignored), and prints the blocked flag bd holds for each:
with --json, [{"id","is_blocked"}] in the order the ids were first given,
omitting an id that names no row on either plane, [] for no input. It is
the readiness column a consumer otherwise had to read with `bd sql`, so no
such consumer needs a local-only command.
It is the FLAG, not the listing: `bd blocked` narrows the stored flag to
rows it can name a blocker for, and --flags does not, so a row whose flag
is set with no nameable blocker reads true here and is absent from the
listing. Where the external-dependency policy applies, an unsatisfied
external: blocker sets the flag, as it keeps the row out of `bd ready`.
One library entry on every route: issueops.BlockedFlags over the route's
BlockingAnnotator (the store's accessor on dolt, embedded and HTTP, the
provider's on the proxied route). Over HTTP that is listBlockingAnnotations
with is_blocked behind dependencies.blocking.isBlocked; a server without
that token is refused before the dial, naming it. No wire change. The
listing's filter flags are refused with --flags rather than ignored.
Tests: TestE2E_BlockedFlagsOverHTTP (served JSON equals the local command's
after mapping ids to titles, unknown id omitted, repeat collapsed, [] on
empty stdin, filters refused; a stale-flagged row with no blocker reads
true and is absent from `bd blocked --json`, which fails if --flags is
derived from the listing; an older server's handshake refused naming the
token with no dial); TestProxiedServerBlocked/flags_reads_ids_from_stdin on
the proxied route; corpus gc-blocked-flags, pending until the reference
server advertises the token. CHANGELOG entry.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(blocked): bd blocked --flags --raw, the stored is_blocked column
`bd blocked --flags --raw [--json]` answers the stored is_blocked column
for each id read from stdin, exactly what reading is_blocked from the issues
and wisps tables returns, without the external-dependency policy: a row
blocked only by an unsatisfied external: ref reads false (the policy-aware
--flags reads true), and a stale flag reads true either way. Same output
shape as --flags. This lets a consumer move off a `bd sql` read of the
column with no change in what it reads. The default --flags is unchanged.
The role is taken from beneath every decorator: storage.UnwrapStore on the
direct routes, uow.UnwrapProvider on the proxied route. Over HTTP the
innermost store is the http client, whose annotation read is the server's
answer: against a server that does not advertise
policy.external_dependencies that answer is the column, and --raw is
served. A server that advertises it decorates the flag it answers, the v0
wire has no member asking it for the column, and the decorated flag cannot
be turned back into the column client-side, so --raw against such a server
is refused before the annotation read, naming the token, rather than
answered with the decorated flag. --raw without --flags is refused.
Tests: TestE2E_BlockedFlagsOverHTTP/raw_is_the_stored_column (served --raw
equals local --raw; the externally-blocked-only row reads true with --flags
and false with --raw on both; locally blocked rows agree; a stale-flagged
row reads true in both modes; --raw without --flags exits 1; a server
advertising the policy refuses --raw with no annotation request);
TestProxiedServerBlocked/flags_raw_is_the_stored_column; corpus
gc-blocked-flags-raw (pending until the reference server advertises
dependencies.blocking.isBlocked). Routing --raw through the decorated
annotator fails the e2e. CHANGELOG entry.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(changelog): bd blocked --flags --raw is for the local and proxied routes
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(blocked): give the e2e gate-kid its own priority; document --flags answers closed ids
TestE2E_BlockedOverHTTP seeded gate-parent and gate-kid at one priority, so
when both landed in the same second their order fell to the generated ids,
which differ between the local and the served workspace: the comparison
failed intermittently. gate-kid now has its own priority. Ten runs on the
embedded lane pass.
bd blocked --flags answers a closed id too, with its stored flag; a SQL read
of the column filtered on status <> 'closed' leaves it out. Said in the
command's doc and the CHANGELOG.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(blocked): apply label filters to external-only blocked rows
The external-dependency policy appends a row blocked only by an
unsatisfied external: ref after the label query has run, so --label,
--label-any and --exclude-label never judged it: bd blocked and
GET /v0/beads/blocked listed it whatever its labels, on every route.
Both decorators now filter those rows by the request's labels. The uow
decorator reads them through GetMany, chunked at MaxGetManyIDs, so their
labels are hydrated; GetIssuesByIDs left them empty, which also printed
them unlabelled on the proxied route.
Review fixes (adopt-pr apply-fixes, iteration 1):
- major #1: label filters on external-only rows in the store and uow
decorators, with absence-asserting arms in the unit, served httpclient
(server- and client-enforced policy), e2e and proxied tiers; the spec
and the role doc say such rows are narrowed like every other row.
- minor #2: the policy.external_dependencies token prose names the
blocked listing (spec, routes.go, regenerated types.gen.go).
- minor #6: SERVE_RUNBOOK and the allowUnlimited comment record the
blocked listing's unbounded whole-set answer and why.
- minor #10: BlockedLister telemetry-outermost and typed-unsupported pins
beside the ReadyCounter pair, plus the externaldeps wrap pin.
- nit #9: F-blocked names --flags --raw's refusal against a server that
applies the policy (ledger.go and the regenerated ledger doc).
- nit #11: the remote BlockedLister no longer builds a local lister it
then discards.
Review fixes (adopt-pr apply-fixes, iteration 2; the review approved at
940 with one minor and two nits, none gating):
- minor (the --flags refusal list was unpinned): the new
TestBlockedFlagsRefusesEveryListingFilter pins blockedFlagFilterFlags
to the flags blockedCmd registers, in both directions. Every
registered flag except --flags and --raw must be refused, and every
entry must name a registered flag. The test walks Flags() rather than
LocalFlags(), so it does not merge the root's persistent flags into
blockedCmd, and it skips inherited root flags and cobra's --help.
Mutants:
- four fail it: a new filter flag left off the list, a dropped
entry, a misspelled entry, and --raw added to the list;
- a simulated earlier in-process run (merged root flags plus --help)
stays green, and fails once either skip is removed.
- nit (the remote branch's span): the remote BlockedLister comment now
says why that lister is not wrapped again. The inner lister already
carries telemetry's span, so wrapping would nest a second span of the
same name. The comment also says what runs outside that span.
- nit (duplicated order comparator): left as is. finishBlockedIssues
predates this PR, so merging the two sorts would refactor existing
code with no review pass left after this one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| testdata | ||
| BUILD.bazel | ||
| corpus_parts_test.go | ||
| corpus_test.go | ||
| gate_check_served_test.go | ||
| harness_test.go | ||
| README.md | ||
| reference_proxy_test.go | ||
| required_corpus_tier_test.go | ||
| served_pinned_shards.txt | ||
Served command corpus
This suite is the working definition of "bd fully supports a remote HTTP
backend". It runs every distinct bd command shape that an orchestrator (gc)
and its packs and formulas issue against a work store. Each shape runs
against a served workspace and is compared with the same command on an
embedded workspace.
What runs
The shapes are dealt round robin, in corpus order, across corpusParts
parts (corpus_test.go). Each part is a top-level test,
TestServedCorpusPart1 .. TestServedCorpusPartN, and each does the
following for its share of the shapes:
- Builds
bdfrom this tree.BEADS_TEST_BD_BINARY=<path>reuses a prebuilt binary instead. - Starts an in-process
bd serve(httpapi.Listenover embedded Dolt) the way a deployed server is bound:AllowNonLoopbackplus a bearer token file. The socket is still127.0.0.1, but the bind mode decides every behaviour the server keys on, including its refusal of unlimited reads. - Puts a reference proxy in front of the server. The proxy makes it answer
like the deployed reference server described in
testdata/reference_server.json:- the context handshake publishes exactly that server's capability set and
its
ContextResponsemembers (so nowire_revision); - an operation the reference server does not publish gets the server's own 404;
- a query parameter or top-level request member it does not declare gets
the server's own 400
unknown_parameter.
- the context handshake publishes exactly that server's capability set and
its
- Runs
bd connecton one workspace andbd initon another. Both get theseedfromtestdata/corpus.yaml: one graph of named rows, plus bulk graphs that put more than one server page of ready rows and wisps in the store. Both sides useBEADS_ACTOR=workerand setCLAUDE_SESSION_ID. The embedded side is initialized and seeded on its own goroutine while steps 2 and 3 run and the served side is seeded (prepareSide): the two sides share no directory, env or process, and each runs the seed in order, so the seeded data does not depend on the overlap. - Runs each of its shapes as a subtest, on both sides. The embedded run must
exit with the shape's
expect_exitand match itsexpect_stderr. If it does not, the shape is wrong or bd changed, and the subtest fails as a corpus bug that no pending mark or allowlist entry hides. The served run must:- exit like the embedded run;
- carry no refusal (
ErrUnsupported, a v0-wire refusal, the loopback-only 400, or a reference-profile 404/400); - match the embedded output under the shape's
comparemode, after timestamps and revision tokens are dropped (but notdefer_untilordue_at: a shape sets those itself, as absolute dates, and they must match exactly); - for writes, give the same JSON from every
verifyread on both sides afterwards.
A part takes its own server, workspaces and seed, so no shape can see another part's writes. That is safe because no shape depends on another shape's effects:
- every check compares the served run with the embedded run after the same commands, so it holds whatever ran before it on both sides;
- each part keeps corpus order, so its reads still run on the bare seed, before any of its writes;
- each write owns its target ids (
e2e-t-*, or ids it creates).
TestCorpusPartsDealEveryShapeOnce checks that every shape is in exactly one
part, in corpus order. A new shape must keep that independence: it may rely on
the seed and its own setup, never on another shape having run.
Every part seeds again, so running all the parts in one process takes
longer than one run over every shape did; each part logs its setup time
(part N setup ...). Run one part with
-run 'TestServedCorpusPart2$'.
There are four parts. In the embedded lane each part's setup (server start
plus both seeds, run concurrently) takes ~62-76s and the shapes ~242s in
total. Four parts ran 114-146s each and three ran 170-180s. Three would save
one setup (~75 test-seconds a run), but the longest part is the lane's
served long pole and sits on the CI gate's critical path, so four parts win.
To change the count, update corpusParts, servedCorpusParts,
served_pinned_shards.txt and the target's shard_count together.
CGO_ENABLED=1 BEADS_TEST_EMBEDDED_DOLT=1 BEADS_HTTP_TEST_REQUIRED=1 \
go test -tags gms_pure_go -run 'TestServedCorpus|TestCorpus' ./backend/http/corpus/
In CI, //backend/http/corpus:corpus_served_test runs the suite with both
variables set, in the Embedded Dolt lane (--config=embedded; see
engdocs/TESTING.md). Its
served_pinned_shards.txt pins part k to Bazel shard k, so the parts run
concurrently. corpus_test on its own sets neither variable, so it skips
the parts.
Two environment variables change the run:
BEADS_CORPUS_REPORT=<file>writes a JSON summary with per-shape status and request counts.BEADS_CORPUS_SKIP_PENDING=1skips pending shapes without running them.
The data
| File | Holds |
|---|---|
testdata/corpus.yaml |
assets, the seed, and the shapes |
testdata/allowlist.yaml |
permanent refusals, each with its reason and the refusal text the served run must match |
testdata/reference_server.json |
the reference server's wire surface |
Each shape in corpus.yaml has these fields:
| Field | Meaning |
|---|---|
id |
gc-* for orchestrator subprocess shapes, mc-* for pack/formula/prompt shapes |
source |
where the shape comes from |
args |
the argv; {file:NAME} expands to an asset path and {rev:ID} to that side's current revision of ID |
stdin |
text piped to bd on both sides |
setup |
commands run on both sides before the shape; one that fails fails the shape |
compare |
json (default), ids, count, text or exit |
ordered |
under json and ids, array order is significant; by default arrays are sorted before comparing |
prefix_of |
for a limited ids listing: its unlimited form. Each side's limited rows must be a prefix, in order, of that side's own unlimited listing, both sides return as many rows, and the two unlimited listings hold the same ids. A limit that cuts through a tie set (same priority, created_at in the same second) lands on different rows on two workspaces seeded seconds apart, so the limited rows are not compared across sides directly |
verify |
reads run after the shape |
ignore_fields |
JSON keys dropped, at any depth, before a json comparison and before verify output is compared; for example comment ids, which each workspace mints itself |
expect_exit, expect_stderr |
expected outcome on both sides (exit 0 by default). An allowlisted shape must state expect_exit; only its embedded run is held to these |
pending |
a known failure |
requests |
exactly how many requests the served run of args makes through the reference proxy, not counting setup, verify or {rev:ID} reads. Each request is a sequential round trip, so the count is the shape's latency floor over a gateway. A change that makes the shape cheaper updates the pin |
pending: {slice, reason} marks a known failure and names the slice or
slices that fix it, for example S6c+S15. A pending shape still runs:
- while it fails, the subtest is skipped and the skip records what was observed;
- once it passes, the subtest fails with "now passes; remove the pending mark".
So removing a mark is how a slice proves itself, and any other failure is unexpected and fails the suite.
Updating the reference profile
reference_server.json was derived from two things:
- the deployed server's live
GET /v0/beads/contextcapabilities; - its published OpenAPI document (bd 1.3.0, no
wire_revision). For each operation the profile keeps the method, the path template, the query parameter names, and the top-level request members.
When the deployed server is upgraded (SRV-1), regenerate the file the same
way. Then remove the SRV-1 pending marks that now pass.
Known fidelity limits
- Responses pass through unfiltered. A response member the reference server would not send still reaches the client.
- Only top-level request members are checked; nested members (for example
batchApplyitems) are not. - The served backend underneath is embedded Dolt, not the reference deployment's storage backend.
- Out of scope, because they are local by design or never touch a remote
store:
init,bootstrap,migrate,doctor,flatten,gc, and thedoltsubcommands.context,version,backup statusandprimeare in the corpus, but only checked for exit codes.