beads/internal/httpapi/wire_bijection_test.go
Julian Knutsen 4450fe99bb
feat(http): bd blocked over bd serve (blocked.list), plus bd blocked --flags [--raw] (#7649)
* feat(http): bd blocked over bd serve (blocked.list) through issueops.BlockedLister

`bd blocked` refused against an HTTP workspace: it called the legacy
GetBlockedIssues, which the HTTP backend stubs as unsupported. It now runs
one role on every route, and bd serve publishes that role.

- Role: issueops.BlockedLister.ListBlocked(BlockedRequest{ParentID, Labels,
  LabelsAny, ExcludeLabels}) -> BlockedResult{Items}. One body,
  internal/storage/issueops.ExecuteListBlockedInTx over the unchanged
  GetBlockedIssuesInTx; labels normalized in the role (BlockedFilter);
  Items never nil and in a total order (priority, created_at desc, id).
- Legs: DoltStore (read tx), EmbeddedDoltStore (read conn), uow (read unit
  of work through the use case's GetBlockedIssues, so the
  external-dependency uow decorator applies). Census: storage.DoltStorage
  gains BlockedLister(); HookFiringStore passes it through (a read),
  telemetry wraps it, uow.notifyingProvider, externaldeps.uowProvider and
  httpapi.timedProvider build it.
- externaldeps.Store.BlockedLister is built over its own GetBlockedIssues,
  so local `bd blocked` output is unchanged. Over a remote store it takes
  the server's listing when the server advertises
  policy.external_dependencies, else decorates the server's listing here
  through EdgeReader and BatchGetter (decorateBlocked, factored out of
  GetBlockedIssues).
- Wire: GET /v0/beads/blocked (listBlockedIssues; parent, label,
  label_any, exclude_label) answering BlockedPage {items, has_more} with
  the new x-go-type-pinned BlockedIssue schema, behind the new
  blocked.list token. apigen and the wire-shape golden regenerated; the
  golden diff is additive only, so wire_revision stays 2.
  Config.BlockedLister is a required role, wired in bd serve.
- Client: httpclient.Store.BlockedLister dials through Store.dispatch, so
  an older server is refused BEFORE the dial with *wire.CapabilityError
  naming blocked.list. The legacy Store.GetBlockedIssues rides the role
  (page and row cap applied to its answer) and leaves the unsupported
  shell. Divergence ledger row F-blocked retired.
- CLI: `bd blocked` takes BlockedLister() from the store on the direct
  routes and from the provider's BlockedListerSource on the proxied route;
  runBlockedProxiedServer is deleted. --json output is the same rows in
  the same order, [] when empty.

Tests: backend/conformance RunBlockedLister (10 cases: direct blocker,
child attributed to its blocked parent, closed blocker, wisp plane, pinned
status, parent filter, label filters with normalization, order, empty
slice, request not mutated) on dolt, embedded, uow and the served HTTP
client; httpapi handler tests (mapping, unknown/repeated params,
projection, capability); httpclient stub tests (pre-dial refusal naming
the token, encoding, projection, legacy door paging); skew and vocabulary
counts; TestE2E_BlockedOverHTTP (served `bd blocked --json` equals the
local command's after mapping ids and timestamps, member order included,
external blocker named, older-server refusal with no dial); corpus
gc-blocked re-marked and gc-blocked-parent added, both pending until the
reference server advertises blocked.list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(blocked): pin server-side external decoration, refuse has_more, serve the legacy door over http

Review fixes for the blocked.list slice.

- Server-side external decoration is now pinned on the compositions bd
  serve answers from. TestWrapUOWProviderBlockedListerNamesExternalBlockers
  lists through the wrapped unit-of-work provider's own BlockedLister
  accessor (the bd serve provider path) and requires an external-only row
  and a mixed row to carry their external: refs; building that lister over
  the inner provider fails it. TestServedBlockedListerCarriesExternalRefs-
  WhenTheServerEnforces serves the store-shaped composition
  (externaldeps.Store.BlockedLister, policy advertised) and requires the
  operation's own answer, through the undecorated client role, to carry the
  refs, and the client's decorated chain to return it unchanged; serving
  the inner store's lister fails it.
- The client refuses an answer reporting has_more with the new typed
  httpclient.TruncatedListingError instead of returning a short listing:
  the operation has no cursor, and a caller reads a missing row as
  unblocked. The legacy door inherits it.
- externaldeps.Store.GetBlockedIssues over an http store reads the extra
  sources through the served roles (EdgeReader, BatchGetter) instead of the
  raw GetIssuesByIDs / GetDependencyRecordsForIssues the http backend
  refuses, and the decoration skips the source read when there is nothing
  to load. The legacy door on the CLI chain (bd mol stale, the wisp
  commands) now answers over http in both policy modes;
  TestServedLegacyGetBlockedIssuesThroughTheDecorator pins it, paging
  included. bd ready --explain still refuses on its other raw reads; the
  CHANGELOG says so.
- Docs: the spec and the retired F-blocked ledger row state that the
  listing is unpaged, bounded by the client's 32 MiB response limit
  (refused, never read in part), and that has_more is refused.
- Contract: new cases for waits-for and conditional-blocks blockers, no
  type exclusions (gate, epic), and a stale flag with no nameable blocker
  (on the legs that can write the column: dolt, embedded; skipped
  elsewhere with the reason). Order seeds explicit created_at values with
  five rows tied on priority and created_at, two of them wisps, so only the
  id tiebreak orders them; dropping the tiebreak now fails every run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(blocked): bd blocked --flags, the blocked flag per id from stdin

`bd blocked --flags [--json]` reads ids from stdin, one per line (blank
lines and repeats ignored), and prints the blocked flag bd holds for each:
with --json, [{"id","is_blocked"}] in the order the ids were first given,
omitting an id that names no row on either plane, [] for no input. It is
the readiness column a consumer otherwise had to read with `bd sql`, so no
such consumer needs a local-only command.

It is the FLAG, not the listing: `bd blocked` narrows the stored flag to
rows it can name a blocker for, and --flags does not, so a row whose flag
is set with no nameable blocker reads true here and is absent from the
listing. Where the external-dependency policy applies, an unsatisfied
external: blocker sets the flag, as it keeps the row out of `bd ready`.

One library entry on every route: issueops.BlockedFlags over the route's
BlockingAnnotator (the store's accessor on dolt, embedded and HTTP, the
provider's on the proxied route). Over HTTP that is listBlockingAnnotations
with is_blocked behind dependencies.blocking.isBlocked; a server without
that token is refused before the dial, naming it. No wire change. The
listing's filter flags are refused with --flags rather than ignored.

Tests: TestE2E_BlockedFlagsOverHTTP (served JSON equals the local command's
after mapping ids to titles, unknown id omitted, repeat collapsed, [] on
empty stdin, filters refused; a stale-flagged row with no blocker reads
true and is absent from `bd blocked --json`, which fails if --flags is
derived from the listing; an older server's handshake refused naming the
token with no dial); TestProxiedServerBlocked/flags_reads_ids_from_stdin on
the proxied route; corpus gc-blocked-flags, pending until the reference
server advertises the token. CHANGELOG entry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(blocked): bd blocked --flags --raw, the stored is_blocked column

`bd blocked --flags --raw [--json]` answers the stored is_blocked column
for each id read from stdin, exactly what reading is_blocked from the issues
and wisps tables returns, without the external-dependency policy: a row
blocked only by an unsatisfied external: ref reads false (the policy-aware
--flags reads true), and a stale flag reads true either way. Same output
shape as --flags. This lets a consumer move off a `bd sql` read of the
column with no change in what it reads. The default --flags is unchanged.

The role is taken from beneath every decorator: storage.UnwrapStore on the
direct routes, uow.UnwrapProvider on the proxied route. Over HTTP the
innermost store is the http client, whose annotation read is the server's
answer: against a server that does not advertise
policy.external_dependencies that answer is the column, and --raw is
served. A server that advertises it decorates the flag it answers, the v0
wire has no member asking it for the column, and the decorated flag cannot
be turned back into the column client-side, so --raw against such a server
is refused before the annotation read, naming the token, rather than
answered with the decorated flag. --raw without --flags is refused.

Tests: TestE2E_BlockedFlagsOverHTTP/raw_is_the_stored_column (served --raw
equals local --raw; the externally-blocked-only row reads true with --flags
and false with --raw on both; locally blocked rows agree; a stale-flagged
row reads true in both modes; --raw without --flags exits 1; a server
advertising the policy refuses --raw with no annotation request);
TestProxiedServerBlocked/flags_raw_is_the_stored_column; corpus
gc-blocked-flags-raw (pending until the reference server advertises
dependencies.blocking.isBlocked). Routing --raw through the decorated
annotator fails the e2e. CHANGELOG entry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(changelog): bd blocked --flags --raw is for the local and proxied routes

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(blocked): give the e2e gate-kid its own priority; document --flags answers closed ids

TestE2E_BlockedOverHTTP seeded gate-parent and gate-kid at one priority, so
when both landed in the same second their order fell to the generated ids,
which differ between the local and the served workspace: the comparison
failed intermittently. gate-kid now has its own priority. Ten runs on the
embedded lane pass.

bd blocked --flags answers a closed id too, with its stored flag; a SQL read
of the column filtered on status <> 'closed' leaves it out. Said in the
command's doc and the CHANGELOG.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(blocked): apply label filters to external-only blocked rows

The external-dependency policy appends a row blocked only by an
unsatisfied external: ref after the label query has run, so --label,
--label-any and --exclude-label never judged it: bd blocked and
GET /v0/beads/blocked listed it whatever its labels, on every route.
Both decorators now filter those rows by the request's labels. The uow
decorator reads them through GetMany, chunked at MaxGetManyIDs, so their
labels are hydrated; GetIssuesByIDs left them empty, which also printed
them unlabelled on the proxied route.

Review fixes (adopt-pr apply-fixes, iteration 1):
- major #1: label filters on external-only rows in the store and uow
  decorators, with absence-asserting arms in the unit, served httpclient
  (server- and client-enforced policy), e2e and proxied tiers; the spec
  and the role doc say such rows are narrowed like every other row.
- minor #2: the policy.external_dependencies token prose names the
  blocked listing (spec, routes.go, regenerated types.gen.go).
- minor #6: SERVE_RUNBOOK and the allowUnlimited comment record the
  blocked listing's unbounded whole-set answer and why.
- minor #10: BlockedLister telemetry-outermost and typed-unsupported pins
  beside the ReadyCounter pair, plus the externaldeps wrap pin.
- nit #9: F-blocked names --flags --raw's refusal against a server that
  applies the policy (ledger.go and the regenerated ledger doc).
- nit #11: the remote BlockedLister no longer builds a local lister it
  then discards.

Review fixes (adopt-pr apply-fixes, iteration 2; the review approved at
940 with one minor and two nits, none gating):
- minor (the --flags refusal list was unpinned): the new
  TestBlockedFlagsRefusesEveryListingFilter pins blockedFlagFilterFlags
  to the flags blockedCmd registers, in both directions. Every
  registered flag except --flags and --raw must be refused, and every
  entry must name a registered flag. The test walks Flags() rather than
  LocalFlags(), so it does not merge the root's persistent flags into
  blockedCmd, and it skips inherited root flags and cobra's --help.
  Mutants:
  - four fail it: a new filter flag left off the list, a dropped
    entry, a misspelled entry, and --raw added to the list;
  - a simulated earlier in-process run (merged root flags plus --help)
    stays green, and fails once either skip is removed.
- nit (the remote branch's span): the remote BlockedLister comment now
  says why that lister is not wrapped again. The inner lister already
  carries telemetry's span, so wrapping would nest a second span of the
  same name. The comment also says what runs outside that span.
- nit (duplicated order comparator): left as is. finishBlockedIssues
  predates this PR, so merging the two sorts would refactor existing
  code with no review pass left after this one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
2026-10-17 12:29:13 +00:00

213 lines
8.3 KiB
Go

package httpapi
import (
"reflect"
"sort"
"strings"
"testing"
"github.com/steveyegge/beads/internal/eventsjournal"
"github.com/steveyegge/beads/internal/types"
"github.com/steveyegge/beads/issueops"
)
// pinnedSchemas are the schemas welded to canonical Go structs with x-go-type.
// Because the generator emits a type alias for them, the compiler cannot see
// the spec's property list at all — this test is the ONLY guard that the
// document still describes the struct it claims to describe.
//
// EVERY x-go-type-pinned schema belongs here, not just the top-level response
// bodies. Dependency, Comment and BondRef are nested inside issue bodies, so a
// field added to one of them ships on the wire just as silently — and more
// silently than an Issue field, which a populated `omitempty` tag would at
// least surface in the byte-pinned CLI protocol corpus. internal/httpapi/
// pinning.go proves those three are aliases; only this test proves the document
// still describes them.
// The import path is per-entry rather than one constant, because the canonical
// structs do not all live in one package: ten are internal/types, and the
// three that are not — the cycle pair and IssueBlocking — live in the role
// package issueops, where the results of issueops.CycleDetector and
// issueops.BlockingAnnotator are declared, and EventRecord lives in
// internal/eventsjournal, which owns the journal's published envelope for both
// front doors. Asserting the declared path per schema keeps the pin's TARGET
// checked, which is what a bare x-go-type name would not do.
var pinnedSchemas = []struct {
name string
goType string
importPath string
value any
}{
{"Issue", "types.Issue", canonicalTypesImport, types.Issue{}},
{"IssueWithCounts", "types.IssueWithCounts", canonicalTypesImport, types.IssueWithCounts{}},
{"IssueDetails", "types.IssueDetails", canonicalTypesImport, types.IssueDetails{}},
{"IssueWithDependencyMetadata", "types.IssueWithDependencyMetadata", canonicalTypesImport, types.IssueWithDependencyMetadata{}},
{"TreeNode", "types.TreeNode", canonicalTypesImport, types.TreeNode{}},
{"Dependency", "types.Dependency", canonicalTypesImport, types.Dependency{}},
{"Comment", "types.Comment", canonicalTypesImport, types.Comment{}},
{"BondRef", "types.BondRef", canonicalTypesImport, types.BondRef{}},
{"Statistics", "types.Statistics", canonicalTypesImport, types.Statistics{}},
{"BlockedIssue", "types.BlockedIssue", canonicalTypesImport, types.BlockedIssue{}},
{"Cycle", "issueops.Cycle", canonicalRolesImport, issueops.Cycle{}},
{"CycleMember", "issueops.CycleMember", canonicalRolesImport, issueops.CycleMember{}},
{"IssueBlocking", "issueops.IssueBlocking", canonicalRolesImport, issueops.IssueBlocking{}},
{"EventRecord", "eventsjournal.Record", canonicalJournalImport, eventsjournal.Record{}},
}
const (
canonicalTypesImport = "github.com/steveyegge/beads/internal/types"
canonicalRolesImport = "github.com/steveyegge/beads/issueops"
// The journal record lives in neither: it is the leaf package that owns the
// journal's cross-binary concerns, and its Record is what `bd events tail`
// marshals.
canonicalJournalImport = "github.com/steveyegge/beads/internal/eventsjournal"
)
// omittedProperties lists JSON field names a pinned schema deliberately does
// not document, keyed by schema name. It STARTS EMPTY and must stay that way
// unless there is a reason worth writing down here: every entry is a hole in
// the only check that the wire contract matches the struct.
var omittedProperties = map[string]map[string]string{}
// TestWireTagBijection asserts two-way set equality between the JSON tags of
// the canonical structs and the properties of the schemas pinned to them.
//
// Two-way, and reflection-based, on purpose. A round-trip fixture test is not
// a substitute: a newly added `omitempty` field that the fixture does not
// mention round-trips cleanly in both directions and the drift ships silently.
// Here, a field added to types.Issue without a spec entry fails CI, and a spec
// property with no Go field fails CI.
func TestWireTagBijection(t *testing.T) {
doc := loadSpec(t)
schemas := mapAt(t, mapAt(t, doc, "components"), "schemas")
for _, pinned := range pinnedSchemas {
t.Run(pinned.name, func(t *testing.T) {
schema := mapAt(t, schemas, pinned.name)
// The pin itself is part of the contract: without x-go-type the
// generator would emit a mirror struct and this test would be
// comparing the spec against a type nothing serializes.
if got, _ := schema["x-go-type"].(string); got != pinned.goType {
t.Fatalf("x-go-type = %q, want %q", got, pinned.goType)
}
imp := mapAt(t, schema, "x-go-type-import")
if got, _ := imp["path"].(string); got != pinned.importPath {
t.Fatalf("x-go-type-import.path = %q, want %q", got, pinned.importPath)
}
specProps := schemaProperties(t, doc, schema)
goProps := jsonTagNames(t, reflect.TypeOf(pinned.value))
for name, why := range omittedProperties[pinned.name] {
if why == "" {
t.Fatalf("omitted property %q carries no justification", name)
}
delete(goProps, name)
delete(specProps, name)
}
if missing := diff(goProps, specProps); len(missing) > 0 {
t.Errorf("Go fields with no spec property: %v\n"+
"add them to the %s schema in internal/httpapi/spec/openapi.v0.yaml — they are on the wire already",
missing, pinned.name)
}
if extra := diff(specProps, goProps); len(extra) > 0 {
t.Errorf("spec properties with no Go field: %v\n"+
"the %s schema promises fields that %s never marshals",
extra, pinned.name, pinned.goType)
}
})
}
}
// jsonTagNames collects the serialized names of a struct, walking embedded
// structs the way encoding/json does. Fields tagged `json:"-"` carry no wire
// name and are excluded by construction, which is what keeps internal-only
// fields (content hashes, row versions, routing overrides, the lite-partial
// marker) out of the contract without an allowlist entry.
//
// An exported field with NO json tag is a wire field too — encoding/json keys
// it by its Go name — so it counts under that name. Treating a missing tag as
// "not on the wire" would have left the only hole in this gate: an untagged
// field ships in every response while the schema says nothing about it.
func jsonTagNames(t *testing.T, rt reflect.Type) map[string]bool {
t.Helper()
for rt.Kind() == reflect.Pointer {
rt = rt.Elem()
}
if rt.Kind() != reflect.Struct {
t.Fatalf("%s is not a struct", rt)
}
out := map[string]bool{}
for i := range rt.NumField() {
f := rt.Field(i)
name := strings.Split(f.Tag.Get("json"), ",")[0]
if f.Anonymous && name == "" {
et := f.Type
for et.Kind() == reflect.Pointer {
et = et.Elem()
}
if et.Kind() == reflect.Struct {
// Promoted fields are wire fields. Shadowing (an outer field
// with the same JSON name as an embedded one) collapses into
// the same set entry, exactly as it collapses into one key on
// the wire.
for n := range jsonTagNames(t, et) {
out[n] = true
}
continue
}
// An embedded non-struct (or embedded pointer to one) marshals
// under its TYPE name.
name = f.Name
}
if name == "-" || !f.IsExported() {
continue
}
if name == "" {
name = f.Name
}
out[name] = true
}
return out
}
// schemaProperties collects a schema's property names. allOf branches are
// unioned in (and local $refs resolved) so the check keeps working if the
// document ever composes — though it cannot today: oapi-codegen v2.6.0 drops
// x-go-type from an allOf-bearing component, which internal/httpapi/pinning.go
// turns into a build failure.
func schemaProperties(t *testing.T, doc, schema map[string]any) map[string]bool {
t.Helper()
out := map[string]bool{}
if props, ok := schema["properties"].(map[string]any); ok {
for name := range props {
out[name] = true
}
}
if branches, ok := schema["allOf"].([]any); ok {
for _, raw := range branches {
branch, ok := raw.(map[string]any)
if !ok {
t.Fatalf("allOf branch is %T, want a mapping", raw)
}
for name := range schemaProperties(t, doc, resolveRef(t, doc, branch)) {
out[name] = true
}
}
}
return out
}
func diff(a, b map[string]bool) []string {
var out []string
for k := range a {
if !b[k] {
out = append(out, k)
}
}
sort.Strings(out)
return out
}