Commit graph

5 commits

Author SHA1 Message Date
Julian Knutsen
9a2c918b0e
Admit the ephemeral plane from a list request, on the facade and on the wire (#5476)
* feat(reader): admit the ephemeral plane from a list request

`issueops.Reader.Ready` has had an ephemeral gate since it was written;
`List` has not. The wisp plane was reachable from a list request only as
a SIDE EFFECT of `IncludeInfra`, which also drops the infra-type
exclusions, or by naming an infra type, which routes to that plane
ALONE. A caller that wants the ephemeral rows of ordinary types — a
message bead, say — could ask for them on the ready query and not on the
listing, and the wire said so in prose: listIssues published no
`include_ephemeral`, and the document called the asymmetry out.

ListRequest.IncludeEphemeral closes it, with the same semantics
ReadyRequest.IncludeEphemeral already has.

  false (zero value)  durable rows only — today's listing, unchanged
  true                the wisp plane merged IN ADDITION; the answer is a
                      superset of the false answer under the same
                      filters, and never ephemeral-only
  WispType            a predicate on the wisp_type COLUMN, which BOTH
                      tables carry, so it selects no plane: it narrows
                      whatever was admitted. WispType alone answers with
                      nothing, since no durable row carries a
                      classification; WispType + IncludeEphemeral is
                      lawful and composes as an ordinary AND
  order               ONE order across both planes, by SortBy, as if
                      they were a single table — not durable rows
                      followed by ephemeral ones. A Limit truncates the
                      merged order, and the keyset position is applied
                      to BOTH legs, so a cursored walk over the merge
                      drops no row and repeats none. The non-snapshot
                      caveat is the one every multi-page walk already
                      has; the ephemeral plane only turns over faster
  --ready             CARRIED, not dropped. The ready query has an
                      ephemeral gate of its own, so
                      ReadyFilterFromIssueFilter now reads the plane bit
                      off the list filter. That also carries
                      IncludeInfra's plane half, which the projection
                      used to drop silently — `bd list --ready
                      --include-infra` now answers with the wisps `bd
                      count --include-infra` already counts

It is a PLANE knob and not a TYPE knob, which is the whole difference
from the three fields beside it: templates, gates and infra types stay
hidden by their own predicates. IncludeInfra remains strictly wider.

The wire gets `include_ephemeral` on listIssues, spec-first. There is no
capability-token change: it is an ordinary optional query parameter, so
an older server answers it with 400 `unknown_parameter` — the designed
skew signal, whose recovery is to degrade rather than retry.

Conformance runs on all three legs (server-backed store, embedded store,
unit of work) and covers admission, the merged order, every page bound
across the interleaving, a cursored walk across the plane boundary, and
the WispType interaction. The two seams arrange the merge differently —
one merge-sorts two independently ordered legs in Go and trims, the
other ORDERs a SQL UNION ALL — so the bounded and cursored arms are a
genuine two-body case rather than one body twice.

* fix(reader): correct the flag's reach, and settle the merge seam it lands on

Review follow-up. Three of these are documentation being wrong about the
code; one is the code itself.

THE MOTIVATING EXAMPLE WAS FALSE. `message` is one of the DEFAULT INFRA
TYPES (agent, role, message), so "the ephemeral rows of ordinary types —
a message bead, say" named the one row this flag cannot reach: the
infra-type exclusion is a TYPE predicate and survives the plane knob, as
the golden already recorded and nothing asserted. `bd ready` cannot
deliver it either — ReadyWorkExcludeTypes appends the infra set
unconditionally. The field doc, the operation description and the
parameter text now say that ephemeral agent/role/message rows need
IncludeInfra, and what the flag alone reaches is the ephemeral rows of
the types a listing already shows.

The contract case is what makes it more than a corrected sentence. The
fixture's only wisp was a task, which comes back under EITHER knob, so a
body that wired the plane flag to IncludeInfra's branch answered every
row correctly. An infra-typed wisp is hidden twice over — once by the
plane, once by the type — and is the only row that can tell the two
apart. Three rows now pin it: absent under IncludeEphemeral, present
under IncludeInfra, present under both. Wiring the flag into the
type branch fails it.

TWO MORE DOC CORRECTIONS on the same field. It admits the wisps TABLE,
which holds no-history rows (ephemeral = 0) alongside true ephemerals —
stated, rather than left as "the ephemeral plane". And it is NOT
identical to ReadyRequest.IncludeEphemeral: ready reads both planes
either way and adds a per-ROW ephemeral predicate when unset, where this
selects which TABLES are read. The difference is observable on exactly
those no-history rows, which a DEFAULT ready answer already contains and
a default listing does not.

The ready projection's predicate is negative, on a field whose zero
value means admit, so it fires for a filter that merely never set
SkipWisps rather than for one that asked. Every caller is handed a
BuildListFilter result, which decides that field for every request, so
"unset" is unreachable today — named at the site so the next hand-built
filter does not walk into it.

THE SEAM DISAGREEMENT, which is the code change. A cross-plane duplicate
id was tolerated by the per-table seam (collapse to the canonical wisp,
answer) and HARD-ERRORED by the union seam, and this flag routes every
wisp-inclusive read onto the union. Two implementations of one role
answering a page and an error is a divergence, not a design, and leaving
it silent while making it reachable is worse than either behavior.

The union now resolves it the way its sibling does, at all four sites,
and the strict flag is gone rather than left as a footgun. Three things
say that is the right verdict and not merely the convenient one: the
per-table seam has said so since be-iabdi ("hard-erroring breaks every
lookup city-wide"); `bd doctor` detects the state with a query of its
own, so answering blinds no detector; and its fix deletes the stale
ISSUES copy, which is exactly the row the union now drops. Its own check
text names the hard failure as the damage — "stale issues-table copies
break every lookup for the affected IDs". The four tests that pinned the
error now pin the wisp-canonical answer; inverting the preference fails
all four.

Preferring the wisp needs a second pass: the outer ORDER BY can place
either copy first, so which leg produced a row is known in full only
after the scan.

Last, a stale comment: issueops/count.go claimed SearchIssuesInTx was
the corruption detector. It has not been one since be-iabdi. The count
trusts the disjoint-table invariant and no read enforces it, so a
dual-resident id is counted twice and listed once; doctor is the
detector and the repair.

---------

Co-authored-by: Review Sim <ci@beads.test>
2026-08-09 05:06:36 -07:00
CI Bot
aa10a725e5 feat(list): put bd list's row cap and counts hydration on ListRequest (bd-b34o7)
STAGE ONE OF TWO. Fields, not methods, and no route flips — those are stage 2
(bd-5k21g). `issueops.ListRequest` gains MaxRows, MaxRowsSource and SkipCounts,
each with a conformance contract running at all three backends and zero parks.

MAXROWS IS THE EXACT INVERSE OF OFFSET, which is why it needed no new shape.
Offset is honored by the unit-of-work body — it renders OFFSET — and refused by
the store-backed one with a typed *ErrUnsupported. The cap points the other way:
the store search path enforces it after the scan (EnforceMaxRowsCap), and the
unit-of-work query path (internal/storage/domain/db) reads no cap at all, so a
request carrying one would come back as the FULL uncapped answer with nothing on
it to say so — a caller that set a circuit breaker handed exactly the runaway
query it was guarding against. It refuses instead, naming the operation and the
backend. `bd list --max-rows --proxied-server` has always refused one layer up
in the CLI for that reason; this is the same refusal for the callers that are
not the CLI. RunReaderListMaxRowsIsHonoredOrRefused mirrors
RunReaderOffsetIsHonoredOrRefused, including the never-silently-ignored clause,
and asserts the complement too: a cap the result set FITS inside must still come
back as an ordinary page, or a body that refused every cap out of hand would
pass.

MAXROWSSOURCE is the third field and the bead named two. It is attribution
only — it decides no answer — but `handleMaxRowsError` falls back to
"BEADS_MAX_ROWS" when it is empty, so a request that carried the number without
the source would make stage 2's route flip start telling operators to raise the
wrong knob. The contract asserts it comes back on the cap error.

SKIPCOUNTS is the SkipLabels precedent one column over: the three cardinality
joins are dropped and DependencyCount/DependentCount/CommentCount come back
zero, to be read as unknown rather than as none. It has to land WITH MaxRows —
stage 2 moves `bd list`'s TEXT path onto the role, and that path calls
SearchIssues today and projects no counts at all, so arriving on a counted page
without a way to opt out is a straight perf regression. The expensive member is
the reverse-blocker aggregate, which joins on a COALESCE key the pure-Go
analyzer cannot auto-index — the known embedded sore point, and the reason the
knob is worth having at all.

The contract's second half is the load-bearing one: NOTHING ELSE MOVES. Same
rows, same order, same has-more, same Parent — asserted by running the identical
request both ways rather than against a written-out expectation. Parent is the
tripwire: it rides the same mega-query and is not a cardinality, so an
implementation that made an aggregate join inner, or suppressed the parent join
too, would answer a strict subset and still look like it had "skipped the
counts".

WHAT THE SQL DOES. sqlbuild.SearchCountsSQL's two trailing hydration bools
become one CountsHydration value, so a path cannot pick up one opt-out and drop
the other, and both stores plus the unit-of-work repository read the pair
through one helper. A suppressed aggregate is projected as a constant IN PLACE
rather than removed: the scan reads the six extra columns positionally, so the
shape is invariant. Ready work passes the zero value at every call site —
types.WorkFilter carries neither knob, which is what "SkipLabels and SkipCounts
are not carried onto the ReadyFlag arm" means, now stated where a caller reads
it. MaxRows IS carried there; reader_ready_scope.go already claimed so, and this
is the commit that makes the claim true, since until now the field existed only
on the filter.

ONE HAND-ROLL DELETED. `bd list` built the filter through the shared builder and
then reached in and stamped filter.MaxRows/MaxRowsSource onto it — the
"build it, then change it" half-step this programme exists to make unreachable,
and the reason the cap was invisible to every implementation of Reader.List. The
cap is resolved in gatherListInput onto the request, BuildListFilter is now the
only writer of those two filter fields, and the proxied refusal reads the
already-resolved value (rejectResolvedMaxRowsUnderProxiedServer) so a malformed
BEADS_MAX_ROWS still warns exactly once.

PROVABLY INERT AT THE CLI, measured rather than argued: 22 `bd list` / `bd
children` invocations — every output mode, both max-rows knobs, the malformed
and negative cases — run against a binary built from integration/facade-wave-2
and one built from this commit, over ONE shared workspace so both read identical
data, differ on exactly one line of 617. `bd list --max-rows -1 --status bogus`
now reports the malformed flag instead of the invalid status; both exit 1, it
needs two bad arguments to reach, and it is recorded as A-lst1-3.

NO OPENAPI OPERATION AND NO NEW QUERY PARAMETER, deliberately, recorded as
A-lst1-1. `bd list`'s operation (GET /v0/beads/issues) already exists and is
already on the role, and neither field can be published on it yet: `bd serve`
always builds a provider-backed server, which REFUSES max_rows, so the parameter
would be one the reference server can never honor; and skip_counts would set
three unconditional integers on a PINNED response component to zero, leaving a
client unable to tell "no dependencies" from "not computed" — the failure
comments_omitted exists to prevent on the detail view. `make api-check`
regenerates nothing.

Verified: build, vet, gofmt clean; golangci-lint 0 issues; the two new cases at
dolt, embeddeddolt (BEADS_TEST_EMBEDDED_DOLT=1) and uow, plus the whole Reader
contract at all three; the builder golden (35 recorded cases plus two new ones
for the new fields) and the counts-SQL shape test; internal/httpapi and
make api-check; internal/storage/{issueops,domain/...}, internal/workapi,
internal/types, internal/telemetry, the root package; the embedded
`bd list`/`bd children`/max-rows CLI suites, TestProxiedServerList under
BEADS_TEST_PROXIED_SERVER=1 (including reject_max_rows_flag,
reject_max_rows_env and allow_max_rows_zero), TestProxiedLookup*, and the
protocol corpus package.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 16:36:31 +00:00
CI Bot
1c35a769d3 feat(issueops): add the Reader role and wire both front doors through it
Reader is Lifecycle's read counterpart, built as its mirror image: a role in
the leaf `issueops` package, reached through a capability accessor on the
storage seam, implemented at the same six sites. `Storage` gains
`IssueReader() (issueops.Reader, error)` beside `IssueLifecycle()`, and the
governing rule holds — a new capability got a new role and a new accessor,
not four more methods on an existing one.

SUBSUME, NOT COMPOSE. Reader takes the whole high-level request and performs
filter and default construction INTERNALLY, supplying its own ConfigSource
from the store or unit of work it already holds. Under compose both front
doors would each build a ConfigSource, load config, call a builder and execute
— two callers again, which is the topology that shipped a bare IssueFilter to
a viewer and silently served in_progress rows as "ready". Under subsume the
ritual is not callable from a front door: handlers and RunE bodies can only
say rd.List(ctx, req), so the failure mode is unwritable rather than merely
discouraged.

The three read endpoints are implemented against the frozen OpenAPI v0
document and consume the accessor; the 501 stubs and their self-deleting
exemption list are gone. `bd show --json` reaches the same role through the
same accessor, resolving the canonical id first so fuzzy and cross-repo
lookup stay CLI-local.

The workapi builders are re-typed onto the leaf request types, so there is no
second params struct to drift; both recorded goldens replay byte-identical,
which is the proof the re-typing changed nothing.

Load-bearing details:

- `sort` is forwarded EXPLICITLY as priority when absent. The storage layer
  reads an empty policy as hybrid, which demotes older high-priority work and
  therefore changes the item SET once a limit truncates — while the document
  still reads correctly. Pinned by a handler test on the recorded filter and
  by the parity oracle's order assertion.
- The parity oracle compares FULL ITEM JSON, not id sets. An id-set oracle
  passes while ready ships a bare Issue against the CLI's IssueWithCounts.
- internal/workapi no longer imports internal/storage/uow; it names the slice
  of a unit of work it reads through, so the uow provider can implement the
  role without closing an import cycle.
- internal/utils' cgo store test moves to an external test package for the
  same reason, with its two unexported-symbol cases split out.

Verified: issueops is still a leaf (internal/types + stdlib only); the
protocol corpus is unchanged structurally and behaviourally under
BEADS_PROTOCOL_REQUIRE_DOLT=1; a real `bd serve` answers all six operations
against a real workspace with CLI and HTTP JSON identical field for field.
2026-07-31 18:37:25 +00:00
CI Bot
6b04f94ef0 refactor(workapi): add BuildReadyFilter with a golden recorded from both old builders (bd-ehi)
First half of the bd-ehi collapse, kept separate so the golden table is
provably generated from the OLD code: this commit adds
internal/workapi/ready.go (BuildReadyFilter, ReadyParams,
DefaultReadyLimit) without touching either of cmd/bd's copies, and records
what BOTH of them produce for 39 command lines into
internal/workapi/testdata/ready_filter_golden.json.

Recording both columns is the point. `bd ready`'s defaults are built twice
today - inline in ready.go's RunE for the direct path, and in
gatherReadyInput for the proxied path - and the golden is what turns "they
look the same" into a fact. They agree on 33 of the 39 cases, including
Status: "open" (the tracked description says only one copy hard-codes it;
that has not been true since ready_input.go was added in 911ff8025). The
six disagreements are pinned by TestReadyFilterGoldenDivergences with the
branch that was kept and why:

  - Offset: only gather sets it. The direct RunE rejects --offset before
    any filter exists, so this is not observable.
  - MaxRows/MaxRowsSource: only the direct path sets them, and only the
    direct path can enforce them. Kept in cmd/bd rather than ReadyParams.
  - --max-rows -1 / --claim --gated / --offset -1: each rejected by one
    builder because the other path's RunE rejects it upstream.

DefaultReadyLimit = 100 is today's CLI default, not a new choice: it is
readyCmd's --limit default at HEAD. 0 still means unlimited, so the builder
does not substitute it.

The generator (cmd/bd/zz_ready_filter_golden_gen_test.go) is scaffolding
and is deleted in the follow-up commit that removes ready.go's inline copy.
It clones readyCmd's real flag definitions rather than re-declaring them,
and its args -> ReadyParams mapping cannot launder a mistake into a false
pass: the golden records the ReadyParams, so a field set in params but not
on the command line makes the replay fail rather than succeed.

Agent-Signature: claude-code-opus-5-unknown-reasoning on behalf of CI Bot
2026-07-31 07:19:59 +00:00
CI Bot
af56541bb3 refactor(workapi): add internal/workapi with the list-filter builder and a pre-extraction golden (bd-fv4)
First half of the bd-fv4 extraction, kept separate so the golden table is
provably generated from the OLD code: this commit adds
internal/workapi/list.go (BuildListFilter, ListParams, ListConfig,
ConfigSource, DefaultListLimit) without touching cmd/bd's copy, records
what cmd/bd's buildListFilter produces for 35 inputs into
internal/workapi/testdata/list_filter_golden.json, and asserts the moved
builder reproduces it byte for byte.

DefaultListLimit = 50 is today's CLI default, not a new choice: both
cmd/bd/list.go's --limit flag and internal/config's list.limit default are
50 at HEAD.

The generator (cmd/bd/zz_list_filter_golden_gen_test.go) is scaffolding and
is deleted in the follow-up commit that removes cmd/bd/list_filter.go. Its
params -> listInput conversion cannot launder a mistake into a false pass:
the golden records the ListParams, so a mis-mapped field there makes the
replay fail rather than succeed.

Agent-Signature: claude-code-opus-5-unknown-reasoning on behalf of CI Bot
2026-07-31 05:42:40 +00:00