beads/issueops/reader_ready_scope.go
Julian Knutsen 9a2c918b0e
Admit the ephemeral plane from a list request, on the facade and on the wire (#5476)
* feat(reader): admit the ephemeral plane from a list request

`issueops.Reader.Ready` has had an ephemeral gate since it was written;
`List` has not. The wisp plane was reachable from a list request only as
a SIDE EFFECT of `IncludeInfra`, which also drops the infra-type
exclusions, or by naming an infra type, which routes to that plane
ALONE. A caller that wants the ephemeral rows of ordinary types — a
message bead, say — could ask for them on the ready query and not on the
listing, and the wire said so in prose: listIssues published no
`include_ephemeral`, and the document called the asymmetry out.

ListRequest.IncludeEphemeral closes it, with the same semantics
ReadyRequest.IncludeEphemeral already has.

  false (zero value)  durable rows only — today's listing, unchanged
  true                the wisp plane merged IN ADDITION; the answer is a
                      superset of the false answer under the same
                      filters, and never ephemeral-only
  WispType            a predicate on the wisp_type COLUMN, which BOTH
                      tables carry, so it selects no plane: it narrows
                      whatever was admitted. WispType alone answers with
                      nothing, since no durable row carries a
                      classification; WispType + IncludeEphemeral is
                      lawful and composes as an ordinary AND
  order               ONE order across both planes, by SortBy, as if
                      they were a single table — not durable rows
                      followed by ephemeral ones. A Limit truncates the
                      merged order, and the keyset position is applied
                      to BOTH legs, so a cursored walk over the merge
                      drops no row and repeats none. The non-snapshot
                      caveat is the one every multi-page walk already
                      has; the ephemeral plane only turns over faster
  --ready             CARRIED, not dropped. The ready query has an
                      ephemeral gate of its own, so
                      ReadyFilterFromIssueFilter now reads the plane bit
                      off the list filter. That also carries
                      IncludeInfra's plane half, which the projection
                      used to drop silently — `bd list --ready
                      --include-infra` now answers with the wisps `bd
                      count --include-infra` already counts

It is a PLANE knob and not a TYPE knob, which is the whole difference
from the three fields beside it: templates, gates and infra types stay
hidden by their own predicates. IncludeInfra remains strictly wider.

The wire gets `include_ephemeral` on listIssues, spec-first. There is no
capability-token change: it is an ordinary optional query parameter, so
an older server answers it with 400 `unknown_parameter` — the designed
skew signal, whose recovery is to degrade rather than retry.

Conformance runs on all three legs (server-backed store, embedded store,
unit of work) and covers admission, the merged order, every page bound
across the interleaving, a cursored walk across the plane boundary, and
the WispType interaction. The two seams arrange the merge differently —
one merge-sorts two independently ordered legs in Go and trims, the
other ORDERs a SQL UNION ALL — so the bounded and cursored arms are a
genuine two-body case rather than one body twice.

* fix(reader): correct the flag's reach, and settle the merge seam it lands on

Review follow-up. Three of these are documentation being wrong about the
code; one is the code itself.

THE MOTIVATING EXAMPLE WAS FALSE. `message` is one of the DEFAULT INFRA
TYPES (agent, role, message), so "the ephemeral rows of ordinary types —
a message bead, say" named the one row this flag cannot reach: the
infra-type exclusion is a TYPE predicate and survives the plane knob, as
the golden already recorded and nothing asserted. `bd ready` cannot
deliver it either — ReadyWorkExcludeTypes appends the infra set
unconditionally. The field doc, the operation description and the
parameter text now say that ephemeral agent/role/message rows need
IncludeInfra, and what the flag alone reaches is the ephemeral rows of
the types a listing already shows.

The contract case is what makes it more than a corrected sentence. The
fixture's only wisp was a task, which comes back under EITHER knob, so a
body that wired the plane flag to IncludeInfra's branch answered every
row correctly. An infra-typed wisp is hidden twice over — once by the
plane, once by the type — and is the only row that can tell the two
apart. Three rows now pin it: absent under IncludeEphemeral, present
under IncludeInfra, present under both. Wiring the flag into the
type branch fails it.

TWO MORE DOC CORRECTIONS on the same field. It admits the wisps TABLE,
which holds no-history rows (ephemeral = 0) alongside true ephemerals —
stated, rather than left as "the ephemeral plane". And it is NOT
identical to ReadyRequest.IncludeEphemeral: ready reads both planes
either way and adds a per-ROW ephemeral predicate when unset, where this
selects which TABLES are read. The difference is observable on exactly
those no-history rows, which a DEFAULT ready answer already contains and
a default listing does not.

The ready projection's predicate is negative, on a field whose zero
value means admit, so it fires for a filter that merely never set
SkipWisps rather than for one that asked. Every caller is handed a
BuildListFilter result, which decides that field for every request, so
"unset" is unreachable today — named at the site so the next hand-built
filter does not walk into it.

THE SEAM DISAGREEMENT, which is the code change. A cross-plane duplicate
id was tolerated by the per-table seam (collapse to the canonical wisp,
answer) and HARD-ERRORED by the union seam, and this flag routes every
wisp-inclusive read onto the union. Two implementations of one role
answering a page and an error is a divergence, not a design, and leaving
it silent while making it reachable is worse than either behavior.

The union now resolves it the way its sibling does, at all four sites,
and the strict flag is gone rather than left as a footgun. Three things
say that is the right verdict and not merely the convenient one: the
per-table seam has said so since be-iabdi ("hard-erroring breaks every
lookup city-wide"); `bd doctor` detects the state with a query of its
own, so answering blinds no detector; and its fix deletes the stale
ISSUES copy, which is exactly the row the union now drops. Its own check
text names the hard failure as the damage — "stale issues-table copies
break every lookup for the affected IDs". The four tests that pinned the
error now pin the wisp-canonical answer; inverting the preference fails
all four.

Preferring the wisp needs a second pass: the outer ORDER BY can place
either copy first, so which leg produced a row is known in full only
after the scan.

Last, a stale comment: issueops/count.go claimed SearchIssuesInTx was
the corruption detector. It has not been one since be-iabdi. The count
trusts the disjoint-table invariant and no read enforces it, so a
dual-resident id is counted twice and listed once; doctor is the
detector and the repair.

---------

Co-authored-by: Review Sim <ci@beads.test>
2026-08-09 05:06:36 -07:00

128 lines
6.9 KiB
Go

package issueops
import (
"fmt"
"strings"
)
// readyScopeField is one ListRequest field the ReadyFlag arm cannot carry,
// paired with the `bd list` flag that sets it. Naming both is the point of the
// refusal: a library caller looks for the struct field and a CLI caller for
// the flag, and neither should have to map one onto the other to learn which
// part of their request could not be honored.
type readyScopeField struct {
name string
flag string
set func(ListRequest) bool
}
// readyScopeFields is the drop set: every field a list request can set that
// does not survive the trip to the blocker-aware ready query.
//
// It was read off the two functions between this request and that query —
// internal/workapi.BuildListFilter, which turns the request into a storage
// filter, and internal/workapi.ReadyFilterFromIssueFilter, which projects that
// filter onto the narrower ready-work filter — and it is exactly the
// intersection: fields the builder copies onto the filter and the projection
// then discards.
//
// Four groups are deliberately ABSENT, because listing them here would refuse
// requests that are answered correctly today:
//
// - What the projection carries: IssueType, all five label forms, Assignee,
// NoAssignee, the exact Priority, ParentID, MolType, WispType,
// MetadataFields, HasMetadataKey, ExcludeTypes (and with it IncludeGates
// and IncludeInfra's type suppression), IncludeEphemeral (and with it
// IncludeInfra's plane half, which the ready query admits through its own
// ephemeral gate), Limit, Offset, and the MaxRows cap with its
// attribution.
//
// - THE HYDRATION KNOBS, SkipLabels and SkipCounts. They are the one group
// the "exactly the intersection" rule above would otherwise sweep in: the
// builder does copy them onto the filter and the projection does discard
// them. They stay out because they select what is HYDRATED rather than
// which rows match, so a ReadyFlag request that sets one is answered with
// the rows it asked for and merely pays for a column it did not want.
// Refusing that would be refusing a correct answer over its cost, and the
// promise is already stated where a caller reads it (ListRequest.ReadyFlag).
//
// - Status and AllFlag. The builder resolves both to "open" under ReadyFlag
// before the projection ever runs, so IssueFilter.Statuses is never
// populated on this path and there is nothing for the projection to drop.
// Ready work is open work; that override is pinned by the builder's golden
// file (internal/workapi/testdata/list_filter_golden.json,
// ready_flag_overrides_status).
//
// - NoPinnedFlag. Pinned is dropped by the projection, but the ready-work
// WHERE clause excludes pinned rows unconditionally
// (internal/storage/sqlbuild/ready.go), so a caller who asks for the
// pinned rows to be left out gets exactly that. The opposite request,
// PinnedFlag, asks for pinned rows only and would be answered with a set
// that can never contain one — it is in the list below.
var readyScopeFields = []readyScopeField{
{"IDFilter", "--id", func(r ListRequest) bool { return r.IDFilter != "" }},
{"TitleSearch", "--title", func(r ListRequest) bool { return r.TitleSearch != "" }},
{"SpecPrefix", "--spec", func(r ListRequest) bool { return r.SpecPrefix != "" }},
{"TitleContains", "--title-contains", func(r ListRequest) bool { return r.TitleContains != "" }},
{"DescContains", "--desc-contains", func(r ListRequest) bool { return r.DescContains != "" }},
{"NotesContains", "--notes-contains", func(r ListRequest) bool { return r.NotesContains != "" }},
{"ExternalContains", "--external-contains", func(r ListRequest) bool { return r.ExternalContains != "" }},
{"ExternalRef", "--external-ref", func(r ListRequest) bool { return r.ExternalRef != "" }},
{"CreatedAfter", "--created-after", func(r ListRequest) bool { return r.CreatedAfter != nil }},
{"CreatedBefore", "--created-before", func(r ListRequest) bool { return r.CreatedBefore != nil }},
{"UpdatedAfter", "--updated-after", func(r ListRequest) bool { return r.UpdatedAfter != nil }},
{"UpdatedBefore", "--updated-before", func(r ListRequest) bool { return r.UpdatedBefore != nil }},
{"ClosedAfter", "--closed-after", func(r ListRequest) bool { return r.ClosedAfter != nil }},
{"ClosedBefore", "--closed-before", func(r ListRequest) bool { return r.ClosedBefore != nil }},
{"DeferAfter", "--defer-after", func(r ListRequest) bool { return r.DeferAfter != nil }},
{"DeferBefore", "--defer-before", func(r ListRequest) bool { return r.DeferBefore != nil }},
{"DueAfter", "--due-after", func(r ListRequest) bool { return r.DueAfter != nil }},
{"DueBefore", "--due-before", func(r ListRequest) bool { return r.DueBefore != nil }},
{"DeferredFlag", "--deferred", func(r ListRequest) bool { return r.DeferredFlag }},
{"OverdueFlag", "--overdue", func(r ListRequest) bool { return r.OverdueFlag }},
{"EmptyDesc", "--empty-description", func(r ListRequest) bool { return r.EmptyDesc }},
{"NoLabels", "--no-labels", func(r ListRequest) bool { return r.NoLabels }},
{"NoParent", "--no-parent", func(r ListRequest) bool { return r.NoParent }},
{"PinnedFlag", "--pinned", func(r ListRequest) bool { return r.PinnedFlag }},
{"PriorityMin", "--priority-min", func(r ListRequest) bool { return r.PriorityMin != nil }},
{"PriorityMax", "--priority-max", func(r ListRequest) bool { return r.PriorityMax != nil }},
// The keyset position is a pair and only the timestamp half decides
// whether one was supplied; AfterID alone is not a position.
{"AfterCreatedAt/AfterID", "cursor", func(r ListRequest) bool { return r.AfterCreatedAt != nil }},
}
// ValidateReadyFlagScope refuses a list request that combines ReadyFlag with a
// filter the blocker-aware ready query cannot carry, naming every field it
// could not honor. A generic refusal would leave the caller to bisect their
// own request.
//
// This is the enforcement half of ListRequest.ReadyFlag's promise; the other
// half is the doc comment there. It lives beside that promise rather than in
// any implementation, and it is called from the single builder every
// implementation of Reader runs (internal/workapi.BuildListFilter), so no
// backend can be missing it — a per-backend copy is the drift this role exists
// to remove.
//
// A request without ReadyFlag is never refused here: every field below is
// honored by the ordinary listing.
func ValidateReadyFlagScope(req ListRequest) error {
if !req.ReadyFlag {
return nil
}
var named []string
for _, f := range readyScopeFields {
if f.set(req) {
named = append(named, f.name+" ("+f.flag+")")
}
}
if len(named) == 0 {
return nil
}
return fmt.Errorf("%w: --ready cannot filter on %s; the blocker-aware ready query carries only part of the list vocabulary, so answering would return every ready issue rather than the ones asked for — drop --ready, or drop what it cannot carry",
ErrValidation, strings.Join(named, ", "))
}