Commit graph

6 commits

Author SHA1 Message Date
Vishnu J
f5a20fb66b
fix(list): honor --status when combined with --ready (#5924)
* fix(list): honor --status when combined with --ready (#5832)

`bd list --status X --ready` silently dropped --status and returned the
unfiltered ready set (typically open issues). BuildListFilter pinned
StatusOpen whenever ReadyFlag was set, and ReadyFilterFromIssueFilter
hard-coded the same pin, so a custom status or in_progress selector
never reached GetReadyWork.

--ready without --status still defaults to open. An explicit --status
is now the ready-work status selector, matching how other list filters
already compose with --ready. `--status all` remains the no-filter
spelling and keeps the open default.

* fix(list): status-aware --ready footer and honor-path contract docs (#5832)

The query intersection from the prior commit is unchanged. Round-1
review asked the ReadyFlag/Statuses contract comments and the text
footer to describe the honor path: default --ready is still open,
an explicit --status is named instead of "excludes in_progress".
2026-09-02 18:12:07 -07:00
Stephanie Jarmak
7e825441b4
feat(workapi,storage): opt-in Brief projection for the list and ready pages (#5554)
STORAGE LAYER ONLY, per CONTRIBUTING_PR_GUIDELINES.md. issueops.ListRequest
and issueops.ReadyRequest gain Brief; no CLI flag and no HTTP parameter sets
it yet, so no front door's payload changes. The wiring stacks on top.

WHAT IT DROPS is the free-form text: description, design, acceptance_criteria,
notes, payload and waiters are not selected, and the row comes back carrying
types.Issue.IsLitePartial. #5546 measured a 16,051-issue store where
`bd ready --json` returned 454,021 bytes and notes plus description were 76% of
it, on the call an agent makes to decide what to work on. Measured through
issueops.Reader on 100 rows sized from those per-row ratios: 362,301 bytes
full, 24,001 brief, a 93.4% reduction.

THE PRIMITIVE WAS HALF-BUILT AND UNREACHABLE. #3906 landed
types.IssueFilter.Lite, IssueSelectColumnsLite, HeavyDropList,
ScanIssueLiteFrom and Issue.IsLitePartial, then wired filter.Lite into exactly
one place: SearchIssuesInTx, the UNCOUNTED store-backed search. Every read that
answers with IssueWithCounts, which is every --json route of both commands,
both implementations of issueops.Reader, and both HTTP reads, runs the counts
mega-query instead and ignored the field. A request that set it would have come
back correct, fully hydrated, and with nothing on it to say the projection had
not happened. That silent no-op is what this closes.

THE APPLICATION POINT IS THE COUNTS MEGA-QUERY, and it is one point rather than
four because sqlbuild.SearchCountsSQL and issueops.ScanReadyWorkRowWithCounts
are already the pair every counted read passes through: the store seam's list
(search_counts.go) and ready (ready_work_counts.go), and the unit-of-work
seam's list (issue_search_counts.go) and ready (ready_work_union.go, via
fetchCountsByIDs). sqlbuild.CountsHydration is already the value that carries
hydration opt-outs across that pair on both seams, so Lite is a third field on
it rather than a new channel.

IT CHANGES THE PROJECTION'S SHAPE, which SkipLabels and SkipCounts do not, and
that is why ScanReadyWorkRowWithCounts now takes the whole hydration instead of
a bool: the value that chose the SELECT list is the value that drives the scan,
so the two cannot be given different answers. Substituting empty strings in the
heavy positions would have kept one scan path and was rejected, because the row
would carry blank text with IsLitePartial=false, which is indistinguishable
from a genuinely textless row, the ambiguity ga-clgh and CommentsOmitted
already record.

WHERE IS INDEPENDENT OF SELECT, so this bounds the size of a row and never
which rows match: DescContains, NotesContains and EmptyDesc keep selecting
exactly what they select today, and no sort key, keyset cursor or wisp-merge
dedup reads a dropped column (CompareIssuesBy orders on nine fields, none of
them heavy).

THE CLAIM REFUSES IT, which is not the same call the page makes and is the
sharper case of the rule Limit and Offset already state. A claim does not read
its row through the page's query at all: ExecuteClaimNext refetches the winning
row whole with GetIssueInTx and hydrates its counts itself, so the projection
has nothing to apply to. Accepting the field would answer a MUTATING request
with a fully hydrated row carrying IsLitePartial=false, the caller's one signal
that it did not get what it asked for, reporting that it did. The check lives
in issueops.ValidateClaimNextRequest, which is the shared rule both ready
claimers and BatchCloser's claim already run, so all four backends refuse it on
the same line.

BuildReadyCountFilter CARRIES it, which is the opposite treatment and the
opposite reason. A count reads no field of any row, so the projection cannot
make the number wrong, and the count is not always cheap enough for that to be
the end of it: the unit-of-work seam has no COUNT(*) over the ready predicate
and sizes the set by running the unbounded page and taking its length
(uow/ready_counter.go), so dropping the field there would hydrate every heavy
column of the whole ready set to answer `bd ready --brief`, a larger read than
the page it was asked for. Carrying it also keeps that builder what its doc
says it is: the listing's filter with the PAGE removed and nothing else.

TWO ASYMMETRIES on the page path, both deliberate and both stated in code:

  - Brief IS carried onto the --ready arm (ReadyFilterFromIssueFilter) where
    SkipLabels and SkipCounts are dropped. Those two hide a number the ready
    renderings print; this drops a body no listing prints, so carrying it is
    what makes `bd list --ready` and `bd ready` answer one request one way.
  - types.WorkFilter carries Lite and neither of the other two, so the ready
    paths get readyHydrationFor rather than hydrationFor. Before this, both
    uow ready call sites passed a hardcoded CountsHydration{}, a literal that
    could not carry a filter's answer at all, which is how that path would have
    been missed.

CONFORMANCE at all four registered sites, one Test per case, for the two page
cases and for the claim refusal. Every field the cases assert stripped is
seeded heavy first and the premise is checked, so a backend that never
round-trips one of the six fails loudly instead of turning that clause into one
that cannot fail. Identity is asserted too (ID, Title, Status, IssueType,
Priority), or a body that stripped the row to its id would satisfy every
emptiness clause. IsLitePartial is asserted in both directions.

THE LITE COLUMN LIST IS PINNED BY ORDER, not only by membership. Both lists are
scanned positionally, so two columns of the same SQL type can be transposed
with no membership change at all: the pre-existing set-based parity test stays
green, the scan succeeds, and every row silently carries one column's value in
another's field. TestIssueSelectColumnsLite_IsFullMinusHeavyInOrder derives the
expected list by deleting HeavyDropList from IssueSelectColumns in place, which
makes the lite list a derivation rather than a second hand-maintained copy.

RED BEFORE GREEN, five sabotages, each verified and reverted:

  - deleting the lite branch from CountsHydration.IssueColumns fails both page
    cases with "expected 58 destination arguments in Scan, not 52";
  - accepting Brief without plumbing it to either filter, the silent no-op this
    change exists to close, fails both page cases on all six text fields and on
    IsLitePartial, which is the run that proves the assertions are
    load-bearing;
  - transposing created_by and owner in IssueSelectColumnsLite fails only the
    new order test, on IssueSelectColumnsLite[9] = owner, want created_by,
    while the set-based test stays green, which is the gap it was added for;
  - dropping the claim's refusal fails the claimer contract with "ClaimNext
    with brief: error = <nil>, want ErrValidation"; and
  - clearing Brief in BuildReadyCountFilter fails
    TestBuildReadyCountFilterIsTheUnboundedReadyFilter, which already existed to
    catch a ReadyRequest field the count builder forgets to carry and gains
    Brief in its fixture here.

The recorded ready-filter golden gains "Lite": false in 62 cases because
types.WorkFilter gained the field and the test compares marshaled JSON
byte-for-byte. False is what a request that did not ask for the projection
resolves that field to, so the replay stays exact; the test comment says so.

Gates: go build ./..., go vet ./..., gofmt, internal/storage/... (dolt,
embeddeddolt with BEADS_TEST_EMBEDDED_DOLT=1, uow, issueops, sqlbuild,
domain/db), internal/workapi/..., backend/conformance/..., issueops/...,
make api-check.

Refs: #5546, #3906

Co-authored-by: sjarmak <t@t.co>
2026-08-10 10:45:23 -07:00
Julian Knutsen
9a2c918b0e
Admit the ephemeral plane from a list request, on the facade and on the wire (#5476)
* feat(reader): admit the ephemeral plane from a list request

`issueops.Reader.Ready` has had an ephemeral gate since it was written;
`List` has not. The wisp plane was reachable from a list request only as
a SIDE EFFECT of `IncludeInfra`, which also drops the infra-type
exclusions, or by naming an infra type, which routes to that plane
ALONE. A caller that wants the ephemeral rows of ordinary types — a
message bead, say — could ask for them on the ready query and not on the
listing, and the wire said so in prose: listIssues published no
`include_ephemeral`, and the document called the asymmetry out.

ListRequest.IncludeEphemeral closes it, with the same semantics
ReadyRequest.IncludeEphemeral already has.

  false (zero value)  durable rows only — today's listing, unchanged
  true                the wisp plane merged IN ADDITION; the answer is a
                      superset of the false answer under the same
                      filters, and never ephemeral-only
  WispType            a predicate on the wisp_type COLUMN, which BOTH
                      tables carry, so it selects no plane: it narrows
                      whatever was admitted. WispType alone answers with
                      nothing, since no durable row carries a
                      classification; WispType + IncludeEphemeral is
                      lawful and composes as an ordinary AND
  order               ONE order across both planes, by SortBy, as if
                      they were a single table — not durable rows
                      followed by ephemeral ones. A Limit truncates the
                      merged order, and the keyset position is applied
                      to BOTH legs, so a cursored walk over the merge
                      drops no row and repeats none. The non-snapshot
                      caveat is the one every multi-page walk already
                      has; the ephemeral plane only turns over faster
  --ready             CARRIED, not dropped. The ready query has an
                      ephemeral gate of its own, so
                      ReadyFilterFromIssueFilter now reads the plane bit
                      off the list filter. That also carries
                      IncludeInfra's plane half, which the projection
                      used to drop silently — `bd list --ready
                      --include-infra` now answers with the wisps `bd
                      count --include-infra` already counts

It is a PLANE knob and not a TYPE knob, which is the whole difference
from the three fields beside it: templates, gates and infra types stay
hidden by their own predicates. IncludeInfra remains strictly wider.

The wire gets `include_ephemeral` on listIssues, spec-first. There is no
capability-token change: it is an ordinary optional query parameter, so
an older server answers it with 400 `unknown_parameter` — the designed
skew signal, whose recovery is to degrade rather than retry.

Conformance runs on all three legs (server-backed store, embedded store,
unit of work) and covers admission, the merged order, every page bound
across the interleaving, a cursored walk across the plane boundary, and
the WispType interaction. The two seams arrange the merge differently —
one merge-sorts two independently ordered legs in Go and trims, the
other ORDERs a SQL UNION ALL — so the bounded and cursored arms are a
genuine two-body case rather than one body twice.

* fix(reader): correct the flag's reach, and settle the merge seam it lands on

Review follow-up. Three of these are documentation being wrong about the
code; one is the code itself.

THE MOTIVATING EXAMPLE WAS FALSE. `message` is one of the DEFAULT INFRA
TYPES (agent, role, message), so "the ephemeral rows of ordinary types —
a message bead, say" named the one row this flag cannot reach: the
infra-type exclusion is a TYPE predicate and survives the plane knob, as
the golden already recorded and nothing asserted. `bd ready` cannot
deliver it either — ReadyWorkExcludeTypes appends the infra set
unconditionally. The field doc, the operation description and the
parameter text now say that ephemeral agent/role/message rows need
IncludeInfra, and what the flag alone reaches is the ephemeral rows of
the types a listing already shows.

The contract case is what makes it more than a corrected sentence. The
fixture's only wisp was a task, which comes back under EITHER knob, so a
body that wired the plane flag to IncludeInfra's branch answered every
row correctly. An infra-typed wisp is hidden twice over — once by the
plane, once by the type — and is the only row that can tell the two
apart. Three rows now pin it: absent under IncludeEphemeral, present
under IncludeInfra, present under both. Wiring the flag into the
type branch fails it.

TWO MORE DOC CORRECTIONS on the same field. It admits the wisps TABLE,
which holds no-history rows (ephemeral = 0) alongside true ephemerals —
stated, rather than left as "the ephemeral plane". And it is NOT
identical to ReadyRequest.IncludeEphemeral: ready reads both planes
either way and adds a per-ROW ephemeral predicate when unset, where this
selects which TABLES are read. The difference is observable on exactly
those no-history rows, which a DEFAULT ready answer already contains and
a default listing does not.

The ready projection's predicate is negative, on a field whose zero
value means admit, so it fires for a filter that merely never set
SkipWisps rather than for one that asked. Every caller is handed a
BuildListFilter result, which decides that field for every request, so
"unset" is unreachable today — named at the site so the next hand-built
filter does not walk into it.

THE SEAM DISAGREEMENT, which is the code change. A cross-plane duplicate
id was tolerated by the per-table seam (collapse to the canonical wisp,
answer) and HARD-ERRORED by the union seam, and this flag routes every
wisp-inclusive read onto the union. Two implementations of one role
answering a page and an error is a divergence, not a design, and leaving
it silent while making it reachable is worse than either behavior.

The union now resolves it the way its sibling does, at all four sites,
and the strict flag is gone rather than left as a footgun. Three things
say that is the right verdict and not merely the convenient one: the
per-table seam has said so since be-iabdi ("hard-erroring breaks every
lookup city-wide"); `bd doctor` detects the state with a query of its
own, so answering blinds no detector; and its fix deletes the stale
ISSUES copy, which is exactly the row the union now drops. Its own check
text names the hard failure as the damage — "stale issues-table copies
break every lookup for the affected IDs". The four tests that pinned the
error now pin the wisp-canonical answer; inverting the preference fails
all four.

Preferring the wisp needs a second pass: the outer ORDER BY can place
either copy first, so which leg produced a row is known in full only
after the scan.

Last, a stale comment: issueops/count.go claimed SearchIssuesInTx was
the corruption detector. It has not been one since be-iabdi. The count
trusts the disjoint-table invariant and no read enforces it, so a
dual-resident id is counted twice and listed once; doctor is the
detector and the repair.

---------

Co-authored-by: Review Sim <ci@beads.test>
2026-08-09 05:06:36 -07:00
Julian Knutsen
e29abaea37
Converge Offset and MaxRows on all three legs, and fix the panic and the breaker hole under them (#5435)
Offset and MaxRows were mirror-image splits: the unit-of-work seam honored
Offset and refused MaxRows; the two store legs did the opposite. Three contract
cases absorbed that with an "either HONORED or REFUSED with a typed
*ErrUnsupported" disjunction.

A contract that asserts a capability is either honored or refused is not pinning
semantics — it is ratifying a split. All three are gone, along with
assertReaderUnsupported. Converging upward rather than uow-matches-server is the
owner's one ratified exception, because the server refusing MaxRows was a
capability gap rather than a semantics choice.

Offset now runs in the shared page epilogue on every leg rather than in whichever
seam could render it: the skip must happen in the caller's display order, which
is not SQL-expressible for --sort id, and a row the database skips is a row the
query matched.

Two bugs fixed under it.

The panic: domain/db rendered LIMIT 18446744073709551615 OFFSET k for an
unbounded query, and Dolt sized a buffer from limit+offset and returned a
recovered makeslice panic instead of rows. The contract case had been shaped
around it, bounding every request, with a comment calling it a storage-seam bug
that predates the role. It now drives the unbounded shape.

The breaker hole, found by review: proxied bd list --watch with an offset and a
cap counted the cap against post-skip rows while the page route counted
pre-skip, so the same flags produced opposite verdicts — L=10, k=2, M=3 over 5
rows returned a page on one route and ErrTooManyRows on the other. Newly
reachable since 3b2385922 removed the proxied --max-rows rejection ahead of the
watch dispatch, and it falsified the comment claiming no caller combines an
offset with a cap.

MaxRows semantics are now ratified and documented: the cap counts the rows the
query TOUCHED (offset+limit), not the rows the caller receives. A skipped row is
still a matched row. Review reported the Limit+Offset == MaxRows boundary as an
off-by-one; measured under those semantics the code was already correct and the
doc was wrong, so the fix was the doc plus pinning the behaviour.

A seventh cannot-fail test was caught mid-slice: an unreachable "under a
defensive cap" branch in searchWindowFor absorbed the exact regression the new
MaxRows-behind-offset arm exists to catch, and the mutation returned
INCONCLUSIVE. Removed.

Equivalence of the epilogue skip to a SQL OFFSET was checked rather than
assumed. One narrow uncovered exception stands: under --sort title, SQL
collation and Go byte-compare can disagree on non-ASCII and then pick different
skip sets. Conformance seeds ASCII.
2026-08-08 11:37:09 -07:00
CI Bot
1108af95ee refactor(workapi): one execution epilogue for every list and ready read
Both surfaces already shared CONSTRUCTION — one BuildListFilter, one
BuildReadyFilter, both over the same issueops request types, both pinned
by the builders' golden files. They executed separately: the two Reader
implementations applied the over-fetch/sort/trim/HasMore tail, and each
CLI route applied its own copy. That tail had already drifted twice, and
both times it was the same bug in the same three lines written in two
places — a --ready page under a sort the database cannot express came
back untrimmed from one implementation, and the proxied route answered
`--sort id --limit 5` with every row while the direct route answered
with five.

workapi.FinishPage is now that tail, once. It is called by both
issueops.Reader implementations, by both of `bd list`'s routes, and by
the proxied `bd ready` listing. Nothing about the contract widened: the
epilogue takes rows, an order, a page limit and the seam's own has-more,
and answers with the page and the verdict. A seam that reports has-more
natively seeds it; a seam that does not passes false and lets the
over-fetched row speak.

`bd ready`'s DIRECT route keeps its own epilogue, and the comment in
ready_input.go now says so instead of claiming otherwise: it answers the
strictly larger question "how many rows were hidden" with a second
counting query and publishes the total in its pagination meta, which the
role has no way to express and the CLI cannot stop printing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 22:27:01 +00:00
CI Bot
1c35a769d3 feat(issueops): add the Reader role and wire both front doors through it
Reader is Lifecycle's read counterpart, built as its mirror image: a role in
the leaf `issueops` package, reached through a capability accessor on the
storage seam, implemented at the same six sites. `Storage` gains
`IssueReader() (issueops.Reader, error)` beside `IssueLifecycle()`, and the
governing rule holds — a new capability got a new role and a new accessor,
not four more methods on an existing one.

SUBSUME, NOT COMPOSE. Reader takes the whole high-level request and performs
filter and default construction INTERNALLY, supplying its own ConfigSource
from the store or unit of work it already holds. Under compose both front
doors would each build a ConfigSource, load config, call a builder and execute
— two callers again, which is the topology that shipped a bare IssueFilter to
a viewer and silently served in_progress rows as "ready". Under subsume the
ritual is not callable from a front door: handlers and RunE bodies can only
say rd.List(ctx, req), so the failure mode is unwritable rather than merely
discouraged.

The three read endpoints are implemented against the frozen OpenAPI v0
document and consume the accessor; the 501 stubs and their self-deleting
exemption list are gone. `bd show --json` reaches the same role through the
same accessor, resolving the canonical id first so fuzzy and cross-repo
lookup stay CLI-local.

The workapi builders are re-typed onto the leaf request types, so there is no
second params struct to drift; both recorded goldens replay byte-identical,
which is the proof the re-typing changed nothing.

Load-bearing details:

- `sort` is forwarded EXPLICITLY as priority when absent. The storage layer
  reads an empty policy as hybrid, which demotes older high-priority work and
  therefore changes the item SET once a limit truncates — while the document
  still reads correctly. Pinned by a handler test on the recorded filter and
  by the parity oracle's order assertion.
- The parity oracle compares FULL ITEM JSON, not id sets. An id-set oracle
  passes while ready ships a bare Issue against the CLI's IssueWithCounts.
- internal/workapi no longer imports internal/storage/uow; it names the slice
  of a unit of work it reads through, so the uow provider can implement the
  role without closing an import cycle.
- internal/utils' cgo store test moves to an external test package for the
  same reason, with its two unexported-symbol cases split out.

Verified: issueops is still a leaf (internal/types + stdlib only); the
protocol corpus is unchanged structurally and behaviourally under
BEADS_PROTOCOL_REQUIRE_DOLT=1; a real `bd serve` answers all six operations
against a real workspace with CLI and HTTP JSON identical field for field.
2026-07-31 18:37:25 +00:00