beads/internal/workapi/ready.go
Stephanie Jarmak 7e825441b4
feat(workapi,storage): opt-in Brief projection for the list and ready pages (#5554)
STORAGE LAYER ONLY, per CONTRIBUTING_PR_GUIDELINES.md. issueops.ListRequest
and issueops.ReadyRequest gain Brief; no CLI flag and no HTTP parameter sets
it yet, so no front door's payload changes. The wiring stacks on top.

WHAT IT DROPS is the free-form text: description, design, acceptance_criteria,
notes, payload and waiters are not selected, and the row comes back carrying
types.Issue.IsLitePartial. #5546 measured a 16,051-issue store where
`bd ready --json` returned 454,021 bytes and notes plus description were 76% of
it, on the call an agent makes to decide what to work on. Measured through
issueops.Reader on 100 rows sized from those per-row ratios: 362,301 bytes
full, 24,001 brief, a 93.4% reduction.

THE PRIMITIVE WAS HALF-BUILT AND UNREACHABLE. #3906 landed
types.IssueFilter.Lite, IssueSelectColumnsLite, HeavyDropList,
ScanIssueLiteFrom and Issue.IsLitePartial, then wired filter.Lite into exactly
one place: SearchIssuesInTx, the UNCOUNTED store-backed search. Every read that
answers with IssueWithCounts, which is every --json route of both commands,
both implementations of issueops.Reader, and both HTTP reads, runs the counts
mega-query instead and ignored the field. A request that set it would have come
back correct, fully hydrated, and with nothing on it to say the projection had
not happened. That silent no-op is what this closes.

THE APPLICATION POINT IS THE COUNTS MEGA-QUERY, and it is one point rather than
four because sqlbuild.SearchCountsSQL and issueops.ScanReadyWorkRowWithCounts
are already the pair every counted read passes through: the store seam's list
(search_counts.go) and ready (ready_work_counts.go), and the unit-of-work
seam's list (issue_search_counts.go) and ready (ready_work_union.go, via
fetchCountsByIDs). sqlbuild.CountsHydration is already the value that carries
hydration opt-outs across that pair on both seams, so Lite is a third field on
it rather than a new channel.

IT CHANGES THE PROJECTION'S SHAPE, which SkipLabels and SkipCounts do not, and
that is why ScanReadyWorkRowWithCounts now takes the whole hydration instead of
a bool: the value that chose the SELECT list is the value that drives the scan,
so the two cannot be given different answers. Substituting empty strings in the
heavy positions would have kept one scan path and was rejected, because the row
would carry blank text with IsLitePartial=false, which is indistinguishable
from a genuinely textless row, the ambiguity ga-clgh and CommentsOmitted
already record.

WHERE IS INDEPENDENT OF SELECT, so this bounds the size of a row and never
which rows match: DescContains, NotesContains and EmptyDesc keep selecting
exactly what they select today, and no sort key, keyset cursor or wisp-merge
dedup reads a dropped column (CompareIssuesBy orders on nine fields, none of
them heavy).

THE CLAIM REFUSES IT, which is not the same call the page makes and is the
sharper case of the rule Limit and Offset already state. A claim does not read
its row through the page's query at all: ExecuteClaimNext refetches the winning
row whole with GetIssueInTx and hydrates its counts itself, so the projection
has nothing to apply to. Accepting the field would answer a MUTATING request
with a fully hydrated row carrying IsLitePartial=false, the caller's one signal
that it did not get what it asked for, reporting that it did. The check lives
in issueops.ValidateClaimNextRequest, which is the shared rule both ready
claimers and BatchCloser's claim already run, so all four backends refuse it on
the same line.

BuildReadyCountFilter CARRIES it, which is the opposite treatment and the
opposite reason. A count reads no field of any row, so the projection cannot
make the number wrong, and the count is not always cheap enough for that to be
the end of it: the unit-of-work seam has no COUNT(*) over the ready predicate
and sizes the set by running the unbounded page and taking its length
(uow/ready_counter.go), so dropping the field there would hydrate every heavy
column of the whole ready set to answer `bd ready --brief`, a larger read than
the page it was asked for. Carrying it also keeps that builder what its doc
says it is: the listing's filter with the PAGE removed and nothing else.

TWO ASYMMETRIES on the page path, both deliberate and both stated in code:

  - Brief IS carried onto the --ready arm (ReadyFilterFromIssueFilter) where
    SkipLabels and SkipCounts are dropped. Those two hide a number the ready
    renderings print; this drops a body no listing prints, so carrying it is
    what makes `bd list --ready` and `bd ready` answer one request one way.
  - types.WorkFilter carries Lite and neither of the other two, so the ready
    paths get readyHydrationFor rather than hydrationFor. Before this, both
    uow ready call sites passed a hardcoded CountsHydration{}, a literal that
    could not carry a filter's answer at all, which is how that path would have
    been missed.

CONFORMANCE at all four registered sites, one Test per case, for the two page
cases and for the claim refusal. Every field the cases assert stripped is
seeded heavy first and the premise is checked, so a backend that never
round-trips one of the six fails loudly instead of turning that clause into one
that cannot fail. Identity is asserted too (ID, Title, Status, IssueType,
Priority), or a body that stripped the row to its id would satisfy every
emptiness clause. IsLitePartial is asserted in both directions.

THE LITE COLUMN LIST IS PINNED BY ORDER, not only by membership. Both lists are
scanned positionally, so two columns of the same SQL type can be transposed
with no membership change at all: the pre-existing set-based parity test stays
green, the scan succeeds, and every row silently carries one column's value in
another's field. TestIssueSelectColumnsLite_IsFullMinusHeavyInOrder derives the
expected list by deleting HeavyDropList from IssueSelectColumns in place, which
makes the lite list a derivation rather than a second hand-maintained copy.

RED BEFORE GREEN, five sabotages, each verified and reverted:

  - deleting the lite branch from CountsHydration.IssueColumns fails both page
    cases with "expected 58 destination arguments in Scan, not 52";
  - accepting Brief without plumbing it to either filter, the silent no-op this
    change exists to close, fails both page cases on all six text fields and on
    IsLitePartial, which is the run that proves the assertions are
    load-bearing;
  - transposing created_by and owner in IssueSelectColumnsLite fails only the
    new order test, on IssueSelectColumnsLite[9] = owner, want created_by,
    while the set-based test stays green, which is the gap it was added for;
  - dropping the claim's refusal fails the claimer contract with "ClaimNext
    with brief: error = <nil>, want ErrValidation"; and
  - clearing Brief in BuildReadyCountFilter fails
    TestBuildReadyCountFilterIsTheUnboundedReadyFilter, which already existed to
    catch a ReadyRequest field the count builder forgets to carry and gains
    Brief in its fixture here.

The recorded ready-filter golden gains "Lite": false in 62 cases because
types.WorkFilter gained the field and the test compares marshaled JSON
byte-for-byte. False is what a request that did not ask for the projection
resolves that field to, so the replay stays exact; the test comment says so.

Gates: go build ./..., go vet ./..., gofmt, internal/storage/... (dolt,
embeddeddolt with BEADS_TEST_EMBEDDED_DOLT=1, uow, issueops, sqlbuild,
domain/db), internal/workapi/..., backend/conformance/..., issueops/...,
make api-check.

Refs: #5546, #3906

Co-authored-by: sjarmak <t@t.co>
2026-08-10 10:45:23 -07:00

154 lines
6.4 KiB
Go

package workapi
import (
"fmt"
"strings"
"github.com/steveyegge/beads/internal/types"
"github.com/steveyegge/beads/internal/utils"
"github.com/steveyegge/beads/issueops"
)
// DefaultReadyLimit is the default number of rows a ready-work query returns
// when the caller does not ask for a specific limit. Every frontend registers
// its limit knob from this constant so the surfaces cannot drift apart; 0
// still means unlimited.
const DefaultReadyLimit = 100
// LimitOr resolves a request's optional limit: nil takes the surface's shared
// default, and an explicit 0 stays 0, which means unlimited at both storage
// seams. The pointer is what keeps "unset" and "explicitly unlimited"
// distinguishable — collapsing them would make one constant unable to serve
// both surfaces.
func LimitOr(limit *int, fallback int) int {
if limit == nil {
return fallback
}
return *limit
}
// BuildReadyFilter turns a ready request into the storage-level work filter.
// It is the single definition of what `bd ready` means: open issues only (an
// empty WorkFilter would default to open plus in_progress), issue-type alias
// expansion, label and exclude-type normalization, limit defaulting, and the
// assignee rule that lets Unassigned win over a stale Assignee.
//
// Labels, LabelsAny and ExcludeLabels are normalized here, so a caller can
// pass raw user input. A caller that has to decide something from the label
// sets - the CLI's directory-label default is the one such case - reads them
// back off the returned filter, where they are already normalized, rather than
// normalizing its own copy: a value it then puts on the filter itself is its
// own to shape, and running it through here would change it.
func BuildReadyFilter(in issueops.ReadyRequest) (types.WorkFilter, error) {
filter := types.WorkFilter{
// Open only, not in_progress - the same set `bd list --ready` shows.
Status: types.StatusOpen,
Type: utils.NormalizeIssueType(in.IssueType),
Limit: LimitOr(in.Limit, DefaultReadyLimit),
// Carried for the callers that consume this filter as a VALUE and run
// their own query (cmd/bd's proxied `bd ready`), where the seam beneath
// them renders it. Both implementations of issueops.Reader take it back
// off and skip in the shared page epilogue; see BuildListFilter and
// FinishPageAt.
Offset: in.Offset,
Unassigned: in.Unassigned,
SortPolicy: types.SortPolicy(in.Sort),
Labels: utils.NormalizeLabels(in.Labels),
LabelsAny: utils.NormalizeLabels(in.LabelsAny),
ExcludeLabels: utils.NormalizeLabels(in.ExcludeLabels),
LabelPattern: in.LabelPattern,
LabelRegex: in.LabelRegex,
IncludeDeferred: in.IncludeDeferred,
IncludeEphemeral: in.IncludeEphemeral,
ExcludeTypes: normalizeExcludeTypes(in.ExcludeTypes),
HasMetadataKey: in.HasMetadataKey,
Lite: in.Brief,
}
if in.Priority != nil {
p := *in.Priority
filter.Priority = &p
}
if in.Assignee != "" && !in.Unassigned {
a := in.Assignee
filter.Assignee = &a
}
if in.ParentID != "" {
pid := in.ParentID
filter.ParentID = &pid
}
if in.MolType != nil {
filter.MolType = in.MolType
}
if len(in.MetadataFields) > 0 {
filter.MetadataFields = in.MetadataFields
}
if err := ValidateMetadataFilters(in.MetadataFields, in.HasMetadataKey); err != nil {
return filter, err
}
if !filter.SortPolicy.IsValid() {
// A deterministic request-validation failure, so it matches
// ErrValidation: every role whose filter vocabulary this builds —
// Reader.Ready, ReadyClaimer.ClaimNext and BatchCloser's ClaimNext —
// promises a caller can classify one with errors.Is rather than by
// reading prose. The wrap is %.0w rather than a "%w: " prefix because
// this text is what `bd ready --sort bogus` prints verbatim behind
// "Error: "; prefixing it would change user-visible copy to say
// something the reader already knows.
return filter, fmt.Errorf("invalid sort policy '%s'. Valid values: hybrid, priority, oldest%.0w", in.Sort, issueops.ErrValidation)
}
return filter, nil
}
// BuildReadyCountFilter turns a ready request into the storage-level filter a
// COUNT of the ready set runs against: BuildReadyFilter's filter with the page
// removed, and the single definition of what `bd ready`'s published total means.
//
// It refuses a request carrying a page. issueops.ReadyCounter.CountReady
// promises its answer equals len(Reader.Ready(r with Limit=0).Items), and a
// Limit would make that "how many of the first N" while an Offset would
// subtract the rows it skipped from the size of a set that still holds them.
//
// The zeroed limit is set on a LOCAL copy: a nil Limit means the shared ready
// default at BuildReadyFilter, so an unlimited count has to say so explicitly.
func BuildReadyCountFilter(in issueops.ReadyRequest) (types.WorkFilter, error) {
if in.Limit != nil {
return types.WorkFilter{}, fmt.Errorf("%w: a ready count does not take a limit", issueops.ErrValidation)
}
if in.Offset != 0 {
return types.WorkFilter{}, fmt.Errorf("%w: a ready count does not take an offset", issueops.ErrValidation)
}
unlimited := 0
counted := in
counted.Limit = &unlimited
// Brief is CARRIED, not refused and not cleared, which is the opposite of
// what ClaimNext does with it. A count reads no field of any row, so the
// projection cannot make the number wrong; and the count is not always
// cheap enough for that to be the end of it. The unit-of-work seam has no
// COUNT(*) over the ready predicate and sizes the set by running the
// unbounded page and taking its length (uow/ready_counter.go), so clearing
// the field here would hydrate every heavy column of the whole ready set to
// answer `bd ready --brief`, which is the cost the projection exists to
// avoid and larger than the page it was asked for. Carrying it also keeps
// the count filter what this builder says it is: the listing's filter with
// the PAGE removed, and nothing else removed.
return BuildReadyFilter(counted)
}
// normalizeExcludeTypes splits comma-separated exclusions and expands type
// aliases, so --exclude-type=mr,epic and --exclude-type mr --exclude-type epic
// mean the same thing.
func normalizeExcludeTypes(raw []string) []types.IssueType {
var out []types.IssueType
for _, group := range raw {
for _, t := range strings.Split(group, ",") {
t = strings.TrimSpace(t)
if t != "" {
out = append(out, types.IssueType(utils.NormalizeIssueType(t)))
}
}
}
return out
}