STORAGE LAYER ONLY, per CONTRIBUTING_PR_GUIDELINES.md. issueops.ListRequest and issueops.ReadyRequest gain Brief; no CLI flag and no HTTP parameter sets it yet, so no front door's payload changes. The wiring stacks on top. WHAT IT DROPS is the free-form text: description, design, acceptance_criteria, notes, payload and waiters are not selected, and the row comes back carrying types.Issue.IsLitePartial. #5546 measured a 16,051-issue store where `bd ready --json` returned 454,021 bytes and notes plus description were 76% of it, on the call an agent makes to decide what to work on. Measured through issueops.Reader on 100 rows sized from those per-row ratios: 362,301 bytes full, 24,001 brief, a 93.4% reduction. THE PRIMITIVE WAS HALF-BUILT AND UNREACHABLE. #3906 landed types.IssueFilter.Lite, IssueSelectColumnsLite, HeavyDropList, ScanIssueLiteFrom and Issue.IsLitePartial, then wired filter.Lite into exactly one place: SearchIssuesInTx, the UNCOUNTED store-backed search. Every read that answers with IssueWithCounts, which is every --json route of both commands, both implementations of issueops.Reader, and both HTTP reads, runs the counts mega-query instead and ignored the field. A request that set it would have come back correct, fully hydrated, and with nothing on it to say the projection had not happened. That silent no-op is what this closes. THE APPLICATION POINT IS THE COUNTS MEGA-QUERY, and it is one point rather than four because sqlbuild.SearchCountsSQL and issueops.ScanReadyWorkRowWithCounts are already the pair every counted read passes through: the store seam's list (search_counts.go) and ready (ready_work_counts.go), and the unit-of-work seam's list (issue_search_counts.go) and ready (ready_work_union.go, via fetchCountsByIDs). sqlbuild.CountsHydration is already the value that carries hydration opt-outs across that pair on both seams, so Lite is a third field on it rather than a new channel. IT CHANGES THE PROJECTION'S SHAPE, which SkipLabels and SkipCounts do not, and that is why ScanReadyWorkRowWithCounts now takes the whole hydration instead of a bool: the value that chose the SELECT list is the value that drives the scan, so the two cannot be given different answers. Substituting empty strings in the heavy positions would have kept one scan path and was rejected, because the row would carry blank text with IsLitePartial=false, which is indistinguishable from a genuinely textless row, the ambiguity ga-clgh and CommentsOmitted already record. WHERE IS INDEPENDENT OF SELECT, so this bounds the size of a row and never which rows match: DescContains, NotesContains and EmptyDesc keep selecting exactly what they select today, and no sort key, keyset cursor or wisp-merge dedup reads a dropped column (CompareIssuesBy orders on nine fields, none of them heavy). THE CLAIM REFUSES IT, which is not the same call the page makes and is the sharper case of the rule Limit and Offset already state. A claim does not read its row through the page's query at all: ExecuteClaimNext refetches the winning row whole with GetIssueInTx and hydrates its counts itself, so the projection has nothing to apply to. Accepting the field would answer a MUTATING request with a fully hydrated row carrying IsLitePartial=false, the caller's one signal that it did not get what it asked for, reporting that it did. The check lives in issueops.ValidateClaimNextRequest, which is the shared rule both ready claimers and BatchCloser's claim already run, so all four backends refuse it on the same line. BuildReadyCountFilter CARRIES it, which is the opposite treatment and the opposite reason. A count reads no field of any row, so the projection cannot make the number wrong, and the count is not always cheap enough for that to be the end of it: the unit-of-work seam has no COUNT(*) over the ready predicate and sizes the set by running the unbounded page and taking its length (uow/ready_counter.go), so dropping the field there would hydrate every heavy column of the whole ready set to answer `bd ready --brief`, a larger read than the page it was asked for. Carrying it also keeps that builder what its doc says it is: the listing's filter with the PAGE removed and nothing else. TWO ASYMMETRIES on the page path, both deliberate and both stated in code: - Brief IS carried onto the --ready arm (ReadyFilterFromIssueFilter) where SkipLabels and SkipCounts are dropped. Those two hide a number the ready renderings print; this drops a body no listing prints, so carrying it is what makes `bd list --ready` and `bd ready` answer one request one way. - types.WorkFilter carries Lite and neither of the other two, so the ready paths get readyHydrationFor rather than hydrationFor. Before this, both uow ready call sites passed a hardcoded CountsHydration{}, a literal that could not carry a filter's answer at all, which is how that path would have been missed. CONFORMANCE at all four registered sites, one Test per case, for the two page cases and for the claim refusal. Every field the cases assert stripped is seeded heavy first and the premise is checked, so a backend that never round-trips one of the six fails loudly instead of turning that clause into one that cannot fail. Identity is asserted too (ID, Title, Status, IssueType, Priority), or a body that stripped the row to its id would satisfy every emptiness clause. IsLitePartial is asserted in both directions. THE LITE COLUMN LIST IS PINNED BY ORDER, not only by membership. Both lists are scanned positionally, so two columns of the same SQL type can be transposed with no membership change at all: the pre-existing set-based parity test stays green, the scan succeeds, and every row silently carries one column's value in another's field. TestIssueSelectColumnsLite_IsFullMinusHeavyInOrder derives the expected list by deleting HeavyDropList from IssueSelectColumns in place, which makes the lite list a derivation rather than a second hand-maintained copy. RED BEFORE GREEN, five sabotages, each verified and reverted: - deleting the lite branch from CountsHydration.IssueColumns fails both page cases with "expected 58 destination arguments in Scan, not 52"; - accepting Brief without plumbing it to either filter, the silent no-op this change exists to close, fails both page cases on all six text fields and on IsLitePartial, which is the run that proves the assertions are load-bearing; - transposing created_by and owner in IssueSelectColumnsLite fails only the new order test, on IssueSelectColumnsLite[9] = owner, want created_by, while the set-based test stays green, which is the gap it was added for; - dropping the claim's refusal fails the claimer contract with "ClaimNext with brief: error = <nil>, want ErrValidation"; and - clearing Brief in BuildReadyCountFilter fails TestBuildReadyCountFilterIsTheUnboundedReadyFilter, which already existed to catch a ReadyRequest field the count builder forgets to carry and gains Brief in its fixture here. The recorded ready-filter golden gains "Lite": false in 62 cases because types.WorkFilter gained the field and the test compares marshaled JSON byte-for-byte. False is what a request that did not ask for the projection resolves that field to, so the replay stays exact; the test comment says so. Gates: go build ./..., go vet ./..., gofmt, internal/storage/... (dolt, embeddeddolt with BEADS_TEST_EMBEDDED_DOLT=1, uow, issueops, sqlbuild, domain/db), internal/workapi/..., backend/conformance/..., issueops/..., make api-check. Refs: #5546, #3906 Co-authored-by: sjarmak <t@t.co>
179 lines
7.4 KiB
Go
179 lines
7.4 KiB
Go
package workapi
|
|
|
|
import (
|
|
"cmp"
|
|
"strings"
|
|
|
|
"github.com/steveyegge/beads/internal/types"
|
|
"github.com/steveyegge/beads/internal/utils"
|
|
)
|
|
|
|
// CompareIssuesBy orders two issues by one of `bd list --sort`'s fields.
|
|
// An unknown field compares equal, which leaves the underlying order intact.
|
|
func CompareIssuesBy(a, b *types.Issue, sortBy string) int {
|
|
switch sortBy {
|
|
case "priority":
|
|
return cmp.Compare(a.Priority, b.Priority)
|
|
case "created":
|
|
return b.CreatedAt.Compare(a.CreatedAt)
|
|
case "updated":
|
|
return b.UpdatedAt.Compare(a.UpdatedAt)
|
|
case "closed":
|
|
if a.ClosedAt == nil && b.ClosedAt == nil {
|
|
return 0
|
|
} else if a.ClosedAt == nil {
|
|
return 1
|
|
} else if b.ClosedAt == nil {
|
|
return -1
|
|
}
|
|
return b.ClosedAt.Compare(*a.ClosedAt)
|
|
case "status":
|
|
return cmp.Compare(a.Status, b.Status)
|
|
case "id":
|
|
return utils.NaturalCompareIDs(a.ID, b.ID)
|
|
case "title":
|
|
return cmp.Compare(strings.ToLower(a.Title), strings.ToLower(b.Title))
|
|
case "type":
|
|
return cmp.Compare(a.IssueType, b.IssueType)
|
|
case "assignee":
|
|
return cmp.Compare(a.Assignee, b.Assignee)
|
|
}
|
|
return 0
|
|
}
|
|
|
|
// ReadyFilterFromIssueFilter projects a list filter onto the ready-work
|
|
// filter, which is how `--ready` reuses the whole list vocabulary against the
|
|
// blocker-aware query. WorkFilter is the narrower of the two, so this is a
|
|
// deliberate lossy projection: what it does not carry, ready work does not
|
|
// filter on.
|
|
func ReadyFilterFromIssueFilter(filter types.IssueFilter) types.WorkFilter {
|
|
wf := types.WorkFilter{
|
|
Status: types.StatusOpen,
|
|
Limit: filter.Limit,
|
|
Offset: filter.Offset,
|
|
Labels: filter.Labels,
|
|
LabelsAny: filter.LabelsAny,
|
|
ExcludeLabels: filter.ExcludeLabels,
|
|
LabelPattern: filter.LabelPattern,
|
|
LabelRegex: filter.LabelRegex,
|
|
ParentID: filter.ParentID,
|
|
MolType: filter.MolType,
|
|
WispType: filter.WispType,
|
|
ExcludeTypes: filter.ExcludeTypes,
|
|
MetadataFields: filter.MetadataFields,
|
|
HasMetadataKey: filter.HasMetadataKey,
|
|
MaxRows: filter.MaxRows,
|
|
MaxRowsSource: filter.MaxRowsSource,
|
|
// Carried, where SkipLabels and SkipCounts are dropped: those two hide
|
|
// a number the ready renderings print, and this bounds a body no
|
|
// listing prints. Dropping it here would answer `bd list --ready
|
|
// --brief` with full rows and nothing to say why.
|
|
Lite: filter.Lite,
|
|
}
|
|
if filter.IssueType != nil {
|
|
wf.Type = string(*filter.IssueType)
|
|
}
|
|
if filter.Priority != nil {
|
|
wf.Priority = filter.Priority
|
|
}
|
|
if filter.Assignee != nil {
|
|
wf.Assignee = filter.Assignee
|
|
}
|
|
if filter.NoAssignee {
|
|
wf.Unassigned = true
|
|
}
|
|
// The ephemeral PLANE, which the two filters spell differently: a list
|
|
// filter admits it by leaving SkipWisps off (or, for an infra type, by
|
|
// routing to it outright), and a work filter by setting IncludeEphemeral.
|
|
// Reading both spellings is what carries ListRequest.IncludeEphemeral —
|
|
// and IncludeInfra's plane half with it — onto the --ready arm instead of
|
|
// dropping it there, which would answer a request that named the plane
|
|
// with the durable set and no error.
|
|
//
|
|
// THE PREDICATE IS NEGATIVE, on a field whose ZERO VALUE means admit. That
|
|
// is deliberate but it is a footgun worth naming: this fires for any filter
|
|
// that merely never set SkipWisps, not only for one that asked. Every
|
|
// caller today is handed a filter built by BuildListFilter, which decides
|
|
// that field for every request, so "unset" is not reachable here — a
|
|
// hand-built types.IssueFilter{} passed in from somewhere new would get the
|
|
// wisp plane it never asked for. Keep the construction on the builder.
|
|
if (filter.Ephemeral != nil && *filter.Ephemeral) || !filter.SkipWisps {
|
|
wf.IncludeEphemeral = true
|
|
}
|
|
return wf
|
|
}
|
|
|
|
// WithFetchOneExtra bumps Limit by one so the caller can detect "more results
|
|
// than the limit" (GH#3212) by comparing the row count against the original
|
|
// limit.
|
|
//
|
|
// `--limit N --max-rows N` (equal) needs special handling so this bump doesn't
|
|
// collide with the MaxRows cap check: EffectiveSearchLimit treats
|
|
// Limit==MaxRows as "no overage sniff" (returns Limit verbatim, no +1 — see
|
|
// EffectiveSearchLimit's doc and the WithLimit_LimitAtCap_NoError storage
|
|
// test), but bumping Limit to N+1 alone flips that to Limit>MaxRows, which
|
|
// *does* sniff for overage — so the probe row itself would trip
|
|
// EnforceMaxRowsCap(N+1, N, ...) even though the delivered set is always
|
|
// trimmed back to N. Bumping MaxRows by one in lockstep, only in this exact
|
|
// N==M case, keeps the probe row inside the (temporarily N+1) cap so
|
|
// EnforceMaxRowsCap never fires on it while still fetching N+1 rows for the
|
|
// truncation check. This can't false-negative a real violation: the query's
|
|
// own LIMIT is capped at N+1 either way, so "more than N+1 matches exist" was
|
|
// already undetectable at Limit==MaxRows before this bump ever existed (the
|
|
// same "no overage detection above the equal cap" contract EffectiveSearchLimit
|
|
// documents for the unbumped case).
|
|
//
|
|
// This does not change the Limit>MaxRows (tighter-cap) or Limit<MaxRows cases:
|
|
// EffectiveSearchLimit's branch selection there is unaffected by a one-row bump
|
|
// to Limit, and MaxRows is left untouched, so a genuine tighter-cap violation
|
|
// still fires with the correct (unbumped) Cap value in the error message.
|
|
// The pair of steps this performs — bump the cap in lockstep at Limit==MaxRows,
|
|
// then bump Limit — is also written as one function for the seam that renders
|
|
// its probe row inside the query rather than onto a filter; see
|
|
// internal/storage/issueops.SearchProbeLimit. Change one and change the other,
|
|
// or the two implementations of issueops.Reader.List stop agreeing about when a
|
|
// cap fires.
|
|
func WithFetchOneExtra(filter types.IssueFilter) types.IssueFilter {
|
|
if filter.Limit > 0 {
|
|
if filter.MaxRows > 0 && filter.Limit == filter.MaxRows {
|
|
filter.MaxRows++
|
|
}
|
|
filter.Limit++
|
|
}
|
|
return filter
|
|
}
|
|
|
|
// WithRowsBeforeThePage moves an OFFSET off the query and onto the epilogue: it
|
|
// widens the filter's row bound to cover the rows the epilogue will skip, and
|
|
// clears the Offset the builder carried so the query does not skip them too.
|
|
// An unbounded filter needs no widening — it already carries every matching row.
|
|
//
|
|
// It is the filter half of FinishPageAt, and the reason both halves exist is
|
|
// there: the skip belongs after the display order, and the rows it drops are
|
|
// rows the MaxRows cap has to have counted. Apply it BEFORE WithFetchOneExtra,
|
|
// which sizes the probe row against the bound this leaves.
|
|
//
|
|
// The builders still write Offset onto the filter, for the callers that consume
|
|
// it as a VALUE and run their own query — `bd list --watch`, the proxied
|
|
// hierarchical --parent walk, proxied `bd ready` — where there is no shared
|
|
// epilogue to move it to. Those callers get the same cap boundary anyway: the
|
|
// seam beneath them widens its own bound by the offset and keeps the skip
|
|
// whenever a cap is set, for the reason stated above (internal/storage/domain/db,
|
|
// searchWindow).
|
|
func WithRowsBeforeThePage(filter types.IssueFilter, offset int) types.IssueFilter {
|
|
filter.Offset = 0
|
|
if offset > 0 && filter.Limit > 0 {
|
|
filter.Limit += offset
|
|
}
|
|
return filter
|
|
}
|
|
|
|
// WithReadyRowsBeforeThePage is WithRowsBeforeThePage for the ready-work filter,
|
|
// which is a different type carrying the same two fields.
|
|
func WithReadyRowsBeforeThePage(filter types.WorkFilter, offset int) types.WorkFilter {
|
|
filter.Offset = 0
|
|
if offset > 0 && filter.Limit > 0 {
|
|
filter.Limit += offset
|
|
}
|
|
return filter
|
|
}
|