* feat(search): re-cut SearchIssueSummaries atop Lite as searchProjection[T]
Re-cuts release/be-nu4.3-stack onto current main, resolved against the
merged Lite foundation (#3906) instead of the stale pre-Lite conflict set.
SearchIssueSummaries moves from a standalone search_summary.go into a
summaryProjection literal alongside issueProjection/issueLiteProjection/
idProjection in issueops/search.go, sharing the generic searchInTx core
instead of a parallel top-level function — retiring the anti-pattern
coffeegoddd's CHANGES_REQUESTED flagged. LessSummary shares lessFields
with Less via a summarySortFields adapter so the two comparators can't
drift on NULL-handling or tie-break rules.
Adds parity coverage (SearchIssues vs SearchIssueSummaries agree on ID
set/order and on every IssueSummary field) and the three-way wide/lite/
summary A/B benchmark, extending dolt_benchmark_test.go — which still
only compiles together with the projection method, the fact that
retired the earlier "land the bench file first" plan.
* test(feat): red — fix parity test to compare order directly + add comparator/wisp-limit coverage (refs be-be3pj)
- TestSearchIssuesAndSearchIssueSummaries_Parity now compares
SearchIssues/SearchIssueSummaries IDs order-sensitively (slices.Equal)
instead of via the sort-then-compare equalStringSlices helper, which
would silently pass an ordering/top-N regression.
- Add TestLessSummaryMatchesLessAcrossSortKeys: Less vs LessSummary
produce identical order across every SortDefs key + "id", both
directions.
- Add TestSearchIssueSummariesInTx_MergesAcrossIssuesAndWisps: be-yrtwi
regression pattern for summaryProjection specifically, covering
wisp-merge top-N truncation under a limit.
* fix(types): give IssueSummary the same JSON tags as Issue
IssueSummary carried no struct tags, so encoding/json emitted Go's default
field names ({"ID":…,"Title":…}). The type exists to back list-shaped
rendering and `bd list --json` is one of that command's primary modes, so
the wiring follow-up would have silently changed the wire shape for every
consumer parsing bd output. Zero production callers today makes this a
zero-risk moment to fix it.
Tags mirror the same-named fields on types.Issue exactly, including
priority carrying no omitempty (0 is a valid P0). Three tests pin it: a
reflection test comparing every tag against Issue's so the two cannot
drift, an exact-key-set marshal test, and an omitempty test.
Reviewer item 1 (maphew, 2026-08-03).
* docs(storage): pin the --sort id contract on SearchIssueSummaries
The interface said SortBy/SortDesc are honored identically to SearchIssues,
which is true but left the id case looking like a gap in the summary path.
It is not a gap, and the doc now says why: sqlbuild.OrderBy emits no ORDER BY
for id (sqlbuild.IsGoSideSort), so *neither* path gets a SQL id order, and
both carry a comparator used only where the issues+wisps merge re-sorts
(sqlbuild.Less / sqlbuild.LessSummary).
The user-facing id order is cmd/bd's own, applied after the call via
utils.NaturalCompareIDs (cmd/bd/list.go sortIssues -> compareIssuesBy) —
numeric-aware, so bd-2 sorts before bd-10. Adding a lexicographic
storage-side summary sorter would therefore diverge from SearchIssues and
still not match what bd list renders.
Reviewer question (maphew, 2026-08-03): the contract no longer depends on
an unfiled follow-up to be readable.
* fix(backend): alias IssueSummary into the public backend contract
#5321 landed the public backend surface and its reflection drift guard on
2026-08-03, about two hours after this branch's last push, so the 08-11
maintenance rebase brought the branch under a gate that did not exist when
the engine interface was widened. TestPublicSurfaceComplete fails on both
platforms at the rebased head:
internal type github.com/steveyegge/beads/internal/types.IssueSummary
is reachable from the public backend contract but has no alias in
package backend
The gate is right and the rebase broke nothing. SearchIssueSummaries
returns []*types.IssueSummary on storage.Storage, which backend.DoltStorage
aliases whole, so the row type is now part of the contract an out-of-tree
backend writes against — and without the alias that backend cannot name it.
Verified this is ours rather than base breakage: the same test passes on
origin/main and fails only on this branch.
* ci: retrigger after infra-flake red on dc92c3c3
Run 34381971686 went red on two failures, neither of them in this PR's
15-file diff:
- Test (storage domain + uow): internal/tracker lost its Dolt
testcontainer mid-suite after a Ryuk reaper handshake reset
(54 failed, 1994 passed). Known parallel-binary reaper flake; the
fix is PR #5770, open and unmerged upstream since 2026-08-15. Third
recorded recurrence, same package and same mapped port as the prior
two.
- PR Core (wrapper timing): TestDoltServer_LogFile_CapturesOutput in
internal/storage/dbproxy/server, a package this PR does not touch.
Not reproducible locally in 55 attempts against clean main, 30 of
them under 2x CPU load with the CI-pinned dolt 2.2.0.
We are pull-only on the upstream repo and cannot rerun jobs, so an
empty commit is the only retrigger lever available.
* test(issueops): tie the summary column list to its scanner, delete a third copy
Review point (minor 2): IssueSummaryColumns, ScanIssueSummaryFrom's dest list
and search_summary_merge_test.go's summaryColumnNames were three hand-synced
copies of the same list, with only the real-database field-parity test to
catch a drift between them.
Two changes rather than one guard over three lists:
- summaryColumnNames now derives from IssueSummaryColumns instead of
restating it, so the fixture cannot drift at all. summaryMergeRow places
its values by column NAME against that list rather than by literal
position, so adding a column widens the row automatically instead of
silently shifting every value one slot left.
- TestIssueSummaryColumnsMatchScanner pins the remaining pair by count,
using a counting scanner that needs no database. This is the summary
counterpart of TestIssueSelectColumns_LitePlusHeavyEqualsFull.
TestIssueSummaryColumnsAreRealIssueColumns covers what a count cannot: a
misspelled column keeps the widths equal and fails only at query time against
a real database, as an opaque unknown-column error from whichever caller runs
first. Requiring membership in IssueSelectColumns catches it in a unit test.
Membership, not order: unlike IssueSelectColumnsLite, IssueSummaryColumns is
deliberately not a subsequence of the full list, so there is no analogue of
TestIssueSelectColumnsLite_IsFullMinusHeavyInOrder.
Both guards verified red-before-green by mutation: adding a column without a
dest fails the first, and misspelling one fails the second.
* fix(types): carry the wisp-plane markers through the summary projection
Review point (MAJOR): IssueSummary's doc comment promises a summary-backed
list serializes to the same wire shape a full-Issue-backed one does, and
issueops.searchInTx merges wisp rows into every result whose filter does not
set SkipWisps. But IssueSummary had no Ephemeral, NoHistory, WispType or
StorageClass, so a wisp rendered through it dropped ephemeral, no_history,
wisp_type and storage_class from `bd list --json` while every other key stayed
identical -- a wisp and a durable bead became indistinguishable on the wire.
maphew raised this on 2026-08-03 and it was still open.
Adding the fields, not excluding wisps. Excluding them would break the
same-wisp-merge-semantics claim that is this PR's central design argument, and
would make the wiring follow-up silently drop wisps from `bd list` entirely --
a strictly worse outcome than the missing keys.
Correcting our own 2026-08-03 reasoning, which argued the other way. That
comment observed that no list *renderer* reads Ephemeral or WispType back off
the row, which is true and beside the point: `--json` does not read fields,
it marshals the struct. It also treated the cost as the one D3 exists to
remove. These four are narrow scalars -- two TINYINT(1), two short VARCHARs,
already present on both the issues and wisps tables and already scanned by
ScanIssueFrom -- not the multi-KB TEXT bodies and JSON parse the projection is
narrow to avoid. Zero callers today makes this free; after the wiring lands it
is a wire-format change.
Verified red-before-green at both levels:
- TestIssueSummaryCarriesWispMarkers failed on all four fields before the
change (reflection against Issue, asserting tag AND type -- WispType and
StorageClass are named string types that a plain string would impersonate
under a tag-only check).
- TestIssueSummaryWispKeysMatchIssue marshals the same marker values through
both types and compares the four keys byte for byte, including a durable
row that must emit none of them.
- TestSearchIssueSummaries_FieldsMatchSearchIssues now seeds an ephemeral
wisp and a no-history wisp (mutually exclusive, so one row each) and
compares all four markers against SearchIssues on a real Dolt database. It
refuses to pass vacuously: fewer than two wisp rows in the result is a
fatal, since zero-vs-zero marker comparisons would otherwise succeed if
the merge stopped returning wisps. Mutating ScanIssueSummaryFrom to stop
mapping wisp_type fails it on both seeded rows.
ScanIssueSummaryFrom maps the four exactly as ScanIssueFrom does, including a
NULL storage_class staying empty for EffectiveStorageClass to resolve per
plane rather than being pinned to a literal.
* perf(sqlbuild): compare sortFields through pointers, not two copies per call
Review point (minor 3): the shared-comparator refactor made Less build two
sortFields values per comparison and pass them by value, a small regression on
the pre-existing issues+wisps merge sort inside a perf PR.
lessFields and sortKeyCompareFields now take *sortFields. sortFields is a
~150-byte struct, so passing by value copied it twice into lessFields and
twice more into sortKeyCompareFields on every comparison; that is now zero
copies after the adapters run. The adapters still build one value per row --
that normalization is exactly what buys Less and LessSummary a single shared
body, and giving it up would reintroduce the drift this PR exists to remove.
No allocation either way, before or after: escape analysis reports nothing in
Less/LessSummary/lessFields escaping to heap (the only escapes in sort.go are
the pre-existing SortDefs map and OrderBy's string formatting), and both arms
measure 0 B/op, 0 allocs/op. A throwaway A/B in one binary, -benchtime=3s
-count=3 on the priority key, put the by-value median at 26.33 ns/op and the
pointer median at 25.35 ns/op, with non-overlapping ranges -- a consistent but
small ~4% win.
Stating that plainly rather than claiming more: the value here is removing four
struct copies per comparison from the hot path, not the wall-clock number,
which is within a couple of nanoseconds. Behavior is unchanged; the sqlbuild
suite, including the parity and LessID-honors-sortDesc tests, passes.
* test(dolt): vary SortBy across the summary parity table
Review point (minor 4): none of the nine parity cases set SortBy, so
LessSummary-versus-SQL ORDER BY agreement was proven by unit test alone
(TestLessSummaryMatchesLessAcrossSortKeys) and never against a real database.
Every case ran at the default order, priority ASC.
Nine rows added, covering created, title, status, updated and assignee -- the
first two in both directions -- plus id in both directions. Each carries a
Limit, so the merge's sort-before-trim actually decides which rows survive
rather than the ordering being cosmetic: an ordering bug that only shows up in
which rows a top-N keeps is the failure mode this table exists to catch.
id is included deliberately even though sqlbuild.OrderBy emits no ORDER BY for
it. It is the one Go-side key, so these two rows pin that both paths still
agree on the order the merge comparator produces where SQL contributes none.
All nine pass against a real Dolt database.
* docs(storage): document the summary projection's caller contract
Two review points, both about the contract being legible to a caller who has
only the interface and the extension guide.
Minor 1 -- IncludeDependencies is a silent no-op on the summary path, because
types.IssueSummary has no Dependencies field to populate. hydrateSummaryLabels
says why in its own comment; the interface a caller actually reads did not.
SearchIssueSummaries' doc now states it, alongside the neighbouring facts that
SkipLabels IS honored identically and that wisps are not excluded.
Minor 5 -- engdocs/EXTENDING.md documented the Lite projection's contract and
never mentioned the third one. It now carries a parallel section: what a caller
may read, why there is no IsLitePartial equivalent (a narrower type makes a
missing field a compile error rather than a silent zero value, which is the
substantive difference from Lite), the dependency and wisp rules above, the
ordering guarantee, and where each is enforced -- including the note that
IssueSummaryColumns is deliberately not an ordered subsequence of the full
list, so it has no analogue of the lite path's in-order guard.
The Lite section is left intact, its own Backend coverage subsection included;
the new section follows it rather than interleaving.
* chore(bazel): enumerate the two new test files in their go_test srcs
The PR adds internal/types/issue_summary_test.go and
internal/storage/issueops/search_summary_merge_test.go. Both are untagged,
and both packages carry a base-added BUILD.bazel whose go_test enumerates
srcs explicitly, so bazel silently would not have compiled or run them:
measured on //internal/types:types_test, the unrepaired binary lists 0 of
the 5 new tests (68 total) and the repaired one lists 5 of 5 (73), with
both builds exiting 0 -- a fake green, since bazel reports "1 test passes"
either way.
Base parity at 43a526b828 is clean (102 packages / 1192 untagged test files
enumerated, 0 srcs gaps), so the omission is PR-introduced rather than an
unenforced list. make bazel-sync-check and tools/bazel/go_srcs.py --check
both exit 0 on the broken head; generated with make bazel-sync (gazelle),
which is idempotent here.
Maintainer fixup applied during adopt-pr rebase (PR #3458).
* docs: announce the required SearchIssueSummaries method and inventory its span
Review follow-through for #3458 (maintainer fixup, iteration 1).
CHANGELOG.md: backend/backend.go promises that a required method added to
the engine interface is called out in CHANGELOG.md. SearchIssueSummaries is
one and had no entry. Adds an [Unreleased] "Changed" bullet in the #4911
form, next to the #6731 GetReadyWorkWithCountsAndTotal announcement: the
method and its signature, that callers are unaffected while implementers
must add it to compile, and its contract (SortBy/SortDesc and SkipLabels as
in SearchIssues, IncludeDependencies a silent no-op, wisps merged with their
markers set).
engdocs/design/otel/otel-data-model.md: the direct-operations block calls
itself a complete inventory and was exact at the PR base. The PR's new
storage.SearchIssueSummaries span was the one missing name. Inserted in
alphabetical order and bumped the count to 74. A set-diff against the
s.op(ctx, "...") names in internal/telemetry/storage.go now finds 74 == 74
with no drift in either direction.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
6 KiB
Extending bd
This file documents contracts that callers of the storage API must honor. It is not user-facing; it is for code that embeds bd or talks to the storage layer directly.
Lite SELECT shape — IssueFilter.Lite
store.SearchIssues(ctx, query, filter) accepts an IssueFilter value.
When filter.Lite == true, the storage layer issues a narrower SELECT
that omits these heavy TEXT columns:
descriptiondesignacceptance_criterianoteswaiterspayload
Contract for callers
Code that calls store.SearchIssues with IssueFilter.Lite == true:
- MUST NOT read
Description,Design,AcceptanceCriteria,Notes,Payload, orWaitersfrom any returned*types.Issue. These fields are zero-valued after a lite scan; they did not come from the row. Reading them yields no signal. - MAY read every other field — identity, status, priority, timestamps, labels, dependencies, metadata, etc. Lite preserves them.
- MUST detect lite-fetched records via
issue.IsLitePartialif branching behavior on hydration depth is required. The field is internal-only (json:"-") — it never crosses the wire.
To recover the full body for a specific issue after a lite listing,
call store.GetIssue(ctx, id) — GetIssue always returns the full row.
Default behavior
IssueFilter.Lite defaults to false. Every existing call site that
does not opt in retains today's behavior: heavy columns are fully
hydrated, and Issue.IsLitePartial is false.
Where the contract is enforced
- Column lists:
internal/storage/issueops/scan.go(IssueSelectColumns,IssueSelectColumnsLite,HeavyDropList). - Scan helpers:
ScanIssueFrom(full) andScanIssueLiteFrom(lite, setsIsLitePartial). - SELECT dispatch:
internal/storage/issueops/search.go—SearchIssuesInTxselectsissueProjectionorissueLiteProjectiononfilter.Lite; both aresearchProjection[*types.Issue]literals sharing the wisp-merge and hydration machinery insearchTableInTxT. - Schema-parity guard:
internal/storage/issueops/scan_test.go::TestIssueSelectColumns_LitePlusHeavyEqualsFullfails CI if a future column is added toIssueSelectColumnswithout being classified intoIssueSelectColumnsLiteorHeavyDropList.
Backend coverage
filter.Lite is currently honored only by the issueops-backed stores
(Dolt, embedded Dolt) via the dispatch above. The proxied-server
(internal/storage/domain/db) path — issueSQLRepositoryImpl.searchTable
/ fetchIssuesByIDs — does not check filter.Lite yet: it always issues
the full issueSelectColumns SELECT and returns fully-hydrated issues
with IsLitePartial == false. This is correct-but-unoptimized (no lite
callers exist yet, so the difference is invisible today); wiring
filter.Lite through the domain/db stack is deferred to the CLI-wiring
follow-up (be-uwvs.2+), not part of this foundation.
Summary projection — store.SearchIssueSummaries
SearchIssueSummaries is a third projection alongside the full and lite
shapes above. It returns []*types.IssueSummary rather than
[]*types.Issue, selecting only IssueSummaryColumns — narrower than
IssueSelectColumnsLite, because it drops the routing/claim scalars
(metadata, row_lock, the leases.* overlay) a list-shaped render never
reads, not just the heavy TEXT bodies.
Contract for callers
- MAY read every field
types.IssueSummarydeclares. All of them come from the row, so there is noIsLitePartialequivalent and no partial state to detect — a field the type does not have is a compile error, not a silent zero value. That is the difference from Lite: Lite returns a full-shaped*types.Issuewith some fields hollowed out, the summary projection returns a narrower type. - MUST call
store.SearchIssuesinstead when dependencies are needed.types.IssueSummaryhas noDependenciesfield, soIssueFilter.IncludeDependenciesis a silent no-op on this path.SkipLabelsis honored exactly as it is onSearchIssues. - MAY rely on wisps behaving identically. The issues+wisps merge in
searchInTxruns for any filter that does not setSkipWisps, andtypes.IssueSummarycarries the four wisp-plane markers (Ephemeral,NoHistory,WispType,StorageClass), so a merged wisp row serializes the same through either projection. - MAY rely on identical ordering.
SortBy/SortDescrender the same SQLORDER BY, and the post-merge re-sort usessqlbuild.LessSummary, which shares one comparator body withsqlbuild.Less.
Where the contract is enforced
- Column list and scanner:
internal/storage/issueops/scan.go(IssueSummaryColumns,ScanIssueSummaryFrom). - SELECT dispatch:
summaryProjectionininternal/storage/issueops/search.go— asearchProjection[*types.IssueSummary]literal on the samesearchTableInTxTcore as the other two, so WHERE clauses, wisp merge, dedup, trim andMaxRowscannot drift between them. - Column/scanner guards:
scan_test.go::TestIssueSummaryColumnsMatchScanner(count agreement) andTestIssueSummaryColumnsAreRealIssueColumns(every column exists on the issues/wisps tables). NoteIssueSummaryColumnsis deliberately not a subsequence ofIssueSelectColumnsin order, so it has no analogue ofTestIssueSelectColumnsLite_IsFullMinusHeavyInOrder. - Wire-shape guards:
internal/types/issue_summary_test.gopinsIssueSummary's JSON tags againstIssue's field by field, including the wisp markers. - End-to-end parity:
internal/storage/dolt/search_parity_test.gocomparesSearchIssuesagainstSearchIssueSummarieson a real database, across filters and sort keys, and field by field for durable and wisp rows.
Backend coverage
No in-tree caller reads SearchIssueSummaries yet — this is the storage
foundation, and the bd list wiring that spends it is tracked separately.
The proxied-server (internal/storage/domain/db) path does not implement
the narrow projection either, for the same reason filter.Lite is not
wired through it yet (above).