beads/engdocs
Jim Wordelman 132601bfa0
feat(storage): opt-in lite SELECT shape (Issue body columns deferred) (#3906)
* feat(storage): be-uwvs.1 lite SELECT shape foundation

Add IssueFilter.Lite + IssueSelectColumnsLite + HeavyDropList +
ScanIssueLiteFrom so listing paths can opt out of materializing heavy
TEXT columns (description, design, acceptance_criteria, notes, payload,
waiters). Default Lite=false preserves today's behavior everywhere;
Lite=true (once wired to a call site) would return issues with
IsLitePartial=true and heavy fields blank.

A schema-parity test (cols(Lite) ∪ HeavyDropList == cols(Full)) guards
against future IssueSelectColumns drift, and happy-path tests exercise
both scan helpers.

Out of scope (per bead): CLI wiring, WorkFilter.Lite,
per-backend correctness tests, benchmark — those are be-uwvs.{2,3,4,5}.

Refs: be-uwvs (designer §1/§4/§5), be-jp5s (architecture D2/R-07).

Note (rework, PR #3906): this commit is quad341's original patch
cherry-picked onto current main, minus its internal/storage/issueops/
search.go hunk. That hunk targeted searchTableInTx, which #4150
("perf(id_parser): narrow SearchIssues to id-only projection", merged
2026-07-04) replaced with a generic searchProjection[T] dispatch; the
old hunk has no target to apply to and is dropped here rather than
force-merged. search.go is therefore unchanged by this commit — no
caller can reach the Lite branch yet. The follow-up commit re-homes the
filter.Lite switch inside the new projection system.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: release gate PASS for be-uwvs.1 (lite SELECT foundation)

* rework(storage): re-home lite SELECT switch into projection dispatch (PR #3906)

Re-homes quad341's filter.Lite SELECT-shape switch (PR #3906) into the
generic searchProjection[T] dispatch that #4150 ("perf(id_parser): narrow
SearchIssues to id-only projection", merged 2026-07-04) introduced in
internal/storage/issueops/search.go, replacing the searchTableInTx
function the PR originally patched.

- internal/storage/issueops/search.go: add issueLiteProjection, a
  searchProjection[*types.Issue] literal mirroring issueProjection but
  with columns()->IssueSelectColumnsLite and scan()->ScanIssueLiteFrom.
  SearchIssuesInTx now selects issueProjection or issueLiteProjection
  based on filter.Lite before calling the shared searchInTx wrapper, so
  the Lite branch gets wisp-merge, id-shrink Pattern B, and label/dep
  hydration for free, same as the full projection. No new top-level
  function and no patched SQL-string branch inside searchTableInTxT,
  matching the PR's own "adding a narrow-projection variant means adding
  a projection literal" design note already on issueProjection/idProjection.

- internal/storage/issueops/scan.go: classify the two column groups added
  to IssueSelectColumns since this PR's original patch predates them:
  `row_lock` (optimistic-concurrency token) and the `leases.lease_expires_at`
  / `leases.heartbeat_at` overlay (migration 0054, joined via
  sqlbuild.LeaseJoin). Both are small, non-TEXT, routing-relevant columns,
  so both go into IssueSelectColumnsLite/ScanIssueLiteFrom rather than
  HeavyDropList, matching the maintainer review's reasoning for the lease
  columns and extending it to row_lock (not discussed in the review because
  it wasn't yet on main when the review was written). issueLiteProjection
  sets joinLeases: true so the FROM clause gets the LEFT JOIN these columns
  require.

Verification (2026-07-23, current upstream/main = 1bbca0c1f):
- go build -tags gms_pure_go ./...                            -> clean
- go vet -tags gms_pure_go ./...                               -> clean
- gofmt -l on changed files                                    -> clean
- go test -tags gms_pure_go -run 'Lite|IssueSelectColumns' \
    ./internal/storage/...                                     -> PASS
    (TestIssueSelectColumns_LitePlusHeavyEqualsFull,
     TestScanIssueLiteFrom_LeavesHeavyFieldsBlank,
     TestScanIssueFrom_PopulatesHeavyFields,
     TestIssueSelectColumns_MatchClassic)
- go test -tags gms_pure_go ./internal/storage/issueops/...    -> PASS
- go test -tags gms_pure_go ./internal/storage/embeddeddolt/... \
    ./internal/storage/schema/... \
    ./internal/storage/versioncontrolops/... \
    ./internal/types/...                                       -> PASS

Scope preserved from the PR: foundation only. filter.Lite still has no
caller on main (grepped for `.Lite = true` / `Lite: true` outside this
diff -- none found); CLI wiring (bd list / bd dep tree / bd dep graph
--json / WorkFilter.Lite) remains deferred to be-uwvs.2+ per the PR body.

Rework of gastownhall/beads#3906 (feat/be-uwvs.1-lite-select-foundation by
quad341), per maphew's 2026-07-05 review comment on that PR, which
identified #4150 as the timing conflict and named the lease columns as
needing classification.

Agent-Signature: claude-sonnet-medium on behalf of maphew

* chore(storage): drop stale release-gate artifact from PR #3906

release-gates/be-uwvs.1-gate.md is a contributor-process artifact from
the original PR (contains stale fork references) and isn't something
this repo's process wants carried on the branch; docs/EXTENDING.md stays
as the legitimate caller-contract doc introduced by this PR.

Per Opus review (APPROVE-WITH-NITS) on the rework in
rework(storage): re-home lite SELECT switch into projection dispatch
(6882a4513).

Agent-Signature: claude-sonnet-medium on behalf of maphew

* docs(storage): note domain/db path doesn't honor filter.Lite yet

Per Codex review: the proxied-server (domain/db) path —
issueSQLRepositoryImpl.searchTable / fetchIssuesByIDs in
internal/storage/domain/db/issue_search.go — always issues the full
issueSelectColumns SELECT and scanIssue scanner, ignoring filter.Lite
entirely. Confirmed: neither function references filter.Lite.

Adjudicated as document-and-defer, no code change: no domain/db caller
sets Lite=true yet, so this is graceful degradation (correct results,
no lite optimization) rather than a bug, and wiring Lite through
domain/db belongs to the be-uwvs.2+ CLI-wiring follow-up alongside the
rest of the caller work.

- docs/EXTENDING.md: new "Backend coverage" subsection under the Lite
  SELECT shape contract; also fixes the "SELECT dispatch" bullet, which
  still named the pre-#4150 searchTableInTx function, to name the
  current issueProjection/issueLiteProjection dispatch.
- internal/types/types.go: same one-liner added to the IssueFilter.Lite
  field doc comment.

Agent-Signature: claude-sonnet-medium on behalf of maphew

* fix(storage): be-uwvs.1 wire less comparator into issueLiteProjection

issueLiteProjection was missing the less: sqlbuild.Less field that
issueProjection carries, per maphew's 2026-07-24 review comment on
PR #3906 flagging this as the remaining follow-up once #4160 landed.
Without it, searchInTx's merge branch (issues+wisps) has no comparator
to re-sort the concatenation of two independently-ordered legs before
trimming to filter.Limit when a Lite search takes that path — the trim
would keep whichever leg happens to come first regardless of sort rank,
unlike the full (non-Lite) projection which already re-sorts correctly.

Also relocates the PR's new docs/EXTENDING.md to engdocs/EXTENDING.md:
the file's own text says "It is not user-facing; it is for code that
embeds bd or talks to the storage layer directly," and
test/docsync.TestEveryDocsPageIsPublished requires every docs/ page to
carry a docs.json navigation entry (Mintlify-published) or live under
engdocs/ instead. engdocs/ is the existing home for this class of
internal contract doc, so move it there rather than adding a new
docs.json entry or exemption. Updates the two code comments that
pointed at the old path.

Verification (2026-07-25, rebased onto current origin/main):
- go build ./...                                     -> clean
- go build -tags gms_pure_go ./...                    -> clean
- go vet ./...                                        -> clean
- go test ./internal/storage/issueops/... ./internal/types/... ./test/docsync/...
                                                        -> PASS
- go test ./... (full suite)                          -> only pre-existing,
  environment-induced failures unrelated to this diff (verified identical
  on a clean origin/main baseline checkout: internal/beads, internal/config,
  internal/configfile, cmd/bd/doctor TMPDIR/shared-dolt-server/config-leak
  artifacts of this sandbox, plus TestCommitBeadsConfigSkipsGitHooks). Docker
  unavailable in this environment, so the cgo/Dolt-backed gate tests in
  cmd/bd (lite_filter_spy-based) self-skip, same as on baseline.

Rebased gastownhall/beads#3906 (feat/be-uwvs.1-lite-select-foundation)
onto current origin/main; the only conflict was the additive
IssueFilter struct-field collision with #3908/#4160's MaxRows/
MaxRowsSource (both kept).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(issueops): be-yrtwi regression test for issueLiteProjection merge-sort

issueLiteProjection was missing the less: sqlbuild.Less comparator that
issueProjection carries (fixed in 64778027e), so sortMergedResults
silently no-op'd on the issues+wisps merge branch and trimToSearchLimit
kept issues-leg concatenation order instead of the true globally-sorted
top-N. No test exercised IssueFilter{Lite: true} through SearchIssuesInTx
end-to-end, so this bug had no regression guard.

Adds a sqlmock-based end-to-end test (matching this package's existing
convention — no test in internal/storage/issueops uses a live DB) that
seeds an issues leg and a wisps leg where the true created_at-DESC top-2
lives entirely in the wisps leg, and asserts SearchIssuesInTx returns
that true order rather than the stale issues-leg rows. Verified this
fails against the pre-64778027e state (temporarily stripped less: and
confirmed red) and passes with it restored.

* fix(storage): classify leases.granted_node into IssueSelectColumnsLite (PR #3906)

Merging upstream/main into the PR #3906 rework branch (feat/be-uwvs.1-lite-
select-foundation) brought in wy-jpd3.7's replica-aware leases, which added
leases.granted_node to LeaseSelectColumns and therefore to IssueSelectColumns
(sqlbuild package). TestIssueSelectColumns_LitePlusHeavyEqualsFull caught the
drift: the new column was in neither IssueSelectColumnsLite nor HeavyDropList,
so the schema-parity invariant (lite ∪ drop == full) broke.

granted_node is a small, non-TEXT lease-overlay column — the same class as
lease_expires_at/heartbeat_at, which IssueSelectColumnsLite already retains
(see #4150) — so it belongs in the lite list, not HeavyDropList. Adds it to
IssueSelectColumnsLite, scans it in ScanIssueLiteFrom into
issue.LeaseGrantedNode (mirroring ScanIssueFrom), and updates the sqlmock
column/row fixtures in search_lite_merge_test.go (be-yrtwi's issueLiteProjection
merge-sort regression test) to the new column count.

This is downstream of #4160's `less: sqlbuild.Less` comparator wiring on
issueLiteProjection, already present on this branch and preserved by the
merge (search.go issueLiteProjection literal) — this commit only follows up
the schema-parity fallout from bringing that history in. Bead mybd-hu2k.
Agent-Signature: claude-fable-5-medium on behalf of matt wilkie

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-26 15:29:08 -07:00
..
adr docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
decisions docs(pipeline): pin the CLI docs to the released bd; repair main's broken CLI reference (#4876) 2026-07-17 18:03:28 -07:00
design docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
staged-for-removal docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
AGENT_SIGNING.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
ATTRIBUTION.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
CI_CLEANUP_PLAN.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
CI_REQUIRED_CHECK_TOPOLOGY.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
CI_TEST_SURFACE_AUDIT.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
CLAUDE.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
COLLISION_MATH.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
CONTRIBUTOR_NAMESPACE_ISOLATION.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
DOC_INVENTORY.md docs: loss-check merge legacy docs/ originals into the site pages 2026-07-12 10:37:07 -07:00
ERROR_HANDLING.md docs: port user docs into the Mintlify site tree under docs/ 2026-07-12 10:37:07 -07:00
EXCLUSIVE_LOCK.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
EXTENDING.md feat(storage): opt-in lite SELECT shape (Issue body columns deferred) (#3906) 2026-07-26 15:29:08 -07:00
ICU-POLICY.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
INTEGRATION_CHARTER.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
INTERNALS.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
LINTING.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
messaging.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
PERFORMANCE_TESTING.md docs(engdocs): point storage test and bench references at the Dolt packages 2026-07-18 02:43:57 +00:00
PROJECT_CHARTER.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
README_TESTING.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
RELEASE-STABILITY-GATE.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
RELEASING.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
REPO_CONTEXT.md docs(ci): add Mintlify preview, docsync guard, and broken-links workflow 2026-07-12 10:37:07 -07:00
RULES_AUDIT_DESIGN.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
SECURITY-DEPENDENCY-EXCEPTIONS.md Bump Go to 1.26.5 (#4942) 2026-07-23 18:54:32 -07:00
TESTING.md docs(engdocs): point storage test and bench references at the Dolt packages 2026-07-18 02:43:57 +00:00
TESTING_PHILOSOPHY.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
UI_PHILOSOPHY.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00