diff --git a/CHANGELOG.md b/CHANGELOG.md index 8234c58a9..28bc20404 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1079,6 +1079,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ready set should fail to compile rather than silently fall back to an unbounded query. +- **`storage.Storage` gains a required `SearchIssueSummaries` method** + ([#3458](https://github.com/gastownhall/beads/pull/3458)). It is a + narrow-projection variant of `SearchIssues` for list-shaped rendering: it + returns `[]*types.IssueSummary` (exported as `backend.IssueSummary`), which + carries the list columns, labels, and the four wisp-plane markers but none of + the TEXT/JSON columns. Nothing in `bd` calls it yet, so no command's output + changes. Consumers that only *call* the interface are unaffected; any external + type that *implements* it (a custom store, mock, or proxy, or an + `internal/storage/backends` registrant) must add the method to compile: + `SearchIssueSummaries(ctx context.Context, query string, filter types.IssueFilter) ([]*types.IssueSummary, error)`. + It honors `SortBy`/`SortDesc` and `SkipLabels` exactly as `SearchIssues` + does, `IncludeDependencies` is a silent no-op (a summary has nowhere to put + dependency records), and wisps are merged in unless `SkipWisps` is set, with + their markers set so a wisp row stays distinguishable from a durable issue. + - **Push `--dry-run` now honors `--create-only`** ([#6337](https://github.com/gastownhall/beads/issues/6337)). The sequential tracker push previewed "Would update" for every already-linked issue, even diff --git a/backend/types.go b/backend/types.go index 55eef4f09..65e2590aa 100644 --- a/backend/types.go +++ b/backend/types.go @@ -26,6 +26,7 @@ type ( IssueDetails = types.IssueDetails IssueFilter = types.IssueFilter IssueSnapshot = types.IssueSnapshot + IssueSummary = types.IssueSummary IssueType = types.IssueType IssueWithCounts = types.IssueWithCounts IssueWithDependencyMetadata = types.IssueWithDependencyMetadata diff --git a/engdocs/EXTENDING.md b/engdocs/EXTENDING.md index 7f2222f13..8cfe741eb 100644 --- a/engdocs/EXTENDING.md +++ b/engdocs/EXTENDING.md @@ -66,3 +66,63 @@ with `IsLitePartial == false`. This is correct-but-unoptimized (no lite callers exist yet, so the difference is invisible today); wiring `filter.Lite` through the domain/db stack is deferred to the CLI-wiring follow-up (be-uwvs.2+), not part of this foundation. + +## Summary projection — `store.SearchIssueSummaries` + +`SearchIssueSummaries` is a third projection alongside the full and lite +shapes above. It returns `[]*types.IssueSummary` rather than +`[]*types.Issue`, selecting only `IssueSummaryColumns` — narrower than +`IssueSelectColumnsLite`, because it drops the routing/claim scalars +(`metadata`, `row_lock`, the `leases.*` overlay) a list-shaped render never +reads, not just the heavy TEXT bodies. + +### Contract for callers + +- **MAY** read every field `types.IssueSummary` declares. All of them come + from the row, so there is no `IsLitePartial` equivalent and no partial + state to detect — a field the type does not have is a compile error, not + a silent zero value. That is the difference from Lite: Lite returns a + full-shaped `*types.Issue` with some fields hollowed out, the summary + projection returns a narrower type. +- **MUST** call `store.SearchIssues` instead when dependencies are needed. + `types.IssueSummary` has no `Dependencies` field, so + `IssueFilter.IncludeDependencies` is a silent no-op on this path. + `SkipLabels` is honored exactly as it is on `SearchIssues`. +- **MAY** rely on wisps behaving identically. The issues+wisps merge in + `searchInTx` runs for any filter that does not set `SkipWisps`, and + `types.IssueSummary` carries the four wisp-plane markers (`Ephemeral`, + `NoHistory`, `WispType`, `StorageClass`), so a merged wisp row serializes + the same through either projection. +- **MAY** rely on identical ordering. `SortBy`/`SortDesc` render the same + SQL `ORDER BY`, and the post-merge re-sort uses `sqlbuild.LessSummary`, + which shares one comparator body with `sqlbuild.Less`. + +### Where the contract is enforced + +- Column list and scanner: `internal/storage/issueops/scan.go` + (`IssueSummaryColumns`, `ScanIssueSummaryFrom`). +- SELECT dispatch: `summaryProjection` in + `internal/storage/issueops/search.go` — a + `searchProjection[*types.IssueSummary]` literal on the same + `searchTableInTxT` core as the other two, so WHERE clauses, wisp merge, + dedup, trim and `MaxRows` cannot drift between them. +- Column/scanner guards: + `scan_test.go::TestIssueSummaryColumnsMatchScanner` (count agreement) and + `TestIssueSummaryColumnsAreRealIssueColumns` (every column exists on the + issues/wisps tables). Note `IssueSummaryColumns` is deliberately **not** a + subsequence of `IssueSelectColumns` in order, so it has no analogue of + `TestIssueSelectColumnsLite_IsFullMinusHeavyInOrder`. +- Wire-shape guards: `internal/types/issue_summary_test.go` pins + `IssueSummary`'s JSON tags against `Issue`'s field by field, including the + wisp markers. +- End-to-end parity: `internal/storage/dolt/search_parity_test.go` compares + `SearchIssues` against `SearchIssueSummaries` on a real database, across + filters and sort keys, and field by field for durable and wisp rows. + +### Backend coverage + +No in-tree caller reads `SearchIssueSummaries` yet — this is the storage +foundation, and the `bd list` wiring that spends it is tracked separately. +The proxied-server (`internal/storage/domain/db`) path does not implement +the narrow projection either, for the same reason `filter.Lite` is not +wired through it yet (above). diff --git a/engdocs/design/otel/otel-data-model.md b/engdocs/design/otel/otel-data-model.md index 6545b53d7..1d6f5ec39 100644 --- a/engdocs/design/otel/otel-data-model.md +++ b/engdocs/design/otel/otel-data-model.md @@ -172,7 +172,7 @@ are inventoried here rather than repeated. There are two families. **Direct storage operations** — `storage.`, one per storage method -(73 spans): +(74 spans): ``` AddDependency AddIssueComment AddLabel CloseIssue CloseIssueChecked @@ -190,9 +190,9 @@ IterEvents IterIssueComments IterIssues IterReadyWork IterWisps ListWisps MergeMetadata MergeSlotAcquire MergeSlotCheck MergeSlotCreate MergeSlotRelease RecordProvenanceEvent RemoveDependency RemoveLabel ReopenIssue RunInIssueLifecycleTransaction RunInTransaction SearchIssueIDs SearchIssues -SearchIssuesWithCounts SetConfig SetLocalMetadata SlotClear SlotGet SlotSet -UnclaimIssue UnclaimIssueIfAssignee UpdateIssue UpdateIssueChecked -UpdateIssueType +SearchIssueSummaries SearchIssuesWithCounts SetConfig SetLocalMetadata +SlotClear SlotGet SlotSet UnclaimIssue UnclaimIssueIfAssignee UpdateIssue +UpdateIssueChecked UpdateIssueType ``` **Issueops role operations** — `storage..`, emitted when a caller diff --git a/internal/jira/tracker_test.go b/internal/jira/tracker_test.go index 47aa5a99c..c24a6d94c 100644 --- a/internal/jira/tracker_test.go +++ b/internal/jira/tracker_test.go @@ -769,6 +769,9 @@ func (s *configStore) SearchIssues(_ context.Context, _ string, _ types.IssueFil func (s *configStore) SearchIssueIDs(_ context.Context, _ string, _ types.IssueFilter) ([]string, error) { return nil, nil } +func (s *configStore) SearchIssueSummaries(_ context.Context, _ string, _ types.IssueFilter) ([]*types.IssueSummary, error) { + return nil, nil +} func (s *configStore) AddDependency(_ context.Context, _ *types.Dependency, _ string) error { return nil } diff --git a/internal/storage/dolt/dolt_benchmark_test.go b/internal/storage/dolt/dolt_benchmark_test.go index 90bd5faf7..deefe5786 100644 --- a/internal/storage/dolt/dolt_benchmark_test.go +++ b/internal/storage/dolt/dolt_benchmark_test.go @@ -1382,6 +1382,91 @@ func BenchmarkPerfIDProjectionVsHydration_5K(b *testing.B) { }) } +// BenchmarkPerfSummaryProjectionVsHydration_5K is the three-way wide/lite/ +// summary A/B: SearchIssueSummaries (narrow projection, IssueSummaryColumns) +// vs SearchIssues with Lite:true (IssueSelectColumnsLite, defers 6 heavy TEXT +// columns) vs SearchIssues full (IssueSelectColumns, hydrates everything). +// Mirrors BenchmarkPerfIDProjectionVsHydration_5K's shape; the extra middle +// arm exists because #3906 (Lite) landed after PR #3458's original two-way +// A/B was measured, so summary-vs-full alone would overclaim, some of that +// win is now Lite's. +func BenchmarkPerfSummaryProjectionVsHydration_5K(b *testing.B) { + store, cleanup := setupBenchStore(b) + defer cleanup() + + const total = 5000 + // Heavy TEXT payloads approximate real issues. Neither the summary nor the + // lite projection scans these columns; SkipLabels does not help with them. + bigText := strings.Repeat("lorem ipsum dolor sit amet consectetur ", 24) // ~936 B + bigJSON := json.RawMessage(`{"k":"` + strings.Repeat("v", 256) + `"}`) + + issues := make([]*types.Issue, 0, total) + for i := 0; i < total; i++ { + issues = append(issues, &types.Issue{ + ID: fmt.Sprintf("summarybench-%05d", i), + Title: fmt.Sprintf("summarybench issue %05d", i), + Description: bigText, + Design: bigText, + AcceptanceCriteria: bigText, + Notes: bigText, + Metadata: bigJSON, + Status: types.StatusOpen, + Priority: (i % 4) + 1, + IssueType: types.TypeTask, + Labels: []string{"area-summary", fmt.Sprintf("bucket-%03d", i%100)}, + }) + } + createBenchIssueBatch(b, store, issues) + + ctx := context.Background() + // "summarybench" appears in every id and title, so id/title LIKE matches + // the full set — maximizing per-row hydration cost, the dimension under + // test. + const query = "summarybench" + + // Guard: all three arms must see the same cardinality, else the + // comparison would be measuring row count rather than projection width. + if ids, err := store.SearchIssueIDs(ctx, query, types.IssueFilter{}); err != nil { + b.Fatalf("setup SearchIssueIDs: %v", err) + } else if len(ids) != total { + b.Fatalf("fixture: expected %d matches, got %d", total, len(ids)) + } + + b.Run("SearchIssueSummaries_narrow", func(b *testing.B) { + b.ReportAllocs() + for i := 0; i < b.N; i++ { + if got, err := store.SearchIssueSummaries(ctx, query, types.IssueFilter{}); err != nil { + b.Fatalf("SearchIssueSummaries: %v", err) + } else if len(got) != total { + b.Fatalf("SearchIssueSummaries: got %d want %d", len(got), total) + } + } + }) + + b.Run("SearchIssues_Lite", func(b *testing.B) { + b.ReportAllocs() + filter := types.IssueFilter{Lite: true} + for i := 0; i < b.N; i++ { + if got, err := store.SearchIssues(ctx, query, filter); err != nil { + b.Fatalf("SearchIssues Lite: %v", err) + } else if len(got) != total { + b.Fatalf("SearchIssues Lite: got %d want %d", len(got), total) + } + } + }) + + b.Run("SearchIssues_full", func(b *testing.B) { + b.ReportAllocs() + for i := 0; i < b.N; i++ { + if got, err := store.SearchIssues(ctx, query, types.IssueFilter{}); err != nil { + b.Fatalf("SearchIssues full: %v", err) + } else if len(got) != total { + b.Fatalf("SearchIssues full: got %d want %d", len(got), total) + } + } + }) +} + func BenchmarkPerfAddDependencyCycleCheck_DiamondDAG(b *testing.B) { store, cleanup := setupBenchStore(b) defer cleanup() diff --git a/internal/storage/dolt/queries.go b/internal/storage/dolt/queries.go index 344325b7a..38a57bff2 100644 --- a/internal/storage/dolt/queries.go +++ b/internal/storage/dolt/queries.go @@ -37,6 +37,18 @@ func (s *DoltStore) SearchIssueIDs(ctx context.Context, query string, filter typ return result, err } +// SearchIssueSummaries is the narrow-projection variant of SearchIssues; +// returns []*types.IssueSummary instead of full issues. +func (s *DoltStore) SearchIssueSummaries(ctx context.Context, query string, filter types.IssueFilter) ([]*types.IssueSummary, error) { + var result []*types.IssueSummary + err := s.withReadTx(ctx, func(tx *sql.Tx) error { + var err error + result, err = issueops.SearchIssueSummariesInTx(ctx, tx, query, filter) + return err + }) + return result, err +} + func (s *DoltStore) SearchIssuesWithCounts(ctx context.Context, query string, filter types.IssueFilter) ([]*types.IssueWithCounts, error) { var result []*types.IssueWithCounts err := s.withReadTx(ctx, func(tx *sql.Tx) error { diff --git a/internal/storage/dolt/search_parity_test.go b/internal/storage/dolt/search_parity_test.go index 08f4b0a5e..9b28eb422 100644 --- a/internal/storage/dolt/search_parity_test.go +++ b/internal/storage/dolt/search_parity_test.go @@ -2,6 +2,7 @@ package dolt import ( "context" + "slices" "sort" "testing" @@ -66,6 +67,249 @@ func TestSearchIssuesAndSearchIssueIDs_Parity(t *testing.T) { } } +// TestSearchIssuesAndSearchIssueSummaries_Parity asserts that SearchIssues and +// SearchIssueSummaries return the same ID set, in the same order, across the +// same representative filters TestSearchIssuesAndSearchIssueIDs_Parity checks +// above. SearchIssueSummaries shares the same generic searchInTx core +// (summaryProjection, internal/storage/issueops/search.go) as SearchIssues and +// SearchIssueIDs; this is that contract's third leg. +func TestSearchIssuesAndSearchIssueSummaries_Parity(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + seedSearchParityFixture(ctx, t, store) + + cases := []struct { + name string + query string + filter types.IssueFilter + }{ + {name: "no filter, full set"}, + {name: "open status only", filter: types.IssueFilter{Statuses: []types.Status{types.StatusOpen}}}, + {name: "priority filter", filter: types.IssueFilter{Priority: ptr(1)}}, + {name: "substring search (id_parser fallback)", query: "search-parity"}, + {name: "substring search no match", query: "no-such-prefix-anywhere"}, + {name: "label-driven (DISTINCT join)", filter: types.IssueFilter{Labels: []string{"alpha"}}}, + {name: "ephemeral only", filter: types.IssueFilter{Ephemeral: ptr(true)}}, + {name: "non-ephemeral only", filter: types.IssueFilter{Ephemeral: ptr(false)}}, + {name: "limit applied", filter: types.IssueFilter{Limit: 2}}, + // SortBy-varying rows. Without these the SQL ORDER BY that + // sqlbuild.LessSummary mirrors is only ever exercised at its default + // (priority ASC), so LessSummary-vs-SQL agreement on every other key + // is proven by unit test alone (TestLessSummaryMatchesLessAcrossSortKeys) + // and never against a real database. Each key is covered in both + // directions, and each is paired with a Limit so the merge's + // sort-before-trim actually decides which rows survive. + {name: "sort created asc + limit", filter: types.IssueFilter{SortBy: "created", Limit: 3}}, + {name: "sort created desc + limit", filter: types.IssueFilter{SortBy: "created", SortDesc: true, Limit: 3}}, + {name: "sort title asc + limit", filter: types.IssueFilter{SortBy: "title", Limit: 3}}, + {name: "sort title desc + limit", filter: types.IssueFilter{SortBy: "title", SortDesc: true, Limit: 3}}, + {name: "sort status asc + limit", filter: types.IssueFilter{SortBy: "status", Limit: 3}}, + {name: "sort updated desc + limit", filter: types.IssueFilter{SortBy: "updated", SortDesc: true, Limit: 3}}, + {name: "sort assignee asc + limit", filter: types.IssueFilter{SortBy: "assignee", Limit: 3}}, + // "id" is the one Go-side key (sqlbuild.IsGoSideSort): SQL emits no + // ORDER BY for it, so this row pins that both paths still agree on the + // order the merge comparator produces. + {name: "sort id asc + limit", filter: types.IssueFilter{SortBy: "id", Limit: 3}}, + {name: "sort id desc + limit", filter: types.IssueFilter{SortBy: "id", SortDesc: true, Limit: 3}}, + } + + for _, tc := range cases { + tc := tc + t.Run(tc.name, func(t *testing.T) { + issues, err := store.SearchIssues(ctx, tc.query, tc.filter) + if err != nil { + t.Fatalf("SearchIssues: %v", err) + } + summaries, err := store.SearchIssueSummaries(ctx, tc.query, tc.filter) + if err != nil { + t.Fatalf("SearchIssueSummaries: %v", err) + } + + fromIssues := make([]string, len(issues)) + for i, issue := range issues { + fromIssues[i] = issue.ID + } + fromSummaries := make([]string, len(summaries)) + for i, s := range summaries { + fromSummaries[i] = s.ID + } + + // Order-sensitive: both paths share the same ORDER BY, so unlike + // equalStringSlices (which sorts before comparing and only proves + // the same ID set), this must also catch drift in relative row order. + if !slices.Equal(fromIssues, fromSummaries) { + t.Errorf("SearchIssues vs SearchIssueSummaries disagree:\n SearchIssues: %v\n SearchIssueSummaries: %v", + fromIssues, fromSummaries) + } + }) + } +} + +// TestSearchIssueSummaries_FieldsMatchSearchIssues asserts that every field +// types.IssueSummary carries agrees with the corresponding types.Issue field +// for the same row, including the nullable/derived fields most likely to +// silently drift from ScanIssueFrom if ScanIssueSummaryFrom's column list or +// scan order falls out of sync: Pinned, Labels, Assignee, and ClosedAt. +// +// The fixture seeds wisps as well as durable beads, because the wisp-plane +// markers are the fields with the most room to drift: they are read from a +// second table (searchInTx merges wisps into every non-SkipWisps result) and +// they are the newest columns in IssueSummaryColumns. A durable-only fixture +// would leave all four at their zero values and pass whether or not the +// summary path reads them at all. +func TestSearchIssueSummaries_FieldsMatchSearchIssues(t *testing.T) { + store, cleanup := setupTestStore(t) + defer cleanup() + + ctx, cancel := testContext(t) + defer cancel() + + pinned := &types.Issue{ + ID: "search-summary-pinned", + Title: "pinned reference issue", + Status: types.StatusOpen, + Priority: 1, + IssueType: types.TypeTask, + Assignee: "alice", + Pinned: true, + } + if err := store.CreateIssue(ctx, pinned, "tester"); err != nil { + t.Fatalf("CreateIssue (pinned): %v", err) + } + if err := store.AddLabel(ctx, pinned.ID, "needs-summary-field", "tester"); err != nil { + t.Fatalf("AddLabel: %v", err) + } + + const closedID = "search-summary-closed" + closedIssue := &types.Issue{ + ID: closedID, + Title: "closed reference issue", + Status: types.StatusOpen, + Priority: 2, + IssueType: types.TypeBug, + } + if err := store.CreateIssue(ctx, closedIssue, "tester"); err != nil { + t.Fatalf("CreateIssue (closed): %v", err) + } + if err := store.CloseIssue(ctx, closedID, "done", "tester", "s1"); err != nil { + t.Fatalf("CloseIssue: %v", err) + } + + // Wisp-plane rows. Ephemeral and NoHistory are mutually exclusive + // (types.Issue.Validate), so they need one row each; both carry a WispType + // so a dropped wisp_type column cannot pass as an empty-string match. + ephemeralWisp := &types.Issue{ + Title: "ephemeral summary wisp", + Status: types.StatusOpen, + Priority: 2, + IssueType: types.TypeTask, + Ephemeral: true, + WispType: types.WispTypeHeartbeat, + } + if err := store.CreateIssue(ctx, ephemeralWisp, "tester"); err != nil { + t.Fatalf("CreateIssue (ephemeral wisp): %v", err) + } + noHistoryWisp := &types.Issue{ + Title: "no-history summary wisp", + Status: types.StatusOpen, + Priority: 3, + IssueType: types.TypeTask, + NoHistory: true, + WispType: types.WispTypeEscalation, + } + if err := store.CreateIssue(ctx, noHistoryWisp, "tester"); err != nil { + t.Fatalf("CreateIssue (no-history wisp): %v", err) + } + + issues, err := store.SearchIssues(ctx, "", types.IssueFilter{}) + if err != nil { + t.Fatalf("SearchIssues: %v", err) + } + summaries, err := store.SearchIssueSummaries(ctx, "", types.IssueFilter{}) + if err != nil { + t.Fatalf("SearchIssueSummaries: %v", err) + } + if len(summaries) != len(issues) { + t.Fatalf("SearchIssueSummaries returned %d rows, SearchIssues returned %d", len(summaries), len(issues)) + } + + byID := make(map[string]*types.Issue, len(issues)) + for _, issue := range issues { + byID[issue.ID] = issue + } + + // Guard against a vacuous pass: if the wisp merge stopped returning wisps, + // every marker assertion below would compare zero to zero and succeed. + wispRows := 0 + for _, s := range summaries { + if s.Ephemeral || s.NoHistory { + wispRows++ + } + } + if wispRows < 2 { + t.Fatalf("expected both seeded wisps in the summary result, got %d wisp rows out of %d; "+ + "the marker assertions below would be vacuous", wispRows, len(summaries)) + } + + for _, s := range summaries { + issue, ok := byID[s.ID] + if !ok { + t.Fatalf("SearchIssueSummaries returned %q, not present in SearchIssues result", s.ID) + } + if s.Title != issue.Title { + t.Errorf("%s: Title = %q, want %q", s.ID, s.Title, issue.Title) + } + if s.Status != issue.Status { + t.Errorf("%s: Status = %q, want %q", s.ID, s.Status, issue.Status) + } + if s.Priority != issue.Priority { + t.Errorf("%s: Priority = %d, want %d", s.ID, s.Priority, issue.Priority) + } + if s.IssueType != issue.IssueType { + t.Errorf("%s: IssueType = %q, want %q", s.ID, s.IssueType, issue.IssueType) + } + if s.Assignee != issue.Assignee { + t.Errorf("%s: Assignee = %q, want %q", s.ID, s.Assignee, issue.Assignee) + } + if s.Pinned != issue.Pinned { + t.Errorf("%s: Pinned = %v, want %v", s.ID, s.Pinned, issue.Pinned) + } + if !equalStringSlices(s.Labels, issue.Labels) { + t.Errorf("%s: Labels = %v, want %v", s.ID, s.Labels, issue.Labels) + } + if !s.CreatedAt.Equal(issue.CreatedAt) { + t.Errorf("%s: CreatedAt = %v, want %v", s.ID, s.CreatedAt, issue.CreatedAt) + } + if !s.UpdatedAt.Equal(issue.UpdatedAt) { + t.Errorf("%s: UpdatedAt = %v, want %v", s.ID, s.UpdatedAt, issue.UpdatedAt) + } + switch { + case s.ClosedAt == nil && issue.ClosedAt == nil: + // both nil: fine + case s.ClosedAt == nil || issue.ClosedAt == nil: + t.Errorf("%s: ClosedAt nil-ness mismatch: summary=%v issue=%v", s.ID, s.ClosedAt, issue.ClosedAt) + case !s.ClosedAt.Equal(*issue.ClosedAt): + t.Errorf("%s: ClosedAt = %v, want %v", s.ID, *s.ClosedAt, *issue.ClosedAt) + } + if s.Ephemeral != issue.Ephemeral { + t.Errorf("%s: Ephemeral = %v, want %v", s.ID, s.Ephemeral, issue.Ephemeral) + } + if s.NoHistory != issue.NoHistory { + t.Errorf("%s: NoHistory = %v, want %v", s.ID, s.NoHistory, issue.NoHistory) + } + if s.WispType != issue.WispType { + t.Errorf("%s: WispType = %q, want %q", s.ID, s.WispType, issue.WispType) + } + if s.StorageClass != issue.StorageClass { + t.Errorf("%s: StorageClass = %q, want %q", s.ID, s.StorageClass, issue.StorageClass) + } + } +} + func seedSearchParityFixture(ctx context.Context, t *testing.T, store *DoltStore) { t.Helper() diff --git a/internal/storage/embeddeddolt/list_queries.go b/internal/storage/embeddeddolt/list_queries.go index a165b074e..c801aaeb8 100644 --- a/internal/storage/embeddeddolt/list_queries.go +++ b/internal/storage/embeddeddolt/list_queries.go @@ -31,6 +31,18 @@ func (s *EmbeddedDoltStore) SearchIssueIDs(ctx context.Context, query string, fi return result, err } +// SearchIssueSummaries is the narrow-projection variant of SearchIssues; +// returns []*types.IssueSummary instead of full issues. +func (s *EmbeddedDoltStore) SearchIssueSummaries(ctx context.Context, query string, filter types.IssueFilter) ([]*types.IssueSummary, error) { + var result []*types.IssueSummary + err := s.withConn(ctx, false, func(tx *sql.Tx) error { + var err error + result, err = issueops.SearchIssueSummariesInTx(ctx, tx, query, filter) + return err + }) + return result, err +} + func (s *EmbeddedDoltStore) SearchIssuesWithCounts(ctx context.Context, query string, filter types.IssueFilter) ([]*types.IssueWithCounts, error) { var result []*types.IssueWithCounts err := s.withConn(ctx, false, func(tx *sql.Tx) error { diff --git a/internal/storage/issueops/BUILD.bazel b/internal/storage/issueops/BUILD.bazel index 1efa3a7be..331484351 100644 --- a/internal/storage/issueops/BUILD.bazel +++ b/internal/storage/issueops/BUILD.bazel @@ -173,6 +173,7 @@ go_test( "search_goside_sort_test.go", "search_lite_merge_test.go", "search_missing_table_test.go", + "search_summary_merge_test.go", "search_test.go", "tree_walk_test.go", "update_closed_fields_test.go", diff --git a/internal/storage/issueops/scan.go b/internal/storage/issueops/scan.go index fb52fa9da..c7f81f8cc 100644 --- a/internal/storage/issueops/scan.go +++ b/internal/storage/issueops/scan.go @@ -57,6 +57,28 @@ var HeavyDropList = []string{ "payload", } +// IssueSummaryColumns is the column list for types.IssueSummary hydration — +// the narrowest projection, used by list-shaped rendering paths that never +// dereference TEXT/JSON columns. Labels are hydrated separately (they live +// in a join table, not a column); see hydrateSummaryLabels in search.go. +// ScanIssueSummaryFrom below scans it positionally and must stay in agreement; +// TestIssueSummaryColumnsMatchScanner pins that agreement by count. +// +// The trailing four are the wisp-plane markers. searchInTx merges the wisps +// table into every non-SkipWisps result, so they are load-bearing rather than +// optional: without them a wisp row and a durable one are indistinguishable in +// `bd list --json`. All four are narrow scalars (TINYINT(1)/short VARCHAR) +// present on both the issues and wisps tables, so they cost none of the +// TEXT/JSON hydration this projection exists to skip. +// +// Unlike IssueSelectColumnsLite, this list is deliberately NOT a subsequence +// of IssueSelectColumns in order (pinned sits beside the other scalars here, +// not between wisp_type and is_template); the two are scanned by different +// functions, so only each list's own agreement with its scanner matters. +const IssueSummaryColumns = `id, title, status, priority, issue_type, assignee, + pinned, created_at, updated_at, closed_at, + ephemeral, no_history, wisp_type, storage_class` + // IssueScanner is the common interface between *sql.Row and *sql.Rows, // allowing a single scan function to work with both single-row and // multi-row query results. @@ -415,6 +437,63 @@ func ScanIssueLiteFrom(s IssueScanner, extra ...any) (*types.Issue, error) { return &issue, nil } +// ScanIssueSummaryFrom scans a narrow issue summary from any source +// implementing IssueScanner. The caller must ensure the query selected +// exactly IssueSummaryColumns in order. Labels are not populated here — the +// caller must hydrate them separately (see hydrateSummaryLabels in search.go). +func ScanIssueSummaryFrom(s IssueScanner) (*types.IssueSummary, error) { + var summary types.IssueSummary + var assignee sql.NullString + var pinned sql.NullInt64 + var createdAtStr, updatedAtStr sql.NullString + var closedAt sql.NullTime + // Wisp-plane markers. Mapped exactly as ScanIssueFrom maps the same four + // columns, so a row read through either scanner carries identical values: + // the TINYINT(1) pair is truthy-on-nonzero, wisp_type takes any non-NULL + // string, and a NULL storage_class stays empty for EffectiveStorageClass to + // resolve per plane rather than being pinned to a literal here. + var ephemeral, noHistory sql.NullInt64 + var wispType, storageClass sql.NullString + + if err := s.Scan( + &summary.ID, &summary.Title, &summary.Status, &summary.Priority, &summary.IssueType, + &assignee, &pinned, &createdAtStr, &updatedAtStr, &closedAt, + &ephemeral, &noHistory, &wispType, &storageClass, + ); err != nil { + return nil, err + } + + if assignee.Valid { + summary.Assignee = assignee.String + } + if pinned.Valid && pinned.Int64 != 0 { + summary.Pinned = true + } + if createdAtStr.Valid { + summary.CreatedAt = ParseTimeString(createdAtStr.String) + } + if updatedAtStr.Valid { + summary.UpdatedAt = ParseTimeString(updatedAtStr.String) + } + if closedAt.Valid { + summary.ClosedAt = &closedAt.Time + } + if ephemeral.Valid && ephemeral.Int64 != 0 { + summary.Ephemeral = true + } + if noHistory.Valid && noHistory.Int64 != 0 { + summary.NoHistory = true + } + if wispType.Valid { + summary.WispType = types.WispType(wispType.String) + } + if storageClass.Valid { + summary.StorageClass = types.StorageClass(storageClass.String) + } + + return &summary, nil +} + // ParseTimeString parses a time string from database TEXT columns (non-nullable). // Supports RFC3339Nano, RFC3339, and MySQL DATETIME format. func ParseTimeString(s string) time.Time { diff --git a/internal/storage/issueops/scan_test.go b/internal/storage/issueops/scan_test.go index 11081087a..3f2b179ca 100644 --- a/internal/storage/issueops/scan_test.go +++ b/internal/storage/issueops/scan_test.go @@ -230,3 +230,65 @@ func TestScanIssueFrom_PopulatesHeavyFields(t *testing.T) { t.Errorf("ScanIssueFrom: Waiters = %v, want [\"a\"]", issue.Waiters) } } + +// countingScanner records how many destination pointers a scan function asks +// for, without needing a database. Scan itself is a no-op, so the caller's +// post-scan mapping runs against zero-valued dests — harmless, and irrelevant +// to the count this fixture exists to observe. +type countingScanner struct{ dests int } + +func (c *countingScanner) Scan(dest ...any) error { + c.dests = len(dest) + return nil +} + +// TestIssueSummaryColumnsMatchScanner ties IssueSummaryColumns to +// ScanIssueSummaryFrom's dest list by count. The two are hand-synced and read +// positionally, so a column added to one and not the other does not fail +// loudly: the SELECT and the Scan disagree on width, and database/sql reports +// it as a generic scan error from whichever query happens to run first, far +// from the edit that caused it. +// +// This is the summary-projection counterpart of +// TestIssueSelectColumns_LitePlusHeavyEqualsFull, which does the same job for +// the lite projection. +func TestIssueSummaryColumnsMatchScanner(t *testing.T) { + t.Parallel() + + cols := parseSelectColumns(IssueSummaryColumns) + + var scanner countingScanner + if _, err := ScanIssueSummaryFrom(&scanner); err != nil { + t.Fatalf("ScanIssueSummaryFrom with counting scanner: %v", err) + } + + if scanner.dests != len(cols) { + t.Errorf("IssueSummaryColumns has %d columns but ScanIssueSummaryFrom scans %d dests.\n"+ + " columns: %v\n"+ + "Action: add the column to both, in the same position — they are read positionally.", + len(cols), scanner.dests, cols) + } +} + +// TestIssueSummaryColumnsAreRealIssueColumns pins every column in +// IssueSummaryColumns to one that actually exists on the issues and wisps +// tables, by requiring it to appear in IssueSelectColumns — the canonical full +// list both tables are already queried with. +// +// Count parity alone cannot catch a typo: misspelling a column while adding +// its dest keeps the widths equal and fails only at query time, against a real +// database, as an opaque "unknown column" from whichever caller runs first. +// Note this is a membership check, not an order check — unlike +// IssueSelectColumnsLite, IssueSummaryColumns is deliberately not a +// subsequence of the full list (see the const's doc comment). +func TestIssueSummaryColumnsAreRealIssueColumns(t *testing.T) { + t.Parallel() + + full := columnSet(parseSelectColumns(IssueSelectColumns)) + for _, col := range parseSelectColumns(IssueSummaryColumns) { + if _, ok := full[col]; !ok { + t.Errorf("IssueSummaryColumns contains %q, which is not in IssueSelectColumns; "+ + "either it is misspelled or it is not a column on the issues/wisps tables", col) + } + } +} diff --git a/internal/storage/issueops/search.go b/internal/storage/issueops/search.go index df0ff5b95..4a0b10438 100644 --- a/internal/storage/issueops/search.go +++ b/internal/storage/issueops/search.go @@ -32,6 +32,15 @@ func SearchIssueIDsInTx(ctx context.Context, tx DBTX, query string, filter types return searchInTx(ctx, tx, query, filter, idProjection) } +// SearchIssueSummariesInTx is the narrow-projection variant of +// SearchIssuesInTx for list-shaped rendering paths: applies the same WHERE +// clauses, wisp-merge semantics, and sort order as SearchIssuesInTx, but +// projects only the columns in IssueSummaryColumns (plus a separate label +// hydration pass) and returns []*types.IssueSummary instead of full issues. +func SearchIssueSummariesInTx(ctx context.Context, tx DBTX, query string, filter types.IssueFilter) ([]*types.IssueSummary, error) { + return searchInTx(ctx, tx, query, filter, summaryProjection) +} + // SearchWispsPlaneInTx searches the wisps plane ALONE: every row stored in // the wisps table, whatever its ephemeral, no_history or wisp_type values, and // never the issues table. @@ -149,6 +158,21 @@ var idProjection = searchProjection[string]{ }, } +// summaryProjection is the narrow list-rendering variant: reads +// IssueSummaryColumns instead of the wide or lite column sets, scans with +// ScanIssueSummaryFrom, and hydrates only labels (hydrateSummaryLabels) since +// types.IssueSummary carries no Dependencies field to hydrate. idShrink stays +// false (the zero value) — IssueSummaryColumns is already narrower than +// IssueSelectColumnsLite, so Pattern B's extra id-scan round-trip isn't a +// clear win here the way it is for the wide issueProjection. +var summaryProjection = searchProjection[*types.IssueSummary]{ + columns: func(_ FilterTables) string { return IssueSummaryColumns }, + scan: func(rows *sql.Rows) (*types.IssueSummary, error) { return ScanIssueSummaryFrom(rows) }, + id: func(s *types.IssueSummary) string { return s.ID }, + hydrate: hydrateSummaryLabels, + less: sqlbuild.LessSummary, +} + // hydrateIssueLabelsAndDeps bulk-loads labels (and optionally dependencies) // for the given issues. searchTableInTxT runs against exactly one of the // issues/wisps tables, so every ID here belongs to tables.Labels — we use @@ -195,6 +219,29 @@ func missingOptionalWispTable(err error) bool { return ok && sqlbuild.OptionalWispTable(name) } +// hydrateSummaryLabels bulk-loads labels for the given summaries. Mirrors the +// label half of hydrateIssueLabelsAndDeps; types.IssueSummary has no +// Dependencies field, so there is no dependency-hydration half to mirror. +func hydrateSummaryLabels(ctx context.Context, tx DBTX, tables FilterTables, summaries []*types.IssueSummary, filter types.IssueFilter) error { + if filter.SkipLabels { + return nil + } + ids := make([]string, len(summaries)) + for i, s := range summaries { + ids[i] = s.ID + } + labelMap, err := GetLabelsForIssuesFromTableInTx(ctx, tx, tables.Labels, ids) + if err != nil { + return fmt.Errorf("hydrate summary labels: %w", err) + } + for _, s := range summaries { + if labels, ok := labelMap[s.ID]; ok { + s.Labels = labels + } + } + return nil +} + // searchInTx is the shared wisp-merge wrapper. Ephemeral routing, the // empty-wisps probe, the issues+wisps queries, and overlap detection live // here once. Both SearchIssuesInTx and SearchIssueIDsInTx use this body — diff --git a/internal/storage/issueops/search_summary_merge_test.go b/internal/storage/issueops/search_summary_merge_test.go new file mode 100644 index 000000000..5f23c686c --- /dev/null +++ b/internal/storage/issueops/search_summary_merge_test.go @@ -0,0 +1,111 @@ +package issueops + +import ( + "context" + "database/sql/driver" + "testing" + "time" + + "github.com/DATA-DOG/go-sqlmock" + "github.com/steveyegge/beads/internal/types" +) + +// summaryColumnNames is derived from IssueSummaryColumns rather than +// hand-listed beside it. The hand-copied version was one of three lists that +// had to be edited in lockstep (the const, ScanIssueSummaryFrom's dests, and +// this fixture); deriving it deletes one of the three outright, and +// TestIssueSummaryColumnsMatchScanner in scan_test.go pins the remaining pair. +func summaryColumnNames() []string { + return parseSelectColumns(IssueSummaryColumns) +} + +// summaryMergeRow builds one full-width IssueSummaryColumns row with only +// identity/status/priority/created_at populated; every other column, including +// the wisp-plane markers, is NULL — this fixture exercises merge ordering, not +// hydration. +// +// Values are placed by column NAME against summaryColumnNames rather than by +// literal position, so adding a column to IssueSummaryColumns widens this row +// automatically instead of silently shifting every value one slot left. +func summaryMergeRow(id string, createdAt time.Time) []driver.Value { + byName := map[string]driver.Value{ + "id": id, + "title": id, + "status": "open", + "priority": 2, + "issue_type": "task", + "created_at": createdAt.Format(time.RFC3339), + } + cols := summaryColumnNames() + row := make([]driver.Value, len(cols)) + for i, col := range cols { + row[i] = byName[col] // absent from the map => nil => SQL NULL + } + return row +} + +// TestSearchIssueSummariesInTx_MergesAcrossIssuesAndWisps is +// TestSearchIssuesInTx_Lite_MergesAcrossIssuesAndWisps's sibling for +// summaryProjection (SearchIssueSummariesInTx): coverage for be-be3pj +// acceptance criterion 3 — wisp-merge behavior under a limit (top-N +// truncation after merge) was previously exercised only for issueProjection +// and issueLiteProjection, never for summaryProjection. summaryProjection +// already carries less: sqlbuild.LessSummary (unlike issueLiteProjection +// before be-yrtwi's fix), so this test is prophylactic: it locks in the +// currently-correct merge+trim behavior rather than reproducing a live bug. +// +// Fixture: the issues table's 2 rows are both older than either wisps-table +// row. A Limit=2 search over IssueFilter{SortBy: "created"} must return the +// true top-2 by created_at DESC — both wisp rows — not the issues-leg rows +// that would survive a naive concatenate-then-trim (always the issues leg, +// since results := append(filtered, wispResults...)). SkipLabels avoids +// mocking the per-leg label hydration queries. +func TestSearchIssueSummariesInTx_MergesAcrossIssuesAndWisps(t *testing.T) { + t.Parallel() + + day := func(d int) time.Time { return time.Date(2026, 6, d, 0, 0, 0, 0, time.UTC) } + + _, mock, tx := beginMockTx(t) + + // Issues leg: SQL already orders these DESC by created_at (day5, day1) — + // both older than anything in the wisps leg below. + mock.ExpectQuery(`(?s)FROM issues.*LIMIT 2`). + WillReturnRows(sqlmock.NewRows(summaryColumnNames()). + AddRow(summaryMergeRow("bd-issue-mid", day(5))...). + AddRow(summaryMergeRow("bd-issue-old", day(1))...)) + + mock.ExpectQuery(`SELECT 1 FROM wisps LIMIT 1`). + WillReturnRows(sqlmock.NewRows([]string{"1"}).AddRow(1)) + + // Wisps leg: SQL already orders these DESC by created_at (day12, day10) — + // both newer than either issues-leg row. True global top-2 by created + // DESC is entirely this leg. + mock.ExpectQuery(`(?s)FROM wisps.*LIMIT 2`). + WillReturnRows(sqlmock.NewRows(summaryColumnNames()). + AddRow(summaryMergeRow("bd-wisp-newest", day(12))...). + AddRow(summaryMergeRow("bd-wisp-new", day(10))...)) + + filter := types.IssueFilter{ + SortBy: "created", + Limit: 2, + SkipLabels: true, + } + + got, err := SearchIssueSummariesInTx(context.Background(), tx, "", filter) + if err != nil { + t.Fatalf("SearchIssueSummariesInTx: %v", err) + } + + if len(got) != 2 || got[0].ID != "bd-wisp-newest" || got[1].ID != "bd-wisp-new" { + ids := make([]string, len(got)) + for i, g := range got { + ids[i] = g.ID + } + t.Fatalf("SearchIssueSummariesInTx merge: got %v, want [bd-wisp-newest bd-wisp-new] "+ + "(true created-DESC top-2 across issues+wisps, not issues-leg concatenation order)", ids) + } + + if err := mock.ExpectationsWereMet(); err != nil { + t.Fatalf("unmet SQL expectations: %v", err) + } +} diff --git a/internal/storage/sqlbuild/sort.go b/internal/storage/sqlbuild/sort.go index 7f233976c..50ab7182a 100644 --- a/internal/storage/sqlbuild/sort.go +++ b/internal/storage/sqlbuild/sort.go @@ -113,16 +113,84 @@ func OrderBy(sortBy string, sortDesc bool, table string) string { }) } +// sortFields holds exactly the columns Less/LessSummary compare. Sharing one +// comparator core (lessFields/sortKeyCompareFields) between types.Issue and +// types.IssueSummary means the correctness-critical NULL-first and tie-break +// rules are written once instead of drifting across two hand-copied switch +// statements. +type sortFields struct { + id string + priority int + createdAt time.Time + updatedAt time.Time + closedAt *time.Time + status string + issueType string + assignee string + title string +} + +func issueSortFields(i *types.Issue) sortFields { + return sortFields{ + id: i.ID, + priority: i.Priority, + createdAt: i.CreatedAt, + updatedAt: i.UpdatedAt, + closedAt: i.ClosedAt, + status: string(i.Status), + issueType: string(i.IssueType), + assignee: i.Assignee, + title: i.Title, + } +} + +func summarySortFields(s *types.IssueSummary) sortFields { + return sortFields{ + id: s.ID, + priority: s.Priority, + createdAt: s.CreatedAt, + updatedAt: s.UpdatedAt, + closedAt: s.ClosedAt, + status: string(s.Status), + issueType: string(s.IssueType), + assignee: s.Assignee, + title: s.Title, + } +} + // Less is the Go-side mirror of OrderBy for merge sorts over rows fetched // from separate queries (issues + wisps). It must order exactly the way the // SQL does, including NULL-first ascending semantics for nullable columns; // otherwise a post-merge limit cut keeps a different row set than SQL // selected. func Less(a, b *types.Issue, sortBy string, sortDesc bool) bool { + af, bf := issueSortFields(a), issueSortFields(b) + return lessFields(&af, &bf, sortBy, sortDesc) +} + +// LessSummary is Less's sibling for types.IssueSummary, used by the +// summaryProjection merge sort (internal/storage/issueops.searchInTx). It +// shares lessFields with Less so the two can never drift on the NULL/tie-break +// rules that make a post-merge limit cut safe. +func LessSummary(a, b *types.IssueSummary, sortBy string, sortDesc bool) bool { + af, bf := summarySortFields(a), summarySortFields(b) + return lessFields(&af, &bf, sortBy, sortDesc) +} + +// lessFields takes its operands by pointer, not by value. sortFields is a +// ~150-byte struct and this runs O(n log n) times inside the issues+wisps +// merge sort, so passing it by value copied it twice here and twice more into +// sortKeyCompareFields on every comparison. The adapters above still build one +// value per row — that normalization is what buys Less and LessSummary a +// single shared body — but nothing copies it again after that. The locals do +// not escape, so they stay on the stack and this adds no allocation. +func lessFields(a, b *sortFields, sortBy string, sortDesc bool) bool { if sortBy == "id" { // This key used to ignore sortDesc, so a reversed id merge kept the // byte-FIRST rows (the sibling bug idSrcPage.sortGoSide's doc named). - return LessID(a.ID, b.ID, sortDesc) + // Living in the shared body means LessSummary honors sortDesc too, + // rather than needing its own copy of the same fix. + return LessID(a.id, b.id, sortDesc) } def, ok := SortDefs[sortBy] if !ok { @@ -133,46 +201,46 @@ func Less(a, b *types.Issue, sortBy string, sortDesc bool) bool { if sortDesc { descending = !descending } - if c := sortKeyCompare(a, b, sortBy); c != 0 { + if c := sortKeyCompareFields(a, b, sortBy); c != 0 { if descending { return c > 0 } return c < 0 } - if (sortBy == "" || sortBy == "priority") && !a.CreatedAt.Equal(b.CreatedAt) { - return a.CreatedAt.After(b.CreatedAt) + if (sortBy == "" || sortBy == "priority") && !a.createdAt.Equal(b.createdAt) { + return a.createdAt.After(b.createdAt) } - return a.ID < b.ID + return a.id < b.id } -// sortKeyCompare three-way compares the primary sort column in ascending -// order, with MySQL NULL-first semantics for nullable columns. -func sortKeyCompare(a, b *types.Issue, sortBy string) int { +// sortKeyCompareFields three-way compares the primary sort column in +// ascending order, with MySQL NULL-first semantics for nullable columns. +func sortKeyCompareFields(a, b *sortFields, sortBy string) int { switch sortBy { case "created": - return compareTimesAsc(a.CreatedAt, b.CreatedAt) + return compareTimesAsc(a.createdAt, b.createdAt) case "updated": - return compareTimesAsc(a.UpdatedAt, b.UpdatedAt) + return compareTimesAsc(a.updatedAt, b.updatedAt) case "closed": switch { - case a.ClosedAt == nil && b.ClosedAt == nil: + case a.closedAt == nil && b.closedAt == nil: return 0 - case a.ClosedAt == nil: + case a.closedAt == nil: return -1 - case b.ClosedAt == nil: + case b.closedAt == nil: return 1 } - return compareTimesAsc(*a.ClosedAt, *b.ClosedAt) + return compareTimesAsc(*a.closedAt, *b.closedAt) case "status": - return strings.Compare(string(a.Status), string(b.Status)) + return strings.Compare(a.status, b.status) case "type": - return strings.Compare(string(a.IssueType), string(b.IssueType)) + return strings.Compare(a.issueType, b.issueType) case "assignee": - return strings.Compare(a.Assignee, b.Assignee) + return strings.Compare(a.assignee, b.assignee) case "title": - return strings.Compare(strings.ToLower(a.Title), strings.ToLower(b.Title)) + return strings.Compare(strings.ToLower(a.title), strings.ToLower(b.title)) } - return a.Priority - b.Priority + return a.priority - b.priority } func compareTimesAsc(a, b time.Time) int { diff --git a/internal/storage/sqlbuild/sqlbuild_test.go b/internal/storage/sqlbuild/sqlbuild_test.go index dcd4bd2b9..ac2d9bfc2 100644 --- a/internal/storage/sqlbuild/sqlbuild_test.go +++ b/internal/storage/sqlbuild/sqlbuild_test.go @@ -2,6 +2,8 @@ package sqlbuild import ( "fmt" + "slices" + "sort" "strings" "testing" "time" @@ -89,6 +91,126 @@ func TestLessIDHonorsSortDesc(t *testing.T) { } } +// TestLessSummaryMirrorsOrderBy is TestLessMirrorsOrderBy's sibling for +// LessSummary/types.IssueSummary. Less and LessSummary share the lessFields +// core, so this mostly guards against a future adapter (summarySortFields) +// dropping or mis-mapping a field rather than the tie-break logic itself. +func TestLessSummaryMirrorsOrderBy(t *testing.T) { + t.Parallel() + + now := time.Now().UTC() + older := now.Add(-time.Hour) + a := &types.IssueSummary{ID: "a", Priority: 1, CreatedAt: now} + b := &types.IssueSummary{ID: "b", Priority: 2, CreatedAt: now} + if !LessSummary(a, b, "", false) || LessSummary(b, a, "", false) { + t.Error("default sort must order priority 1 before priority 2") + } + c := &types.IssueSummary{ID: "c", Priority: 1, CreatedAt: older} + if !LessSummary(a, c, "", false) { + t.Error("equal priority must order newer created_at first (created_at DESC)") + } + d := &types.IssueSummary{ID: "d", Priority: 1, CreatedAt: now} + if !LessSummary(a, d, "", false) || LessSummary(d, a, "", false) { + t.Error("full tie must break by id ASC") + } +} + +// TestLessSummaryMatchesLessAcrossSortKeys is TestLessSummaryMirrorsOrderBy's +// broader sibling: instead of spot-checking only the default tie-break chain, +// it sorts a parallel Issue/IssueSummary fixture with Less and LessSummary +// for every SortDefs key (plus "id") in both directions and asserts the two +// produce identical ID order. Less and LessSummary already share the +// lessFields core, so this mainly guards summarySortFields: if a future field +// is added to sortFields without updating summarySortFields, the two +// adapters silently diverge and this is what would catch it. +func TestLessSummaryMatchesLessAcrossSortKeys(t *testing.T) { + t.Parallel() + + day := func(d int) time.Time { return time.Date(2026, 1, d, 0, 0, 0, 0, time.UTC) } + closedOn := func(d int) *time.Time { c := day(d); return &c } + + type row struct { + id string + priority int + createdAt time.Time + updatedAt time.Time + closedAt *time.Time + status types.Status + issueType types.IssueType + assignee string + title string + } + // Distinct (and, for priority, partially tied) values on every field so + // each sort key — including the tie-break chain the "" and "priority" + // keys share — produces a non-trivial, checkable order. + rows := []row{ + {"p1", 3, day(1), day(5), nil, types.StatusOpen, types.TypeBug, "alice", "Zebra"}, + {"p2", 1, day(3), day(2), closedOn(10), types.StatusBlocked, types.TypeEpic, "bob", "apple"}, + {"p3", 2, day(5), day(4), closedOn(8), types.StatusClosed, types.TypeTask, "carol", "Mango"}, + {"p4", 1, day(2), day(1), nil, types.StatusInProgress, types.TypeChore, "dave", "banana"}, + {"p5", 4, day(4), day(3), closedOn(6), types.StatusOpen, types.TypeBug, "eve", "Orange"}, + } + + issues := make([]*types.Issue, len(rows)) + summaries := make([]*types.IssueSummary, len(rows)) + for i, r := range rows { + issues[i] = &types.Issue{ + ID: r.id, Priority: r.priority, CreatedAt: r.createdAt, UpdatedAt: r.updatedAt, + ClosedAt: r.closedAt, Status: r.status, IssueType: r.issueType, Assignee: r.assignee, Title: r.title, + } + summaries[i] = &types.IssueSummary{ + ID: r.id, Priority: r.priority, CreatedAt: r.createdAt, UpdatedAt: r.updatedAt, + ClosedAt: r.closedAt, Status: r.status, IssueType: r.issueType, Assignee: r.assignee, Title: r.title, + } + } + + keys := make([]string, 0, len(SortDefs)+1) + for key := range SortDefs { + keys = append(keys, key) + } + keys = append(keys, "id") + + for _, key := range keys { + for _, desc := range []bool{false, true} { + name := key + if name == "" { + name = "default" + } + if desc { + name += "/desc" + } else { + name += "/asc" + } + t.Run(name, func(t *testing.T) { + t.Parallel() + + sortedIssues := append([]*types.Issue(nil), issues...) + sort.Slice(sortedIssues, func(i, j int) bool { + return Less(sortedIssues[i], sortedIssues[j], key, desc) + }) + sortedSummaries := append([]*types.IssueSummary(nil), summaries...) + sort.Slice(sortedSummaries, func(i, j int) bool { + return LessSummary(sortedSummaries[i], sortedSummaries[j], key, desc) + }) + + gotIssues := make([]string, len(sortedIssues)) + for i, is := range sortedIssues { + gotIssues[i] = is.ID + } + gotSummaries := make([]string, len(sortedSummaries)) + for i, s := range sortedSummaries { + gotSummaries[i] = s.ID + } + + if !slices.Equal(gotIssues, gotSummaries) { + t.Errorf("Less vs LessSummary order diverge for sortBy=%q sortDesc=%v:\n Less: %v\n LessSummary: %v", + key, desc, gotIssues, gotSummaries) + } + }) + } + } +} + func TestBuildReadyWorkOrderPriorityFIFO(t *testing.T) { t.Parallel() diff --git a/internal/storage/storage.go b/internal/storage/storage.go index b90f62833..28b9536c7 100644 --- a/internal/storage/storage.go +++ b/internal/storage/storage.go @@ -409,6 +409,29 @@ type Storage interface { // returns only matching issue IDs. Use when full row hydration is wasted // (e.g., partial-ID resolution in internal/utils/id_parser.go). SearchIssueIDs(ctx context.Context, query string, filter types.IssueFilter) ([]string, error) + // SearchIssueSummaries is a narrow-projection variant of SearchIssues that + // returns []*types.IssueSummary instead of full issues, for list-shaped + // rendering paths that never dereference TEXT/JSON columns. SortBy/SortDesc + // are honored identically to SearchIssues — including SortBy=="id", which + // both paths deliberately leave to the caller: sqlbuild.OrderBy emits no + // ORDER BY for it (sqlbuild.IsGoSideSort), so storage orders id-sorted rows + // only where the issues+wisps merge re-sorts them (sqlbuild.LessSummary, + // mirroring sqlbuild.Less for SearchIssues). The user-facing id order is + // cmd/bd's, applied post-call via utils.NaturalCompareIDs — numeric-aware + // and intentionally different from the lexicographic storage comparator, so + // a storage-side id sorter here would diverge from SearchIssues rather than + // complete it. + // + // Two filter fields behave differently here than on SearchIssues, both + // because types.IssueSummary has no Dependencies field to populate: + // IncludeDependencies is a silent no-op (SearchIssues hydrates dependency + // records; this path has nowhere to put them, so a caller that needs them + // must use SearchIssues), while SkipLabels is honored exactly as it is + // there. Wisps are NOT excluded: the issues+wisps merge runs for any filter + // that does not set SkipWisps, and types.IssueSummary carries the four + // wisp-plane markers so a merged wisp row serializes identically to the one + // SearchIssues would return. + SearchIssueSummaries(ctx context.Context, query string, filter types.IssueFilter) ([]*types.IssueSummary, error) // Dependencies AddDependency(ctx context.Context, dep *types.Dependency, actor string) error diff --git a/internal/telemetry/storage.go b/internal/telemetry/storage.go index bd11798e6..d60574eef 100644 --- a/internal/telemetry/storage.go +++ b/internal/telemetry/storage.go @@ -275,6 +275,17 @@ func (s *InstrumentedStorage) SearchIssueIDs(ctx context.Context, query string, return ids, err } +func (s *InstrumentedStorage) SearchIssueSummaries(ctx context.Context, query string, filter types.IssueFilter) ([]*types.IssueSummary, error) { + attrs := []attribute.KeyValue{attribute.String("bd.query", query)} + ctx, span, t := s.op(ctx, "SearchIssueSummaries", attrs...) + summaries, err := s.inner.SearchIssueSummaries(ctx, query, filter) + if err == nil { + span.SetAttributes(attribute.Int("bd.result.count", len(summaries))) + } + s.done(ctx, span, t, err, attrs...) + return summaries, err +} + // ── Dependencies ──────────────────────────────────────────────────────────── func (s *InstrumentedStorage) AddDependency(ctx context.Context, dep *types.Dependency, actor string) error { diff --git a/internal/types/BUILD.bazel b/internal/types/BUILD.bazel index c87b51074..15f4c8d94 100644 --- a/internal/types/BUILD.bazel +++ b/internal/types/BUILD.bazel @@ -17,6 +17,7 @@ go_test( srcs = [ "explain_test.go", "id_generator_test.go", + "issue_summary_test.go", "storage_class_test.go", "types_test.go", ], diff --git a/internal/types/issue_summary_test.go b/internal/types/issue_summary_test.go new file mode 100644 index 000000000..17acb9686 --- /dev/null +++ b/internal/types/issue_summary_test.go @@ -0,0 +1,264 @@ +package types + +import ( + "encoding/json" + "reflect" + "testing" + "time" +) + +// TestIssueSummaryJSONTagsMatchIssue pins IssueSummary's wire shape to Issue's, +// field by field. IssueSummary is a narrow projection of Issue, so any field it +// carries must serialize under the same key with the same omitempty behavior — +// otherwise a summary-backed `bd list --json` and a full-Issue-backed one emit +// different documents for the same bead. Comparing against Issue's own tags +// rather than a hand-copied literal means the two cannot drift when a tag on +// Issue changes. +func TestIssueSummaryJSONTagsMatchIssue(t *testing.T) { + issueTags := make(map[string]string) + issueType := reflect.TypeOf(Issue{}) + for i := 0; i < issueType.NumField(); i++ { + f := issueType.Field(i) + issueTags[f.Name] = f.Tag.Get("json") + } + + summaryType := reflect.TypeOf(IssueSummary{}) + for i := 0; i < summaryType.NumField(); i++ { + f := summaryType.Field(i) + want, ok := issueTags[f.Name] + if !ok { + t.Errorf("IssueSummary.%s has no same-named field on Issue; choose its json tag deliberately and update this test", f.Name) + continue + } + if got := f.Tag.Get("json"); got != want { + t.Errorf("IssueSummary.%s json tag = %q, want %q (must match Issue.%s)", f.Name, got, want, f.Name) + } + } +} + +// TestIssueSummaryMarshalsIssueWireKeys is the concrete counterpart to the +// reflection test above: it marshals a fully-populated DURABLE summary and +// asserts the exact key set. An untagged struct would emit Go's default +// capitalized field names ("ID", "Title", …) and silently break every consumer +// parsing bd output, which is the regression this pins. +// +// The exact-count assertion carries a second claim now that IssueSummary also +// holds the wisp-plane markers: a durable bead must not sprout ephemeral, +// no_history, wisp_type or storage_class keys. The wisp direction is pinned by +// TestIssueSummaryWispKeysMatchIssue below. +func TestIssueSummaryMarshalsIssueWireKeys(t *testing.T) { + closedAt := time.Date(2026, 3, 4, 5, 6, 7, 0, time.UTC) + summary := IssueSummary{ + ID: "bd-1", + Title: "narrow projection", + Status: StatusClosed, + Priority: 1, + IssueType: TypeTask, + Assignee: "someone", + Pinned: true, + Labels: []string{"alpha"}, + CreatedAt: time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC), + UpdatedAt: time.Date(2026, 2, 3, 4, 5, 6, 0, time.UTC), + ClosedAt: &closedAt, + } + + encoded, err := json.Marshal(summary) + if err != nil { + t.Fatalf("marshal IssueSummary: %v", err) + } + + var decoded map[string]json.RawMessage + if err := json.Unmarshal(encoded, &decoded); err != nil { + t.Fatalf("unmarshal IssueSummary: %v", err) + } + + want := []string{ + "id", "title", "status", "priority", "issue_type", + "assignee", "pinned", "labels", "created_at", "updated_at", "closed_at", + } + for _, key := range want { + if _, ok := decoded[key]; !ok { + t.Errorf("marshaled IssueSummary missing key %q; got %s", key, encoded) + } + } + if len(decoded) != len(want) { + t.Errorf("marshaled IssueSummary has %d keys, want %d; got %s", len(decoded), len(want), encoded) + } +} + +// TestIssueSummaryOmitsEmptyLikeIssue pins the omitempty half of the contract: +// Priority carries no omitempty because 0 is a valid priority (P0/critical), +// while the optional fields drop out of the document entirely when unset. A +// summary that emitted "priority" only for non-zero values would make P0 beads +// indistinguishable from unset ones in `bd list --json`. +func TestIssueSummaryOmitsEmptyLikeIssue(t *testing.T) { + encoded, err := json.Marshal(IssueSummary{ID: "bd-1", Title: "t"}) + if err != nil { + t.Fatalf("marshal zero-value IssueSummary: %v", err) + } + + var decoded map[string]json.RawMessage + if err := json.Unmarshal(encoded, &decoded); err != nil { + t.Fatalf("unmarshal IssueSummary: %v", err) + } + + for _, key := range []string{"id", "title", "priority", "created_at", "updated_at"} { + if _, ok := decoded[key]; !ok { + t.Errorf("zero-value IssueSummary should still emit %q; got %s", key, encoded) + } + } + for _, key := range []string{"status", "issue_type", "assignee", "pinned", "labels", "closed_at"} { + if _, ok := decoded[key]; ok { + t.Errorf("zero-value IssueSummary should omit %q; got %s", key, encoded) + } + } +} + +// wispMarkerFields are the Issue fields that identify a row as living on the +// wisp plane. issueops.searchInTx merges wisp rows into every SearchIssueSummaries +// result that does not set SkipWisps, so these travel the summary path too. +var wispMarkerFields = []string{"Ephemeral", "NoHistory", "WispType", "StorageClass"} + +// TestIssueSummaryCarriesWispMarkers pins the wisp half of the wire-shape +// contract the doc comment on IssueSummary states. +// +// The durable-row half is already covered: TestIssueSummaryJSONTagsMatchIssue +// checks that every field IssueSummary *has* serializes like Issue's. That +// leaves the fields it does *not* have unchecked, and one group of those is +// not optional — issueops.searchInTx merges the wisps table into every +// non-SkipWisps result, so a summary-backed `bd list --json` renders wisp rows +// alongside durable ones. A marker Issue carries and IssueSummary drops +// disappears from that document with no other signal, which is exactly the +// "every consumer parsing bd output breaks silently" failure the doc comment +// promises against. +// +// Type equality is asserted as well as the tag: WispType and StorageClass are +// named string types whose zero values marshal identically to a plain string, +// so a field declared as string here would satisfy a tag-only check while +// still dropping the type's own validation surface. +func TestIssueSummaryCarriesWispMarkers(t *testing.T) { + issueType := reflect.TypeOf(Issue{}) + summaryType := reflect.TypeOf(IssueSummary{}) + + for _, name := range wispMarkerFields { + issueField, ok := issueType.FieldByName(name) + if !ok { + t.Fatalf("Issue has no %s field; update wispMarkerFields to match the wisp plane's markers", name) + } + + summaryField, ok := summaryType.FieldByName(name) + if !ok { + t.Errorf("IssueSummary has no %s field: searchInTx merges wisp rows into every non-SkipWisps "+ + "SearchIssueSummaries result, so a summary-backed `bd list --json` would drop %q for those rows", + name, issueField.Tag.Get("json")) + continue + } + if got, want := summaryField.Tag.Get("json"), issueField.Tag.Get("json"); got != want { + t.Errorf("IssueSummary.%s json tag = %q, want %q (must match Issue.%s)", name, got, want, name) + } + if got, want := summaryField.Type, issueField.Type; got != want { + t.Errorf("IssueSummary.%s type = %v, want %v (must match Issue.%s)", name, got, want, name) + } + } +} + +// TestIssueSummaryWispKeysMatchIssue is the concrete counterpart to +// TestIssueSummaryCarriesWispMarkers: the reflection test proves the fields +// exist with matching tags, this one proves a wisp row actually serializes the +// same through both types. It marshals the same marker values as an Issue and +// as an IssueSummary and compares the four keys byte for byte, so a divergence +// in omitempty behavior, key name, or the named types' own marshaling shows up +// as a value mismatch rather than passing on field presence alone. +// +// The durable case is included deliberately: parity means a durable bead emits +// no marker keys through either type, not merely that a wisp emits some. +func TestIssueSummaryWispKeysMatchIssue(t *testing.T) { + // Ephemeral and NoHistory are mutually exclusive on the wisp plane + // (Issue.Validate rejects both set), so they get one row each rather than + // one row setting both. + cases := []struct { + name string + ephemeral bool + noHistory bool + wispType WispType + storageClass StorageClass + }{ + { + name: "ephemeral heartbeat wisp", + ephemeral: true, + wispType: WispTypeHeartbeat, + storageClass: StorageClassEphemeral, + }, + { + name: "no-history escalation wisp", + noHistory: true, + wispType: WispTypeEscalation, + storageClass: StorageClassEphemeral, + }, + { + name: "durable bead emits no marker keys", + }, + } + + markerKeys := []string{"ephemeral", "no_history", "wisp_type", "storage_class"} + + for _, tc := range cases { + tc := tc + t.Run(tc.name, func(t *testing.T) { + issue := Issue{ + ID: "bd-1", + Title: "narrow projection", + Status: StatusOpen, + Priority: 2, + IssueType: TypeTask, + Ephemeral: tc.ephemeral, + NoHistory: tc.noHistory, + WispType: tc.wispType, + StorageClass: tc.storageClass, + } + summary := IssueSummary{ + ID: issue.ID, + Title: issue.Title, + Status: issue.Status, + Priority: issue.Priority, + IssueType: issue.IssueType, + Ephemeral: tc.ephemeral, + NoHistory: tc.noHistory, + WispType: tc.wispType, + StorageClass: tc.storageClass, + } + + fromIssue := marshalToKeys(t, issue) + fromSummary := marshalToKeys(t, summary) + + for _, key := range markerKeys { + want, inIssue := fromIssue[key] + got, inSummary := fromSummary[key] + if inIssue != inSummary { + t.Errorf("key %q present in Issue=%v, in IssueSummary=%v; a summary-backed "+ + "`bd list --json` must emit the same marker keys a full-Issue-backed one does", + key, inIssue, inSummary) + continue + } + if inIssue && string(got) != string(want) { + t.Errorf("key %q = %s through IssueSummary, want %s (Issue's value)", key, got, want) + } + } + }) + } +} + +// marshalToKeys marshals v and returns its top-level keys, so two documents +// can be compared key by key without depending on field order. +func marshalToKeys(t *testing.T, v any) map[string]json.RawMessage { + t.Helper() + encoded, err := json.Marshal(v) + if err != nil { + t.Fatalf("marshal %T: %v", v, err) + } + var decoded map[string]json.RawMessage + if err := json.Unmarshal(encoded, &decoded); err != nil { + t.Fatalf("unmarshal %T: %v", v, err) + } + return decoded +} diff --git a/internal/types/types.go b/internal/types/types.go index 9d01e47f9..eda2340c8 100644 --- a/internal/types/types.go +++ b/internal/types/types.go @@ -192,6 +192,51 @@ type Issue struct { IsLitePartial bool `json:"-"` } +// IssueSummary is a read-only narrow projection of Issue for list-shaped +// rendering paths that don't dereference TEXT/JSON columns. Populated by +// storage.SearchIssueSummaries, which SELECTs only the columns listed here. +// Shape ratified by be-nu4.3.1 addendum: Pinned IS included, Metadata is NOT +// — adding Metadata would re-introduce the JSON parse cost D3 exists to +// eliminate. +// +// IssueSummary is read-only. No write methods accept it. +// +// The JSON tags mirror the same-named fields on Issue exactly (name, casing, +// and omitempty), because this type backs list-shaped rendering and `bd list +// --json` is one of that command's primary modes: a summary-backed list must +// serialize to the same wire shape a full-Issue-backed one does, or every +// consumer parsing bd output breaks silently. Keep them in sync with Issue. +// +// That promise covers wisp rows, not only durable ones: issueops.searchInTx +// merges the wisps table into every result whose filter does not set +// SkipWisps, so the four wisp-plane markers below are part of the projection +// rather than an optional extra. They are narrow scalar columns (two +// TINYINT(1), two short VARCHARs), so carrying them costs none of the +// TEXT/JSON hydration D3 exists to eliminate. +type IssueSummary struct { + ID string `json:"id"` + Title string `json:"title"` + Status Status `json:"status,omitempty"` + Priority int `json:"priority"` // No omitempty: 0 is valid (P0/critical) + IssueType IssueType `json:"issue_type,omitempty"` + Assignee string `json:"assignee,omitempty"` + Pinned bool `json:"pinned,omitempty"` + Labels []string `json:"labels,omitempty"` + CreatedAt time.Time `json:"created_at"` + UpdatedAt time.Time `json:"updated_at"` + ClosedAt *time.Time `json:"closed_at,omitempty"` + + // ===== Wisp-plane markers ===== + // A summary-backed list renders wisps as well as durable beads (see the + // doc comment above), and these four are what distinguish one. Dropping + // them would make a wisp indistinguishable from a durable bead in + // `bd list --json` while every other key stayed identical. + Ephemeral bool `json:"ephemeral,omitempty"` + NoHistory bool `json:"no_history,omitempty"` + WispType WispType `json:"wisp_type,omitempty"` + StorageClass StorageClass `json:"storage_class,omitempty"` +} + // ComputeContentHash creates a deterministic hash of the issue's content. // Uses all substantive fields (excluding ID, timestamps, and compaction metadata) // to ensure that identical content produces identical hashes across all clones. @@ -2120,7 +2165,12 @@ type IssueFilter struct { SkipWisps bool // Q2: skip wisps table merge entirely (for callers that never return ephemeral results) NoIDShrink bool // Q3: force Pattern A (full 47-col scan) even when Limit > 0 - Offset int + Offset int + // SortBy and SortDesc are honored by SearchIssues, SearchIssueIDs, and + // SearchIssueSummaries alike. All three sort implementations (SQL ORDER BY + // and the Go-side merge comparators) must order identically for a given + // SortBy value, or a post-merge limit cut can keep a different row set + // than SQL selected. SortBy string SortDesc bool