beads/internal/storage/sqlbuild/sort.go
Steve Yegge 96388ea019
fix(search): order Go-side-sort pages before every trim; honor sortDesc for id (#5639)
* fix(search): order Go-side-sort pages before every trim; honor sortDesc for id

Two live defects, both named-but-deferred in #5587's commit message and
both reachable from bd query '<expr>' --sort id (DefaultQueryLimit means a
bound is always set):

1. sqlbuild.Less ignored sortDesc for "id" — the one Go-side sort key —
   so every merge sort feeding a reversed id page kept the byte-FIRST
   rows. idSrcPage.sortGoSide's doc named this sibling bug explicitly.

2. #5587's fix (a page bound is only ever pushed under an order the query
   can express) covered the union seam only. The per-table seams
   (searchTable/scanFilterIDs, and the counts twin's
   finishSearchCountsPage) received the full matching set — searchWindowFor
   already withholds the page bound — but trimmed it with no sort anywhere:
   measured, 10 rows sort id desc limit 3 answered [000 001 002] where
   [009 008 007] was owed. The store-backed seam (issueops.searchTableInTxT)
   was worse off: it still pushed LIMIT into the leg SQL with no ORDER BY,
   so each leg contributed an arbitrary engine-ordered subset.

Fixes, one per seam, all converging on the same rule:

- sqlbuild.Less: sortDesc now flips the id comparison (byte order, matching
  the SQL the key would render if it could).
- domain/db: new sortRowsGoSide helper (the per-table copy of
  idSrcPage.sortGoSide) runs before finishWindow at searchTable's wide
  path, scanFilterIDs, and finishSearchCountsPage.
- issueops.searchTableInTxT: a Go-side sort leg no longer carries a SQL
  LIMIT; the leg scans the complete matching set, orders it in Go, and
  applies the same EffectiveSearchLimit bound after the order exists
  (goSideSortAndTrim), so cap accounting is unchanged and Pattern B
  hydrates only surviving rows.
- idProjection gains a comparator for the one key its payload can order
  ("id"), removing the merged-id-search landmine the old nil documented.

Tests: red-before-green verified on all three seams. sqlbuild unit test
pins Less; sqlmock tests pin the store-backed leg shape (no LIMIT under a
Go-side sort, bound still pushed under a SQL sort) and the ordered+trimmed
page; the domain/db suite adds per-table membership cases (asc, the
measured desc repro, and a four-route parity check: wisps-empty probe,
SkipWisps, wide path, counts twin must answer one page) plus a reversed
union page fence.

(bd-jao3t, bd-69c1a)

* fix(search): extend the Go-side-sort bound rule to the store-backed counts seam; one LessID comparator

Review round (adversarial pass on PR #5639):

F1 — runFilterSearchQueryInTx (issueops/search_counts.go) had the identical
defect the first commit fixed in searchTableInTxT: LIMIT eff pushed
unconditionally while sqlbuild.OrderBy renders no ORDER BY for "id". This
is the seam bd query '<expr>' --sort id actually reaches on the
store-shaped backends (storequerier → SearchIssuesWithCounts →
SearchIssuesWithCountsInTx), where BuildQueryPlan always pushes a bound —
so the first commit's repro claim held only for the uow/domain-db bodies.
Now the counts leg scans the complete matching set under a Go-side sort
and applies the same eff bound after ordering (goSideSortAndTrim),
leaving merge/trim/cap accounting exactly what the SQL LIMIT handed them.
bd list was already safe here (workapi.SQLLimit zeroes the pushed limit
for --sort id); only the querier path pushed a bound.

F2 — the byte-order id comparison existed in five hand-rolled copies
(Less, idProjection.less, goSideSortAndTrim, sortRowsGoSide,
idSrcPage.sortGoSide) — the exact duplication that let Less drift from
sortGoSide in the first place, and that hid F1's sixth site. All five now
call the one exported comparator, sqlbuild.LessID.

F3 — the ascending per-table suite case is annotated as the weak witness
it is (engine order for a PK-clustered prefix scan tends to be
byte-ascending already); the DESC cases are the red-before-green fences.

New shape test pins the counts legs: under --sort id neither the issues
nor the wisps counts query may carry a LIMIT (red on the previous commit:
the leg query carried LIMIT 3).

(bd-jao3t, bd-69c1a)
2026-08-11 00:24:28 -07:00

186 lines
5.6 KiB
Go

package sqlbuild
import (
"fmt"
"strings"
"time"
"github.com/steveyegge/beads/internal/types"
)
// SortDef maps a user-facing sort key to its column and default direction.
type SortDef struct {
Column string
DefaultDir string
}
// SortDefs is the canonical sort-key table for issue list/search ordering.
var SortDefs = map[string]SortDef{
"": {"priority", "ASC"},
"priority": {"priority", "ASC"},
"created": {"created_at", "DESC"},
"updated": {"updated_at", "DESC"},
"closed": {"closed_at", "DESC"},
"status": {"status", "ASC"},
"type": {"issue_type", "ASC"},
"assignee": {"assignee", "ASC"},
"title": {"title", "ASC"},
}
// UnionSortColumnsSQL projects every sortable column under a stable sort_*
// alias so a UNION ALL outer query can ORDER BY any sort key.
const UnionSortColumnsSQL = `priority AS sort_priority,
created_at AS sort_created,
updated_at AS sort_updated,
closed_at AS sort_closed,
status AS sort_status,
issue_type AS sort_type,
assignee AS sort_assignee,
LOWER(title) AS sort_title`
// IsGoSideSort reports sort keys that are applied in Go after the query
// instead of in SQL.
func IsGoSideSort(sortBy string) bool {
return sortBy == "id"
}
// LessID orders two ids for the Go-side "id" sort key: byte order — the order
// SQL would apply — flipped by desc. This is THE membership comparator for a
// Go-side id sort; every seam (Less, the union page's sortGoSide, the
// per-table sortRowsGoSide, the store-backed goSideSortAndTrim) calls it
// rather than restating the comparison, because the sortDesc bug it fixes
// came from exactly such a restated copy drifting from its siblings. The
// natural-numeric order a human reads (bd-9 before bd-10) is display-layer
// (workapi.CompareIssuesBy), applied to the delivered page — never here.
func LessID(a, b string, desc bool) bool {
if desc {
return a > b
}
return a < b
}
func flipDir(dir string) string {
if dir == "ASC" {
return "DESC"
}
return "ASC"
}
// OrderByForColumns renders the ORDER BY clause for a sort key, mapping sort
// keys to column expressions via col. Used directly by UNION consumers whose
// columns are aliased; per-table callers should use OrderBy.
func OrderByForColumns(sortBy string, sortDesc bool, col func(sortKey string) string) string {
if IsGoSideSort(sortBy) {
return ""
}
def, ok := SortDefs[sortBy]
if !ok {
def = SortDefs[""]
sortBy = ""
}
dir := def.DefaultDir
if sortDesc {
dir = flipDir(dir)
}
if sortBy == "" || sortBy == "priority" {
return fmt.Sprintf("ORDER BY %s %s, %s DESC, %s ASC", col("priority"), dir, col("created"), col("id"))
}
// A nullable sort column (closed_at, assignee) treats NULL as lowest: first on ASC
// and last on DESC. Lead with an explicit (col IS NULL) key so the contract does not
// depend on a driver's default NULL ordering. NOT NULL columns keep the plain clause.
if sortBy == "closed" || sortBy == "assignee" {
return fmt.Sprintf("ORDER BY (%s IS NULL) %s, %s %s, %s ASC", col(sortBy), flipDir(dir), col(sortBy), dir, col("id"))
}
return fmt.Sprintf("ORDER BY %s %s, %s ASC", col(sortBy), dir, col("id"))
}
// OrderBy renders the ORDER BY clause against real table columns, optionally
// qualified ("i" -> "i.priority"). Ties always break by id ASC; the default
// priority sort additionally breaks by created_at DESC.
func OrderBy(sortBy string, sortDesc bool, table string) string {
qual := ""
if table != "" {
qual = table + "."
}
return OrderByForColumns(sortBy, sortDesc, func(k string) string {
switch k {
case "id":
return qual + "id"
case "title":
return "LOWER(" + qual + "title)"
}
return qual + SortDefs[k].Column
})
}
// Less is the Go-side mirror of OrderBy for merge sorts over rows fetched
// from separate queries (issues + wisps). It must order exactly the way the
// SQL does, including NULL-first ascending semantics for nullable columns;
// otherwise a post-merge limit cut keeps a different row set than SQL
// selected.
func Less(a, b *types.Issue, sortBy string, sortDesc bool) bool {
if sortBy == "id" {
// This key used to ignore sortDesc, so a reversed id merge kept the
// byte-FIRST rows (the sibling bug idSrcPage.sortGoSide's doc named).
return LessID(a.ID, b.ID, sortDesc)
}
def, ok := SortDefs[sortBy]
if !ok {
def = SortDefs[""]
sortBy = ""
}
descending := def.DefaultDir == "DESC"
if sortDesc {
descending = !descending
}
if c := sortKeyCompare(a, b, sortBy); c != 0 {
if descending {
return c > 0
}
return c < 0
}
if (sortBy == "" || sortBy == "priority") && !a.CreatedAt.Equal(b.CreatedAt) {
return a.CreatedAt.After(b.CreatedAt)
}
return a.ID < b.ID
}
// sortKeyCompare three-way compares the primary sort column in ascending
// order, with MySQL NULL-first semantics for nullable columns.
func sortKeyCompare(a, b *types.Issue, sortBy string) int {
switch sortBy {
case "created":
return compareTimesAsc(a.CreatedAt, b.CreatedAt)
case "updated":
return compareTimesAsc(a.UpdatedAt, b.UpdatedAt)
case "closed":
switch {
case a.ClosedAt == nil && b.ClosedAt == nil:
return 0
case a.ClosedAt == nil:
return -1
case b.ClosedAt == nil:
return 1
}
return compareTimesAsc(*a.ClosedAt, *b.ClosedAt)
case "status":
return strings.Compare(string(a.Status), string(b.Status))
case "type":
return strings.Compare(string(a.IssueType), string(b.IssueType))
case "assignee":
return strings.Compare(a.Assignee, b.Assignee)
case "title":
return strings.Compare(strings.ToLower(a.Title), strings.ToLower(b.Title))
}
return a.Priority - b.Priority
}
func compareTimesAsc(a, b time.Time) int {
switch {
case a.Before(b):
return -1
case a.After(b):
return 1
}
return 0
}