beads/internal/storage/sqlbuild/sqlbuild.go
Julian Knutsen 7e7dce8177
fix(storage): stop reporting a broken wisp plane as an empty one in search (re-land of #5805) (#5932)
* fix(storage): stop reporting a broken wisp plane as an empty one in search

A wisp search folded any table-not-exist error into "there are no wisps". The
tolerance is there for pre-migration databases, which have no wisp plane at
all, but the wisp query reads more tables than the wisp plane owns: its FROM
clause carries sqlbuild.LeaseJoin and its hydration reads wisp_labels. So a
database that has wisps but has lost wisp_labels is reported as having no wisps
-- with a nil error, from an ordinary list.

Reproduced end-to-end against embedded Dolt, with the row present in the same
transaction that is told it is not:

  RENAME TABLE wisp_labels TO wisp_labels_renamed_away
  SearchIssues(ephemeral)  before: 1 row   after: 0 rows, nil error
  SearchIssues(no filter)  before: 2 rows  after: 1 row,  nil error

GetIssueInTx already surfaces a missing wisp_labels as an error (pinned by the
existing embedded-Dolt wisp_label_table_error_propagates case), so the two read
paths disagreed today about whether that database is broken or empty.

Narrow the tolerance to the tables the wisp plane may legitimately not have.
sqlbuild.OptionalWispTable becomes the one place that set is written down --
issueops.optionalBlockedTable now reads it, so the classic and domain/db stacks
cannot drift on what a missing table means, which is the same reason the query
text itself lives in sqlbuild. The check is gated on the existing
IsTableNotExist classification via dberrors.MissingTableName, so nothing that
was previously an error becomes tolerated.

The leases half of the same over-broad check is included for the same reason
but is not, on its own, a reachable wrong answer today: the ephemeral branch
falls through to an issues-plane query over the same join, which re-raises the
missing table, so the caller does get an error. That was verified rather than
assumed, and the tests pin it at the guard rather than claiming a user-visible
repro.

Tests are red-first: sqlmock cases on both stacks and an end-to-end
RENAME TABLE against real embedded Dolt. Both directions have controls -- a
pre-migration database missing wisps must still search to an empty result, and
a missing wisp_dependencies must stay invisible -- because over-tightening to
"never tolerate" passes every must-error assertion and only those controls
catch it.

* test(storage): cover the union hydrate-wisps guard, which nothing reached

The review council mutated all four guarded call sites and found one that
no test in the repository reaches: the domain/db searchUnion hydrate-wisps
check. Reverting it to the old blanket table-not-exist test survived the
full domain tree and the nine-minute uow contract suite with embedded Dolt
enabled.

It is the site that matters most. The ephemeral branch needs an explicit
filter; the union is what a plain `bd list` runs. And the union is exactly
where the two stacks diverge: the leg's FROM clause is plan.FromSQL, with
no lease join and no label read, so a database missing leases or
wisp_labels answers the union query successfully and returns wisp ids --
the breakage surfaces only in fetchIssuesByIDs, which adds
sqlbuild.LeaseJoin and hydrates labels. Tolerating it there sends every
wisp through reassembleBySrc as a miss and drops it behind a nil error.

Also adds the fail-open control the domain/db file was missing. The
council's other surviving mutant made the guard return true for any error
it could not parse a table name out of; issueops caught it, domain/db was
green across the whole package.

Red before green (real edits, restored by trap):

  union guard -> blanket IsTableNotExist   TestSearchUnionBrokenWispPlane
                                           IsAnError, both subtests; every
                                           other test stays green
  fail open on unparseable errors          both UnrelatedErrorPropagates
                                           tests, ephemeral and union

The e2e wisp_dependencies control keeps its name but loses its overclaim:
a search that never asks for dependencies cannot see that table missing,
so it stays green whatever the tolerance set is. The sqlmock control is
what pins that entry; this one proves the query still answers end to end.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-21 16:58:43 -07:00

142 lines
6.6 KiB
Go

// Package sqlbuild holds the pure SQL-text builders shared by the classic
// issueops stack (production, *sql.Tx) and the domain/db repository stack
// (proxied-server, Runner). Functions here map (filter, tables) -> (sql, args)
// and are execution-context-free: no database handles, no transactions.
//
// Both stacks must produce identical row sets for identical filters (pinned
// by the Seam A parity suite in internal/storage/domain/db). Keeping the
// query text in one place is what prevents the two implementations from
// drifting; see bd-6dnrw.46 for the unify-vs-duplicate decision. Execution
// orchestration (issues+wisps merge strategy, hydration, probes) is
// intentionally NOT unified and stays in each stack.
package sqlbuild
import "strings"
// FilterTables configures table names for the filter builders, allowing the
// same filter logic to target both the issues and wisps table families.
type FilterTables struct {
Main string // "issues" or "wisps"
Labels string // "labels" or "wisp_labels"
Dependencies string // "dependencies" or "wisp_dependencies"
Comments string // "comments" or "wisp_comments"
}
var (
IssuesFilterTables = FilterTables{Main: "issues", Labels: "labels", Dependencies: "dependencies", Comments: "comments"}
WispsFilterTables = FilterTables{Main: "wisps", Labels: "wisp_labels", Dependencies: "wisp_dependencies", Comments: "wisp_comments"}
)
// OptionalWispTable reports whether name is a wisp-plane table a database may
// legitimately not have, which is the set a wisp query may treat as "no wisps"
// rather than a failure. Both stacks read it so the two cannot disagree about
// what a missing table means.
func OptionalWispTable(name string) bool {
return strings.EqualFold(name, "wisps") || strings.EqualFold(name, "wisp_dependencies")
}
// DepTargetExpr resolves a dependency row's target across the three
// mutually-exclusive target columns.
const DepTargetExpr = "COALESCE(depends_on_issue_id, depends_on_wisp_id, depends_on_external)"
// IssueBaseColumns is the column list for the issues/wisps row itself,
// without the lease overlay. Use IssueSelectColumns for full hydration;
// this split exists so callers that alias the main table (QualifyColumns)
// can qualify the row columns without mangling the leases.* references.
const IssueBaseColumns = `id, content_hash, title, description, design, acceptance_criteria, notes,
status, priority, issue_type, assignee, estimated_minutes,
created_at, created_by, owner, updated_at, started_at, closed_at, external_ref, spec_id,
compaction_level, compacted_at, compacted_at_commit, original_size, source_repo, close_reason, closed_by_session,
sender, ephemeral, no_history, wisp_type, pinned, is_template,
await_type, await_id, timeout_ns, waiters,
mol_type,
event_kind, actor, target, payload,
due_at, defer_until,
work_type, source_system, metadata, row_lock, storage_class`
// IssueBaseColumnsLite is IssueBaseColumns minus the heavy TEXT columns
// (issueops.HeavyDropList). It lives here rather than beside its scan function
// for the same reason IssueBaseColumns does: the counts mega-query aliases the
// main table and has to qualify the row columns without mangling the leases.*
// references, and that query is rendered in this package. issueops re-exports
// the full list as IssueSelectColumnsLite and pins it against IssueBaseColumns
// with the schema-parity test.
const IssueBaseColumnsLite = `id, content_hash, title,
status, priority, issue_type, assignee, estimated_minutes,
created_at, created_by, owner, updated_at, started_at, closed_at, external_ref, spec_id,
compaction_level, compacted_at, compacted_at_commit, original_size, source_repo, close_reason, closed_by_session,
sender, ephemeral, no_history, wisp_type, pinned, is_template,
await_type, await_id, timeout_ns,
mol_type,
event_kind, actor, target,
due_at, defer_until,
work_type, source_system, metadata, row_lock, storage_class`
// LeaseSelectColumns is the lease overlay for full issue hydration. Leases
// live in the ephemeral leases table (bd-lrgn1), not on the issues row, so
// every query selecting these must also add LeaseJoin to its FROM clause —
// a query that forgets the join fails loudly on the leases.* reference.
const LeaseSelectColumns = `leases.lease_expires_at, leases.heartbeat_at, leases.granted_node`
// IssueSelectColumns is the canonical column list for full issue hydration.
// Every query that reads a complete types.Issue should use this constant
// and include LeaseJoin(table) in its FROM clause; the scan side is
// issueops.ScanIssueFrom, which scans positionally and must stay in
// column-for-column agreement with this list.
const IssueSelectColumns = IssueBaseColumns + `,
` + LeaseSelectColumns
// IssueSelectColumnsLite is the lite counterpart of IssueSelectColumns. The
// scan side is issueops.ScanIssueLiteFrom and the same LeaseJoin requirement
// applies.
const IssueSelectColumnsLite = IssueBaseColumnsLite + `,
` + LeaseSelectColumns
// LeaseJoin returns the FROM-clause fragment that overlays the ephemeral
// leases table onto the given issues/wisps table reference (a table name or
// alias). LEFT JOIN: rows without a live claim have no lease row and hydrate
// nil lease fields.
func LeaseJoin(tableRef string) string {
return "LEFT JOIN leases ON leases.issue_id = " + tableRef + ".id"
}
// QueryBatchSize bounds IN-clause sizes when long ID lists are folded into
// WHERE fragments.
const QueryBatchSize = 200
// QualifyColumns prefixes every column in a comma-separated list (e.g.
// IssueSelectColumns) with the given table qualifier such as "i.".
func QualifyColumns(columns, prefix string) string {
raw := strings.ReplaceAll(columns, "\n", " ")
raw = strings.ReplaceAll(raw, "\t", " ")
parts := strings.Split(raw, ",")
for i, p := range parts {
parts[i] = prefix + strings.TrimSpace(p)
}
return strings.Join(parts, ", ")
}
// InPlaceholders renders "?,?,..." and the matching arg slice for an IN clause.
func InPlaceholders[T ~string](values []T) (string, []any) {
ph := make([]string, len(values))
args := make([]any, len(values))
for i, v := range values {
ph[i] = "?"
args[i] = string(v)
}
return strings.Join(ph, ","), args
}
// CompactNonEmptyStrings drops empty entries, returning nil when none remain.
func CompactNonEmptyStrings(values []string) []string {
if len(values) == 0 {
return nil
}
out := make([]string, 0, len(values))
for _, value := range values {
if value != "" {
out = append(out, value)
}
}
return out
}