diff --git a/.golangci.yml b/.golangci.yml index a41860ae3..d065b0e64 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -254,7 +254,7 @@ linters: linters: - forbidigo # Inside cmd/bd the rule is deny-by-default, so each entry below is a - # HOLE in it and carries the reason that hole exists. 61 of the 388 + # HOLE in it and carries the reason that hole exists. 59 of the 389 # non-test files under cmd/bd are on this list; widening it is an edit # here and a diff a reviewer sees. internal/httpapi has no entry and is # meant not to. @@ -303,7 +303,14 @@ linters: # issueops request and name no filter type, so the hole they needed is # closed rather than merely unused. Removing an entry here is how a role # commit proves its front doors reached the role. - - path: '^cmd/bd/(close|duplicates|find_duplicates|gate|gc|graph|info|label|orphans|purge|search|todo|wisp)(_proxied_server)?\.go$' + # + # `purge` left it the same way when the Sweeper role took the candidate + # filter off both `bd purge` and `bd prune`: cmd/bd/purge.go now builds an + # issueops.SweepRequest, and cmd/bd/purge_proxied_server.go is down to the + # provider's capability accessor. `gc` stays — it is a different command + # that still selects its own closed beads, and it now shares only the pure + # recheck (workapi.FilterSweepCandidates) with the role. + - path: '^cmd/bd/(close|duplicates|find_duplicates|gate|gc|graph|info|label|orphans|search|todo|wisp)(_proxied_server)?\.go$' linters: - forbidigo # Molecule reporting: each selects the members of one molecule. diff --git a/PROPOSAL-bd-serve-http-api.md b/PROPOSAL-bd-serve-http-api.md new file mode 100644 index 000000000..dfc275c50 --- /dev/null +++ b/PROPOSAL-bd-serve-http-api.md @@ -0,0 +1,2807 @@ +# PROPOSAL: `bd serve` — a loopback HTTP API over one shared work-surface contract + +**Status:** design, ready for implementation as sliced beads PRs +**Author:** design merge (five parallel section drafts, reconciled under judge-panel binding corrections) +**Date:** 2026-07-31 +**Repo:** `github.com/steveyegge/beads` (worktree `sd-server` @ `b1694a50a`) +**Motivating incidents:** hosted viewer's bare-`IssueFilter{}` leak and `err != nil || issue == nil → 404` bug; 49 hand-duplicated `*_proxied_server.go` command duals in `cmd/bd`; Gas City's subprocess forks of `bd ready` / `bd list` / `bd show` / `bd update --claim`. + +--- + +## Contents + +1. [Thesis, non-goals, constraints, and scope boundaries](#1-thesis-non-goals-constraints-and-scope-boundaries) +2. [The shared library contract](#2-the-shared-library-contract) — the public `issueops.Reader` role over `internal/workapi`: filter builders, not-found normalization, detail assembly, claim typing, parity enforcement table +3. [The wire: OpenAPI document, schemas, errors, pagination](#3-the-wire-openapi-document-schemas-errors-pagination) — the six operations, `x-go-type` pinning, problem+json, keyset cursor +4. [Codegen, CLI migration, server lifecycle and security](#4-codegen-cli-migration-server-lifecycle-and-security) — drift gate, extraction slices M1–M4, mode gate, bind posture, concurrency +5. [Test strategy, slice plan, risks, and open questions](#5-test-strategy-slice-plan-risks-and-open-questions) + +Appendix A: [Reconciliation with the issueops Lifecycle refactor](#reconciliation-with-the-issueops-lifecycle-refactor) — owner decisions (Reader role + subsume), rationale, strongest objection, merge order, residual risks +Appendix B: [Council review — dispositions](#council-review--dispositions) — per-reviewer verdicts, applied/deferred ledger, post-review consistency check + +Design rules cited as "rule N" throughout refer to the numbered list in +[§ 1 Binding constraints (design rules from review)](#binding-constraints-design-rules-from-review); +"decided parameter N" refers to [§ 1 Binding constraints (decided parameters)](#binding-constraints-decided-parameters). + +--- + +## 1. Thesis, non-goals, constraints, and scope boundaries + +### Thesis + +`bd serve` is a cobra subcommand that runs a loopback-bound HTTP server exposing six operations +(`GET /healthz`, `GET /v0/beads/context`, `GET /v0/beads/ready`, `GET /v0/beads/issues`, +`GET /v0/beads/issues/{id}`, `POST /v0/beads/issues/{id}:claim`) over the existing UOW seam, +described by a hand-written OpenAPI document from which the Go wire types are generated. The +problem it solves is not "beads needs a web server" — it is that beads already has multiple +hand-written surfaces over the same data and they have already drifted in production: `cmd/bd` +contains 49 `*_proxied_server.go` command duals (34 of them on the UOW seam), ready-work defaults +are constructed from scratch twice inside the CLI alone, and the hosted viewer — a third +hand-written surface — shipped both a bare `IssueFilter{}` that leaks gates/wisps/templates/closed +issues that `bd list` hides, and an `err != nil || issue == nil → 404` bug that reports dropped +connections as missing issues. The shared-contract framing is therefore the point: the deliverable +is ONE implementation of filter/default construction, result shaping, detail assembly, not-found +normalization, and the claim commit protocol, which the direct CLI, the proxied CLI, and the HTTP +handlers all consume, so that a fourth surface cannot be written by hand and a field cannot be +silently dropped. The HTTP layer itself is deliberately thin and boring — stdlib mux, one UOW per +request, no new lifecycle machinery — because everything interesting lives at the contract +boundary. That boundary stops at filter and result-shaping construction: query execution is +explicitly NOT abstracted, because both existing seams (`storage.Storage` and +`domain.IssueUseCase` via UOW) already implement every needed query with one-line call sites, and +an execution port would exist only to launder their signature differences. Drift prevention is +enforced by mechanism, not convention: the spec's `Issue` schema is pinned to the canonical +`internal/types` struct via `x-go-type` with a two-way JSON-tag bijection test, CI regenerates and +diffs the generated types, and CLI-vs-HTTP parity oracles compare full item JSON, not ID sets. +Success for v0 is that Gas City can replace its highest-volume `bd` subprocess forks (`ready`, +`list`, `show`, `update --claim`) with HTTP calls whose responses are field-identical to the CLI's +`--json` output — and that after this slice, the duplicated builder code those four commands used +to carry no longer exists to drift. One scoped exception is stated here rather than discovered +downstream: the cache reconciler's full active-universe scan needs the ephemeral tier and +label-free rows (`TierBoth` + `SkipLabels`, served today by `bd list` UNIONED with a second +`bd query "ephemeral=true AND ..."` fork — gascity caching_store_reconcile.go:61-63, +bdstore.go:2262-2264), and v0's list surface deliberately hard-wires `SkipWisps=true` with labels +always hydrated, so that loop stays on subprocess in v0 (§ 3, "what this list surface deliberately +cannot serve"; slice 11). + +### Non-goals + +Explicitly out of scope for v0. Each is a decision, not an omission. + +- **Multi-tenancy.** No `{project}` path segment, no control-plane operations, no per-request + workspace selection. The server is single-workspace by construction (workspace resolution + mutates process-global state: `os.Setenv("BEADS_DIR")` at `cmd/bd/main.go:519,748`). +- **Authentication or TLS.** Loopback is the v0 trust boundary — identical to the loopback Dolt + SQL server and dbproxy the command already fronts. The named future mechanism for non-loopback + is the existing rotating 0600 shared secret in `internal/storage/dbproxy/identity` + (`crypto/subtle` compare), reused, not reinvented. Not in-slice. +- **A write surface beyond claim.** No create/update/close/reopen endpoints. The claim endpoint + exists because Gas City's dispatch loop needs an atomic CAS with a typed conflict; general + writes wait for the public issueops facade vocabulary (see "What this does NOT solve"). +- **Migrating the other ~46 command pairs.** Only `bd ready`, `bd list`, `bd show`, + `bd update --claim` move onto the shared contract in this slice. +- **An execution port/adapter layer — AMENDED by the issueops reconciliation (owner decision; + see "Reconciliation with the issueops Lifecycle refactor").** The read surface IS exposed as + the public `issueops.Reader` role with `IssueReader()` accessors, following the facade's + role/accessor convention, with filter/default construction subsumed inside the + implementations. `internal/workapi` remains the single implementation substrate behind every + Reader, and no OTHER execution abstraction may be introduced — no generic Port abstracting + `storage.Storage` and the UOW use-cases beyond this one role. The council's original finding + is recorded as the considered-and-overridden counter-argument: such a port exists only to + launder `SearchPage` vs `[]*types.Issue`, forces limit+1 over-fetch semantics onto the direct + path, and costs ~5 adapter implementations over one-line reads. +- **Serving embedded (non-SQL-server) workspaces.** Typed refusal via the existing + `storage.ErrUnsupported{Op, Backend}` (`internal/storage/errors_unsupported.go:8`). +- **Fuzzy/substring ID resolution over HTTP.** `GET /v0/beads/issues/{id}` is exact-match (with + the issue→wisp fallback `bd show` has); fuzzy resolution stays a CLI affordance. This is what + lets Gas City delete its decode-array-index-`[0]` + `ErrIDCollision` guard. +- **Hooks.** `bd serve` fires no hooks. The UOW seam has never constructed a `HookFiringStore`; + a user-controlled subprocess per HTTP mutation is a latency and process-lifecycle hazard. +- **CLI auto-commit machinery.** The per-command auto-commit globals in cobra's + `PersistentPostRunE` are never touched; the claim write commits explicitly per request through + the one existing transactional helper (`uow.RunTxResult`, `internal/storage/uow/tx.go:76`). +- **Server lifecycle machinery.** No flock, no pidfile, no `.beads/serve-info.json` discovery + file, no idle timeout. dbproxy needs those because clients race to auto-spawn it; `bd serve` is + operator-invoked and the TCP bind is the mutual exclusion (for a fixed port — see rule 7 and + § 4 for the honest ephemeral-default caveat). +- **Wire-level result caps.** No clamping or rejection of `limit=0`; no rate limiting, request + quotas, or body-size ceremony beyond what the six operations need. Any future cap is an + operator knob, not a wire refusal. +- **Serving the OpenAPI document over HTTP**, dashboards, metrics endpoints, or any operation + beyond the six. Every route must appear in the spec-parity set-equality test; additions are + additive later. A structured request log to stderr is NOT covered by this non-goal — it is a + required floor (§ 4, "Observability floor"): no metrics endpoint is defensible, no log is not. +- **Lite/partial hydration (`skip_labels`).** `IssueFilter.Lite` sets `Issue.IsLitePartial`, + which is `json:"-"` — a lite row is wire-indistinguishable from an issue with empty fields. A + future explicit projection parameter can document the omission contract safely. + +### Binding constraints (decided parameters) + +These are settled. Design and implementation happen WITHIN them; none may be relitigated in a +slice PR. + +1. **Clean single-tenant OSS surface; vendor-neutral source.** The hosted product adapts to OSS, + not the reverse — no commercial or hosted-product names appear in OSS code or spec. +2. **Spec-first.** The hand-written OpenAPI document is the source of truth and Go types are + generated from it with CI drift enforcement — because a spec reverse-engineered from handlers + is documentation, not a contract. +3. **Loopback-only by default.** Bind numeric 127.0.0.1; refuse non-loopback without explicit + opt-in — v0 ships no auth, so the bind policy IS the security model. +4. **`bd serve` cobra subcommand with `--addr`.** A subcommand gets its own flag surface, help, + and lifecycle; a root-level `--serve` flag would entangle every other command's flag parsing. +5. **The API fronts the UOW implementation only.** One-UOW-per-request is the natural transaction + shape and the owner's explicit simplification; the server never constructs a + `storage.Storage`. All three SQL-server modes are in scope for the gate: the exported + providers already exist on main (`uow.NewDoltServerUOWProvider`, + `uow.NewExternalDoltServerUOWProvider` — `internal/storage/uow/doltserver_provider.go:16`, + `external_doltserver_provider.go:18`, both already called by `cmd/bd/uow_factory.go:77,132`). + On main today, `uowProvider` is wired only in proxied mode (`cmd/bd/main.go:1378-1388`), so + the serve command must construct the matching provider for server/external-server workspaces + itself; the delivery plan states which slice wires which mode, and any mode not yet wired gets + a typed refusal whose message does not promise what the gate refuses. +6. **MVP surface = exactly the six operations listed in the thesis.** Bounded by what Gas City + verifiably calls today; every additional operation is permanent wire surface. +7. **CLI migration scope = the four backing commands only** (`bd ready`, `bd list`, `bd show`, + `bd update --claim`) — enough to prove the shared contract kills real drift without a 49-pair + big bang. +8. **Delivery target is OSS beads main** (`github.com/steveyegge/beads`); the Gas City fork + rebase and its Postgres conformance are downstream dependencies, not in-slice work. + +### Binding constraints (design rules from review) + +These override any section or slice that disagrees. Each is grep-verified against the tree. + +1. **No second wire struct.** The spec's `Issue`/`IssueDetails`/`IssueWithCounts` schemas are + pinned to the canonical `internal/types` structs via `x-go-type`; no hand-written or generated + mirror struct, no Go-to-Go mapping function anywhere. Enforcement is a two-way JSON-tag ↔ + spec-property bijection test — a round-trip fixture test is insufficient (a new `omitempty` + field absent from the fixture passes both directions). +2. **`limit=0` means unlimited, exactly as in the CLI.** Gas City calls `bd ready ... --limit 0` + (`gascity internal/beads/bdstore.go:2513`) and `bd list --json --limit 0` (`bdstore.go:1923`, + the reconciler's forever full active-set scan). A v0 that 400s or clamps `limit=0` fails the + brief. (The reconciler's own scan stays on subprocess in v0 for tier/labels reasons — § 3, + "what this list surface deliberately cannot serve" — but its `limit 0` posture is the + contract every migrated full-set reader inherits, and `bd ready --limit 0` migrates in v0.) Scope (security review): this is a guarantee for the default loopback bind, where the + reconciler lives; under `--allow-non-loopback` an unlimited fully-buffered read is a + network-reachable memory-exhaustion primitive and `limit=0` is refused there (§ 3, limit + semantics). +3. **`/v0/beads/ready` returns `IssueWithCounts` items, not bare `Issue`.** `bd ready --json` + emits `[]*types.IssueWithCounts` on both CLI paths (`cmd/bd/ready.go:246,539`; + `cmd/bd/ready_proxied_server.go:441`); the HTTP surface must carry + `dependency_count`/`dependent_count`/`comment_count`/`parent` or it drifts at the field level + on day one. +4. **Not-found is normalized inside the shared get/detail function.** On the UOW seam, + `db.issueSQLRepositoryImpl.Get` returns `(nil, sql.ErrNoRows)` + (`internal/storage/domain/db/issue.go:394-410`) — never `storage.ErrNotFound`. The CLI's own + proxied code already treats both sentinels as not-found (`cmd/bd/create_proxied_server.go:497`). + The shared function normalizes {wrapped `sql.ErrNoRows`, nil-issue-with-nil-error} to the + `storage.ErrNotFound` sentinel so no future caller can reintroduce the bug; an integration + test against real Dolt asserts 404-on-unknown-id. Handler-only mapping is insufficient. +5. **One claim retry/commit implementation.** `uow.RunTxResult` + (`internal/storage/uow/tx.go:76`) — already on main with serialization retry and + nothing-to-commit tolerance — is consumed by BOTH the proxied CLI and the server. The bespoke + `cenkalti/backoff` loop in `cmd/bd/update_proxied_server.go:100-116` does not survive as a + second copy, and no new `ClaimViaUOW` wrapper is lifted. +6. **No execution abstraction other than the `issueops.Reader` role.** AMENDED by owner decision + — the reconciliation section supersedes this rule's original absolute form in one respect: + the read surface IS exposed as the public `issueops.Reader` role with `IssueReader()` + accessors, filter/default construction subsumed inside the implementations, and + `internal/workapi` remains the single implementation behind every Reader; no OTHER execution + abstraction may be introduced. The council's original finding is preserved verbatim as the + considered-and-overridden counter-argument: "The shared contract stops at filter and + result-shaping construction; both seams already implement every needed query with one-line + call sites, and a Port would exist only to launder `SearchPage` vs `[]*types.Issue` while + forcing limit+1 over-fetch semantics onto the direct CLI." +7. **No flock/pidfile/serve-info.json.** The TCP bind is the singleton — which holds only for an + explicitly fixed port; under the default ephemeral `--addr 127.0.0.1:0` N serves run in + parallel with no exclusion at all (§ 4 states this honestly and blesses a fixed-port `--addr` + for deployments). Concurrent serves are data-safe because every claim is a CAS inside a Dolt + transaction. +8. **Parity oracles compare full item JSON**, modulo an explicit, documented allowlist. An + ID-set oracle would have passed while the ready endpoint shipped bare `Issue` against the + CLI's `IssueWithCounts`. +9. **Defaults come from one shared constant per surface pair.** List default limit is 50 + (`cmd/bd/list.go:743`), ready default is 100 (`cmd/bd/ready.go:796`); the HTTP defaults are + the same constants imported from the shared package, never re-stated literals. +10. **The mode gate does not over-promise.** Gate on SQL-server-ness (`usesSQLServer()`, + `cmd/bd/store_factory.go:20`), wire the existing exported providers for server and + external-server modes in the slice that claims them, and make every refusal message describe + exactly what is refused and why. +11. **Extractions land as separate no-op refactor slices**, each verified against the byte-pinned + protocol corpus at `cmd/bd/protocol`, so any semantic change is bisectable to one move. +12. **Document only the status codes the six operations need.** Every documented status+code pair + is permanent wire surface; no 413/429/504/403-read_only for mechanisms v0 does not ship. +13. **Detail assembly is shared.** Labels + dependencies-with-metadata + counts + + comments-omitted semantics exist once, consumed by `bd show` (both variants) and the HTTP get + handler — the anti-drift slice must not ship a new copy of the very shaping logic that + drifted in the hosted viewer. +14. **No invented seam capabilities.** Every mechanism a section leans on must be grep-verified. + Known false claims from drafts, excluded here: "workspacegate lease acquired with the + provider" (only `cmd/bd/doctor/gitignore.go` touches `internal/workspacegate` outside the + package itself); "every Port method maps 1:1 onto both seams"; "UOW GetIssue wraps + `storage.ErrNotFound`" (it surfaces `sql.ErrNoRows`, see rule 4). +15. **One CLI-side default stays out of the library.** The cwd-derived directory-label + substitution (`cmd/bd/ready.go:141-145`) is meaningless in a server process; CLI callers + apply it to the request before invoking the shared builder, and the server never does. + +### What this does NOT solve + +Three known gaps are adjacent to this design and deliberately not closed by it. For each: what +the gap is, whether this design leaves it on a credible path, and how it is tracked. None of +these is silently "handled". + +#### The other ~46 direct/proxied command pairs + +`cmd/bd` has 49 `*_proxied_server.go` duals (34 on the UOW seam); this slice migrates the four +backing the MVP operations and leaves ~45 duals in place, still duplicated, still able to drift. +What this design does for them: it establishes the pattern (pure, seam-independent filter/default +builders and shaping functions; per-seam execution as one-line calls; `uow.RunTxResult` as the +single transactional protocol) and proves it on the four commands with the worst verified +duplication — including retiring the one bespoke retry loop (`update_proxied_server.go:100-116`) +as the exemplar for write-shaped duals. The path is credible for any command whose duplication is +filter/shape-shaped (most of the 34 UOW duals), because the extraction is a move plus two +one-line call sites, verifiable against the protocol corpus. The path is NOT yet paved for +commands whose duplication is multi-step transaction orchestration; those are the trigger for +introducing shared execution machinery later — explicitly not before. Tracked: each future +command-family migration is filed as its own bead (one family per issue, dependency-linked to +the extraction slices of this design), and each lands as a separate no-op refactor slice per +rule 11. This design commits to the pattern and the first four instances, not to a schedule for +the rest, and does not claim the pattern is proven beyond what it ships. + +#### The gascity/beads fork rebase and its Postgres conformance + +Gas City consumes a fork (`gascity/beads @ feat/hosted-multibackend`, pinned via a go.mod +replace), and the fork carries a Postgres backend that implements `storage.Storage` but not the +UOW seam. Two consequences this design does not fix: (1) the fork must rebase over this work — +new shared packages, a deleted `cmd/bd/list_filter.go`, and modified call sites in the touched +`cmd/bd` files; (2) `bd serve` on the fork's Postgres backend will refuse with the typed +`storage.ErrUnsupported` until the fork either grows a Postgres UOW provider or adapts the gate, +because the decided parameter 5 makes the server UOW-only. What this design does to keep the path +credible: extractions are moves rather than rewrites (minimizing rebase conflict surface), every +slice PR lists its deleted/renamed files, and the CLI-vs-HTTP parity oracle is written as a +seam-parametrized scenario table so the fork can add its backend as another row rather than +writing a new harness. Tracked: as downstream-dependency beads on the fork side (rebase; Postgres +UOW conformance decision), referenced from this design's delivery epic — per decided parameter 8 +they are dependencies of adoption, not in-slice work, and nothing in this repo blocks on them. + +#### The unmerged `feat/public-issueops-facade` branch + +The branch (at `/data/projects/beads-public-issueops-simple`) is 35 commits ahead of origin/main +at base `8bb0d36be` — current-main lineage; the earlier "32 ahead / 12 behind @ `53f3424cd`" +figure is stale. It defines a public WRITE vocabulary whose binding contract is +`issueops.Lifecycle` {Create, Update, Close, Reopen} — the rename from `Operations` is binding +per the facade's final design spec (`/var/tmp/w46-final-design.md`), not yet in code at branch +HEAD, so cite the spec, not branch line numbers — with `UpdateRequest.Claim` CAS semantics. The +facade mints NO error sentinels of its own: it relocates the existing 18 storage/domain error +symbols into the leaf `issueops` package and aliases them back from `internal/storage` with the +same pointer, so existing `errors.Is` keys keep working unedited. Nor is it consumer-free: the +branch routes the four direct CLI write verbs through the facade via +`cmd/bd/issueops_adapter.go`. Its proxied-server rewire is the facade's own deferred follow-up +PR, gated by its spec on proxied contract-pin tests being committed and green first — a +precondition slices 1-4 of this plan partly supply. + +Two binding decisions are now OWNER-DECIDED (rationale, strongest objection, and merge order in +"Reconciliation with the issueops Lifecycle refactor" near the end of this document): + +1. **`issueops.Reader` is the public read ROLE** — the read counterpart of `Lifecycle`, exposed + via `IssueReader()` accessors following the facade's role/accessor convention, with + `internal/workapi` as the single shared implementation substrate behind every Reader. +2. **Subsume, not compose** — filter/default construction happens once, inside the Reader + implementations, behind the accessor; both front doors can only say `rd.List(ctx, req)`. + +The claim write still does not wait for the facade: it uses `domain.IssueUseCase.ClaimIssue` via +`workapi.ClaimOnUOW`, on main today and deliberately below `Lifecycle` (see § 2, claim-outcome +typing, post-facade justification). The read-endpoint slices, however, now gate on the facade's +API-shape phase merging to OSS main (the leaf `issueops` package must exist before `Reader` can +be declared) — a schedule coupling flagged in § 5 Risks and open questions rather than hidden. +The composition obligation stands: the shared contract's vocabulary reuses the existing +`storage.Err*` sentinels and CAS-outcome shapes rather than minting new types. +One ordering decision is made now to pre-empt a future fight: the HTTP 409 `code` strings become +frozen wire surface the moment Gas City consumes them, so if the facade lands with a conflicting +claim vocabulary, the facade adapts to the wire — not the reverse. Tracked: a coordination bead +linking the facade branch to the claim-endpoint slice, checked before that slice ships (the +facade owner reviews the claim endpoint's documented semantics against `UpdateRequest.Claim`); +plus the standing note that any future write endpoints (non-goal above) are specified against the +facade's `Lifecycle` vocabulary once it merges, not invented independently. + +## 2. The shared library contract + +### Scope: what is in the contract, and the explicit UOW-vs-Storage resolution + +The HTTP API fronts UOW only (decided), but the direct CLI has no UOW provider (`uowProvider` is +built only in proxied mode, cmd/bd/main.go:1378-1388) and the proxied CLI has no `storage.Storage`, +so an execution-interface contract over either seam would exile one CLI family. The resolution: +**the contract contains no execution layer at all** (binding rule 6). Both seams already implement +every MVP query with one-line call sites — `store.GetReadyWorkWithCounts` returns +`[]*types.IssueWithCounts` (internal/storage/storage.go:168) while +`uw.IssueUseCase().GetReadyWorkWithCounts` returns `domain.SearchCountsPage{Items, HasMore}` +(internal/storage/domain/issue.go:140-143, :261) — and that mismatch never enters the contract +because the contract stops one call before it. + +**IN the contract** (`internal/workapi`, new package): + +1. Filter/default construction: `BuildReadyFilter` (collapsing the duplicated ~110-line builders at + cmd/bd/ready.go:107-218 and ready_input.go:28-174) and `BuildListFilter` (the ~200-line + `buildListFilter` moved from cmd/bd/list_filter.go:119-314), plus the `ConfigSource` seam and + its two adapters (moved from list_filter.go:48-117). +2. Shared default constants (rule 9): `DefaultReadyLimit = 100`, `DefaultListLimit = 50`, and + `types.WireSchemaVersion = 1` (new `internal/types/wire.go`, consumed by cmd/bd/output.go:12's + `JSONSchemaVersion` and by the `/v0/beads/context` handler). +3. Not-found normalization (rule 4): `IsNotFound` / `ResolveIssue`. +4. Detail assembly (rule 13): `DetailSource` + `BuildIssueDetails`, replacing the duplicated blocks + at cmd/bd/show.go:144-241 and show_proxied_server.go:478-556 (+ helpers :127-168). +5. Claim-outcome typing: `ClaimOutcome`, `ClaimConflictError`, `ClassifyClaimError`, per-attempt + `ClaimOnUOW` — the retry/commit protocol is explicitly NOT here (rule 5; it is `uow.RunTxResult`, + internal/storage/uow/tx.go:76-121, already consumed by proxied `bd ready --claim` at + ready_proxied_server.go:184). + +**Deliberately LEFT on the two existing seams / in cmd/bd** (each with the reason): + +- **Query execution and page shapes.** `[]*types.IssueWithCounts` vs `domain.SearchCountsPage` + stays per-seam; an execution Port would exist only to launder that mismatch and would force the + limit+1 over-fetch onto the direct path (rule 6). Each caller executes its seam's method itself. +- **Overflow/truncation detection.** Direct CLI: `withFetchOneExtra` + `effectiveLimit` trim + + `PaginationMeta` (cmd/bd/list.go:42-76, :587-605); UOW/HTTP: `SearchCountsPage.HasMore`. Two + detectors remain per-seam by design; tied together only by the parity oracle (gap G4). +- **cwd-derived directory-label scoping** (ready.go:141-145, ready_input.go:101-105). Callers + pre-fill `ReadyParams.LabelsAny`: the CLI applies `config.GetDirectoryLabels()`, the server never + does — a server process's cwd is meaningless. The one caller-side default; the builder cannot + reach it (import policy below bans that accessor). +- **Fuzzy/substring ID resolution and cross-repo routing** (`resolveAndGetIssueWithRouting`, + `openRoutedReadStore`): interactive CLI affordances. The contract takes exact IDs only. +- **MaxRows resolution and proxied rejection**: `resolveMaxRows` / + `rejectMaxRowsUnderProxiedServer` stay cobra-side (cmd/bd/max_rows.go:35-106; list.go:522, + ready.go:57-70). `ReadyParams` carries the fields; who fills them is seam policy. +- **Output envelopes**: `outputJSONWithPagination`, `BD_JSON_ENVELOPE`, stderr hints, the `bd show` + array envelope — byte-pinned by the cmd/bd/protocol corpus, CLI-only. HTTP envelopes are the + httpapi layer's generated types. +- **The claim retry/commit protocol**: `uow.RunTxResult` (rule 5); the contract types the outcome, + never the transaction lifecycle. +- **The keyset cursor token codec**: `ListParams` accepts the decoded position + (`AfterCreatedAt`/`AfterID`, 1:1 onto `types.IssueFilter`'s keyset fields, types.go:1618-1633); + token encode/decode is HTTP-layer (§ 3, "Pagination"). + +### The Reader role — the public read seam (owner-decided; reconciliation section) + +`issueops.Reader` is the guarded issue-query role: the read counterpart of `Lifecycle`. It lives +in the leaf `issueops` package at the repo root (imports `internal/types` + stdlib only — +verified leaf-legal: `types.IssueWithCounts`, `types.IssueDetails`, `types.MolType`, +`types.WispType` all live in `internal/types`). One method per read operation; each takes the +high-level request and performs filter/default construction INTERNALLY (subsume). No +constructor; the accessor is the API. + +```go +type Reader interface { + Ready(ctx context.Context, req ReadyRequest) (IssuePage, error) + List(ctx context.Context, req ListRequest) (IssuePage, error) + Get(ctx context.Context, req GetRequest) (*types.IssueDetails, error) +} + +type ReadyRequest struct { // field-for-field the vocabulary slice 2 built as workapi.ReadyParams + IssueType string; Assignee string; Unassigned bool + Labels, LabelsAny, ExcludeLabels []string // raw; normalized inside + LabelPattern, LabelRegex string + Priority *int // nil = unset (replaces Priority+PrioritySet pair) + ParentID string; MolType *types.MolType + IncludeDeferred, IncludeEphemeral bool + ExcludeTypes []string + MetadataFields map[string]string; HasMetadataKey string + Sort string // "" = hybrid; validated inside + Limit *int // nil = default 100; 0 = unlimited (CLI-identical) + Offset int // honored where the backend supports it +} +// NOTE: MaxRows is NOT on the contract — the ready extraction as built keeps it CLI-side, and +// serve's --max-rows (open question 2) is an httpapi-layer operator clamp, not a query field. + +type ListRequest struct { + // field-for-field workapi.ListParams as built on refactor/bd-fv4-workapi-list-filter + // (Status..IDFilter strings; label trio + pattern/regex; *Contains fields; the ten + // *time.Time windows; EmptyDesc/NoAssignee/NoLabels/SkipLabels; priority trio; + // Pinned/Include* flags; ExcludeTypes; ParentID/NoParent; MolType/WispType; + // Deferred/Overdue; MetadataFields map[string]string; HasMetadataKey; AllFlag/ReadyFlag; + // SortBy/Reverse; Offset) with exactly TWO deltas: + Limit *int // nil = default 50; 0 = unlimited — REPLACES ListParams.SQLLimit; + // the sort-pushdown/fetch-all-and-trim decision moves INSIDE the impl + AfterCreatedAt *time.Time // decoded keyset position; token codec stays HTTP-layer + AfterID string +} + +type GetRequest struct { + ID string // exact canonical ID; issue-to-wisp fallback happens inside (ResolveIssue) + IncludeDependents bool // bd show --include-dependents; HTTP passes zero-value in v0 + IncludeComments bool // bd show --include-comments; HTTP passes zero-value in v0 +} + +type IssuePage struct { // one page type: ready and list both carry IssueWithCounts (rule 3) + Items []*types.IssueWithCounts + HasMore bool +} +``` + +`ReadyRequest` and `ListRequest` derive field-for-field from the workapi params structs the +extraction slices build; the deltas are exactly the three listed above (`Limit *int` replacing +default/SQLLimit mechanics, the keyset pair on `ListRequest`, `Priority *int` replacing the +`Priority`+`PrioritySet` pair). In the Reader slice the workapi builders are re-typed to accept +these leaf request types, so no duplicated params struct exists to drift. + +**Operation-to-method map.** + +| Operation | Reader method | +|---|---| +| `GET /v0/beads/ready` | `Reader.Ready` | +| `GET /v0/beads/issues` | `Reader.List` | +| `GET /v0/beads/issues/{id}` | `Reader.Get` | +| `GET /healthz` | none — process liveness, no DB touch | +| `GET /v0/beads/context` | none — contextinfo snapshot, not an issue query | +| `POST /v0/beads/issues/{id}:claim` | none — a WRITE; stays on `domain.IssueUseCase.ClaimIssue` via `workapi.ClaimOnUOW` inside caller-owned `uow.RunTxResult`, BELOW the facade (see claim-outcome typing) | + +**Accessors — exactly the Lifecycle pattern** (`IssueReader` grep-verified free in both trees; +no collision with the internal `IssueLifecycleStore` lane): + +```go +// internal/storage +type Storage interface { /* existing 71 */ + IssueLifecycle() (issueops.Lifecycle, error) + IssueReader() (issueops.Reader, error) +} +// internal/storage/dolt — implementation is NOT leaf; imports workapi freely +func (s *DoltStore) IssueReader() (issueops.Reader, error) { return &storeReader{st: s}, nil } +// internal/storage/embeddeddolt (cgo file; !cgo stub mirrors whatever IssueLifecycle does) +func (s *EmbeddedDoltStore) IssueReader() (issueops.Reader, error) { return &storeReader{st: s}, nil } +// internal/storage — reads fire no hooks: passthrough BY RECURSION, accessor exists for seam uniformity +func (h *HookFiringStore) IssueReader() (issueops.Reader, error) { return h.inner.IssueReader() } +// internal/telemetry — adds its layer by recursion (read spans; plain passthrough if none exist today) +func (s *InstrumentedStorage) IssueReader() (issueops.Reader, error) { + inner, err := s.Unwrap().IssueReader(); if err != nil { return nil, err } + return s.WrapIssueReader(inner), nil +} +// internal/storage/uow — the provider grows the same accessor; EACH METHOD OPENS EXACTLY ONE UOW, +// runs the whole workapi pipeline inside it, and closes (detached-close protection lives here). +func (p *) IssueReader() (issueops.Reader, error) { return &uowReader{p: p}, nil } +``` + +**Transactions: one UOW per Reader method.** Because the methods are request-granular, the uow +reader is one-UOW-per-call — which IS this proposal's one-UOW-per-request shape (this dissolves +the council's transaction-fragmentation objection, which targeted a fine-grained compose-shaped +Reader). The detached-close protection moves inside the uow reader. Implementation internals: + +- `uowReader.List`: `NewUOW` → `workapi.NewUOWConfigSource(uw)` → `LoadListConfig` → + `BuildListFilter` → `IssueUseCase().SearchIssuesWithCounts` → rewrap `domain.SearchCountsPage` + as `issueops.IssuePage`. +- `storeReader.List`: `workapi.NewStoreConfigSource(st)` → `LoadListConfig` → `BuildListFilter` + → limit+1 over-fetch (or fetch-all for DB-inexpressible sorts, the logic today at + `cmd/bd/list.go:42-76`) → trim + `HasMore`. +- `Ready`: `workapi.BuildReadyFilter` → `GetReadyWorkWithCounts` per seam; the store side + synthesizes `HasMore` by over-fetch, the uow side has it natively. +- `Get`: `workapi.BuildIssueDetails` over workapi's `NewStoreDetailSource` / `NewUOWDetailSource` + (unchanged from the detail extraction) inside the one UOW; not-found normalization (rule 4) + now sits doubly behind the accessor. + +`ConfigSource` is supplied by the implementation from what it already holds — +`NewStoreConfigSource(s.st)` in dolt/embedded, `NewUOWConfigSource(uw)` around the per-call UOW +— never by front doors, which is the drift kill: handlers and RunE bodies can only say +`rd.List(ctx, req)`; the skip-the-ritual failure mode is unwritable. + +### Package path, file layout, import policy + +`internal/workapi` — "the work-surface API both frontends delegate to". Verified free (no +collision; `internal/query` and `internal/storage/issueops` are taken). It stays `internal/` +PERMANENTLY as the shared implementation substrate behind every `Reader` — the room behind every +door. The public promotion is the `issueops.Reader` role (decided, not hypothetical — see "The +Reader role" above), never methods on `Lifecycle`; the builders are re-typed to accept the leaf +request types in the Reader slice so there is no duplicated params struct to drift. Its +vocabulary is facade-consistent (existing `storage.Err*` sentinels re-used, never new ones; +plain param structs; CAS-style outcomes). + +``` +internal/types/wire.go WireSchemaVersion = 1 (new; output.go:12 becomes = types.WireSchemaVersion) +internal/workapi/doc.go package rationale + import policy +internal/workapi/defaults.go DefaultReadyLimit, DefaultListLimit +internal/workapi/ready.go ReadyParams, BuildReadyFilter +internal/workapi/list.go ListParams, ListConfig, ConfigSource, NewStoreConfigSource, + NewUOWConfigSource, LoadListConfig, BuildListFilter (+ unexported + applyStatusFilter, validStatusList, parseMetadataFields) +internal/workapi/notfound.go IsNotFound, NotFound, IssueSource, ResolveIssue +internal/workapi/detail.go DetailSource, DetailOptions, BuildIssueDetails +internal/workapi/detail_store.go NewStoreDetailSource (over storage.DoltStorage) +internal/workapi/detail_uow.go NewUOWDetailSource (over uow.UnitOfWork) +internal/workapi/claim.go ClaimOutcome, ClaimErrorKind, ClaimConflictError, + ClassifyClaimError, ClaimOnUOW +internal/workapi/*_test.go moved list_filter_status_test.go + ready_input_test.go cases, + golden equality tables, new table-driven tests +``` + +cmd/bd deltas: `list_filter.go` deleted (moved); `ready.go` inline builder :107-218 deleted (the +direct path routes through `gatherReadyInput` as proxied already does at ready_proxied_server.go:18) +and `ready_input.go` thins to flags→`ReadyParams`; the `show.go`/`show_proxied_server.go` assembly +blocks and five `proxied*` detail helpers deleted in favor of `BuildIssueDetails`; +`update_proxied_server.go`'s bespoke backoff loop (:93-125) rewritten onto `uow.RunTxResult`; +`output.go` pins `JSONSchemaVersion` to the shared constant. + +Import policy (workapi imports exactly): +`internal/types`, `internal/storage` (sentinels, `ValidateMetadataKey`, `DoltStorage`), +`internal/storage/uow`, `internal/storage/domain` (`DepListFilter`), `internal/utils`, +`internal/config` (ONLY `GetCustomTypesFromYAML`, the workspace-scoped custom-types fallback that +`loadListFilterConfig` already performs at list_filter.go:91-95 — workspace config is meaningful in +a single-workspace server; client-cwd-derived reads like `GetDirectoryLabels` are banned), +`database/sql` (ErrNoRows). No cobra, no net/http, no `os.Getenv`. + +Enforcement is mechanical, matching the repo's existing idiom for the charter's storage boundary +(`.golangci.yml` depguard rule `dolt-storage-boundary`), not review-and-doc-comment: + +- **`.golangci.yml` depguard rule `workapi-frontend-boundary`**: for + `**/internal/workapi/**` files, deny `github.com/spf13/cobra` and `net/http` with a desc + pointing at this policy — the package is the shared substrate under both frontends and must + not know either exists. +- **`scripts/ci/pr-policy.sh` grep** for the symbol-level half depguard cannot express (workapi + legitimately imports `internal/config` for `GetCustomTypesFromYAML`): fail the PR if + `internal/workapi/` matches `config\.GetDirectoryLabels|os\.Getenv` — cwd/env-derived reads + are meaningless in a server process (rule 15). + +`doc.go` still states the rationale, but the boundary itself fails CI, not review attention. + +### Shared default constants (rule 9) + +```go +package workapi + +const ( + DefaultReadyLimit = 100 // was the literal in cmd/bd/ready.go:796 + DefaultListLimit = 50 // was the literal in cmd/bd/list.go:743 +) +``` + +Cobra flag registration changes to `IntP("limit", "n", workapi.DefaultReadyLimit, ...)` / +`workapi.DefaultListLimit`; the HTTP query decoder maps an absent `limit` to `nil`, and the builders +default `nil` to the same constants. `limit=0` means **unlimited on both surfaces, identically** +(rule 2): the builders pass 0 through untouched, and `WorkFilter.Limit==0` / `IssueFilter.Limit==0` +already mean unlimited at both seams. No clamp, no 400, anywhere in the contract. Alongside these, +`internal/types/wire.go` declares `const WireSchemaVersion = 1` — the one schema_version constant, +consumed by the CLI JSON envelope (output.go's `JSONSchemaVersion`) and `GET /v0/beads/context`. + +### Filter and default builders + +```go +// ready.go +type ReadyParams struct { + Assignee string + Unassigned bool + Type string // raw; Build normalizes via utils.NormalizeIssueType + ExcludeTypes []string // raw, comma-splittable; Build splits + normalizes + Labels []string // AND; Build normalizes via utils.NormalizeLabels + LabelsAny []string // OR; CLI pre-fills cwd directory labels, server never does + ExcludeLabels []string + LabelPattern, LabelRegex string + Priority *int // nil = unset (preserves P0-vs-unset, ready.go:179-182) + ParentID string + MolType string // Build validates via types.MolType.IsValid + MetadataFields []string // raw "k=v"; Build parses via parseMetadataFields + HasMetadataKey string // Build validates via storage.ValidateMetadataKey + Sort string // "" => hybrid; validated via types.SortPolicy.IsValid + Limit *int // nil => DefaultReadyLimit; 0 => unlimited (CLI-identical) + Offset int // honored under the UOW seam only (proxied CLI today) + IncludeDeferred bool + IncludeEphemeral bool + MaxRows int // direct CLI fills (resolveMaxRows); proxied rejects the flag; HTTP leaves 0 in v0 + MaxRowsSource string +} + +// BuildReadyFilter is the ONE place ready defaults exist. It hard-codes +// Status:"open" (today duplicated at ready.go:162 and ready_input.go:118 — +// and the reason the hosted viewer's empty WorkFilter silently meant +// open+in_progress). Also: label normalization, exclude-type splitting, +// metadata parsing/validation, sort validation, limit defaulting. +func BuildReadyFilter(p ReadyParams) (types.WorkFilter, error) +``` + +```go +// list.go +// ListParams is the field-for-field port of cmd/bd/list_input.go's listInput +// (:18-96) minus presentation-only fields (longFormat, prettyFormat, +// flatFormat, depsMode, watchMode, noPager, formatStr, jsonOutput, +// repoOverride, limitChanged, effectiveLimit), with exactly three changes: +// Limit *int // nil => DefaultListLimit; 0 => unlimited +// MetadataFields []string // raw "k=v"; parse moves from gatherListInput into Build +// AfterCreatedAt *time.Time; AfterID string // decoded keyset position -> +// // IssueFilter.AfterCreatedAt/AfterID (types.go:1618-1633) +// All other fields keep listInput's names (exported) and types verbatim: +// Status..IDFilter strings; label trio + pattern/regex; *Contains fields; the +// ten *time.Time windows; EmptyDesc/NoAssignee/NoLabels/SkipLabels; +// Priority/Min/Max (+Set flags); PinnedFlag/NoPinnedFlag/Include{Templates, +// Gates,Infra}; ExcludeTypeStrs; ParentID/NoParent; MolType/WispType; +// DeferredFlag/OverdueFlag; HasMetadataKey; AllFlag/ReadyFlag; Offset; SortBy/Reverse. +type ListParams struct { ... } + +type ListConfig struct { /* moved listFilterConfig: customStatuses, customTypes, infraSet + methods */ } + +type ConfigSource interface { // moved listFilterConfigSource, list_filter.go:48-52 + GetCustomStatuses(ctx context.Context) ([]types.CustomStatus, error) + GetCustomTypes(ctx context.Context) ([]string, error) + GetInfraTypes(ctx context.Context) (map[string]bool, error) +} +func NewStoreConfigSource(st storage.DoltStorage) ConfigSource // moved directConfigSource :54-64 +func NewUOWConfigSource(uw uow.UnitOfWork) ConfigSource // moved proxiedConfigSource :66-76 +func LoadListConfig(ctx context.Context, src ConfigSource) (ListConfig, error) // moved :78-106 + +// BuildListFilter is the verbatim move of buildListFilter (list_filter.go:119-314): +// closed + custom done/frozen exclusion, Pinned=false, IsTemplate=false, gate +// exclusion, infra-type exclusion, SkipWisps, with the same opt-outs +// (AllFlag, IncludeGates, IncludeInfra, IncludeTemplates). +func BuildListFilter(p ListParams, cfg ListConfig) (types.IssueFilter, error) +``` + +Builder error strings are the canonical copies of today's messages (e.g. `invalid sort policy '%s'. +Valid values: hybrid, priority, oldest`); the CLI wraps them with `HandleErrorRespectJSON("%v", err)` +so corpus error output stays byte-identical; the HTTP layer maps them to 400. `ConfigSource` is one +of only two interfaces in the package; both satisfy the rule-6 test — two implementations exist on +main today (list_filter.go:54-76). Post-reconciliation, HTTP handlers never wire a `ConfigSource`: +the `uowReader` supplies `NewUOWConfigSource(uw)` internally around its per-call UOW (§ 2, "The +Reader role"), and the handler only says `rd.List(ctx, req)`. + +### Not-found normalization (rule 4) + +The two seams miss differently, verified: the direct seam returns wrapped `storage.ErrNotFound` and +self-routes wisps (`DoltStore.GetIssue` → `storageissueops.GetIssueInTx`, +internal/storage/issueops/get_issue.go:18-35); the UOW seam's +`db.issueSQLRepositoryImpl.Get` returns `(nil, sql.ErrNoRows)` +(internal/storage/domain/db/issue.go:394-410) which the domain layer wraps with `"get %s: %w"` +(domain/issue.go:387-396) — it is NEVER `storage.ErrNotFound`. The CLI's own proxied code already +treats both sentinels as not-found (`uowIssueExists`, cmd/bd/create_proxied_server.go:495-499). +The contract makes that the law: + +```go +// notfound.go + +// IsNotFound reports the miss condition across both seams: +// wrapped storage.ErrNotFound (direct) or wrapped sql.ErrNoRows (UOW). +func IsNotFound(err error) bool { + return errors.Is(err, storage.ErrNotFound) || errors.Is(err, sql.ErrNoRows) +} + +// NotFound is the canonical normalized miss: errors.Is(NotFound(id), storage.ErrNotFound). +func NotFound(id string) error { return fmt.Errorf("%w: issue %s", storage.ErrNotFound, id) } + +// IssueSource is the resolution slice of both seams. DetailSource embeds it. +type IssueSource interface { + GetIssue(ctx context.Context, id string) (*types.Issue, error) + GetWisp(ctx context.Context, id string) (*types.Issue, error) +} + +// ResolveIssue is THE shared exact-ID issue→wisp resolution (the +// proxiedGetIssueOrWisp protocol, show_proxied_server.go:127-140, promoted): +// GetIssue; on miss — IsNotFound(err) OR (nil issue, nil err) — try GetWisp; +// on double miss return (nil, false, NotFound(id)). A non-miss error passes +// through UNCHANGED and is never normalized to not-found: a dropped +// connection can never read as 404, nor a missing issue as 500. +func ResolveIssue(ctx context.Context, src IssueSource, id string) (issue *types.Issue, isWisp bool, err error) +``` + +Normalization lives HERE, in the shared function every caller goes through — the HTTP get/claim +handlers, `BuildIssueDetails`, and the migrated proxied show/update paths — not in the HTTP error +mapper alone. The mapper keys 404 exclusively on `errors.Is(err, storage.ErrNotFound)`, the only +way a miss can surface after normalization. `isWisp` is a routing hint for seams with per-table +APIs (UOW); the direct adapter self-routes and ignores it (its `GetIssue` already probes both +tables; its `GetWisp` returns `NotFound(id)` immediately, reporting `isWisp=false`). + +### Shared detail assembly (rule 13) + +```go +// detail.go +type DetailSource interface { + IssueSource + Labels(ctx context.Context, id string, isWisp bool) ([]string, error) + // Dependencies: outgoing edges, full items with dep metadata. + Dependencies(ctx context.Context, id string, isWisp bool) ([]*types.IssueWithDependencyMetadata, error) + // DependentsShallow: incoming edges projected to the 5-field shallow shape + // (ID, Status, IssueType, Priority, Title) + DependencyType — the projection + // both existing paths already emit (show.go:168-181, show_proxied_server.go: + // 500-512); part of the adapter contract so the direct side keeps streaming via Iter. + DependentsShallow(ctx context.Context, id string, isWisp bool) ([]*types.IssueWithDependencyMetadata, error) + CountDependents(ctx context.Context, id string, isWisp bool) (int64, error) + CountDependencies(ctx context.Context, id string, isWisp bool) (int64, error) + CountComments(ctx context.Context, id string, isWisp bool) (int64, error) + Comments(ctx context.Context, id string, isWisp bool) ([]*types.Comment, error) +} + +func NewStoreDetailSource(st storage.DoltStorage) DetailSource +func NewUOWDetailSource(uw uow.UnitOfWork) DetailSource + +type DetailOptions struct { + IncludeDependents bool // bd show --include-dependents (CLI); HTTP passes zero-value opts in v0 + IncludeComments bool // bd show --include-comments (CLI); HTTP passes zero-value opts in v0 +} +// The HTTP GET /v0/beads/issues/{id} handler passes a zero-value GetRequest to +// Reader.Get in v0 (the uow reader calls BuildIssueDetails with DetailOptions{}) +// — mirroring bd show's count-only default; no include_* +// query params ship (§ 3 exposes none; open question 3 covers a comments knob). + +// BuildIssueDetails is the single assembly: ResolveIssue → Labels → +// Dependencies → three counts (count-only default, be-ijck6q) → optional +// DependentsShallow + epic progress (EpicTotal/Closed/Closeable from +// parent-child deps) → optional Comments, else CommentsOmitted=true iff +// CommentCount>0 (ga-clgh) → Parent = first DepParentChild dependency. +// A miss returns the normalized NotFound error. +func BuildIssueDetails(ctx context.Context, src DetailSource, id string, opts DetailOptions) (*types.IssueDetails, error) +``` + +Adapter method mapping — every row grep-verified on main (rule 14), no invented capability: + +| DetailSource method | Store adapter (self-routing; ignores isWisp) | UOW adapter (explicit axis) | +|---|---|---| +| GetIssue / GetWisp | `st.GetIssue` (GetIssueInTx probes issues then wisps) / immediate NotFound | `IssueUseCase().GetIssue` / `GetWisp` | +| Labels | `st.GetLabels` (GetLabelsInTx table="" self-routes, internal/storage/issueops/labels.go:14-18) | `LabelUseCase().GetLabels` / `GetWispLabels` | +| Dependencies | `st.GetDependenciesWithMetadata` (isActiveWisp self-route, dolt/dependencies.go:156-159) | `DependencyUseCase().ListWithIssueMetadata` / `ListWispWithIssueMetadata`, `DepListFilter{Direction: Out}` | +| DependentsShallow | `st.IterDependentsWithMetadata` + shallow projection (today's show.go:161-186 moved) | `ListWithIssueMetadata`/wisp variant, `Direction: In`, + projection (today's :498-512 moved) | +| CountDependents / CountDependencies | `st.CountDependents` / `CountDependencies` (aggregate permanent+wisp tables, dolt/counts.go:60-71) | `CountByIssueID` / `CountByWispID` with `Direction: In` / `Out` | +| CountComments | `st.CountIssueComments` | `CommentUseCase().CountCommentsForIssue` / `CountCommentsForWisp` | +| Comments | `st.IterIssueComments` materialized (today's show.go:210-223 moved) | `GetCommentsForIssue` / `GetCommentsForWisp` | + +Consumers after migration: direct `bd show` (adapter over its routed `result.Store`), proxied +`bd show` (UOW adapter; `proxiedBuildDetails` and the five helpers deleted), and the HTTP +`GET /v0/beads/issues/{id}` handler — three consumers, one assembly. CLI-only concerns stay +outside: fuzzy resolution feeds the exact ID in, the array-of-details envelope wraps the result, +and `--thread`/`--refs`/`--children`/`--as-of`/watch mode are untouched. + +### Claim-outcome typing (and the one retry/commit implementation, rule 5) + +The CAS already has one implementation per seam with identical semantics — +`domain.(*issueUseCaseImpl).claim` (issue.go:458-492) and `storageissueops.ClaimIssueInTx` +(`internal/storage/issueops`, the INTERNAL package — distinct from the public leaf `issueops`; +the facade branch's `storageissueops` import-alias convention is adopted in prose throughout) — +both wrapping `storage.ErrAlreadyClaimed`/`ErrNotClaimable` (post-facade, the canonical +declarations live in the leaf `issueops` package via the errors relocation, aliased back from +`internal/storage` with the same pointer — `errors.Is` keys on `storage.Err*` keep working +unedited). The direct seam's +`ClaimIssue(ctx, id, actor) error` (bulk_issues.go:16) yields only sentinels; the domain seam +returns `ClaimResult{AlreadyClaimed, PriorAssignee}` (issue.go:222-225). The contract adds typing +and one per-attempt helper — no transaction lifecycle: + +```go +// claim.go +type ClaimOutcome struct { + Issue *types.Issue // fresh same-transaction re-read after the CAS landed + AlreadyYours bool // domain ClaimResult.AlreadyClaimed: idempotent same-actor re-claim +} + +type ClaimErrorKind int +const ( + ClaimErrNone ClaimErrorKind = iota + ClaimErrNotFound // errors.Is storage.ErrNotFound (post-normalization) + ClaimErrAlreadyClaimed // errors.Is storage.ErrAlreadyClaimed + ClaimErrNotClaimable // errors.Is storage.ErrNotClaimable + ClaimErrWriteConflict // uow.IsSerializationError (retries exhausted) +) + +// ClassifyClaimError is the errors.Is dispatch both surfaces use — the typed +// replacement for downstream substring matching on "already claimed by". +func ClassifyClaimError(err error) ClaimErrorKind + +// ClaimConflictError carries structured Holder/Status for a lost CAS, read +// back in the SAME transaction — never parsed from message text. Unwrap() +// returns the sentinel chain, so errors.Is(err, storage.ErrAlreadyClaimed) +// keeps working; CLI output never sees this type and is unaffected. +type ClaimConflictError struct { + Kind ClaimErrorKind // ClaimErrAlreadyClaimed | ClaimErrNotClaimable + Holder string // conflicting assignee ("" for not_claimable) + Status string // issue status at conflict time + Err error +} + +// ClaimOnUOW is ONE ATTEMPT over one open UOW: ResolveIssue (shared dispatch +// + not-found normalization), then the matching CAS (IssueUseCase().ClaimIssue +// or ClaimWisp — the domain methods that expose AlreadyClaimed, which +// ApplyUpdate discards), then a fresh same-tx re-read for the response body. +// On a lost CAS it re-reads the row and returns *ClaimConflictError. +// NO retry, NO commit: callers run it inside uow.RunTxResult. +func ClaimOnUOW(ctx context.Context, uw uow.UnitOfWork, id, actor string) (ClaimOutcome, error) +``` + +The retry/commit protocol has exactly one implementation: **`uow.RunTxResult`** +(internal/storage/uow/tx.go:76-121 — serialization-error retry with backoff, permanent-error +short-circuit, nothing-to-commit tolerance, test-covered on main), already how proxied +`bd ready --claim` commits (ready_proxied_server.go:184-199). The HTTP claim handler is +`uow.RunTxResult(ctx, provider, func(ctx, uw) (workapi.ClaimOutcome, string, error) { o, err := +workapi.ClaimOnUOW(ctx, uw, id, actor); ...; return o, "bd serve: claim by ", nil })`. +The migration slice rewrites `applyUpdateProxiedOne`'s bespoke `cenkalti/backoff` loop +(update_proxied_server.go:93-125) as `uow.RunTxResult` around the existing attempt body — after +which the bespoke loop is deleted and cannot drift because it does not exist. No `ClaimViaUOW` +wrapper is introduced (rule 5). `bd update --claim` keeps `ApplyUpdate(spec{Claim:true})` for +atomic claim+field updates; its per-attempt resolve (the GetIssue→GetWisp fallback at +update_proxied_server.go:146-158) migrates onto `ResolveIssue` so wisp dispatch is shared too. +Honest residue: `ApplyUpdate` internally decides wispness via `isWispID` (domain/issue.go:499) — a +pre-existing second wispness decision inside the domain layer, out of scope here; flagged as gap G5. + +**Post-facade justification: why the claim endpoint stays BELOW `issueops.Lifecycle`** (recorded +so the choice is not re-litigated when the facade merges). Verified on the facade branch: +(a) `Lifecycle` verbs run their own `RunTxResult` with a fixed `"update issue"` commit message — +routing claim through `Lifecycle.Update{Claim: true}` would double-wrap the retry and lose the +`"bd serve: claim by "` commit message; (b) `ApplyUpdate` discards domain +`ClaimResult.AlreadyClaimed`, which the wire's `already_claimed` field maps; (c) on a lost CAS +the facade returns only the sentinel with the transaction already rolled back, making the 409 +`assignee`/`issue_status` same-transaction re-read extensions unreachable (message parsing is +forbidden); (d) per-issue commit granularity is moot for a single-issue claim — one commit +either way; (e) the facade's governing rule constrains the PUBLIC capability surface, not +internal seam consumers — `bd serve`'s claim rides the same seam proxied `bd update --claim` +uses today, and the facade itself defers the proxied rewire. + +### Result-shaping vocabulary (what the wire carries, verified) + +- **Ready items are `[]*types.IssueWithCounts`** (rule 3): direct `bd ready --json` at + ready.go:266, proxied at ready_proxied_server.go:98. HTTP `/ready` carries the same items or it + drifts at the field level (dependency_count/dependent_count/comment_count/parent, + types.go:976-982) on day one. +- **List items are ALSO `IssueWithCounts`**: `bd list --json` calls `SearchIssuesWithCounts` on + both paths (cmd/bd/list.go:576-578, list_proxied_server.go:79). `/issues` items are + `IssueWithCounts`; a bare-`Issue` list would fail the full-item-JSON oracle immediately. +- **Detail is `types.IssueDetails`** (types.go:984-1013) from `BuildIssueDetails`, a single object + over HTTP (the array is a CLI multi-arg envelope). +- **`skip_labels`/lite is not in the contract**: `IssueFilter.SkipLabels` sets `Issue.IsLitePartial` + which is `json:"-"` — wire-indistinguishable from an empty issue. The CLI keeps + `newSkipLabelsListJSONResponse` locally; HTTP does not expose it in v0 (consequence for the + one downstream `--skip-labels` caller, the reconciler full scan: § 3, "what this list + surface deliberately cannot serve"). +- All three item types are marshaled from the canonical Go structs directly — the spec pins them via + `x-go-type` (rule 1; mechanics in the spec/codegen section). No `convert.go`, no `apigen.Issue` + mirror, no Go-to-Go mapping anywhere. + +### What enforces CLI/HTTP parity — mechanism by mechanism + +An unenforced convention is not a guarantee. For each shared behaviour, the concrete artifact: + +| # | Shared behaviour | Enforcing artifact (type / function / named test) | +|---|---|---| +| 1 | Ready defaults (Status:"open", hybrid sort, normalization) | `workapi.BuildReadyFilter` — single implementation; both old builders deleted, so a second copy is a compile-time impossibility. Golden test `TestBuildReadyFilterGolden` (tables written FROM the old builders BEFORE deletion) pins old==new across an input matrix. | +| 2 | List exclusion defaults (~200 lines: closed/done/frozen, pinned, templates, gates, infra, wisps) | `workapi.BuildListFilter` — verbatim move, old file deleted; moved suite `list_filter_status_test.go` + golden test `TestBuildListFilterGolden`. | +| 3 | Default page sizes (ready 100, list 50) | `DefaultReadyLimit`/`DefaultListLimit` constants referenced by BOTH the cobra flag registrations and the HTTP param decoder (compile-time), plus `TestLimitDefaults_SingleSource` asserting each cobra `Flag.DefValue == strconv.Itoa(const)` (DefValue is a string snapshot and could otherwise drift). | +| 4 | `limit=0` = unlimited on both surfaces | Builders pass 0 through; seam semantics already unlimited. Enforced by the `limit=0` cases of `TestServerListCursorPagination` and the Tier-C parity oracles `TestProxiedServerServeReadyParity`/`TestProxiedServerServeListParity` (§ 5): HTTP `limit=0` output equals `bd list --limit 0 --json` / `bd ready --limit 0 --json` against seeded Dolt. | +| 5 | Wire item shape (Issue / IssueWithCounts / IssueDetails) | The canonical structs themselves via `x-go-type` (no second wire struct, rule 1) + two-way JSON-tag bijection test `TestWireTagBijection` (spec properties ↔ struct tags, set-equality both directions, explicit in-test allowlist) — a fixture round-trip is NOT relied on. Runs in the required PR-time path. | +| 6 | Not-found semantics (404 iff missing; 500 never masks as 404 and vice versa) | `workapi.ResolveIssue` + `IsNotFound` — normalization inside the shared function every caller uses, so no future caller can reintroduce the bug; HTTP mapper keys only on `errors.Is(err, storage.ErrNotFound)`. Named integration test `TestServerGetUnknownID404` against real Dolt (UOW seam, where the raw miss is wrapped `sql.ErrNoRows`) + unit `TestResolveIssue_MissShapes` covering {wrapped ErrNoRows, wrapped ErrNotFound, nil-issue-nil-error, non-miss error passthrough}. | +| 7 | Detail assembly (labels, deps-with-metadata, counts, comments-omitted, epic progress, parent) | `workapi.BuildIssueDetails` — three consumers, one implementation, old blocks deleted (compile-time). Full-item parity test `TestProxiedServerServeShowParity`: `GET /issues/{id}` JSON equals `bd show --json`[0] modulo documented allowlist; fixture set MUST include a wisp, an epic with children, and an issue with comments (count-only path). | +| 8 | Claim CAS semantics + conflict taxonomy | Single CAS in the domain / `internal/storage/issueops` layer (existing, per-seam); `workapi.ClaimOnUOW` + `ClassifyClaimError` for outcome/conflict typing; retry/commit solely `uow.RunTxResult` (bespoke loop deleted — compile-time). E2E `TestProxiedServerServeClaimCrossCheck`: HTTP claim then `bd update --claim` by another actor yields the same classification both ways; concurrent-claim race asserts exactly one 200; plus the rule-8 item-level oracle — `ClaimResponse.issue` full-item-JSON vs a same-actor `bd update --claim --json`[0] (§ 5, test 18). | +| 9 | schema_version reported by CLI envelope and `/context` | `types.WireSchemaVersion` consumed by both (compile-time); protocol corpus pins the CLI value byte-level. | +| 10 | Full-surface parity oracle (rule 8) | `TestProxiedServerServeReadyParity` and `TestProxiedServerServeListParity`: decoded-map equality of FULL item JSON (not ID sets) between HTTP and CLI for the same filter matrix, modulo an explicit, in-test, commented allowlist. Plus `TestParityMatrix_CoversAllParams`: reflection over `ReadyParams`/`ListParams` fields asserting every field appears in the matrix registry, so adding a param without oracle coverage fails CI. | +| 11 | workapi frontend-boundary import policy (rule 15) | `.golangci.yml` depguard rule `workapi-frontend-boundary` (deny cobra + net/http in `internal/workapi`, same idiom as `dolt-storage-boundary`) + `scripts/ci/pr-policy.sh` grep banning `config.GetDirectoryLabels`/`os.Getenv` in the package (§ 2, import policy). Fails CI, not review. | + +**Honest gaps — conventions that remain unenforced (say so, per the mandate):** + +- **G1 — query-param ↔ flag-name correspondence.** Nothing machine-checks that HTTP `label_any` + maps to the same `ReadyParams` field as `--label-any`. The matrix-completeness test (row 10) + catches a mis-wired mapping by value divergence, but the param SPELLING is convention + review. +- **G2 — cobra help-text numerals** ("default 50" in usage strings) are not tied to the constants. +- **G3 — builder error strings** double as corpus-pinned CLI output and HTTP 400 `detail`. The + corpus enforces the CLI side only; acceptable because `detail` is prose, `code` is the contract. +- **G4 — overflow detection** stays two implementations by design (direct `withFetchOneExtra` vs + UOW `HasMore`); only the parity oracle's truncation cases tie them together. +- **G5 — wisp dispatch**: `ResolveIssue` is shared by get/detail/claim, but `ApplyUpdate`'s + internal `isWispID` remains a second decider in the domain layer (pre-existing; untested-for). +- **G6 — `bd ready --claim` JSON shaping** (`buildReadyIssueOutput` vs + `buildReadyIssueOutputProxied`) stays duplicated in cmd/bd; claim-ready is outside the MVP + surface — documented residual drift, not silently absorbed. +- **G7 — list ORDER divergence, and oracle blindness to it.** HTTP `/v0/beads/issues` is + always `(created_at DESC, id ASC)` (welded to the cursor, § 3); `bd list`'s default is the + priority ordering (sqlbuild/sort.go:70-71). This is a deliberate, documented divergence in + ordering only — item set and item JSON stay identical, so the rule-8 parity oracles, which + sort both sides by id before comparing, are structurally unable to detect ANY ordering + regression. Ordering is carried solely by `TestServerListCursorPagination`'s first-page + ordering assertion; it is not on the parity allowlist because the oracles never see it. + +### Migration deltas the extraction slices must triage (not silently absorb) + +Golden tables (rows 1-2 above) are written from the OLD code before deletion. The two ready +builders differ today in exactly two fields, and the unified `ReadyParams` carries both with the +policy unchanged and stated in the extraction PR: `MaxRows`/`MaxRowsSource` exist only in the +direct inline copy (proxied bulk ready REJECTS an explicit cap, ready.go:57-70, as list does at +list.go:522 — seam policy that stays in cmd/bd); `Offset` exists only in `gatherReadyInput` +(proxied-only, rejected on direct list at list.go:531). Any further delta the golden tables surface +is triaged in review — the extraction slices are otherwise no-op refactors, verified against the +byte-pinned cmd/bd/protocol corpus, landing separately per rule 11. + +## 3. The wire: OpenAPI document, schemas, errors, pagination + +### The document + +The spec lives at `internal/httpapi/spec/openapi.v0.yaml` — OpenAPI 3.0.3, hand-written, the +source of truth per decided parameter 2. `internal/httpapi/spec/embed.go` embeds it +(`//go:embed openapi.v0.yaml`) so the parity and bijection tests parse the same bytes that ship. +Its `info.description` declares it spec-first and vendor-neutral (no product names, no `x-gc-*` +extensions — the marker is `x-bd-source: spec-first`). + +Codegen follows the hosted-proven arrangement exactly: `oapi-codegen` v2.6.0, types-only, no +server stubs, no router, no YAML config. `internal/httpapi/apigen/doc.go` carries + +```go +//go:generate go tool oapi-codegen -generate types,skip-prune -package apigen -o types.gen.go ../spec/openapi.v0.yaml +``` + +and `go.mod` (module is on Go 1.26.5, so `tool` directives are supported) gains +`tool github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen` pinned at v2.6.0. Generated +output is committed at `internal/httpapi/apigen/types.gen.go`; the CI drift gate +(regenerate → `git diff --exit-code -- internal/httpapi/apigen/`) runs in the PR-time policy +job, never in the concurrency-cancelled `Main` workflow. + +### Schema pinning: no second wire struct (one-way door) + +Every schema that already has a canonical Go wire type is pinned to it with `x-go-type` / +`x-go-type-import`, so oapi-codegen emits a type alias (`type Issue = types.Issue`) instead of a +mirror struct: + +| Spec schema | Pinned Go type | Verified at | +|---|---|---| +| `Issue` | `types.Issue` | internal/types/types.go:20-171 (JSON tags) | +| `IssueWithCounts` | `types.IssueWithCounts` | types.go:976-982 | +| `IssueDetails` | `types.IssueDetails` | types.go:986-1013 | + +Nested wire types (`Dependency`, `Comment`, `IssueWithDependencyMetadata`) ride along inside the +pinned structs and are documented as sub-schemas of the pinned ones; they are never re-declared +as generated Go types. There is **no hand-written `toAPI()` mapping and no generated Go struct +that mirrors `types.Issue` anywhere** — the CLI's `--json`, JSONL interchange, and HTTP all +marshal the one canonical struct. This is what makes a future `types.Issue` field addition +appear on the HTTP wire automatically instead of silently vanishing (the hosted viewer's drift +disease). + +Consequence, marked explicitly: **pinning is a one-way door in both directions.** `types.Issue`'s +JSON encoding *is* the HTTP contract from v0 onward; removing or renaming a serialized field on +`types.Issue` becomes an HTTP breaking change, not just a CLI one. That is the intended +trade — one contract, one compatibility domain. + +**Evolution policy for the welded domain (while `/v0` is live).** The repo already has a +sanctioned deliberate-change protocol for the CLI wire: `make corpus-regen` plus a +`JSONSchemaVersion` bump (§ 4, CLI migration mechanics). After pinning, that same maneuver would +silently change `/v0` responses for clients pinned to the path, with only a `schema_version` +field in `/context` they have no reason to re-poll. So the policy is stated here, not left to +discovery: + +- **Additive-only while `/v0` is the newest path version.** Changes to the `x-go-type`-pinned + structs (`Issue`, `IssueWithCounts`, `IssueDetails` and their ride-along sub-types) are limited + to new `omitempty` fields. Each addition must land with its spec `properties` entry in the same + PR — the bijection test makes forgetting a compile-visible CI failure — and appears on both + surfaces simultaneously, which is the feature, not a leak. +- **A `WireSchemaVersion` bump is forbidden while `/v0` is the only HTTP path version.** A bump + is by definition a breaking change to the pinned schemas, and a single-struct domain cannot + serve two shapes from one path. The sanctioned breaking-change maneuver becomes: cut `/v1` + (new spec document, new path prefix) in the same change that bumps `WireSchemaVersion` and + regenerates the corpus. +- **What happens on that day, mechanically:** the bijection test re-binds the canonical structs + to the *newest* path version's spec (the test's "no second wire struct" rule is scoped to the + live version). `/v0` either (a) is deprecated and removed on a documented sunset, or (b) is + frozen by snapshotting the pre-bump shapes as a deliberate, version-suffixed mirror struct + served only by `/v0` handlers — a frozen mirror cannot drift, so a round-trip fixture (adequate + for a shape that can no longer gain fields) replaces the bijection test for it. Choosing (a) + vs (b) is deferred to that day (recorded as open question 8); what is NOT deferred is that the + corpus-regen + `JSONSchemaVersion`-bump procedure alone is no longer sufficient once `/v0` ships. + +Because `x-go-type` means the spec's property list is documentation rather than +compiler-enforced, the guard is a **two-way JSON-tag bijection test** +(`internal/httpapi/wire_bijection_test.go`): reflect over the JSON tags of `types.Issue`, +`types.IssueWithCounts`, and `types.IssueDetails` and assert exact set-equality with the spec's +`Issue`/`IssueWithCounts`/`IssueDetails` `properties`. Fields tagged `json:"-"` (`ContentHash`, +`RowVersion`, `SourceRepo`, `IDPrefix`, `PrefixOverride`, `IsLitePartial` — types.go:22,89,111-113,170) +carry no wire name and are excluded by construction. The deliberate-omission allowlist **starts +empty**; any future entry requires an in-test comment justifying the omission. A round-trip +fixture test is NOT an acceptable substitute: a new `omitempty` field absent from the fixture +round-trips cleanly in both directions and the drift ships silently. The bijection test runs in +the required PR-time path (`go test ./internal/httpapi/...` under `ci-pr-core`), because with +pinning it is the only guard on the Issue schema. + +Envelope schemas that have no pre-existing canonical Go type — `ReadyPage`, `IssuesPage`, +`ClaimRequest`, `ClaimResponse`, `ContextResponse`, `Health`, `Problem` — are spec-native and +generated normally. They are new wire surface, not mirrors, so rule "no second wire struct" does +not apply to them; drift is impossible because the generated type is the only implementation. + +### The six operations + +All success bodies are `application/json; charset=utf-8`; every non-2xx body from any handler is +`application/problem+json` (§ error envelope). Base path has no `{project}` segment +(single-tenant, decided). The claim path is declared in the spec as the literal AIP-136-style +`/v0/beads/issues/{id}:claim`; the route-parity test carries the explicit spec-path ↔ mux-pattern +mapping (router mechanics: § 4, "Server lifecycle and security"). + +| operationId | Method + path | Request | 200 body | Documented non-2xx | +|---|---|---|---|---| +| `health` | `GET /healthz` | — | `Health` `{"status":"ok"}` (process LIVENESS only, no DB touch; readiness is probed via `/v0/beads/ready?limit=1` — § 4, concurrency limits) | — | +| `getContext` | `GET /v0/beads/context` | — | `ContextResponse` | 500 | +| `listReadyWork` | `GET /v0/beads/ready` | query → `types.WorkFilter` (table below) | `ReadyPage` | 400, 500, 503 | +| `listIssues` | `GET /v0/beads/issues` | query → `types.IssueFilter` (table below) | `IssuesPage` | 400, 500, 503 | +| `getIssue` | `GET /v0/beads/issues/{id}` | path `id`: exact canonical ID, no fuzzy/substring resolution; wisp fallback | `IssueDetails` | 404, 500, 503 | +| `claimIssue` | `POST /v0/beads/issues/{id}:claim` | body `ClaimRequest` `{"actor": ""}` | `ClaimResponse` | 400, 404, 409, 500, 503 | + +One code applies to **every** route, `/healthz` included, and is documented once in the spec +rather than repeated per row: the middleware-level 400 `invalid_argument` for a foreign `Host` +header (§ 4, "Browser exposure and DNS rebinding") — it runs before any handler. + +Per rule 12, **only** the codes above are documented. Deliberately NOT in the spec: 413 (an +oversized claim body tripping `http.MaxBytesReader` maps to 400 `invalid_argument`), 429/504 +(the connection cap and semaphore QUEUE rather than refuse; a semaphore wait that times out +surfaces as the already-documented 503 `busy`, and a tripped per-request deadline as the generic +500 — § 4 — so neither new code is ever emitted), 403 `read_only` (no read-only serve +mode ships in v0), 501 (`*storage.ErrUnsupported` is a startup refusal, not a request-time +response). Router-level 404/405 for unregistered paths/methods are emitted in the problem +envelope for uniformity but are not per-operation spec surface. + +**Unknown query parameters are rejected, on every operation.** With oapi-codegen generating +types only — no server stubs, no generated request validation — the query decoders in +`internal/httpapi/params.go` are hand-written over stdlib `url.Values`, and Go's default is to +silently ignore any key the decoder does not read. That default is banned, in the spec and in +the decoders: each decoder iterates the request's raw query keys, and any key outside the +operation's parameter table → 400 `invalid_argument` with extension members +`param: ""` and `reason: "unknown_parameter"` (§ error envelope). Operations +with no documented query parameters (`/healthz`, `/context`, get, claim) reject every query key +the same way. This is not pedantry: a silently ignored FILTER parameter widens the result set. +The concrete skew failure — a client one version ahead sends a newly added filter (the +"additive later" params the tables below name, e.g. `mol_type`); an older server drops it; a +dispatch probe that asked for "unassigned, routed to me" gets back work routed to someone else — is the +hosted viewer's bare-`IssueFilter{}` leak class all over again, triggered by version skew +instead of a bare struct. Strict rejection is also the only reliable per-parameter capability +probe a client has, since `capabilities` is operation-level (§ version skew). Enforced by +Tier-A test 8c. + +#### `GET /v0/beads/ready` — query parameters → `types.WorkFilter` (types.go:1812-1872) + +| Param | Type/format | WorkFilter field | Notes | +|---|---|---|---| +| `assignee` | string | `Assignee *string` | | +| `unassigned` | bool | `Unassigned` | | +| `type` | string | `Type` | normalized via `utils.NormalizeIssueType` in the shared builder | +| `exclude_type` | repeated, comma-splittable | `ExcludeTypes []IssueType` | ignored when `type` set (WorkFilter contract) | +| `label` | repeated | `Labels` | AND semantics | +| `label_any` | repeated | `LabelsAny` | OR semantics | +| `exclude_label` | repeated | `ExcludeLabels` | | +| `label_pattern` | string | `LabelPattern` | glob | +| `label_regex` | string | `LabelRegex` | | +| `priority` | int | `Priority *int` | | +| `parent` | string | `ParentID *string` | recursive descendants | +| `metadata_field` | repeated `key=value` (split on first `=`) | `MetadataFields map[string]string` | keys validated via `storage.ValidateMetadataKey` (metadata.go:216) | +| `has_metadata_key` | string | `HasMetadataKey` | | +| `include_ephemeral` | bool | `IncludeEphemeral` | Gas City passes `--include-ephemeral` | +| `include_deferred` | bool | `IncludeDeferred` | | +| `sort` | `hybrid`\|`priority`\|`oldest` | `SortPolicy` | validated via `types.SortPolicy.IsValid` (types.go:1746-1758); empty → hybrid | +| `limit` | int ≥ 0 | `Limit` | absent → shared ready-default constant = 100 (cmd/bd/ready.go:796); `0` → unlimited (rule 2) | + +Not exposed: `offset` (a proxied-CLI-ism), `max_rows`/`max_rows_source` (operator knob, not wire), +`molecule_id`/`mol_type`/`wisp_type` (no consumer; additive later). `Status`/`Statuses` are not +params: the shared builder hard-codes `Status:"open"` — the empty-WorkFilter +open+in_progress default is unreachable from the wire. + +**Response `ReadyPage` = `{items: [IssueWithCounts], has_more: bool}` (rule 3).** `bd ready +--json` emits `[]*types.IssueWithCounts` on both CLI paths (cmd/bd/ready.go:246,299,509; +ready_proxied_server.go:181-210,441-471), so the HTTP items carry `dependency_count`, +`dependent_count`, `comment_count`, `parent` from day one — executed via +`IssueUseCase.GetReadyWorkWithCounts` → `domain.SearchCountsPage` (domain/issue.go:140-142,261). +`has_more` comes from `SearchCountsPage.HasMore`. No cursor on ready: the SortPolicy orders +(hybrid/priority) admit no keyset predicate and the consumer pattern is snapshot-and-requery on a +patrol cadence. Adding a cursor later under a fixed sort is additive — not a one-way door. + +#### `GET /v0/beads/issues` — query parameters → `types.IssueFilter` (types.go:1584-1736) + +Constructed exclusively through the shared `buildListFilter` extraction, so the default +exclusions are byte-identical to `bd list` (cmd/bd/list_filter.go:119-143,240-261,302-310) and the +hosted viewer's bare-`IssueFilter{}` leak is structurally impossible. + +| Param | Type/format | IssueFilter field / builder behavior | Notes | +|---|---|---|---| +| `status` | comma-separated or repeated | `Status` (single) / `Statuses` (multi) via `applyStatusFilter` | custom statuses honored | +| `type` | string | `IssueType *IssueType` | | +| `assignee` | string | `Assignee *string` | | +| `label` / `label_any` / `exclude_label` | repeated | `Labels` / `LabelsAny` / `ExcludeLabels` | AND / OR / NOT | +| `parent` | string | `ParentID *string` | | +| `all` | bool | disables the default `ExcludeStatus` construction (closed + custom done/frozen categories) | default false ⇒ exclusions apply | +| `include_templates` | bool | default false ⇒ `IsTemplate=&false` | | +| `include_gates` | bool | default false ⇒ `"gate"` appended to `ExcludeTypes` | | +| `include_infra` | bool | default false ⇒ configured infra types appended to `ExcludeTypes` | | +| `created_before` / `created_after` | RFC3339 / RFC3339Nano | `CreatedBefore` / `CreatedAfter *time.Time` | Gas City passes RFC3339Nano | +| `metadata_field` | repeated `key=value` | `MetadataFields` | | +| `has_metadata_key` | string | `HasMetadataKey` | | +| `limit` | int ≥ 0 | `Limit` | absent → shared list-default constant = 50 (cmd/bd/list.go:743); `0` → unlimited (rule 2) | +| `cursor` | opaque string | `AfterCreatedAt` / `AfterID` | § pagination; sort forced on EVERY request, cursored or not (below) | + +**Order is forced on every request, not only under a cursor.** The handler sets +`SortBy="created"`, `SortDesc=false` unconditionally on every list request. This is load-bearing: +`buildListFilter` passes the caller's sort straight through (list_filter.go:123-124), and an empty +`SortBy` renders `ORDER BY priority , created DESC, id ASC` +(internal/storage/sqlbuild/sort.go:70-71) — so forcing the order only when a cursor is present +would make page 1 priority-ordered while its `next_cursor` encodes a `(created_at DESC, id ASC)` +position, and page 2 would skip and duplicate rows relative to page 1. Consequence, stated in the +spec's endpoint description as deliberate: **HTTP list order diverges from `bd list`'s default +priority ordering** — every `/v0/beads/issues` page is `(created_at DESC, id ASC)`. This is an +ordering-only divergence (same item set, same item JSON); see gap G7 for why the parity oracles +are structurally blind to it and which test carries it instead. + +Default-off wisp merge (`SkipWisps=true`, list_filter.go:310) applies exactly as in the CLI. Not +exposed in v0: `sort`/`reverse` (welded to the cursor contract, § pagination), `offset` (keyset +supersedes it; never load-bearing), `skip_labels` (§ traps), `q`-style text search, +`title_contains` and friends, time windows other than created (no consumer; all additive later). + +**What this list surface deliberately cannot serve in v0 — the reconciler full scan.** Gas +City's cache reconciler snapshots the COMPLETE active universe every 30/60/120 s per store as +`{limit 0, SkipLabels: true, TierMode: TierBoth}` (caching_store_reconcile.go:61-63): the +persistent tier via `bd list`, the ephemeral tier via a SECOND subprocess, +`bd query "ephemeral=true AND ..."` — "the installed bd list surface does not expose ephemeral +rows" (bdstore.go:2262-2264). v0's `/issues` hard-wires `SkipWisps=true` and always hydrates +labels, so that snapshot is unreachable over HTTP — by scope, not by accident — and this +document says so in the thesis and slice 11 instead of counting the reconciler among the +deletable forks (migrating it without `skip_labels` would also hydrate labels for the full +active set every cycle, past the reconciler's own scan-warn threshold). The additive path is +real and needs no new storage capability: `IssueFilter.Ephemeral *bool` (types.go:1648) and +`SkipWisps` (types.go:1700) already exist to back an ephemeral/wisp-inclusion parameter, and a +labels-omission knob may ship only together with an explicit omission marker on the NEW page +envelope (e.g. `omitted: ["labels"]`), where the `IsLitePartial` `json:"-"` trap does not apply +(§ traps). Neither knob has a `bd list` counterpart to run the rule-8 parity oracle against — +`bd list` cannot emit ephemeral rows at all — which is the second reason they are out of the +MVP: they would be the only wire surface with no CLI oracle behind it. + +**Response `IssuesPage` = `{items: [IssueWithCounts], has_more: bool, next_cursor?: string}`.** +`bd list --json` calls `SearchIssuesWithCounts` on both CLI paths (cmd/bd/list.go:574-580, +list_proxied_server.go:78), so the HTTP items carry the same counts fields or the full-item-JSON +oracle (rule 8) fails immediately — same reasoning as `/ready` (rule 3). `next_cursor` is present +iff `has_more` is true. + +#### `GET /v0/beads/issues/{id}` + +Response is a **single `IssueDetails` object, never an array** (the array is a CLI +multi-argument-ism; this deletes Gas City's decode-array-index-`[0]` + `ErrIDCollision` guard). +Shape per types.go:986-1013: embedded `Issue` + `labels[]` + `dependencies[]` +(`IssueWithDependencyMetadata`: full issue + `dependency_type`) + `parent` + count-only-mode +cardinality fields `dependency_count`/`dependent_count`/`comment_count` (`*int64`) with +`Dependents`/`Comments` slices nil and `comments_omitted` semantics preserved exactly as `bd +show`'s default — assembled by the shared detail function (rule 13; § 2, "Shared detail +assembly", owns its signature). This is a superset of Gas City's decoded 18-field contract. + +#### `POST /v0/beads/issues/{id}:claim` + +`ClaimRequest` = `{"actor": string}`, required, non-empty; the server never infers an actor (its +git identity is meaningless for remote callers). The handler validates the actor at the wire +edge, BEFORE the CAS, because the domain layer checks only `actor == ""` +(domain/issue.go:462-463) — a whitespace-only or near-1MB actor would otherwise pass, be +persisted to the assignee column, and be interpolated into the Dolt commit message +(`"bd serve: claim by "`), corrupting the audit trail the claim model rests on. +Rules: trim, reject empty-after-trim, reject length > 256 bytes, reject control characters +(including newline — a multiline actor would forge extra commit-message lines); all → 400 +`invalid_argument`. The spec pins these as `minLength`/`maxLength`/`pattern` on the `actor` +property so clients see the contract, not just the refusal. `ClaimResponse` = `{issue: Issue, +already_claimed: bool}` — `already_claimed` maps `domain.ClaimResult.AlreadyClaimed` +(domain/issue.go:222-225): a same-actor re-claim is idempotent and returns 200 with +`already_claimed: true`, matching CLI semantics. There is no constant `claimed: true` member — +a field that can only ever be one value is dead wire surface. `issue` is re-read inside the same +transaction after the CAS. The write path is `uow.RunTxResult` (rule 5); its retry semantics +surface on the wire only as the 503 `busy` row below. + +#### `GET /v0/beads/context` + +`ContextResponse` is a spec-native schema (§ schema pinning) SOURCED FROM — not a mirror of — +`domain.ContextInfo` (domain/context.go:36-53), captured via +`contextinfo.NewContextProvider(workDir, version)` (internal/storage/contextinfo/provider.go:12). +Its exact field set, enumerated here because each one is permanent wire surface: + +```json +{"api_version": "v0", "bd_version": "...", "schema_version": 1, + "backend": "dolt", "dolt_mode": "...", "database": "...", + "beads_dir": "...", "repo_root": "...", "project_id": "...", + "capabilities": ["ready.list", "issues.list", "issues.get", "issues.claim"]} +``` + +Seven fields carry `ContextInfo` values (`bd_version`←`BdVersion`, `backend`←`Backend`, +`dolt_mode`←`DoltMode`, `database`←`Database`, `beads_dir`←`BeadsDir`, `repo_root`←`RepoRoot`, +`project_id`←`ProjectID`); three are HTTP-only (`api_version`, `schema_version`, +`capabilities` — `ContextInfo` has none of them). Each kept identifier is justified, not +inherited: `database`/`project_id` are logical names, not host topology; `beads_dir`/`repo_root` +ARE absolute host paths, kept because the default deployment is same-host loopback and they are +the single-workspace server's only workspace-identity handshake (they replace Gas City's direct +`.beads/metadata.json` read) — a network peer learning filesystem layout from them is part of +the disclosure an operator accepts with the `--allow-non-loopback` warning. + +The remaining nine `ContextInfo` fields are **deliberately withheld**, and one of them is +**hard-excluded**: + +- `SyncRemote` is hard-excluded, not merely unserved-for-now. `GetContextInfo` populates it + unconditionally from config `sync.remote`, falling back to the deprecated `sync.git-remote` + (fs/context.go:87-93), and the key is documented as "any Dolt-compatible remote URL" + (internal/config/yaml_config.go:45). Remote URLs routinely embed credentials + (`https://x-access-token:TOKEN@github.com/org/repo`), so serving the field verbatim would hand + a repo credential to every loopback client — and to every network peer under + `--allow-non-loopback`. It never joins the response, in any future version, without an + explicit credential-redaction rule designed first. Same rule for any successor sync-URL field. +- `ServerHost`/`ServerPort` name the **unauthenticated Dolt SQL bind endpoint** — advertising it + invites clients to bypass the API and dial the DB directly; `ProxiedDir`/`DataDir` are + server-local filesystem topology with no client use; `Role` is sync plumbing with no v0 + consumer. +- `CWDRepoRoot`/`IsRedirected`/`IsWorktree` are cwd-derived, meaningless in a server process + (same reasoning as rule 15). + +Withholding is the safe default: adding any of them later is additive; shipping one now is +permanent. The allowlist is enforced, not aspirational: `TestContextResponseAllowlist` (§ 5, +test 8b) serializes the handler's response for a `ContextInfo` with every field populated — +`SyncRemote` set to a URL embedding a fake credential — and fails on any JSON key outside the +ten above or on the credential substring appearing anywhere in the body, so a future +`ContextInfo` field cannot leak onto the wire by default. + +`schema_version` is sourced from a shared `WireSchemaVersion` constant that +`cmd/bd/output.go`'s `JSONSchemaVersion = 1` (output.go:12) also consumes, so the value reported +over HTTP and over stdout cannot diverge. It is diagnostic, never a gate: because the constant +is shared with the CLI stdout envelope, it can move for CLI-only reasons with zero HTTP wire +change, so clients MUST NOT branch HTTP behavior on it (§ version skew). `capabilities` is +derived from the route table's **implemented** handlers (one source; transitional 501 stubs are excluded by construction, so +the endpoint never advertises an operation that would 501 — § 5 slice 6); additions are +additive, never a door. This endpoint replaces Gas City's +three-way context hack (`bd context --json`, `bd version` string surgery, `.beads/metadata.json` +reads). Documented codes: 200, 500, plus the middleware-level 400 for a foreign `Host` header +(§ 4, reachable on every route) — v0 serves a startup snapshot and does not touch the DB +per request; if a later slice makes it a DB-probing readiness endpoint, 503 is added then +(additive). + +### Version skew: what a client may gate on (the client contract) + +"What breaks when the server is a version ahead of or behind my client" is answered elsewhere +for item fields (additive-only evolution + tolerant decode, § schema pinning) and for cursors +(opaque, versioned, § pagination). The remaining skew surfaces — codes, parameters, and the +gating rule itself — are stated here as contract, in the spec's `ContextResponse` and +`Problem` descriptions, because an unstated gating rule regenerates exactly the pathology this +API retires: version-string surgery and substring matching. Which field gates what: + +- **`capabilities` gates operation presence.** It is derived from the implemented route table + (§ context above); a client that needs `issues.claim` checks the list, never the version. +- **`bd_version` gates feature availability.** It is the release's structured semver and the + only field a client may compare as a version — for BEHAVIORAL changes tied to a release (a + fixed bug, a changed default), never for operation or parameter presence. +- **`api_version` gates the path major.** It changes only when `/v1` is cut (§ schema pinning, + evolution policy). +- **Parameter presence is probed, not versioned, in v0.** `capabilities` is operation-level, + so the strict unknown-parameter rejection (§ the six operations) is the per-parameter gate: + a 400 with `reason: "unknown_parameter"` + `param: ""` is a loud, machine-attributable + "this server does not know this parameter" — never a silently widened result set. If + probe-by-request proves awkward in practice, the additive fix is an `api_revision` context + field exposing the spec's `info.version` (bumped on every spec change, enforceable by the + same pr-policy drift gate) so parameter-level gating never regresses to semver surgery — + recorded as open question 7 rather than shipped, because it grows the § context allowlist. +- **Clients MUST default-branch on unknown `code` values within a status class.** The code + vocabulary grows additively (rule 12), and `code` is the only dispatchable member — so a + client that switches exhaustively on `code` breaks on the first addition, which would make + "adding codes later is additive" false for precisely the clients this envelope invites. + Contract: dispatch on the codes you know; treat an unknown code as its status class (unknown + 4xx → client bug, fail loud; unknown 503 → retry per `Retry-After`; other unknown 5xx → + server fault). Same rule for unknown `reason` values (§ sentinel mapping). The spec states + this on the `Problem` schema. +- **`schema_version` is NOT an HTTP gate.** It is `types.WireSchemaVersion`, deliberately + shared with the CLI stdout envelope (§ 2, parity row 9) so the two surfaces report one + number — which means a CLI-envelope-only corpus change bumps the value a remote client sees + with ZERO HTTP wire change. A client that gates HTTP behavior on `schema_version` trips + spuriously on the first CLI-side corpus regen: the shared constant quietly welds the two + compatibility domains the error-envelope section keeps independent, and the weld is + tolerable ONLY because nothing on the HTTP side may key on the value. Report it, log it, + never branch on it. (While `/v0` is the newest path version any bump is forbidden outright — + § schema pinning — so the field is constant `1` today; if a CLI-envelope-only schema change + ever collides with that freeze, the unweld — a separate CLI-envelope version constant — is + additive on the HTTP side.) + +### Error envelope: RFC 9457 problem+json, one shape everywhere (one-way door) + +Every non-2xx byte the server emits is `application/problem+json`: + +```json +{"status": 409, "title": "Conflict", "code": "already_claimed", + "detail": "issue bd-abc already claimed by alice", + "assignee": "alice", "issue_status": "in_progress"} +``` + +Required members: `status`, `title`, `code`. `code` is the stable machine-readable string — +the only member clients may dispatch on; it is what deletes Gas City's ~100 lines of +substring-matching on conflict text (bdstore.go:812-828). `detail` is optional prose, never +load-bearing. `type` is omitted (`about:blank` implied; no URI ceremony in v0). Extension +members are per-code as listed below; adding members or codes later is additive — additive for +the client only under the § version-skew contract (default-branch on unknown `code` values +within a status class; an exhaustive switch on `code` breaks on the first addition) — but +**renaming or removing a documented status+code pair is a wire break — the code vocabulary is +a one-way door**, which is why rule 12 caps it at what the six operations need. + +Why this envelope and not the two shapes already in play: + +1. **The CLI `{error, schema_version}` envelope** (cmd/bd/output.go:40,71) is a stdout protocol, + byte-pinned by the `cmd/bd/protocol` corpus. It carries no HTTP status semantics and no + machine code; extending it would churn a pinned corpus and weld two independent compatibility + domains (CLI stdout schema, HTTP wire) together. It stays CLI-only. +2. **The hosted `{"error": "..."}` handler shape** is the documented cautionary tale: split + three ways (problem+json from its identity gate, bare `{"error"}` from handlers, 403 modeled + both ways, 500 undocumented). Adopting it would import the inconsistency we exist to kill. + +RFC 9457 is the registered standard, gives typed conflicts via extension members, and the hosted +identity gate already emits it — so the hosted product converges toward OSS (decided +parameter 1). We fix the hosted spec's omission by documenting 500 (and 503 where reachable) on +every DB-backed operation. + +#### Sentinel → status mapping + +One table, in one file (`internal/httpapi/problem.go`), matched exclusively with +`errors.Is`/`errors.As` — never `err != nil → status`: + +| Condition | Status | `code` | Extensions | +|---|---|---|---| +| request parse/validation failure: unknown query parameter (§ 3, unknown-parameter rejection), bad param value, invalid `sort`/`status`/`type`, bad `metadata_field` key, malformed/oversized claim body, invalid actor (empty after trim, > 256 bytes, or control characters — § 3 claim), `limit=0` under `--allow-non-loopback` (§ 3 limit semantics), foreign `Host` header (§ 4 bind posture) | 400 | `invalid_argument` | `param` — the offending query parameter, body field, or header name; present on every condition in this row except a body that fails to parse at all. `reason` ∈ {`unknown_parameter`, `invalid_value`} — always present | +| undecodable or wrong-version cursor | 400 | `invalid_cursor` | — | +| `errors.Is(err, storage.ErrNotFound)` | 404 | `not_found` | — | +| `errors.Is(err, storage.ErrAlreadyClaimed)` | 409 | `already_claimed` | `assignee`, `issue_status` | +| `errors.Is(err, storage.ErrNotClaimable)` | 409 | `not_claimable` | `issue_status` | +| `uow.RunTxResult` returns with `uow.IsSerializationError(err)` true (retry budget exhausted; uow/tx.go:76, uow/errors.go:16), or the in-flight semaphore wait timed out (§ 4, concurrency limits) | 503 | `busy` | header `Retry-After: 5` on exhausted claim retries (the server just observed ≥ 15 s of write contention; a 1 s comeback invites a convoy of claim requests each holding a semaphore slot up to 15 s, starving reads exactly when dispatch is busiest) / `Retry-After: 1` on a semaphore-wait timeout (slot pressure clears quickly); static `detail` (scrubbing rule below) | +| `NewUOW`/driver/net failure — Dolt server or dbproxy down/idle-stopped | 503 | `db_unavailable` | header `Retry-After: 5`; static `detail` (scrubbing rule below) | +| anything else, including panic recovery | 500 | `internal` | — static `detail`; real error logged server-side only (scrubbing rule below) | + +**400s carry a machine-readable pointer, not just prose (decided during client review).** One +umbrella `invalid_argument` code over every validation failure is rule-12-minimal, but it +forces a choice on clients that the envelope's own principle forbids: `detail` is never +load-bearing, yet a bad `sort` value, a bad `metadata_field` key, and an unknown parameter +would be distinguishable only by parsing prose — the substring-matching pathology this API +exists to delete, reintroduced at the 400 boundary. So the distinction is carried by extension +members, not new codes: `param` points at the offending input by name, and `reason` separates +the two client postures — `invalid_value` ("my value is malformed": a client bug, fail loud) +from `unknown_parameter` ("this server does not know this parameter": version skew, degrade or +fall back — § version skew). `invalid_cursor` needs neither: the dedicated code already does +this job for cursors. Extension members are additive vocabulary (§ one-way doors), and the +`reason` set may grow; clients MUST default-branch on unknown `reason` values exactly as they +do on unknown codes. + +**5xx detail scrubbing (decided — formerly open question 4, closed as safe-by-default).** For +the `busy`, `db_unavailable`, and `internal` rows, `detail` on the wire is a fixed static +string per code (e.g. "database temporarily unavailable; retry"); the underlying error is +written to the server log only, never to the response body. This is mandatory, not an +operator-verbosity preference: `NewUOW`/driver failures routinely embed the DSN and topology — +go-sql-driver renders connection targets as `user@tcp(127.0.0.1:PORT)/db` and net dial errors +carry `dial tcp 127.0.0.1:PORT` — and query-layer errors can carry SQL fragments. The moment +the same binary is bound with `--allow-non-loopback` (an explicitly supported mode), a verbose +5xx `detail` becomes an information-disclosure channel to network peers. The envelope's own +principle — `code` is the contract, `detail` is prose — is tightened here to **prose is never +sensitive**. The 4xx rows keep their specific, parameter-naming `detail`: they reflect the +client's own input back, not server internals. `TestProblemMapping` (§ 5) asserts the mapped +5xx bodies never contain the input error's message text. + +Sentinel declaration sites, post-facade: the canonical declarations of the `Err*` sentinels live +in the leaf `issueops` package (the facade's errors relocation), aliased back from +`internal/storage` with the same pointer — `errors.Is` keys on `storage.Err*` keep working +unedited and remain the mapper's keys; keying on `issueops.Err*` is equivalent and optional. +`ClaimedByFragment`/`NotClaimableStatusFragment` stay in `internal/storage` (not among the 18 +relocated symbols). + +The 409 extensions are populated from a fresh read in the same transaction, never by parsing the +sentinel message fragments (`storage.ClaimedByFragment`/`NotClaimableStatusFragment`, which +exist for CLI text reconstruction only). `*storageissueops.ErrTooManyRows` +(`internal/storage/issueops`, the internal package) is +deliberately absent: with no operator opt-in, v0 never sets `MaxRows` on a server-built filter +(a cap is an operator knob per rule 2), so the error is unreachable; when the knob ships +(`--max-rows`, recommended for v0 by SRE review, owner sign-off pending — open question 2), a +400 `too_many_rows` row is added in the same change. + +**Not-found is normalized in the shared function, not the handler (rule 4).** The dual miss +shapes and the normalization mechanics are specified once in § 2, "Not-found normalization" — +the consequence for this table is that `problem.go` needs only the single +`errors.Is(err, storage.ErrNotFound)` row, and no future caller can reintroduce the bug in +either direction (missing-issue-500, or the hosted viewer's every-error-is-404). +`TestServerGetUnknownID404` asserts 404-on-unknown-id end to end against real Dolt; a simulated +mid-request connection drop must yield 503/500, never 404. + +### Pagination + +**Keyset cursor over `(created_at DESC, id ASC)`, on `GET /v0/beads/issues` only (one-way +door).** The predicate and total order are already implemented and documented on +`types.IssueFilter.AfterCreatedAt`/`AfterID` (types.go:1617-1633): rows strictly after the +position under `(created_at < ca) OR (created_at = ca AND id > id)`; `id` is the primary key, so +same-second groups larger than a page still page completely with no dropped or duplicated row. +It composes with every other filter, including `created_before`. + +- **Encoding:** `base64url(JSON {"v": 1, "ca": "", "id": ""})`. + Opaque by contract — clients MUST NOT construct or parse it; the `v` member is the server's + private evolution hinge. Undecodable, unpadded-garbage, or unknown-`v` input → 400 + `invalid_cursor`. +- **Server side:** the handler forces `SortBy="created"`, `SortDesc=false` on EVERY list + request — cursored or not — and a cursor additionally decodes into `AfterCreatedAt`/`AfterID`. + Per the field contract (types.go:1637-1639) this yields `ORDER BY created_at DESC, id ASC`, + the order the predicate assumes. Forcing only in the cursor-decode path would be a real bug: + an un-cursored page 1 would fall through to the default priority ordering + (sqlbuild/sort.go:70-71 via list_filter.go:123-124) while `next_cursor` encodes a keyset + position in the created order, so page 2 would skip and duplicate rows. This is why v0's list + has no `sort` param: sort order is welded to the cursor contract, and that weld is the + one-way part. New sort orders require new (cursorless or separately-cursored) surface. + `TestServerListCursorPagination` asserts the first (un-cursored) page is already in + `(created_at DESC, id ASC)` order, because the id-sorted parity oracles cannot see ordering. +- **Response:** `has_more` from `domain.SearchPage.HasMore`, which the repo computes by limit+1 + over-fetch and trim (domain/db/issue_search.go:81-102,476-481, limitOffsetSQL:511-521). + `next_cursor` is present iff `has_more`, built from the last item of the page. + +**Limit semantics (rule 2, one-way door):** + +- `limit` absent → the shared default constant: **50** for list (= `bd list`'s default, + cmd/bd/list.go:743), **100** for ready (= `bd ready`'s default, cmd/bd/ready.go:796). Both + surfaces read the same constants (owned by the shared-contract package), so the document that + promises no divergence cannot itself diverge (rule 9). +- `limit=0` → **unlimited, exactly as in the CLI.** Gas City verifiably runs + `bd ready --limit 0` and `bd list --limit 0` (its reconciler does an unfiltered active-set + scan forever); a v0 that 400s or clamps it fails the brief. Verified mechanics: `Limit<=0` + emits no SQL LIMIT clause and `trimToLimit` returns false (issue_search.go:476-481,511-517), + so an unlimited page reports `has_more=false` and no `next_cursor` — correct by construction, + no special-casing needed. On the default loopback bind, a cap — if ever wanted — is an + operator knob (`MaxRows` vocabulary), never a wire-level refusal; whether that knob is worth + shipping stays open question 2. **Under `--allow-non-loopback` the calculus inverts (decided + during security review):** `limit=0` is refused with 400 `invalid_argument` (detail: + "unlimited reads are loopback-only; pass an explicit limit"). A single + `GET /v0/beads/issues?limit=0` buffers the entire active set plus its JSON marshal, and 16 + concurrent scans multiply it (§ 5 risks) — that must not be reachable by arbitrary network + peers via a shipped flag, and the mitigation must not wait on a post-hoc knob nobody enabled. + In-vocabulary (rule 12: `invalid_argument`, no new code), mode-dependent, documented in the + spec's `limit` parameter description, and invisible to Gas City's loopback reconciler (rule 2 + carve-out noted at the rule). +- `limit` < 0 → 400 `invalid_argument`. + +### Traps and how the wire handles them + +**`IsLitePartial` is `json:"-"` (types.go:170).** `IssueFilter.Lite` produces rows whose heavy +fields are zero-valued with only the non-serialized `IsLitePartial` marker set — a lite row is +wire-indistinguishable from an issue with a genuinely empty description. Similarly +`IssueFilter.SkipLabels` (types.go:1694-1697) leaves `Labels` nil, indistinguishable under +`json:"labels,omitempty"` from "no labels". Wire handling: **v0 never sets `Lite` or +`SkipLabels` on any server-built filter and exposes no `skip_labels`/`view=lite` parameter**; +labels are always hydrated. Belt-and-braces: the server's execution seam (UOW → domain/db) does +not implement `Lite` at all and always returns fully-hydrated issues (types.go:1729-1734), so a +lite-partial row is unreachable from `bd serve` by construction. Future exposure is additive +but carries a precondition recorded here: a lite/skip-labels view may only ship together with an +explicit wire marker for the omission (e.g. a response-level `partial` field), because the +canonical structs cannot express it. Gas City's `--skip-labels` subprocess caller is the +reconciler full scan, which is not a "gives up an optimization" case: that loop stays on +subprocess entirely in v0 for the stronger reason that it also needs the ephemeral tier v0's +list does not expose (§ 3, "what this list surface deliberately cannot serve"); nothing about +its CLI usage changes. + +**Metadata typing.** The brief's trap ("bd type-infers `--set-metadata k=true` into JSON +bool/number") has shifted on main but the wire consequence stands: since GH#4146, +`MetadataEditValue` always stores `--set-metadata` values as JSON strings +(internal/storage/metadata.go:314-324), but typed values still enter via the explicit +`--metadata`/`--metadata-json` path (`MergeMetadataJSON`, metadata.go:246-249) and persist in +every pre-GH#4146 row. Wire handling: `Issue.metadata` is `json.RawMessage` (types.go:102) and +the spec declares it `type: object, additionalProperties: true` with an explicit description +that **values may be any JSON type and clients MUST NOT strict-decode to `map[string]string`** +(the strict decode of one typed bead once poisoned an entire claim batch). The parity/bijection +fixtures include bool, number, and nested-object metadata values so a strict-decoding +regression can never land. `metadata_field` query values filter by top-level key=value equality +exactly as the CLI flag does — same `map[string]string` vocabulary, same semantics. + +**`timeout` is nanoseconds.** `types.Issue.Timeout` is `time.Duration` (types.go:145), which +`encoding/json` marshals as an int64 nanosecond count. Pinning means the spec must document +reality: `timeout: {type: integer, format: int64, description: nanoseconds}`. Pre-existing wire +fact, not a choice. + +### One-way doors, summarized + +| Decision | One-way? | +|---|---| +| URL shapes `/v0/beads/*`, `/healthz` (Gas City's probe already asserts `$GC_BEADS_API/v0/beads/ready`) | **Yes** | +| `x-go-type` pinning: `types.Issue` JSON encoding = HTTP contract | **Yes** (by design — one compatibility domain) | +| Ready items = `IssueWithCounts` (counts fields permanent) | **Yes** | +| problem+json envelope + each documented status/`code` pair | **Yes** (additions additive; removals/renames breaking) | +| Keyset order `(created_at DESC, id ASC)` welded to `/v0/beads/issues` | **Yes** (new orders need new surface) | +| `limit=0` = unlimited; defaults 50/100 from shared constants | **Yes** (clients will depend on both) | +| Page envelopes as top-level objects `{items, ...}` | Door opened in the extensible position | +| Cursor internal encoding (`v` member) | No — opaque + versioned, server-private | +| No cursor on `/ready`; no `sort` on `/issues`; no `skip_labels` | No — all additive later | +| `capabilities` list contents, problem extension members | No — additive | + +## 4. Codegen, CLI migration, server lifecycle and security + +### Codegen and the drift gate + +The wiring copies the hosted product's proven mechanism exactly (hand-written spec, `oapi-codegen` +types-only, `go:generate` + Go `tool` directive, three-part CI gate), relocated and renamed for OSS. +Verified against the hosted repo: the generate line lives in `cmd/beads-web/apigen/doc.go` +(`//go:generate go tool oapi-codegen -generate types -package apigen -o types.gen.go ...`), the tool +directive at `go.mod:251` (`tool github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen`, pinned +v2.6.0), and the gate is Makefile `spec`/`spec-ci`: regenerate, `git diff --quiet -- $(SPEC_FILES)` +or fail, then `go test ./cmd/beads-web/ -run 'TestSpec' -count=1`. + +OSS layout: + +``` +internal/httpapi/spec/openapi.v0.yaml # hand-written, source of truth +internal/httpapi/spec/embed.go # //go:embed openapi.v0.yaml (parity tests read it) +internal/httpapi/apigen/doc.go # go:generate line below +internal/httpapi/apigen/types.gen.go # generated, committed +``` + +`internal/httpapi/apigen/doc.go`: + +```go +// Package apigen holds Go types generated from the bd serve OpenAPI spec +// (spec-first). Do NOT hand-edit types.gen.go — change the spec and run +// `make api-gen`. +// +//go:generate go tool oapi-codegen -generate types,skip-prune -package apigen -o types.gen.go ../spec/openapi.v0.yaml +package apigen +``` + +`go.mod` gains `tool github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen` pinned at v2.6.0 (the +hosted-proven version; beads is on Go 1.26.5, tool directives supported; the repo currently has no +`tool` directives, so this is the first — review the go.sum diff in isolation). Because `Issue`, +`IssueWithCounts`, and `IssueDetails` are pinned to the canonical Go types via +`x-go-type`/`x-go-type-import` (no second wire struct, per the binding corrections), `types.gen.go` +contains only the envelope/request types (`ReadyPage`, `IssuesPage`, `ClaimRequest`, +`ClaimResponse`, `ContextResponse`, `Health`, `Problem`) referencing `internal/types` directly. + +Makefile targets (names final): + +```make +api-gen: ## regenerate OpenAPI types from internal/httpapi/spec/openapi.v0.yaml + go generate ./internal/httpapi/apigen +api-check: api-gen ## three-part spec drift gate (regen, diff-or-fail, named tests) + @git diff --quiet -- internal/httpapi/apigen internal/httpapi/spec || \ + { git --no-pager diff --stat -- internal/httpapi; \ + echo "openapi drift: run 'make api-gen' and commit"; exit 1; } + go test ./internal/httpapi/ -run 'TestSpecRouteParity|TestWireTagBijection|TestProblemMapping' -count=1 +``` + +CI wiring: one line appended to `scripts/ci/pr-policy.sh` in the existing `ci_time` idiom — +`ci_time "check openapi spec gate" -- make api-check` — so the gate runs in the required PR-time +`ci-pr-policy` job (`Makefile:186`). It must NOT live only in the `Main` workflow, which is +perpetually concurrency-cancelled and hides breakage. The three named tests (all CGO-free, so they +also run under `ci-pr-core`'s `go test ./...`): + +- `TestSpecRouteParity` (`internal/httpapi/spec_parity_test.go`) — parses the embedded YAML and + asserts exact set-equality between spec `paths` (method+path) and the server's route table; each + route entry carries an explicit `specPath` so the `{id}:claim` ↔ ServeMux-pattern mapping is + declared, not inferred. Modeled on the hosted `TestSpecBijectionAndScopeParity`. +- `TestWireTagBijection` (`internal/httpapi/wire_bijection_test.go`) — reflects over the JSON tags + of `types.Issue`, `types.IssueWithCounts`, and `types.IssueDetails` and asserts TWO-WAY + set-equality with the spec's schema properties, modulo an explicit in-test allowlist for + documented omissions. This is the only guard on the x-go-type-pinned schemas (the compiler cannot + see the spec), so a new `types.Issue` field fails CI until the spec documents it, and vice versa. + A round-trip fixture test is NOT a substitute (a new `omitempty` field absent from the fixture + passes both directions); the fixture round-trip (including bool/number `metadata` values from + `--set-metadata k=true` / `k=42` inference) exists additionally, as a decode-strictness check. +- `TestProblemMapping` (`internal/httpapi/problem_test.go`) — table test driving every sentinel + (wrapped and bare) through the error mapper and asserting status+code, including the + nil-issue and wrapped-`sql.ErrNoRows` not-found cases. + +### CLI migration mechanics + +Per binding rule 11, every extraction lands as its own no-op refactor slice — never the mega-slice. +Each slice is bisectable against the byte-pinned protocol corpus. + +**How the corpus protects the migration.** `cmd/bd/protocol` (32 files) is the producer half of a +cross-version contract system: `TestCorpusGolden` runs a deterministic command plan against a live, +source-built, Dolt-backed `bd`, canonicalizes (timestamps → ``, sorted keys/arrays), and +byte-compares against committed goldens under `cmd/bd/protocol/testdata/corpus/`. A diff is a hard +failure; deliberate wire changes require `make corpus-regen` plus a `JSONSchemaVersion` bump in +`cmd/bd/output.go` — and once `/v0` ships, a bump additionally requires a new HTTP path version +per the welded-domain evolution policy (§ 3, schema pinning). The extraction slices therefore +have an executable no-op oracle: they must land +with ZERO corpus diff and no `JSONSchemaVersion` change. Two additional guards: (1) golden +filter-equality tables are written FROM the old inline builder code BEFORE it is deleted +(old-construction == new-builder output across an input matrix); any delta found is triaged as an +explicit, corpus-verified behavior fix in the PR — never silently absorbed; (2) the known local +trap: `cmd/bd` subprocess tests silently reuse a stale repo-root `bd` — rebuild it before trusting +local corpus runs. + +Slice-by-slice (shared package `internal/workapi`; its internal shape is specified in § 2 — this +section fixes only the file mechanics; slices M1–M4 here are slices 1–4 of § 5's delivery plan): + +**Slice M1 — `bd list` (no-op).** `cmd/bd/list_filter.go` (343 lines) is DELETED; its contents move +verbatim to `internal/workapi/list.go` with exported names: `buildListFilter` (`list_filter.go:119`) +→ `workapi.BuildListFilter`, `loadListFilterConfig` (`:78`) → `workapi.LoadListConfig`, +`listFilterConfig`/`listFilterConfigSource` → `workapi.ListConfig`/`workapi.ConfigSource`, and the +two adapters `directConfigSource`/`proxiedConfigSource` (`:54`/`:66`) → +`workapi.NewStoreConfigSource(storage.DoltStorage)` / `workapi.NewUOWConfigSource(uow.UnitOfWork)`. +Call sites edited: `cmd/bd/list.go:539` and `cmd/bd/list_proxied_server.go:55`. `list_input.go` +(`gatherListInput`, `:102`) stays in `cmd/bd` — cobra flag parsing is CLI-local — now emitting +`workapi.ListParams`. `list_filter_status_test.go` cases move to `internal/workapi`. The shared +default constant `workapi.DefaultListLimit = 50` is introduced here and `list.go:743`'s flag +registration consumes it (binding rule 9: HTTP later reads the same constant; 0 = unlimited). + +**Slice M2 — `bd ready` (no-op).** The inline duplicate filter construction at +`cmd/bd/ready.go:104-215` is DELETED; the direct RunE path calls `gatherReadyInput(cmd)` exactly as +the proxied path already does (`ready_proxied_server.go:18`). `cmd/bd/ready_input.go` shrinks to a +cobra→`workapi.ReadyParams` mapper and calls `workapi.BuildReadyFilter` — the single home of the +ready defaults (`Status:"open"`, sort validation, label/type normalization, +`workapi.DefaultReadyLimit = 100` consumed by `ready.go:796`, 0 = unlimited). The cwd-derived +directory-label substitution (`config.GetDirectoryLabels()`, `ready.go:141-145`) is applied in +`cmd/bd` BEFORE calling the builder and never enters the library — meaningless in a server process. +`resolveMaxRows` also stays cobra-side, feeding `ReadyParams.MaxRows`. + +**Slice M3 — `bd show` detail assembly (no-op).** New `internal/workapi/detail.go`: +`BuildIssueDetails(ctx, DetailSource, id, DetailOptions)` plus `NewStoreDetailSource(storage.DoltStorage)` and +`NewUOWDetailSource(uow.UnitOfWork)` (the DetailSource interface carries the explicit `isWisp` axis; +shape in the contract section). DELETED in favor of it: the direct assembly block in +`cmd/bd/show.go:147-235` (labels + `GetDependenciesWithMetadata` + +`CountDependents`/`CountDependencies`/`CountIssueComments`) and the proxied helpers +`proxiedGetIssueOrWisp`/`proxiedListDeps`/`proxiedCountDeps`/`proxiedCountComments` +(`show_proxied_server.go:127-170`) together with the assembly core of `proxiedBuildDetails` +(`:478-556`). Binding rule 4 is implemented HERE, in the shared function: on the UOW seam the miss +shape is a bare/wrapped `sql.ErrNoRows` (`internal/storage/domain/db/issue.go:403-405` returns +`sql.ErrNoRows` unwrapped; the domain layer wraps it) and is NEVER `storage.ErrNotFound`; +`BuildIssueDetails` normalizes {`errors.Is(err, sql.ErrNoRows)`, `errors.Is(err, +storage.ErrNotFound)`, nil-issue-with-nil-error} to the `storage.ErrNotFound` sentinel — the same +dual-sentinel treatment the CLI's own proxied code already applies +(`cmd/bd/create_proxied_server.go:497`) — so no future caller can reintroduce the +missing-issue-500. CLI-local and untouched: fuzzy/substring ID resolution, `--current`, watch mode, +the array-of-one `--json` envelope, and the `--include-dependents`/`--thread`/`--refs`/`--children` +extras, which layer additional queries on top of the shared assembly. + +**Slice M4 — claim consolidation (no-op for outcomes, one implementation for retry/commit).** +Binding rule 5: the ONLY retry/commit implementation is `uow.RunTxResult` +(`internal/storage/uow/tx.go:76` — fresh UOW per attempt, 25ms→15s serialization backoff, +nothing-to-commit tolerance, commit-with-message; test-covered on main). The bespoke +`cenkalti/backoff` loop in `applyUpdateProxiedOne` (`cmd/bd/update_proxied_server.go:93-126`) is +DELETED and its per-attempt body (`applyUpdateProxiedAttempt`) becomes the `TxFuncResult` passed to +`RunTxResult`; retry classification is identical (retry only on `uow.IsSerializationError`, +permanent otherwise), so this is a mechanical re-expression locked by the `lease_claim` and +`errors_contract` corpus files and the update-conditional test suites. The `updateIDFailure` +rendering on exhaustion stays in cobra-land. The direct path: post-facade it is NOT untouched — +direct `bd update` (including `--claim`) routes through `issueops.Lifecycle.Update` via +`cmd/bd/issueops_adapter.go` (same CAS, same sentinels underneath, so the classification +contract is unchanged). Slice M4's scope is unaffected either way: it is proxied-only +(`cmd/bd/update_proxied_server.go`, a file the facade branch does not touch). +The HTTP claim handler (server slice) calls the very same `RunTxResult` — zero claim-protocol code +in `internal/httpapi` beyond the sentinel→status mapping. + +The other ~46 direct/proxied command pairs are untouched, explicitly out of scope. + +### Server lifecycle and security + +**Command and flags.** New `cmd/bd/serve.go` (no `serve` command exists today): `bd serve +--addr 127.0.0.1:0 [--allow-non-loopback]`. No other flags in v0 (one pending exception: the +off-by-default `--max-rows` recommended by SRE review, awaiting owner sign-off — open +question 2). The bound address is printed as +the first stdout line (`bd serve: listening on http://127.0.0.1:PORT`); no default port is +blessed, but deployments are steered to an explicit fixed port (rule 7 note below — the +ephemeral default carries no mutual exclusion). + +**Mode gate — reconciled honestly (rule 10).** Verified on main: `uowProvider` is constructed only +when `proxiedServerMode` (`cmd/bd/main.go:1378-1388` → `newProxiedServerUOWProvider`, +`cmd/bd/uow_factory.go`, which routes to `uow.NewDoltServerUOWProvider` for the managed child and +`uow.NewExternalDoltServerUOWProvider` when `info.External` is set — i.e. the external-dolt +topology). Plain server / shared-server mode (`usesSQLServer()` true via `serverMode` or +`doltserver.IsSharedServerMode()`, `cmd/bd/store_factory.go:20-32`) builds a `DoltStore` and has NO +UOW provider on main. Therefore: + +| Workspace mode | v0 behavior | Delivered by | +|---|---|---| +| proxied (managed Dolt child) | works — serve reuses the `uowProvider` built in `PersistentPreRunE` | serve-skeleton slice | +| external dolt (proxied topology with `info.External`) | works — same `uowProvider` path | serve-skeleton slice | +| dolt server / shared-server mode | typed refusal, then wired | server-mode wiring slice: export `uow.NewSQLServerUOWProvider(ctx, endpoint, database, user, password, tlsName, teamServer)` as a thin wrapper over the existing unexported `openAndInitSchema` (`internal/storage/uow/dolt_sql_provider.go:179`), fed from the live doltserver state (127.0.0.1 + `state.Port`); serve is also excluded from post-run maintenance (`effectiveRootStorePolicy`) in this slice, since the non-proxied `PersistentPostRunE` branch otherwise runs auto-backup/export/push | +| embedded | permanent typed refusal (decided parameter 5) | — | + +Refusal messages must not promise what the gate refuses. Embedded: +`&storage.ErrUnsupported{Op: "serve", Backend: "embedded-dolt"}` with detail "bd serve requires a +Dolt SQL server; this workspace uses embedded Dolt" (`embedded-dolt` names the distinct +`EmbeddedDoltStore` backend implementation, in-vocabulary for the field). Server-mode (until its +slice lands): `&storage.ErrUnsupported{Op: "serve", Backend: "dolt"}` with detail "bd serve +supports proxied-server workspaces (managed or external dolt) today; dolt server mode support is +tracked". The mode belongs in the detail TEXT, not the `Backend` field: the type documents +`Backend` as a backend name (`// e.g. "sqlite"`, errors_unsupported.go:10) and is the embryo of +the pluggable-backend seam's error taxonomy, so stuffing a topology string like +`"dolt-server-mode"` into it would hand every downstream `errors.As` dispatch on `Backend` a +mixed backend/mode vocabulary that is wire-adjacent and hard to walk back. +`storage.ErrUnsupported` (`internal/storage/errors_unsupported.go:8`) is reused, not reinvented. + +**Bind posture.** `validateServeAddr(addr, allowNonLoopback)` follows +`validateManagedServerConfigPolicy` (`cmd/bd/proxied_server.go:449-498`) exactly: `net.SplitHostPort`; +the host MUST parse as a numeric IP literal (`net.ParseIP != nil`) — hostnames including +`"localhost"` are refused, because a name can silently license other listeners (the proxied-server +validator documents that a "localhost" host also licenses the default unix socket); unix sockets are +not supported at all; `!ip.IsLoopback()` requires `--allow-non-loopback` and prints a loud stderr +warning that v0 has no authentication. Trust model on loopback: identical to the managed Dolt child +(root/empty password) the server fronts — HTTP adds no new exposure on the same boundary. No auth, +no TLS, no CORS in v0; `Cache-Control: no-store` on every response; claim body capped at 1 MB via +`http.MaxBytesReader`. The designated future auth mechanism (only when non-loopback grows real +users) is the existing rotating 0600 shared secret with `crypto/subtle` comparison +(`internal/storage/dbproxy/identity`) surfaced as a bearer token — reused, vendor-neutral. + +**Browser exposure and DNS rebinding (defense-in-depth, stated rather than accidental).** An +unauthenticated loopback HTTP service is reachable from any browser running on the host, so the +browser attack surface must be reasoned about explicitly, not left to accident. Two properties +protect it today: the server never emits `Access-Control-Allow-Origin`, so a cross-origin page +cannot READ responses; and the claim handler REQUIRES `Content-Type: application/json` (anything +else → 400, in-vocabulary) — a JSON content type is not CORS-"simple", so a cross-origin claim +always triggers a preflight the server never approves. Making the content-type check explicit +matters: without it, an attacker's `text/plain` POST carrying JSON would skip the preflight. +Neither property stops **DNS rebinding**, where the attacker's page re-resolves its own hostname +to 127.0.0.1 and the browser then issues plain same-origin requests with no CORS in play. The +standard local-dev-server defense is applied: every request's `Host` header must match a small +allowlist — `127.0.0.1`, `[::1]`, or `localhost` (any port), plus the configured bind address +when bound non-loopback — anything else → 400 `invalid_argument` before any handler runs. A +rebound request arrives bearing the attacker's hostname in `Host` (browsers preserve the +original name) and is rejected. `TestProxiedServerServeModeGate` (§ 5) pins the foreign-`Host` +rejection. + +**No flock, no pidfile, no serve-info.json (rule 7).** dbproxy needs that machinery because clients +race to auto-spawn it; `bd serve` is operator-invoked and the TCP bind IS the mutual exclusion — +`EADDRINUSE` produces a clean error naming the address. Stated honestly rather than over-claimed: +that exclusion exists ONLY under an explicitly fixed port. The default `--addr 127.0.0.1:0` +allocates an ephemeral port, so N concurrent serves against one workspace run in parallel +silently, each on its own port, with no enumeration mechanism (no pidfile, by this same rule) — +data-safe, but "which instance are clients actually talking to" has no answer beyond each +instance's startup log line. The deployment recommendation, blessed in `bd serve --help` and the +slice-11 runbook, is therefore an explicit fixed-port `--addr`; the ephemeral default is for +ad-hoc and test use, where the printed bound address is consumed immediately. Two concurrent +serves against one workspace +are read-safe and claim-safe (the CAS is arbitrated in the SQL server). A discovery file would be a +brand-new unversioned file-format contract nobody asked for. Likewise no invented workspacegate +lease: verified that nothing on main outside `cmd/bd/doctor/gitignore.go` touches +`internal/workspacegate`, so serve acquires nothing — parity with every other command — and this is +documented rather than papered over. + +**One UOW per request, rollback guaranteed.** Every read handler opens one UOW from the provider and +closes it via a detached context: + +```go +uw, err := provider.NewUOW(ctx) // request ctx: honors client deadline for BEGIN +... +defer func() { + closeCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second) + defer cancel() + uw.Close(closeCtx) // RollbackUnlessCommitted (uow.go) +}() +``` + +Rationale, verified in the tx layer: `Close` → `RollbackUnlessCommitted` sends `ROLLBACK` on the +pinned `*sql.Conn`; if that send fails the conn is POISONED, not returned (`doltserver_tx.go:88-95` +— go-sql-driver's ResetSession does no COM_RESET_CONNECTION, so a session with an open transaction +must never be reused). Correctness is therefore already protected, but closing with the cancelled +request context would burn one pinned session per client disconnect; the detached context lets the +ROLLBACK complete and the conn return clean. Reads never call `Commit`. The generic +`uow.RunTxRead` closes with the caller's context, so serve uses its own `withUOW` helper with the +detached close instead. + +**The claim endpoint** is the only committing path: `uow.RunTxResult(ctx, provider, work)` with +commit message `"bd serve: claim by "` — the identical implementation the proxied CLI +consumes after Slice M4 (rule 5). `backoff.WithContext` already bounds the 15s retry budget by the +request context. Retry exhaustion maps to 503 `busy` with `Retry-After: 5` (retryable by contract, +the server-side analogue of the CLI's "retries exhausted on write conflicts" failure). Not +`Retry-After: 1`: the server has just spent up to 15 s failing to commit under write contention, +and inviting every 409/503 loser back one second later convoys claim requests that each occupy a +semaphore slot for up to 15 s — degrading read availability exactly when dispatch is busiest. A +5 s comeback keeps slots circulating; the `busy`-vs-`db_unavailable` triage distinction is +unchanged. "Nothing to +commit" is tolerated inside `RunTxResult` (wisp claims write only `dolt_ignored` tables). + +**The write path gets the same detached-close protection as reads — fixed in `RunTxResult`, not +forked.** Verified: `RunTxResult`'s per-attempt cleanup is `defer uw.Close(ctx)` with the +CALLER's context (uow/tx.go:90). On a client disconnect or the 60 s deadline expiring +mid-attempt, that `Close` runs with an already-cancelled context, the ROLLBACK `ExecContext` +fails immediately, and `doltserver_tx` poisons the pinned conn — precisely the failure mode the +read path's detached close above exists to prevent. Routing the only committing endpoint through +the one helper that still closes with the live request context would be an internal +contradiction: under flaky clients, or claim contention pushing requests toward the deadline, the +write path would churn one poisoned session per disconnect. The fix lands in `RunTxResult` +itself, not as a serve-side variant (rule 5 — one implementation, so no fork): the per-attempt +close adopts the same `context.WithoutCancel` + 5 s timeout pattern as `withUOW`, and the proxied +CLI inherits the same robustness (today a Ctrl-C mid-claim burns a session the same way). +Delivered with slice 9 (§ 5). + +**Concurrency limits.** Facts: `openDB` sets no pool limits (`dolt_sql_provider.go:168-177`, +`database/sql` default = unlimited) and every UOW pins one connection (`BeginTx`, +`dolt_sql_provider.go:50-56`), so an unbounded request burst means unbounded SQL connections. +serve bounds in-flight requests with a BLOCKING semaphore — `const serveMaxInflight = 16`, acquired +with the request context — which queues rather than refuses, so no 429/refusal status enters the +permanent wire surface (rule 12). The queue is bounded in TIME, not only in width: semaphore +acquisition waits at most 10 s (inside the request's 60 s deadline); a timed-out acquisition maps +to the already-documented 503 `busy` with `Retry-After: 1` — in-vocabulary, so rule 12 holds — +instead of parking the request for the full deadline and then answering a non-retryable 500. +16 pinned conns sit under Dolt's default `max_connections` only when serve is the sole tenant — +which is FALSE in external-dolt mode, where other `bd` CLI invocations, reconciler-driven forks, +and any second serve instance share the same server's `max_connections` budget; the slice-11 +runbook documents that budget arithmetic rather than assuming it away. The in-flight and wait +constants can become operator flags later without wire impact. + +**Pool limits on the serve-owned `*sql.DB` (defense in depth, not semaphore trust).** The HTTP +semaphore bounds handler concurrency, not connections: poisoned-conn replacement after a failed +ROLLBACK, `RunTxResult`'s retry attempts (a fresh UOW — a fresh pinned conn — per attempt), and +any future semaphore-exempt endpoint that touches the DB all escape it. serve therefore sets +explicit limits on the provider's pool at startup (via a small exported knob on the uow provider — +the `*sql.DB` is provider-internal today): `SetMaxOpenConns(serveMaxInflight + 4)` (headroom for +retry/replacement churn), `SetMaxIdleConns(serveMaxInflight)`, `SetConnMaxIdleTime(5m)`, +`SetConnMaxLifetime(1h)`. This is the repo's own idiom applied to the one seam that lacks it: +`embeddeddolt/open.go:51-54` and `dolt/transaction.go:162` set pool limits and +`dolt/connection_pool_test.go` proves the cap bounds driver opens — only uow's `openDB` sets +nothing. One operational-footprint consequence is documented rather than discovered: idle pooled +conns are live TCP connections through dbproxy, whose idle-watcher counts them as active +(`dbproxy/proxy/server.go:297`, `activeConns`), so a running serve keeps the watcher +armed-never-firing and pins the managed Dolt child alive for serve's lifetime; +`SetConnMaxIdleTime` bounds how long that outlasts serve's last actual traffic. + +The semaphore bounds the DB tier only; two front-tier gaps are closed explicitly rather than +left to it: + +- **Accepted connections are capped.** Go's `http.Server` spawns a goroutine per connection, and + a handler parked on a full semaphore holds its goroutine, fd, and buffers — so a flood of + valid requests (or a buggy client in a retry loop) accumulates unbounded blocked goroutines + behind 16 slots. The listener is wrapped in `netutil.LimitListener(l, 64)` + (golang.org/x/net/netutil — already in the module graph); excess connections wait in the + kernel accept backlog, not in Go memory. Like the semaphore, a cap that queues is not wire + surface (rule 12); the constant can become an operator flag later. +- **`/healthz` and `GET /v0/beads/context` bypass the DB semaphore** — both serve + startup-snapshot data and touch no DB (§ 3), so liveness and identity stay observable while + all 16 DB slots are held by long scans. They still count against the connection cap. This + makes `/healthz` LIVENESS-only by definition: it answers 200 while Dolt is down, wedged, or an + hour dead — a documented property, not a gap. The READINESS story is explicit rather than + absent: v0 adds no DB-probing endpoint (rule 12 minimality), so the documented operator + readiness probe — in `bd serve --help` and the slice-11 runbook, suitable for systemd/k8s + probes — is `GET /v0/beads/ready?limit=1`: a real query through the full semaphore + UOW path, + where 200 means ready and 503 means live-but-not-ready. A dedicated readiness endpoint later + is additive (§ 3's context note already reserves that door). + +**Timeouts at every layer.** `http.Server{ReadHeaderTimeout: 10s, ReadTimeout: 30s, IdleTimeout: +120s, WriteTimeout: 0, MaxHeaderBytes: 64<<10}`. `WriteTimeout` is deliberately 0: `limit=0` means +unlimited exactly as in the CLI (binding rule 2 — Gas City's reconciler does full active-set scans), +and a large response must not be truncated mid-body; slowloris exposure is covered by the header and +idle timeouts on a loopback socket. DB dial/ping honors context (`openDB` uses `PingContext`). + +`WriteTimeout: 0` removes the server's only whole-request backstop, so the deadline moves into +the handlers: EVERY handler — reads included, not just claim — wraps semaphore acquisition + UOW ++ query in a 60s `context.WithTimeout` (for claim that is ≥ the 15s retry budget plus commit; +generous by design, and like the semaphore and connection-cap constants it can become an +operator flag later without wire impact). A +request parked behind sixteen `limit=0` scans therefore times out and releases its slot instead +of pinning a goroutine indefinitely — without this, ReadTimeout covers only the header/body read +and nothing bounds the post-read lifetime of a blocked request. The deadline deliberately +excludes the response write: a large `limit=0` body must not be truncated mid-write, and a +slow-reading client holds one of the 64 `LimitListener` slots (bounded), not unbounded memory. A +server-side deadline surfaces as the generic 500 with the real error logged — with one carved-out +case already in vocabulary: a semaphore-acquisition wait that times out surfaces as the retryable +503 `busy` (above) — v0 documents only +the status codes its six operations need (rule 12), so there is no 504/413/429 vocabulary. + +**The wedge scenario, closed end to end.** The failure mode the two bounds above exist for: Dolt +stops answering but keeps connections alive (`dolt gc`, lock contention). Without them, the 16 +slots fill with un-cancellable queries; every subsequent request — 13 dispatch probes per 30 s +patrol plus reconciler full scans per store — queues on the blocking semaphore with no wait +bound; goroutines accumulate behind the 64-connection cap; `/healthz` stays green +(liveness-only, above); and when Dolt unwedges, a thundering herd of stale queued requests +executes. With them: the 60 s handler deadline cancels in-flight queries +(`QueryContext`/`ExecContext` propagate cancellation through go-sql-driver) and their conns are +returned or poisoned-and-replaced under the pool cap; the 10 s semaphore-wait bound sheds the +queue as retryable 503 `busy` instead of accumulating it; and the semaphore-saturation log event +(observability floor, below) is the 3 a.m. signal that distinguishes "wedged" from "no traffic". +The server self-recovers instead of converting a hung Dolt into a silently growing black hole, +and no new status vocabulary is introduced at any point. + +**Graceful shutdown.** serve blocks on `rootCtx.Done()` — already signal-aware via +`setupGracefulShutdown` (`cmd/bd/main.go:1719-1726`, SIGINT/SIGTERM/SIGHUP) — then +`srv.Shutdown(ctx, 20s)` (listener closes immediately; in-flight requests, including a claim +mid-commit, drain), `srv.Close()` on overrun, exit 0. The drain budget is 20 s, not 10, and the +number is reconciled with the claim path rather than picked: `srv.Shutdown` does NOT cancel +in-flight handler contexts, so a claim inside its 15 s serialization-retry budget would outlive +a 10 s drain and have its TCP connection killed by `srv.Close()` — possibly AFTER the Dolt +commit landed, leaving the client unable to know whether the claim succeeded. 20 s covers the +retry budget plus commit; for the residual window (a claim near its 60 s deadline at shutdown), +the client-retry contract closes the ambiguity: the claim is idempotent per actor, so +re-claiming after a killed connection returns 200 with `already_claimed: true` if the commit had +landed — the spec description and runbook state this recovery explicitly. One signal consequence +is documented rather than discovered: `setupGracefulShutdown` registers SIGHUP, so closing the +terminal of a foreground `bd serve` triggers a graceful shutdown — run it under a supervisor +(systemd, tmux, nohup) for anything longer than an experiment. The provider is closed where it is closed for +every proxied command today: `PersistentPostRunE` (`main.go:1509-1517`, which in proxied mode does +nothing but close `uowProvider` — no auto-commit/export/push machinery can fire). + +**Observability floor (the operator's 3 a.m. contract).** "No metrics endpoint" is a defensible +non-goal; no log is not — without a request log the operator of a failing serve cannot +distinguish no-traffic from all-traffic-hanging from all-traffic-503ing, and the endpoints alone +cannot tell them (`/healthz` is liveness-only, `/context` is a startup snapshot). serve ships a +minimum floor in the skeleton slice as an acceptance criterion (§ 5, slice 6), not a nicety: + +- **One structured line per request, to stderr** (stdout carries only the bound-address line): + method, path, status, problem `code` when non-2xx, total duration, semaphore-wait duration, + and UOW-acquire duration. The three timings are what separate "Dolt is slow" from "serve is + saturated" from "handler bug". +- **Startup lines**: bound address, workspace root, resolved mode (proxied / external / server), + and the effective limits (semaphore, connection cap, pool caps, deadline). +- **Shutdown lines**: signal received, drain start, drain complete — or forced close, with the + count of connections killed. +- **A saturation event** when the in-flight semaphore blocks a request for more than 1 s, and + one when a semaphore wait times out — the wedge-detection signal named above. +- **A `request_id` extension member** on every problem response (additive, allowed by the + envelope's rules; documented once on the Problem schema like the middleware 400), echoed in + the request log line, so a client-reported failure correlates to exactly one log line. + +Precedent, not invention: dbproxy already writes structured trace lines through its `tracef` +logger (`dbproxy/proxy/server.go:105`); serve must not ship with less visibility than the proxy +it fronts. Plain structured key=value text on stderr; log rotation, levels, and a metrics +endpoint remain non-goals. + +**When the SQL server dies mid-request.** In-flight statements fail; the mapper classifies +driver/net/`ErrBadConn`-class failures from `NewUOW` or query execution to a retryable 503 (never +404 — the hosted viewer's every-error-is-404 bug is structurally excluded because 404 is reachable +only via the normalized `ErrNotFound` sentinel from the shared function, rule 4). The provider is +retained across failures: the next request's `NewUOW` re-pings and, on the managed path, the +dbproxy respawn machinery re-establishes the child. Required integration test: kill the Dolt server +under a live serve → request returns 503 → restart Dolt → next request returns 200 with NO serve +restart. If self-healing proves incomplete at implementation time, the fix is a provider-recycle on +`driver.ErrBadConn` inside serve — a small, contained addition; flagged as the one open risk here. + +**Auto-commit — decided.** serve never touches the CLI's per-command protocol (`doltAutoCommit`, +`commandDidWrite`, `PersistentPostRunE` maintenance): those are non-atomic package globals sized for +one-shot commands. Durability is per-request: the claim's `uw.Commit` inside `RunTxResult` creates +exactly one Dolt commit per successful claim — identical cost and semantics to proxied +`bd update --claim` today. This is also the concurrency-correct choice by the codebase's own +testimony: `PersistentPreRunE` defaults auto-commit OFF in server mode because "firing DOLT_COMMIT +after every write under concurrent load causes 'database is read only' errors" (`cmd/bd/main.go`, +verified comment). Reads never commit; version-control cadence beyond per-claim commits stays the +operator's existing workflow. + +**Hooks — decided.** serve fires NO hooks. Facts: the UOW seam itself fires none, but the proxied +CLI update path DOES fire `on_update` synchronously in cobra-land (`fireProxiedUpdateHooks`, +`cmd/bd/update_proxied_server.go:230-252`, runner resolved from the process cwd, +`proxiedHookRunner` `:254-268`). So this is an honest, documented front-end divergence, not silent +drift: an HTTP claim will not run `on_update`, a CLI claim will. Justification: a hook is a +user-controlled subprocess per mutation — in a concurrent server that is an unbounded-latency +multiplier, an orphaned-child hazard at shutdown, and its cwd-derived hook-dir resolution is +meaningless in a server process; hooks are workspace-owner policy, part of the front end, not of the +shared contract (CAS semantics and result shaping remain identical). Recorded in the spec +description and `bd serve --help`; revisit behind an explicit `--fire-hooks` opt-in only on demand. + +**Who is the actor on an HTTP claim, and is the CAS fence real?** The actor is the required +`{"actor": "..."}` body field, validated at the wire edge (§ 3 claim: trimmed, non-empty, +≤ 256 bytes, no control characters — the domain layer alone checks only `actor == ""`). The server NEVER infers it: the CLI's derivation chain +(`getActorWithGit`, `cmd/bd/main.go:612-641` — `--actor` flag > `BEADS_ACTOR` > `BD_ACTOR` > git +`user.name` > `$USER` > "unknown") describes the server operator's identity, not the caller's, and +is meaningless for a remote client. Can a client spoof it? Yes: v0 has no authentication, so any +loopback client can present any actor — which is EXACTLY the CLI's trust model today, where any +local process can run `bd update --claim --actor alice`. Actor has always been caller-asserted +provenance for the audit trail, not authenticated identity. What this means for the fence: the CAS +is real as a mutual-exclusion guarantee and advisory as an identity guarantee. The claim is a +single-row compare-and-set inside the request transaction (`domain.issueUseCaseImpl.claim`, +`internal/storage/domain/issue.go:458-491`: `Updated` wins; held-by-another → +`storage.ErrAlreadyClaimed`; non-open → `storage.ErrNotClaimable`; same-actor re-claim → idempotent +success with `AlreadyClaimed: true`), so under any mix of honest and spoofed actors, N concurrent +claimers on one issue yield exactly one 200 and N−1 typed 409s, and no update is ever lost — the +property Gas City's dispatch actually depends on. A spoofer cannot STEAL a live claim through the +claim endpoint: the CAS refuses non-open rows, and eviction (`bd update --force`) is not exposed +over HTTP. What spoofing buys is misattribution: claiming an OPEN issue under someone else's name, +or a state-neutral idempotent "success" by presenting the current holder's name. Within the +loopback boundary that is the pre-existing local-user trust model; under `--allow-non-loopback` it +extends to every network peer, which is precisely why that flag prints the no-auth warning. The v1 +path, if remote exposure becomes real, is the dbproxy identity secret authenticating the client +channel, with actor then bound server-side to the authenticated principal. + +## 5. Test strategy, slice plan, risks, and open questions + +Package names below follow § 2: `internal/workapi` is the shared +filter/shaping contract, `internal/httpapi` is the HTTP layer. Every mechanism referenced here was +verified against the tree at `/data/projects/beads/.claude/worktrees/sd-server`. + +### Test strategy + +The repo has four test tiers, and this design uses all four. Verified mechanics: + +- **Pure unit tests** (no build tag) run in the required PR job via `make ci-pr-core` → + `scripts/ci/pr-core.sh` → `go test -race -short -skip '^TestEmbedded' ./...`. Anything + load-bearing that can run without Dolt MUST live here, because the cgo tiers are conditional. +- **`*_embedded_test.go`**: `//go:build cgo`, functions named `TestEmbedded*` (e.g. + `TestEmbeddedReady`, cmd/bd/ready_embedded_test.go:19). Excluded from ci-pr-core by the + `-skip '^TestEmbedded'` pattern; run in dedicated shards. +- **`*_proxied_integration_test.go`**: `//go:build cgo`, functions named `TestProxiedServer*`, + runtime-gated by `requireProxiedServerEnv` (`BEADS_TEST_PROXIED_SERVER=1` + + `testutil.RequireDoltBinary`, cmd/bd/proxied_integration_helpers_test.go:28-34). These drive the + freshly built `bd` binary as a subprocess with `BEADS_DOLT_PROXIED_SERVER=1` env (`bdProxiedRun`). + They run in 15-way shards in `.github/workflows/pr-risk.yml` (PR-time, only when + `ci-embedded-tier.sh` flips `full_embedded=true`) and in `main.yml` (push-to-main, which is + perpetually concurrency-cancelled — never rely on it as the only gate). New `TestProxiedServer*` + tests hash-distribute across shards until pinned in + `.github/scripts/proxied-cmd-test-shards.txt`; pin the serve tests there when they land. +- **Protocol corpus** (`cmd/bd/protocol`): `TestCorpusGolden` runs a deterministic command plan + against a from-source Dolt-backed `bd`, canonicalizes, and byte-compares against + `testdata/corpus/`. It is the primary no-op-refactor guard for slices 1-4. Any deliberate wire + change requires `make corpus-regen` plus a `JSONSchemaVersion` bump — the extraction slices must + require neither. +- Subprocess-test gotcha (known, recurring): rebuild the binary under test; a stale repo-root `bd` + silently passes old behavior. All new subprocess tests take the built-binary path from the + existing helpers, never `./bd`. + +#### Named tests and what each one proves + +**Tier A — pure, in ci-pr-core (these are the drift gates; they must never move to a cgo shard):** + +1. `TestWireTagBijection` (`internal/httpapi/wire_bijection_test.go`) — **rule 1**. Reflects over + the JSON tags of `types.Issue`, `types.IssueWithCounts` (which embeds `*Issue` and adds + `dependency_count`/`dependent_count`/`comment_count`/`parent`, internal/types/types.go:976-982), + and `types.IssueDetails`, walking embedded structs, and asserts **two-way set equality** with the + property names of the spec's `Issue`/`IssueWithCounts`/`IssueDetails` schemas (parsed from the + `//go:embed`-ed YAML). Divergence is allowed only through an explicit in-test allowlist where + every entry carries a comment justifying it. Proves: a field added to `types.Issue` without a + spec edit fails CI, and a spec property with no Go field fails CI. A round-trip fixture cannot + prove this (a new `omitempty` field absent from the fixture passes both directions), which is + why this test is reflection-based, not fixture-based. +2. `TestSpecRouteParity` (`internal/httpapi/spec_parity_test.go`) — the analog of the hosted + `TestSpecBijectionAndScopeParity`: exact set equality between the route table's + (method, specPath) pairs and the spec's `paths`. The `{id}:claim` custom-method route carries an + explicit `specPath` in the route table (ServeMux registers `POST /v0/beads/issues/{idop}`), so + the mapping is declared, never inferred. +3. `TestSpecDefaultsMatchSharedConstants` (`internal/httpapi/spec_parity_test.go`) — **rule 9**. + Parses the spec's `limit` parameter `default:` values and asserts equality with + `workapi.DefaultListLimit` (= 50, the value at cmd/bd/list.go:743) and + `workapi.DefaultReadyLimit` (= 100, cmd/bd/ready.go:796) — the same constants the cobra flag + registrations consume after the extraction slices. Also asserts the spec documents `limit=0` as + unlimited on both list and ready (**rule 2**), so the document cannot drift from the behavior. +4. `TestSpecStatusCodesMatchHandlerTable` — **rule 12**. Asserts the set of documented status codes + per operation equals exactly the reachable set in `problem.go`'s mapping table. No 413/429/504 + can be documented unless a mechanism ships that can emit it, and no handler can emit an + undocumented status. +5. `TestProblemMapping` (`internal/httpapi/problem_test.go`) — table over every sentinel: + `storage.ErrNotFound` and wrapped `sql.ErrNoRows` → 404 `not_found`; + `storage.ErrAlreadyClaimed`/`ErrNotClaimable` → 409 with extensions; serialization-exhausted → + the retryable code; arbitrary error → 500, **never** 404. Also pins the § 3 scrubbing rule: + for the 500/503 rows, an input error whose message embeds a fake DSN + (`user@tcp(127.0.0.1:3306)/db`) or dial target yields a problem body containing only the + static per-code `detail` — none of the input message may appear on the wire. +6. `TestBuildListFilterGolden` / `TestBuildReadyFilterGolden` (`internal/workapi`) — golden + equality tables **generated from the old inline builders before deletion** (the table literals + are produced in the extraction PR from the pre-move code): for a matrix of inputs, old + construction == `workapi.Build*Filter` output. The ready table additionally pins + `BuildReadyFilter(zero)` → `Status:"open"`, `Limit:100`. Any delta found while writing the + tables (e.g. the proxied path's dropped `MaxRows`) is triaged in the PR as an explicit behavior + fix, never silently absorbed. +7. `TestBuildIssueDetailsNotFoundNormalization` (`internal/workapi/detail_test.go`) — **rule 4, + unit half**. Fake `DetailSource` returning (a) an error wrapping `sql.ErrNoRows` — the real UOW + seam shape, `db.issueSQLRepositoryImpl.Get` returns bare `sql.ErrNoRows` + (internal/storage/domain/db/issue.go:394-410) — and (b) `(nil, nil)`. Both must normalize to + `errors.Is(err, storage.ErrNotFound)` **inside the shared function**, matching the dual-sentinel + treatment the proxied CLI already applies (cmd/bd/create_proxied_server.go:497). No caller-side + mapping is trusted. +8. `TestCursorRoundTrip` / `TestCursorRejects` (`internal/httpapi/cursor_test.go`) — codec + round-trip; tampered/undecodable cursor → typed error → 400 `invalid_cursor`. + +8b. `TestContextResponseAllowlist` (`internal/httpapi/context_test.go`) — **the § 3 context + field allowlist, enforced**. Builds the context handler over a `domain.ContextInfo` with + EVERY field populated with a distinct sentinel — `SyncRemote` set to a URL embedding a fake + credential — decodes the response into `map[string]any`, and asserts (a) the key set is + exactly the ten allowlisted properties and (b) the credential substring appears nowhere in + the raw body. Proves a future `ContextInfo` field (or a careless "serialize the struct" + refactor) cannot leak a sync-remote credential or host topology onto the wire by default. + +8c. `TestParamsRejectUnknown` (`internal/httpapi/params_test.go`) — **the § 3 + unknown-parameter rule, enforced at the decoder**. Table over every operation: a request + carrying an unrecognized query key (including a near-miss typo like `labels` and a + plausible next-version param like `mol_type`) → 400 `invalid_argument` with `param` naming + the offending key and `reason: "unknown_parameter"`; the parameterless operations + (`/healthz`, `/context`, get, claim) reject any query key at all; a known key with a bad + value yields `reason: "invalid_value"` with the same `param` member. Pure — drives the + hand-written decoders directly with `url.Values`, no server, no DB. Proves the decoders can + never regress to Go's silent-ignore default (which would silently WIDEN filtered result + sets under version skew) and pins the § 3 rule that strict rejection is the client's + per-parameter capability probe. + +**Tier B — cgo handler tests** (`internal/httpapi`, real UOW provider via +`testutil.EnsureDoltContainerForTestMain`, internal/testutil/testdoltserver.go:228): + +9. `TestServerListCursorPagination` — same-second `created_at` groups page without loss or + duplication across the keyset boundary; `limit=0` returns the full set with `has_more=false` + (**rule 2**; if `SearchPage.HasMore` turns out to be limit-probe-based, the handler forces + `has_more=false` at limit 0 and this test pins it). Also asserts the FIRST, un-cursored page + arrives in `(created_at DESC, id ASC)` order over seed data whose priority order differs from + its created order — pinning that the handler forces the sort on every request, not only in + the cursor-decode path (gap G7: the id-sorted parity oracles cannot catch this). +10. `TestServerReadyReturnsCounts` — **rule 3**. Response items strict-decode into + `types.IssueWithCounts` and carry correct nonzero `dependency_count`/`dependent_count`/ + `comment_count`/`parent` for seeded data — the field-level guard that the ready endpoint + matches `bd ready --json`'s actual element type. +11. `TestServerGetUnknownID404` — **rule 4, integration half, against real Dolt**: unknown id + through the real UOW provider → 404 `not_found` problem+json; a seeded id → 200; an + induced non-not-found failure (closed provider) → 500, proving 404 is unreachable from + arbitrary errors. +12. `TestServerClaimMatrix` — fresh claim 200; same-actor idempotent re-claim; foreign holder → + 409 `already_claimed` with `assignee` extension; closed issue → 409 `not_claimable`; unknown + id → 404; whitespace-only, > 256-byte, and newline-bearing actors → 400 `invalid_argument` + carrying `param: "actor"`, `reason: "invalid_value"`, with no row written (§ 3 wire-edge + actor rules); N concurrent claimers → exactly one success + (exercises `uow.RunTxResult`'s serialization retry, internal/storage/uow/tx.go:76 — the + single implementation, **rule 5**). +13. `TestServerDoltRestartRecovers` — kill the Dolt server mid-session → mapped retryable 503; + restart → next request 200 with the same provider (per-request self-heal requirement). +14. `TestDetailSourceConformance` (`internal/workapi`, cgo) — **rule 13**: the store-backed and + UOW-backed `DetailSource` adapters, run over the same seeded data (issue with labels, deps, + comments, and a wisp), produce byte-identical `BuildIssueDetails` JSON. Two implementations, + one assembly, machine-checked. + +**Tier C — end-to-end CLI/HTTP parity** (`cmd/bd/serve_proxied_integration_test.go`, cgo + +`BEADS_TEST_PROXIED_SERVER=1`, subprocess `bd serve --addr 127.0.0.1:0` with the printed bound +address parsed). These are the anti-drift oracles; per **rule 8** every one compares **full item +JSON** (decode both sides to `[]map[string]any`, sort by id, `go-cmp` after deleting only +allowlisted field paths, each with a written justification), never ID sets: + +15. `TestProxiedServerServeReadyParity` — `GET /v0/beads/ready` items vs `bd ready --json` for the + same filters (including `--limit 0`), over seeded data with deps/comments so the counts fields + are load-bearing. An ID-set oracle would have passed while the endpoint shipped bare `Issue`; + this one cannot. +16. `TestProxiedServerServeListParity` — `GET /v0/beads/issues` under defaults vs `bd list --json`, + seeded with a gate, a template, a closed issue, an infra-type issue, and a wisp — proving the + ~200 lines of exclusions apply identically (the hosted viewer's bare-`IssueFilter{}` bug class). + Also asserts `limit=0` HTTP output equals `bd list --limit 0 --json`. +17. `TestProxiedServerServeShowParity` — `GET /v0/beads/issues/{id}` vs `bd show --json ` + element [0], field-for-field; allowlist contains exactly one entry (the CLI's array-of-one + envelope vs the HTTP bare object). +18. `TestProxiedServerServeClaimCrossCheck` — claim over HTTP, then `bd update --claim` by a second + actor: the CLI sees the same conflict, and the HTTP 409's `assignee` extension matches; then + `bd show --json` confirms the HTTP-claimed state. Plus the item-level parity case claim was + missing (rule 8 — every other MVP operation has a full-item oracle, and "parity holds by + construction" is exactly the reasoning rule 8 forbids): a same-actor CLI re-claim + (`bd update --claim --json`, idempotent under CLI semantics) of the HTTP-claimed issue + full-item-JSON-compares its element `[0]` (the proxied path emits `[]*types.Issue`, + update_proxied_server.go:48) against `ClaimResponse.issue`, under the same allowlist + discipline as tests 15-17 — so a later CLI-side enrichment of update output (e.g. label + hydration) fails the oracle instead of drifting silently against the HTTP body. +19. `TestProxiedServerServeModeGate` — embedded workspace → typed `storage.ErrUnsupported` message, + exit 1; dolt-server-mode workspace before the server-mode slice lands → the staged refusal text + (asserting it does NOT claim server mode is supported, **rule 10**); non-loopback `--addr` + without opt-in → refusal; with `--allow-non-loopback` set, `?limit=0` → 400 + `invalid_argument` (the § 3 unlimited-read refusal is keyed on the flag, so this is testable + on a loopback bind); a request with a foreign `Host` header (e.g. `Host: evil.example`) → + 400 (the DNS-rebinding allowlist); SIGTERM → graceful drain, exit 0. + +**Corpus:** slices 1-4 are accepted only with `TestCorpusGolden` unchanged (no regen, no +`JSONSchemaVersion` bump) — `ready_front`, `json_contract`, `lease_claim`, and `errors_contract` +blobs are the byte-level proof the extractions were no-ops. + +**CI placement (explicit, because the CI map hides breakage):** tests 1-8c run in `ci-pr-core` +(pure, required). The regen-and-diff drift gate (`go generate ./internal/httpapi/...` + +`git diff --exit-code`) is added to `scripts/ci/pr-policy.sh` (invoked by the required +`make ci-pr-policy`). Tiers B and C run in the pr-risk cgo shards; the serve tests get pinned +entries in `proxied-cmd-test-shards.txt`. Nothing load-bearing lives only in `main.yml`. + +### Slice plan + +Ordered; each slice is independently shippable and leaves main green. Per **rule 11**, slices 1-4 +are pure no-op refactors, one extraction each, individually bisectable against the corpus. Per +**rule 10**: slice 6 delivers `bd serve` for **proxied** mode (where `uowProvider` already exists, +cmd/bd/main.go:1378-1388); slice 10 delivers **server / external-server / shared-server** modes by +constructing the existing exported providers at serve startup; between slices 6 and 10 those modes +get a typed refusal whose text promises nothing ("not yet supported by bd serve; supported today +under the proxied server"). + +1. **Extract list-filter construction** (no-op refactor). + Outcome: `buildListFilter` + `listFilterConfig` + the config-source seam move verbatim to + `internal/workapi/list.go` as `BuildListFilter`/`ListConfig`/`ConfigSource`, exported, with + `DefaultListLimit = 50`; both `bd list` variants delegate; `cmd/bd/list_filter.go` deleted. + Files: new `internal/workapi/list.go` + tests (incl. moved `list_filter_status_test.go` cases and + `TestBuildListFilterGolden`); delete `cmd/bd/list_filter.go`; edit `cmd/bd/list.go`, + `cmd/bd/list_input.go`, `cmd/bd/list_proxied_server.go`; `.golangci.yml` (depguard rule + `workapi-frontend-boundary`) and `scripts/ci/pr-policy.sh` (banned-accessor grep) per § 2's + import-policy enforcement — the boundary is machine-enforced from the package's first commit. + Accept: corpus unchanged; `list_embedded_test.go` + `list_proxied_integration_test.go` green; + golden table old==new; `grep -rn buildListFilter cmd/bd` returns nothing; lint fails on a + deliberate cobra import in `internal/workapi` (verified once, then reverted). +2. **Collapse ready defaults** (no-op refactor). + Outcome: one `BuildReadyFilter` (`Status:"open"`, `DefaultReadyLimit = 100`, normalization) in + `internal/workapi/ready.go`; the inline duplicate at `cmd/bd/ready.go:104-215` deleted; direct + path consumes `gatherReadyInput` like the proxied path; the cwd directory-label default stays in + `cmd/bd` (caller pre-fills the request). + Files: new `internal/workapi/ready.go` + tests; edit `cmd/bd/ready.go`, `cmd/bd/ready_input.go` + (+ moved `ready_input_test.go` cases). + Accept: corpus `ready_front` unchanged; `ready_test.go`, `ready_embedded_test.go`, + `ready_proxied_integration_test.go`, `metadata_ready_test.go`, `ready_max_rows_test.go` green; + golden table old==new; the proxied MaxRows delta (if confirmed) called out and decided in the PR. +3. **Shared detail assembly + not-found normalization** (no-op refactor). + Outcome: `BuildIssueDetails` + `DetailSource` (with the isWisp axis) + store/UOW adapters in + `internal/workapi/detail.go`; `bd show`'s direct assembly block (`show.go:147-235`) and the + proxied helpers (`proxiedGetIssueOrWisp` etc.) deleted; the shared function normalizes + {wrapped `sql.ErrNoRows`, nil-issue-with-nil-error} → `storage.ErrNotFound` (**rules 4, 13**). + Fuzzy resolution, `--current`, watch, and the array-of-one envelope stay CLI-local. + Files: new `internal/workapi/detail.go` + `detail_test.go` + `TestDetailSourceConformance`; edit + `cmd/bd/show.go`, `cmd/bd/show_proxied_server.go`. + Accept: corpus `show` blob unchanged; `show_test.go`, `show_embedded_test.go`, + `show_proxied_integration_test.go`, `cli_coverage_show_test.go` green; normalization unit test + green. +4. **One claim retry implementation** (no-op refactor, **rule 5**). + Outcome: `applyUpdateProxiedOne`'s bespoke `cenkalti/backoff` loop + (cmd/bd/update_proxied_server.go:~94-125) reimplemented as a `uow.RunTxResult` consumer + (T = the attempt's {issue, fail} pair); the whole-attempt-redo semantics, nothing-to-commit + tolerance, and the retries-exhausted stderr/exit contract are preserved by RunTxResult's + existing tested behavior. + Files: edit `cmd/bd/update_proxied_server.go` only. + Accept: corpus `lease_claim` + `errors_contract` unchanged; `update_proxied_integration_test.go`, + `update_conditional_proxied_test.go`, `update_proxied_server_test.go` green; no `backoff.Retry` + call remains in `update_proxied_server.go`. + Note: the facade's own deferred proxied-rewire PR is a downstream rebase consumer of this + RunTxResult consolidation (it requires proxied contract-pin tests committed and green first; + this slice contributes to that precondition). +5. **OpenAPI spec, codegen, drift gates** (inert — no runtime change). + Outcome: hand-written `internal/httpapi/spec/openapi.v0.yaml` (all six operations; `Issue`/ + `IssueWithCounts`/`IssueDetails` pinned to the canonical types via `x-go-type`, **rule 1**; only + the status codes the six operations need, **rule 12**; `limit` defaults 50/100 and `0=unlimited` + documented, with the `--allow-non-loopback` `limit=0` refusal in the `limit` description (§ 3); + `ClaimRequest.actor` constraints as `minLength`/`maxLength`/`pattern` (§ 3); the + unknown-query-parameter rejection rule and the 400 `param`/`reason` extension members + documented on every operation and on the `Problem` schema (§ 3); the 5xx static- + `detail` scrubbing rule in the problem writer + `TestProblemMapping` (§ 3); + the `/issues` description documents the always-forced `(created_at DESC, id ASC)` + order and its deliberate divergence from `bd list`'s default priority ordering, § 3 / gap G7); + `apigen/types.gen.go` via oapi-codegen types-only (go.mod `tool` directive); + Makefile `api-gen`/`api-check` (names per § 4); regen-diff check in `scripts/ci/pr-policy.sh`. + Files: `internal/httpapi/spec/{openapi.v0.yaml,embed.go}`, `internal/httpapi/apigen/{doc.go, + types.gen.go}`, `internal/httpapi/{wire_bijection_test.go,spec_parity_test.go(partial), + problem.go,problem_test.go}`, `Makefile`, `scripts/ci/pr-policy.sh`, `go.mod`. + Accept: tests 1, 3, 4, 5 green in ci-pr-core; `make api-check` clean, and fails on a spec + edit without regen; go.mod diff contains only the tool directive and its closure. +6. **`bd serve` skeleton: lifecycle, mode gate, /healthz, /v0/beads/context** (proxied mode live). + Outcome: `cmd/bd/serve.go` (`--addr` default `127.0.0.1:0`, `--allow-non-loopback`, + numeric-loopback validation per the `proxied_server.go:446-476` policy, printed bound address, + graceful shutdown on the existing signal context); `internal/httpapi` Server/route table/problem + writer; typed refusals: embedded → permanent `storage.ErrUnsupported`; server/external-server → + staged refusal (**rule 10** honest text); no flock/pidfile/serve-info.json (**rule 7** — the TCP + bind is the mutual exclusion); one-UOW-per-request with rollback via a detached close context; + `/v0/beads/context` from the `contextinfo.NewContextProvider` snapshot, serializing only the + § 3 field allowlist (SyncRemote hard-excluded). The § 4 front-tier hardening lands with the + skeleton, not later: `Host`-header allowlist middleware, `netutil.LimitListener` connection + cap, the DB semaphore (bounded 10 s acquisition wait → 503 `busy`) with `/healthz` + + `/context` bypass, the per-request 60s deadline, and explicit pool limits on the provider's + `*sql.DB` (§ 4, concurrency limits). The § 4 observability floor lands here too: the + per-request structured stderr line (method, path, status, problem `code`, duration, + semaphore-wait, UOW-acquire), startup and shutdown lines, the semaphore-saturation event, and + the `request_id` problem member. + Files: `cmd/bd/serve.go`; `internal/httpapi/{server.go,routes.go,handlers.go(healthz,context)}`; + `cmd/bd/serve_proxied_integration_test.go` (mode-gate + lifecycle cases). + Accept: test 2 green (route table = spec paths; unimplemented ops registered as transitional + 501 stubs so route/spec parity holds all-at-once). The stubs are an explicit, temporary + rule-12 exemption, not a silent one: 501 is never documented spec surface, so + `TestSpecStatusCodesMatchHandlerTable` (test 4) carries a commented in-test exemption listing + exactly the stubbed operations, deleted together with the last stub in slice 9 — after which + the exemption list being non-empty fails the test. `/v0/beads/context` `capabilities` is + derived from implemented handlers only and never lists a stubbed op (§ 3), so a release cut + between slices 6 and 9 advertises only what works. Tests 8b and 19 green. The observability + floor is an ACCEPTANCE CRITERION of this slice, not a convention: the lifecycle integration + cases assert the startup lines, one request log line per request (with the § 4 fields), the + shutdown drain lines, and the saturation event under a synthetic semaphore-full condition. + +6b. **Reader role** (owner-decided; inserted by the issueops reconciliation — GATED on the + `feat/public-issueops-facade` merge: the leaf `issueops` package must exist on OSS main + before `Reader` can be declared). + Outcome: the leaf `issueops.Reader` contract (Reader interface, + `ReadyRequest`/`ListRequest`/`GetRequest`/`IssuePage` — § 2, "The Reader role"); + `IssueReader()` accessors on `Storage`, `DoltStore`, `EmbeddedDoltStore` (cgo + !cgo stub), + `HookFiringStore` (passthrough by recursion), `InstrumentedStorage` (wrap by recursion), the + UOW provider (+ the `configStore` test fake); the `storeReader` and `uowReader` + implementations over `workapi` (builders re-typed to the leaf request types; one UOW per + Reader method with detached-close inside the uow reader). + Files: new `issueops/reader.go` (leaf) + per-backend `*_reader.go` in + `internal/storage/{dolt,embeddeddolt,uow}` + accessor additions + tests. + Accept: cross-backend conformance tests green (store vs uow Reader agree on the parity + oracle corpus shapes); `go list -deps` verification that the leaf package imports only + `internal/types` + stdlib. +7. **Read endpoints: `/v0/beads/ready` + `/v0/beads/issues`** (gated on 6b). + Outcome: param decoding → `provider.IssueReader().Ready` / `.List` — handlers NEVER call + `workapi` builders or wire `ConfigSource`; the Reader subsumes filter/default construction + and one-UOW-per-call execution; ready returns + `IssueWithCounts` items (**rule 3**); list gets the opaque keyset cursor (token codec stays + HTTP-layer, decoded position passed as `ListRequest.AfterCreatedAt`/`AfterID`); `limit=0` + unlimited on + both (**rule 2**; refused with 400 under `--allow-non-loopback`, § 3); sentinel mapping live. + Files: `internal/httpapi/{params.go,cursor.go,handlers.go}` + tests. + Accept: tests 8, 8c, 9, 10 green (8c includes the strict unknown-parameter rejection on + every operation, § 3); parity oracles 15, 16 green. +8. **Detail endpoint: `GET /v0/beads/issues/{id}`** (gated on 6b). + Outcome: HTTP handler consumes `provider.IssueReader().Get` — which runs + `workapi.BuildIssueDetails` through the UOW detail source inside the reader (no new + assembly copy, **rule 13**); exact-ID + wisp fallback; 404 only via the normalized sentinel. + Files: `internal/httpapi/handlers.go`. + Accept: tests 11, 17 green. +9. **Claim endpoint: `POST /v0/beads/issues/{id}:claim`**. + Outcome: handler consumes `uow.RunTxResult` (the same single implementation the proxied CLI now + uses after slice 4, **rule 5**), commit message `"bd serve: claim by "`; wire-edge + actor validation before the CAS (trim / length / control characters, § 3); typed 409s + with `assignee`/`issue_status` extensions; 501 stub replaced; hooks do not fire (seam-consistent + with every proxied write today); `uow.RunTxResult`'s per-attempt close switched to the + detached-context pattern (§ 4, "the write path gets the same detached-close protection" — + fixed in the one implementation per rule 5, so the proxied CLI inherits it). + Files: `internal/httpapi/handlers.go` (+ the `{idop}` suffix split); + `internal/storage/uow/tx.go` (detached per-attempt close); spec already covers the op. + Accept: tests 12, 18 green; concurrent-claim race: exactly one 200; a client disconnect + mid-claim leaves the provider serving subsequent requests without a burned session (the + detached-close verification). + The § 1 coordination-bead check happens before this slice ships, in `Lifecycle` vocabulary: + the facade owner reviews the endpoint's documented semantics against + `Lifecycle.Update`/`UpdateRequest.Claim` together with the recorded reasons the claim stays + BELOW `Lifecycle` (§ 2, post-facade justification: fixed commit message, discarded + `AlreadyClaimed`, unreachable same-tx 409 re-reads, moot granularity, internal-seam scope). +10. **Server / external-server / shared-server mode support** (**rule 10** delivery). + Outcome: `bd serve` startup constructs a UOW provider for non-proxied SQL-server workspaces by + calling the existing exported `uow.NewDoltServerUOWProvider` / + `uow.NewExternalDoltServerUOWProvider` (internal/storage/uow/doltserver_provider.go:16, + external_doltserver_provider.go:18 — already tested, already consumed by cmd/bd/uow_factory.go) + through the same config-resolution path `newProxiedServerUOWProvider` uses; serve is excluded + from post-run auto-commit maintenance in these modes; staged refusal deleted. + Files: `cmd/bd/serve.go`; small extraction in `cmd/bd/uow_factory.go`; possibly + `cmd/bd/main.go` post-run policy. + Accept: integration test against an external dolt sql-server workspace answers all six + operations; embedded still refused; test 19's staged-refusal assertion updated. + +10b. **CLI re-point** (no-op refactor; inserted by the issueops reconciliation; follows 6b). + Outcome: direct + proxied `bd ready` / `bd list` / `bd show` route through the + `IssueReader()` accessors instead of calling the `workapi` builders at the command layer; + the fetch-one-extra / `SQLLimit` / client-side-sort trim logic (today at + `cmd/bd/list.go:42-76`) relocates INTO the store reader. CLI-local concerns stay in + `cmd/bd`: fuzzy/substring resolution, `--current`, watch mode, the array-of-one show + envelope, the cwd directory-label default (CLI pre-fills `ReadyRequest.LabelsAny`), MaxRows + resolution and proxied rejection, output envelopes. Commands outside the four-command scope + (`count.go`, `search.go`, `graph_apply.go`) keep calling `BuildListFilter` directly — they + are not part of the six-operation anti-drift surface, and the facade's governing rule + constrains the public capability surface, not internal helper use. + Files: `cmd/bd/{ready,list,show}*.go`; `internal/storage/dolt` store reader. + Accept: behavior-sensitive and corpus-refereed exactly like slices 1-4 — protocol corpus + unchanged, goldens old==new, bisectable to one move. +11. **Docs + downstream handoff**. + Outcome: engdocs page (surface, error codes, cursor contract, loopback posture, no-hooks and + no-auto-commit decisions, claim-throughput note) plus the operator runbook material this + proposal commits to: the fixed-port deployment recommendation and the ephemeral-default + honesty note (§ 4, rule 7), the `GET /v0/beads/ready?limit=1` readiness-probe recipe with + `/healthz` documented as liveness-only (§ 4, concurrency limits), the SIGHUP/supervisor + note and the ambiguous-shutdown re-claim recovery (§ 4, graceful shutdown), the + connection-budget arithmetic for shared external Dolt servers, and the request-log line + format (§ 4, observability floor) — all written **vendor-neutrally** per binding + constraint 1: it addresses generic "automation clients / orchestrators", names no downstream + consumer, product, or fork, and motivates each decision from the API contract itself (e.g. + "clients that classified claim conflicts by substring-matching error text should switch to + the typed 409 `code`"), not from any one consumer's internals. CHANGELOG likewise. The + consumer-specific adoption checklist (delete substring claim classification, the show + `[0]`/`ErrIDCollision` guard, the version-string surgery) is a **downstream deliverable**: + it is handed off to and lives in the consuming fork's own tree, and is never committed to + OSS beads. That checklist must also state the negative honestly: the reconciler's list path + (the `TierBoth` full scan) is NOT on the v0 migration list — it stays on its `bd list` + + `bd query "ephemeral=true..."` subprocess pair (§ 3, "what this list surface deliberately + cannot serve"), so none of that loop's runner/parse/retry machinery is deletable until an + ephemeral/wisp-inclusion parameter (and a labels-omission marker) ships in a later version. + Files: `engdocs/`, `CHANGELOG.md`. Accept: docs-drift scripts pass; no code; a + vendor-neutrality grep of the new docs (no hosted-product or consumer names) is part of PR + review for this slice. + Disposition of THIS document: it is a working design artifact, not a slice deliverable. It + does not land in the OSS tree as-is — it cites downstream-consumer internals throughout, + which binding constraint 1 bars from committed OSS docs. If the owner wants a committed + design record, the engdocs page above is that record; landing this proposal verbatim would + require the same genericization scrub. + +### Risks + +- **The bijection test is the only guard on the wire schema.** With `x-go-type` pinning there is no + compiler check that spec and struct agree; if test 1 ever moves out of `ci-pr-core` (or is + skipped under `-short`), the drift gate is gone. Mitigation: it is pure and fast by construction; + CI placement is an acceptance criterion of slice 5, not a convention. +- **Extraction deltas.** Slices 1-4 move corpus-pinned behavior; the golden-table method surfaces + deltas (the proxied ready `MaxRows` drop is already suspected) but each surfaced delta forces a + product decision mid-slice. Policy: triage in the PR, never silently absorb; a delta that changes + wire bytes moves to its own follow-up slice with a corpus regen. +- **`SearchPage.HasMore` semantics under `limit=0` and under keyset filters are unverified.** If + HasMore is limit-probe-based, `limit=0` must force `has_more=false` explicitly; test 9 pins + whichever way it lands, but the implementation must check before wiring cursors. +- **`limit=0` responses are fully buffered — and the OOM blast radius is NOT "same as the CLI".** + Each unlimited response is double-buffered per request (query-layer rows plus the full JSON + marshal). Today an OOM kills one forked `bd list --limit 0` process and nothing else; under + serve, the same scans run inside ONE long-lived shared process — 16 concurrent + reconciler-style full active-set scans can OOM the process that every client, including the + dispatch loop, depends on. The fork model's isolation is silently lost, and this document says + so rather than calling the exposure equivalent. It also changes the answer to open question 2: + this proposal's recommendation (SRE review) is to ship the off-by-default `--max-rows` + operator flag in v0, not later. The non-loopback half of the hazard stays CLOSED — + `--allow-non-loopback` refuses `limit=0` with 400 (§ 3, limit semantics), so the unbounded + buffer is never network-reachable. +- **RunTxResult adoption (slice 4) is behavior-sensitive at the failure edges**: the + retries-exhausted path currently prints a specific stderr line and returns a per-issue failure + rather than an error. If RunTxResult's error surface cannot express that without contortion, the + slice must adapt the call site, not fork the retry loop — `errors_contract` corpus is the + referee. +- **Provider self-heal after Dolt/dbproxy idle-stop is assumed, not yet proven.** Test 13 is the + proof obligation; if `NewUOW` does not re-establish, serve needs a provider-recycle on bad-conn, + which is a small but real scope addition to slice 6. +- **Server-mode staging gap (slices 6→10).** Decided parameter 5 names three modes; between the + slices only proxied works. The refusal text is written to promise nothing, but docs and release + notes must not advertise `bd serve` for server-mode workspaces until slice 10 ships. +- **Claim throughput is bounded by Dolt's per-write commit** (~0.4 s measured in prior perf work): + roughly 2 claims/sec sustained. Fine for a 30 s dispatch patrol; documented so nobody builds a + hot claim loop against v0. +- **cgo tier conditionality.** Tiers B/C run on PRs only when the risk-tier script flips + `full_embedded=true`; a doc-only PR touching the spec YAML might skip them. The pure gates + (tests 1-8c) are deliberately sufficient to catch schema/route/default drift on their own. +- **go.mod churn from the oapi-codegen tool directive.** A prior PR was reverted for unrelated + go.mod noise; slice 5's acceptance pins the diff to the directive plus its closure. +- **Read-endpoint slices are gated on the facade merge.** Slices 6b/7/8 require the facade's + API-shape phase to merge to OSS main first — the leaf `issueops` package must exist before + `Reader` can be declared. The schedule exposure is an owner call (open question 9's + contingency), managed visibly rather than silently re-deciding the architecture. +- **The CLI re-point slice (10b) moves over-fetch/trim behavior into the store reader.** The + fetch-one-extra/`SQLLimit`/client-side-sort trim relocation is behavior-sensitive and must be + corpus-bisectable exactly like slices 1-4; any surfaced delta is triaged in the PR, never + silently absorbed. + +### Contradictions resolved during assembly + +The five sections were drafted in parallel; where they disagreed, one position was applied and +the tradeoff is recorded here rather than silently dropped. + +- **Ready/list response envelope**: one draft floated a bare JSON array for `/ready` (byte-parity + with `bd ready --json`); § 3's decided `{items, has_more}` object envelope wins — extensible and + consistent with the paginated list endpoint — at the cost of one deliberate CLI/HTTP shape + difference the parity oracles allowlist (wrapper only; item JSON stays field-identical). +- **`/issues` item type**: § 3 drafted `items: [Issue]`; § 2's `IssueWithCounts` wins — verified + `bd list --json` calls `SearchIssuesWithCounts` on both CLI paths (cmd/bd/list.go:574-580, + list_proxied_server.go:78) — at the cost of three counts columns on every list row (the price + of rule 8 field-level parity). +- **Detail `include_*` HTTP knobs**: § 2's `DetailOptions` comments implied HTTP + `include_dependents`/`include_comments` params; § 3's no-params surface wins for v0 (rule 12 + minimality) — the library keeps `DetailOptions` for the CLI, HTTP passes the zero value, and + the comments knob remains open question 3 below. +- **Makefile target names**: `api-gen`/`api-check` (§ 4, "names final") over the plan draft's + `generate-api`/`check-api-drift`. +- **Test names**: normalized to § 5's inventory (`TestWireTagBijection`, `TestProxiedServerServe*` + — the prefix the cgo shard mechanics require — `TestBuild*FilterGolden`, + `TestServerGetUnknownID404`); § 5's route test renamed `TestSpecRouteParity` to match § 4's + `make api-check` gate. +- **Rule numbering**: § 1's design-rules list renumbered 1-15 so the "rule N" citations used by + every other section resolve against it. + +### Open questions for the product owner + +1. **v0 release boundary**: tag/announce v0 after slice 9 (proxied-only, staged refusal for + server modes) — or hold the announcement until slice 10 so decided parameter 5 is fully true on + day one? +2. **Unlimited-read cap (narrowed by security review; recommendation added by SRE review)**: the + security half is decided — `--allow-non-loopback` refuses `limit=0` with 400 (§ 3), so no + unbounded buffer is ever network-reachable. What remains is NOT mere loopback ergonomics: + moving full active-set scans from per-fork CLI processes into one shared process changes the + OOM blast radius from "one command dies" to "the API dies for every client, dispatch + included" (§ 5, Risks). The proposal's recommendation is therefore to ship the off-by-default + `--max-rows` operator flag in v0 (wire semantics unchanged when unset — binding rule 2 + intact; when an operator opts in, refusals surface as a typed 400 `too_many_rows`, a + permanent code-vocabulary addition, which is why this still needs owner sign-off). Owner + either/or: accept the v0 flag as recommended — or keep v0 capless on loopback (exact CLI + parity) and accept the documented shared-process OOM risk? (A streaming JSON encoder was + considered and not chosen: it removes only the marshal copy — the query layer still buffers + every row — so it halves the footprint without bounding it.) Related either/or the security + review surfaced: keep the non-loopback `limit=0` refusal as shipped, or instead make + `--max-rows` mandatory at startup when `--allow-non-loopback` is set (a cap instead of a + refusal)? +3. **Comments in the detail endpoint**: mirror `bd show`'s default exactly (count-only, + `comments_omitted` semantics, no way to fetch bodies over HTTP in v0) — or add + `?include_comments=true` in v0 (one more spec knob, but remote clients have no other way to read + comments)? +4. **500 detail verbosity** — **closed during security review, no longer open**: 500 `internal` + and 503 `db_unavailable`/`busy` carry a generic static `detail`; the real error (which can + embed the DSN or dial target) goes to the server log only. See § 3, "5xx detail scrubbing". + Numbering kept so cross-references to questions 5-6 stay stable. +5. **HTTP contract corpus**: extend the byte-pinned corpus mechanism with a vendorable HTTP-response + corpus in v0 (same producer/consumer contract Gas City already replays for CLI JSON) — or defer + to v0.1 and let the spec + bijection tests carry the contract alone until the surface settles? +6. **Committed design record**: rely on slice 11's vendor-neutral engdocs page as the sole + in-tree design record (this proposal stays out of tree; see slice 11 "Disposition") — or fund + a genericization scrub of this proposal so a scrubbed copy can land in `engdocs/` alongside it? +7. **`api_revision` context field (client version-skew review)**: v0's parameter-level + capability gate is strict unknown-parameter rejection — a client probes by sending and gets + a machine-attributable 400 (§ 3, version skew). Ship an `api_revision` field on `/context` + exposing the spec's `info.version` (bumped on every spec change, enforceable by extending + the pr-policy drift gate: a spec diff without an `info.version` change fails) so clients can + gate on parameters without probing — or keep the context allowlist at its ten fields and let + probe-by-400 carry parameter gating until a real consumer reports the probe as insufficient? +8. **`/v0` disposition on the day a `WireSchemaVersion` bump forces `/v1`** (maintainer review; + documented as deferred-to-that-day in § 3's evolution policy — no decision is needed until a + breaking wire change is actually proposed): when that day comes, sunset-and-remove `/v0` on a + documented deprecation window — or freeze `/v0` by snapshotting the pre-bump shapes as a + deliberate version-suffixed mirror struct served only by `/v0` handlers (a round-trip fixture + replacing the bijection test for the frozen shape)? +9. **Facade-slip contingency (issueops reconciliation)**: the read-endpoint slices (6b/7/8) gate + on the facade's API-shape phase merging to OSS main. If the facade slips, does the owner + accept the delay — or authorize declaring the Reader contract temporarily in an internal + package (relocated to the leaf `issueops` package when the facade lands), trading a later + mechanical move for schedule decoupling? +10. **`IssueReader` accessor naming**: `IssueReader()` sits beside the existing internal + `IssueLifecycleStore` lane — the name is grep-verified free in both trees today, but confirm + the naming convention (`IssueReader` vs e.g. `IssueQueries`) before it becomes public API. +11. **`Offset` on the public contract**: `ReadyRequest`/`ListRequest` carry `Offset` "honored + where the backend supports it" — keep a best-effort field on the public leaf contract, or + drop it and leave offset pagination CLI-internal? +12. **`*int` field conventions**: `Limit *int` (nil = default, 0 = unlimited) and `Priority *int` + (nil = unset) replace the CLI's paired value+`Set` flags on the public request structs — + confirm the pointer-field convention for the leaf contract before it freezes. +13. **v0 tag vs the CLI re-point slice**: does the v0 tag/announcement wait for slice 10b (so the + CLI and HTTP surfaces both sit on the accessors from day one) — or ship after slice 9/10 + with 10b as a fast-follow no-op refactor? + +--- + +## Reconciliation with the issueops Lifecycle refactor + +This section records the owner's reconciliation of this proposal with effort B, the +`feat/public-issueops-facade` branch. It is the authority for the amendments marked +"issueops reconciliation" throughout the document (non-goals bullet, binding rule 6, § 2's +Reader role, slices 6b/10b, risks, open questions 9-13). + +### What effort B changes + +The facade branch (35 commits ahead of origin/main at base `8bb0d36be`) establishes the public +`issueops.Lifecycle` role — {Create, Update, Close, Reopen}, the rename from `Operations` +binding per its final design spec, not yet in code at branch HEAD — exposed via an +`IssueLifecycle()` accessor on `Storage`, with the leaf `issueops` package at the repo root +importing only `internal/types` + stdlib. It relocates the existing 18 storage/domain error +symbols into that leaf package (aliased back from `internal/storage` with the same pointer; no +new sentinels), routes the four direct CLI write verbs through the facade via +`cmd/bd/issueops_adapter.go`, and defers the proxied-server rewire to its own follow-up PR gated +on proxied contract-pin tests. + +### The decision (owner-decided, binding) + +**BOTH, layered.** `issueops.Reader` is the public read ROLE (binding decision 1), and +`internal/workapi` is the single shared IMPLEMENTATION substrate behind every Reader +implementation. Neither replaces the other: the leaf package declares the contract; workapi +keeps the ~310 lines of construction/shaping logic; the per-backend readers glue them under the +accessor. This is **subsume** (binding decision 2): construction happens once, inside the +implementations, and both front doors can only say `rd.List(ctx, req)`. The full contract, the +accessor set, and the implementation internals are specified in § 2, "The Reader role". + +`internal/workapi` stays internal PERMANENTLY — the room behind every door. Reader +implementations call: `BuildListFilter` + `LoadListConfig` + the `ConfigSource` adapters (List), +`BuildReadyFilter` (Ready), `BuildIssueDetails` + `DetailSource` adapters + +`ResolveIssue`/`IsNotFound`/`NotFound` (Get), and `Default{Ready,List}Limit` (nil-Limit +defaulting). `ConfigSource` is supplied by the implementation from what it already holds — never +by front doors, which is the drift kill. Still called directly from `cmd/bd`, honestly: +(a) commands outside the four-command scope (`count.go`, `search.go`, `graph_apply.go`) keep +calling `BuildListFilter` — the governing rule constrains the public capability surface, not +internal helper use; (b) the claim path (proxied `bd update --claim` and the HTTP claim handler) +uses `ClaimOnUOW`/`ClassifyClaimError` directly; (c) CLI-local stays outside the contract +entirely: fuzzy/substring id resolution, `--current`, watch mode, the array-of-one show +envelope, the cwd directory-label default, MaxRows resolution and proxied rejection, output +envelopes, and the cursor token codec (HTTP layer). + +### Rationale (the load-bearing discoveries) + +1. **The leaf constraint binds only the CONTRACT package.** Verified on the facade branch: + `internal/storage/uow/issue_operations.go` and `dolt/issue_operations.go` implement the + public contract while importing `internal/storage` freely — so Reader implementations sit + beside `NewIssueOperations` in dolt/embeddeddolt/uow and import `workapi` and + `internal/config` without restriction. The "subsume is structurally impossible" claim + conflated these layers and is refuted. +2. **Subsume, not compose, neutralizes the council's two sharpest objections.** Request-granular + methods mean the uow reader is one-UOW-per-call (no transaction fragmentation — the objection + targeted a fine-grained compose-shaped Reader), and construction disappearing behind the + accessor means the two front doors cannot re-become two builder callers. +3. **The already-built work is the substrate, not a casualty.** Slice 1's `BuildListFilter` + (456 lines + 2890-line golden + depguard boundary), slice 2's `BuildReadyFilter`, and + slice 3's `BuildIssueDetails`/`DetailSource` are exactly what the Reader implementations + call — the list-filter and claim-retry branches merge now as-is; the ready/detail + extractions finish as specced; one later re-point slice (10b) moves the four commands' call + sites onto accessors and relocates the fetch-one-extra/SQLLimit trim into the store reader + (behavior-sensitive, corpus-refereed) — a follow-up, not rework. +4. **The claim endpoint changes nothing.** Verified on the facade branch: `Lifecycle.Update` + runs its own `RunTxResult` with a fixed "update issue" commit message (double-wrapped retry, + lost `"bd serve: claim by "` message), `ApplyUpdate` discards domain + `ClaimResult.AlreadyClaimed` (the wire's `already_claimed` field needs it), and on a lost CAS + the facade returns only the sentinel with the transaction gone — making the 409 + `assignee`/`issue_status` same-tx re-read extensions unreachable; the per-issue-granularity + argument is moot for a single-issue claim. B's governing rule governs the public capability + surface and does not conscript internal seam consumers — `bd serve`'s claim rides the same + seam proxied `bd update --claim` uses today, and B itself defers the proxied rewire. +5. **Sequencing is nearly free.** fv4-vs-facade overlap is zero files, m2u-vs-facade is one + additive test file, and the facade is by its own spec one API-shape phase from PR-ready, so + it merges last and resolves the single union conflict. The one real cost, surfaced honestly: + declaring `Reader` requires the leaf `issueops` package to exist on OSS main, so the + read-endpoint slices now gate on effort B's merge — flagged to the owner (risk + open + question 9) rather than hidden. + +### The strongest objection (recorded, not hidden) + +The best honest case against subsume-with-a-Reader-role is the council's own finding, sharpened: +a Reader role today is the execution port rule 6 rejected, wearing effort B's naming. It +converts reads that are currently zero-cost passthroughs (both decorators embed and override +only writes; both seams answer every needed query in one line) into ~5 accessor implementations +plus leaf page types; the store implementation must adopt exactly the limit+1 over-fetch the +council refused to force onto the direct CLI path; and it couples this epic's read endpoints to +an unmerged branch whose binding API shape (Lifecycle rename, accessor, leaf-ification) is not +yet in code — speculative coupling of the kind rule 6 exists to prevent. Meanwhile, the argument +runs, the anti-drift goal was already achieved by deletion: after slices 1-3 the old builders no +longer exist, so a second construction copy is a compile error no matter who calls +`BuildListFilter` — compose gets the drift protection without the adapter weight. + +Why it does not win: deletion secures construction UNIQUENESS but not call-site symmetry. Under +compose, each front door still performs the four-step ritual — build a `ConfigSource`, load +config, call the builder, execute on its seam — and a caller that skips or half-performs the +ritual compiles fine; that is precisely the topology that shipped the hosted viewer's bare +`IssueFilter{}` bug this epic exists to kill. Under subsume the ritual is not callable from a +front door at all: handlers and RunE bodies can only say `rd.List(ctx, req)`, so the failure +mode is unwritable — a strictly stronger guarantee than "the shared function exists". The +transaction-fragmentation half of the objection dissolves under subsume specifically: +request-granular methods mean the uow reader opens exactly one UOW per call, the proposal's own +one-UOW-per-request shape; that objection was valid only against a fine-grained compose-shaped +Reader. The over-fetch and adapter costs are real but bounded, paid once inside two +implementations rather than by every caller, and were explicitly accepted by the owner with the +council finding recorded as the counter-argument. The schedule coupling is the one genuinely +live cost, and the answer is to manage it visibly (the facade-merge gate on slice 6b, plus open +question 9's contingency) rather than to let it silently re-decide the architecture. + +### Merge order + +Order: (1) `refactor/bd-fv4-workapi-list-filter`, (2) `bd-m2u-collapse-claim-retry-loop`, +(3) `feat/public-issueops-facade`. All three share base `8bb0d36be` (current-main lineage). + +1. **bd-fv4 first**: reviewed, protocol corpus verified unchanged, zero file overlap with BOTH + other branches (verified: its 17 files touch `cmd/bd/list*/count*/search*/graph_apply.go` + + `internal/workapi/*` + `.golangci.yml` + `scripts/ci/pr-policy.sh`; the facade's 147 files + and bd-m2u's 4 files touch none of them). It also creates the workapi package + depguard + boundary that the ready/detail extractions stack on (their branches literally contain its + commits `af56541bb`/`e2d82a54d`). +2. **bd-m2u second**: disjoint from fv4 (`update_proxied_server*`, `uow/tx.go`, + `uow/tx_test.go`), merges clean. Landing fv4+m2u (and then the ready/detail extractions, + which slot in anywhere before the facade — the facade touches none of their extraction + targets) first also supplies part of the "proxied contract-pin tests committed and green + FIRST" precondition B's own spec sets for its deferred proxied-rewire PR. +3. **feat/public-issueops-facade last** — the only branch not PR-ready (at HEAD the Lifecycle + rename, `IssueLifecycle()` accessor, `beads_issueops.go` deletion, and 18-symbol relocation + are all still pending; the YAGNI cuts and relocation are partly uncommitted working-tree + state). After its API-shape phase is committed, it rebases onto main (now containing + fv4+m2u). + +**The single conflict — `internal/storage/uow/tx_test.go` — resolves as the UNION of both sides** +(the hunks are disjoint regions): keep the facade's widened `mockUnitOfWork` (five use-case +fields + accessor methods returning them), its `mockUnitOfWorkProvider.newUOWCalls` counter, its +`sqlStateError` type and `TestRunTx_RetriesOnPostgresSerializationStates`; AND keep bd-m2u's +added `time` import plus `TestRunTxResultWithin_ExhaustedBudgetReturnsSerializationError` +verbatim. The union is semantically coherent, verified: bd-m2u's test uses a zero-value +`mockUnitOfWorkProvider{}` and `newMySQLError(1213)`, both untouched by the facade's edits; the +facade's `issue_operations.go` calls `RunTx`/`RunTxResult` whose signatures bd-m2u preserved (it +only added `RunTxResultWithin` and exported `DefaultTxRetryMaxElapsed`, and the facade does NOT +touch `tx.go`; the facade's `errors.go` SQLSTATE widening composes cleanly since bd-m2u does not +touch `errors.go`). Post-merge check: re-run the facade's five counted baselines — the uow +package count becomes 128 (127 + bd-m2u's added test), all other counts (72/55/802/40) +unchanged. + +### Residual risks + +- The read-endpoint gate: slices 6b/7/8 wait on the facade's API-shape phase reaching OSS main + (leaf `issueops` must exist before `Reader` can be declared); contingency is open question 9. +- The CLI re-point (10b) relocates over-fetch/trim behavior into the store reader — + behavior-sensitive, corpus-bisectable like slices 1-4. +- The facade's binding shape (Lifecycle rename, accessor, leaf-ification) is spec-committed but + not yet code at branch HEAD; if the shape shifts during its API-shape phase, the Reader + contract inherits the shift before slice 6b, not after. + +--- + +## Council review — dispositions + +Four reviewer passes were applied to this document in sequence: `council:maintainer`, +`council:security`, `council:sre`, `council:consumer`. Every reviewer verdict was **revised**. +This appendix is the ledger: what each pass changed (one line), every item each pass deliberately +did NOT apply with its reason, and the result of the post-review consistency check across the +values multiple reviewers touched. Owner-facing either/ors raised by the reviews are consolidated +in § 5, "Open questions" (questions 2, 6, 7, 8) — each tension appears there exactly once. + +### council:maintainer — revised + +**Applied (one line):** forced `(created_at DESC, id ASC)` list order on every request with the +sqlbuild skip-and-duplicate bug mechanics, honest gap G7, and the test-9 first-page ordering +assertion; the welded-domain evolution policy (additive-only `/v0`, `WireSchemaVersion` bump ⇒ +cut `/v1`); depguard `workapi-frontend-boundary` rule + `pr-policy.sh` grep replacing +review-enforced import policy (parity row 11, slice-1 accept); slice-11 vendor-neutral +genericization + the "Disposition of THIS document" paragraph + open question 6; context +`capabilities` derived from implemented routes with the transitional 501-stub test exemption; +the exact 10-field context wire surface with the 9 withheld `ContextInfo` fields; mode-gate +refusals keyed on `Backend:"dolt"` vocabulary. + +**Deferred:** + +- Genericizing the ~23 Gas City / downstream-consumer references scattered through the proposal + body itself — the reviewer's requirement was to resolve the disposition, not rewrite the + document: the applied slice-11 "Disposition" paragraph states the proposal does not land in + the OSS tree as-is and that landing it would require the same scrub, and open question 6 puts + the scrub-vs-engdocs-only choice to the owner. A full-body scrub would be a whole-document + rewrite, against the targeted-edit mandate and colliding with three parallel reviewer passes. +- Choosing between `/v0` sunset-and-remove vs frozen version-suffixed mirror structs on the day + a `WireSchemaVersion` bump forces `/v1` — the new evolution policy deliberately defers the + (a)/(b) choice to that day while making non-deferrable the rule that corpus-regen + + `JSONSchemaVersion` bump alone is insufficient once `/v0` ships; picking now would be + speculative design for an event with no date, and option (b) touches the "no second wire + struct" rule that the praise list protects for the live version. (Recorded as open + question 8.) + +### council:security — revised + +**Applied (one line):** hardened the context 10-field allowlist with `SyncRemote` hard-excluded +for credential-leak mechanics and enforcement test 8b; 5xx detail scrubbing decided (static +per-code `detail`, real error server-log only — closed former open question 4, pinned by +test 5); `netutil.LimitListener(l, 64)` accepted-connection cap and the `/healthz` + `/context` +DB-semaphore bypass; the 60 s per-request deadline; `limit=0` → 400 under +`--allow-non-loopback` with binding rule 2 scoped to loopback (open question 2 narrowed); +wire-edge actor validation on claim (test 12); the Host-allowlist / DNS-rebinding middleware 400 +with the mandatory content-type check (test 19); all landed as additive slice-5/6/7/9 +deliverables, no new slices. + +**Deferred:** + +- Refusing `limit=0` whenever the resolved bind is non-loopback (address-keyed) rather than + whenever `--allow-non-loopback` is set (flag-keyed) — flag-keyed was chosen deliberately: it + is a stated posture ("asking for network exposure turns off unlimited reads"), strictly safer + (never weaker than address-keyed), and testable in CI on a loopback bind without binding + 0.0.0.0; noted in test 19's description. +- A dedicated load/exhaustion test for the LimitListener cap and per-request deadline — + resource-exhaustion tests of this kind are flaky in shared CI and the reviewer did not + require one; the mechanisms are specified as slice-6 deliverables and the deadline's + user-visible mapping (generic 500) is already covered by the status-code parity machinery. + +### council:sre — revised + +**Applied (one line):** the observability floor (structured stderr request/startup/shutdown +lines, saturation event, `request_id` extension) as a slice-6 acceptance criterion; the 10 s +bounded semaphore wait → 503 `busy` and the wedge scenario closed end to end; pool limits on +the serve-owned `*sql.DB` with the sole-tenant honesty on the "16 conns fit" claim; the +detached-close fix landing in `uow.RunTxResult` itself (rule 5); the limit=0 risk rewritten as +a shared-process OOM blast radius with the ship-`--max-rows`-in-v0 recommendation (open +question 2 sharpened); drain raised 10 s → 20 s with the idempotent same-actor re-claim +recovery and SIGHUP/supervisor note; the singleton claim made honest (default `127.0.0.1:0` +allows N parallel serves; fixed-port `--addr` blessed); claim retry-exhaustion `Retry-After` +1 → 5 with the convoy rationale; slice-11 runbook deliverables enumerated. + +**Deferred:** + +- An actual v0 operator flag for the read-handler deadline (the "flag-tunable" half of the + wedge-protection item) — the document's decided command surface is deliberately minimal ("no + other flags in v0", § 4); the deadline is specified as a generous 60 s constant explicitly + documented as promotable to an operator flag later without wire impact — the same posture the + doc already takes for the semaphore and connection-cap constants, satisfying the tunability + intent without growing the v0 flag surface. +- Streaming JSON encoder for list/ready responses — considered and rejected in favor of the + `--max-rows` recommendation, with the reason recorded in open question 2: streaming removes + only the marshal copy — the query layer still buffers every row — so it halves the footprint + without bounding it. +- Lowering the HTTP-side claim retry budget (the alternative branch of the Retry-After minor) — + binding rule 5 pins the single `uow.RunTxResult` implementation with its tested 15 s budget + shared with the proxied CLI; the `Retry-After: 5` branch of the reviewer's either/or was + applied instead, which addresses the convoy without forking or parameterizing the one retry + implementation. + +### council:consumer — revised + +**Applied (one line):** strict unknown-parameter rejection on every operation with the +`param`/`reason` 400 extension members and test 8c; the reconciler's TierBoth+SkipLabels full +active-universe scan truthfully scoped out of v0 (thesis exception, the "what this list surface +deliberately cannot serve" paragraph with the verified additive path, the skip_labels trap +correction, slice-11 negative statement); the version-skew client-contract section +(`capabilities`/`bd_version`/`api_version` gates, probe-by-400 parameter gating, +default-branch-on-unknown-codes, `schema_version` not-an-HTTP-gate weld) plus open question 7; +test 18 extended with the item-level claim parity oracle. + +**Deferred:** + +- Option (a) of the reconciler change: actually expose an ephemeral/wisp-inclusion parameter + and a labels-omission knob on `/v0/beads/issues` in v0 — took the reviewer's sanctioned + option (b) — truthful scoping — instead: the knobs would be the only MVP wire surface with no + CLI parity oracle behind it (`bd list` cannot emit ephemeral rows, so rule-8 full-item + oracles cannot run), and the labels-omission marker is a new page-envelope design decision; + the additive path is documented concretely in § 3 with the verified field citations so a + later slice can ship it without re-derivation. +- Ship `api_revision` (spec `info.version`) as a `ContextResponse` field now — the reviewer + asked only to "consider" it; shipping it would grow the security reviewer's enforced + ten-field context allowlist (test 8b asserts exactly ten keys) and permanent wire surface, so + it is recorded as concrete either/or open question 7 with the enforcement mechanism + (pr-policy: spec diff without `info.version` change fails) spelled out, while v0's + parameter-level gate is the strict unknown-parameter probe. +- Unweld `WireSchemaVersion` from `JSONSchemaVersion` (separate CLI-envelope version constant) — + the reviewer asked only that `schema_version` be documented as not-an-HTTP-gate and the weld + named; actually splitting the constant would revert § 2's shared-constant design (parity + row 9) that earlier passes preserved — the skew section instead states the weld, forbids + HTTP-side keying on the value, and notes the unweld is additive on the HTTP side if a + CLI-only schema change ever collides with the `/v0` bump freeze. + +### Post-review consistency check (values touched by more than one pass) + +Spot-checked after all four passes; each value below holds exactly one consistent position +across the document. No contradictory pairs required fixing. + +- **List limit defaults**: 50 (list) / 100 (ready), `0` = unlimited — identical in § 1 rule 9, + both parameter tables, § 3 pagination, the one-way-doors table, and test 4's assertion. +- **In-flight bound**: `serveMaxInflight = 16` (DB semaphore) inside `LimitListener(l, 64)` + (connection cap), pool `SetMaxOpenConns(16 + 4)` — § 4 concurrency, pool-limits, and slice 6 + agree; the "16 conns fit under max_connections" claim carries the SRE sole-tenant caveat in + its single location. +- **Timeouts**: 60 s per-request deadline (security-introduced, SRE-annotated, one value), + 10 s semaphore wait, 20 s shutdown drain (§ 4 explicitly reconciles "20 s, not 10" against + the 15 s claim retry budget), `Retry-After: 5` on retry exhaustion / `db_unavailable` vs + `Retry-After: 1` on semaphore-wait timeout — the § 3 busy row and § 4 narrative state the + same split. +- **Status-code table**: the § 3 sentinel table's rows (400 with `param`/`reason` extensions, + 400 `invalid_cursor`, 404, 409 ×2, 503 `busy`/`db_unavailable`, 500) match the + six-operations table's per-route documented codes plus the once-documented middleware 400; + the deliberate non-codes (413/429/504/403/501) are consistent with the queue-not-refuse + semaphore and the deadline→500 mapping after the SRE carve-out. +- **Context field list**: exactly ten fields (7 from `ContextInfo` + `api_version`, + `schema_version`, `capabilities`) in the § 3 enumeration, the version-skew section, test 8b + ("any JSON key outside the ten"), and open question 7's "ten fields". + +**Consolidation performed in this pass**: the maintainer's deferred-to-that-day `/v0` +sunset-vs-freeze either/or was present in § 3's evolution policy but absent from § 5's owner +list — added as open question 8 with a cross-reference from the policy text. The security and +SRE cap questions were already merged into the single open question 2 (loopback `--max-rows` +either/or plus the non-loopback refusal-vs-mandatory-cap variant); no duplicates remained. The +table of contents gained the appendix entry. diff --git a/backend/conformance/sweeper_contract.go b/backend/conformance/sweeper_contract.go new file mode 100644 index 000000000..090d80a91 --- /dev/null +++ b/backend/conformance/sweeper_contract.go @@ -0,0 +1,560 @@ +package conformance + +import ( + "context" + "errors" + "fmt" + "reflect" + "testing" + "time" + + "github.com/steveyegge/beads/internal/types" + publicops "github.com/steveyegge/beads/issueops" +) + +// This file holds the semantic contract every implementation of +// publicops.Sweeper must satisfy. Each case asserts what issueops/sweeper.go +// PROMISES, cited by line, rather than what any one backend happens to do; a +// backend that genuinely disagrees is parked at its own wiring site with +// skipKnownDivergence so the case still runs on the ones that agree. +// +// THERE ARE TWO BODIES BEHIND THE THREE WIRINGS. dolt and embeddeddolt share +// internal/storage/issueops.SweepInTx and differ only in how they reach a +// transaction; the unit-of-work provider reaches the same four questions +// through the domain use cases and is genuinely separate code. So the wirings +// are one vote plus an engine check, and a second independent vote — and the +// selection half of both votes runs through the SAME internal/workapi +// functions, which is why the cases below spend their assertions on what only +// a real backend can show: which rows are actually gone, what a dry run left +// alone, and whether the version-control plane recorded anything. +// +// EVERY CASE SCOPES ITSELF WITH AN ID PATTERN, and here that is not hygiene +// but a precondition. A sweep is asked of a whole TIER, so a case that swept +// without a pattern would delete the rows the next case seeded — and, on the +// ephemeral tier, would be a legal unfiltered request that the role does not +// refuse. Each Run therefore seeds under fixture.IssuePrefix + its own tag and +// passes that tag as SweepRequest.IDPattern. +// +// EVERY CASE PINS closed_at EXPLICITLY. The recheck the role performs is +// against a cutoff, and a seeded row whose closed_at defaulted to "now" would +// make "closed before now" a race with the test's own clock. + +// SweeperFixture supplies adapter-specific storage access for the +// bulk-clearance assertions. Every field is named and typed exactly like the +// per-backend roleFixtureKit hook it is filled from, so a wiring is kit plus +// accessor plus prefix with no adapter in between. +type SweeperFixture struct { + // IssuePrefix namespaces the ids each assertion seeds, so several of them + // can share one database. + IssuePrefix string + // Sweeper is the surface under test. + Sweeper publicops.Sweeper + // CreateIssue seeds a durable issue in the issues plane. + CreateIssue func(context.Context, *types.Issue, string) error + // CreateWisp seeds an ephemeral issue in the wisps plane. It is a separate + // field rather than an Ephemeral flag on CreateIssue because the three + // adapters reach the two planes through different verbs. + CreateWisp func(context.Context, *types.Issue, string) error + // QueryScalar runs a single-row query and scans it. It is how these cases + // observe the ONE thing a sweep result cannot be trusted to report about + // itself: whether the rows are really gone. + QueryScalar func(context.Context, string, []any, ...any) error + // CountHistory reports how many history entries the fixture's branch has. + // A nil hook means "this backend cannot observe history", and the case + // that needs it SKIPS with that reason rather than passing quietly. + CountHistory func(context.Context) (int, error) +} + +// sweeperClosedAt is the stamp every seeded candidate carries, and +// sweeperCutoff is an instant strictly after it. Fixed values rather than +// offsets from time.Now so the cutoff cases assert an interval rather than +// race the clock. +var ( + sweeperClosedAt = time.Date(2026, 3, 1, 12, 0, 0, 0, time.UTC) + sweeperCutoff = time.Date(2026, 4, 1, 12, 0, 0, 0, time.UTC) +) + +// RunSweeperRefusesAnUnfilteredDurableSweep pins the safety invariant the role +// took off the CLI (issueops/sweeper.go:59-73): a durable sweep with neither a +// cutoff nor a pattern is ErrValidation, and it is the WHOLE POINT of the +// guard living below the front doors that this is asserted at the role rather +// than at a handler. +// +// It asserts the refusal AND its effect. A guard that returned an error after +// deleting would satisfy an errors.Is assertion perfectly, and on this +// operation that is the failure worth spending a seeded row to rule out. +func RunSweeperRefusesAnUnfilteredDurableSweep(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + id := sweeperSeedClosedIssue(t, ctx, fixture, "gate", false) + + result, err := fixture.Sweeper.Sweep(ctx, publicops.SweepRequest{Tier: publicops.SweepDurable}) + if !errors.Is(err, publicops.ErrValidation) { + t.Fatalf("Sweep(durable, no cutoff, no pattern) error = %v, want ErrValidation: "+ + "the require-a-filter guard is a safety invariant of the ROLE, so a second front door inherits it", err) + } + if result.Swept != 0 { + t.Errorf("refused sweep reported Swept = %d, want 0", result.Swept) + } + sweeperAssertIssueRows(t, ctx, fixture, 1, id) + + // The ephemeral tier carries no such gate: an unfiltered sweep of closed + // wisps is the ordinary use of `bd purge`. Asked with a pattern that + // matches nothing so the case does not clear another case's seeds. + if _, err := fixture.Sweeper.Sweep(ctx, publicops.SweepRequest{ + Tier: publicops.SweepEphemeral, + IDPattern: fixture.IssuePrefix + "-gate-nothing-*", + }); err != nil { + t.Fatalf("Sweep(ephemeral) error = %v, want nil: the gate is the durable tier's, not every tier's", err) + } +} + +// RunSweeperRefusesAMalformedRequest pins the other two ErrValidation clauses +// (issueops/sweeper.go): an unset or unrecognized Tier, and an IDPattern that +// is not a well-formed glob. +// +// The pattern case is a DEFECT FIX with teeth. Both front doors used to +// discard filepath.Match's error, so `--pattern '['` reported "nothing to +// prune" — indistinguishable from a correct pattern over an empty set, on a +// command whose entire job is to delete what it matched. +func RunSweeperRefusesAMalformedRequest(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + for _, test := range []struct { + name string + request publicops.SweepRequest + }{ + {"unset tier", publicops.SweepRequest{IDPattern: "*"}}, + {"unrecognized tier", publicops.SweepRequest{Tier: publicops.SweepTier("wisps"), IDPattern: "*"}}, + {"malformed pattern", publicops.SweepRequest{Tier: publicops.SweepDurable, IDPattern: "[bad"}}, + } { + t.Run(test.name, func(t *testing.T) { + if _, err := fixture.Sweeper.Sweep(ctx, test.request); !errors.Is(err, publicops.ErrValidation) { + t.Fatalf("Sweep(%s) error = %v, want ErrValidation", test.name, err) + } + }) + } +} + +// RunSweeperClearsOneTierAndLeavesTheOther pins the disjointness that makes +// `bd purge` and `bd prune` ONE capability with a tier parameter +// (issueops/sweeper.go, SweepTier): a sweep of one tier can never touch a row +// of the other. +// +// Both planes are seeded with ids the SAME pattern admits, so a body that +// forgot the tier — or that let the pattern do the work the filter should do — +// deletes the wrong rows rather than merely reporting a wrong number. +func RunSweeperClearsOneTierAndLeavesTheOther(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + issueID := sweeperSeedClosedIssue(t, ctx, fixture, "tier", false) + wispID := sweeperSeedClosedIssue(t, ctx, fixture, "tier", true) + + result := sweeperSweep(t, ctx, fixture, publicops.SweepRequest{ + Tier: publicops.SweepDurable, + IDPattern: sweeperPattern(fixture, "tier"), + }) + if result.Swept != 1 { + t.Fatalf("durable sweep reported Swept = %d, want 1", result.Swept) + } + sweeperAssertIssueRows(t, ctx, fixture, 0, issueID) + sweeperAssertWispRows(t, ctx, fixture, 1, wispID) + + result = sweeperSweep(t, ctx, fixture, publicops.SweepRequest{ + Tier: publicops.SweepEphemeral, + IDPattern: sweeperPattern(fixture, "tier"), + }) + if result.Swept != 1 { + t.Fatalf("ephemeral sweep reported Swept = %d, want 1", result.Swept) + } + sweeperAssertWispRows(t, ctx, fixture, 0, wispID) +} + +// RunSweeperProtectsPinnedRows pins the protection no request field overrides +// (issueops.SweepSkips.Pinned): a pinned closed row is held back and counted, +// and the sweep still clears everything beside it. +func RunSweeperProtectsPinnedRows(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + plain := sweeperSeedClosedIssue(t, ctx, fixture, "pin", false) + pinned := sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "pin", "keep", false), func(issue *types.Issue) { + issue.Pinned = true + }) + + result := sweeperSweep(t, ctx, fixture, publicops.SweepRequest{ + Tier: publicops.SweepDurable, + IDPattern: sweeperPattern(fixture, "pin"), + }) + + if result.Swept != 1 { + t.Errorf("Swept = %d, want 1 — the pinned row must not be one of them", result.Swept) + } + if result.Skipped.Pinned != 1 { + t.Errorf("Skipped.Pinned = %d, want 1", result.Skipped.Pinned) + } + sweeperAssertIssueRows(t, ctx, fixture, 0, plain) + sweeperAssertIssueRows(t, ctx, fixture, 1, pinned) +} + +// RunSweeperHonorsTheCutoffAndThePattern pins the two narrowing fields. +// +// The CUTOFF is HALF-OPEN, and that is asserted rather than assumed: a row +// closed exactly at the cutoff is KEPT (issueops.SweepRequest.ClosedBefore, +// "strictly before"). An off-by-one there deletes a bead closed the instant a +// scheduled sweep names, which is the one row a caller reasoning about the +// boundary is thinking of. +// +// The PATTERN is matched in Go rather than translated to SQL, so the case +// includes a character class — the construct a LIKE translation would get +// wrong, on the operation where getting it wrong deletes rows. +func RunSweeperHonorsTheCutoffAndThePattern(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + old := sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "when", "old", false), nil) + atCutoff := sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "when", "edge", false), func(issue *types.Issue) { + at := sweeperCutoff + issue.ClosedAt = &at + }) + + cutoff := sweeperCutoff + result := sweeperSweep(t, ctx, fixture, publicops.SweepRequest{ + Tier: publicops.SweepDurable, + ClosedBefore: &cutoff, + IDPattern: sweeperPattern(fixture, "when"), + }) + if result.Swept != 1 { + t.Errorf("Swept = %d, want 1 — ClosedBefore is strictly before, so the row closed AT the cutoff stays", result.Swept) + } + sweeperAssertIssueRows(t, ctx, fixture, 0, old) + sweeperAssertIssueRows(t, ctx, fixture, 1, atCutoff) + + // A character class, and a row the class excludes. + inClass := sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "glob", "a1", false), nil) + outOfClass := sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "glob", "z9", false), nil) + result = sweeperSweep(t, ctx, fixture, publicops.SweepRequest{ + Tier: publicops.SweepDurable, + IDPattern: fixture.IssuePrefix + "-glob-[a-c]*", + }) + if result.Swept != 1 { + t.Errorf("Swept = %d, want 1 for the character-class pattern", result.Swept) + } + sweeperAssertIssueRows(t, ctx, fixture, 0, inClass) + sweeperAssertIssueRows(t, ctx, fixture, 1, outOfClass) +} + +// RunSweeperDryRunChangesNothing pins the promise a preview is worth reading +// for (issueops.Sweeper.Sweep, "A DRY RUN CHANGES NOTHING"): the preview +// reports the same counts the real sweep goes on to report, and leaves every +// row where it was. +// +// The two are compared against EACH OTHER rather than against a literal, +// because that is the property a caller relies on — a preview whose number +// differs from the run it precedes is worse than no preview. +func RunSweeperDryRunChangesNothing(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + ids := []string{ + sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "dry", "1", false), nil), + sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "dry", "2", false), nil), + } + request := publicops.SweepRequest{ + Tier: publicops.SweepDurable, + IDPattern: sweeperPattern(fixture, "dry"), + } + + preview := sweeperSweep(t, ctx, fixture, sweeperWithDryRun(request, true)) + if !preview.DryRun { + t.Error("preview.DryRun = false; the result must echo the request it answered") + } + if preview.Swept != 2 { + t.Fatalf("preview Swept = %d, want 2", preview.Swept) + } + sweeperAssertIssueRows(t, ctx, fixture, 2, ids...) + + // A second preview: a dry run that had deleted anything would report a + // smaller number here, which is the failure this repetition rules out. + if again := sweeperSweep(t, ctx, fixture, sweeperWithDryRun(request, true)); again.Swept != preview.Swept { + t.Fatalf("second preview Swept = %d, first = %d: the first preview changed the store", again.Swept, preview.Swept) + } + + real := sweeperSweep(t, ctx, fixture, request) + if real.Swept != preview.Swept { + t.Errorf("real Swept = %d, preview said %d: a preview that disagrees with its run is worse than none", + real.Swept, preview.Swept) + } + if real.DryRun { + t.Error("real.DryRun = true") + } + sweeperAssertIssueRows(t, ctx, fixture, 0, ids...) +} + +// RunSweeperProtectsCitedRows pins ProtectReferenced +// (issueops.SweepRequest.ProtectReferenced): a candidate whose id appears in a +// not-done bead's description is held back, counted, and named in the bounded +// sample — and the SAME request without the protection deletes it. +// +// The two halves matter together. Asserting only the protection would pass for +// an implementation that skipped every candidate; asserting only the +// unprotected sweep would pass for one that never scanned at all. +// +// The citation is at a WORD BOUNDARY and there is a near-miss beside it, so an +// implementation that fell back to a bare substring search protects a row +// nothing cites. +func RunSweeperProtectsCitedRows(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + cited := sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "ref", "cited", false), nil) + free := sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "ref", "free", false), nil) + + citing := sweeperIssue(fixture, "ref", "live", false) + citing.Status = types.StatusOpen + citing.ClosedAt = nil + // The near-miss: `x` must NOT protect anything, and the parenthesised + // occurrence must. + citing.Description = fmt.Sprintf("superseded by (%s). unrelated: %sx", cited, free) + // The citing bead is OPEN, so the pattern below must not admit it as a + // candidate — it would be skipped by the closed recheck either way, but a + // case that relied on that would be asserting the wrong protection. + sweeperSeedOpen(t, ctx, fixture, citing) + + request := publicops.SweepRequest{ + Tier: publicops.SweepDurable, + IDPattern: sweeperPattern(fixture, "ref"), + ProtectReferenced: true, + } + result := sweeperSweep(t, ctx, fixture, request) + + if result.Skipped.Referenced != 1 { + t.Fatalf("Skipped.Referenced = %d, want 1 (sample %v)", result.Skipped.Referenced, result.ReferencedIDs) + } + if len(result.ReferencedIDs) != 1 || result.ReferencedIDs[0] != cited { + t.Errorf("ReferencedIDs = %v, want [%s]", result.ReferencedIDs, cited) + } + if result.Swept != 1 { + t.Errorf("Swept = %d, want 1 — the uncited candidate goes", result.Swept) + } + sweeperAssertIssueRows(t, ctx, fixture, 1, cited) + sweeperAssertIssueRows(t, ctx, fixture, 0, free) + + // Without the protection the same candidate goes, and nothing is reported + // as protected — a caller reading Referenced=0 without having asked has + // learned nothing, which the field doc says out loud. + request.ProtectReferenced = false + result = sweeperSweep(t, ctx, fixture, request) + if result.Skipped.Referenced != 0 { + t.Errorf("Skipped.Referenced = %d with ProtectReferenced false, want 0", result.Skipped.Referenced) + } + if result.Swept != 1 { + t.Errorf("Swept = %d, want 1 — the cited candidate is no longer protected", result.Swept) + } + sweeperAssertIssueRows(t, ctx, fixture, 0, cited) +} + +// RunSweeperEmptyMatchIsZeroAndNil pins the not-found story +// (issueops.Sweeper.Sweep, "A REQUEST THAT MATCHES NOTHING"): a zero result and +// a nil error, which is the answer a scheduled sweep gets every time it runs +// after the first. +func RunSweeperEmptyMatchIsZeroAndNil(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + for _, tier := range []publicops.SweepTier{publicops.SweepDurable, publicops.SweepEphemeral} { + t.Run(string(tier), func(t *testing.T) { + result, err := fixture.Sweeper.Sweep(ctx, publicops.SweepRequest{ + Tier: tier, + IDPattern: fixture.IssuePrefix + "-none-ever-*", + }) + if err != nil { + t.Fatalf("Sweep(nothing matches) error = %v, want nil", err) + } + if result.Swept != 0 { + t.Errorf("Swept = %d, want 0", result.Swept) + } + if len(result.ReferencedIDs) != 0 { + t.Errorf("ReferencedIDs = %v, want empty", result.ReferencedIDs) + } + }) + } +} + +// RunSweeperRecordsAtMostOneHistoryEntry pins the versioning clause +// (issueops.Sweeper.Sweep, "A DRY RUN CHANGES NOTHING, including history"): a +// dry run and a no-op record NONE, and a sweep that deleted rows records at +// most ONE — the deletion is one act, not one per row. +// +// "AT MOST" rather than "exactly" is the honest promise across these backends: +// the server-backed store records a Dolt commit, the embedded one commits +// outside the SQL transaction and records none here, and an ephemeral sweep +// touches only tables the version-control plane ignores. A case demanding +// exactly one would be asserting a property of one wiring. +func RunSweeperRecordsAtMostOneHistoryEntry(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + if fixture.CountHistory == nil { + t.Skip("this backend cannot observe history, so the entry-per-call clause is unobservable here") + } + ids := []string{ + sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "hist", "1", false), nil), + sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, "hist", "2", false), nil), + } + request := publicops.SweepRequest{ + Tier: publicops.SweepDurable, + IDPattern: sweeperPattern(fixture, "hist"), + } + + before := sweeperHistory(t, ctx, fixture) + sweeperSweep(t, ctx, fixture, sweeperWithDryRun(request, true)) + if after := sweeperHistory(t, ctx, fixture); after != before { + t.Errorf("history went %d -> %d across a DRY RUN, want no entry at all", before, after) + } + + before = sweeperHistory(t, ctx, fixture) + sweeperSweep(t, ctx, fixture, publicops.SweepRequest{ + Tier: publicops.SweepDurable, + IDPattern: fixture.IssuePrefix + "-hist-nothing-*", + }) + if after := sweeperHistory(t, ctx, fixture); after != before { + t.Errorf("history went %d -> %d across a sweep that deleted nothing, want no entry", before, after) + } + + before = sweeperHistory(t, ctx, fixture) + sweeperSweep(t, ctx, fixture, request) + after := sweeperHistory(t, ctx, fixture) + if after < before || after > before+1 { + t.Errorf("history went %d -> %d across one sweep of %d rows, want at most one more entry", + before, after, len(ids)) + } +} + +// RunSweeperDoesNotMutateTheCallerRequest pins the no-mutation promise +// (issueops.SweepRequest: "Implementations never mutate caller-owned request +// values"). ClosedBefore is the pointer that would otherwise be written +// through, and a normalization step is exactly the kind of edit that does it. +func RunSweeperDoesNotMutateTheCallerRequest(t *testing.T, ctx context.Context, fixture SweeperFixture) { + t.Helper() + cutoff := sweeperCutoff + request := publicops.SweepRequest{ + Tier: publicops.SweepDurable, + Actor: "sweeper-contract", + ClosedBefore: &cutoff, + IDPattern: sweeperPattern(fixture, "immutable"), + ProtectReferenced: true, + DryRun: true, + } + snapshot := request + snapshotCutoff := cutoff + + if _, err := fixture.Sweeper.Sweep(ctx, request); err != nil { + t.Fatalf("Sweep() error = %v", err) + } + + if !reflect.DeepEqual(request, snapshot) { + t.Errorf("request changed across the call: got %+v, want %+v", request, snapshot) + } + if !cutoff.Equal(snapshotCutoff) { + t.Errorf("the caller's cutoff changed: got %v, want %v", cutoff, snapshotCutoff) + } +} + +// --- fixture helpers ------------------------------------------------------- + +// sweeperPattern is the glob that scopes one case's ids, and the reason every +// case has one: a sweep is asked of a whole tier. +func sweeperPattern(fixture SweeperFixture, tag string) string { + return fixture.IssuePrefix + "-" + tag + "-*" +} + +func sweeperIssue(fixture SweeperFixture, tag, name string, ephemeral bool) *types.Issue { + closedAt := sweeperClosedAt + return &types.Issue{ + ID: fmt.Sprintf("%s-%s-%s", fixture.IssuePrefix, tag, name), + Title: tag + " " + name, + Status: types.StatusClosed, + Priority: 2, + IssueType: types.TypeTask, + Ephemeral: ephemeral, + ClosedAt: &closedAt, + } +} + +// sweeperSeed writes one issue through the plane its Ephemeral flag names and +// returns its id. mutate runs on the issue before it is written, which is how +// a case seeds a pinned row or moves a closed_at. +func sweeperSeed(t *testing.T, ctx context.Context, fixture SweeperFixture, issue *types.Issue, mutate func(*types.Issue)) string { + t.Helper() + if mutate != nil { + mutate(issue) + } + create := fixture.CreateIssue + if issue.Ephemeral { + create = fixture.CreateWisp + } + if err := create(ctx, issue, "sweeper-seed"); err != nil { + t.Fatalf("seeding %s: %v", issue.ID, err) + } + return issue.ID +} + +func sweeperSeedClosedIssue(t *testing.T, ctx context.Context, fixture SweeperFixture, tag string, ephemeral bool) string { + t.Helper() + name := "durable" + if ephemeral { + name = "wisp" + } + return sweeperSeed(t, ctx, fixture, sweeperIssue(fixture, tag, name, ephemeral), nil) +} + +func sweeperSeedOpen(t *testing.T, ctx context.Context, fixture SweeperFixture, issue *types.Issue) string { + t.Helper() + issue.Status = types.StatusOpen + issue.ClosedAt = nil + return sweeperSeed(t, ctx, fixture, issue, nil) +} + +func sweeperWithDryRun(request publicops.SweepRequest, dryRun bool) publicops.SweepRequest { + request.DryRun = dryRun + return request +} + +func sweeperSweep(t *testing.T, ctx context.Context, fixture SweeperFixture, request publicops.SweepRequest) publicops.SweepResult { + t.Helper() + result, err := fixture.Sweeper.Sweep(ctx, request) + if err != nil { + t.Fatalf("Sweep(%+v) error = %v", request, err) + } + return result +} + +// sweeperAssertIssueRows counts the named ids in the ISSUES plane. It is the +// only assertion in this file that does not trust the result the sweep +// reported about itself, which on a destructive operation is the assertion +// that matters. +func sweeperAssertIssueRows(t *testing.T, ctx context.Context, fixture SweeperFixture, want int, ids ...string) { + t.Helper() + sweeperAssertRows(t, ctx, fixture, "issues", want, ids...) +} + +func sweeperAssertWispRows(t *testing.T, ctx context.Context, fixture SweeperFixture, want int, ids ...string) { + t.Helper() + sweeperAssertRows(t, ctx, fixture, "wisps", want, ids...) +} + +func sweeperAssertRows(t *testing.T, ctx context.Context, fixture SweeperFixture, table string, want int, ids ...string) { + t.Helper() + var got int + args := make([]any, len(ids)) + placeholders := "" + for i, id := range ids { + args[i] = id + if i > 0 { + placeholders += ", " + } + placeholders += "?" + } + query := fmt.Sprintf("SELECT COUNT(*) FROM %s WHERE id IN (%s)", table, placeholders) + if err := fixture.QueryScalar(ctx, query, args, &got); err != nil { + t.Fatalf("counting %s rows for %v: %v", table, ids, err) + } + if got != want { + t.Errorf("%s rows for %v = %d, want %d", table, ids, got, want) + } +} + +func sweeperHistory(t *testing.T, ctx context.Context, fixture SweeperFixture) int { + t.Helper() + entries, err := fixture.CountHistory(ctx) + if err != nil { + t.Fatalf("CountHistory: %v", err) + } + return entries +} diff --git a/cmd/bd/closed_delete_candidates.go b/cmd/bd/closed_delete_candidates.go index b9a6b97aa..d32316363 100644 --- a/cmd/bd/closed_delete_candidates.go +++ b/cmd/bd/closed_delete_candidates.go @@ -4,60 +4,38 @@ import ( "time" "github.com/steveyegge/beads/internal/types" + "github.com/steveyegge/beads/internal/workapi" + "github.com/steveyegge/beads/issueops" ) -type closedDeletionCandidateStats struct { - PinnedSkipped int - NilSkipped int - NonClosedSkipped int - MissingClosedAtSkipped int - RecentClosedAtSkipped int -} +// The closed-candidate recheck used to live here in full, shared by `bd gc`, +// `bd purge` and `bd prune`. purge and prune are behind issueops.Sweeper now +// and the role applies it below both front doors; `bd gc` is not, so what is +// left here is a thin call into the SAME pure function +// (workapi.FilterSweepCandidates) plus gc's own warning line. +// +// One definition, two callers: gc and the role cannot come to disagree about +// what "a closed bead safe to delete" means, which is exactly the drift this +// file existed to prevent between purge and prune in the first place. -func (s closedDeletionCandidateStats) SafetySkipped() int { - return s.NilSkipped + s.NonClosedSkipped + s.MissingClosedAtSkipped + s.RecentClosedAtSkipped -} +type closedDeletionCandidateStats = issueops.SweepSkips +// filterClosedDeletionCandidates keeps the closed, unpinned, old-enough +// candidates and reports what it held back. `bd gc` matches no glob, so it +// passes the empty pattern, which admits everything. func filterClosedDeletionCandidates(issues []*types.Issue, cutoff *time.Time) ([]*types.Issue, closedDeletionCandidateStats) { - filtered := make([]*types.Issue, 0, len(issues)) - var stats closedDeletionCandidateStats - - for _, issue := range issues { - if issue == nil { - stats.NilSkipped++ - continue - } - if issue.Pinned { - stats.PinnedSkipped++ - continue - } - if issue.Status != types.StatusClosed { - stats.NonClosedSkipped++ - continue - } - if issue.ClosedAt == nil { - stats.MissingClosedAtSkipped++ - continue - } - if cutoff != nil && !issue.ClosedAt.Before(*cutoff) { - stats.RecentClosedAtSkipped++ - continue - } - filtered = append(filtered, issue) - } - - return filtered, stats + return workapi.FilterSweepCandidates(issues, "", cutoff) } func warnClosedDeletionSafetySkips(stats closedDeletionCandidateStats) { - if stats.SafetySkipped() == 0 { + if workapi.SweepDefenseSkips(stats) == 0 { return } WarnError("skipped %d deletion candidate(s) after closed_at safety recheck (nil=%d, non_closed=%d, missing_closed_at=%d, too_recent=%d)", - stats.SafetySkipped(), - stats.NilSkipped, - stats.NonClosedSkipped, - stats.MissingClosedAtSkipped, - stats.RecentClosedAtSkipped, + workapi.SweepDefenseSkips(stats), + stats.Unreadable, + stats.NotClosed, + stats.UnknownClosedAt, + stats.ClosedAtOrAfterCutoff, ) } diff --git a/cmd/bd/closed_delete_candidates_test.go b/cmd/bd/closed_delete_candidates_test.go deleted file mode 100644 index f0982addf..000000000 --- a/cmd/bd/closed_delete_candidates_test.go +++ /dev/null @@ -1,72 +0,0 @@ -package main - -import ( - "testing" - "time" - - "github.com/steveyegge/beads/internal/types" -) - -func TestFilterClosedDeletionCandidatesRechecksClosedAtCutoff(t *testing.T) { - cutoff := time.Date(2026, 3, 27, 20, 1, 44, 0, time.UTC) - oldClosedAt := cutoff.Add(-time.Second) - recentClosedAt := cutoff.Add(time.Second) - - candidates := []*types.Issue{ - {ID: "old-closed", Status: types.StatusClosed, ClosedAt: &oldClosedAt}, - {ID: "recent-closed", Status: types.StatusClosed, ClosedAt: &recentClosedAt}, - {ID: "missing-closed-at", Status: types.StatusClosed}, - {ID: "open-with-old-closed-at", Status: types.StatusOpen, ClosedAt: &oldClosedAt}, - {ID: "pinned-old", Status: types.StatusClosed, ClosedAt: &oldClosedAt, Pinned: true}, - nil, - } - - filtered, stats := filterClosedDeletionCandidates(candidates, &cutoff) - - if len(filtered) != 1 || filtered[0].ID != "old-closed" { - t.Fatalf("filtered IDs = %v, want only old-closed", closedDeletionCandidateIDs(filtered)) - } - if stats.PinnedSkipped != 1 { - t.Fatalf("PinnedSkipped = %d, want 1", stats.PinnedSkipped) - } - if stats.RecentClosedAtSkipped != 1 { - t.Fatalf("RecentClosedAtSkipped = %d, want 1", stats.RecentClosedAtSkipped) - } - if stats.MissingClosedAtSkipped != 1 { - t.Fatalf("MissingClosedAtSkipped = %d, want 1", stats.MissingClosedAtSkipped) - } - if stats.NonClosedSkipped != 1 { - t.Fatalf("NonClosedSkipped = %d, want 1", stats.NonClosedSkipped) - } - if stats.NilSkipped != 1 { - t.Fatalf("NilSkipped = %d, want 1", stats.NilSkipped) - } -} - -func TestFilterClosedDeletionCandidatesWithoutCutoffStillRequiresClosedAt(t *testing.T) { - closedAt := time.Date(2026, 4, 26, 18, 30, 18, 0, time.UTC) - candidates := []*types.Issue{ - {ID: "closed", Status: types.StatusClosed, ClosedAt: &closedAt}, - {ID: "closed-missing-time", Status: types.StatusClosed}, - } - - filtered, stats := filterClosedDeletionCandidates(candidates, nil) - - if len(filtered) != 1 || filtered[0].ID != "closed" { - t.Fatalf("filtered IDs = %v, want only closed", closedDeletionCandidateIDs(filtered)) - } - if stats.MissingClosedAtSkipped != 1 { - t.Fatalf("MissingClosedAtSkipped = %d, want 1", stats.MissingClosedAtSkipped) - } - if stats.RecentClosedAtSkipped != 0 { - t.Fatalf("RecentClosedAtSkipped = %d, want 0 without cutoff", stats.RecentClosedAtSkipped) - } -} - -func closedDeletionCandidateIDs(issues []*types.Issue) []string { - ids := make([]string, len(issues)) - for i, issue := range issues { - ids[i] = issue.ID - } - return ids -} diff --git a/cmd/bd/prune.go b/cmd/bd/prune.go index 1694fb926..510a62101 100644 --- a/cmd/bd/prune.go +++ b/cmd/bd/prune.go @@ -4,6 +4,7 @@ import ( "github.com/spf13/cobra" "github.com/steveyegge/beads/internal/metrics" + "github.com/steveyegge/beads/issueops" ) var pruneCmd = &cobra.Command{ @@ -53,14 +54,17 @@ EXAMPLES: ignoreRefs, _ := cmd.Flags().GetBool("ignore-references") return runPurgeOrPrune(cmd, purgeScope{ - cmdName: "prune", - pastTense: "pruned", - countKey: "pruned_count", - dryRunCountKey: "prune_count", - subjectNoun: "closed bead", - ephemeralOnly: false, - requireFilter: true, - ignoreReferences: ignoreRefs, + cmdName: "prune", + pastTense: "pruned", + countKey: "pruned_count", + dryRunCountKey: "prune_count", + subjectNoun: "closed bead", + tier: issueops.SweepDurable, + // The flag is the NEGATIVE of the request field: --ignore-references + // asks the role NOT to protect. The reporting is unconditional, so + // `--json` keeps publishing the keys with the flag set. + protectReferenced: !ignoreRefs, + reportsReferences: true, }) }, } diff --git a/cmd/bd/prune_bench_test.go b/cmd/bd/prune_bench_test.go deleted file mode 100644 index 1380f7663..000000000 --- a/cmd/bd/prune_bench_test.go +++ /dev/null @@ -1,140 +0,0 @@ -//go:build cgo - -package main - -import ( - "context" - "fmt" - "strings" - "testing" - "time" - - "github.com/steveyegge/beads/internal/storage" - "github.com/steveyegge/beads/internal/types" -) - -// mockRefStore satisfies storage.DoltStorage using only SearchIssues and -// GetIssueComments; all other methods panic if called. -type mockRefStore struct { - storage.DoltStorage - openBeads []*types.Issue -} - -func (m *mockRefStore) SearchIssues(_ context.Context, _ string, _ types.IssueFilter) ([]*types.Issue, error) { - return m.openBeads, nil -} - -func (m *mockRefStore) GetIssueComments(_ context.Context, _ string) ([]*types.Comment, error) { - return nil, nil -} - -// GetCustomStatusesDetailed lets buildReferencedSet enumerate active custom -// statuses; this fixture configures none. -func (m *mockRefStore) GetCustomStatusesDetailed(_ context.Context) ([]types.CustomStatus, error) { - return nil, nil -} - -func TestCandidateIDMatcherWordBoundaries(t *testing.T) { - candidates := map[string]bool{ - "be-ref-001": true, - "be-ref-001.1": true, - "be-ref-002": true, - } - matcher := newCandidateIDMatcher(candidates) - - found := make(map[string]bool) - matcher.findAll("see (be-ref-001), be-ref-001.1 and xbe-ref-002 but not be-ref-002x", found) - - if !found["be-ref-001"] { - t.Fatal("expected be-ref-001 to match at punctuation boundaries") - } - if !found["be-ref-001.1"] { - t.Fatal("expected be-ref-001.1 to match at punctuation boundaries") - } - if found["be-ref-002"] { - t.Fatal("did not expect be-ref-002 to match inside word boundaries") - } -} - -// TestPruneLargeFixture asserts that buildReferencedSet completes in <5s on a -// 10K-open-bead × ~5KB-body fixture (NFR-02 from be-5sn). -func TestPruneLargeFixture(t *testing.T) { - if testing.Short() { - t.Skip("skipping large fixture bench") - } - - const ( - openBeadCount = 10_000 - candidateCount = 100 - seededRefCount = 20 - bodyPadding = 5_000 // bytes of filler per bead - maxDurationSeconds = 5 - ) - - // Build 100 closed candidate IDs (be-ref-000 … be-ref-099). - candidates := make(map[string]bool, candidateCount) - for i := 0; i < candidateCount; i++ { - candidates[fmt.Sprintf("be-ref-%03d", i)] = true - } - - // Build the seeded ID set (first 20 candidates). - seeded := make([]string, seededRefCount) - i := 0 - for id := range candidates { - if i >= seededRefCount { - break - } - seeded[i] = id - i++ - } - - // Padding text that does NOT contain any candidate ID substring. - pad := strings.Repeat("x", bodyPadding) - - // Create 10K open beads. Every 500th bead gets one seeded ID planted in - // its description so that exactly seededRefCount beads carry references. - openBeads := make([]*types.Issue, openBeadCount) - seededIdx := 0 - for j := 0; j < openBeadCount; j++ { - desc := pad - if seededIdx < seededRefCount && j%500 == 0 { - desc = fmt.Sprintf("%s %s %s", pad[:100], seeded[seededIdx], pad[100:]) - seededIdx++ - } - openBeads[j] = &types.Issue{ - ID: fmt.Sprintf("be-open-%05d", j), - Status: types.StatusOpen, - Description: desc, - } - } - - st := &mockRefStore{openBeads: openBeads} - ctx := context.Background() - - start := time.Now() - refSet, err := buildReferencedSet(ctx, st, candidates) - elapsed := time.Since(start) - - if err != nil { - t.Fatalf("buildReferencedSet error: %v", err) - } - - // Performance assertion (NFR-02). The race detector adds multi-x overhead - // that invalidates a wall-clock budget (buildReferencedSet measures ~4x - // slower under -race), so only enforce the bound in non-race builds. The - // correctness assertions below still run under -race, which is where the - // large-fixture pass earns its keep as a data-race check. - if !raceEnabled && elapsed > maxDurationSeconds*time.Second { - t.Errorf("buildReferencedSet took %v; must complete in <%ds on 10K-bead fixture", elapsed, maxDurationSeconds) - } - - // Correctness assertion: exactly the seeded IDs should be in the reference set. - if len(refSet) != seededIdx { - t.Errorf("expected %d referenced IDs, got %d", seededIdx, len(refSet)) - } - for _, id := range seeded[:seededIdx] { - if !refSet[id] { - t.Errorf("expected %s to be in refSet but it was not", id) - } - } -} diff --git a/cmd/bd/prune_proxied_integration_test.go b/cmd/bd/prune_proxied_integration_test.go index 157eae463..f4ebdba6a 100644 --- a/cmd/bd/prune_proxied_integration_test.go +++ b/cmd/bd/prune_proxied_integration_test.go @@ -177,9 +177,15 @@ func TestProxiedServerPrune(t *testing.T) { if got := readDoltLogCountSince(t, db, before); got < 1 { t.Errorf("prune should create a dolt commit, got %d new commits", got) } + // The label is the ROLE's, not this command's: one sweep, one entry, + // naming the operation and the tier it cleared. It used to read + // "bd: prune N bead(s)" here and "bd: delete N issue(s)" on the direct + // route, which named neither the command nor the tier — the two front + // doors now write the same sentence, and so does an HTTP sweep, where + // there is no `bd prune` to name. msg := readDoltLogTopMessage(t, db) - if !strings.HasPrefix(msg, "bd: prune ") { - t.Errorf("dolt commit message should begin with 'bd: prune ', got: %q", msg) + if !strings.HasPrefix(msg, "bd: sweep ") || !strings.Contains(msg, "durable") { + t.Errorf("dolt commit message should name the sweep and its tier, got: %q", msg) } }) } diff --git a/cmd/bd/purge.go b/cmd/bd/purge.go index 47a790550..de9191798 100644 --- a/cmd/bd/purge.go +++ b/cmd/bd/purge.go @@ -1,24 +1,26 @@ package main import ( - "context" "fmt" - "path/filepath" - "sort" "strconv" "strings" "time" "github.com/spf13/cobra" "github.com/steveyegge/beads/internal/metrics" - "github.com/steveyegge/beads/internal/storage" - "github.com/steveyegge/beads/internal/types" "github.com/steveyegge/beads/internal/ui" + "github.com/steveyegge/beads/issueops" ) -// purgeScope parameterizes the shared purge/prune implementation so both -// commands can share filter plumbing, preview/dry-run/force semantics, and -// messaging without copying 200 lines of boilerplate. +// purgeScope is what `bd purge` and `bd prune` still differ by once the +// operation itself is behind issueops.Sweeper: a tier, a reference policy, and +// the words each command prints. +// +// It used to carry the SEMANTICS too — ephemeralOnly, requireFilter, +// ignoreReferences — and each route read them and re-derived the same +// selection, the same pinned protection and the same safety gate from them. +// All of that is request fields now, so what is left here is presentation plus +// the two request values a flag chooses. type purgeScope struct { // cmdName is the user-visible command name (e.g. "purge", "prune"). // Used in messages and the suggested `--force` hint. @@ -33,18 +35,18 @@ type purgeScope struct { // (e.g. "closed ephemeral bead", "closed bead"). "(s)" is appended by // the printer when multiple items are involved. subjectNoun string - // ephemeralOnly restricts the filter to ephemeral beads when true. - // When false, restricts to non-ephemeral beads — the scopes are - // deliberately disjoint so `prune` never touches wisps that `purge` - // would handle, and vice versa. - ephemeralOnly bool - // requireFilter forces the user to pass --older-than or --pattern. - // Without this gate, `bd prune --force` would silently delete every - // closed non-ephemeral bead in the repo. - requireFilter bool - // ignoreReferences, when true, bypasses the reference-aware skip in prune. - // Always false for purge — ephemeral beads' references are themselves transient. - ignoreReferences bool + // tier is the plane this command sweeps. The two are DISJOINT, which is + // what makes purge and prune one capability rather than two: `prune` never + // touches a wisp that `purge` would handle, and vice versa. + tier issueops.SweepTier + // protectReferenced asks the role to skip candidates cited by a bead that + // is not done. `bd prune` asks unless --ignore-references; `bd purge` + // never does, because a wisp's citations are as transient as the wisp. + protectReferenced bool + // reportsReferences publishes the reference-skip keys under --json. It is + // separate from protectReferenced because `bd prune --ignore-references` + // still publishes them, as zeroes, which is the shipped shape. + reportsReferences bool } var purgeCmd = &cobra.Command{ @@ -87,138 +89,39 @@ EXAMPLES: countKey: "purged_count", dryRunCountKey: "purge_count", subjectNoun: "closed ephemeral bead", - ephemeralOnly: true, - requireFilter: false, + tier: issueops.SweepEphemeral, }) }, } -// buildReferencedSet scans every non-closed bead's description, notes, and -// comments for literal occurrences of any candidate ID and returns the set of -// candidate IDs that were found. Uses a Statuses filter (not ExcludeStatus) -// to avoid the PG ExcludeStatus coverage gap (be-jdeief). -func buildReferencedSet(ctx context.Context, st storage.DoltStorage, candidateIDs map[string]bool) (map[string]bool, error) { - if len(candidateIDs) == 0 { - return nil, nil +// openSweeper hands back the bulk-clearance role for whichever route this +// invocation is on, each through its OWN capability accessor — the store's for +// the direct route and the provider's for the proxied one. +// +// Neither branch builds a filter, matches a glob, rechecks closed_at or scans +// for citations. That is the whole of what moved behind the role, and it used +// to be written out twice, once per route, with the selection and the deletion +// in two DIFFERENT transactions on this one. +func openSweeper() (issueops.Sweeper, error) { + if usesProxiedServer() { + return proxiedSweeper() } - matcher := newCandidateIDMatcher(candidateIDs) - - // Scan every non-done bead: built-in active statuses plus any configured - // custom statuses whose category is not "done". A repo can define custom - // statuses (status.custom) in active/wip/frozen categories; a bead in such - // a status that cites a closed bead must protect it from prune exactly like - // a built-in open bead does. Reading custom statuses is required, not - // best-effort: if we cannot enumerate them we must not under-scan and risk - // deleting a referenced bead, so the error propagates and aborts the prune. - notClosedStatuses := []types.Status{ - types.StatusOpen, - types.StatusInProgress, - types.StatusBlocked, - types.StatusDeferred, - types.StatusPinned, - types.StatusHooked, - } - customStatuses, err := st.GetCustomStatusesDetailed(ctx) - if err != nil { - return nil, fmt.Errorf("reading custom statuses for reference scan: %w", err) - } - for _, cs := range customStatuses { - if cs.Category != types.CategoryDone { - notClosedStatuses = append(notClosedStatuses, types.Status(cs.Name)) - } - } - notClosed := types.IssueFilter{Statuses: notClosedStatuses} - openBeads, err := st.SearchIssues(ctx, "", notClosed) - if err != nil { - return nil, err - } - - refSet := make(map[string]bool) - scanText := func(text string) { - matcher.findAll(text, refSet) - } - - for _, iss := range openBeads { - scanText(iss.Description) - scanText(iss.Notes) - comments, err := st.GetIssueComments(ctx, iss.ID) - if err != nil { + if store == nil { + if err := ensureStoreActive(); err != nil { return nil, err } - for _, c := range comments { - scanText(c.Text) - } } - return refSet, nil + return store.Sweeper() } -type candidateIDMatcher struct { - byFirstByte map[byte][]string -} - -func newCandidateIDMatcher(candidateIDs map[string]bool) candidateIDMatcher { - byFirstByte := make(map[byte][]string) - for id := range candidateIDs { - if id == "" { - continue - } - byFirstByte[id[0]] = append(byFirstByte[id[0]], id) - } - for first := range byFirstByte { - ids := byFirstByte[first] - sort.Slice(ids, func(i, j int) bool { - if len(ids[i]) == len(ids[j]) { - return ids[i] < ids[j] - } - return len(ids[i]) > len(ids[j]) - }) - byFirstByte[first] = ids - } - return candidateIDMatcher{byFirstByte: byFirstByte} -} - -func (m candidateIDMatcher) findAll(text string, found map[string]bool) { - for i := 0; i < len(text); i++ { - ids := m.byFirstByte[text[i]] - if len(ids) == 0 || !isWordBoundaryAt(text, i) { - continue - } - for _, id := range ids { - end := i + len(id) - if end <= len(text) && strings.HasPrefix(text[i:], id) && isWordBoundaryAt(text, end) { - found[id] = true - break - } - } - } -} - -func isWordBoundaryAt(s string, idx int) bool { - var before, after byte - if idx > 0 { - before = s[idx-1] - } - if idx < len(s) { - after = s[idx] - } - return isASCIIWordByte(before) != isASCIIWordByte(after) -} - -func isASCIIWordByte(b byte) bool { - return b == '_' || - ('0' <= b && b <= '9') || - ('A' <= b && b <= 'Z') || - ('a' <= b && b <= 'z') -} - -// runPurgeOrPrune implements the shared delete-closed-beads flow used by -// both `bd purge` (ephemeral scope) and `bd prune` (non-ephemeral scope). -// The caller's scope controls the filter, messaging, and safety gate. +// runPurgeOrPrune implements the shared delete-closed-beads flow used by both +// `bd purge` (ephemeral tier) and `bd prune` (durable tier). +// +// It is ONE body across both routes now rather than two: the proxied twin held +// a second copy of the selection, the recheck, the reference scan and every +// line of output, and the two routes disagreed about which transaction the +// deletion ran in. func runPurgeOrPrune(cmd *cobra.Command, scope purgeScope) error { - if usesProxiedServer() { - return runPurgeOrPruneProxied(cmd, scope) - } - CheckReadonly(scope.cmdName) force, _ := cmd.Flags().GetBool("force") @@ -226,7 +129,15 @@ func runPurgeOrPrune(cmd *cobra.Command, scope purgeScope) error { olderThan, _ := cmd.Flags().GetString("older-than") pattern, _ := cmd.Flags().GetString("pattern") - if scope.requireFilter && olderThan == "" && pattern == "" { + // The ROLE refuses an unfiltered durable sweep — that guard is + // workapi.ValidateSweepRequest, below every front door, so an HTTP handler + // or a library embedder inherits it instead of having to remember it. This + // branch is here for the MESSAGE and nothing else: the role's refusal + // names request fields, and a person who typed `bd prune --force` needs to + // be told which two flags to reach for. The contract case + // RunSweeperRefusesAnUnfilteredDurableSweep is what proves the guard + // survives this branch being deleted. + if scope.tier == issueops.SweepDurable && olderThan == "" && pattern == "" { return HandleErrorWithHint( fmt.Sprintf("bd %s requires --older-than or --pattern", scope.cmdName), "Protects against accidental bulk deletion. Use `--pattern '*'` to\n"+ @@ -234,228 +145,194 @@ func runPurgeOrPrune(cmd *cobra.Command, scope purgeScope) error { " / `--pattern ''` to narrow the deletion.") } - if store == nil { - if err := ensureStoreActive(); err != nil { - return HandleErrorRespectJSON("%v", err) - } + request := issueops.SweepRequest{ + Actor: actor, + Tier: scope.tier, + IDPattern: pattern, + ProtectReferenced: scope.protectReferenced, + // A --dry-run and an UNCONFIRMED run ask the role the same question — + // "what would this do" — so both send DryRun. --force is this + // command's confirmation, not a request field, and a sweep that + // answered the confirmation prompt by deleting would be the worst + // possible reading of it. + DryRun: dryRun || !force, } - - ctx := rootCtx - - statusClosed := types.StatusClosed - ephemeralFlag := scope.ephemeralOnly - filter := types.IssueFilter{ - Status: &statusClosed, - Ephemeral: &ephemeralFlag, - } - - var cutoff *time.Time if olderThan != "" { days, err := parseHumanDuration(olderThan) if err != nil { return HandleErrorRespectJSON("invalid --older-than value %q: %v", olderThan, err) } - cutoffTime := time.Now().UTC().AddDate(0, 0, -days) - cutoff = &cutoffTime - filter.ClosedBefore = cutoff + cutoff := time.Now().UTC().AddDate(0, 0, -days) + request.ClosedBefore = &cutoff } - closedIssues, err := store.SearchIssues(ctx, "", filter) + sweeper, err := openSweeper() if err != nil { - return HandleErrorRespectJSON("listing issues: %v", err) + return HandleErrorRespectJSON("%v", err) } - - if pattern != "" { - var matched []*types.Issue - for _, issue := range closedIssues { - if ok, _ := filepath.Match(pattern, issue.ID); ok { - matched = append(matched, issue) - } - } - closedIssues = matched - } - - var safetyStats closedDeletionCandidateStats - closedIssues, safetyStats = filterClosedDeletionCandidates(closedIssues, cutoff) - pinnedCount := safetyStats.PinnedSkipped - warnClosedDeletionSafetySkips(safetyStats) - - // Reference-aware skip (prune only): filter closed beads cited by open beads. - referencedCount := 0 - var referencedSample []string - if scope.cmdName == "prune" && !scope.ignoreReferences { - candidateIDs := make(map[string]bool, len(closedIssues)) - for _, iss := range closedIssues { - candidateIDs[iss.ID] = true - } - refSet, err := buildReferencedSet(ctx, store, candidateIDs) - if err != nil { - return HandleErrorRespectJSON("scanning open beads for references: %v", err) - } - nonReferenced := closedIssues[:0] - for _, iss := range closedIssues { - if refSet[iss.ID] { - referencedCount++ - if len(referencedSample) < 100 { - referencedSample = append(referencedSample, iss.ID) - } - } else { - nonReferenced = append(nonReferenced, iss) - } - } - closedIssues = nonReferenced - } - - if len(closedIssues) == 0 { - if jsonOutput { - stats := map[string]interface{}{ - scope.countKey: 0, - "message": fmt.Sprintf("No %ss to %s", scope.subjectNoun, scope.cmdName), - } - if scope.cmdName == "prune" { - stats["referenced_skipped"] = referencedCount - stats["referenced_count"] = referencedCount - if len(referencedSample) > 0 { - stats["referenced_ids_sample"] = referencedSample - } - } - return outputJSON(stats) - } - msg := fmt.Sprintf("No %ss to %s", scope.subjectNoun, scope.cmdName) - if olderThan != "" { - msg += fmt.Sprintf(" (older than %s)", olderThan) - } - if pattern != "" { - msg += fmt.Sprintf(" (matching %q)", pattern) - } - fmt.Println(msg) - if referencedCount > 0 { - fmt.Println(ui.MutedStyle.Render(fmt.Sprintf( - " (%d closed bead(s) protected by open-bead references — use --ignore-references to override)", - referencedCount))) - } - return nil - } - - issueIDs := make([]string, len(closedIssues)) - for i, issue := range closedIssues { - issueIDs[i] = issue.ID - } - - if dryRun { - result, err := store.DeleteIssues(ctx, issueIDs, false, false, true) - if jsonOutput { - stats := map[string]interface{}{ - "dry_run": true, - scope.dryRunCountKey: len(issueIDs), - "dependencies": 0, - "labels": 0, - "events": 0, - } - if err == nil { - stats["dependencies"] = result.DependenciesCount - stats["labels"] = result.LabelsCount - stats["events"] = result.EventsCount - } - if pinnedCount > 0 { - stats["pinned_skipped"] = pinnedCount - } - if scope.cmdName == "prune" { - stats["referenced_skipped"] = referencedCount - stats["referenced_count"] = referencedCount - if len(referencedSample) > 0 { - stats["referenced_ids_sample"] = referencedSample - } - } - return outputJSON(stats) - } - fmt.Printf("Would %s %d %s(s)\n", scope.cmdName, len(issueIDs), scope.subjectNoun) - if err == nil { - fmt.Printf(" Dependencies: %d\n", result.DependenciesCount) - fmt.Printf(" Labels: %d\n", result.LabelsCount) - fmt.Printf(" Events: %d\n", result.EventsCount) - } - if pinnedCount > 0 { - fmt.Printf(" Pinned (skipped): %d\n", pinnedCount) - } - if referencedCount > 0 { - fmt.Printf(" %s %d\n", ui.MutedStyle.Render("Referenced (skipped):"), referencedCount) - sample := referencedSample - if len(sample) > 5 { - sample = sample[:5] - } - idStrs := make([]string, len(sample)) - for i, id := range sample { - idStrs[i] = ui.IDStyle.Render(id) - } - suffix := "" - if referencedCount > 5 { - suffix = ui.MutedStyle.Render(", ...") - } - fmt.Printf(" %s %s%s\n", ui.MutedStyle.Render("Referenced IDs (sample):"), strings.Join(idStrs, ", "), suffix) - } - fmt.Printf("\n(Dry-run mode — no changes made)\n") - return nil - } - - if !force { - fmt.Printf("Found %d %s(s) to %s\n", len(issueIDs), scope.subjectNoun, scope.cmdName) - if pinnedCount > 0 { - fmt.Printf("Skipping %d pinned bead(s)\n", pinnedCount) - } - if referencedCount > 0 { - fmt.Println(ui.MutedStyle.Render(fmt.Sprintf("Skipping %d referenced bead(s)", referencedCount))) - } - hint := fmt.Sprintf("bd %s --force", scope.cmdName) - if olderThan != "" { - hint += " --older-than " + olderThan - } - if pattern != "" { - hint += " --pattern " + pattern - } - return HandleErrorWithHint( - fmt.Sprintf("would %s %d bead(s)", scope.cmdName, len(issueIDs)), - fmt.Sprintf("Use --force to confirm or --dry-run to preview.\n %s", hint)) - } - - result, err := store.DeleteIssues(ctx, issueIDs, false, true, false) + result, err := sweeper.Sweep(rootCtx, request) if err != nil { return HandleErrorRespectJSON("%s failed: %v", scope.cmdName, err) } - commandDidWrite.Store(true) - if result.DeletedCount > 0 { - commandMayEmptyJSONLExport.Store(true) + warnSweepDefenseSkips(result.Skipped) + + switch { + case result.Swept == 0: + return emitSweepEmpty(scope, olderThan, pattern, result) + case dryRun: + return emitSweepDryRun(scope, result) + case !force: + return emitSweepConfirm(scope, olderThan, pattern, result) } + commandDidWrite.Store(true) + commandMayEmptyJSONLExport.Store(true) + return emitSweepResult(scope, result) +} + +// warnSweepDefenseSkips reports the candidates the role's own recheck threw +// out. A non-zero count means the tier query and the recheck disagreed about +// which rows are closed, which is worth a line on stderr whatever the output +// mode — see issueops.SweepSkips. +func warnSweepDefenseSkips(skips issueops.SweepSkips) { + total := skips.Unreadable + skips.NotClosed + skips.UnknownClosedAt + skips.ClosedAtOrAfterCutoff + if total == 0 { + return + } + WarnError("skipped %d deletion candidate(s) after closed_at safety recheck (nil=%d, non_closed=%d, missing_closed_at=%d, too_recent=%d)", + total, + skips.Unreadable, + skips.NotClosed, + skips.UnknownClosedAt, + skips.ClosedAtOrAfterCutoff, + ) +} + +// addReferenceStats attaches the reference-skip members to a --json payload. +// `bd prune` publishes them whether or not the protection was asked for, which +// is the shipped shape; `bd purge` never does. +func addReferenceStats(scope purgeScope, stats map[string]interface{}, result issueops.SweepResult) { + if !scope.reportsReferences { + return + } + stats["referenced_skipped"] = result.Skipped.Referenced + stats["referenced_count"] = result.Skipped.Referenced + if len(result.ReferencedIDs) > 0 { + stats["referenced_ids_sample"] = result.ReferencedIDs + } +} + +func emitSweepEmpty(scope purgeScope, olderThan, pattern string, result issueops.SweepResult) error { if jsonOutput { stats := map[string]interface{}{ - scope.countKey: result.DeletedCount, - "dependencies": result.DependenciesCount, - "labels": result.LabelsCount, - "events": result.EventsCount, - } - if pinnedCount > 0 { - stats["pinned_skipped"] = pinnedCount - } - if scope.cmdName == "prune" { - stats["referenced_skipped"] = referencedCount - stats["referenced_count"] = referencedCount - if len(referencedSample) > 0 { - stats["referenced_ids_sample"] = referencedSample - } + scope.countKey: 0, + "message": fmt.Sprintf("No %ss to %s", scope.subjectNoun, scope.cmdName), } + addReferenceStats(scope, stats, result) return outputJSON(stats) } - fmt.Printf("%s %s %d %s(s)\n", ui.RenderPass("✓"), capitalize(scope.pastTense), result.DeletedCount, scope.subjectNoun) - fmt.Printf(" Dependencies removed: %d\n", result.DependenciesCount) - fmt.Printf(" Labels removed: %d\n", result.LabelsCount) - fmt.Printf(" Events removed: %d\n", result.EventsCount) - if pinnedCount > 0 { - fmt.Printf(" Pinned (skipped): %d\n", pinnedCount) + msg := fmt.Sprintf("No %ss to %s", scope.subjectNoun, scope.cmdName) + if olderThan != "" { + msg += fmt.Sprintf(" (older than %s)", olderThan) } - if referencedCount > 0 { - fmt.Printf(" %s %d\n", ui.MutedStyle.Render("Referenced (skipped):"), referencedCount) + if pattern != "" { + msg += fmt.Sprintf(" (matching %q)", pattern) + } + fmt.Println(msg) + if result.Skipped.Referenced > 0 { + fmt.Println(ui.MutedStyle.Render(fmt.Sprintf( + " (%d closed bead(s) protected by open-bead references — use --ignore-references to override)", + result.Skipped.Referenced))) + } + return nil +} + +func emitSweepDryRun(scope purgeScope, result issueops.SweepResult) error { + if jsonOutput { + stats := map[string]interface{}{ + "dry_run": true, + scope.dryRunCountKey: result.Swept, + "dependencies": result.Dependencies, + "labels": result.Labels, + "events": result.Events, + } + if result.Skipped.Pinned > 0 { + stats["pinned_skipped"] = result.Skipped.Pinned + } + addReferenceStats(scope, stats, result) + return outputJSON(stats) + } + fmt.Printf("Would %s %d %s(s)\n", scope.cmdName, result.Swept, scope.subjectNoun) + fmt.Printf(" Dependencies: %d\n", result.Dependencies) + fmt.Printf(" Labels: %d\n", result.Labels) + fmt.Printf(" Events: %d\n", result.Events) + if result.Skipped.Pinned > 0 { + fmt.Printf(" Pinned (skipped): %d\n", result.Skipped.Pinned) + } + if result.Skipped.Referenced > 0 { + fmt.Printf(" %s %d\n", ui.MutedStyle.Render("Referenced (skipped):"), result.Skipped.Referenced) + sample := result.ReferencedIDs + if len(sample) > 5 { + sample = sample[:5] + } + idStrs := make([]string, len(sample)) + for i, id := range sample { + idStrs[i] = ui.IDStyle.Render(id) + } + suffix := "" + if result.Skipped.Referenced > 5 { + suffix = ui.MutedStyle.Render(", ...") + } + fmt.Printf(" %s %s%s\n", ui.MutedStyle.Render("Referenced IDs (sample):"), strings.Join(idStrs, ", "), suffix) + } + fmt.Printf("\n(Dry-run mode — no changes made)\n") + return nil +} + +func emitSweepConfirm(scope purgeScope, olderThan, pattern string, result issueops.SweepResult) error { + fmt.Printf("Found %d %s(s) to %s\n", result.Swept, scope.subjectNoun, scope.cmdName) + if result.Skipped.Pinned > 0 { + fmt.Printf("Skipping %d pinned bead(s)\n", result.Skipped.Pinned) + } + if result.Skipped.Referenced > 0 { + fmt.Println(ui.MutedStyle.Render(fmt.Sprintf("Skipping %d referenced bead(s)", result.Skipped.Referenced))) + } + hint := fmt.Sprintf("bd %s --force", scope.cmdName) + if olderThan != "" { + hint += " --older-than " + olderThan + } + if pattern != "" { + hint += " --pattern " + pattern + } + return HandleErrorWithHint( + fmt.Sprintf("would %s %d bead(s)", scope.cmdName, result.Swept), + fmt.Sprintf("Use --force to confirm or --dry-run to preview.\n %s", hint)) +} + +func emitSweepResult(scope purgeScope, result issueops.SweepResult) error { + if jsonOutput { + stats := map[string]interface{}{ + scope.countKey: result.Swept, + "dependencies": result.Dependencies, + "labels": result.Labels, + "events": result.Events, + } + if result.Skipped.Pinned > 0 { + stats["pinned_skipped"] = result.Skipped.Pinned + } + addReferenceStats(scope, stats, result) + return outputJSON(stats) + } + fmt.Printf("%s %s %d %s(s)\n", ui.RenderPass("✓"), capitalize(scope.pastTense), result.Swept, scope.subjectNoun) + fmt.Printf(" Dependencies removed: %d\n", result.Dependencies) + fmt.Printf(" Labels removed: %d\n", result.Labels) + fmt.Printf(" Events removed: %d\n", result.Events) + if result.Skipped.Pinned > 0 { + fmt.Printf(" Pinned (skipped): %d\n", result.Skipped.Pinned) + } + if result.Skipped.Referenced > 0 { + fmt.Printf(" %s %d\n", ui.MutedStyle.Render("Referenced (skipped):"), result.Skipped.Referenced) } return nil } diff --git a/cmd/bd/purge_proxied_server.go b/cmd/bd/purge_proxied_server.go index 85b8b294d..a1da9ab1f 100644 --- a/cmd/bd/purge_proxied_server.go +++ b/cmd/bd/purge_proxied_server.go @@ -1,359 +1,32 @@ package main import ( - "context" + "errors" "fmt" - "path/filepath" - "strings" - "time" - "github.com/spf13/cobra" - - "github.com/steveyegge/beads/internal/storage/domain" "github.com/steveyegge/beads/internal/storage/uow" - "github.com/steveyegge/beads/internal/types" - "github.com/steveyegge/beads/internal/ui" + "github.com/steveyegge/beads/issueops" ) -type purgeProxiedTxResult struct { - empty bool - dryRun bool - needsConfirm bool - - issueIDs []string - deleteResult domain.DeleteIssuesResult - deleteErr error - pinnedCount int - referencedCount int - referencedSample []string - safetyStats closedDeletionCandidateStats -} - -func runPurgeOrPruneProxied(cmd *cobra.Command, scope purgeScope) error { - CheckReadonly(scope.cmdName) - - force, _ := cmd.Flags().GetBool("force") - dryRun, _ := cmd.Flags().GetBool("dry-run") - olderThan, _ := cmd.Flags().GetString("older-than") - pattern, _ := cmd.Flags().GetString("pattern") - - if scope.requireFilter && olderThan == "" && pattern == "" { - return HandleErrorWithHint( - fmt.Sprintf("bd %s requires --older-than or --pattern", scope.cmdName), - "Protects against accidental bulk deletion. Use `--pattern '*'` to\n"+ - " include all closed beads in this scope, or `--older-than 1d`\n"+ - " / `--pattern ''` to narrow the deletion.") - } - +// proxiedSweeper hands back the guarded bulk-clearance surface for the +// proxied-server provider, through the provider's OWN capability accessor — +// the same two-step proxiedCounter performs, and for the same reason: the +// accessor is where each layer is added. +// +// This file used to hold a second copy of `bd purge` and `bd prune`: it parsed +// the flags again, opened a unit of work, built the candidate filter, matched +// the glob, re-applied the pinned and closed_at rechecks, ran its own +// reference scan, and then printed 190 lines of output that had to be kept +// character-identical to the direct route's by hand. All of that is behind the +// role now, so the proxied route differs from the direct one by which accessor +// it asks and nothing else. +func proxiedSweeper() (issueops.Sweeper, error) { if uowProvider == nil { - return HandleErrorRespectJSON("proxied-server UOW provider not initialized") + return nil, errors.New("proxied-server UOW provider not initialized") } - ctx := rootCtx - - res, err := uow.RunTxResult(ctx, uowProvider, func(ctx context.Context, uw uow.UnitOfWork) (purgeProxiedTxResult, string, error) { - var result purgeProxiedTxResult - - statusClosed := types.StatusClosed - ephemeralFlag := scope.ephemeralOnly - filter := types.IssueFilter{ - Status: &statusClosed, - Ephemeral: &ephemeralFlag, - } - - var cutoff *time.Time - if olderThan != "" { - days, err := parseHumanDuration(olderThan) - if err != nil { - return result, "", fmt.Errorf("invalid --older-than value %q: %w", olderThan, err) - } - cutoffTime := time.Now().UTC().AddDate(0, 0, -days) - cutoff = &cutoffTime - filter.ClosedBefore = cutoff - } - - page, err := uw.IssueUseCase().SearchIssues(ctx, "", filter) - if err != nil { - return result, "", fmt.Errorf("listing issues: %w", err) - } - closedIssues := page.Items - - if pattern != "" { - var matched []*types.Issue - for _, issue := range closedIssues { - if ok, _ := filepath.Match(pattern, issue.ID); ok { - matched = append(matched, issue) - } - } - closedIssues = matched - } - - closedIssues, result.safetyStats = filterClosedDeletionCandidates(closedIssues, cutoff) - result.pinnedCount = result.safetyStats.PinnedSkipped - - if scope.cmdName == "prune" && !scope.ignoreReferences { - candidateIDs := make(map[string]bool, len(closedIssues)) - for _, iss := range closedIssues { - candidateIDs[iss.ID] = true - } - refSet, err := buildReferencedSetProxied(ctx, uw, candidateIDs) - if err != nil { - return result, "", fmt.Errorf("scanning open beads for references: %w", err) - } - nonReferenced := closedIssues[:0] - for _, iss := range closedIssues { - if refSet[iss.ID] { - result.referencedCount++ - if len(result.referencedSample) < 100 { - result.referencedSample = append(result.referencedSample, iss.ID) - } - } else { - nonReferenced = append(nonReferenced, iss) - } - } - closedIssues = nonReferenced - } - - if len(closedIssues) == 0 { - result.empty = true - return result, "", nil - } - - result.issueIDs = make([]string, len(closedIssues)) - for i, issue := range closedIssues { - result.issueIDs[i] = issue.ID - } - - if dryRun { - result.dryRun = true - result.deleteResult, result.deleteErr = uw.IssueUseCase().DeleteIssues(ctx, domain.DeleteIssuesParams{ - IDs: result.issueIDs, - DryRun: true, - }, actor) - return result, "", nil - } - - if !force { - result.needsConfirm = true - return result, "", nil - } - - deleteResult, err := uw.IssueUseCase().DeleteIssues(ctx, domain.DeleteIssuesParams{ - IDs: result.issueIDs, - }, actor) - if err != nil { - return result, "", fmt.Errorf("%s failed: %w", scope.cmdName, err) - } - result.deleteResult = deleteResult - - return result, fmt.Sprintf("bd: %s %d bead(s)", scope.cmdName, deleteResult.DeletedCount), nil - }) - if err != nil { - return HandleErrorRespectJSON("%v", err) - } - - warnClosedDeletionSafetySkips(res.safetyStats) - - switch { - case res.empty: - return emitProxiedPruneEmpty(scope, olderThan, pattern, res.referencedCount, res.referencedSample) - case res.dryRun: - return emitProxiedPruneDryRun(scope, res.issueIDs, res.deleteResult, res.deleteErr, res.pinnedCount, res.referencedCount, res.referencedSample) - case res.needsConfirm: - return emitProxiedPruneConfirm(scope, res.issueIDs, olderThan, pattern, res.pinnedCount, res.referencedCount) - default: - return emitProxiedPruneResult(scope, res.deleteResult, res.pinnedCount, res.referencedCount, res.referencedSample) + src, ok := uowProvider.(uow.SweeperSource) + if !ok { + return nil, fmt.Errorf("proxied-server provider %T does not offer the bulk-clearance surface", uowProvider) } -} - -func buildReferencedSetProxied(ctx context.Context, uw uow.UnitOfWork, candidateIDs map[string]bool) (map[string]bool, error) { - if len(candidateIDs) == 0 { - return nil, nil - } - matcher := newCandidateIDMatcher(candidateIDs) - - notClosedStatuses := []types.Status{ - types.StatusOpen, - types.StatusInProgress, - types.StatusBlocked, - types.StatusDeferred, - types.StatusPinned, - types.StatusHooked, - } - customStatuses, err := uw.ConfigUseCase().GetCustomStatuses(ctx) - if err != nil { - return nil, fmt.Errorf("reading custom statuses for reference scan: %w", err) - } - for _, cs := range customStatuses { - if cs.Category != types.CategoryDone { - notClosedStatuses = append(notClosedStatuses, types.Status(cs.Name)) - } - } - notClosed := types.IssueFilter{Statuses: notClosedStatuses} - page, err := uw.IssueUseCase().SearchIssues(ctx, "", notClosed) - if err != nil { - return nil, err - } - openBeads := page.Items - - openIDs := make([]string, len(openBeads)) - for i, iss := range openBeads { - openIDs[i] = iss.ID - } - commentsByIssue, err := uw.CommentUseCase().GetCommentsForIssues(ctx, openIDs) - if err != nil { - return nil, err - } - - refSet := make(map[string]bool) - scanText := func(text string) { - matcher.findAll(text, refSet) - } - - for _, iss := range openBeads { - scanText(iss.Description) - scanText(iss.Notes) - for _, c := range commentsByIssue[iss.ID] { - scanText(c.Text) - } - } - return refSet, nil -} - -func emitProxiedPruneEmpty(scope purgeScope, olderThan, pattern string, referencedCount int, referencedSample []string) error { - if jsonOutput { - stats := map[string]interface{}{ - scope.countKey: 0, - "message": fmt.Sprintf("No %ss to %s", scope.subjectNoun, scope.cmdName), - } - if scope.cmdName == "prune" { - stats["referenced_skipped"] = referencedCount - stats["referenced_count"] = referencedCount - if len(referencedSample) > 0 { - stats["referenced_ids_sample"] = referencedSample - } - } - return outputJSON(stats) - } - msg := fmt.Sprintf("No %ss to %s", scope.subjectNoun, scope.cmdName) - if olderThan != "" { - msg += fmt.Sprintf(" (older than %s)", olderThan) - } - if pattern != "" { - msg += fmt.Sprintf(" (matching %q)", pattern) - } - fmt.Println(msg) - if referencedCount > 0 { - fmt.Println(ui.MutedStyle.Render(fmt.Sprintf( - " (%d closed bead(s) protected by open-bead references — use --ignore-references to override)", - referencedCount))) - } - return nil -} - -func emitProxiedPruneDryRun(scope purgeScope, issueIDs []string, result domain.DeleteIssuesResult, resultErr error, pinnedCount, referencedCount int, referencedSample []string) error { - if jsonOutput { - stats := map[string]interface{}{ - "dry_run": true, - scope.dryRunCountKey: len(issueIDs), - "dependencies": 0, - "labels": 0, - "events": 0, - } - if resultErr == nil { - stats["dependencies"] = result.DependenciesCount - stats["labels"] = result.LabelsCount - stats["events"] = result.EventsCount - } - if pinnedCount > 0 { - stats["pinned_skipped"] = pinnedCount - } - if scope.cmdName == "prune" { - stats["referenced_skipped"] = referencedCount - stats["referenced_count"] = referencedCount - if len(referencedSample) > 0 { - stats["referenced_ids_sample"] = referencedSample - } - } - return outputJSON(stats) - } - fmt.Printf("Would %s %d %s(s)\n", scope.cmdName, len(issueIDs), scope.subjectNoun) - if resultErr == nil { - fmt.Printf(" Dependencies: %d\n", result.DependenciesCount) - fmt.Printf(" Labels: %d\n", result.LabelsCount) - fmt.Printf(" Events: %d\n", result.EventsCount) - } - if pinnedCount > 0 { - fmt.Printf(" Pinned (skipped): %d\n", pinnedCount) - } - if referencedCount > 0 { - fmt.Printf(" %s %d\n", ui.MutedStyle.Render("Referenced (skipped):"), referencedCount) - sample := referencedSample - if len(sample) > 5 { - sample = sample[:5] - } - idStrs := make([]string, len(sample)) - for i, id := range sample { - idStrs[i] = ui.IDStyle.Render(id) - } - suffix := "" - if referencedCount > 5 { - suffix = ui.MutedStyle.Render(", ...") - } - fmt.Printf(" %s %s%s\n", ui.MutedStyle.Render("Referenced IDs (sample):"), strings.Join(idStrs, ", "), suffix) - } - fmt.Printf("\n(Dry-run mode — no changes made)\n") - return nil -} - -func emitProxiedPruneConfirm(scope purgeScope, issueIDs []string, olderThan, pattern string, pinnedCount, referencedCount int) error { - fmt.Printf("Found %d %s(s) to %s\n", len(issueIDs), scope.subjectNoun, scope.cmdName) - if pinnedCount > 0 { - fmt.Printf("Skipping %d pinned bead(s)\n", pinnedCount) - } - if referencedCount > 0 { - fmt.Println(ui.MutedStyle.Render(fmt.Sprintf("Skipping %d referenced bead(s)", referencedCount))) - } - hint := fmt.Sprintf("bd %s --force", scope.cmdName) - if olderThan != "" { - hint += " --older-than " + olderThan - } - if pattern != "" { - hint += " --pattern " + pattern - } - return HandleErrorWithHint( - fmt.Sprintf("would %s %d bead(s)", scope.cmdName, len(issueIDs)), - fmt.Sprintf("Use --force to confirm or --dry-run to preview.\n %s", hint)) -} - -func emitProxiedPruneResult(scope purgeScope, result domain.DeleteIssuesResult, pinnedCount, referencedCount int, referencedSample []string) error { - if jsonOutput { - stats := map[string]interface{}{ - scope.countKey: result.DeletedCount, - "dependencies": result.DependenciesCount, - "labels": result.LabelsCount, - "events": result.EventsCount, - } - if pinnedCount > 0 { - stats["pinned_skipped"] = pinnedCount - } - if scope.cmdName == "prune" { - stats["referenced_skipped"] = referencedCount - stats["referenced_count"] = referencedCount - if len(referencedSample) > 0 { - stats["referenced_ids_sample"] = referencedSample - } - } - return outputJSON(stats) - } - fmt.Printf("%s %s %d %s(s)\n", ui.RenderPass("✓"), capitalize(scope.pastTense), result.DeletedCount, scope.subjectNoun) - fmt.Printf(" Dependencies removed: %d\n", result.DependenciesCount) - fmt.Printf(" Labels removed: %d\n", result.LabelsCount) - fmt.Printf(" Events removed: %d\n", result.EventsCount) - if pinnedCount > 0 { - fmt.Printf(" Pinned (skipped): %d\n", pinnedCount) - } - if referencedCount > 0 { - fmt.Printf(" %s %d\n", ui.MutedStyle.Render("Referenced (skipped):"), referencedCount) - } - return nil + return src.Sweeper() } diff --git a/cmd/bd/serve.go b/cmd/bd/serve.go index 14c0d2888..6952b9d74 100644 --- a/cmd/bd/serve.go +++ b/cmd/bd/serve.go @@ -77,7 +77,17 @@ WHAT THIS DOES NOT DO An actor on an HTTP request is caller-asserted provenance for the audit trail, not authenticated identity — the same thing it has always been on the CLI, - where any local process can pass any --actor.`, + where any local process can pass any --actor. + +DESTRUCTIVE OPERATIONS + + POST /v0/beads/issues:sweep deletes closed beads in bulk — the operation + behind bd purge and bd prune — and nothing it deletes comes back. It shares + the library surface those commands call, so it inherits their guards: pinned + beads are never swept, and a durable sweep with neither a cutoff nor an id + pattern is refused rather than clearing every closed bead in the workspace. + Combined with the trust model above, that means anyone who can reach this + address can erase closed work; bind it accordingly.`, Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { return runServe() @@ -152,7 +162,8 @@ func runServe() error { if serveAllowNonLoopback { fmt.Fprintf(os.Stderr, "bd serve: WARNING: --allow-non-loopback binds %s beyond loopback. "+ - "This API has no authentication and no TLS: any peer that can reach it can read every issue and claim work as any actor.\n", + "This API has no authentication and no TLS: any peer that can reach it can read every issue, claim work as any actor, "+ + "and bulk-delete closed beads.\n", serveAddr) } diff --git a/cmd/bd/serve_test.go b/cmd/bd/serve_test.go index 286c58b62..7081adf96 100644 --- a/cmd/bd/serve_test.go +++ b/cmd/bd/serve_test.go @@ -258,7 +258,14 @@ func TestServeBuildsOnlyAProviderBackedServer(t *testing.T) { continue } switch key.Name { - case "Reader", "Claimer", "EdgeReader": + // NOT the whole role set — Settings, Stats, CycleDetector and + // ReadyCounter are absent, which is a gap someone should close + // by inverting this into "any key that is not one of the + // non-role fields". Sweeper is here because it is the one + // whose absence would cost most: a roles-backed server over + // the embedded store answers a BULK DELETE while its + // per-request atomicity claim is false. + case "Reader", "Claimer", "EdgeReader", "Sweeper": t.Errorf("%s: bd serve sets httpapi.Config.%s. The roles source bypasses the unit-of-work "+ "provider serveModeGate vouched for, and internal/httpapi cannot tell an embedded-backed "+ "role from any other — read serveModeGate before changing this", diff --git a/engdocs/ADDING_AN_ISSUEOPS_ROLE.md b/engdocs/ADDING_AN_ISSUEOPS_ROLE.md index 6bd6dc92b..382f03847 100644 --- a/engdocs/ADDING_AN_ISSUEOPS_ROLE.md +++ b/engdocs/ADDING_AN_ISSUEOPS_ROLE.md @@ -100,6 +100,16 @@ in; 13 is the HTTP surface, which lands with the command rather than after it. `hook_counter.go`). Either way the accessor must EXIST on the decorator: it is declared, never inherited, and there is a reflection test that says so. + **The read/write split is about the HOOK VOCABULARY, not about whether the + role writes.** `internal/hooks` publishes on_create, on_update and on_close + and nothing else, so a write the vocabulary cannot name recurses like a + read: `issueops.Sweeper` deletes rows and `hook_sweeper.go` is three lines, + because there is no on_delete to fire and the rows a hook script would be + handed are gone. Say which it is and why in the file, and assert it in + `role_accessor_decorator_test.go`'s wrapped/unwrapped table — the whole + point of that table is that "recurses" reads identically whether it was a + decision or an omission. + 7. **The telemetry wrapper.** `internal/telemetry/.go`. No read/write distinction here — telemetry spans reads too, so every method gets `storage.op` / `storage.done`. diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index 3ae2573b3..ccd9e9a0e 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -25,6 +25,24 @@ func (e HealthStatus) Valid() bool { } } +// Defines values for SweepRequestTier. +const ( + Durable SweepRequestTier = "durable" + Ephemeral SweepRequestTier = "ephemeral" +) + +// Valid indicates whether the value is a known member of the SweepRequestTier enum. +func (e SweepRequestTier) Valid() bool { + switch e { + case Durable: + return true + case Ephemeral: + return true + default: + return false + } +} + // Defines values for QueryIssuesParamsSort. const ( QueryIssuesParamsSortAssignee QueryIssuesParamsSort = "assignee" @@ -120,7 +138,7 @@ type ContextResponse struct { // BeadsDir Absolute path of the served workspace's `.beads` directory. A host path, kept because it is the single-workspace server's only workspace-identity handshake; disclosing it to network peers is part of what an operator accepts when binding beyond loopback. BeadsDir string `json:"beads_dir"` - // Capabilities The operations this server actually implements, derived from its route table. v0's vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.get`, `issues.claim`, `stats.get`, `config.list`, `config.get`, `dependencies.cycles`, `dependencies.list`; it grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation — never the version string. + // Capabilities The operations this server actually implements, derived from its route table. v0's vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.get`, `issues.claim`, `issues.sweep`, `stats.get`, `config.list`, `config.get`, `dependencies.cycles`, `dependencies.list`; it grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation — never the version string. Capabilities []string `json:"capabilities"` // Database Logical database name (not a host or a DSN). @@ -316,6 +334,79 @@ type StatsResponse struct { Summary Statistics `json:"summary"` } +// SweepRequest Which closed beads to clear. The predicate is FIXED at "closed beads of one tier" and the two narrowing members only narrow it: there is no status, no assignee, no label and no free-text query here, because every one of those would be another way to spell a destructive selection that a caller could get subtly wrong. +// +// `additionalProperties: false`, so an unknown member is a `400` naming the member — the same posture the query-parameter rule takes, and for the same reason: on this operation a silently ignored narrowing term widens what is erased. +type SweepRequest struct { + // Actor Caller-asserted attribution for wherever the backend records it, under the same rules and for the same reasons as `ClaimRequest`'s `actor`: trimmed, refused when empty after trimming, over 256 BYTES, or carrying any control character. Optional — a deleted bead leaves no row to attribute the deletion on. + Actor *string `json:"actor,omitempty"` + + // ClosedBefore Keep only beads closed STRICTLY BEFORE this instant (RFC 3339). A bead closed exactly at it is kept, which is the half-open interval every other time bound on this surface uses. `bd prune --older-than 30d` resolves the duration itself and sends the resulting instant. + ClosedBefore *time.Time `json:"closed_before,omitempty"` + + // DryRun Report what the sweep WOULD do and delete nothing. The counts, the skips and the refusals are the same ones the real sweep would produce, computed against the same snapshot — and nothing is recorded in history either. + DryRun *bool `json:"dry_run,omitempty"` + + // Pattern Keep only beads whose id matches this shell glob (`*`, `?`, `[...]`, `\` escapes; `*` also crosses `-` and `.`, since an id is not a path). Absent matches every bead in the tier. A MALFORMED glob is a `400`, never a pattern that matches nothing. + Pattern *string `json:"pattern,omitempty"` + + // ProtectReferenced Skip candidates whose id is CITED — as a literal, at word boundaries — in the description, notes or comments of any bead that is not done, so a decision trail a live bead still points at is not deleted out from under it. It costs a full scan of the not-done set and its comments, which is why it is asked for rather than always on. `bd prune` asks unless `--ignore-references`; `bd purge` never does. + ProtectReferenced *bool `json:"protect_referenced,omitempty"` + + // Tier Which plane to clear. `ephemeral` is the wisp tier (`bd purge`) and `durable` is the issue tier (`bd prune`). The two are DISJOINT: a sweep of one can never touch a bead of the other. Required, with no default — a caller handed the wrong tier has nothing to notice until the beads are gone. + Tier SweepRequestTier `json:"tier"` +} + +// SweepRequestTier Which plane to clear. `ephemeral` is the wisp tier (`bd purge`) and `durable` is the issue tier (`bd prune`). The two are DISJOINT: a sweep of one can never touch a bead of the other. Required, with no default — a caller handed the wrong tier has nothing to notice until the beads are gone. +type SweepRequestTier string + +// SweepResult What one sweep did. Every number describes the SAME snapshot, because the selection and the deletion ran in one transaction. +// +// It is NOT `x-go-type`-pinned, for the reason `ReadyCount` is not: there is no canonical Go struct whose JSON encoding is this contract. The CLI publishes these numbers under its own per-command keys (`purged_count`, `pruned_count`), which are a stdout presentation rather than a wire type, so pinning would weld this body to one of them. +type SweepResult struct { + // Dependencies Dependency edge rows removed with them. Reported because a sweep's visible effect is much larger than its bead count. + Dependencies int `json:"dependencies"` + + // DryRun Echoes the request, so a result carries whether its numbers describe beads that are gone or beads that would go. + DryRun bool `json:"dry_run"` + + // Events Event rows removed with the swept beads. + Events int `json:"events"` + + // Labels Label rows removed with the swept beads. + Labels int `json:"labels"` + + // ReferencedIds A BOUNDED SAMPLE of the ids `skipped.referenced` counts — at most 100, in the order the candidate query returned them. It is a sample, not the set: compare its length against 100 to tell a truncated one from a complete one. Absent when nothing was protected. + ReferencedIds *[]string `json:"referenced_ids,omitempty"` + + // Skipped The candidates a sweep declined to delete, bucketed by WHY. They are separate counters rather than one number because they mean different things: the first two are PROTECTIONS, and the last four are the sweep declining to trust its own input. + Skipped SweepSkips `json:"skipped"` + + // Swept How many beads were deleted, or under `dry_run` would be. + Swept int `json:"swept"` +} + +// SweepSkips The candidates a sweep declined to delete, bucketed by WHY. They are separate counters rather than one number because they mean different things: the first two are PROTECTIONS, and the last four are the sweep declining to trust its own input. +type SweepSkips struct { + // ClosedAtOrAfterCutoff Candidates whose close timestamp did not satisfy `closed_before`. See `not_closed`. + ClosedAtOrAfterCutoff int `json:"closed_at_or_after_cutoff"` + + // NotClosed Candidates the tier query returned that the recheck found were not closed. A NON-ZERO VALUE HERE IS A DEFENSE FIRING, not a normal outcome: the query asked for exactly the beads this excludes. + NotClosed int `json:"not_closed"` + + // Pinned Candidates protected by the pinned flag. No request member overrides it — a caller who wants a pinned bead gone unpins it first. + Pinned int `json:"pinned"` + + // Referenced Candidates protected by `protect_referenced`. Always 0 when that member is false or absent, so a 0 read without having asked says nothing about whether beads are cited. + Referenced int `json:"referenced"` + + // UnknownClosedAt Closed candidates carrying no close timestamp. See `not_closed`. + UnknownClosedAt int `json:"unknown_closed_at"` + + // Unreadable Rows the tier query returned as nothing at all. A defense of the same kind, on a shape rather than a value. + Unreadable int `json:"unreadable"` +} + // IssueID defines model for IssueID. type IssueID = string @@ -574,3 +665,6 @@ type GetStatsParams struct { // ClaimIssueJSONRequestBody defines body for ClaimIssue for application/json ContentType. type ClaimIssueJSONRequestBody = ClaimRequest + +// SweepIssuesJSONRequestBody defines body for SweepIssues for application/json ContentType. +type SweepIssuesJSONRequestBody = SweepRequest diff --git a/internal/httpapi/claim.go b/internal/httpapi/claim.go index c90336a31..ed0388641 100644 --- a/internal/httpapi/claim.go +++ b/internal/httpapi/claim.go @@ -34,9 +34,10 @@ const ( // maxActorBytes is the document's cap on `actor`. The schema's maxLength // counts characters; this byte limit is the binding one. maxActorBytes = 256 - // maxClaimBodyBytes bounds the request body. The only member is an actor of - // at most a few hundred bytes, so this is pure refusal of the absurd. - maxClaimBodyBytes = 1 << 20 + // maxJSONBodyBytes bounds every request body on this surface. The largest + // documented one is a handful of short members, so this is pure refusal of + // the absurd. + maxJSONBodyBytes = 1 << 20 // claimContentType is the one media type this operation accepts, and // refusing anything else is a CSRF control, not pedantry: a JSON content // type is not CORS-"simple", so a cross-origin claim always triggers a @@ -175,7 +176,7 @@ func (s *Server) requireJSONContent(w http.ResponseWriter, r *http.Request) bool // assignee column AND interpolated into the storage commit message — where an // unvalidated newline forges audit-trail lines that look like separate commits. func (s *Server) claimActor(w http.ResponseWriter, r *http.Request) (string, bool) { - members, res := decodeClaimBody(w, r) + members, res := decodeJSONObjectBody(w, r) if res != nil { s.fail(w, r, *res) return "", false @@ -223,12 +224,15 @@ func (s *Server) claimActor(w http.ResponseWriter, r *http.Request) (string, boo return trimmed, true } -// decodeClaimBody reads the body as a JSON object of raw members. Decoding the -// members rather than the generated struct is what makes the schema's +// decodeJSONObjectBody reads the body as a JSON object of raw members. Decoding +// the members rather than the generated struct is what makes the schema's // additionalProperties: false enforceable by NAME: encoding/json's // DisallowUnknownFields reports the offender only inside an error string, and -// this endpoint exists to let clients stop parsing prose. -func decodeClaimBody(w http.ResponseWriter, r *http.Request) (map[string]json.RawMessage, *Result) { +// these endpoints exist to let clients stop parsing prose. +// +// Shared with the sweep, which has the same posture and the same reason for +// it; the member vocabulary each operation accepts is its own. +func decodeJSONObjectBody(w http.ResponseWriter, r *http.Request) (map[string]json.RawMessage, *Result) { // A body with no nameable part: `param` is documented absent on exactly // this case and present on every other 400. unparseable := func(detail string) *Result { @@ -237,11 +241,11 @@ func decodeClaimBody(w http.ResponseWriter, r *http.Request) (map[string]json.Ra } var members map[string]json.RawMessage - dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, maxClaimBodyBytes)) + dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, maxJSONBodyBytes)) if err := dec.Decode(&members); err != nil { var tooLarge *http.MaxBytesError if errors.As(err, &tooLarge) { - return nil, unparseable(fmt.Sprintf("request body is larger than %d bytes", maxClaimBodyBytes)) + return nil, unparseable(fmt.Sprintf("request body is larger than %d bytes", maxJSONBodyBytes)) } return nil, unparseable("request body must be a JSON object") } @@ -364,6 +368,7 @@ var ( _ uow.IssueClaimerSource = timedProvider{} _ uow.ReadyCounterSource = timedProvider{} _ uow.QuerierSource = timedProvider{} + _ uow.SweeperSource = timedProvider{} ) // IssueReader builds the reader OVER THIS WRAPPER rather than delegating to the @@ -443,6 +448,15 @@ func (p timedProvider) Querier() (issueops.Querier, error) { return uow.NewQuerier(p) } +// Sweeper builds the sweeper OVER THIS WRAPPER, for the same reason as the two +// above and with the sharpest version of the hazard: the sweep is the longest +// unit of work on this surface, so a sweeper bound to the untimed provider +// would report uow_ms=0.000 for exactly the request whose duration an operator +// most wants in the log. +func (p timedProvider) Sweeper() (issueops.Sweeper, error) { + return uow.NewSweeper(p) +} + func (p timedProvider) NewUOW(ctx context.Context) (uow.UnitOfWork, error) { start := time.Now() uw, err := p.inner.NewUOW(ctx) diff --git a/internal/httpapi/claim_test.go b/internal/httpapi/claim_test.go index ead9aef8a..e9108b93e 100644 --- a/internal/httpapi/claim_test.go +++ b/internal/httpapi/claim_test.go @@ -611,10 +611,10 @@ func TestClaimRejectsMalformedRequests(t *testing.T) { // rather than over a socket: a megabyte in flight makes the assertion about // timing instead of about the rule. func TestClaimBodyCapIsEnforcedWhileReading(t *testing.T) { - oversized := `{"actor":"` + strings.Repeat("x", maxClaimBodyBytes) + `"}` + oversized := `{"actor":"` + strings.Repeat("x", maxJSONBodyBytes) + `"}` r := httptest.NewRequest(http.MethodPost, claimPath, strings.NewReader(oversized)) - members, res := decodeClaimBody(httptest.NewRecorder(), r) + members, res := decodeJSONObjectBody(httptest.NewRecorder(), r) if res == nil { t.Fatalf("a %d-byte body was accepted (%d members)", len(oversized), len(members)) } diff --git a/internal/httpapi/problem.go b/internal/httpapi/problem.go index e885ba2b2..2523779fa 100644 --- a/internal/httpapi/problem.go +++ b/internal/httpapi/problem.go @@ -161,6 +161,9 @@ const ( OpListDependencies = "listDependencies" OpCountReadyWork = "countReadyWork" OpQueryIssues = "queryIssues" + // OpSweepIssues is the one DESTRUCTIVE operation on this surface: bulk + // clearance of closed beads from one tier, behind issueops.Sweeper. + OpSweepIssues = "sweepIssues" ) // operationCodes is the per-operation problem vocabulary: exactly the codes @@ -220,6 +223,13 @@ var operationCodes = map[string][]Code{ // recovery is the same as for any other malformed parameter value, and a // code it cannot act on differently is a code it should not have to know. OpQueryIssues: {CodeInvalidArgument, CodeBusy, CodeDBUnavailable, CodeInternal}, + // The 400 here is the widest on this surface, and most of it is not this + // handler's: an unknown or malformed body member is, but the unfiltered + // durable sweep, the unrecognized tier and the malformed glob are all the + // ROLE's ErrValidation reaching the wire through failSweepErr. No 404 — + // this operation names no id — and no 409: a sweep has nothing to conflict + // with, since a bead another sweep already took is simply not in the set. + OpSweepIssues: {CodeInvalidArgument, CodeBusy, CodeDBUnavailable, CodeInternal}, OpClaimIssue: { CodeInvalidArgument, CodeNotFound, CodeAlreadyClaimed, CodeNotClaimable, CodeBusy, CodeDBUnavailable, CodeInternal, diff --git a/internal/httpapi/roles_test.go b/internal/httpapi/roles_test.go index 26fbc2253..f9ec63515 100644 --- a/internal/httpapi/roles_test.go +++ b/internal/httpapi/roles_test.go @@ -313,6 +313,9 @@ func rolesConfig(cfg Config) Config { if cfg.Querier == nil { cfg.Querier = &roleQuerier{} } + if cfg.Sweeper == nil { + cfg.Sweeper = &roleSweeper{} + } return cfg } @@ -347,6 +350,36 @@ func (d *roleCycleDetector) DetectCycles(_ context.Context, req issueops.DetectC return d.report, nil } +// roleSweeper is the store-shaped source's bulk-clearance role. Like the +// others it answers with whatever a case configured, so every configured-roles +// case can hand Listen a COMPLETE source without deciding what a sweep of its +// fixture should do — which matters more here than for the reads, because the +// alternative to a complete source is a server that binds and then +// nil-dereferences on the one request that deletes beads. +type roleSweeper struct { + result issueops.SweepResult + err error + + mu sync.Mutex + calls []issueops.SweepRequest +} + +func (s *roleSweeper) Sweep(_ context.Context, req issueops.SweepRequest) (issueops.SweepResult, error) { + s.mu.Lock() + s.calls = append(s.calls, req) + s.mu.Unlock() + if s.err != nil { + return issueops.SweepResult{}, s.err + } + return s.result, nil +} + +func (s *roleSweeper) requests() []issueops.SweepRequest { + s.mu.Lock() + defer s.mu.Unlock() + return append([]issueops.SweepRequest(nil), s.calls...) +} + // countedPage is the fixture both database sources answer with, so a body // difference between them can only come from the construction. func countedPage() []*types.IssueWithCounts { @@ -441,6 +474,25 @@ func TestListenRequiresExactlyOneDatabaseSource(t *testing.T) { cfg: rolesConfigWithout(func(c *Config) { c.Querier = nil }), wantErr: "no database source", }, + { + // The newest role, and the one whose absence would be worst: a + // Config written against the previous release binds, advertises + // issues.sweep in its own capability list, and nil-dereferences on + // the first request that deletes beads. + name: "no sweeper", + cfg: rolesConfigWithout(func(c *Config) { c.Sweeper = nil }), + wantErr: "no database source", + }, + { + name: "a sweeper alone", + cfg: Config{Sweeper: &roleSweeper{}}, + wantErr: "no database source", + }, + { + name: "a provider and a sweeper", + cfg: Config{Provider: &fakeProvider{}, Sweeper: &roleSweeper{}}, + wantErr: "exactly one database source", + }, { name: "a cycle detector alone", cfg: Config{CycleDetector: &roleCycleDetector{}}, @@ -822,16 +874,7 @@ func TestConfiguredRolesAnswerEveryDatabaseRoute(t *testing.T) { // package. func TestConfiguredRolesKeepTheDocumentedRefusals(t *testing.T) { t.Run("a missing issue is 404", func(t *testing.T) { - ts := newTestServer(t, Config{ - Reader: &roleReader{err: fmt.Errorf("get bd-404: %w", storage.ErrNotFound)}, - Claimer: &roleClaimer{}, - Settings: &roleSettings{}, - Stats: &roleStats{}, - CycleDetector: &roleCycleDetector{}, - EdgeReader: &roleEdgeReader{}, - ReadyCounter: &roleReadyCounter{}, - Querier: &roleQuerier{}, - }) + ts := newTestServer(t, rolesConfig(Config{Reader: &roleReader{err: fmt.Errorf("get bd-404: %w", storage.ErrNotFound)}})) resp := ts.get(t, "/v0/beads/issues/bd-404") if resp.StatusCode != http.StatusNotFound { t.Fatalf("status = %d, want 404: %s", resp.StatusCode, readAll(t, resp)) @@ -845,16 +888,7 @@ func TestConfiguredRolesKeepTheDocumentedRefusals(t *testing.T) { // The builders run INSIDE the role on this path, so their refusal // arrives at the handler exactly as it does from a unit-of-work reader — // and must still be mapped to its parameter rather than to a 500. - ts := newTestServer(t, Config{ - Reader: &roleReader{err: errors.New("invalid status bogus")}, - Claimer: &roleClaimer{}, - Settings: &roleSettings{}, - Stats: &roleStats{}, - CycleDetector: &roleCycleDetector{}, - EdgeReader: &roleEdgeReader{}, - ReadyCounter: &roleReadyCounter{}, - Querier: &roleQuerier{}, - }) + ts := newTestServer(t, rolesConfig(Config{Reader: &roleReader{err: errors.New("invalid status bogus")}})) resp := ts.get(t, "/v0/beads/issues?status=bogus") if resp.StatusCode != http.StatusBadRequest { t.Fatalf("status = %d, want 400: %s", resp.StatusCode, readAll(t, resp)) @@ -866,21 +900,14 @@ func TestConfiguredRolesKeepTheDocumentedRefusals(t *testing.T) { }) t.Run("a foreign holder is 409 with its state", func(t *testing.T) { - ts := newTestServer(t, Config{ - Reader: &roleReader{}, - Stats: &roleStats{}, - CycleDetector: &roleCycleDetector{}, - EdgeReader: &roleEdgeReader{}, + ts := newTestServer(t, rolesConfig(Config{ Claimer: &roleClaimer{err: &issueops.ClaimConflictError{ IssueID: "bd-1", Assignee: "bob", Status: types.StatusInProgress, Err: fmt.Errorf("claim bd-1: %w", storage.ErrAlreadyClaimed), }}, - Settings: &roleSettings{}, - ReadyCounter: &roleReadyCounter{}, - Querier: &roleQuerier{}, - }) + })) resp := ts.claim(t, claimPath, `{"actor":"alice"}`) if resp.StatusCode != http.StatusConflict { t.Fatalf("status = %d, want 409: %s", resp.StatusCode, readAll(t, resp)) @@ -895,16 +922,7 @@ func TestConfiguredRolesKeepTheDocumentedRefusals(t *testing.T) { }) t.Run("a role failure is the generic 500", func(t *testing.T) { - ts := newTestServer(t, Config{ - Reader: &roleReader{err: errors.New("backend is unreachable")}, - Claimer: &roleClaimer{}, - Settings: &roleSettings{}, - Stats: &roleStats{}, - CycleDetector: &roleCycleDetector{}, - EdgeReader: &roleEdgeReader{}, - ReadyCounter: &roleReadyCounter{}, - Querier: &roleQuerier{}, - }) + ts := newTestServer(t, rolesConfig(Config{Reader: &roleReader{err: errors.New("backend is unreachable")}})) resp := ts.get(t, "/v0/beads/ready") if resp.StatusCode != http.StatusInternalServerError { t.Fatalf("status = %d, want 500: %s", resp.StatusCode, readAll(t, resp)) @@ -1001,16 +1019,7 @@ func TestARoleThatAnswersWithNothingIsNotDereferenced(t *testing.T) { // tell a broken role from an absent issue. t.Run("a reader with no detail view is the documented miss", func(t *testing.T) { silent := newTestServer(t, rolesConfig(Config{})) - missed := newTestServer(t, Config{ - Reader: &roleReader{err: fmt.Errorf("get bd-1: %w", storage.ErrNotFound)}, - Claimer: &roleClaimer{}, - Settings: &roleSettings{}, - Stats: &roleStats{}, - CycleDetector: &roleCycleDetector{}, - EdgeReader: &roleEdgeReader{}, - ReadyCounter: &roleReadyCounter{}, - Querier: &roleQuerier{}, - }) + missed := newTestServer(t, rolesConfig(Config{Reader: &roleReader{err: fmt.Errorf("get bd-1: %w", storage.ErrNotFound)}})) got := silent.get(t, "/v0/beads/issues/bd-1") want := missed.get(t, "/v0/beads/issues/bd-1") @@ -1033,16 +1042,9 @@ func TestARoleThatAnswersWithNothingIsNotDereferenced(t *testing.T) { // fault reaches the log as a stack trace instead of as an error, and the // panic path writes no request_error line for an operator to alert on. t.Run("a claimer with no issue is the generic failure", func(t *testing.T) { - ts := newTestServer(t, Config{ - Reader: &roleReader{}, - Claimer: &roleClaimer{result: issueops.ClaimResult{Changed: true}}, - Settings: &roleSettings{}, - Stats: &roleStats{}, - CycleDetector: &roleCycleDetector{}, - EdgeReader: &roleEdgeReader{}, - ReadyCounter: &roleReadyCounter{}, - Querier: &roleQuerier{}, - }) + ts := newTestServer(t, rolesConfig(Config{ + Claimer: &roleClaimer{result: issueops.ClaimResult{Changed: true}}, + })) resp := ts.claim(t, claimPath, `{"actor":"alice"}`) if resp.StatusCode != http.StatusInternalServerError { diff --git a/internal/httpapi/routes.go b/internal/httpapi/routes.go index 2426b2556..711f6e742 100644 --- a/internal/httpapi/routes.go +++ b/internal/httpapi/routes.go @@ -181,6 +181,25 @@ var routeTable = []route{ implemented: true, handler: (*Server).handleClaim, }, + { + op: OpSweepIssues, + method: http.MethodPost, + // A collection-level custom method, spelled the way countReadyWork's + // is and needing no specPath declaration for the same reason: the + // segment is a LITERAL. Only a WILDCARD segment is inexpressible as a + // ServeMux pattern, which is what makes the claim route the one + // exception. + // + // It is registered AFTER the claim's `/v0/beads/issues/{idop}`, and + // ServeMux precedence is by specificity rather than by order, so the + // literal wins over the wildcard for this exact path. That is what + // keeps a sweep from being parsed as a claim of an issue called + // ":sweep"; TestSweepPathReachesItsHandler drives the documented path. + pattern: "/v0/beads/issues:sweep", + capability: "issues.sweep", + implemented: true, + handler: (*Server).handleSweep, + }, } // specPathOf is the document path for a route, defaulting to the router diff --git a/internal/httpapi/server.go b/internal/httpapi/server.go index 897109fda..1f9c6c86e 100644 --- a/internal/httpapi/server.go +++ b/internal/httpapi/server.go @@ -207,6 +207,12 @@ type Config struct { EdgeReader issueops.EdgeReader ReadyCounter issueops.ReadyCounter Querier issueops.Querier + // Sweeper is the DESTRUCTIVE one, and it is required on the same terms as + // every other role rather than opt-in. A Config that could omit it would + // be a Config that quietly decides whether this build erases beads, which + // is a decision that belongs to the operator who chose to run bd serve at + // all — not to whether a caller remembered a field. + Sweeper issueops.Sweeper // Workspace is the startup snapshot GET /v0/beads/context answers from. // Only the allowlisted fields are ever serialized — see contextResponse, // which names the whole set and the reasons for the exclusions. @@ -250,6 +256,7 @@ type Server struct { issueEdges issueops.EdgeReader issueReadyCounter issueops.ReadyCounter issueQuerier issueops.Querier + issueSweeper issueops.Sweeper listener net.Listener http *http.Server @@ -354,6 +361,7 @@ func Listen(cfg Config) (*Server, error) { issueEdges: cfg.EdgeReader, issueReadyCounter: cfg.ReadyCounter, issueQuerier: cfg.Querier, + issueSweeper: cfg.Sweeper, sem: make(chan struct{}, maxInflight), semTimeout: semAcquireTimeout, @@ -457,12 +465,12 @@ func Listen(cfg Config) (*Server, error) { // as this check is concerned, exactly as it was when the check was a hand- // written boolean per role. func sourceRoles(cfg Config) []any { - return []any{cfg.Reader, cfg.Claimer, cfg.Settings, cfg.Stats, cfg.CycleDetector, cfg.EdgeReader, cfg.ReadyCounter, cfg.Querier} + return []any{cfg.Reader, cfg.Claimer, cfg.Settings, cfg.Stats, cfg.CycleDetector, cfg.EdgeReader, cfg.ReadyCounter, cfg.Querier, cfg.Sweeper} } // roleSourceNames spells sourceRoles for the refusal message, in the same // order, so a caller reading the error learns the whole set it must pass. -const roleSourceNames = "Reader, Claimer, Settings, Stats, CycleDetector, EdgeReader, ReadyCounter and Querier" +const roleSourceNames = "Reader, Claimer, Settings, Stats, CycleDetector, EdgeReader, ReadyCounter, Querier and Sweeper" func anyRoleSet(cfg Config) bool { return slices.ContainsFunc(sourceRoles(cfg), func(r any) bool { return r != nil }) @@ -720,6 +728,25 @@ func (s *Server) querier(r *http.Request) (issueops.Querier, error) { return src.Querier() } +// sweeper returns the guarded bulk-clearance surface for one request. +// +// The same two sources as every role above, for the same reasons, and held by +// INTERFACE so uow.SweeperSource is load-bearing rather than decorative. It +// goes out unwrapped for the reason readyCounter does: SweepResult is a VALUE, +// so there is no pointer for a caller-supplied role to hand back nil in. +// +// The role this returns is the ONLY thing standing between a POST body and a +// mass delete — the require-a-filter gate, the pinned protection and the +// closed_at recheck are all inside it — which is why the Config field it comes +// from is required rather than optional. +func (s *Server) sweeper(r *http.Request) (issueops.Sweeper, error) { + if s.provider == nil { + return s.issueSweeper, nil + } + var src uow.SweeperSource = timedProvider{inner: s.provider, rec: requestInfo(r.Context())} + return src.Sweeper() +} + // WithUOW runs fn inside one unit of work and guarantees the rollback. // // The close context is DETACHED on purpose. Close sends ROLLBACK on the pinned diff --git a/internal/httpapi/server_test.go b/internal/httpapi/server_test.go index 31acac5b8..1bdfdafd5 100644 --- a/internal/httpapi/server_test.go +++ b/internal/httpapi/server_test.go @@ -578,8 +578,8 @@ func TestCapabilitiesAdvertiseEveryImplementedOperation(t *testing.T) { } want := []string{ "config.get", "config.list", "dependencies.cycles", "dependencies.list", - "issues.claim", "issues.get", "issues.list", "issues.query", "ready.count", - "ready.list", "stats.get", + "issues.claim", "issues.get", "issues.list", "issues.query", + "issues.sweep", "ready.count", "ready.list", "stats.get", } if !slices.Equal(got, want) { t.Errorf("capabilities = %v, want %v", got, want) diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index 6066e94b8..532ab9799 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -57,12 +57,12 @@ info: Operations with no parameters at all (`GET /healthz`, `GET /v0/beads/context`, `GET /v0/beads/dependencies/cycles`, `GET /v0/beads/issues/{id}`, `POST /v0/beads/issues/{id}:claim`, - `GET /v0/beads/config`, `GET /v0/beads/config/{key}`) reject every query - key the same way. + `POST /v0/beads/issues:sweep`, `GET /v0/beads/config`, + `GET /v0/beads/config/{key}`) reject every query key the same way. Both are uniform rules, reachable on EVERY route including `GET /healthz`, - and are stated here instead of being repeated on all twelve operations. The + and are stated here instead of being repeated on every operation. The per-operation `responses` below therefore list what an operation produces BEYOND these two: a generated client must treat `400 invalid_argument` as possible everywhere, including on the operations whose documented statuses @@ -1145,6 +1145,91 @@ paths: '503': $ref: '#/components/responses/Unavailable' + /v0/beads/issues:sweep: + post: + operationId: sweepIssues + summary: Delete closed beads in bulk + description: >- + Bulk clearance of CLOSED beads from ONE tier: the operation behind + `bd purge` (`tier: ephemeral`) and `bd prune` (`tier: durable`). It is + the only DESTRUCTIVE operation on this surface, and nothing it deletes + comes back. + + + A collection-level custom method rather than `DELETE + /v0/beads/issues?...`: this describes a SET and acts on it, and a + `DELETE` with a filtering query string is the shape where a dropped + parameter widens what is erased. Sending a body makes every narrowing + term a member the server refuses by name if it does not know it. + + + ## What it selects, in order + + + The closed rows of `tier`, then `pattern`, then the two protections — + pinned beads are never swept and `protect_referenced` holds back beads + another live bead cites. The ORDER is part of the answer: `skipped` + counts what the request actually reached, so a pinned bead the pattern + excluded is not counted as protected. + + + ## The safety gate is the ROLE'S, not this handler's + + + A `durable` sweep with neither `closed_before` nor `pattern` is + refused with `400` / `invalid_argument`. That refusal comes from the + same library surface `bd prune` calls, not from a check written here, + which is what makes this endpoint incapable of erasing every closed + bead in a workspace by omission. A caller that really means everything + closed sends `pattern: "*"`. + + + ## One transaction + + + The selection and the deletion share one transaction, so the set the + response describes IS the set that was deleted. The cost is that a + sweep is all-or-nothing: one large enough to exceed the backend's write + timeout fails whole and deletes nothing. Narrow the request rather than + expecting progress. + + + `dry_run: true` answers the same question and changes nothing — + including history. Ask it first. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/SweepRequest' + responses: + '200': + description: >- + The sweep ran (or, under `dry_run`, was costed). A request that + matched nothing is a 200 with `swept: 0`, not a 404 — an empty set + of closed beads is the steady state of a swept workspace. + content: + application/json: + schema: + $ref: '#/components/schemas/SweepResult' + '400': + description: >- + Invalid request: an unparseable or oversized body, an unknown body + member, a `tier` outside the enum, a malformed `pattern`, an + unparseable `closed_before`, an `actor` that is empty after + trimming or carries control characters — or an unfiltered + `durable` sweep, which the library surface refuses as a safety + invariant. + x-bd-codes: [invalid_argument] + content: + application/problem+json: + schema: + $ref: '#/components/schemas/Problem' + '500': + $ref: '#/components/responses/InternalError' + '503': + $ref: '#/components/responses/Unavailable' + /v0/beads/config: get: operationId: listSettings @@ -2175,6 +2260,7 @@ components: The operations this server actually implements, derived from its route table. v0's vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.get`, `issues.claim`, + `issues.sweep`, `stats.get`, `config.list`, `config.get`, `dependencies.cycles`, `dependencies.list`; it grows additively, and an operation never appears here unless it is fully implemented. This is how a client @@ -2209,6 +2295,172 @@ components: pinning would weld this body to a CLI presentation type instead of preventing a drift. + SweepRequest: + type: object + additionalProperties: false + required: [tier] + description: >- + Which closed beads to clear. The predicate is FIXED at "closed beads of + one tier" and the two narrowing members only narrow it: there is no + status, no assignee, no label and no free-text query here, because + every one of those would be another way to spell a destructive + selection that a caller could get subtly wrong. + + + `additionalProperties: false`, so an unknown member is a `400` naming + the member — the same posture the query-parameter rule takes, and for + the same reason: on this operation a silently ignored narrowing term + widens what is erased. + properties: + tier: + type: string + enum: [ephemeral, durable] + description: >- + Which plane to clear. `ephemeral` is the wisp tier (`bd purge`) and + `durable` is the issue tier (`bd prune`). The two are DISJOINT: a + sweep of one can never touch a bead of the other. Required, with no + default — a caller handed the wrong tier has nothing to notice + until the beads are gone. + actor: + type: string + minLength: 1 + maxLength: 256 + pattern: '^[^\u0000-\u001F\u007F]+$' + description: >- + Caller-asserted attribution for wherever the backend records it, + under the same rules and for the same reasons as `ClaimRequest`'s + `actor`: trimmed, refused when empty after trimming, over 256 + BYTES, or carrying any control character. Optional — a deleted bead + leaves no row to attribute the deletion on. + closed_before: + type: string + format: date-time + description: >- + Keep only beads closed STRICTLY BEFORE this instant (RFC 3339). A + bead closed exactly at it is kept, which is the half-open interval + every other time bound on this surface uses. `bd prune + --older-than 30d` resolves the duration itself and sends the + resulting instant. + pattern: + type: string + description: >- + Keep only beads whose id matches this shell glob (`*`, `?`, + `[...]`, `\` escapes; `*` also crosses `-` and `.`, since an id is + not a path). Absent matches every bead in the tier. A MALFORMED + glob is a `400`, never a pattern that matches nothing. + protect_referenced: + type: boolean + default: false + description: >- + Skip candidates whose id is CITED — as a literal, at word + boundaries — in the description, notes or comments of any bead that + is not done, so a decision trail a live bead still points at is not + deleted out from under it. It costs a full scan of the not-done set + and its comments, which is why it is asked for rather than always + on. `bd prune` asks unless `--ignore-references`; `bd purge` never + does. + dry_run: + type: boolean + default: false + description: >- + Report what the sweep WOULD do and delete nothing. The counts, the + skips and the refusals are the same ones the real sweep would + produce, computed against the same snapshot — and nothing is + recorded in history either. + + SweepResult: + type: object + required: [dry_run, swept, dependencies, labels, events, skipped] + description: >- + What one sweep did. Every number describes the SAME snapshot, because + the selection and the deletion ran in one transaction. + + + It is NOT `x-go-type`-pinned, for the reason `ReadyCount` is not: there + is no canonical Go struct whose JSON encoding is this contract. The CLI + publishes these numbers under its own per-command keys (`purged_count`, + `pruned_count`), which are a stdout presentation rather than a wire + type, so pinning would weld this body to one of them. + properties: + dry_run: + type: boolean + description: >- + Echoes the request, so a result carries whether its numbers + describe beads that are gone or beads that would go. + swept: + type: integer + description: How many beads were deleted, or under `dry_run` would be. + dependencies: + type: integer + description: >- + Dependency edge rows removed with them. Reported because a sweep's + visible effect is much larger than its bead count. + labels: + type: integer + description: Label rows removed with the swept beads. + events: + type: integer + description: Event rows removed with the swept beads. + skipped: + $ref: '#/components/schemas/SweepSkips' + referenced_ids: + type: array + description: >- + A BOUNDED SAMPLE of the ids `skipped.referenced` counts — at most + 100, in the order the candidate query returned them. It is a + sample, not the set: compare its length against 100 to tell a + truncated one from a complete one. Absent when nothing was + protected. + items: + type: string + + SweepSkips: + type: object + required: + - pinned + - referenced + - not_closed + - unknown_closed_at + - closed_at_or_after_cutoff + - unreadable + description: >- + The candidates a sweep declined to delete, bucketed by WHY. They are + separate counters rather than one number because they mean different + things: the first two are PROTECTIONS, and the last four are the sweep + declining to trust its own input. + properties: + pinned: + type: integer + description: >- + Candidates protected by the pinned flag. No request member + overrides it — a caller who wants a pinned bead gone unpins it + first. + referenced: + type: integer + description: >- + Candidates protected by `protect_referenced`. Always 0 when that + member is false or absent, so a 0 read without having asked says + nothing about whether beads are cited. + not_closed: + type: integer + description: >- + Candidates the tier query returned that the recheck found were not + closed. A NON-ZERO VALUE HERE IS A DEFENSE FIRING, not a normal + outcome: the query asked for exactly the beads this excludes. + unknown_closed_at: + type: integer + description: Closed candidates carrying no close timestamp. See `not_closed`. + closed_at_or_after_cutoff: + type: integer + description: >- + Candidates whose close timestamp did not satisfy `closed_before`. + See `not_closed`. + unreadable: + type: integer + description: >- + Rows the tier query returned as nothing at all. A defense of the + same kind, on a shape rather than a value. + ReadyPage: type: object required: [items, has_more] diff --git a/internal/httpapi/sweep.go b/internal/httpapi/sweep.go new file mode 100644 index 000000000..e1bfaaae5 --- /dev/null +++ b/internal/httpapi/sweep.go @@ -0,0 +1,264 @@ +package httpapi + +import ( + "encoding/json" + "errors" + "net/http" + "slices" + "strings" + "time" + + "github.com/steveyegge/beads/internal/httpapi/apigen" + "github.com/steveyegge/beads/issueops" +) + +// The request body's member vocabulary. The schema is +// additionalProperties: false, so anything else is refused BY NAME — the same +// posture the unknown-query-parameter rule takes, and on this operation for a +// sharper reason: a narrowing term the server silently ignored would widen +// what is erased. +const ( + sweepTierMember = "tier" + sweepActorMember = "actor" + sweepClosedBeforeMember = "closed_before" + sweepPatternMember = "pattern" + sweepProtectReferencedMember = "protect_referenced" + sweepDryRunMember = "dry_run" +) + +// sweepMembers is the whole vocabulary, in one place, so the unknown-member +// refusal and the decoding below cannot come to disagree about what this +// operation accepts. +var sweepMembers = []string{ + sweepTierMember, + sweepActorMember, + sweepClosedBeforeMember, + sweepPatternMember, + sweepProtectReferencedMember, + sweepDryRunMember, +} + +// handleSweep answers POST /v0/beads/issues:sweep — the one DESTRUCTIVE +// operation on this surface. +// +// WHAT THIS HANDLER DOES NOT DO is the whole point of it. It does not decide +// which beads are closed, does not match the glob, does not recheck closed_at, +// does not protect pinned beads, and — the one that matters most — does not +// implement the require-a-filter safety gate. All of that is +// issueops.Sweeper, the same library surface `bd prune` calls, so this +// endpoint could not erase every closed bead in a workspace by omission even +// if a future edit here forgot the rule existed. That is the reason the gate +// was moved INTO the role rather than left in the CLI handler that historically +// owned it: without it, a second front door is one handler away from an +// unguarded mass delete. +// +// Everything above the role here is argument validation: the media type, the +// body shape, and the six members the document publishes. +// +// NO ACTOR IS INFERRED, for the reason the claim gives: the server's own +// identity is meaningless to a remote caller. Unlike the claim, the actor is +// OPTIONAL here — a deleted bead leaves no row to attribute the deletion on — +// and it is validated by the same rules when present, because it reaches the +// same commit-message interpolation. +func (s *Server) handleSweep(w http.ResponseWriter, r *http.Request) { + if !s.requireNoQuery(w, r) { + return + } + if !s.requireJSONContent(w, r) { + return + } + request, ok := s.sweepRequest(w, r) + if !ok { + return + } + + sweeper, err := s.sweeper(r) + if err != nil { + s.failErr(w, r, err) + return + } + result, err := sweeper.Sweep(r.Context(), request) + if err != nil { + s.failSweepErr(w, r, err) + return + } + writeJSON(w, sweepResponse(result)) +} + +// sweepRequest decodes the body into the role's request, member by member. +// +// Member by member rather than straight into apigen.SweepRequest so that every +// refusal can NAME the member it is about: unmarshaling the generated struct +// reports a type mismatch only inside an error string, and this endpoint +// exists so clients can stop parsing prose. +func (s *Server) sweepRequest(w http.ResponseWriter, r *http.Request) (issueops.SweepRequest, bool) { + members, res := decodeJSONObjectBody(w, r) + if res != nil { + s.fail(w, r, *res) + return issueops.SweepRequest{}, false + } + + var unknown []string + for name := range members { + if !slices.Contains(sweepMembers, name) { + unknown = append(unknown, name) + } + } + if len(unknown) > 0 { + // One offender, chosen deterministically so a client dispatching on + // `param` never sees it depend on map order. + offender := slices.Min(unknown) + requestInfo(r.Context()).refuse(offender) + s.fail(w, r, InvalidArgument(offender, ReasonUnknownParameter, + "this operation's request body carries "+sweepMemberList()+" and nothing else")) + return issueops.SweepRequest{}, false + } + + var request issueops.SweepRequest + + raw, ok := members[sweepTierMember] + if !ok { + s.fail(w, r, InvalidArgument(sweepTierMember, ReasonInvalidValue, + "`"+sweepTierMember+"` is required and has no default")) + return issueops.SweepRequest{}, false + } + var tier *string + if err := json.Unmarshal(raw, &tier); err != nil || tier == nil { + s.fail(w, r, InvalidArgument(sweepTierMember, ReasonInvalidValue, + "`"+sweepTierMember+"` must be a string")) + return issueops.SweepRequest{}, false + } + // The enum check uses the GENERATED validator, which is derived from the + // document rather than a second hand-written copy of the vocabulary. The + // role refuses an unrecognized tier too — this is here so the refusal can + // name the member, not because the role's is in doubt. + if !apigen.SweepRequestTier(*tier).Valid() { + s.fail(w, r, InvalidArgument(sweepTierMember, ReasonInvalidValue, + "`"+sweepTierMember+"` must be \"ephemeral\" or \"durable\"")) + return issueops.SweepRequest{}, false + } + request.Tier = issueops.SweepTier(*tier) + + if raw, ok := members[sweepActorMember]; ok { + var value *string + if err := json.Unmarshal(raw, &value); err != nil || value == nil { + s.fail(w, r, InvalidArgument(sweepActorMember, ReasonInvalidValue, + "`"+sweepActorMember+"` must be a string")) + return issueops.SweepRequest{}, false + } + // The claim's rules, unchanged: trim, then refuse empty, over-long and + // any control character. `param` reads "actor" there too, which is what + // makes the two operations one vocabulary for a client. + trimmed, res := validateActor(*value) + if res != nil { + s.fail(w, r, *res) + return issueops.SweepRequest{}, false + } + request.Actor = trimmed + } + + if raw, ok := members[sweepClosedBeforeMember]; ok { + var value *time.Time + if err := json.Unmarshal(raw, &value); err != nil || value == nil { + s.fail(w, r, InvalidArgument(sweepClosedBeforeMember, ReasonInvalidValue, + "`"+sweepClosedBeforeMember+"` must be an RFC 3339 timestamp")) + return issueops.SweepRequest{}, false + } + request.ClosedBefore = value + } + + if raw, ok := members[sweepPatternMember]; ok { + var value *string + if err := json.Unmarshal(raw, &value); err != nil || value == nil { + s.fail(w, r, InvalidArgument(sweepPatternMember, ReasonInvalidValue, + "`"+sweepPatternMember+"` must be a string")) + return issueops.SweepRequest{}, false + } + // A malformed glob is NOT refused here. The role refuses it, and + // routing it through the role is what keeps one definition of what a + // pattern is: filepath.Match's, matched in Go on both front doors. + request.IDPattern = *value + } + + for _, flag := range []struct { + member string + dest *bool + }{ + {sweepProtectReferencedMember, &request.ProtectReferenced}, + {sweepDryRunMember, &request.DryRun}, + } { + raw, ok := members[flag.member] + if !ok { + continue + } + var value *bool + if err := json.Unmarshal(raw, &value); err != nil || value == nil { + s.fail(w, r, InvalidArgument(flag.member, ReasonInvalidValue, + "`"+flag.member+"` must be a boolean")) + return issueops.SweepRequest{}, false + } + *flag.dest = *value + } + + return request, true +} + +func sweepMemberList() string { + quoted := make([]string, len(sweepMembers)) + for i, name := range sweepMembers { + quoted[i] = "`" + name + "`" + } + return strings.Join(quoted, ", ") +} + +// failSweepErr answers a failed sweep. +// +// issueops.ErrValidation is a 400 HERE and nowhere else yet, and that is +// deliberate rather than an oversight in ClassifyError: this is the first +// operation whose ROLE performs request validation the handler does not +// duplicate — the require-a-filter gate, the tier vocabulary and the glob are +// all refused below the wire — so it is the first place a role's refusal must +// reach the client as its own fault instead of as a 500. Widening +// ClassifyError would change what every other operation returns for an error +// it has never produced, which is a bigger claim than this commit is making. +func (s *Server) failSweepErr(w http.ResponseWriter, r *http.Request, err error) { + if errors.Is(err, issueops.ErrValidation) { + // No `param`: the refusal is about the REQUEST rather than one member + // of it — an unfiltered durable sweep is two absent members at once — + // and the document's `param` is documented absent on exactly that + // case. The detail carries the role's own sentence, which names what + // to send instead. + s.fail(w, r, InvalidArgument("", ReasonInvalidValue, err.Error())) + return + } + s.failErr(w, r, err) +} + +// sweepResponse projects the role's result onto the wire type. It is a field +// list rather than an alias because SweepResult is deliberately not +// x-go-type-pinned: there is no canonical Go struct whose JSON encoding is +// this body (see the schema's own description), so the projection is where the +// two shapes are held together and TestSweepResponseCarriesEveryRoleField is +// what keeps a new result field from being dropped here in silence. +func sweepResponse(result issueops.SweepResult) apigen.SweepResult { + body := apigen.SweepResult{ + DryRun: result.DryRun, + Swept: result.Swept, + Dependencies: result.Dependencies, + Labels: result.Labels, + Events: result.Events, + Skipped: apigen.SweepSkips{ + Pinned: result.Skipped.Pinned, + Referenced: result.Skipped.Referenced, + NotClosed: result.Skipped.NotClosed, + UnknownClosedAt: result.Skipped.UnknownClosedAt, + ClosedAtOrAfterCutoff: result.Skipped.ClosedAtOrAfterCutoff, + Unreadable: result.Skipped.Unreadable, + }, + } + if len(result.ReferencedIDs) > 0 { + ids := append([]string(nil), result.ReferencedIDs...) + body.ReferencedIds = &ids + } + return body +} diff --git a/internal/httpapi/sweep_test.go b/internal/httpapi/sweep_test.go new file mode 100644 index 000000000..cb64ff7bc --- /dev/null +++ b/internal/httpapi/sweep_test.go @@ -0,0 +1,313 @@ +package httpapi + +import ( + "net/http" + "reflect" + "strings" + "testing" + "time" + + "github.com/steveyegge/beads/issueops" +) + +// The pins for POST /v0/beads/issues:sweep. The role is the surface under +// scrutiny everywhere else in this package's storage tests; what is asserted +// here is the WIRE EDGE — that the handler decodes the document's six members +// into the role's request faithfully, refuses what the document refuses, and +// does not re-implement anything the role owns. + +const sweepPath = "/v0/beads/issues:sweep" + +func (ts *testServer) sweep(t *testing.T, body string) *http.Response { + t.Helper() + return ts.claimRequest(t, sweepPath, "application/json", body) +} + +// TestSweepPathReachesItsHandler is the routing assertion the claim row's twin +// makes: the sweep path is a LITERAL segment registered beside the claim's +// wildcard `/v0/beads/issues/{idop}`, and ServeMux precedence is by specificity +// rather than by registration order. A 404 or a claim-shaped refusal here would +// mean the sweep is being parsed as a claim of an issue named ":sweep". +func TestSweepPathReachesItsHandler(t *testing.T) { + sweeper := &roleSweeper{result: issueops.SweepResult{Swept: 3}} + ts := newTestServer(t, rolesConfig(Config{Sweeper: sweeper})) + + resp := ts.sweep(t, `{"tier":"ephemeral"}`) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, want 200: %s", resp.StatusCode, readAll(t, resp)) + } + if len(sweeper.requests()) != 1 { + t.Fatalf("the sweep role was called %d times, want 1 — the path reached another handler", + len(sweeper.requests())) + } +} + +// TestSweepForwardsEveryDocumentedMember is the operation's central pin: each +// of the six body members reaches the role's request unchanged. +// +// It is asserted on the REQUEST the role received rather than on the response, +// because a body carrying the right numbers says nothing about which set was +// swept — this is the operation where the handler dropping a narrowing member +// widens what is erased. +func TestSweepForwardsEveryDocumentedMember(t *testing.T) { + sweeper := &roleSweeper{} + ts := newTestServer(t, rolesConfig(Config{Sweeper: sweeper})) + + resp := ts.sweep(t, `{ + "tier": "durable", + "actor": " alice ", + "closed_before": "2026-03-01T12:00:00Z", + "pattern": "bd-old-*", + "protect_referenced": true, + "dry_run": true + }`) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, want 200: %s", resp.StatusCode, readAll(t, resp)) + } + + reqs := sweeper.requests() + if len(reqs) != 1 { + t.Fatalf("%d sweeps, want 1", len(reqs)) + } + got := reqs[0] + want := issueops.SweepRequest{ + Tier: issueops.SweepDurable, + // TRIMMED, by the same rule and the same function the claim's actor + // goes through: it reaches the same commit-message interpolation. + Actor: "alice", + IDPattern: "bd-old-*", + ProtectReferenced: true, + DryRun: true, + } + cutoff := time.Date(2026, 3, 1, 12, 0, 0, 0, time.UTC) + want.ClosedBefore = &cutoff + if got.ClosedBefore == nil || !got.ClosedBefore.Equal(cutoff) { + t.Errorf("ClosedBefore = %v, want %v", got.ClosedBefore, cutoff) + } + got.ClosedBefore, want.ClosedBefore = nil, nil + if !reflect.DeepEqual(got, want) { + t.Errorf("request = %+v, want %+v", got, want) + } +} + +// TestSweepDefaultsTheOptionalMembers: a body carrying only `tier` reaches the +// role as an unfiltered request of that tier, and NOT as a dry run. +// +// The dry-run default is the one worth pinning. `dry_run` absent must mean +// false — that is what the document says — and a handler that defaulted it the +// safe-looking way would make the endpoint silently incapable of deleting +// anything, which is a worse failure than it sounds: a caller would retry. +func TestSweepDefaultsTheOptionalMembers(t *testing.T) { + sweeper := &roleSweeper{} + ts := newTestServer(t, rolesConfig(Config{Sweeper: sweeper})) + + if resp := ts.sweep(t, `{"tier":"ephemeral"}`); resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, want 200: %s", resp.StatusCode, readAll(t, resp)) + } + reqs := sweeper.requests() + if len(reqs) != 1 { + t.Fatalf("%d sweeps, want 1", len(reqs)) + } + if want := (issueops.SweepRequest{Tier: issueops.SweepEphemeral}); reqs[0] != want { + t.Errorf("request = %+v, want %+v", reqs[0], want) + } +} + +// TestSweepPublishesTheWholeResult pins the projection, which exists because +// SweepResult is deliberately not x-go-type-pinned: there is no compile-time +// tie between the role's result and this body, so a field added to the role and +// forgotten here would vanish from the wire in silence. +func TestSweepPublishesTheWholeResult(t *testing.T) { + sweeper := &roleSweeper{result: issueops.SweepResult{ + DryRun: true, + Swept: 7, + Dependencies: 3, + Labels: 2, + Events: 11, + Skipped: issueops.SweepSkips{ + Pinned: 1, Referenced: 2, NotClosed: 3, + UnknownClosedAt: 4, ClosedAtOrAfterCutoff: 5, Unreadable: 6, + }, + ReferencedIDs: []string{"bd-1", "bd-2"}, + }} + ts := newTestServer(t, rolesConfig(Config{Sweeper: sweeper})) + + resp := ts.sweep(t, `{"tier":"durable","pattern":"*","dry_run":true}`) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, want 200: %s", resp.StatusCode, readAll(t, resp)) + } + body := decodeBody(t, resp) + + for _, want := range []struct { + key string + val any + }{ + {"dry_run", true}, + {"swept", float64(7)}, + {"dependencies", float64(3)}, + {"labels", float64(2)}, + {"events", float64(11)}, + } { + if body[want.key] != want.val { + t.Errorf("%s = %v, want %v", want.key, body[want.key], want.val) + } + } + + skipped, ok := body["skipped"].(map[string]any) + if !ok { + t.Fatalf("skipped = %v, want an object", body["skipped"]) + } + for key, want := range map[string]float64{ + "pinned": 1, "referenced": 2, "not_closed": 3, + "unknown_closed_at": 4, "closed_at_or_after_cutoff": 5, "unreadable": 6, + } { + if skipped[key] != want { + t.Errorf("skipped.%s = %v, want %v", key, skipped[key], want) + } + } + + ids, ok := body["referenced_ids"].([]any) + if !ok || len(ids) != 2 || ids[0] != "bd-1" { + t.Errorf("referenced_ids = %v, want the role's sample", body["referenced_ids"]) + } +} + +// TestSweepOmitsAnEmptyReferencedSample: the member is documented ABSENT when +// nothing was protected, so a client can tell "nothing cited" from "the +// protection was not asked for" without a second field. +func TestSweepOmitsAnEmptyReferencedSample(t *testing.T) { + ts := newTestServer(t, rolesConfig(Config{Sweeper: &roleSweeper{}})) + + resp := ts.sweep(t, `{"tier":"ephemeral"}`) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, want 200: %s", resp.StatusCode, readAll(t, resp)) + } + if _, present := decodeBody(t, resp)["referenced_ids"]; present { + t.Error("referenced_ids is present with nothing protected; the document says absent") + } +} + +// TestSweepRefusesTheDocumentedBodies walks the 400 table. Every case names the +// member it is about in `param`, which is the whole reason the handler decodes +// member by member instead of unmarshaling the generated struct. +func TestSweepRefusesTheDocumentedBodies(t *testing.T) { + for _, test := range []struct { + name string + body string + param string + }{ + {"no tier", `{"pattern":"*"}`, "tier"}, + {"tier is not a string", `{"tier":3}`, "tier"}, + {"tier is null", `{"tier":null}`, "tier"}, + {"tier outside the enum", `{"tier":"wisps"}`, "tier"}, + {"unknown member", `{"tier":"durable","scope":"all"}`, "scope"}, + {"actor is not a string", `{"tier":"ephemeral","actor":7}`, "actor"}, + {"actor is blank", `{"tier":"ephemeral","actor":" "}`, "actor"}, + {"closed_before is not a timestamp", `{"tier":"ephemeral","closed_before":"yesterday"}`, "closed_before"}, + {"pattern is not a string", `{"tier":"ephemeral","pattern":true}`, "pattern"}, + {"dry_run is not a boolean", `{"tier":"ephemeral","dry_run":"yes"}`, "dry_run"}, + {"protect_referenced is not a boolean", `{"tier":"ephemeral","protect_referenced":1}`, "protect_referenced"}, + } { + t.Run(test.name, func(t *testing.T) { + sweeper := &roleSweeper{} + ts := newTestServer(t, rolesConfig(Config{Sweeper: sweeper})) + + resp := ts.sweep(t, test.body) + if resp.StatusCode != http.StatusBadRequest { + t.Fatalf("status = %d, want 400: %s", resp.StatusCode, readAll(t, resp)) + } + body := decodeBody(t, resp) + if body["code"] != string(CodeInvalidArgument) { + t.Errorf("code = %v, want %s", body["code"], CodeInvalidArgument) + } + if body["param"] != test.param { + t.Errorf("param = %v, want %q", body["param"], test.param) + } + // A refused body must never reach the role: the sweep is + // destructive, and "refused after acting" is the one failure shape + // worth ruling out at every edge. + if got := len(sweeper.requests()); got != 0 { + t.Errorf("the role was called %d times for a refused body, want 0", got) + } + }) + } +} + +// TestSweepPublishesTheRolesRefusalAsA400 is the reason failSweepErr exists. +// +// The require-a-filter gate, the tier vocabulary and the glob are all refused +// BELOW the wire, by issueops.Sweeper — that is what makes this endpoint +// incapable of an unguarded mass delete. A role refusal reaching the client as +// a 500 would tell it the server was broken when the request was, and would +// hide the one sentence that says what to send instead. +func TestSweepPublishesTheRolesRefusalAsA400(t *testing.T) { + // The sentinel has to MATCH, not merely be mentioned in the text. + sweeper := &roleSweeper{err: wrapValidation("a durable sweep requires a closed-before cutoff or an id pattern")} + ts := newTestServer(t, rolesConfig(Config{Sweeper: sweeper})) + + // A body the HANDLER accepts and the ROLE refuses: the durable tier with no + // narrowing member is exactly that request, which is the point. + resp := ts.sweep(t, `{"tier":"durable"}`) + if resp.StatusCode != http.StatusBadRequest { + t.Fatalf("status = %d, want 400: %s", resp.StatusCode, readAll(t, resp)) + } + body := decodeBody(t, resp) + if body["code"] != string(CodeInvalidArgument) { + t.Errorf("code = %v, want %s", body["code"], CodeInvalidArgument) + } + if detail, _ := body["detail"].(string); !strings.Contains(detail, "closed-before cutoff") { + t.Errorf("detail = %q, want the role's own sentence: it is what says what to send instead", detail) + } + // No `param`: the refusal is about two absent members at once, and the + // document declares `param` absent on exactly that case. + if _, present := body["param"]; present { + t.Errorf("param = %v, want it absent for a whole-request refusal", body["param"]) + } +} + +// TestSweepRefusesAForeignMediaType: the CSRF control the claim documents +// applies here for a sharper reason. A JSON content type is not CORS-"simple", +// so a cross-origin sweep always triggers a preflight this server never +// approves; accepting text/plain would let an attacker's page drive a mass +// delete from any browser on the host. +func TestSweepRefusesAForeignMediaType(t *testing.T) { + sweeper := &roleSweeper{} + ts := newTestServer(t, rolesConfig(Config{Sweeper: sweeper})) + + resp := ts.claimRequest(t, sweepPath, "text/plain", `{"tier":"ephemeral"}`) + if resp.StatusCode != http.StatusBadRequest { + t.Fatalf("status = %d, want 400: %s", resp.StatusCode, readAll(t, resp)) + } + if body := decodeBody(t, resp); body["param"] != "Content-Type" { + t.Errorf("param = %v, want Content-Type", body["param"]) + } + if got := len(sweeper.requests()); got != 0 { + t.Errorf("the role was called %d times for a refused media type, want 0", got) + } +} + +// TestSweepPublishesNoQueryParameters: this operation's whole vocabulary is its +// body, so a query key is version skew rather than a bad value. +func TestSweepPublishesNoQueryParameters(t *testing.T) { + ts := newTestServer(t, rolesConfig(Config{Sweeper: &roleSweeper{}})) + + resp := ts.claimRequest(t, sweepPath+"?tier=durable", "application/json", `{"tier":"durable","pattern":"*"}`) + if resp.StatusCode != http.StatusBadRequest { + t.Fatalf("status = %d, want 400: %s", resp.StatusCode, readAll(t, resp)) + } + body := decodeBody(t, resp) + if body["param"] != "tier" || body["reason"] != string(ReasonUnknownParameter) { + t.Errorf("body = %v, want unknown_parameter on param tier", body) + } +} + +// wrapValidation builds an error that MATCHES issueops.ErrValidation while +// reading like the role's own refusal, which is what failSweepErr keys on. +func wrapValidation(detail string) error { + return &validationError{detail: detail} +} + +type validationError struct{ detail string } + +func (e *validationError) Error() string { return issueops.ErrValidation.Error() + ": " + e.detail } +func (e *validationError) Unwrap() error { return issueops.ErrValidation } diff --git a/internal/jira/tracker_test.go b/internal/jira/tracker_test.go index 4c3c9dbc8..827548829 100644 --- a/internal/jira/tracker_test.go +++ b/internal/jira/tracker_test.go @@ -682,6 +682,10 @@ func (s *configStore) ReadyCounter() (issueops.ReadyCounter, error) { func (s *configStore) Querier() (issueops.Querier, error) { return nil, &storage.ErrUnsupported{Op: "Querier", Backend: "jira-config-stub"} } + +func (s *configStore) Sweeper() (issueops.Sweeper, error) { + return nil, &storage.ErrUnsupported{Op: "Sweeper", Backend: "jira-config-stub"} +} func (s *configStore) SetConfig(_ context.Context, _, _ string) error { return nil } func (s *configStore) SetLocalMetadata(_ context.Context, _, _ string) error { return nil } func (s *configStore) GetLocalMetadata(_ context.Context, _ string) (string, error) { diff --git a/internal/storage/dolt/sweeper.go b/internal/storage/dolt/sweeper.go new file mode 100644 index 000000000..16e0e08e1 --- /dev/null +++ b/internal/storage/dolt/sweeper.go @@ -0,0 +1,101 @@ +package dolt + +import ( + "context" + "database/sql" + "fmt" + + "github.com/steveyegge/beads/internal/storage" + storeops "github.com/steveyegge/beads/internal/storage/issueops" + "github.com/steveyegge/beads/internal/storage/schema" + "github.com/steveyegge/beads/internal/workapi" + "github.com/steveyegge/beads/issueops" +) + +// Sweeper returns the guarded bulk-clearance surface for this store. +func (s *DoltStore) Sweeper() (issueops.Sweeper, error) { + if s == nil { + return nil, &storage.ErrUnsupported{Op: "Sweeper", Backend: "nil"} + } + return &sweeper{store: s}, nil +} + +// sweeper clears closed rows from one tier inside ONE transaction. +// +// There is no shared constructor package for this role, for the reason +// cycleDetector gives: the work is a candidate query, a recheck, an optional +// reference scan and a delete that must all see one snapshot, and a +// transaction is not reachable through storage.DoltStorage. The sharing +// happens one level down instead — this body and the embedded store's are a +// few lines each around issueops.SweepInTx — and two wrappers over one body is +// still ONE vote, which the conformance contract says out loud. +// +// A front door cannot construct this: the type is unexported and the accessor +// is the only door. +type sweeper struct{ store *DoltStore } + +var _ issueops.Sweeper = (*sweeper)(nil) + +// Sweep clears the request's tier. +// +// VALIDATION HAPPENS BEFORE THE TRANSACTION, so a refused request costs no +// database work — which is what makes issueops.Sweeper's "a refusal changes +// nothing" true of the connection as well as of the rows. +// +// A DRY RUN TAKES A READ TRANSACTION. It writes nothing by construction, so +// giving it a write transaction and an empty commit would make the preview +// look like a mutation to everything watching the store. +// +// THE VERSION-CONTROL ENTRY IS ONE PER SWEEP, recorded here rather than in the +// shared body because only this backend records one at all: the embedded store +// commits outside the SQL transaction and the unit-of-work provider labels its +// own. An ephemeral sweep touches only the wisp tables, which this plane +// ignores, so DOLT_COMMIT finds nothing to commit and the sweep records no +// entry — the "at most one" the role promises, not "exactly one". +func (s *sweeper) Sweep(ctx context.Context, req issueops.SweepRequest) (issueops.SweepResult, error) { + if err := workapi.ValidateSweepRequest(req); err != nil { + return issueops.SweepResult{}, err + } + + var result issueops.SweepResult + run := func(tx *sql.Tx) error { + var err error + result, err = storeops.SweepInTx(ctx, tx, req) + return err + } + if req.DryRun { + if err := s.store.withReadTx(ctx, run); err != nil { + return issueops.SweepResult{}, err + } + return result, nil + } + + if err := s.store.withWriteTx(ctx, func(tx *sql.Tx) error { + if err := run(tx); err != nil { + return err + } + if result.Swept == 0 { + return nil + } + for _, table := range sweptTables { + _ = schema.DrainCall(ctx, tx, "CALL DOLT_ADD(?)", table) + } + msg := fmt.Sprintf("bd: sweep %d %s bead(s)", result.Swept, req.Tier) + if err := schema.DrainCall(ctx, tx, "CALL DOLT_COMMIT('-m', ?, '--author', ?)", + msg, s.store.commitAuthorString()); err != nil && !isDoltNothingToCommit(err) { + return fmt.Errorf("dolt commit: %w", err) + } + return nil + }); err != nil { + return issueops.SweepResult{}, err + } + return result, nil +} + +// sweptTables are the versioned tables a sweep can touch, staged before the +// commit. It is the same list DeleteIssues stages, because a sweep IS a +// delete of a selected set. +var sweptTables = []string{ + "issues", "dependencies", "labels", "comments", "events", + "child_counters", "issue_snapshots", "compaction_snapshots", +} diff --git a/internal/storage/dolt/sweeper_contract_test.go b/internal/storage/dolt/sweeper_contract_test.go new file mode 100644 index 000000000..a4f28418e --- /dev/null +++ b/internal/storage/dolt/sweeper_contract_test.go @@ -0,0 +1,82 @@ +package dolt + +import ( + "context" + "testing" + + "github.com/steveyegge/beads/backend/conformance" +) + +// TestSweeperContract runs the Sweeper contract against the server-backed +// store, which reaches internal/storage/issueops.SweepInTx through its own +// write transaction and is the ONE wiring that records a version-control entry +// for a sweep. +// +// The cases are subtests of one parent so the whole role suite shares one +// store and one copy-on-write branch. That sharing is what makes the id +// pattern in every case load-bearing rather than tidy: a sweep is asked of a +// whole TIER, so a case that swept without one would delete the next case's +// seeds. setupTestStore already marks the PARENT parallel; no subtest here +// calls t.Parallel, and RecordsAtMostOneHistoryEntry takes a before/after +// delta that is only meaningful while they run sequentially. +func TestSweeperContract(t *testing.T) { + fixture, ctx, cleanup := newDoltSweeperFixture(t, "swp") + defer cleanup() + + t.Run("RefusesAnUnfilteredDurableSweep", func(t *testing.T) { + conformance.RunSweeperRefusesAnUnfilteredDurableSweep(t, ctx, fixture) + }) + t.Run("RefusesAMalformedRequest", func(t *testing.T) { + conformance.RunSweeperRefusesAMalformedRequest(t, ctx, fixture) + }) + t.Run("ClearsOneTierAndLeavesTheOther", func(t *testing.T) { + conformance.RunSweeperClearsOneTierAndLeavesTheOther(t, ctx, fixture) + }) + t.Run("ProtectsPinnedRows", func(t *testing.T) { + conformance.RunSweeperProtectsPinnedRows(t, ctx, fixture) + }) + t.Run("HonorsTheCutoffAndThePattern", func(t *testing.T) { + conformance.RunSweeperHonorsTheCutoffAndThePattern(t, ctx, fixture) + }) + t.Run("DryRunChangesNothing", func(t *testing.T) { + conformance.RunSweeperDryRunChangesNothing(t, ctx, fixture) + }) + t.Run("ProtectsCitedRows", func(t *testing.T) { + conformance.RunSweeperProtectsCitedRows(t, ctx, fixture) + }) + t.Run("EmptyMatchIsZeroAndNil", func(t *testing.T) { + conformance.RunSweeperEmptyMatchIsZeroAndNil(t, ctx, fixture) + }) + t.Run("RecordsAtMostOneHistoryEntry", func(t *testing.T) { + conformance.RunSweeperRecordsAtMostOneHistoryEntry(t, ctx, fixture) + }) + t.Run("DoesNotMutateTheCallerRequest", func(t *testing.T) { + conformance.RunSweeperDoesNotMutateTheCallerRequest(t, ctx, fixture) + }) +} + +// newDoltSweeperFixture composes the frozen role kit with this backend's +// accessor. +func newDoltSweeperFixture(t *testing.T, prefix string) (conformance.SweeperFixture, context.Context, func()) { + t.Helper() + store, storeCleanup := setupTestStore(t) + ctx, cancel := testContext(t) + stop := func() { + cancel() + storeCleanup() + } + sweeper, err := store.Sweeper() + if err != nil { + stop() + t.Fatalf("Sweeper(): %v", err) + } + kit := newDoltRoleFixtureKit(store, prefix) + return conformance.SweeperFixture{ + IssuePrefix: kit.IssuePrefix, + Sweeper: sweeper, + CreateIssue: kit.CreateIssue, + CreateWisp: kit.CreateWisp, + QueryScalar: kit.QueryScalar, + CountHistory: kit.CountHistory, + }, ctx, stop +} diff --git a/internal/storage/embeddeddolt/sweeper.go b/internal/storage/embeddeddolt/sweeper.go new file mode 100644 index 000000000..ed7897cc2 --- /dev/null +++ b/internal/storage/embeddeddolt/sweeper.go @@ -0,0 +1,53 @@ +//go:build cgo + +package embeddeddolt + +import ( + "context" + "database/sql" + + "github.com/steveyegge/beads/internal/storage" + storeops "github.com/steveyegge/beads/internal/storage/issueops" + "github.com/steveyegge/beads/internal/workapi" + "github.com/steveyegge/beads/issueops" +) + +// Sweeper returns the guarded bulk-clearance surface for this store. +func (s *EmbeddedDoltStore) Sweeper() (issueops.Sweeper, error) { + if s == nil { + return nil, &storage.ErrUnsupported{Op: "Sweeper", Backend: "nil"} + } + return &sweeper{store: s}, nil +} + +// sweeper clears closed rows from one tier inside one connection's +// transaction. +// +// It is a sibling of the server-backed store's body rather than a shared +// package for the reason that one gives: the work needs a TRANSACTION, which +// storage.DoltStorage does not publish, so the sharing happens below both of +// them at issueops.SweepInTx. The two stores differ here only in how they +// reach a transaction and in whether they record a version-control entry — +// this one's commit runs outside the SQL transaction, so it records none. +type sweeper struct{ store *EmbeddedDoltStore } + +var _ issueops.Sweeper = (*sweeper)(nil) + +// Sweep clears the request's tier. Validation happens before the connection is +// opened, so a refused request costs no database work. +func (s *sweeper) Sweep(ctx context.Context, req issueops.SweepRequest) (issueops.SweepResult, error) { + if err := workapi.ValidateSweepRequest(req); err != nil { + return issueops.SweepResult{}, err + } + var result issueops.SweepResult + // commit = !DryRun: a preview writes nothing, so it must not take the + // committing path — the same distinction DeleteIssues draws here. + if err := s.store.withConn(ctx, !req.DryRun, func(tx *sql.Tx) error { + var err error + result, err = storeops.SweepInTx(ctx, tx, req) + return err + }); err != nil { + return issueops.SweepResult{}, err + } + return result, nil +} diff --git a/internal/storage/embeddeddolt/sweeper_contract_test.go b/internal/storage/embeddeddolt/sweeper_contract_test.go new file mode 100644 index 000000000..7325c73d6 --- /dev/null +++ b/internal/storage/embeddeddolt/sweeper_contract_test.go @@ -0,0 +1,73 @@ +//go:build cgo + +package embeddeddolt_test + +import ( + "testing" + + "github.com/steveyegge/beads/backend/conformance" +) + +// TestSweeperContract runs the Sweeper contract against the embedded store, +// which hands back the SAME body the server-backed store does +// (internal/storage/issueops.SweepInTx) and differs only in how it reaches a +// transaction and in that its commit runs outside one. That is what this +// wiring catches; it is not an independent vote on the body. +// +// One environment for the whole suite: booting an embedded engine per case +// would dominate the runtime, every case scopes itself to prefix-namespaced +// ids, and the history delta needs the subtests sequential anyway. +func TestSweeperContract(t *testing.T) { + skipUnlessEmbeddedDolt(t) + te := newTestEnv(t, "swp") + ctx := t.Context() + fixture := newEmbeddedSweeperFixture(t, te, "swp") + + t.Run("RefusesAnUnfilteredDurableSweep", func(t *testing.T) { + conformance.RunSweeperRefusesAnUnfilteredDurableSweep(t, ctx, fixture) + }) + t.Run("RefusesAMalformedRequest", func(t *testing.T) { + conformance.RunSweeperRefusesAMalformedRequest(t, ctx, fixture) + }) + t.Run("ClearsOneTierAndLeavesTheOther", func(t *testing.T) { + conformance.RunSweeperClearsOneTierAndLeavesTheOther(t, ctx, fixture) + }) + t.Run("ProtectsPinnedRows", func(t *testing.T) { + conformance.RunSweeperProtectsPinnedRows(t, ctx, fixture) + }) + t.Run("HonorsTheCutoffAndThePattern", func(t *testing.T) { + conformance.RunSweeperHonorsTheCutoffAndThePattern(t, ctx, fixture) + }) + t.Run("DryRunChangesNothing", func(t *testing.T) { + conformance.RunSweeperDryRunChangesNothing(t, ctx, fixture) + }) + t.Run("ProtectsCitedRows", func(t *testing.T) { + conformance.RunSweeperProtectsCitedRows(t, ctx, fixture) + }) + t.Run("EmptyMatchIsZeroAndNil", func(t *testing.T) { + conformance.RunSweeperEmptyMatchIsZeroAndNil(t, ctx, fixture) + }) + t.Run("RecordsAtMostOneHistoryEntry", func(t *testing.T) { + conformance.RunSweeperRecordsAtMostOneHistoryEntry(t, ctx, fixture) + }) + t.Run("DoesNotMutateTheCallerRequest", func(t *testing.T) { + conformance.RunSweeperDoesNotMutateTheCallerRequest(t, ctx, fixture) + }) +} + +func newEmbeddedSweeperFixture(t *testing.T, te *testEnv, prefix string) conformance.SweeperFixture { + t.Helper() + sweeper, err := te.store.Sweeper() + if err != nil { + t.Fatalf("Sweeper(): %v", err) + } + kit := newEmbeddedRoleFixtureKit(te, prefix) + return conformance.SweeperFixture{ + IssuePrefix: kit.IssuePrefix, + Sweeper: sweeper, + CreateIssue: kit.CreateIssue, + CreateWisp: kit.CreateWisp, + QueryScalar: kit.QueryScalar, + CountHistory: kit.CountHistory, + } +} diff --git a/internal/storage/hook_sweeper.go b/internal/storage/hook_sweeper.go new file mode 100644 index 000000000..8fa6a410a --- /dev/null +++ b/internal/storage/hook_sweeper.go @@ -0,0 +1,29 @@ +package storage + +import ( + "github.com/steveyegge/beads/issueops" +) + +// Sweeper returns the inner store's bulk-clearance surface. +// +// IT RECURSES UNWRAPPED, and this is the one WRITE role that does. The +// read roles recurse because nothing completed; this one completes plenty and +// still has no hook to fire, for two reasons that are both about the hook +// vocabulary rather than about the sweep: +// +// - there is no on_delete hook. internal/hooks publishes on_create, +// on_update and on_close, and a sweep is none of them. Firing on_update +// for a row that no longer exists would hand a user's script an id it +// cannot read back. +// - a hook script is handed an ISSUE, and a sweep's result carries counts +// rather than rows — deliberately, because a sweep of ten thousand wisps +// that materialized all ten thousand to fire a subprocess each would be a +// new failure mode rather than a feature. +// +// So the accessor EXISTS on this decorator — declared, never inherited, which +// the reflection test in role_accessor_decorator_test.go asserts — and adds no +// layer. If a delete event ever joins the hook vocabulary, this file is where +// it lands. +func (h *HookFiringStore) Sweeper() (issueops.Sweeper, error) { + return h.inner.Sweeper() +} diff --git a/internal/storage/issueops/sweep.go b/internal/storage/issueops/sweep.go new file mode 100644 index 000000000..a0505f0bf --- /dev/null +++ b/internal/storage/issueops/sweep.go @@ -0,0 +1,135 @@ +package issueops + +import ( + "context" + "database/sql" + "fmt" + + "github.com/steveyegge/beads/internal/types" + "github.com/steveyegge/beads/internal/workapi" + publicops "github.com/steveyegge/beads/issueops" +) + +// SweepInTx is the store-backed body behind issueops.Sweeper: the whole of +// `bd purge` and `bd prune` from candidate query to deletion, inside ONE +// transaction. +// +// It lives here rather than in an importable internal/workapi/store +// package for the reason DetectCycleReportInTx does: the work is several reads +// and a write that must see one snapshot, and storage.DoltStorage publishes +// methods, not transactions. The two Dolt-backed stores share this body and +// differ only in how they reach a transaction, so they are ONE vote and the +// unit-of-work provider is the second. There is nothing here a front door +// could construct — the function takes a transaction — so no depguard entry is +// needed to keep it out of cmd/bd. +// +// It assumes a request already refused by workapi.ValidateSweepRequest. The +// accessors validate BEFORE opening a transaction, so a refusal costs no +// database work and, as issueops.Sweeper promises, changes nothing. +// +// ONE TRANSACTION IS THE POINT. The direct CLI route this replaces searched in +// one transaction and deleted in another, so a row closed, unpinned or cited +// between the two was reported as one thing and treated as another. It is also +// what this gives up: the server-backed store's own DeleteIssues splits large +// wisp deletions into batched transactions to stay inside Dolt's write +// timeout, and a sweep cannot both do that and promise that the set it counted +// is the set it deleted. It promises the latter. +func SweepInTx(ctx context.Context, tx *sql.Tx, req publicops.SweepRequest) (publicops.SweepResult, error) { + result := publicops.SweepResult{DryRun: req.DryRun} + + candidates, err := SearchIssuesInTx(ctx, tx, "", workapi.BuildSweepCandidateFilter(req)) + if err != nil { + return publicops.SweepResult{}, fmt.Errorf("listing sweep candidates: %w", err) + } + + kept, skips := workapi.FilterSweepCandidates(candidates, req.IDPattern, req.ClosedBefore) + result.Skipped = skips + + if req.ProtectReferenced { + referenced, err := sweepReferencedInTx(ctx, tx, kept) + if err != nil { + return publicops.SweepResult{}, err + } + var count int + kept, count, result.ReferencedIDs = workapi.PartitionSweepReferenced(kept, referenced) + result.Skipped.Referenced = count + } + + if len(kept) == 0 { + return result, nil + } + + ids := make([]string, len(kept)) + for i, issue := range kept { + ids[i] = issue.ID + } + + // cascade=false, force=true: a sweep deletes what it selected and ORPHANS + // dependents outside the selection, which is what the direct route has + // always done for a real run. The DRY RUN passes the same flags rather + // than the force=false the route used to pass, so a preview reports what + // the sweep would do instead of refusing over dependents the sweep would + // have orphaned — the old dry run could fail where the real run succeeded, + // and the CLI answered that failure with zeros. + deleted, err := DeleteIssuesInTx(ctx, tx, ids, false, true, req.DryRun) + if err != nil { + return publicops.SweepResult{}, err + } + result.Swept = deleted.DeletedCount + result.Dependencies = deleted.DependenciesCount + result.Labels = deleted.LabelsCount + result.Events = deleted.EventsCount + return result, nil +} + +// sweepReferencedInTx returns which of the candidates are cited by a row that +// is not done, reading the not-done set and its comments on the SAME +// transaction the candidates came off. +// +// Reading the workspace's custom statuses is required rather than +// best-effort: an error propagates and aborts the sweep, because a scan that +// silently missed a custom active status would under-protect and delete a bead +// a live bead still cites (issueops.SweepRequest.ProtectReferenced). +func sweepReferencedInTx(ctx context.Context, tx *sql.Tx, candidates []*types.Issue) (map[string]bool, error) { + if len(candidates) == 0 { + return nil, nil + } + candidateIDs := make(map[string]bool, len(candidates)) + for _, issue := range candidates { + candidateIDs[issue.ID] = true + } + matcher := workapi.NewCandidateIDMatcher(candidateIDs) + + custom, err := ResolveCustomStatusesDetailedInTx(ctx, tx) + if err != nil { + return nil, fmt.Errorf("reading custom statuses for reference scan: %w", err) + } + notDone, err := SearchIssuesInTx(ctx, tx, "", workapi.BuildSweepReferenceScanFilter(custom)) + if err != nil { + return nil, fmt.Errorf("scanning open beads for references: %w", err) + } + + notDoneIDs := make([]string, 0, len(notDone)) + for _, issue := range notDone { + if issue != nil { + notDoneIDs = append(notDoneIDs, issue.ID) + } + } + comments, err := GetCommentsForIssuesInTx(ctx, tx, notDoneIDs) + if err != nil { + return nil, fmt.Errorf("scanning open beads for references: %w", err) + } + + referenced := make(map[string]bool) + for _, issue := range notDone { + if issue == nil { + continue + } + matcher.FindAll(issue.Description, referenced) + matcher.FindAll(issue.Notes, referenced) + for _, c := range comments[issue.ID] { + matcher.FindAll(c.Text, referenced) + } + } + return referenced, nil +} diff --git a/internal/storage/role_accessor_decorator_test.go b/internal/storage/role_accessor_decorator_test.go index b341d3098..addde2aab 100644 --- a/internal/storage/role_accessor_decorator_test.go +++ b/internal/storage/role_accessor_decorator_test.go @@ -11,9 +11,9 @@ import ( "github.com/steveyegge/beads/issueops" ) -// roleAccessorNames is the fourteen-strong capability surface every storage +// roleAccessorNames is the sixteen-strong capability surface every storage // decorator has to answer for. It is written out rather than derived so that -// adding a fifteenth role to DoltStorage without deciding what each decorator +// adding a seventeenth role to DoltStorage without deciding what each decorator // does with it is a compile-or-test failure somewhere, not silence. var roleAccessorNames = []string{ "IssueLifecycle", @@ -27,6 +27,7 @@ var roleAccessorNames = []string{ "CycleDetector", "ReadyCounter", "Querier", + "Sweeper", "Commenter", "ReadyClaimer", "BatchCloser", @@ -68,9 +69,9 @@ func assertRoleAccessorsAreDeclared(t *testing.T, decorator reflect.Type) { } } -// roleAccessorStore is a DoltStorage whose only real methods are the fourteen role -// accessors, each answering with a distinguishable sentinel so a test can tell -// a decorated surface from a passed-through one. +// roleAccessorStore is a DoltStorage whose only real methods are the sixteen +// role accessors, each answering with a distinguishable sentinel so a test can +// tell a decorated surface from a passed-through one. type roleAccessorStore struct { DoltStorage lifecycle issueops.Lifecycle @@ -88,6 +89,7 @@ type roleAccessorStore struct { editor issueops.DependencyEditor readyCounter issueops.ReadyCounter querier issueops.Querier + sweeper issueops.Sweeper err error } @@ -109,6 +111,7 @@ func newRoleAccessorStore() *roleAccessorStore { editor: sentinel, readyCounter: sentinel, querier: sentinel, + sweeper: sentinel, } } @@ -135,6 +138,9 @@ func (s *roleAccessorStore) ReadyCounter() (issueops.ReadyCounter, error) { return s.readyCounter, s.err } func (s *roleAccessorStore) Querier() (issueops.Querier, error) { return s.querier, s.err } +func (s *roleAccessorStore) Sweeper() (issueops.Sweeper, error) { + return s.sweeper, s.err +} func (s *roleAccessorStore) ReadyClaimer() (issueops.ReadyClaimer, error) { return s.claimer, s.err } @@ -143,7 +149,7 @@ func (s *roleAccessorStore) DependencyEditor() (issueops.DependencyEditor, error return s.editor, s.err } -// roleAccessorSentinel implements all fourteen roles at once. Nothing calls its +// roleAccessorSentinel implements all sixteen roles at once. Nothing calls its // methods; identity is the whole point. type roleAccessorSentinel struct{} @@ -216,6 +222,10 @@ func (*roleAccessorSentinel) CountReady(context.Context, issueops.ReadyRequest) func (*roleAccessorSentinel) Query(context.Context, issueops.QueryRequest) (issueops.IssuePage, error) { return issueops.IssuePage{}, nil } + +func (*roleAccessorSentinel) Sweep(context.Context, issueops.SweepRequest) (issueops.SweepResult, error) { + return issueops.SweepResult{}, nil +} func (*roleAccessorSentinel) AddComment(context.Context, issueops.AddCommentRequest) (issueops.AddCommentResult, error) { return issueops.AddCommentResult{}, nil } @@ -276,6 +286,11 @@ func TestHookFiringStoreWrapsTheWriteRolesAndPassesTheReadsThrough(t *testing.T) {"CycleDetector", func() (any, error) { return store.CycleDetector() }, inner.cycles, false}, {"ReadyCounter", func() (any, error) { return store.ReadyCounter() }, inner.readyCounter, false}, {"Querier", func() (any, error) { return store.Querier() }, inner.querier, false}, + // The one WRITE role in the unwrapped column, and the reason is the + // hook vocabulary rather than the role: there is no on_delete hook to + // fire and a swept row is gone, so hook_sweeper.go recurses. Asserting + // it here is what keeps that a decision. + {"Sweeper", func() (any, error) { return store.Sweeper() }, inner.sweeper, false}, } { t.Run(test.name, func(t *testing.T) { surface, err := test.got() @@ -321,6 +336,7 @@ func TestHookFiringStoreRoleAccessorsPropagateInnerErrors(t *testing.T) { {"CycleDetector", func() (any, error) { return store.CycleDetector() }}, {"ReadyCounter", func() (any, error) { return store.ReadyCounter() }}, {"Querier", func() (any, error) { return store.Querier() }}, + {"Sweeper", func() (any, error) { return store.Sweeper() }}, } { t.Run(test.name, func(t *testing.T) { if _, err := test.got(); !errors.Is(err, want) { diff --git a/internal/storage/storage.go b/internal/storage/storage.go index 52bec269b..d40051b9d 100644 --- a/internal/storage/storage.go +++ b/internal/storage/storage.go @@ -217,6 +217,18 @@ type Storage interface { // CONJUNCTION — every field narrows — and no arrangement of its fields // expresses a disjunction. Reads fire no hooks, as for IssueReader. Querier() (issueops.Querier, error) + // Sweeper returns the guarded bulk-clearance surface for this store — + // `bd purge` and `bd prune`, which are one capability over two disjoint + // tiers rather than two. Its own role because it describes a SET and then + // acts on it: no Lifecycle patch names a set, and composing a count with a + // delete would reopen the window this role exists to close. + // + // It is the one WRITE role whose hook decorator recurses UNWRAPPED, and + // that is a fact about the hook vocabulary rather than about the role: + // there is no on_delete hook to fire (internal/hooks publishes create, + // update and close), and the rows a sweep would name it with are gone. + // See hook_sweeper.go. + Sweeper() (issueops.Sweeper, error) // Issue CRUD CreateIssue(ctx context.Context, issue *types.Issue, actor string) error diff --git a/internal/storage/uow/sweeper.go b/internal/storage/uow/sweeper.go new file mode 100644 index 000000000..972743c9c --- /dev/null +++ b/internal/storage/uow/sweeper.go @@ -0,0 +1,174 @@ +package uow + +import ( + "context" + "fmt" + + "github.com/steveyegge/beads/internal/storage/domain" + "github.com/steveyegge/beads/internal/types" + "github.com/steveyegge/beads/internal/workapi" + publicops "github.com/steveyegge/beads/issueops" +) + +// SweeperSource is the capability accessor a unit-of-work provider offers for +// the bulk-clearance role, the sibling of CounterSource and IssueReaderSource. +// A consumer holding a provider by interface asks for the role here instead of +// reaching for a constructor. +type SweeperSource interface { + Sweeper() (publicops.Sweeper, error) +} + +// sweeper clears closed rows from one tier through a unit of work. +type sweeper struct { + provider UnitOfWorkProvider +} + +// Sweeper returns the guarded bulk-clearance surface for this provider. +func (p *doltSQLProvider) Sweeper() (publicops.Sweeper, error) { + return NewSweeper(p) +} + +// NewSweeper constructs a public sweeper backed by provider. +func NewSweeper(provider UnitOfWorkProvider) (publicops.Sweeper, error) { + if isNilUnitOfWorkProvider(provider) { + return nil, fmt.Errorf("new sweeper: unit-of-work provider must not be nil") + } + return &sweeper{provider: provider}, nil +} + +var _ publicops.Sweeper = (*sweeper)(nil) + +// Sweep clears the request's tier inside ONE unit of work. +// +// This is the genuinely separate body: the two store backends share +// issueops.SweepInTx, and this one reaches the same four questions through the +// domain use cases. What it must NOT do differently is which rows go — the +// selection, the pattern, the pinned and closed_at rechecks and the citation +// rule all run through the same internal/workapi functions the shared body +// runs, so the two implementations differ in their plumbing rather than in +// their answer. +// +// ONE UNIT OF WORK matters more here than it does for a read. The candidate +// query and the delete are on the same transaction, so a row closed, unpinned +// or cited between them cannot change what is deleted — the promise +// issueops.Sweeper.Sweep makes and the property the direct CLI route did not +// have before this role existed. +// +// A DRY RUN TAKES A READ-ONLY UNIT OF WORK: it writes nothing, so it must not +// take the committing path and leave a history entry describing a preview. +func (s *sweeper) Sweep(ctx context.Context, req publicops.SweepRequest) (publicops.SweepResult, error) { + if err := workapi.ValidateSweepRequest(req); err != nil { + return publicops.SweepResult{}, err + } + if req.DryRun { + return RunTxRead(ctx, s.provider, func(ctx context.Context, uw UnitOfWork) (publicops.SweepResult, error) { + return sweepInUOW(ctx, uw, req) + }) + } + return RunTxResult(ctx, s.provider, func(ctx context.Context, uw UnitOfWork) (publicops.SweepResult, string, error) { + result, err := sweepInUOW(ctx, uw, req) + if err != nil || result.Swept == 0 { + // A sweep that deleted nothing labels nothing: the role promises + // at most one history entry per call and none for a no-op. + return result, "", err + } + return result, fmt.Sprintf("bd: sweep %d %s bead(s)", result.Swept, req.Tier), nil + }) +} + +// sweepInUOW is the whole sweep on one unit of work, shared by the preview +// path and the committing one so the two cannot answer differently. +func sweepInUOW(ctx context.Context, uw UnitOfWork, req publicops.SweepRequest) (publicops.SweepResult, error) { + result := publicops.SweepResult{DryRun: req.DryRun} + + page, err := uw.IssueUseCase().SearchIssues(ctx, "", workapi.BuildSweepCandidateFilter(req)) + if err != nil { + return publicops.SweepResult{}, fmt.Errorf("listing sweep candidates: %w", err) + } + + kept, skips := workapi.FilterSweepCandidates(page.Items, req.IDPattern, req.ClosedBefore) + result.Skipped = skips + + if req.ProtectReferenced { + referenced, err := sweepReferencedInUOW(ctx, uw, kept) + if err != nil { + return publicops.SweepResult{}, err + } + var count int + kept, count, result.ReferencedIDs = workapi.PartitionSweepReferenced(kept, referenced) + result.Skipped.Referenced = count + } + + if len(kept) == 0 { + return result, nil + } + + ids := make([]string, len(kept)) + for i, issue := range kept { + ids[i] = issue.ID + } + deleted, err := uw.IssueUseCase().DeleteIssues(ctx, domain.DeleteIssuesParams{ + IDs: ids, + DryRun: req.DryRun, + }, req.Actor) + if err != nil { + return publicops.SweepResult{}, err + } + result.Swept = deleted.DeletedCount + result.Dependencies = deleted.DependenciesCount + result.Labels = deleted.LabelsCount + result.Events = deleted.EventsCount + return result, nil +} + +// sweepReferencedInUOW returns which of the candidates a not-done row cites, +// reading the not-done set and its comments on the same unit of work the +// candidates came off. +// +// It reads the comments of the whole not-done set in ONE batch rather than one +// query per row, which is what makes the protection affordable on the corpora +// `bd prune` runs over. +func sweepReferencedInUOW(ctx context.Context, uw UnitOfWork, candidates []*types.Issue) (map[string]bool, error) { + if len(candidates) == 0 { + return nil, nil + } + candidateIDs := make(map[string]bool, len(candidates)) + for _, issue := range candidates { + candidateIDs[issue.ID] = true + } + matcher := workapi.NewCandidateIDMatcher(candidateIDs) + + custom, err := uw.ConfigUseCase().GetCustomStatuses(ctx) + if err != nil { + return nil, fmt.Errorf("reading custom statuses for reference scan: %w", err) + } + page, err := uw.IssueUseCase().SearchIssues(ctx, "", workapi.BuildSweepReferenceScanFilter(custom)) + if err != nil { + return nil, fmt.Errorf("scanning open beads for references: %w", err) + } + notDone := page.Items + + notDoneIDs := make([]string, 0, len(notDone)) + for _, issue := range notDone { + if issue != nil { + notDoneIDs = append(notDoneIDs, issue.ID) + } + } + comments, err := uw.CommentUseCase().GetCommentsForIssues(ctx, notDoneIDs) + if err != nil { + return nil, fmt.Errorf("scanning open beads for references: %w", err) + } + + referenced := make(map[string]bool) + for _, issue := range notDone { + if issue == nil { + continue + } + matcher.FindAll(issue.Description, referenced) + matcher.FindAll(issue.Notes, referenced) + for _, c := range comments[issue.ID] { + matcher.FindAll(c.Text, referenced) + } + } + return referenced, nil +} diff --git a/internal/storage/uow/sweeper_contract_test.go b/internal/storage/uow/sweeper_contract_test.go new file mode 100644 index 000000000..3ba29db88 --- /dev/null +++ b/internal/storage/uow/sweeper_contract_test.go @@ -0,0 +1,80 @@ +package uow + +import ( + "context" + "testing" + + "github.com/steveyegge/beads/backend/conformance" +) + +// TestSweeperContract runs the Sweeper contract against the unit-of-work +// provider — the one implementation that does not run +// internal/storage/issueops.SweepInTx, and the one that reaches the same four +// questions through the domain use cases. It is the SECOND of two votes, not +// the third: the two store backends share the other body. +// +// One provider for the whole suite (each newUOWRoleFixtureProvider boots a +// real Dolt sql-server) and NO t.Parallel: this backend has no per-test +// copy-on-write branch, so dolt_log and the issues table are database-global +// and a parallel subtest would delete another subtest's rows — which on this +// role is not a flake but the operation working as documented. +func TestSweeperContract(t *testing.T) { + ctx := context.Background() + fixture := newUOWSweeperFixture(t, ctx, "swp") + + t.Run("RefusesAnUnfilteredDurableSweep", func(t *testing.T) { + conformance.RunSweeperRefusesAnUnfilteredDurableSweep(t, ctx, fixture) + }) + t.Run("RefusesAMalformedRequest", func(t *testing.T) { + conformance.RunSweeperRefusesAMalformedRequest(t, ctx, fixture) + }) + t.Run("ClearsOneTierAndLeavesTheOther", func(t *testing.T) { + conformance.RunSweeperClearsOneTierAndLeavesTheOther(t, ctx, fixture) + }) + t.Run("ProtectsPinnedRows", func(t *testing.T) { + conformance.RunSweeperProtectsPinnedRows(t, ctx, fixture) + }) + t.Run("HonorsTheCutoffAndThePattern", func(t *testing.T) { + conformance.RunSweeperHonorsTheCutoffAndThePattern(t, ctx, fixture) + }) + t.Run("DryRunChangesNothing", func(t *testing.T) { + conformance.RunSweeperDryRunChangesNothing(t, ctx, fixture) + }) + t.Run("ProtectsCitedRows", func(t *testing.T) { + conformance.RunSweeperProtectsCitedRows(t, ctx, fixture) + }) + t.Run("EmptyMatchIsZeroAndNil", func(t *testing.T) { + conformance.RunSweeperEmptyMatchIsZeroAndNil(t, ctx, fixture) + }) + t.Run("RecordsAtMostOneHistoryEntry", func(t *testing.T) { + conformance.RunSweeperRecordsAtMostOneHistoryEntry(t, ctx, fixture) + }) + t.Run("DoesNotMutateTheCallerRequest", func(t *testing.T) { + conformance.RunSweeperDoesNotMutateTheCallerRequest(t, ctx, fixture) + }) +} + +func newUOWSweeperFixture(t *testing.T, ctx context.Context, prefix string) conformance.SweeperFixture { + t.Helper() + provider := newUOWRoleFixtureProvider(t, ctx, prefix) + // Through the capability accessor, not NewSweeper: a provider that stopped + // offering the role is the regression, and a constructor call would hide + // it. + source, ok := provider.(SweeperSource) + if !ok { + t.Fatalf("provider %T does not offer the Sweeper accessor", provider) + } + sweeper, err := source.Sweeper() + if err != nil { + t.Fatalf("Sweeper(): %v", err) + } + kit := newUOWRoleFixtureKit(provider, prefix) + return conformance.SweeperFixture{ + IssuePrefix: kit.IssuePrefix, + Sweeper: sweeper, + CreateIssue: kit.CreateIssue, + CreateWisp: kit.CreateWisp, + QueryScalar: kit.QueryScalar, + CountHistory: kit.CountHistory, + } +} diff --git a/internal/telemetry/role_accessor_decorator_test.go b/internal/telemetry/role_accessor_decorator_test.go index b58d80307..bfe96af1a 100644 --- a/internal/telemetry/role_accessor_decorator_test.go +++ b/internal/telemetry/role_accessor_decorator_test.go @@ -12,7 +12,7 @@ import ( "github.com/steveyegge/beads/issueops" ) -// roleAccessorNames is the fourteen-strong capability surface every storage +// roleAccessorNames is the sixteen-strong capability surface every storage // decorator has to answer for. It is duplicated from the sibling test in // internal/storage rather than shared because that package cannot import this // one and this one's test helpers cannot be exported back. @@ -28,6 +28,7 @@ var roleAccessorNames = []string{ "CycleDetector", "ReadyCounter", "Querier", + "Sweeper", "Commenter", "ReadyClaimer", "BatchCloser", @@ -58,8 +59,8 @@ func TestInstrumentedStorageDeclaresEveryRoleAccessor(t *testing.T) { } } -// roleAccessorStore is a DoltStorage whose only real methods are the fourteen role -// accessors, each answering with a distinguishable sentinel so a test can tell +// roleAccessorStore is a DoltStorage whose only real methods are the sixteen +// role accessors, each answering with a distinguishable sentinel so a test can tell // an instrumented surface from a passed-through one. type roleAccessorStore struct { storage.DoltStorage @@ -90,6 +91,9 @@ func (s *roleAccessorStore) ReadyCounter() (issueops.ReadyCounter, error) { return s.surface, s.err } func (s *roleAccessorStore) Querier() (issueops.Querier, error) { return s.surface, s.err } +func (s *roleAccessorStore) Sweeper() (issueops.Sweeper, error) { + return s.surface, s.err +} func (s *roleAccessorStore) ReadyClaimer() (issueops.ReadyClaimer, error) { return s.surface, s.err } @@ -98,7 +102,7 @@ func (s *roleAccessorStore) DependencyEditor() (issueops.DependencyEditor, error return s.surface, s.err } -// roleAccessorSentinel implements all fourteen roles at once. Nothing calls its +// roleAccessorSentinel implements all sixteen roles at once. Nothing calls its // methods; identity is the whole point. type roleAccessorSentinel struct{} @@ -171,6 +175,10 @@ func (*roleAccessorSentinel) CountReady(context.Context, issueops.ReadyRequest) func (*roleAccessorSentinel) Query(context.Context, issueops.QueryRequest) (issueops.IssuePage, error) { return issueops.IssuePage{}, nil } + +func (*roleAccessorSentinel) Sweep(context.Context, issueops.SweepRequest) (issueops.SweepResult, error) { + return issueops.SweepResult{}, nil +} func (*roleAccessorSentinel) AddComment(context.Context, issueops.AddCommentRequest) (issueops.AddCommentResult, error) { return issueops.AddCommentResult{}, nil } @@ -193,7 +201,7 @@ func (*roleAccessorSentinel) RemoveDependency(context.Context, issueops.RemoveDe // DoltStorage, and silently drops every span and timing on that role. // // Unlike the hook decorator, this one has no read-role exemption: telemetry -// spans reads as well as writes, so ALL THIRTEEN must come back wrapped. Only +// spans reads as well as writes, so ALL FOURTEEN must come back wrapped. Only // IssueLifecycle and IssueReader had a recursion pin of their own before this // test; the rest had no test at all. func TestInstrumentedStorageInstrumentsEveryRoleAccessor(t *testing.T) { @@ -220,6 +228,7 @@ func TestInstrumentedStorageInstrumentsEveryRoleAccessor(t *testing.T) { {"CycleDetector", func() (any, error) { return wrapped.CycleDetector() }}, {"ReadyCounter", func() (any, error) { return wrapped.ReadyCounter() }}, {"Querier", func() (any, error) { return wrapped.Querier() }}, + {"Sweeper", func() (any, error) { return wrapped.Sweeper() }}, {"Commenter", func() (any, error) { return wrapped.Commenter() }}, {"ReadyClaimer", func() (any, error) { return wrapped.ReadyClaimer() }}, {"BatchCloser", func() (any, error) { return wrapped.BatchCloser() }}, @@ -263,6 +272,7 @@ func TestInstrumentedStorageRoleAccessorsPropagateInnerErrors(t *testing.T) { {"CycleDetector", func() (any, error) { return wrapped.CycleDetector() }}, {"ReadyCounter", func() (any, error) { return wrapped.ReadyCounter() }}, {"Querier", func() (any, error) { return wrapped.Querier() }}, + {"Sweeper", func() (any, error) { return wrapped.Sweeper() }}, {"Commenter", func() (any, error) { return wrapped.Commenter() }}, {"ReadyClaimer", func() (any, error) { return wrapped.ReadyClaimer() }}, {"BatchCloser", func() (any, error) { return wrapped.BatchCloser() }}, diff --git a/internal/telemetry/sweeper.go b/internal/telemetry/sweeper.go new file mode 100644 index 000000000..9a5c99ac4 --- /dev/null +++ b/internal/telemetry/sweeper.go @@ -0,0 +1,38 @@ +package telemetry + +import ( + "context" + + "github.com/steveyegge/beads/issueops" +) + +// Sweeper returns the inner store's bulk-clearance surface wrapped in this +// layer's instrumentation. It recurses instead of delegating: a blind +// delegation would return the inner sweeper unspanned and untimed — and this +// is the longest-running write on the surface, so it is the one whose span +// matters most. +func (s *InstrumentedStorage) Sweeper() (issueops.Sweeper, error) { + inner, err := s.Unwrap().Sweeper() + if err != nil { + return nil, err + } + return s.WrapSweeper(inner), nil +} + +// WrapSweeper instruments guarded bulk clearance with this storage layer's +// existing telemetry meter and tracer. +func (s *InstrumentedStorage) WrapSweeper(inner issueops.Sweeper) issueops.Sweeper { + return &instrumentedSweeper{storage: s, inner: inner} +} + +type instrumentedSweeper struct { + storage *InstrumentedStorage + inner issueops.Sweeper +} + +func (c *instrumentedSweeper) Sweep(ctx context.Context, request issueops.SweepRequest) (result issueops.SweepResult, err error) { + ctx, span, started := c.storage.op(ctx, "Sweeper.Sweep") + result, err = c.inner.Sweep(ctx, request) + c.storage.done(ctx, span, started, err) + return result, err +} diff --git a/internal/workapi/race_detector_off_test.go b/internal/workapi/race_detector_off_test.go new file mode 100644 index 000000000..e5b1a2c2a --- /dev/null +++ b/internal/workapi/race_detector_off_test.go @@ -0,0 +1,6 @@ +//go:build !race + +package workapi + +// raceEnabled is false in non-race builds; see race_detector_on_test.go. +const raceEnabled = false diff --git a/internal/workapi/race_detector_on_test.go b/internal/workapi/race_detector_on_test.go new file mode 100644 index 000000000..0ec6500e1 --- /dev/null +++ b/internal/workapi/race_detector_on_test.go @@ -0,0 +1,11 @@ +//go:build race + +package workapi + +// raceEnabled reports whether the test binary was built with the race +// detector (-race). Wall-clock performance budgets are unreliable under race +// instrumentation (it adds multi-x overhead), so timing-sensitive benches +// consult this flag before enforcing a duration bound. It is a sibling of the +// pair in cmd/bd, which is where the sweep's large-fixture budget used to +// live. +const raceEnabled = true diff --git a/internal/workapi/sweep.go b/internal/workapi/sweep.go new file mode 100644 index 000000000..61d40f8d4 --- /dev/null +++ b/internal/workapi/sweep.go @@ -0,0 +1,274 @@ +package workapi + +import ( + "fmt" + "path/filepath" + "sort" + "strings" + "time" + + "github.com/steveyegge/beads/internal/types" + "github.com/steveyegge/beads/issueops" +) + +// The shared, DATABASE-FREE half of issueops.Sweeper: what a sweep request +// means, which candidates survive the recheck, and what counts as a citation. +// Every implementation of the role runs these, so `bd purge` and `bd prune` +// have one definition rather than one per backend, and the parts that decide +// what the answer MEANS are pinned in sweep_test.go in milliseconds. +// +// What is NOT here is the sweep itself. Selecting rows and deleting them needs +// one transaction (issueops.Sweeper.Sweep), which no interface above a store +// publishes; the bodies live in internal/storage/issueops/sweep.go and in the +// unit-of-work provider. + +// ValidateSweepRequest applies the request rules every Sweeper implementation +// shares, before anything is read. +// +// The require-a-filter refusal for the durable tier is the one that matters, +// and it is HERE rather than in a CLI handler on purpose: it is a safety +// invariant, so a second front door inherits it by calling the role rather +// than by remembering to re-implement it. See issueops.SweepRequest. +func ValidateSweepRequest(in issueops.SweepRequest) error { + switch in.Tier { + case issueops.SweepEphemeral, issueops.SweepDurable: + case "": + return fmt.Errorf("%w: sweep requires a tier (%q or %q), and has no default", + issueops.ErrValidation, issueops.SweepEphemeral, issueops.SweepDurable) + default: + return fmt.Errorf("%w: %q is not a sweep tier; use %q or %q", + issueops.ErrValidation, in.Tier, issueops.SweepEphemeral, issueops.SweepDurable) + } + if in.IDPattern != "" { + // filepath.Match reports a malformed pattern on any subject, so one + // probe against the empty string classifies the pattern itself. The + // front doors used to discard this error, which turned `--pattern '['` + // into "nothing matched" on a command whose job is to delete matches. + if _, err := filepath.Match(in.IDPattern, ""); err != nil { + return fmt.Errorf("%w: --pattern %q is not a valid glob: %v", + issueops.ErrValidation, in.IDPattern, err) + } + } + if in.Tier == issueops.SweepDurable && in.ClosedBefore == nil && in.IDPattern == "" { + return fmt.Errorf("%w: a durable sweep requires a closed-before cutoff or an id pattern; "+ + "pass the pattern \"*\" to sweep every closed issue deliberately", + issueops.ErrValidation) + } + return nil +} + +// BuildSweepCandidateFilter turns a sweep request into the storage-level +// filter that selects its CANDIDATES: the closed rows of one tier, bounded by +// the cutoff. +// +// The pattern is deliberately NOT in the filter. Globs are matched in Go +// (MatchesSweepPattern) because the storage seam has no glob predicate and a +// LIKE translation would silently disagree with filepath.Match on `[...]` and +// on the escape rules — a disagreement that, on this operation, decides which +// rows are deleted. +// +// Call ValidateSweepRequest first; this builder assumes a validated request +// and does not re-refuse one. +func BuildSweepCandidateFilter(in issueops.SweepRequest) types.IssueFilter { + closed := types.StatusClosed + ephemeral := in.Tier == issueops.SweepEphemeral + filter := types.IssueFilter{ + Status: &closed, + Ephemeral: &ephemeral, + } + if in.ClosedBefore != nil { + cutoff := *in.ClosedBefore + filter.ClosedBefore = &cutoff + } + return filter +} + +// MatchesSweepPattern reports whether an id is admitted by a request's +// IDPattern. An empty pattern admits everything. +// +// A malformed pattern reports false here; ValidateSweepRequest is what refuses +// it, so this function is never the place a bad glob is discovered. +func MatchesSweepPattern(pattern, id string) bool { + if pattern == "" { + return true + } + ok, err := filepath.Match(pattern, id) + return err == nil && ok +} + +// FilterSweepCandidates applies the pattern and then the two protections that +// hold a candidate back before any reference scan: the pinned flag, and the +// closed_at recheck. +// +// THE ORDER IS PART OF THE ANSWER, and issueops.Sweeper.Sweep states it: the +// pattern narrows first, so a pinned row the pattern excluded is not counted +// as protected — it was never a candidate. The three closed_at buckets are +// defenses against the tier query returning a row it was not asked for, and +// they are counted rather than dropped so a disagreement between the query and +// the recheck is visible instead of silent. +// +// cutoff is the request's ClosedBefore; a nil cutoff performs the presence +// checks and skips the comparison. +func FilterSweepCandidates(issues []*types.Issue, pattern string, cutoff *time.Time) ([]*types.Issue, issueops.SweepSkips) { + kept := make([]*types.Issue, 0, len(issues)) + var skips issueops.SweepSkips + + for _, issue := range issues { + if issue == nil { + skips.Unreadable++ + continue + } + if !MatchesSweepPattern(pattern, issue.ID) { + continue + } + switch { + case issue.Pinned: + skips.Pinned++ + case issue.Status != types.StatusClosed: + skips.NotClosed++ + case issue.ClosedAt == nil: + skips.UnknownClosedAt++ + case cutoff != nil && !issue.ClosedAt.Before(*cutoff): + skips.ClosedAtOrAfterCutoff++ + default: + kept = append(kept, issue) + } + } + return kept, skips +} + +// SweepDefenseSkips is how many candidates the recheck rejected as +// self-inconsistent — the three closed_at buckets plus the unreadable row. +// It is separate from the two PROTECTIONS (pinned, referenced) because a +// non-zero value here is a defense firing rather than a normal outcome, and +// the front doors warn on exactly that. +func SweepDefenseSkips(s issueops.SweepSkips) int { + return s.Unreadable + s.NotClosed + s.UnknownClosedAt + s.ClosedAtOrAfterCutoff +} + +// NotDoneStatusesForSweep is the status vocabulary a reference scan asks: +// every built-in status that is not "closed", plus every configured custom +// status whose category is not "done". +// +// It is an explicit LIST rather than a not-closed exclusion because that is +// what the storage seam takes reliably on both planes (a Statuses set, not an +// ExcludeStatus). Reading the workspace's custom statuses is required, not +// best-effort: a scan that missed a custom active status would under-protect +// and delete a bead a live bead still cites — see +// issueops.SweepRequest.ProtectReferenced. +func NotDoneStatusesForSweep(custom []types.CustomStatus) []types.Status { + statuses := []types.Status{ + types.StatusOpen, + types.StatusInProgress, + types.StatusBlocked, + types.StatusDeferred, + types.StatusPinned, + types.StatusHooked, + } + for _, cs := range custom { + if cs.Category != types.CategoryDone { + statuses = append(statuses, types.Status(cs.Name)) + } + } + return statuses +} + +// BuildSweepReferenceScanFilter selects the rows a reference scan reads: the +// not-done set, and nothing else about them. +func BuildSweepReferenceScanFilter(custom []types.CustomStatus) types.IssueFilter { + return types.IssueFilter{Statuses: NotDoneStatusesForSweep(custom)} +} + +// CandidateIDMatcher finds literal, word-bounded occurrences of a candidate id +// set in arbitrary text. It is the whole of what "cited" means for +// issueops.SweepRequest.ProtectReferenced. +// +// It is bucketed by FIRST BYTE and each bucket sorted longest-first, because +// the scan is run over every not-done bead's description, notes and comments: +// a naive strings.Contains per candidate is O(candidates x text) and the +// reference protection exists on the command that runs over the largest +// corpora. Longest-first inside a bucket is what makes `be-1.1` win over +// `be-1` where both are candidates and both start at the same offset. +type CandidateIDMatcher struct { + byFirstByte map[byte][]string +} + +// NewCandidateIDMatcher builds a matcher over a candidate id set. Empty ids +// are dropped: they would match at every boundary. +func NewCandidateIDMatcher(candidateIDs map[string]bool) CandidateIDMatcher { + byFirstByte := make(map[byte][]string) + for id := range candidateIDs { + if id == "" { + continue + } + byFirstByte[id[0]] = append(byFirstByte[id[0]], id) + } + for first := range byFirstByte { + ids := byFirstByte[first] + sort.Slice(ids, func(i, j int) bool { + if len(ids[i]) == len(ids[j]) { + return ids[i] < ids[j] + } + return len(ids[i]) > len(ids[j]) + }) + byFirstByte[first] = ids + } + return CandidateIDMatcher{byFirstByte: byFirstByte} +} + +// FindAll adds every candidate id occurring in text at ASCII word boundaries +// to found. +func (m CandidateIDMatcher) FindAll(text string, found map[string]bool) { + for i := 0; i < len(text); i++ { + ids := m.byFirstByte[text[i]] + if len(ids) == 0 || !isWordBoundaryAt(text, i) { + continue + } + for _, id := range ids { + end := i + len(id) + if end <= len(text) && strings.HasPrefix(text[i:], id) && isWordBoundaryAt(text, end) { + found[id] = true + break + } + } + } +} + +func isWordBoundaryAt(s string, idx int) bool { + var before, after byte + if idx > 0 { + before = s[idx-1] + } + if idx < len(s) { + after = s[idx] + } + return isASCIIWordByte(before) != isASCIIWordByte(after) +} + +func isASCIIWordByte(b byte) bool { + return b == '_' || + ('0' <= b && b <= '9') || + ('A' <= b && b <= 'Z') || + ('a' <= b && b <= 'z') +} + +// PartitionSweepReferenced splits candidates into the ones no not-done row +// cites and the ones some row does, and returns the bounded sample +// issueops.SweepResult.ReferencedIDs carries. +// +// The kept slice preserves the candidate order, and so does the sample, so two +// runs over one snapshot report the same ids. +func PartitionSweepReferenced(candidates []*types.Issue, referenced map[string]bool) (kept []*types.Issue, referencedCount int, sample []string) { + kept = make([]*types.Issue, 0, len(candidates)) + for _, issue := range candidates { + if referenced[issue.ID] { + referencedCount++ + if len(sample) < issueops.SweepReferencedSampleLimit { + sample = append(sample, issue.ID) + } + continue + } + kept = append(kept, issue) + } + return kept, referencedCount, sample +} diff --git a/internal/workapi/sweep_test.go b/internal/workapi/sweep_test.go new file mode 100644 index 000000000..ed7c0d934 --- /dev/null +++ b/internal/workapi/sweep_test.go @@ -0,0 +1,352 @@ +package workapi + +import ( + "errors" + "fmt" + "strings" + "testing" + "time" + + "github.com/steveyegge/beads/internal/types" + "github.com/steveyegge/beads/issueops" +) + +// The DATABASE-FREE half of issueops.Sweeper, pinned in milliseconds: what a +// request means, which candidates survive the recheck, and what counts as a +// citation. The conformance contract is left to assert what only a real +// backend can show. + +func TestValidateSweepRequestRequiresATier(t *testing.T) { + for _, test := range []struct { + name string + tier issueops.SweepTier + }{ + {"unset", ""}, + {"unrecognized", issueops.SweepTier("wisps")}, + } { + t.Run(test.name, func(t *testing.T) { + err := ValidateSweepRequest(issueops.SweepRequest{Tier: test.tier, IDPattern: "*"}) + if !errors.Is(err, issueops.ErrValidation) { + t.Fatalf("ValidateSweepRequest(tier=%q) error = %v, want ErrValidation", test.tier, err) + } + }) + } +} + +// TestValidateSweepRequestGatesTheDurableTier is the safety invariant Q9 moved +// below the front doors: an unfiltered DURABLE sweep is a refusal, and the +// same request on the ephemeral tier is not. +func TestValidateSweepRequestGatesTheDurableTier(t *testing.T) { + cutoff := time.Now() + for _, test := range []struct { + name string + request issueops.SweepRequest + refused bool + }{ + {"durable, no filter", issueops.SweepRequest{Tier: issueops.SweepDurable}, true}, + {"durable, cutoff", issueops.SweepRequest{Tier: issueops.SweepDurable, ClosedBefore: &cutoff}, false}, + {"durable, pattern", issueops.SweepRequest{Tier: issueops.SweepDurable, IDPattern: "*"}, false}, + // The deliberate keystroke the refusal points at: "*" is how a caller + // says "everything closed" on purpose. + {"durable, star", issueops.SweepRequest{Tier: issueops.SweepDurable, IDPattern: "*"}, false}, + {"ephemeral, no filter", issueops.SweepRequest{Tier: issueops.SweepEphemeral}, false}, + } { + t.Run(test.name, func(t *testing.T) { + err := ValidateSweepRequest(test.request) + if got := errors.Is(err, issueops.ErrValidation); got != test.refused { + t.Fatalf("ValidateSweepRequest() error = %v, want refused=%v", err, test.refused) + } + }) + } +} + +// TestValidateSweepRequestRefusesAMalformedPattern pins the defect fix: both +// front doors used to discard filepath.Match's error, so a bad glob reported +// "nothing matched" on a command whose job is to delete what it matched. +func TestValidateSweepRequestRefusesAMalformedPattern(t *testing.T) { + err := ValidateSweepRequest(issueops.SweepRequest{Tier: issueops.SweepDurable, IDPattern: "[bad"}) + if !errors.Is(err, issueops.ErrValidation) { + t.Fatalf("ValidateSweepRequest(pattern=%q) error = %v, want ErrValidation", "[bad", err) + } + if !strings.Contains(err.Error(), "[bad") { + t.Fatalf("refusal %q does not quote the pattern the caller sent", err) + } +} + +func TestBuildSweepCandidateFilterSelectsOneClosedTier(t *testing.T) { + cutoff := time.Date(2026, 4, 26, 18, 30, 18, 0, time.UTC) + for _, test := range []struct { + tier issueops.SweepTier + ephemeral bool + }{ + {issueops.SweepEphemeral, true}, + {issueops.SweepDurable, false}, + } { + t.Run(string(test.tier), func(t *testing.T) { + filter := BuildSweepCandidateFilter(issueops.SweepRequest{Tier: test.tier, ClosedBefore: &cutoff}) + if filter.Status == nil || *filter.Status != types.StatusClosed { + t.Fatalf("filter.Status = %v, want closed", filter.Status) + } + if filter.Ephemeral == nil || *filter.Ephemeral != test.ephemeral { + t.Fatalf("filter.Ephemeral = %v, want %v", filter.Ephemeral, test.ephemeral) + } + if filter.ClosedBefore == nil || !filter.ClosedBefore.Equal(cutoff) { + t.Fatalf("filter.ClosedBefore = %v, want %v", filter.ClosedBefore, cutoff) + } + }) + } +} + +// TestBuildSweepCandidateFilterCopiesTheCutoff pins the no-mutation promise on +// the one pointer the request carries. +func TestBuildSweepCandidateFilterCopiesTheCutoff(t *testing.T) { + cutoff := time.Date(2026, 4, 26, 18, 30, 18, 0, time.UTC) + filter := BuildSweepCandidateFilter(issueops.SweepRequest{Tier: issueops.SweepDurable, ClosedBefore: &cutoff}) + if filter.ClosedBefore == &cutoff { + t.Fatal("filter shares the caller's *time.Time; a builder must not hand back an alias of a request field") + } +} + +// TestFilterSweepCandidatesRechecksClosedAtCutoff is the recheck's own table, +// moved here from cmd/bd when the role took it below both front doors. +func TestFilterSweepCandidatesRechecksClosedAtCutoff(t *testing.T) { + cutoff := time.Date(2026, 3, 27, 20, 1, 44, 0, time.UTC) + oldClosedAt := cutoff.Add(-time.Second) + recentClosedAt := cutoff.Add(time.Second) + + candidates := []*types.Issue{ + {ID: "old-closed", Status: types.StatusClosed, ClosedAt: &oldClosedAt}, + {ID: "recent-closed", Status: types.StatusClosed, ClosedAt: &recentClosedAt}, + {ID: "missing-closed-at", Status: types.StatusClosed}, + {ID: "open-with-old-closed-at", Status: types.StatusOpen, ClosedAt: &oldClosedAt}, + {ID: "pinned-old", Status: types.StatusClosed, ClosedAt: &oldClosedAt, Pinned: true}, + nil, + } + + filtered, skips := FilterSweepCandidates(candidates, "", &cutoff) + + if len(filtered) != 1 || filtered[0].ID != "old-closed" { + t.Fatalf("filtered IDs = %v, want only old-closed", sweepCandidateIDs(filtered)) + } + for _, check := range []struct { + name string + got int + }{ + {"Pinned", skips.Pinned}, + {"ClosedAtOrAfterCutoff", skips.ClosedAtOrAfterCutoff}, + {"UnknownClosedAt", skips.UnknownClosedAt}, + {"NotClosed", skips.NotClosed}, + {"Unreadable", skips.Unreadable}, + } { + if check.got != 1 { + t.Errorf("skips.%s = %d, want 1", check.name, check.got) + } + } +} + +func TestFilterSweepCandidatesWithoutCutoffStillRequiresClosedAt(t *testing.T) { + closedAt := time.Date(2026, 4, 26, 18, 30, 18, 0, time.UTC) + candidates := []*types.Issue{ + {ID: "closed", Status: types.StatusClosed, ClosedAt: &closedAt}, + {ID: "closed-missing-time", Status: types.StatusClosed}, + } + + filtered, skips := FilterSweepCandidates(candidates, "", nil) + + if len(filtered) != 1 || filtered[0].ID != "closed" { + t.Fatalf("filtered IDs = %v, want only closed", sweepCandidateIDs(filtered)) + } + if skips.UnknownClosedAt != 1 { + t.Fatalf("skips.UnknownClosedAt = %d, want 1", skips.UnknownClosedAt) + } + if skips.ClosedAtOrAfterCutoff != 0 { + t.Fatalf("skips.ClosedAtOrAfterCutoff = %d, want 0 without a cutoff", skips.ClosedAtOrAfterCutoff) + } +} + +// TestFilterSweepCandidatesNarrowsByPatternBeforeProtecting pins the ORDER +// issueops.Sweeper.Sweep states: a pinned row the pattern excluded is not +// counted as protected, because it was never a candidate. +func TestFilterSweepCandidatesNarrowsByPatternBeforeProtecting(t *testing.T) { + closedAt := time.Date(2026, 4, 26, 18, 30, 18, 0, time.UTC) + candidates := []*types.Issue{ + {ID: "keep-1", Status: types.StatusClosed, ClosedAt: &closedAt}, + {ID: "keep-pinned", Status: types.StatusClosed, ClosedAt: &closedAt, Pinned: true}, + {ID: "other-pinned", Status: types.StatusClosed, ClosedAt: &closedAt, Pinned: true}, + } + + filtered, skips := FilterSweepCandidates(candidates, "keep-*", nil) + + if len(filtered) != 1 || filtered[0].ID != "keep-1" { + t.Fatalf("filtered IDs = %v, want only keep-1", sweepCandidateIDs(filtered)) + } + if skips.Pinned != 1 { + t.Fatalf("skips.Pinned = %d, want 1 — only the pinned row the pattern ADMITTED is a protection", skips.Pinned) + } +} + +func TestMatchesSweepPatternAdmitsEverythingWhenEmpty(t *testing.T) { + if !MatchesSweepPattern("", "anything-at-all") { + t.Fatal("an empty pattern must admit every id") + } + if MatchesSweepPattern("[bad", "anything") { + t.Fatal("a malformed pattern must not match; ValidateSweepRequest is what refuses it") + } +} + +// TestNotDoneStatusesForSweepIncludesActiveCustomStatuses is the clause +// issueops.SweepRequest.ProtectReferenced calls required rather than +// best-effort: a workspace's own active status has to protect what it cites. +func TestNotDoneStatusesForSweepIncludesActiveCustomStatuses(t *testing.T) { + statuses := NotDoneStatusesForSweep([]types.CustomStatus{ + {Name: "reviewing", Category: types.CategoryActive}, + {Name: "shipped", Category: types.CategoryDone}, + }) + found := map[types.Status]bool{} + for _, s := range statuses { + found[s] = true + } + if !found["reviewing"] { + t.Error("an active custom status must be scanned for citations") + } + if found["shipped"] { + t.Error("a done custom status must not be scanned: it is not a live bead") + } + if !found[types.StatusOpen] || !found[types.StatusInProgress] { + t.Error("the built-in active statuses must always be scanned") + } +} + +func TestCandidateIDMatcherWordBoundaries(t *testing.T) { + candidates := map[string]bool{ + "be-ref-001": true, + "be-ref-001.1": true, + "be-ref-002": true, + } + matcher := NewCandidateIDMatcher(candidates) + + found := make(map[string]bool) + matcher.FindAll("see (be-ref-001), be-ref-001.1 and xbe-ref-002 but not be-ref-002x", found) + + if !found["be-ref-001"] { + t.Fatal("expected be-ref-001 to match at punctuation boundaries") + } + if !found["be-ref-001.1"] { + t.Fatal("expected be-ref-001.1 to match at punctuation boundaries") + } + if found["be-ref-002"] { + t.Fatal("did not expect be-ref-002 to match inside word boundaries") + } +} + +// TestPartitionSweepReferencedBoundsTheSample pins the published cap: the +// sample is at most issueops.SweepReferencedSampleLimit long while the COUNT +// is the whole protected set, so a caller can tell a truncated sample from a +// complete one. +func TestPartitionSweepReferencedBoundsTheSample(t *testing.T) { + const protected = issueops.SweepReferencedSampleLimit + 25 + candidates := make([]*types.Issue, 0, protected+1) + referenced := make(map[string]bool, protected) + for i := 0; i < protected; i++ { + id := fmt.Sprintf("be-ref-%03d", i) + candidates = append(candidates, &types.Issue{ID: id}) + referenced[id] = true + } + candidates = append(candidates, &types.Issue{ID: "be-free"}) + + kept, count, sample := PartitionSweepReferenced(candidates, referenced) + + if len(kept) != 1 || kept[0].ID != "be-free" { + t.Fatalf("kept = %v, want only the uncited candidate", sweepCandidateIDs(kept)) + } + if count != protected { + t.Fatalf("referenced count = %d, want %d — the COUNT is the whole set", count, protected) + } + if len(sample) != issueops.SweepReferencedSampleLimit { + t.Fatalf("sample length = %d, want the published cap %d", len(sample), issueops.SweepReferencedSampleLimit) + } + if sample[0] != "be-ref-000" { + t.Fatalf("sample[0] = %q, want the first candidate — the sample keeps candidate order", sample[0]) + } +} + +// TestCandidateIDMatcherLargeFixture is the NFR-02 budget from be-5sn, moved +// here with the scan itself: 10K bodies of ~5KB each, matched against 100 +// candidates, in under five seconds. It is the matcher that made the old +// per-candidate substring scan unaffordable, so the matcher is what the budget +// is written against — the store round trips it used to include measured the +// fixture, not the algorithm. +func TestCandidateIDMatcherLargeFixture(t *testing.T) { + if testing.Short() { + t.Skip("skipping large fixture bench") + } + + const ( + bodyCount = 10_000 + candidateCount = 100 + seededRefCount = 20 + bodyPadding = 5_000 // bytes of filler per bead + maxDurationSeconds = 5 + ) + + candidates := make(map[string]bool, candidateCount) + for i := 0; i < candidateCount; i++ { + candidates[fmt.Sprintf("be-ref-%03d", i)] = true + } + + seeded := make([]string, seededRefCount) + i := 0 + for id := range candidates { + if i >= seededRefCount { + break + } + seeded[i] = id + i++ + } + + // Padding text that does NOT contain any candidate ID substring. + pad := strings.Repeat("x", bodyPadding) + + bodies := make([]string, bodyCount) + seededIdx := 0 + for j := 0; j < bodyCount; j++ { + bodies[j] = pad + if seededIdx < seededRefCount && j%500 == 0 { + bodies[j] = fmt.Sprintf("%s %s %s", pad[:100], seeded[seededIdx], pad[100:]) + seededIdx++ + } + } + + matcher := NewCandidateIDMatcher(candidates) + found := make(map[string]bool) + start := time.Now() + for _, body := range bodies { + matcher.FindAll(body, found) + } + elapsed := time.Since(start) + + // The race detector adds multi-x overhead that invalidates a wall-clock + // budget, so only enforce the bound in non-race builds. The correctness + // assertions below still run under -race, which is where the large-fixture + // pass earns its keep as a data-race check. + if !raceEnabled && elapsed > maxDurationSeconds*time.Second { + t.Errorf("scan took %v; must complete in <%ds on a %d-body fixture", elapsed, maxDurationSeconds, bodyCount) + } + + if len(found) != seededIdx { + t.Errorf("expected %d referenced IDs, got %d", seededIdx, len(found)) + } + for _, id := range seeded[:seededIdx] { + if !found[id] { + t.Errorf("expected %s to be in the reference set but it was not", id) + } + } +} + +func sweepCandidateIDs(issues []*types.Issue) []string { + ids := make([]string, len(issues)) + for i, issue := range issues { + ids[i] = issue.ID + } + return ids +} diff --git a/issue_roles_external_test.go b/issue_roles_external_test.go index 8cc96c13b..624c7f5d8 100644 --- a/issue_roles_external_test.go +++ b/issue_roles_external_test.go @@ -425,3 +425,36 @@ func TestQuerierExposesTypedUnsupportedError(t *testing.T) { t.Fatalf("Querier() error = %v, want *beads.ErrUnsupported", err) } } + +// TestSweeperKeepsTelemetryOutermost is the bulk-clearance role's version of +// the same pin, and it is the ONE WRITE role that answers the way the reads +// do: there is no on_delete hook to fire (internal/storage/hook_sweeper.go), so +// the hook decorator adds no layer and the outermost surface a caller gets is +// the instrumented one. Pinning it here is what keeps that a decision rather +// than a wrapper someone forgot to write. +func TestSweeperKeepsTelemetryOutermost(t *testing.T) { + t.Setenv("BD_OTEL_STDOUT", "true") + instrumented, ok := telemetry.WrapStorage(&dolt.DoltStore{}).(*telemetry.InstrumentedStorage) + if !ok { + t.Fatal("WrapStorage() did not create InstrumentedStorage") + } + + sweeper, err := storage.NewHookFiringStore(instrumented, nil).Sweeper() + if err != nil { + t.Fatalf("Sweeper() error = %v", err) + } + if got := reflect.TypeOf(sweeper).String(); got != "*telemetry.instrumentedSweeper" { + t.Fatalf("outer layer = %s, want the telemetry wrapper unwrapped by the hook decorator", got) + } +} + +func TestSweeperExposesTypedUnsupportedError(t *testing.T) { + sweeper, err := (*dolt.DoltStore)(nil).Sweeper() + if sweeper != nil { + t.Fatalf("Sweeper() sweeper = %T, want nil", sweeper) + } + var unsupported *beads.ErrUnsupported + if !errors.As(err, &unsupported) { + t.Fatalf("Sweeper() error = %v, want *beads.ErrUnsupported", err) + } +} diff --git a/issueops/reader.go b/issueops/reader.go index ab340d012..be8d6a117 100644 --- a/issueops/reader.go +++ b/issueops/reader.go @@ -469,14 +469,14 @@ type IssuePage struct { // types.WorkFilter there at all, so no filter is writable there either — both // are directory-scoped with no per-file exception, so a file added to that // package tomorrow is covered the moment it exists. That same forbidigo rule -// covers cmd/bd deny-by-default with 61 named exceptions, so the files +// covers cmd/bd deny-by-default with 59 named exceptions, so the files // implementing `bd list` and `bd show` cannot write a filter, and neither can // a file they are split or renamed into unless the new name lands on that // list. NOT ENFORCED: the rule forbids NAMING those types, not holding a // value, so the property is "no filter is written there", not "every filter // there came from a builder"; test files are exempt from both rules, because // the oracles hold filters in order to inspect them; `bd ready`'s files are -// among the 61, since its listing and --claim are handed the filter itself and +// among the 59, since its listing and --claim are handed the filter itself and // the blocked-issue views in those files name one directly, so it is guarded by // the builder and the golden files and not by the linter; // GET /healthz and GET /v0/beads/context are not issue queries and are on no diff --git a/issueops/sweeper.go b/issueops/sweeper.go new file mode 100644 index 000000000..c03c0da38 --- /dev/null +++ b/issueops/sweeper.go @@ -0,0 +1,281 @@ +package issueops + +import ( + "context" + "time" +) + +// SweepTier names which of the two planes a sweep clears. It is REQUIRED on +// every request and has no default, for the reason RelatedRequest.Direction +// has none: the two tiers are disjoint sets of rows reached by one operation, +// and a caller handed the wrong one has nothing to notice until the rows are +// gone. +// +// The two are DISJOINT by construction — a sweep of one tier can never touch a +// row of the other — which is what lets `bd purge` and `bd prune` be one +// capability with a parameter rather than two capabilities that resemble each +// other. +type SweepTier string + +const ( + // SweepEphemeral clears CLOSED rows of the wisp tier: `bd purge`. + // Ephemeral rows accumulate fast, carry no long-term value once closed, + // and live in tables the version-control plane ignores. + SweepEphemeral SweepTier = "ephemeral" + // SweepDurable clears CLOSED rows of the issue tier: `bd prune`. This is + // the tier a workspace's real history lives in, which is why it — and only + // it — carries the require-a-filter refusal SweepRequest.ClosedBefore + // describes. + SweepDurable SweepTier = "durable" +) + +// SweepRequest describes one bulk clearance of closed rows: which tier, which +// of its closed rows, and whether to do it or only report it. +// +// IT IS NOT A FILTER-SHAPED READ REQUEST with a delete attached. The predicate +// is FIXED at "closed rows of one tier" and the two fields below only narrow +// it; there is no status, no assignee, no label and no free-text query, +// because every one of those would be a way to spell a destructive selection +// that no front door asks for and that a caller could get subtly wrong. What +// this request can express is exactly what `bd purge` and `bd prune` expose, +// and widening it is a decision rather than an omission to be filled in. +// +// Implementations never mutate caller-owned request values: ClosedBefore is +// read, never written through. +type SweepRequest struct { + // Actor attributes the sweep wherever the backend records attribution. It + // is not an authorization boundary — nothing here checks who the caller + // is — and an empty Actor is accepted, because the deletion of a row + // leaves no row to attribute it on. + Actor string + // Tier selects the plane. It is REQUIRED: the zero value is ErrValidation + // rather than a default, and an unrecognized value is ErrValidation rather + // than a sweep that matches nothing. + Tier SweepTier + // ClosedBefore keeps only rows closed STRICTLY BEFORE this instant. A row + // whose stored closed_at is exactly this instant is kept, which is the + // half-open interval every other time bound in this package uses. + // + // It is one half of the DURABLE TIER'S SAFETY GATE: a SweepDurable request + // with neither ClosedBefore nor IDPattern set is ErrValidation, never an + // unfiltered mass delete. That refusal lives HERE, below every front door, + // rather than in the CLI handler that historically owned it. The gate is a + // safety invariant and not presentation: a second front door — an HTTP + // operation, a library embedder — that inherited the capability without + // the guard would be one handler away from erasing every closed issue in a + // workspace, and a guard that each caller must remember is a guard that + // one caller will not. A caller that really does mean "everything closed" + // says so with IDPattern "*", which is a deliberate keystroke rather than + // an omitted one. + // + // The EPHEMERAL tier carries no such gate: an unfiltered sweep of closed + // wisps is the ordinary use of `bd purge`, and the tier is transient by + // definition. + ClosedBefore *time.Time + // IDPattern keeps only rows whose ID matches this shell glob, in the + // syntax of path/filepath.Match (`*`, `?`, `[...]`, `\` escapes; `*` also + // crosses `-` and `.`, since an id is not a path). Empty matches every row + // in the tier. + // + // A MALFORMED PATTERN IS ErrValidation, not a pattern that matches + // nothing. Both front doors previously discarded filepath.Match's error, + // so `--pattern '['` reported "no beads to prune" — indistinguishable from + // a correct pattern over an empty set, on a command whose whole job is to + // delete what it matched. + // + // It is the other half of the durable tier's gate: see ClosedBefore. + IDPattern string + // ProtectReferenced skips candidates whose ID is CITED by a row that is + // not done — in its description, its notes, or any of its comments — so a + // decision trail a live bead still points at is not deleted out from under + // it. + // + // WHAT COUNTS AS A CITATION is a literal occurrence of the id at ASCII + // word boundaries: `be-1` matches in "see (be-1)." and does not match + // inside `xbe-1` or `be-12`. It is a text scan, not a link: an id spelled + // in prose protects, and a stored dependency edge does not (a sweep + // removes edges pointing at what it deletes — see SweepResult). + // + // WHICH ROWS ARE ASKED is every row whose status is not a done one: the + // built-in active statuses plus every configured custom status whose + // category is not "done". Enumerating the workspace's custom statuses is + // REQUIRED rather than best-effort, because a failure to read them would + // under-scan and delete a bead a live custom-status bead still cites; an + // implementation that cannot read them fails the sweep instead. + // + // It costs a full scan of the not-done set and of that set's comments, so + // it is opt-in per request rather than always-on. `bd prune` asks for it; + // `bd purge` does not, because a wisp's citations are as transient as the + // wisp. + ProtectReferenced bool + // DryRun reports what the sweep WOULD do and deletes nothing. The result + // is otherwise the same result — the same counts, the same skips, the same + // refusals — computed against the same snapshot the real sweep would have + // used, which is what makes a preview worth reading. + DryRun bool +} + +// SweepSkips reports the candidates a sweep declined to delete, bucketed by +// WHY. Every bucket is a row the tier's predicate admitted and the sweep then +// held back, so the buckets and SweepResult.Swept describe one candidate set. +// +// They are separate counters rather than one number because they mean +// different things to the person reading them: Pinned and Referenced are +// PROTECTIONS a user can override or re-express, and the other three are the +// sweep declining to trust its own input. +type SweepSkips struct { + // Pinned counts candidates protected by the pinned flag. Pinning is the + // workspace's own "never sweep this", and no request field overrides it — + // a caller that wants a pinned row gone unpins it first. + Pinned int + // Referenced counts candidates skipped by ProtectReferenced. It is always + // 0 when that field is false; a caller that reads a 0 without having asked + // for the protection has learned nothing about whether rows are cited. + Referenced int + // NotClosed, UnknownClosedAt and ClosedAtOrAfterCutoff count candidates + // the tier's own query returned and the sweep rechecked and rejected: a + // status that is not closed, a closed row with no closed_at stamp at all, + // and a closed_at that does not satisfy ClosedBefore. + // + // A NON-ZERO VALUE IN ANY OF THESE THREE IS A DEFENSE FIRING, not a normal + // outcome, because the query was asked for exactly the rows they exclude. + // They are published rather than silently dropped so an operator can see + // that a delete-everything-closed query and a delete-everything-closed + // recheck disagreed, which is the state in which nothing should be + // deleted on trust. + NotClosed int + UnknownClosedAt int + ClosedAtOrAfterCutoff int + // Unreadable counts rows the tier's query returned as nothing at all. It + // is a defense of the same kind, on a shape rather than a value. + Unreadable int +} + +// SweepResult reports one sweep. +// +// EVERY NUMBER DESCRIBES THE SAME SNAPSHOT, because the whole sweep — the +// candidate query, the recheck, the reference scan and the deletion — runs in +// ONE transaction. See Sweeper.Sweep. +type SweepResult struct { + // DryRun echoes the request. It is here so a result value carries whether + // its numbers describe rows that are gone or rows that would go, without + // the caller having to keep the request beside it. + DryRun bool + // Swept is how many rows were deleted, or — under DryRun — would be. + Swept int + // Dependencies, Labels and Events count the ROWS OF ASSOCIATED DATA the + // sweep removed with them: edges pointing at or out of a swept row, its + // labels, and its recorded events. They are reported because a sweep's + // visible effect is much larger than its row count, and because they are + // the number that explains a large storage reclaim from a small Swept. + Dependencies int + Labels int + Events int + // Skipped reports the candidates the sweep held back and why. + Skipped SweepSkips + // ReferencedIDs is a BOUNDED SAMPLE of the ids Skipped.Referenced counts — + // at most SweepReferencedSampleLimit of them, in the order the candidate + // query returned them. It is a sample and not the set: a caller that needs + // every protected id re-runs with ProtectReferenced false and a DryRun to + // see the difference, or narrows the request. + // + // It is empty when nothing was protected, and never longer than the count + // beside it. + ReferencedIDs []string +} + +// SweepReferencedSampleLimit bounds SweepResult.ReferencedIDs. It is published +// so a caller can tell a truncated sample from a complete one by comparing the +// slice's length against it, rather than against a number copied out of an +// implementation. +const SweepReferencedSampleLimit = 100 + +// Sweeper describes bulk clearance of CLOSED rows — the capability behind +// `bd purge` (the ephemeral tier) and `bd prune` (the durable one) — and, like +// every other capability here, a role with its own accessor. A new capability +// gets a new role interface and its own accessor; never append a method here. +// +// ONE CAPABILITY, NOT TWO. purge and prune name one operation over two +// disjoint tiers: the same candidate query, the same pinned protection, the +// same closed_at recheck, the same deletion. The tier is a REQUEST FIELD +// because that is what it is — a selection — and splitting it into two roles +// would have produced two interfaces over one body, which is the accretion the +// governing rule exists to prevent in the other direction. What genuinely +// differs between them is expressed as request fields too: the durable tier's +// require-a-filter refusal keys on Tier, and the reference protection is +// asked for rather than implied. +// +// IT IS A DIFFERENT QUESTION FROM ITS NEIGHBORS. +// +// - Reader answers with rows. This answers with an EFFECT, and its +// result is a tally of what it did — there is no page of it, no cursor +// into it, and asking twice does not answer twice. +// - Counter answers "how many match". A sweep matches and then ACTS, and +// the two cannot be composed by a caller without reintroducing exactly the +// window this role closes: count, then delete what you counted, is two +// transactions. +// - Lifecycle patches ONE issue the caller names. A sweep names none: it +// describes a set and the implementation resolves it. That is the whole +// reason the safety gate can live here at all — there is a selection to +// refuse. +// +// WRITES, AND THE ONLY DESTRUCTIVE ROLE IN THIS PACKAGE. A sweep deletes rows +// and everything that hangs off them, and nothing it deletes comes back. Its +// completion fires the hook the front doors already fire for a bulk change; +// its refusals change nothing at all. +// +// Deterministic request-validation failures match ErrValidation. Result values +// are unspecified when error is non-nil, with one deliberate exception stated +// on Sweep. +type Sweeper interface { + // Sweep deletes the closed rows of req.Tier that req's two narrowing + // fields admit, and reports what it did. + // + // IT IS ONE TRANSACTION, and that is the central promise of this role + // rather than an implementation note. The candidate query, the pinned and + // closed_at rechecks, the reference scan and the deletion all see one + // snapshot, so: + // + // the set this result describes IS the set it deleted + // + // A row created, closed, unpinned or cited between the selection and the + // deletion cannot change which rows go. The direct CLI route did not have + // that property before this role existed — it selected in one transaction + // and deleted in another, so a bead closed in the window was reported and + // not deleted, and a bead pinned in the window was deleted after being + // judged unpinned. + // + // WHAT THAT COSTS, said out loud because it is a real trade and not a free + // win: a sweep is ALL OR NOTHING. A sweep large enough to exceed a + // backend's write timeout fails whole and deletes nothing, where a + // batching implementation would have deleted some of it. That is the + // better of the two failures — the partial one previously reported a + // deleted count of zero while having deleted rows — but a caller sweeping + // a very large set narrows the request rather than expecting progress. + // + // THE ORDER THE NARROWING HAPPENS IN IS PART OF THE ANSWER, because the + // skip counters are counted along the way: the tier's closed rows, then + // IDPattern, then the pinned and closed_at rechecks, then the reference + // protection. A pinned row excluded by the pattern is therefore NOT + // counted in Skipped.Pinned — it was never a candidate — and the counters + // describe the set the request actually reached. + // + // REFUSALS, all ErrValidation and all before anything is read: + // + // - an unset or unrecognized Tier; + // - a SweepDurable request with neither ClosedBefore nor IDPattern (see + // SweepRequest.ClosedBefore for why this lives here); + // - an IDPattern that is not a well-formed glob. + // + // A REQUEST THAT MATCHES NOTHING IS A ZERO RESULT AND A NIL ERROR, not a + // not-found: an empty set of closed rows is the steady state of a swept + // workspace, and it is the answer a scheduled sweep gets every time it + // runs after the first. + // + // A DRY RUN CHANGES NOTHING, including history: an implementation that + // records a version-control entry for a sweep records none for a dry run + // and none for a sweep that deleted nothing. Where an implementation + // versions at all, one sweep records AT MOST ONE entry — the deletion is + // one act, not one per row. + Sweep(ctx context.Context, req SweepRequest) (SweepResult, error) +}