beads/engdocs/design
Julian Knutsen 3c30a28d07
perf(ready): answer the page and its total in one pass (#6731)
* perf(issueops): answer a ready page and its total in one pass

`bd ready --json --limit 1` against a remote SQL store (~55 ms per round
trip) took 5.4 s and ~100 sequential round trips; `--limit 0` (557 rows)
took 3.8 s. Asking for one row was slower than asking for all of them. The
data is small, so the cost was purely statement count:

- a capped page re-ran the whole ready predicate in a SECOND transaction
  (CountReadyWorkInTx, behind a second defer-wake sweep) just to print
  "Showing 1 of 556";
- every leg re-probed the same facts: wisp_dependencies existence, wisps
  emptiness, and the future-deferred-parent scan (2 probes + 4 child
  queries), once per table family and again in the count pass.

This change:

- Adds a per-transaction probe (probeReadyWorkInTx): one statement of
  EXISTS subqueries answers wisps-present, wisp_dependencies-exists,
  future-deferred-parent-exists and (when sizing) issues/wisps ID collision;
  the deferred-parent children are then read in one UNION ALL instead of
  four queries. The facts and the rendered ID sets are shared by both
  families. A missing-table error (pre-wisp database) falls back to the old
  per-table probes, so their tolerance and error classification are
  unchanged.
- Adds GetReadyWorkWithCountsAndTotalInTx and
  DoltStorage.GetReadyWorkWithCountsAndTotal: the capped page's ID query
  carries COUNT(*) OVER (), evaluated over the full predicate before LIMIT,
  so the total costs no extra statement. The overlap count only runs when
  the probe saw an ID in both tables. Window functions are supported by
  every engine this code runs on (Dolt/go-mysql-server server and embedded,
  MySQL 8, Postgres, SQLite >= 3.25). The externaldeps decorator overrides
  the new method so its exclusions apply to page and total alike, and now
  also overrides ReadyCounter() so the text-mode total honors the same
  exclusions (it used to pass through to the inner store and count
  externally blocked issues); telemetry instruments the new method.
- CountReadyWorkInTx (ReadyCounter role, text `bd ready`) uses the probe
  and counts both families in one statement.

Whole-command statements for `bd ready --json`, counted from a local dolt
sql-server query log (13 of each are startup/config reads this change does
not touch):

  store                          --limit 1   --limit 0
  issues only                    28 -> 18    20 -> 17
  issues + wisps                 34 -> 19    23 -> 18
  issues + wisps + deferred epic 46 -> 20    29 -> 19

TestReadyWorkStatementBudget holds the per-transaction statement count at
or under today's budget on embedded Dolt. TestReadyWorkPageTotal and
TestReadyWorkPageTotalFilterMatrix check the page and total against
hand-written expected ready sets (wisps, deferred parent, blocked, ID
collision; sorts, label/assignee/priority/type/parent/deferred/exclude
filters) and every capped page against the unbounded listing's prefix.

stdout (plain and BD_JSON_ENVELOPE=1) and the stderr hint were
byte-identical before/after for every case above. The one intended output
change: with an external dependency present, the --json total (and the
text "of N") now excludes externally blocked issues, as the page already
did; before, the total counted them.

Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(ready): take the --json total in-band instead of a second count pass

`bd ready --json --limit N` listed a page, then - when the page came back
full - opened a second store call (ReadyCounter.CountReady: a new read
transaction plus its own defer-wake sweep) only to print "Showing N of M"
on stderr and the pagination total in the JSON envelope. On a remote store
(~55 ms per round trip) that pass is why `--limit 1` (5.4 s) was slower
than `--limit 0` (3.8 s).

The --json branch now calls GetReadyWorkWithCountsAndTotal, which returns
the page and the size of the whole ready set from the same transaction.
The total is the same identity the ReadyCounter role promises, taken over
the listing's own filter, so the hint text, the envelope's pagination and
the stdout JSON are identical except that the total now honors
external-dependency exclusions (the externaldeps decorator applies them to
page and total alike; the old second pass counted externally blocked
issues). The text-mode branch still asks the ReadyCounter role, which the
previous commit made cheaper and made apply the same exclusions, so text
and --json report one number.

The embedded CLI test now pins the exact "Showing 2 of N" hint and the
envelope's pagination total against an unbounded listing.

Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 23:50:09 -07:00
..
otel perf(ready): answer the page and its total in one pass (#6731) 2026-09-24 23:50:09 -07:00
bd-serve-v0.md feat(httpapi): serve bd list's order with sort on listIssues (#5666) 2026-08-27 09:21:12 -07:00
dolt-concurrency.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
external-capability-dependencies.md fix(deps): enforce external capability blockers across ready/claim/close (#4769) (#4753) 2026-09-02 16:56:23 -07:00
kv-store.md docs: move internal/maintainer docs from docs/ to engdocs/ 2026-07-12 10:37:07 -07:00
ownership-handoff-contract.md release: make release/1.3.0 tag-ready — revert #6281, stamp 1.3.0, date the changelog (#6564) 2026-09-15 12:13:59 -07:00