beads/.github/scripts
Julian Knutsen 9c4b3d8bab
perf(externaldeps): annotate a page's blocking state in one pass, not one IsBlocked per id (#7556)
* perf(externaldeps): annotate a page's blocking state in one pass, not one IsBlocked per id

The external-dependency BlockingAnnotator overlaid IsBlocked on the
backend's answer one item at a time. Each IsBlocked made several reads
(the stored flag, both edge tables, the blockers' statuses), read the
item's own edges again, and resolved its external refs on its own, so a
foreign project was opened once per item. Over a remote serve,
listBlockingAnnotations cost 9-20 ms per id, and a 1,400-id refresh took
about 12 s.

The annotator now reads the whole page once:
- the local half through a new optional storage.BlockersBatchStore,
  IsBlockedWithBlockersBatch, backed by
  issueops.IsBlockedWithBlockersBatchInTx (the stored flags, the blocked
  ids' blocking edges from both tables, one status read). The dolt and
  embedded dolt stores implement it, and a store without it still gets
  one IsBlocked per id;
- the external half from ONE read of the page's own edges, whose refs
  are resolved together, so each foreign project is opened at most once
  per request.

The answer per id is IsBlocked's, unchanged: a blocked item's BlockedBy
is the local live blockers, then the unsatisfied refs. Both reads now
order a source's blocking edges by (depends_on_id, type). The single
read had no ORDER BY, so its order was the table's, which is a UUID key
on these tables.

Tests:
- TestBlockingAnnotatorReadsAPageOnce pins 1 own-edge read, 1 foreign
  open, 1 batch read and 0 per-id reads for 40 ids. On the old body it
  failed with 40, 30, 0 and 40.
- TestBlockingAnnotatorFallsBackPerIDWithoutTheBatchRead covers a store
  without the batch read.
- TestPolicyBlockingAnnotatorBatchEqualsPerItem (embedded dolt) compares
  the batched answer, and IsBlockedWithBlockersBatch, against the
  per-item path by deep equality. It covers local blockers of every
  blocking type, closed and pinned blockers, parent-chain blocking,
  waits-for and conditional-blocks gates on both planes, satisfied,
  unsatisfied and unconfigured external refs, a non-blocking external
  edge, local plus external, a free row and a missing id.
- TestPolicyBlockingAnnotatorForeignOpensAreSublinear logs N=10 at 1.9 s
  per-item vs 0.26 s batched, and N=40 at 7.5 s vs 0.29 s. Foreign opens
  go from N to 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(dolt): pin server-mode IsBlockedWithBlockersBatch against IsBlocked

Review apply-fixes for #7556, iteration 1. Two minors are taken here, the
merge conflict is left to the pre-approval rebase, and two items are
declined. All changes are a test and a doc comment; no runtime code moves.

Taken: server-Dolt test for the batch arm (Test Evidence Quality)
- TestIsBlockedWithBlockersBatchMatchesPerRow runs the DoltStore arm that
  bd serve and server-backed bd list take, which no test executed before.
  It compares each id's entry with IsBlocked through reflect.DeepEqual.
  The fixture reuses the embedded fixture's shapes as the raw store sees
  them: blocks, a parent-child chain (the list is nil), waits-for,
  conditional-blocks, a wisp source and a wisp blocker, closed and pinned
  blockers, and external refs (blocks and related), which the raw store
  drops. It also covers a free row, a missing id and repeated ids. Two more
  cases reach code the embedded test does not:
  - one id that exists in both issues and wisps, with an edge in each table;
  - 210 blocked ids with 210 distinct blockers, more than queryBatchSize
    (200).
- Mutants were applied to the PR's code with the test binaries rebuilt.
  Five fail the new test and pass the existing embedded
  TestPolicyBlockingAnnotator* tests:
  - edge-chunk: read only the first edge chunk;
  - status-chunk: read only the first status chunk;
  - no-dedup: drop the repeated-id check;
  - table-order: read wisp_dependencies first;
  - server-wrapper: the DoltStore wrapper returns an empty map.
  A sixth mutant, no-order (drop ORDER BY), fails both the new test and the
  existing embedded batch-equals-per-item test.

Taken: BlockersBatchStore doc (Contract and Interface Fidelity)
- The doc now says the interface answers for the raw store alone, with no
  external-dependency policy applied. It is meant to be called from inside
  that policy's decorator, not probed through UnwrapStore by a caller that
  holds the decorated store.

Deferred to the pre-approval rebase: conflict with #7552 (role_accessors.go)
- This change does not rebase. On a scratch merge with main 70887e708a,
  this resolution was validated:
  - keep this PR's
    state := states[id]; if state.blocked { BlockedBy = state.blockers };
  - nest #7552's IncludeIsBlocked flag (IsBlocked = boolPtr(true)) inside
    that branch;
  - keep boolPtr.
- Results on that merged tree:
  - go vet: clean;
  - go test ./internal/storage/externaldeps/: 66 pass, including
    TestBlockingAnnotatorFlagsUnsatisfiedExternalRef;
  - embedded BlockingAnnotator|IsBlocked tests: 22 pass;
  - served httpclient Blocking|Blocked|ExternalDependency|Ready tests with
    BEADS_HTTP_TEST_REQUIRED=1: 96 pass, plus 4 known-divergence skips that
    predate this change.

Declined
- Duplication (route blockedStates' external half through
  blockingStateFromRecords): this refactors contributor code, adds the
  "external dependencies: resolve blockers:" error prefix, and is not
  test-only. Recorded as a follow-up.
- Pattern Drift nit (move the guards next to their methods, rewrap the
  AnnotateBlocking comment): optional churn next to the conflict region.
  Recorded as a follow-up.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:52:37 +00:00
..
bazel-farm-authorize.sh ci(bazel): remote-execution Bazel lanes for allowlisted fork authors (advisory) (#6994) 2026-09-30 14:54:06 -07:00
bazel-gate.sh ci(bazel): cut per-lane fixed cost — LLVM .tar.zst slice, release cross-compile lane, no protoc build (#7379) 2026-10-08 03:13:34 +00:00
ci-gate.sh Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
docs-render-check.sh chore: purge deleted docs-pipeline references from release tooling, CI, and repo plumbing 2026-07-12 11:07:26 -07:00
embedded-cmd-test-shards.txt fix(http): bd promote over http through issueops.Promote and issues.update.persistence (S21b) (#7554) 2026-10-14 05:51:53 +00:00
embedded-storage-test-shard.sh feat(httpapi): opt-in large issues:batchApply envelope (≤1000 items, one transaction) (#7051) 2026-10-01 08:32:56 -07:00
embedded-storage-test-shards.txt perf(externaldeps): annotate a page's blocking state in one pass, not one IsBlocked per id (#7556) 2026-10-14 11:52:37 +00:00
embedded-test-shard.sh Rebalance embedded cmd CI shards (bd-8zg) (#3499) 2026-04-25 17:58:51 -07:00
proxied-cmd-test-shards.txt fix(http): bd show --current, recompute-blocked and search over http through issueops entries (S6e/S25/S26) (#7528) 2026-10-13 04:12:54 +00:00
proxied-test-shard.sh fix(dolt): server-mode bd ready never wakes expired defers (re-proposes #6082) (#6424) 2026-09-30 15:53:22 -07:00
rrc-lane-commands.txt ci(bazel): dolt-int gates server-storage + cmd-dolt in mode remote (item 5 A2) (#7535) 2026-10-13 09:17:54 +00:00
rrc-run-lanes.sh ci(bazel): remote repo contents cache: main-push seed, lane readers, nightly verify (#7453) 2026-10-10 02:50:26 +00:00
rrc-writer-credential.sh ci(bazel): remote repo contents cache: main-push seed, lane readers, nightly verify (#7453) 2026-10-10 02:50:26 +00:00
server-storage-test-shard.sh ci: PR-CI lane runs the full internal/storage/dolt suite against a live Dolt server (#5836) 2026-08-19 20:39:38 -07:00
server-storage-test-shards.txt docs(ci): update shared cloud routing shard rationale (#6432) 2026-09-27 01:57:07 -07:00