fix(storage): a parent-child edge propagates only the parent's exogenous blockedness (#6506) (#6602)

* fix(storage): parent-child cascade propagates only exogenous blockedness (wy-3eb07b, #6506)

A parent-child edge used to cascade the parent's is_blocked bit to its
children WHATEVER set it. So the "close gate on the epic" idiom — P carries
blocks edges onto its own children C1 and C2 so P cannot close before them —
darkened the very children P was waiting for. Neither child could reach
bd ready, so neither could be worked, so neither could close: a permanent
lock, and the one direction cmd/bd/dep.go does not refuse.

The contract (#6506): a parent-child edge propagates only EXOGENOUS
blockedness. P counts as blocked FOR CHILD C iff P has a blocking reason —
an open blocks/conditional-blocks target, or a live waits-for gate — whose
target is NOT one of P's own parent-child children, or P's own parent is
(recursively) exogenously blocked. P's OWN is_blocked is unchanged: it really
does depend on open children. Only the propagation narrows.

DESIGN: a second-pass predicate on the parent row, not WITH RECURSIVE.
Both parent-child legs of shouldBeBlockedIDsUnionScopedSQL keep their
p.is_blocked = 1 conjunct and gain parentCascadesBlockedSQL, which says "p's
blockedness is not solely explained by p's own children":

    EXISTS (blocking reason of p whose target is not p's child)
      OR NOT EXISTS (blocking reason of p whose target IS p's child)

The second disjunct carries the contract's recursive clause through the
EXISTING fixpoint instead of through SQL recursion: a parent that is blocked
with no child-targeted reason of its own is dark either exogenously or
because ITS parent is, and either way its children inherit — one level per
pass, exactly as the cascade always converged.

A recursive CTE was the other candidate and is rejected on cost, not on
engine support. Its seed set is the direct-exogenous set, which includes the
waits-for gate, and the seed cannot be confined by the batch scope (%[3]s)
because a batch's ancestors are by definition outside the batch. That would
put an UNSCOPED whole-table gate evaluation into every batched mark/unmark —
the statement shape #6288 removed, now on every write instead of only on the
full repair. The predicate here is correlated to a parent that is already
known blocked, so it probes idx_dependencies_issue for a handful of rows.
The scope splice, the DISTINCT waits-for barrier and the doctor-count
lockstep are all untouched.

One gap this form does not express: a parent that is BOTH blocked by its own
open child AND darkened by an exogenously blocked ancestor is not
distinguishable from one blocked only by its children, so its children stay
bright. That is an under-propagation in a corner the contract would darken;
it needs a second stored bit (or the CTE above) to see, and it cannot
recreate the lock this commit removes.

Also: AffectedByDepChange{,ForWisp}InTx now seeds the parent's other children
on a parent-child edge change (appendSiblingsUnderParentInTx). Writing that
edge reclassifies the parent's blocking reason on the new child from
exogenous to child-derived, which flips the answer for the SIBLINGS — the
close-gate epic is built one child at a time, and the second edge is what
un-darkens the first. Without it the incremental write path and the full
repair disagree, which the new tests catch as a nonzero doctor count.

Tests (internal/storage/dolt/blocked_consistency_test.go), all four seeded
through the real write path and each verified RED against the pre-fix SQL
except the control:
  - ParentBlockedOnlyByItsOwnChildren: the repro, plus a GetReadyWork
    assertion that the children are workable again.
  - ExogenousParentStillCascades: the control — the fix narrows the cascade,
    it does not remove it. Green before and after.
  - ThreeLevelChainBothWays: GP exogenous darkens P and C; GP blocked only by
    its child P darkens neither. Repaired from an inverted plane too, since
    the chain is the case that needs more than one pass.
  - WaitsForGateOverOwnChildren: a gate whose spawner is the parent's own
    child does not darken; one on an outside spawner does.

TestRecomputeAllIsBlocked_CascadesThroughParentChild is unchanged and green,
including its single-pass lower-bound count — which is why the legs keep the
p.is_blocked = 1 conjunct rather than deriving the cascade from the graph
alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(dep): warn when a parent blocks on its own child (wy-3eb07b, #6506)

`bd dep add P C` where C is already P's parent-child child is the "close gate
on the epic" idiom: P carries a blocks edge onto its own child so P cannot
close before it. dep.go refuses the OTHER direction (a child blocking on its
dotted-ID parent, "would create a deadlock") and says nothing about this one,
which is the direction that used to deadlock for real — the cascade darkened
the child, so it could never reach bd ready, be worked, or close.

Since the cascade fix the edge is harmless, so this is an advisory, not a
refusal: it names what the edge now means (the parent stays blocked; the
child and its siblings are NOT hidden from bd ready) so an operator does not
have to rediscover the contract from a query.

It fires only for blocks/conditional-blocks, only after the edge has actually
landed and committed, and stays silent on any read error — an advisory must
not turn a successful write into an error, and must not precede a refusal for
an edge that was never written (the hierarchy and cycle guards reject this
pair whenever both edges already exist, in either order, which is why the
shape usually arrives one edge at a time or through import).

Unlike the D1 implicit-blocks warning it is NOT gated on an interactive
stderr: that one fires on the documented-default majority path, where
training callers to ignore stderr is the risk; this one fires on a rare
deliberate shape, and an agent wiring a close gate is exactly the caller who
should read what it does.

Wired into both direct paths (`bd dep add` and `bd dep --blocks`); the
proxied-server paths return earlier and are untouched.

Tests beside the existing dep-add warning tests: the predicate table
(including the two controls — right id with a blocks edge is not a child, and
the parent's other children do not match), the message content, and the type
gate proving it returns before touching the store.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(doctor): report parent-blocks-on-own-child edges (wy-3eb07b, #6506)

A new "Parent Close Gates" check lists every blocking edge from a parent onto
one of its own parent-child children — the close-gate idiom, where P blocks
on C1 and C2 so P cannot close before them.

INFORMATIONAL, with no --fix and no effect on the overall result. Before the
cascade fix these edges were a trap: the parent's blocked bit propagated to
its children whatever set it, so the gate hid the very children it was
waiting for and nothing under it could become ready. The cascade now
propagates only exogenous blockedness, so the edges are legal and usually
deliberate. What an operator wants from doctor is therefore the inventory —
which beads in this store use the idiom, and where a pre-6506 store's
mysteriously invisible children came from — not a repair, and certainly not a
warning that would train people to remove a working gate.

It is the mirror of the existing "Child-Parent Dependencies" check, which
finds a CHILD blocking on its PARENT (still a deadlock, still fixable). The
two differ in how they match: that one reads dotted-ID ancestry, this one
matches real parent-child edges, because the close gate is wired between
unrelated ids.

Tests beside the child-parent ones: the empty inventory (with an ordinary
hierarchy present, which must not be mistaken for a gate), and the reported
case with two controls in the same fixture — an exogenous blocker on the
parent and an ordinary blocker on the child, neither of which may appear.

Gate note: these live in package cmd/bd/doctor, which the brief's cmd/bd
selector does not reach; `go test ./cmd/bd/doctor/` was run in full.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* revert(dep): drop the unreachable parent-blocks-own-child warning (wy-3eb07b, #6506)

This reverts 34b7f5b42. The warning it added could never print.

Its precondition is "a blocks edge from P onto C succeeded AND C is already
P's parent-child child", and the store refuses exactly that edge.
CheckBlockingHierarchyInTx (internal/storage/issueops/dependencies.go) rejects
a blocker that is the source's own descendant, and it sits OUTSIDE the
SkipCycleCheck gate, so no caller opts out of it; batch_apply.go:533 applies it
again on the DependencyEditor path `bd dep add` actually uses. Written in the
other order, the parent-child edge that closes the pair is refused as a cycle.
So the reverted code paid a GetDependentsWithMetadata read on EVERY
blocks/conditional-blocks `bd dep add` to emit something no operator could
ever see, and its 102-line test pinned a predicate instead of a behaviour.

What an operator DOES hit is the refusal — and the cascade fix left its stated
reason false. It read "blocked status cascades to descendants, so <blocker>
would inherit the block and never close". Since #6506 a parent-child edge
propagates only the parent's exogenous blockedness, so the descendant does not
inherit and nothing deadlocks. The refusal still stands, because a blocks edge
is not how the hierarchy says "I close after my children" — so the reason now
names the form that is: a waits-for gate over the children, `bd gate`.

The same false sentence was the OpenAPI description of
Problem.blocker_is_ancestor, so it is corrected there too, with types.gen.go
regenerated by `go generate ./internal/httpapi/apigen/` to keep `make
api-check` green.

The doctor "Parent Close Gates" check from 4c5f2f77b is KEPT. A warning on a
write path that refuses the write is noise; an inventory of the edges a store
already carries — from import, from merge, from the pre-refusal past — is what
an operator can act on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(storage): the parent-child exogeneity test is over the whole SUBTREE (wy-3eb07b, #6506)

B1 of the adversarial review: 648ab0af6 tested "is this blocking reason's
target one of my own children" as a DIRECT parent-child child, but #6506's
contract is about blockedness originating outside the parent's own SUBTREE.
At depth 1 the fix worked; at depth 2 it left the permanent lock alive.

  P --blocks--> G,  pc C -> P,  pc C2 -> P,  pc G -> C

G is not P's child, so P read exogenous, so P darkened C, so C darkened G —
and G is the only row whose close can free P. Exactly the shape-1 mutual lock
the issue says no contract intends, one level down. Pinned as
TestParentChildCascade_ParentBlocksItsOwnGrandchild, verified RED at
4c5f2f77b: p/c/c2/g all blocked, all three children missing from ready work,
and the full repair changing nothing.

The test is now a real ancestor walk (subtreeWalkDepth levels of indexed
LEFT JOINs up from the reason's target), so a reason anywhere under the
blocker counts as its own.

Also B2 — the cost regression, which is the same edit, because the shape that
makes the test correct and the shape that makes it affordable are one
decision. The union's parent-child legs no longer carry a per-row correlated
EXISTS. They subtract an uncorrelated set instead:

    AND p.id NOT IN (parents explained by their own subtree)

and the UNBATCHED statements (full repair, doctor count) do not even spell
that set as a subquery: it is read ONCE per fixpoint pass and bound as ids.
A repair pass is four statements and each would otherwise derive the same set
twice; reading it once is the whole difference. The batched mark/unmark keep
the spliced subquery, where it is confined to the batch's own parents by the
existing scope splice and costs a few rows.

MEASURED, gastownhall/beads#6506 bar: full repair and doctor count <= 1.5x
origin/main, write path <= 1.2x. Committed benchmark
(internal/storage/dolt/blocked_scale_bench_test.go, -short-skipped), 1000
issues / 2870 deps / 333 blocked rows, embedded dolt 2.2.0, run INTERLEAVED
main/head to cancel machine drift, four pairs, medians:

                             origin/main 71c4cd08b   this branch   ratio
  full repair                 48 71 43 46 -> 47 ms   66 56 54 55 -> 55 ms  1.18x
  doctor count                55 62 44 53 -> 54 ms   67 61 57 57 -> 57 ms  1.06x
  write path, per CloseIssue  37 30 21 22 -> 26 ms   30 23 16 24 -> 23 ms  0.91x

All three inside the bar; the worst single PAIR is 1.38x on the repair and
1.30x on the count. The two sides disagree on blocked rows by construction
(604 on main, 333 here) — that IS the fix.

NOT inside a bar, and disclosed rather than hidden: the benchmark's fourth
timing, a 200-id RecomputeIsBlockedForIDsInTx (the bulk/import batched path,
which no metric in the review covered), is 53-63 ms on main against 400-556 ms
here — 6.9x at the medians. The cause is structural and measured: a batched
pass runs many small statements, each deriving the set twice, and the set
costs a few ms however it is written. The per-pass read that fixes the
unbatched statements makes the SMALL-batch case worse, not better (one
whole-store read costs more than the subqueries a two-id batch pays), so the
fix is a per-batch-scoped read threaded through expandBatchTemplate's
placeholder plumbing — a separate change, and the reason this timing is
committed rather than left in a scratch file.

Three shapes were built and measured before this one, all at the same scale:

  - the review's own: per-row correlated EXISTS with two more inside,
    3.3-3.7x full repair, 5.2x doctor count;
  - the set as a DISTINCT derived table LEFT JOINed on p.id, which is what B2
    recommended: ~30x. dolt plans a join operand as the inner side of a nested
    loop and rebuilds it per outer row — the correlated shape in a derived
    table's clothes. This is the one finding that contradicts the review, and
    it is why the set is subtracted with NOT IN instead;
  - WITH RECURSIVE for the ancestor walk: blew the driver read timeout on the
    full repair. A recursive CTE pays a query's overhead per ITERATION; the
    seeded walk cost 45 ms per evaluation against 6 ms for four levels of
    joins, which is why the walk has a horizon (subtreeWalkDepth = 4, epic ->
    sub-epic -> leg -> task). Past it a reason reads as exogenous, which
    propagates MORE and hides nothing the old code showed.

Seeding: AffectedByDepChange{,ForWisp}InTx now walks UP from a parent-child
edge's target, not just to it. Hanging G under C changes nothing about C and
everything about any ANCESTOR of C that blocks on G, so each such ancestor's
sibling set is reseeded — otherwise the incremental write path and the full
repair disagree, which the new test catches as a nonzero doctor count. The
walk is filtered to ancestors that carry a blocking reason at all: without
that filter a 70-deep chain reseeds every ancestor's subtree on every edge
written into it, which timed out TestGetReadyWork_ParentFilterReturnsDeepDescendants.

CHANGELOG now names the limitation the review ruled acceptable but undocumented
(a sub-epic blocked both by its own children and by an exogenously blocked
ancestor keeps its children visible), marked tracked upstream; bee files the
follow-up issue at PR time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(doctor): rune-safe Detail truncation, Category on the N/A returns (wy-3eb07b, #6506)

The "Parent Close Gates" check joins its inventory with U+2192, three bytes
wide, and then cut the joined string at byte 200. Two of every three offsets
land mid-rune, so `bd doctor --json` could carry invalid UTF-8 in Detail — a
display nuisance in the terminal, but a value a JSON consumer has to decode.
truncateDetail walks back to a rune boundary first; the test drives every
entry count from 1 to 120 so the cut lands at all three offsets within the
arrow and asserts utf8.ValidString on each.

The same byte cut appears twice more in this file (:101, :561) on ASCII-only
details. Left alone here: they are not this bead's, and changing them would
put a behaviour change in two checks a reviewer of #6506 has no reason to be
reading.

The check's two "N/A" early returns also omitted Category, unlike its three
real returns, which drops them out of the categorised report. Both now carry
CategoryMetadata.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf(storage): read the batched exogeneity set once per batch, not per statement (wy-3eb07b, #6506)

D1, the PR blocker from the round-1 review. The batched mark/unmark templates
spliced parentsExplainedBySubtreeSQL — the "which blocked parents are explained
only by their own subtree" query — into both of their parent-child legs,
batch-scoped. The scope made it look cheap. It was not: dolt pays for a
`p.id NOT IN (subquery)` once per leg per STATEMENT, and a batched pass is a
mark and an unmark, so one 200-id RecomputeIsBlockedForIDsInTx derived the set
four times and measured 6.9x origin/main (53-63 ms -> 400-556 ms). That is the
bulk/import path and the cost shape #6288 removed.

The set is now READ once per batch and BOUND into both statements as ids —
the same trade the unbatched full repair already makes per pass — which is why
markBlockedTemplate/unmarkBlockedTemplate are built per batch now: the number
of bound ids is a property of the batch. In front of the read sits a cheap
indexed guard, batchParentKindsInTx: a parent KIND with no parent-child row in
the batch has an empty candidate set, hence an empty exogeneity set, so its
read is pure overhead, and the read is a ten-join walk under two CTEs that
dolt spends ~8 ms planning whether it returns rows or not. Most batches name
no wisp parent at all; a batch of roots names neither. Without the guard the
2-id case measured 1.9x, with it 0.94x.

expandBatchTemplate now WALKS the template instead of counting "%s": the batch
scope and the bound exogeneity ids alternate inside the union (leg 3's scope,
leg 3's NOT IN, leg 4's scope, leg 4's NOT IN), so only text order says where
each group goes. It returns an error on a ?/bound mismatch rather than handing
the engine an opaque argument-count failure. The %s budget is back to 6 from
8, re-pinned, with two new pins beside it: the ? group is exactly the bound
exogeneity ids (none for an empty set), and the mismatch is refused.

KEPT, on round-1's measurements: the NOT IN subtraction (the derived-table
JOIN measured ~30x in dolt) and the 4-level LEFT JOIN ancestor walk (WITH
RECURSIVE blew the driver read timeout).

MEASURED. n=1000 issues / 2870 deps / 333-604 blocked, embedded dolt 2.2.0,
TestBlockedStateScaleTiming, SIX interleaved origin/main(71c4cd08b)/head pairs,
medians, worst single pair in brackets. The bench now times three batch sizes
instead of one, because the per-statement and per-id costs pull opposite ways:

  metric                     main     head    ratio          bar
  full repair               49 ms    57 ms    1.16x [1.32x]  <= 1.5x  MET
  doctor count (consistent) 48 ms    57 ms    1.18x [1.32x]  <= 1.5x  MET
  write path, per CloseIssue 25 ms   26 ms    1.03x [2.59x]  <= 1.2x  MET
  batched recompute, 200 id 54 ms    69 ms    1.28x [1.57x]  <= 1.5x  MET
  batched recompute,  20 id 17 ms    22 ms    1.29x [1.50x]  (measured)
  batched recompute,   2 id 22 ms    21 ms    0.94x [1.03x]  (measured)

All four stated bars are met on medians. The worst-pair column is at or inside
this box's own noise: over the same six runs origin/main's OWN spread is 2.07x
on the full repair, 2.33x on the write path, 1.92x on the 2-id batch and 1.19x
on the 200-id batch, so a single pair's ratio is not evidence at this
resolution and the medians are what the bars are read off.

DISCLOSED, not a stated bar and not fixed here: the bench FIXTURE BUILD —
2870 AddDependency calls plus 240 CloseIssue through the store — is 75 s ->
102 s, 1.36x [1.61x]. That is the dependency-ADD write path, and the cost is
round 1's incremental seeding (AffectedByDepChangeInTx now walks up to every
reason-carrying ancestor and reseeds its whole sibling set), which is what
buys the incremental/full-repair agreement pinned by the lockstep test in the
next commit. Per-edge it is sub-millisecond; it shows up here because the
fixture writes the hierarchy of a 1000-issue store edge by edge.

GATE: go build ./... ; go vet ./internal/storage/issueops/ ; go test
./internal/storage/issueops/ (ok, includes the rewritten template and sqlmock
pins) ; BEADS_TEST_EMBEDDED_DOLT=1 go test ./internal/storage/dolt/ -run
'ParentChildCascade|RecomputeAllIsBlocked' (ok) ; gofmt -l clean. The full
enumerated gate is on the bead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(storage): pin both sides of the subtree-walk horizon, and the incremental/full-repair lockstep (wy-3eb07b, #6506)

D2 and D3 from the round-1 supervisor ruling.

D2 — THE HORIZON IS NOW PINNED, not commented. "Inside my own subtree" is
decided by a fixed 4-level ancestor walk (subtreeWalkDepth), because a
recursive walk cost 7.5x per evaluation and blew the driver read timeout. Two
tests that differ by ONE EDGE straddle it:

  _ReasonAtTheWalkHorizon    P blocks a descendant exactly 4 pc levels down.
                             The walk reaches P, so the reason is
                             subtree-derived and the chain comes bright.
                             RED on origin/main (4 rows dark that the fix
                             brightens) — so it is not vacuous.
  _ReasonPastTheWalkHorizon  one level further. The reason reads as EXOGENOUS,
                             so the whole chain stays dark, which is
                             origin/main's own answer — and that test PASSES
                             UNCHANGED ON origin/main (verified in a temp
                             worktree at 71c4cd08b, since removed). It also
                             asserts no row of the shape appears in
                             GetReadyWork: no new dark rows AND no new visible
                             rows past the horizon, checkable rather than
                             argued. Raising the constant makes it fail and
                             name the row that changed.

Both also repair from a fully inverted plane, which is the leg that needs the
fixpoint rather than the write path's seeding. The CHANGELOG now names the
4-level horizon beside the existing limitation sentence, and the constant's own
doc names the two pins.

CAN THE HORIZON BE RAISED? Measured, and the answer is "not on this evidence".
Same bench, same box, full repair / doctor count / 20-id batch / 200-id batch:

  depth 4   57 ms /  58 ms / 21 ms /  69 ms     (shipping)
  depth 6  126 ms / 118 ms / 52 ms / 101 ms     2.2x, TWICE reproduced
  depth 8   59 ms /  74 ms / 24 ms /  68 ms     ~= depth 4, TWICE reproduced

The curve is NON-MONOTONIC: 6 falls off a cliff and 8 comes back. That is a
dolt planner artifact, not a cost model, and raising a semantics-bearing
constant on the strength of a number nobody can explain is the wrong trade —
depth 8 also makes every past-horizon shape newly bright, which is a behavior
change and not this round's business. Left at 4, with the numbers recorded for
whoever picks the knob up.

D3 — THE LOCKSTEP. _IncrementalMatchesFullRepairAcrossEdits walks one graph
through the mutations that reclassify a blocking reason WITHOUT touching the
row that carries it, and after every one asserts both the exact flags and
assertConvergedAndStable: detection counts 0 AND a full repair changes 0 rows.
An under-seeded incremental path (the AffectedByDepChange{,ForWisp}InTx
ancestor walk round 1 added) fails here as a nonzero repair over a graph the
write path just finished maintaining. The mutations: re-parent the reason's
target INTO the blocker's subtree (which is how the shape arrives — the store
refuses a blocks edge straight onto an existing own descendant, and that
refusal's reason text is what 3a5203979 corrected), close the descendant,
reopen it, re-parent it back OUT, remove the blocks edge. The
into/out-of-subtree steps are the ones that flip rows the edge does not name
(the parent's other children), which is exactly what
appendSiblingsUnderAncestorsInTx exists to seed. RED on origin/main for the
semantic values; origin/main's own incremental/full-repair agreement holds
there, so what this pins is the agreement, not the numbers.

GATE: BEADS_TEST_EMBEDDED_DOLT=1 go test ./internal/storage/dolt/ -run
'ParentChildCascade' -count=1 — 8/8 ok; gofmt -l clean; go vet clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(doctor): the close-gate inventory walks the same four parent-child levels as the cascade (wy-3eb07b, #6506)

The "Parent Close Gates" check listed only DIRECT parent->child gates, so the
fix's own headline shape (a parent that blocks on its grandchild) never
appeared in the inventory. The check now joins a descendant-pairs set built
to the same four parent-child levels the cascade uses for "inside my own
subtree" (issueops subtreeWalkDepth), as plain joins over the dependency
union rather than a correlated subquery, which dolt declined to plan once
rows were present. Messages say "own-descendant"; the tests add a grandchild
gate (2 listed, the exogenous edge still not) and assert Category.

CHANGELOG cites the nested-gap follow-up, gastownhall/beads#6601.

Tests run: go build ./... 0; go vet ./cmd/bd/doctor/ 0; gofmt -l clean;
env -u BEADS_ACTOR go test ./cmd/bd/doctor/ -run 'ParentBlocksOwnChild|TruncateDetail' -count=1 PASS (3/3).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* ci: allowlist TestBlockedStateScaleTiming's -short skip (wy-3eb07b, #6506)

scripts/check-testing-short.sh reserves testing.Short() for runtime, stress
and large-fixture skips and keys them by file::func. The scale-timing test
added in this PR builds a 1000-issue / ~2000-edge dolt fixture and times the
blocked-state paths over it: a large-fixture skip, and PR Core runs with
-short, so it needs the entry. One line, the same shape as the
TestUOW_ConcurrentMergeOps_NoLostUpdate row from #4732.

Local: ./scripts/ci/pr-policy.sh rc 0 (check testing.Short boundaries
succeeded; the other nine legs unchanged).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wy-3eb07b: build(bazel): register blocked_scale_bench_test.go in internal/storage/dolt

The rebase carried the bench test but not its BUILD.bazel entry, so the
Bazel test lane's gazelle sync and Go test equivalence steps failed while
every Go lane was green. This is CI's own bazel-sync patch for 043f9c47b9,
applied unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(blocked): resolve explained parents through their ancestry; walk ancestors on hinted per-table members

Maintainer fixup for #6602 (adopt-pr review iteration 1, quality scorecard
760/1000, request_changes; amended at iteration 2, 906/1000, approve). It
fixes the review's two major findings, the doctor check's message and its
swallowed query failure, the stale core-builder doc, and the stale
test-comment names. At iteration 2 it also takes both nits and corrects a
CHANGELOG sentence this fixup had made false.

- Mixed-arm leak (major). The explained-parent read decides from a
  parent's own reason rows, and the fixpoint carries the recursive clause
  only for a parent with no reason row of its own. With Q blocked on an
  outside X, P a child of Q, and P blocking on its own child C, P read as
  explained and C came bright under an epic still gated by X, on the
  write path and in full repair alike. pruneExplainedByAncestryInTx now
  shrinks every read to the greatest set whose members' blocked parents
  are all members. A full-repair read is complete, so a blocked parent it
  did not return is unexplained. A batch-scoped read climbs instead,
  reading the blocked parents above the batch by id (explainedAmongInTx).
  The suggested one-level NOT EXISTS is not enough: put a second close
  gate between them (R blocked on X, pc Q -> R, Q blocks on P, pc P -> Q,
  P blocks on C) and P's blocked parent Q is in the set, so R's gate
  never reaches C.
- ancestorChainInTx (major) walked the derived UNION of both dependency
  tables that #7224 removed from isAncestorInTx, on every parent-child
  write and merge replay. It now walks reachabilityMembers, the hinted
  per-table recursive members split out of reachabilityQuery, with the
  reason filter kept in the outer SELECT. TestGraphWalkPlansHonorJoinHints
  (embedded) gains an ancestor-chain subtest that pins the lookup plan;
  the old walk fails it.
- New dolt tests: TestParentChildCascade_OwnGateUnderExogenousParent
  (write path, full repair, and X closed and reopened),
  _NestedGatesBothWays (the two-gate chain both ways, then a close of the
  children's own blocker, which re-evaluates them through a batch whose
  read covers nothing above their parents), and
  _WispGateUnderExogenousParent (a wisp gate under an issue epic, an
  issue gate under a wisp epic, then an explained wisp gate the walk must
  read by id above such a batch). Five mutants of the prune were each
  killed: no prune (the PR head), a one-level check, an over-prune, a
  scoped read treated as complete, and a scoped read that trusts every
  parent above it. The eight existing cascade tests pass under all five.
  Iteration 2 killed two more, each by its own section of the wisp test
  alone: dropping the lookup of a wisp parent from blockedParentEdgesInTx,
  and dropping explainedAmongInTx's wisp leg, which had survived all
  eleven cascade tests.
- The doctor check no longer says the children "stay in bd ready", and
  a failed or timed-out inventory query is now a warning carrying the
  error, as in validation.go's other checks, instead of an OK that reads
  as "no gates". TestCheckParentBlocksOwnChildDB_QueryFailureWarns pins
  it with a closed handle, and it fails with the old status. A warning
  does not touch doctor's overall result, and it can be suppressed.
- The core builder's doc no longer describes the spliced design, and
  three test comments name builders that exist.
- explainedAmongInTx says why it does not tolerate a missing table, as
  the file's wisp reads do: the prune runs only on a non-empty read,
  which came from the same core query over all four tables.
- The CHANGELOG entry no longer calls #6601's shape (a sub-epic gated on
  its own children under an exogenously blocked epic) a known limitation
  that keeps the children visible. The prune darkens them, as
  TestParentChildCascade_OwnGateUnderExogenousParent pins.

Left as-is, all non-gating: the refusal hint's "(bd gate)", an API string
pinned by the golden corpus; a parent-child add/remove timing in the
scale bench, whose 1000 issues cannot show the 50k shape (the plan pin
guards it); and hints for the explained-parent subtree walk and the
doctor query. The measured complexity major is runDiagnostics (50/66),
the same at base; the PR adds two lines to it.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(storage): re-pin the classic batch-apply statement count for the parent-child cascade

The B0 statement-count pins predate this PR's parent-child cascade. The
first full run of the storage tiers on the adopted head measured the
40 (classic) shape at 835 statements on both backends, against the
pinned 786 +/- 2.

All 49 extra statements are reads, seven for each of the shape's seven
distinct parent-child dep adds (its eighth repeats a pair and is a
no-op):
- appendSiblingsUnderAncestorsInTx reseeds the parent's other children:
  one ancestorChainInTx walk and four child probes, +5.
- The batched mark/unmark reads its exogeneity set first
  (batchExogeneity): the parent-kind probe and the issue-parent read, +2.

Measured by logging every statement of the classic apply on the CI base
233d151b37 (786) and on b2b93515bf (835), then diffing the two logs by
statement shape. Those seven shapes are the whole difference, each
exactly +7, and no write changed. The 356 and 712 shapes are unchanged
on both backends.

This is the deliberate re-measure and re-pin the pins' doc comment asks
for. The per-change breakdown is in pinnedEmbeddedStatementCounts.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
This commit is contained in:
Bee 2026-10-08 08:31:34 +00:00 • committed by GitHub
commit 2cae0c1ad8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
22 changed files with 2859 additions and 200 deletions

View file

@ -1486,6 +1486,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
a prefix-routed rig whose store cannot be read: routing still reports that
failure as not-found.
- **An epic that blocks on its own children no longer hides them from `bd
ready`** ([#6506](https://github.com/gastownhall/beads/issues/6506)). A
parent-child edge now propagates only a parent's *exogenous* blockedness —
blockedness whose cause lies outside the parent's own subtree — so the
close-gate idiom, a parent carrying `blocks` edges onto its own children (or
grandchildren) so it cannot close before them, stops darkening the very work
it is waiting for. The parent itself stays blocked, and a close gate does not
unlock the work under an epic that is itself blocked from outside: a sub-epic
blocked BOTH by its own children AND by an exogenously blocked ancestor still
darkens its children
([#6601](https://github.com/gastownhall/beads/issues/6601)).
"Inside my own subtree" is decided to a fixed depth of **four**
parent-child levels — epic → sub-epic → leg → task — so a parent that
blocks on something deeper than that still darkens its whole subtree, exactly
as it did before this fix; that depth is a measured trade (the
bounded walk is a fixed chain of index probes, where a recursive one cost
7.5x per evaluation) and raising it is a one-constant change. Existing stores
repair on the next write that touches the hierarchy, or with
`bd doctor --fix`.
- `bd dep add` no longer explains its refusal of a blocking edge onto your own
descendant by claiming the block would cascade down and never clear — it
does not, as of the fix above. The refusal stands, and now names the
sanctioned way to say it: a waits-for gate over the children.
- **`routes.jsonl` prefixes containing a hyphen now route**
([#5048](https://github.com/gastownhall/beads/issues/5048)). Prefix routing
cut the bead ID at its first `-` and required an exact match, so a route

View file

@ -983,6 +983,11 @@ func runDiagnostics(path string) doctorResult {
result.Checks = append(result.Checks, childParentDepsCheck)
// Don't fail overall check for child→parent deps, just warn
// Check 22b: Parent→own-child close gates (informational, #6506)
parentCloseGateCheck := convertDoctorCheck(doctor.CheckParentBlocksOwnChild(path))
result.Checks = append(result.Checks, parentCloseGateCheck)
// Informational only: these edges are legal and never fail the overall check.
// Check 23: Duplicate issues (from bd validate)
duplicatesCheck := convertDoctorCheck(doctor.CheckDuplicateIssues(path, doctorOrchestrator, orchestratorDuplicatesThreshold))
result.Checks = append(result.Checks, duplicatesCheck)

View file

@ -8,6 +8,7 @@ import (
"os"
"path/filepath"
"strings"
"unicode/utf8"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/storage"
@ -337,6 +338,153 @@ func CheckChildParentDependencies(path string) DoctorCheck {
return checkChildParentDependenciesDB(db)
}
// detailMaxBytes is the budget for a DoctorCheck Detail line: enough to show
// the shape of a finding, short enough not to bury the report.
const detailMaxBytes = 200
// truncateDetail cuts a detail line to detailMaxBytes WITHOUT splitting a
// rune. The gate inventory joins its entries with U+2192 (three bytes), so a
// plain detail[:200] lands mid-rune about two times in three and puts invalid
// UTF-8 into `bd doctor --json`, where it is no longer a display nuisance but
// a value a consumer has to decode.
func truncateDetail(detail string) string {
if len(detail) <= detailMaxBytes {
return detail
}
cut := detailMaxBytes
for cut > 0 && !utf8.RuneStart(detail[cut]) {
cut--
}
return detail[:cut] + "..."
}
// CheckParentBlocksOwnChild reports parents that carry a blocking edge onto one
// of their own parent-child descendants — the "close gate on the epic" idiom,
// where P blocks on C1 and C2 (or on a grandchild) so P cannot close before
// them.
//
// It is INFORMATIONAL and has no fix. Before gastownhall/beads#6506 these
// edges were a trap: the parent's blocked bit cascaded to its children
// whatever set it, so the gate hid the very children it was waiting for and
// nothing in the subtree could ever become ready. The cascade now propagates
// only exogenous blockedness, so the edges are harmless and frequently
// intentional; what an operator wants from doctor is the inventory — which
// beads in this store use the idiom — not a repair. An inventory query that
// fails or times out is a warning, as in the file's other checks, so it cannot
// pass for an empty inventory.
func CheckParentBlocksOwnChild(path string) DoctorCheck {
beadsDir := ResolveBeadsDirForRepo(path)
db, store, err := openStoreDB(beadsDir)
if err != nil {
return DoctorCheck{
Name: "Parent Close Gates",
Status: StatusOK,
Message: "N/A (no database)",
Category: CategoryMetadata,
}
}
defer func() { _ = store.Close() }()
return checkParentBlocksOwnChildDB(db)
}
// checkParentBlocksOwnChildDB is the core logic for CheckParentBlocksOwnChild.
// The pair is symmetric-but-opposite to checkChildParentDependenciesDB above:
// that one finds a CHILD blocking on its PARENT (still a deadlock, still
// fixable); this one finds a PARENT blocking on its own DESCENDANT, and
// matches on real parent-child edges rather than on dotted-ID ancestry,
// because the close-gate idiom is wired with explicit edges between unrelated
// ids. "Descendant" is walked to the same four parent-child levels the cascade
// fix uses to decide "inside my own subtree" (issueops subtreeWalkDepth), so
// the inventory and the fix agree on which gates are gates: a parent that
// blocks on its grandchild is listed, one that blocks five levels down is not.
func checkParentBlocksOwnChildDB(db *sql.DB) DoctorCheck {
//nolint:gosec // G202: doctorDependencyUnionSQL returns a fixed internal SELECT fragment.
query := `
SELECT DISTINCT b.issue_id, b.depends_on_id
FROM (` + doctorDependencyUnionSQL() + `) b
JOIN (
SELECT l1.depends_on_id AS anc, l1.issue_id AS des
FROM (` + doctorDependencyUnionSQL() + `) l1
WHERE l1.type = 'parent-child'
UNION
SELECT l1.depends_on_id, l2.issue_id
FROM (` + doctorDependencyUnionSQL() + `) l1
JOIN (` + doctorDependencyUnionSQL() + `) l2
ON l2.type = 'parent-child' AND l2.depends_on_id = l1.issue_id
WHERE l1.type = 'parent-child'
UNION
SELECT l1.depends_on_id, l3.issue_id
FROM (` + doctorDependencyUnionSQL() + `) l1
JOIN (` + doctorDependencyUnionSQL() + `) l2
ON l2.type = 'parent-child' AND l2.depends_on_id = l1.issue_id
JOIN (` + doctorDependencyUnionSQL() + `) l3
ON l3.type = 'parent-child' AND l3.depends_on_id = l2.issue_id
WHERE l1.type = 'parent-child'
UNION
SELECT l1.depends_on_id, l4.issue_id
FROM (` + doctorDependencyUnionSQL() + `) l1
JOIN (` + doctorDependencyUnionSQL() + `) l2
ON l2.type = 'parent-child' AND l2.depends_on_id = l1.issue_id
JOIN (` + doctorDependencyUnionSQL() + `) l3
ON l3.type = 'parent-child' AND l3.depends_on_id = l2.issue_id
JOIN (` + doctorDependencyUnionSQL() + `) l4
ON l4.type = 'parent-child' AND l4.depends_on_id = l3.issue_id
WHERE l1.type = 'parent-child'
) d ON d.anc = b.issue_id AND d.des = b.depends_on_id
WHERE b.type IN ('blocks', 'conditional-blocks')
ORDER BY b.issue_id, b.depends_on_id
`
rows, err := db.Query(query)
if err != nil {
return DoctorCheck{
Name: "Parent Close Gates",
Status: StatusWarning,
Message: "N/A (query failed)",
Detail: err.Error(),
Category: CategoryMetadata,
}
}
defer rows.Close()
var gates []string
for rows.Next() {
var issueID, dependsOnID string
if err := rows.Scan(&issueID, &dependsOnID); err == nil {
gates = append(gates, fmt.Sprintf("%s→%s", issueID, dependsOnID))
}
}
if err := rows.Err(); err != nil {
return DoctorCheck{
Name: "Parent Close Gates",
Status: StatusWarning,
Message: "N/A (row iteration error)",
Detail: err.Error(),
Category: CategoryMetadata,
}
}
if len(gates) == 0 {
return DoctorCheck{
Name: "Parent Close Gates",
Status: StatusOK,
Message: "No parent→own-descendant blocking edges",
Category: CategoryMetadata,
}
}
detail := truncateDetail(strings.Join(gates, ", "))
return DoctorCheck{
Name: "Parent Close Gates",
Status: StatusOK,
Message: fmt.Sprintf("%d parent→own-descendant blocking edge(s) (informational: these gates do not keep the children out of bd ready)", len(gates)),
Detail: detail,
Category: CategoryMetadata,
}
}
// checkDoltConflicts queries the Dolt server for unresolved merge conflicts (GH-2249).
func checkDoltConflicts(beadsDir string) DoctorCheck {
doltPath := getDatabasePath(beadsDir)

View file

@ -5,6 +5,7 @@ package doctor
import (
"context"
"crypto/sha256"
"database/sql"
"encoding/hex"
"fmt"
"os"
@ -12,6 +13,7 @@ import (
"strings"
"testing"
"time"
"unicode/utf8"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/types"
@ -717,3 +719,156 @@ func TestCheckGitConflicts_DoltBackend_Clean(t *testing.T) {
t.Fatal("Expected no conflicts in clean database")
}
}
// TestCheckParentBlocksOwnChildDB_NoGates verifies the informational check is
// quiet on a store with no close gates — including one that has an ordinary
// parent-child hierarchy, which must not be mistaken for a gate.
func TestCheckParentBlocksOwnChildDB_NoGates(t *testing.T) {
store := newTestDoltStore(t, "test")
ctx := context.Background()
parent := &types.Issue{Title: "Parent epic", Status: types.StatusOpen, Priority: 1, IssueType: types.TypeEpic}
if err := store.CreateIssue(ctx, parent, "test"); err != nil {
t.Fatalf("Failed to create parent: %v", err)
}
child := &types.Issue{Title: "Child task", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}
if err := store.CreateIssue(ctx, child, "test"); err != nil {
t.Fatalf("Failed to create child: %v", err)
}
db := store.DB()
if _, err := db.ExecContext(ctx,
`INSERT INTO dependencies (id, issue_id, depends_on_issue_id, type, created_at, created_by) VALUES (UUID(), ?, ?, 'parent-child', NOW(), 'test')`,
child.ID, parent.ID); err != nil {
t.Fatalf("Failed to insert parent-child edge: %v", err)
}
check := checkParentBlocksOwnChildDB(db)
if check.Status != StatusOK {
t.Errorf("Status = %q, want %q", check.Status, StatusOK)
}
if check.Message != "No parent→own-descendant blocking edges" {
t.Errorf("Message = %q, want the empty-inventory message", check.Message)
}
if check.Category != CategoryMetadata {
t.Errorf("Category = %q, want %q", check.Category, CategoryMetadata)
}
}
// TestCheckParentBlocksOwnChildDB_GateReported verifies the inventory: a parent
// that blocks on its own parent-child child OR grandchild is listed, by both
// ids, and the check stays OK — these edges are legal since gastownhall/beads#6506 and the
// check has no fix.
func TestCheckParentBlocksOwnChildDB_GateReported(t *testing.T) {
store := newTestDoltStore(t, "test")
ctx := context.Background()
parent := &types.Issue{Title: "Parent epic", Status: types.StatusOpen, Priority: 1, IssueType: types.TypeEpic}
if err := store.CreateIssue(ctx, parent, "test"); err != nil {
t.Fatalf("Failed to create parent: %v", err)
}
child := &types.Issue{Title: "Child task", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}
if err := store.CreateIssue(ctx, child, "test"); err != nil {
t.Fatalf("Failed to create child: %v", err)
}
grandchild := &types.Issue{Title: "Grandchild task", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}
if err := store.CreateIssue(ctx, grandchild, "test"); err != nil {
t.Fatalf("Failed to create grandchild: %v", err)
}
other := &types.Issue{Title: "Unrelated blocker", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}
if err := store.CreateIssue(ctx, other, "test"); err != nil {
t.Fatalf("Failed to create unrelated blocker: %v", err)
}
db := store.DB()
edges := []struct{ from, to, depType string }{
{child.ID, parent.ID, "parent-child"},
{grandchild.ID, child.ID, "parent-child"},
{parent.ID, child.ID, "blocks"}, // the close gate
{parent.ID, grandchild.ID, "blocks"}, // a gate two levels down: listed too
{parent.ID, other.ID, "blocks"}, // exogenous: must NOT be listed
{child.ID, other.ID, "blocks"}, // an ordinary blocker on the child
}
for _, e := range edges {
if _, err := db.ExecContext(ctx,
`INSERT INTO dependencies (id, issue_id, depends_on_issue_id, type, created_at, created_by) VALUES (UUID(), ?, ?, ?, NOW(), 'test')`,
e.from, e.to, e.depType); err != nil {
t.Fatalf("Failed to insert %s edge %s->%s: %v", e.depType, e.from, e.to, err)
}
}
check := checkParentBlocksOwnChildDB(db)
if check.Status != StatusOK {
t.Errorf("Status = %q, want %q: the idiom is legal, the check only reports it", check.Status, StatusOK)
}
if check.Fix != "" {
t.Errorf("Fix = %q, want empty: this check has no --fix", check.Fix)
}
if check.Category != CategoryMetadata {
t.Errorf("Category = %q, want %q", check.Category, CategoryMetadata)
}
if !strings.Contains(check.Message, "2 parent→own-descendant") {
t.Errorf("Message = %q, want a count of exactly the two close gates", check.Message)
}
for _, wantDetail := range []string{parent.ID + "→" + child.ID, parent.ID + "→" + grandchild.ID} {
if !strings.Contains(check.Detail, wantDetail) {
t.Errorf("Detail = %q, want it to name %q", check.Detail, wantDetail)
}
}
if strings.Contains(check.Detail, other.ID) {
t.Errorf("Detail = %q, must not list the exogenous blocker %s", check.Detail, other.ID)
}
}
// TestCheckParentBlocksOwnChildDB_QueryFailureWarns verifies that an inventory
// query that fails is a warning, not an OK that reads as "no gates". A closed
// handle fails every query without reaching a server.
func TestCheckParentBlocksOwnChildDB_QueryFailureWarns(t *testing.T) {
db, err := sql.Open("mysql", "root@tcp(127.0.0.1:1)/none")
if err != nil {
t.Fatalf("sql.Open: %v", err)
}
_ = db.Close()
check := checkParentBlocksOwnChildDB(db)
if check.Status != StatusWarning {
t.Errorf("Status = %q, want %q", check.Status, StatusWarning)
}
if check.Message != "N/A (query failed)" {
t.Errorf("Message = %q, want %q", check.Message, "N/A (query failed)")
}
if check.Detail == "" {
t.Error("Detail is empty, want the query error")
}
}
// TestTruncateDetail_CutsOnARuneBoundary pins the --json contract: a cut
// detail line is still valid UTF-8. The gate inventory joins its entries with
// U+2192, three bytes wide, so a byte-indexed cut lands mid-rune for two of
// every three offsets and emits a replacement-character sequence into JSON.
func TestTruncateDetail_CutsOnARuneBoundary(t *testing.T) {
// One entry is 6 bytes: "a", U+2192 (3), "b", ",", " " — so successive
// totals step past 200 at offsets that are not rune starts.
entry := "a→b"
for n := 1; n < 120; n++ {
parts := make([]string, n)
for i := range parts {
parts[i] = entry
}
got := truncateDetail(strings.Join(parts, ", "))
if !utf8.ValidString(got) {
t.Fatalf("n=%d: truncateDetail produced invalid UTF-8: %q", n, got)
}
if len(got) > detailMaxBytes+len("...") {
t.Fatalf("n=%d: truncateDetail returned %d bytes, want <= %d",
n, len(got), detailMaxBytes+len("..."))
}
}
// Short input is returned whole, with no ellipsis.
if got := truncateDetail("wr-p→wr-c"); got != "wr-p→wr-c" {
t.Errorf("short detail was rewritten: %q", got)
}
}

View file

@ -1566,7 +1566,7 @@ type Problem struct {
// BlockerId With `dependency_cycle`, hierarchy refusal only: the ancestor or descendant the edge named as blocker. See `issue_id`.
BlockerId *string `json:"blocker_id,omitempty"`
// BlockerIsAncestor With `dependency_cycle`, hierarchy refusal only: true when `blocker_id` is an ANCESTOR of `issue_id` (which cannot close until its descendants finish, so the gate would never clear), false when it is a DESCENDANT (blocked status cascades, so it would inherit the block and never close). Both polarities are reported; this member is never omitted to mean false. See `issue_id`.
// BlockerIsAncestor With `dependency_cycle`, hierarchy refusal only: true when `blocker_id` is an ANCESTOR of `issue_id` (which cannot close until its descendants finish, so the gate would never clear), false when it is a DESCENDANT (waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children). Both polarities are reported; this member is never omitted to mean false. See `issue_id`.
BlockerIsAncestor *bool `json:"blocker_is_ancestor,omitempty"`
// Blockers With `not_closable`, and ONLY on the live-blocker refusal: the live blockers that refused the close, in the order the refusing check reported them, read from the refusal's typed list rather than parsed out of any message. Never present together with `open_children`.

View file

@ -11982,9 +11982,10 @@ components:
With `dependency_cycle`, hierarchy refusal only: true when
`blocker_id` is an ANCESTOR of `issue_id` (which cannot close until
its descendants finish, so the gate would never clear), false when
it is a DESCENDANT (blocked status cascades, so it would inherit the
block and never close). Both polarities are reported; this member is
never omitted to mean false. See `issue_id`.
it is a DESCENDANT (waiting on your own subtree is a close gate, not
a blocks edge — use a waits-for gate over the children). Both
polarities are reported; this member is never omitted to mean false.
See `issue_id`.
expected_version:
type: string
description: >-

View file

@ -61,14 +61,14 @@
"probe eq-de -\u003e eq-de tables=[]: cycle=true hierarchy=ok cycle-check=cannot add self-dependency: eq-de cannot depend on itself",
"probe eq-de -\u003e eq-de tables=[dependencies]: cycle=true hierarchy=ok cycle-check=cannot add self-dependency: eq-de cannot depend on itself",
"probe eq-de -\u003e eq-de tables=[wisp_dependencies]: cycle=true hierarchy=ok cycle-check=cannot add self-dependency: eq-de cannot depend on itself",
"probe eq-de -\u003e eq-de.1 tables=[]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.1: blocked status cascades to descendants, so eq-de.1 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.1 tables=[dependencies]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.1: blocked status cascades to descendants, so eq-de.1 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.1 tables=[]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.1: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.1 tables=[dependencies]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.1: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.1 tables=[wisp_dependencies]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de -\u003e eq-de.1.1 tables=[]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.1.1: blocked status cascades to descendants, so eq-de.1.1 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.1.1 tables=[dependencies]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.1.1: blocked status cascades to descendants, so eq-de.1.1 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.1.1 tables=[]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.1.1: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.1.1 tables=[dependencies]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.1.1: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.1.1 tables=[wisp_dependencies]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de -\u003e eq-de.2 tables=[]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.2: blocked status cascades to descendants, so eq-de.2 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.2 tables=[dependencies]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.2: blocked status cascades to descendants, so eq-de.2 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.2 tables=[]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.2: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.2 tables=[dependencies]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-de.2: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-de.2 tables=[wisp_dependencies]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de -\u003e eq-db1 tables=[]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de -\u003e eq-db1 tables=[dependencies]: cycle=false hierarchy=ok cycle-check=ok",
@ -91,9 +91,9 @@
"probe eq-de -\u003e eq-dc tables=[]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de -\u003e eq-dc tables=[dependencies]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de -\u003e eq-dc tables=[wisp_dependencies]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de -\u003e eq-dw1 tables=[]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-dw1: blocked status cascades to descendants, so eq-dw1 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-dw1 tables=[]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-dw1: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-dw1 tables=[dependencies]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de -\u003e eq-dw1 tables=[wisp_dependencies]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-dw1: blocked status cascades to descendants, so eq-dw1 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-dw1 tables=[wisp_dependencies]: cycle=true hierarchy=eq-de cannot be blocked by its descendant eq-dw1: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-dw2 tables=[]: cycle=true hierarchy=ok cycle-check=adding dependency would create a cycle",
"probe eq-de -\u003e eq-dw2 tables=[dependencies]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de -\u003e eq-dw2 tables=[wisp_dependencies]: cycle=true hierarchy=ok cycle-check=adding dependency would create a cycle",
@ -121,8 +121,8 @@
"probe eq-de.1 -\u003e eq-de.1 tables=[]: cycle=true hierarchy=ok cycle-check=cannot add self-dependency: eq-de.1 cannot depend on itself",
"probe eq-de.1 -\u003e eq-de.1 tables=[dependencies]: cycle=true hierarchy=ok cycle-check=cannot add self-dependency: eq-de.1 cannot depend on itself",
"probe eq-de.1 -\u003e eq-de.1 tables=[wisp_dependencies]: cycle=true hierarchy=ok cycle-check=cannot add self-dependency: eq-de.1 cannot depend on itself",
"probe eq-de.1 -\u003e eq-de.1.1 tables=[]: cycle=true hierarchy=eq-de.1 cannot be blocked by its descendant eq-de.1.1: blocked status cascades to descendants, so eq-de.1.1 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de.1 -\u003e eq-de.1.1 tables=[dependencies]: cycle=true hierarchy=eq-de.1 cannot be blocked by its descendant eq-de.1.1: blocked status cascades to descendants, so eq-de.1.1 would inherit the block and never close cycle-check=adding dependency would create a cycle",
"probe eq-de.1 -\u003e eq-de.1.1 tables=[]: cycle=true hierarchy=eq-de.1 cannot be blocked by its descendant eq-de.1.1: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de.1 -\u003e eq-de.1.1 tables=[dependencies]: cycle=true hierarchy=eq-de.1 cannot be blocked by its descendant eq-de.1.1: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate) cycle-check=adding dependency would create a cycle",
"probe eq-de.1 -\u003e eq-de.1.1 tables=[wisp_dependencies]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de.1 -\u003e eq-de.2 tables=[]: cycle=false hierarchy=ok cycle-check=ok",
"probe eq-de.1 -\u003e eq-de.2 tables=[dependencies]: cycle=false hierarchy=ok cycle-check=ok",
@ -1143,7 +1143,7 @@
"probe eq-dmissing -\u003e eq-dmissing tables=[wisp_dependencies]: cycle=true hierarchy=ok cycle-check=cannot add self-dependency: eq-dmissing cannot depend on itself",
"add eq-db1 -blocks-\u003e eq-db4: adding dependency would create a cycle",
"add eq-db1 -blocks-\u003e eq-dr: ok",
"add eq-de -blocks-\u003e eq-de.1.1: eq-de cannot be blocked by its descendant eq-de.1.1: blocked status cascades to descendants, so eq-de.1.1 would inherit the block and never close",
"add eq-de -blocks-\u003e eq-de.1.1: eq-de cannot be blocked by its descendant eq-de.1.1: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate)",
"add eq-de.1.1 -blocks-\u003e eq-de: eq-de.1.1 cannot be blocked by its ancestor eq-de: eq-de cannot close until its descendants finish, so the gate would never clear",
"add eq-de.1.1 -blocks-\u003e eq-de.2: ok",
"add eq-de -parent-child-\u003e eq-dw2: adding dependency would create a cycle",
@ -1160,7 +1160,7 @@
"add eq-db2 -related-\u003e eq-db1: dependency eq-db2 -\u003e eq-db1 already exists with type \"blocks\" (requested \"related\"); remove it first with 'bd dep remove' then re-add",
"add eq-dr -blocks-\u003e external:proj:other: ok",
"add eq-de.2 -parent-child-\u003e eq-de.1: ok",
"add eq-de.1 -blocks-\u003e eq-de.2: eq-de.1 cannot be blocked by its descendant eq-de.2: blocked status cascades to descendants, so eq-de.2 would inherit the block and never close",
"add eq-de.1 -blocks-\u003e eq-de.2: eq-de.1 cannot be blocked by its descendant eq-de.2: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate)",
"add eq-dk1 -blocks-\u003e eq-dl: ok",
"add eq-dl -blocks-\u003e eq-dk2: adding dependency would create a cycle",
"add eq-dk2 -blocks-\u003e eq-dk1: eq-dk2 cannot be blocked by its ancestor eq-dk1: eq-dk1 cannot close until its descendants finish, so the gate would never clear"

View file

@ -145,6 +145,7 @@ go_test(
"blocked_recheck_publication_test.go",
"blocked_recheck_raw_test.go",
"blocked_recompute_guard_test.go",
"blocked_scale_bench_test.go",
"blocking_annotator_contract_test.go",
"bootstrap_test.go",
"bootstrapper_contract_test.go",

View file

@ -3,8 +3,10 @@ package dolt
import (
"context"
"database/sql"
"fmt"
"testing"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/types"
)
@ -177,3 +179,870 @@ func TestRecomputeAllIsBlocked_CascadesThroughParentChild(t *testing.T) {
t.Fatalf("after repair: want 0 inconsistencies, got %d", n)
}
}
// --- parent-child cascade: only EXOGENOUS blockedness propagates -------------
//
// gastownhall/beads#6506 (wy-3eb07b). The legs used to cascade on the parent's
// is_blocked bit WHATEVER set it, so the "close gate on the epic" idiom — P
// carries blocks edges onto its own children so it cannot close before them —
// darkened the very children P was waiting for. Neither child could reach bd
// ready, so neither could be worked, so neither could close: a permanent lock.
//
// The contract these four cases pin: a parent-child edge propagates only the
// parent's EXOGENOUS blockedness. P is blocked FOR CHILD C iff P has a
// blocking reason whose target is NOT one of P's own parent-child children, or
// P's own parent is (recursively) exogenously blocked. P's OWN is_blocked is
// unchanged throughout — it really does depend on open children.
// addDepSkippingCycleCheck writes a dependency edge past the per-edge cycle
// check. The close-gate topology cannot be built any other way through the
// store: every edge is legal when it is written, but the pair is refused in
// either order once both exist (CheckBlockingHierarchyInTx rejects a blocker
// that is a descendant; CheckDependencyCycleInTx rejects the parent-child edge
// that closes the loop). That is exactly how the shape arrives in the wild —
// one edge at a time, or through import/merge — and SkipCycleCheck is the
// supported way to reproduce it (the whole-graph END GATE is the batch
// caller's obligation, not this fixture's).
func addDepSkippingCycleCheck(ctx context.Context, t *testing.T, store *DoltStore, source, target string, depType types.DependencyType) {
t.Helper()
// The store-level AddDependencyWithOptions does not forward SkipCycleCheck
// (dolt/dependencies.go builds its own AddDependencyOpts); the transaction
// surface does, and it is the one batch callers use.
err := store.RunInTransaction(ctx, "test: seed close-gate edge", func(tx storage.Transaction) error {
return tx.AddDependencyWithOptions(ctx,
&types.Dependency{IssueID: source, DependsOnID: target, Type: depType},
"tester", storage.DependencyAddOptions{SkipCycleCheck: true})
})
if err != nil {
t.Fatalf("add dep %s -> %s (%s): %v", source, target, depType, err)
}
}
// assertBlockedFlags checks is_blocked for a set of ids in one place so a
// failure names every disagreement rather than the first.
func assertBlockedFlags(ctx context.Context, t *testing.T, store *DoltStore, when string, want map[string]bool) {
t.Helper()
for id, expected := range want {
if got := isBlocked(ctx, t, store.db, id); got != expected {
t.Errorf("%s: %s is_blocked = %v, want %v", when, id, got, expected)
}
}
}
// assertConvergedAndStable pins the lockstep for the case at hand: detection
// counts zero, and the full repair agrees by changing nothing.
func assertConvergedAndStable(ctx context.Context, t *testing.T, store *DoltStore) {
t.Helper()
if n := countInconsistencies(ctx, t, store.db); n != 0 {
t.Errorf("write path left %d inconsistencies; detection and the union must agree", n)
}
if changed := recomputeAll(ctx, t, store.db); changed != 0 {
t.Errorf("full repair changed %d rows over a write-path-consistent graph, want 0", changed)
}
}
// TestParentChildCascade_ParentBlockedOnlyByItsOwnChildren is the repro. P
// blocks on C1 and C2, which are P's own parent-child children. P stays
// blocked; the children must NOT be darkened, and must be reachable as ready
// work — the property whose absence made the idiom a permanent lock.
func TestParentChildCascade_ParentBlockedOnlyByItsOwnChildren(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
for _, id := range []string{"pc-gate-p", "pc-gate-c1", "pc-gate-c2"} {
createPerm(t, ctx, store, id)
}
// The blocks edges go in while no hierarchy exists yet...
addDependencyWithMeta(t, ctx, store, "pc-gate-p", "pc-gate-c1", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pc-gate-p", "pc-gate-c2", types.DepBlocks, "")
// ...and the hierarchy after, which is what closes the gate on the epic.
addDepSkippingCycleCheck(ctx, t, store, "pc-gate-c1", "pc-gate-p", types.DepParentChild)
addDepSkippingCycleCheck(ctx, t, store, "pc-gate-c2", "pc-gate-p", types.DepParentChild)
assertBlockedFlags(ctx, t, store, "after the write path", map[string]bool{
"pc-gate-p": true, // P really does depend on two open children.
"pc-gate-c1": false, // ...but the gate must not darken them.
"pc-gate-c2": false,
})
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, "after the full repair", map[string]bool{
"pc-gate-p": true,
"pc-gate-c1": false,
"pc-gate-c2": false,
})
ready, err := store.GetReadyWork(ctx, types.WorkFilter{Status: types.StatusOpen})
if err != nil {
t.Fatalf("GetReadyWork: %v", err)
}
inReady := map[string]bool{}
for _, issue := range ready {
inReady[issue.ID] = true
}
for _, id := range []string{"pc-gate-c1", "pc-gate-c2"} {
if !inReady[id] {
t.Errorf("%s missing from ready work: the children of a close-gate epic are exactly what must stay workable", id)
}
}
if inReady["pc-gate-p"] {
t.Error("pc-gate-p is ready, but it is blocked on two open children")
}
}
// TestParentChildCascade_ParentBlocksItsOwnGrandchild is the same lock one
// level down, and the case a DIRECT-child reading of "own child" leaves alive
// (the adversarial review's B1 repro).
//
// P blocks on G, which is not P's child but P's GRANDCHILD: pc C -> P,
// pc C2 -> P, pc G -> C. Under a depth-1 test G reads exogenous, so P darkens
// C, so C darkens G — and G is the only row whose close can ever free P. The
// contract is about blockedness originating outside P's own SUBTREE, so the
// whole hierarchy under P must stay bright while P alone stays blocked.
//
// C2 is the sibling control: it is reachable only through P's cascade, so it
// proves the leg was re-evaluated rather than merely skipped for the branch
// the new edge touched.
func TestParentChildCascade_ParentBlocksItsOwnGrandchild(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
for _, id := range []string{"pcg-p", "pcg-c", "pcg-c2", "pcg-g"} {
createPerm(t, ctx, store, id)
}
// The gate goes in first, while P and G are unrelated...
addDependencyWithMeta(t, ctx, store, "pcg-p", "pcg-g", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcg-c", "pcg-p", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pcg-c2", "pcg-p", types.DepParentChild, "")
// ...and the edge that puts G inside P's subtree comes last, which is what
// reclassifies P's reason from exogenous to subtree-derived. Nothing but
// this edge's own seeding can un-darken C and C2.
addDepSkippingCycleCheck(ctx, t, store, "pcg-g", "pcg-c", types.DepParentChild)
want := map[string]bool{
"pcg-p": true, // P still depends on an open G.
"pcg-c": false, // but nothing under P may be darkened by it,
"pcg-c2": false,
"pcg-g": false, // least of all the row whose close frees P.
}
assertBlockedFlags(ctx, t, store, "after the write path", want)
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, "after the full repair", want)
ready, err := store.GetReadyWork(ctx, types.WorkFilter{Status: types.StatusOpen})
if err != nil {
t.Fatalf("GetReadyWork: %v", err)
}
inReady := map[string]bool{}
for _, issue := range ready {
inReady[issue.ID] = true
}
if !inReady["pcg-g"] {
t.Error("pcg-g missing from ready work: the grandchild P waits on is the one row that can end the lock")
}
for _, id := range []string{"pcg-c", "pcg-c2"} {
if !inReady[id] {
t.Errorf("%s missing from ready work", id)
}
}
// The same answer from a fully inverted plane: the grandchild case needs
// the fixpoint, not just the write path's incremental seeding.
if _, err := store.db.ExecContext(ctx, "UPDATE issues SET is_blocked = 1 - is_blocked"); err != nil {
t.Fatalf("invert flags: %v", err)
}
if changed := recomputeAll(ctx, t, store.db); changed == 0 {
t.Fatal("repair reported 0 corrections over an inverted plane")
}
assertBlockedFlags(ctx, t, store, "after repairing an inverted plane", want)
if n := countInconsistencies(ctx, t, store.db); n != 0 {
t.Errorf("after repair: want 0 inconsistencies, got %d", n)
}
}
// TestParentChildCascade_ExogenousParentStillCascades is the control: the fix
// narrows the cascade, it does not remove it. P is blocked by an open issue
// that is NOT one of its children, so both children inherit blocked exactly as
// before.
func TestParentChildCascade_ExogenousParentStillCascades(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
for _, id := range []string{"pc-exo-p", "pc-exo-c1", "pc-exo-c2", "pc-exo-x"} {
createPerm(t, ctx, store, id)
}
addDependencyWithMeta(t, ctx, store, "pc-exo-p", "pc-exo-x", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pc-exo-c1", "pc-exo-p", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pc-exo-c2", "pc-exo-p", types.DepParentChild, "")
assertBlockedFlags(ctx, t, store, "after the write path", map[string]bool{
"pc-exo-x": false,
"pc-exo-p": true,
"pc-exo-c1": true,
"pc-exo-c2": true,
})
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, "after the full repair", map[string]bool{
"pc-exo-p": true,
"pc-exo-c1": true,
"pc-exo-c2": true,
})
}
// TestParentChildCascade_ThreeLevelChainBothWays pins the recursive clause in
// both directions on one graph.
//
// Exogenous chain: GP is blocked by an outside issue, so GP darkens P and P
// darkens C — including C, whose own parent P has no blocking edge of its own.
// That is the clause "P's own parent-child parent is (recursively)
// exogenously blocked", carried one level per fixpoint pass.
//
// Close-gate chain: GP2 is blocked only by its own child P2. Neither P2 nor
// P2's child C2 may be darkened by it.
func TestParentChildCascade_ThreeLevelChainBothWays(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
for _, id := range []string{
"pc3-x", "pc3-gp", "pc3-p", "pc3-c",
"pc3-gp2", "pc3-p2", "pc3-c2",
} {
createPerm(t, ctx, store, id)
}
// Exogenous: GP -> X (outside), then the two-level hierarchy under GP.
addDependencyWithMeta(t, ctx, store, "pc3-gp", "pc3-x", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pc3-p", "pc3-gp", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pc3-c", "pc3-p", types.DepParentChild, "")
// Close gate: GP2 blocks on P2, which is then made GP2's own child, and C2
// hangs under P2.
addDependencyWithMeta(t, ctx, store, "pc3-gp2", "pc3-p2", types.DepBlocks, "")
addDepSkippingCycleCheck(ctx, t, store, "pc3-p2", "pc3-gp2", types.DepParentChild)
addDependencyWithMeta(t, ctx, store, "pc3-c2", "pc3-p2", types.DepParentChild, "")
want := map[string]bool{
"pc3-x": false,
"pc3-gp": true,
"pc3-p": true,
"pc3-c": true,
"pc3-gp2": true, // blocked on its own open child...
"pc3-p2": false, // ...which must not be darkened,
"pc3-c2": false, // ...nor anything under it.
}
assertBlockedFlags(ctx, t, store, "after the write path", want)
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, "after the full repair", want)
// The full repair reaches the same fixpoint from a fully inverted plane,
// not just from the write path's answer: the chain is the case where the
// cascade needs more than one pass.
if _, err := store.db.ExecContext(ctx, "UPDATE issues SET is_blocked = 1 - is_blocked"); err != nil {
t.Fatalf("invert flags: %v", err)
}
if changed := recomputeAll(ctx, t, store.db); changed == 0 {
t.Fatal("repair reported 0 corrections over an inverted plane")
}
assertBlockedFlags(ctx, t, store, "after repairing an inverted plane", want)
if n := countInconsistencies(ctx, t, store.db); n != 0 {
t.Errorf("after repair: want 0 inconsistencies, got %d", n)
}
}
// TestParentChildCascade_WaitsForGateOverOwnChildren pins the same contract for
// the waits-for leg, whose "reason" is a gate rather than a target status.
//
// A gate whose spawner is the parent's OWN child is child-derived and must not
// darken the parent's children; a gate on a spawner outside the hierarchy is
// exogenous and must.
func TestParentChildCascade_WaitsForGateOverOwnChildren(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
for _, id := range []string{
"pcw-p", "pcw-c1", "pcw-c2", "pcw-g",
"pcw-p2", "pcw-c3", "pcw-s", "pcw-sc",
} {
createPerm(t, ctx, store, id)
}
// Own-child gate: P waits for C1, its own child. C1 has an open child of
// its own, so the all-children gate is live and P is blocked.
addDependencyWithMeta(t, ctx, store, "pcw-c1", "pcw-p", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pcw-c2", "pcw-p", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pcw-g", "pcw-c1", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pcw-p", "pcw-c1", types.DepWaitsFor, "")
// Outside gate: P2 waits for a spawner that is not in its hierarchy.
addDependencyWithMeta(t, ctx, store, "pcw-c3", "pcw-p2", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pcw-sc", "pcw-s", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pcw-p2", "pcw-s", types.DepWaitsFor, "")
want := map[string]bool{
"pcw-p": true, // its own child's fanout is still open
"pcw-c1": false, // but the gate is over C1 itself: no darkening
"pcw-c2": false,
"pcw-g": false,
"pcw-p2": true, // gate on an outside spawner: exogenous
"pcw-c3": true, // so the child inherits
"pcw-s": false,
"pcw-sc": false,
}
assertBlockedFlags(ctx, t, store, "after the write path", want)
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, "after the full repair", want)
}
// horizonChain builds "P blocks on a descendant `levels` parent-child edges
// below it" and returns the chain ids, nearest child first.
//
// The order is the only order the store allows: the blocks edge goes in while
// the hierarchy does not exist yet, then the chain from the top down, and the
// LAST edge — the one that finally puts the blocked target inside P's subtree —
// needs SkipCycleCheck, because that is the edge that closes the loop.
func horizonChain(ctx context.Context, t *testing.T, store *DoltStore, prefix string, levels int) []string {
t.Helper()
chain := make([]string, 0, levels)
parent := prefix + "-p"
createPerm(t, ctx, store, parent)
for i := 1; i <= levels; i++ {
id := fmt.Sprintf("%s-d%d", prefix, i)
createPerm(t, ctx, store, id)
chain = append(chain, id)
}
addDependencyWithMeta(t, ctx, store, parent, chain[levels-1], types.DepBlocks, "")
for i, id := range chain {
up := parent
if i > 0 {
up = chain[i-1]
}
if i == levels-1 {
addDepSkippingCycleCheck(ctx, t, store, id, up, types.DepParentChild)
continue
}
addDependencyWithMeta(t, ctx, store, id, up, types.DepParentChild, "")
}
return chain
}
// TestParentChildCascade_ReasonAtTheWalkHorizon pins the deepest shape the
// exogeneity test actually decides: P blocks on a descendant exactly
// subtreeWalkDepth parent-child edges below it. The walk reaches P on its last
// level, so the reason reads as subtree-derived and nothing under P is
// darkened.
//
// Together with the past-horizon test below, this is the pin on the horizon
// itself: these two shapes differ by one edge and must land on opposite sides.
func TestParentChildCascade_ReasonAtTheWalkHorizon(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
// 4 = subtreeWalkDepth (internal/storage/issueops/blocked_consistency.go).
chain := horizonChain(ctx, t, store, "pch4", 4)
want := map[string]bool{"pch4-p": true}
for _, id := range chain {
want[id] = false
}
assertBlockedFlags(ctx, t, store, "after the write path", want)
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, "after the full repair", want)
// And from an inverted plane, which is the leg the fixpoint has to walk.
if _, err := store.db.ExecContext(ctx, "UPDATE issues SET is_blocked = 1 - is_blocked"); err != nil {
t.Fatalf("invert flags: %v", err)
}
if changed := recomputeAll(ctx, t, store.db); changed == 0 {
t.Fatal("repair reported 0 corrections over an inverted plane")
}
assertBlockedFlags(ctx, t, store, "after repairing an inverted plane", want)
if n := countInconsistencies(ctx, t, store.db); n != 0 {
t.Errorf("after repair: want 0 inconsistencies, got %d", n)
}
}
// TestParentChildCascade_ReasonPastTheWalkHorizon is the PIN ON THE
// LIMITATION, not on the fix (gastownhall/beads#6506, fix round 2 / D2).
//
// The ancestor walk in parentsExplainedBySubtreeSQL is four levels deep
// (subtreeWalkDepth), spelled as four pairs of indexed LEFT JOINs because
// WITH RECURSIVE measured 7.5x per evaluation and blew the driver read timeout
// on the full repair. So a blocking reason whose target is FIVE parent-child
// edges below its blocker is out of the walk's reach and reads as EXOGENOUS.
//
// This test states exactly what that costs, so the limitation is a pinned
// behavior rather than a comment: the whole chain under P stays dark, which is
// the pre-#6506 behavior for this one shape. It therefore PASSES UNCHANGED ON
// origin/main — deliberately, and it is the assertion that makes "no new dark
// rows, no new visible rows past the horizon" checkable rather than argued. If
// someone raises the horizon, this test fails and names the row that changed.
//
// The direction matters and is the safe one: past the horizon the cascade
// propagates MORE than the contract asks, never less, so no work is hidden
// that the old code showed. What survives is the old permanent lock for a
// hierarchy deeper than epic -> sub-epic -> leg -> task between a blocker and
// the row it waits on.
func TestParentChildCascade_ReasonPastTheWalkHorizon(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
// 5 = subtreeWalkDepth + 1: one level past what the walk can see.
chain := horizonChain(ctx, t, store, "pch5", 5)
// Every row in the chain stays blocked, P included: origin/main's answer.
want := map[string]bool{"pch5-p": true}
for _, id := range chain {
want[id] = true
}
assertBlockedFlags(ctx, t, store, "after the write path", want)
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, "after the full repair", want)
// No NEW visible row either: nothing in the shape is ready work, which is
// what it means for the past-horizon answer to be unchanged.
ready, err := store.GetReadyWork(ctx, types.WorkFilter{Status: types.StatusOpen})
if err != nil {
t.Fatalf("GetReadyWork: %v", err)
}
for _, issue := range ready {
if _, ours := want[issue.ID]; ours {
t.Errorf("%s is ready work: past the walk horizon the cascade is unchanged from origin/main, where the whole chain is dark", issue.ID)
}
}
if _, err := store.db.ExecContext(ctx, "UPDATE issues SET is_blocked = 1 - is_blocked"); err != nil {
t.Fatalf("invert flags: %v", err)
}
if changed := recomputeAll(ctx, t, store.db); changed == 0 {
t.Fatal("repair reported 0 corrections over an inverted plane")
}
assertBlockedFlags(ctx, t, store, "after repairing an inverted plane", want)
if n := countInconsistencies(ctx, t, store.db); n != 0 {
t.Errorf("after repair: want 0 inconsistencies, got %d", n)
}
}
// TestParentChildCascade_IncrementalMatchesFullRepairAcrossEdits is the
// lockstep proof for the incremental seeding this fix changed
// (AffectedByDepChange{,ForWisp}InTx now walk UP from a parent-child edge's
// target to every reason-carrying ancestor and reseed each ancestor's whole
// sibling set, see appendSiblingsUnderAncestorsInTx).
//
// The property: after EVERY mutation, the flags the incremental write path
// left behind are exactly the flags a full repair computes from scratch —
// detection counts 0 and the repair changes 0 rows (assertConvergedAndStable).
// An under-seeded incremental path shows up here as a nonzero repair on a
// graph the write path just finished maintaining.
//
// The mutations are the ones that reclassify a blocking reason WITHOUT
// touching the row that carries it, which is where seeding is hard to get
// right: moving the reason's target INTO the blocker's subtree, closing and
// reopening it, moving it back OUT, and removing the reason.
//
// Note what is NOT here: adding a blocks edge straight onto an existing own
// descendant. The store refuses that outright (CheckBlockingHierarchyInTx, the
// refusal whose reason text this branch corrects), so the shape can only
// arrive the way it arrives in the wild — one edge at a time, or through
// import/merge — and the re-parent below is the one-edge-at-a-time arrival.
func TestParentChildCascade_IncrementalMatchesFullRepairAcrossEdits(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
for _, id := range []string{"ls-p", "ls-c", "ls-c2", "ls-g", "ls-out"} {
createPerm(t, ctx, store, id)
}
step := func(name string, want map[string]bool) {
t.Helper()
assertBlockedFlags(ctx, t, store, name+" (write path)", want)
// The lockstep itself: detection sees nothing to fix and the full
// repair changes nothing.
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, name+" (after full repair)", want)
}
// The starting graph: P blocks on G while G is OUTSIDE P's subtree, so the
// reason is exogenous and P's two children inherit it. G hangs under a
// root of its own.
addDependencyWithMeta(t, ctx, store, "ls-p", "ls-g", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "ls-c", "ls-p", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "ls-c2", "ls-p", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "ls-g", "ls-out", types.DepParentChild, "")
exogenous := map[string]bool{
"ls-p": true, "ls-c": true, "ls-c2": true, "ls-g": false, "ls-out": false,
}
subtreeDerived := map[string]bool{
"ls-p": true, "ls-c": false, "ls-c2": false, "ls-g": false, "ls-out": false,
}
clear := map[string]bool{
"ls-p": false, "ls-c": false, "ls-c2": false, "ls-g": false, "ls-out": false,
}
step("exogenous blocker outside the subtree", exogenous)
// 1. Re-parent G INTO P's subtree, under C. The same blocks edge is now a
// close gate over P's own grandchild, so P keeps its flag and its whole
// subtree must come bright — including C and C2, which this edge does not
// name. SkipCycleCheck is the only way in: P blocks G, so pc G -> C closes
// a loop.
if err := store.RemoveDependency(ctx, "ls-g", "ls-out", "tester"); err != nil {
t.Fatalf("remove parent-child ls-g -> ls-out: %v", err)
}
addDepSkippingCycleCheck(ctx, t, store, "ls-g", "ls-c", types.DepParentChild)
step("reason re-parented into the subtree", subtreeDerived)
// 2. Close the descendant: a closed target is no reason at all, so P comes
// bright with the edge still in place. Then reopen it.
if err := store.CloseIssue(ctx, "ls-g", "lockstep", "tester", ""); err != nil {
t.Fatalf("close ls-g: %v", err)
}
step("descendant closed", clear)
if err := store.ReopenIssue(ctx, "ls-g", "lockstep", "tester"); err != nil {
t.Fatalf("reopen ls-g: %v", err)
}
step("descendant reopened", subtreeDerived)
// 3. Re-parent G back OUT. The reason turns exogenous again and P's
// cascade comes back on for C and C2.
if err := store.RemoveDependency(ctx, "ls-g", "ls-c", "tester"); err != nil {
t.Fatalf("remove parent-child ls-g -> ls-c: %v", err)
}
addDependencyWithMeta(t, ctx, store, "ls-g", "ls-out", types.DepParentChild, "")
step("reason re-parented out of the subtree", exogenous)
// 4. Remove the reason itself: nothing anywhere carries one.
if err := store.RemoveDependency(ctx, "ls-p", "ls-g", "tester"); err != nil {
t.Fatalf("remove blocks ls-p -> ls-g: %v", err)
}
step("blocks edge removed", clear)
}
// TestParentChildCascade_OwnGateUnderExogenousParent is the contract's
// recursive clause for a parent that carries a reason of its own (the #6602
// review's mixed-arm repro). Q is blocked by an outside issue X, P hangs under
// Q, and P carries a close gate over its own child C. P's own reason is
// subtree-derived, but the epic above it is gated from outside, so P cascades
// and C stays dark: a close gate does not unlock the work under an epic that
// is itself blocked.
//
// Closing and reopening X then flips C without touching a row of the gate.
// Nothing about P changes, so only the seeding from X down the hierarchy can
// re-evaluate C.
func TestParentChildCascade_OwnGateUnderExogenousParent(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
for _, id := range []string{"pcm-x", "pcm-q", "pcm-p", "pcm-c"} {
createPerm(t, ctx, store, id)
}
addDependencyWithMeta(t, ctx, store, "pcm-q", "pcm-x", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcm-p", "pcm-q", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pcm-p", "pcm-c", types.DepBlocks, "")
addDepSkippingCycleCheck(ctx, t, store, "pcm-c", "pcm-p", types.DepParentChild)
step := func(name string, want map[string]bool) {
t.Helper()
assertBlockedFlags(ctx, t, store, name+" (write path)", want)
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, name+" (after full repair)", want)
}
gated := map[string]bool{
"pcm-x": false,
"pcm-q": true, // blocked from outside,
"pcm-p": true, // which P inherits on top of its own gate,
"pcm-c": true, // so the gate's own child stays dark.
}
step("own gate under an exogenously blocked parent", gated)
ready, err := store.GetReadyWork(ctx, types.WorkFilter{Status: types.StatusOpen})
if err != nil {
t.Fatalf("GetReadyWork: %v", err)
}
for _, issue := range ready {
if issue.ID == "pcm-c" {
t.Error("pcm-c is ready work, but the epic above its parent is blocked from outside")
}
}
if _, err := store.db.ExecContext(ctx, "UPDATE issues SET is_blocked = 1 - is_blocked"); err != nil {
t.Fatalf("invert flags: %v", err)
}
if changed := recomputeAll(ctx, t, store.db); changed == 0 {
t.Fatal("repair reported 0 corrections over an inverted plane")
}
assertBlockedFlags(ctx, t, store, "after repairing an inverted plane", gated)
if n := countInconsistencies(ctx, t, store.db); n != 0 {
t.Errorf("after repair: want 0 inconsistencies, got %d", n)
}
if err := store.CloseIssue(ctx, "pcm-x", "lockstep", "tester", ""); err != nil {
t.Fatalf("close pcm-x: %v", err)
}
step("outside blocker closed", map[string]bool{
"pcm-x": false,
"pcm-q": false,
"pcm-p": true, // P still waits on C,
"pcm-c": false, // but nothing above P is blocked any more.
})
if err := store.ReopenIssue(ctx, "pcm-x", "lockstep", "tester"); err != nil {
t.Fatalf("reopen pcm-x: %v", err)
}
step("outside blocker reopened", gated)
}
// TestParentChildCascade_NestedGatesBothWays pins that the recursive clause
// reads the WHOLE chain above a parent, not one level of it.
//
// Exogenous root: R is blocked by an outside issue X. Q hangs under R and
// carries a close gate over its own child P, and P carries one over its own
// child C. Q's reason and P's are both subtree-derived, so asking only "is P's
// blocked parent explained?" finds Q explained and lets C through. But Q is
// explained by its own reasons alone while R above it is gated from outside:
// R cascades into Q, Q into P, and P into C.
//
// No exogenous root: the same two nested gates, Q2 over P2 and P2 over C2,
// with nothing blocked above Q2. Every blocked row is explained by its own
// subtree, so C2 stays bright. This is the control against pruning too much:
// a blocked parent above P2 is not enough to cascade, it has to be one that
// is not explained.
//
// Closing X then frees C three levels below the row whose flag changed, and
// reopening X darkens it again.
//
// Last, a write that re-evaluates the children alone: an outside blocker Y of
// their own closes. The close seeds only Y and the rows waiting on it, so the
// batch's read is asked about P2 and P and nothing above them, and the chain
// has to be read by id as the walk climbs: Q2 is explained, so C2 comes
// bright, and R is not, so C stays dark.
func TestParentChildCascade_NestedGatesBothWays(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
for _, id := range []string{
"pcn-x", "pcn-r", "pcn-q", "pcn-p", "pcn-c",
"pcn-q2", "pcn-p2", "pcn-c2",
} {
createPerm(t, ctx, store, id)
}
// The blocks edges go in while no hierarchy exists; each parent-child edge
// that puts a gate's target under its blocker closes a loop and needs
// SkipCycleCheck.
addDependencyWithMeta(t, ctx, store, "pcn-r", "pcn-x", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcn-q", "pcn-p", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcn-p", "pcn-c", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcn-q", "pcn-r", types.DepParentChild, "")
addDepSkippingCycleCheck(ctx, t, store, "pcn-p", "pcn-q", types.DepParentChild)
addDepSkippingCycleCheck(ctx, t, store, "pcn-c", "pcn-p", types.DepParentChild)
addDependencyWithMeta(t, ctx, store, "pcn-q2", "pcn-p2", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcn-p2", "pcn-c2", types.DepBlocks, "")
addDepSkippingCycleCheck(ctx, t, store, "pcn-p2", "pcn-q2", types.DepParentChild)
addDepSkippingCycleCheck(ctx, t, store, "pcn-c2", "pcn-p2", types.DepParentChild)
step := func(name string, want map[string]bool) {
t.Helper()
assertBlockedFlags(ctx, t, store, name+" (write path)", want)
assertConvergedAndStable(ctx, t, store)
assertBlockedFlags(ctx, t, store, name+" (after full repair)", want)
}
want := map[string]bool{
"pcn-x": false,
"pcn-r": true,
"pcn-q": true,
"pcn-p": true,
"pcn-c": true, // two explained gates under an exogenous one
"pcn-q2": true,
"pcn-p2": true,
"pcn-c2": false, // two explained gates with nothing above them
}
step("nested gates", want)
ready, err := store.GetReadyWork(ctx, types.WorkFilter{Status: types.StatusOpen})
if err != nil {
t.Fatalf("GetReadyWork: %v", err)
}
inReady := map[string]bool{}
for _, issue := range ready {
inReady[issue.ID] = true
}
if inReady["pcn-c"] {
t.Error("pcn-c is ready work, but R two gates above it is blocked from outside")
}
if !inReady["pcn-c2"] {
t.Error("pcn-c2 missing from ready work: every gate above it is explained by its own subtree")
}
if _, err := store.db.ExecContext(ctx, "UPDATE issues SET is_blocked = 1 - is_blocked"); err != nil {
t.Fatalf("invert flags: %v", err)
}
if changed := recomputeAll(ctx, t, store.db); changed == 0 {
t.Fatal("repair reported 0 corrections over an inverted plane")
}
assertBlockedFlags(ctx, t, store, "after repairing an inverted plane", want)
if n := countInconsistencies(ctx, t, store.db); n != 0 {
t.Errorf("after repair: want 0 inconsistencies, got %d", n)
}
if err := store.CloseIssue(ctx, "pcn-x", "lockstep", "tester", ""); err != nil {
t.Fatalf("close pcn-x: %v", err)
}
freed := map[string]bool{}
for id, blocked := range want {
freed[id] = blocked
}
freed["pcn-r"] = false
freed["pcn-c"] = false
step("outside blocker closed", freed)
if err := store.ReopenIssue(ctx, "pcn-x", "lockstep", "tester"); err != nil {
t.Fatalf("reopen pcn-x: %v", err)
}
step("outside blocker reopened", want)
createPerm(t, ctx, store, "pcn-y")
addDependencyWithMeta(t, ctx, store, "pcn-c2", "pcn-y", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcn-c", "pcn-y", types.DepBlocks, "")
want["pcn-y"] = false
want["pcn-c2"] = true // by Y, for now
step("a blocker of their own on both children", want)
if err := store.CloseIssue(ctx, "pcn-y", "done", "tester", ""); err != nil {
t.Fatalf("close pcn-y: %v", err)
}
want["pcn-c2"] = false
step("their own blocker closed", want)
}
// TestParentChildCascade_WispGateUnderExogenousParent is the mixed arm on the
// wisp mirror, and across the two kinds: wisp P carries a close gate over its
// own wisp child C and hangs under a permanent epic Q that an outside issue X
// blocks. The edge up from P lives in wisp_dependencies and names Q in
// depends_on_issue_id, so the walk above P has to cross from the wisp tables
// to the issue tables to find what gates it.
//
// The second component crosses the other way, under the same X: permanent P2
// carries a close gate over its own child C2 and hangs under a wisp epic Q2
// that X blocks. The edge up from P2 lives in dependencies and names Q2 in
// depends_on_wisp_id.
//
// Last, a wisp gate that is explained: wisp Q3 gates its own child P3, which
// gates its own child C3, and nothing blocks Q3 from outside, so C3 is bright.
// Then C3's own outside blocker Y closes. The close seeds only Y and the rows
// waiting on it, so the batch's read is asked about P3 and nothing above it,
// and the walk has to read wisp Q3 by id to find it explained.
func TestParentChildCascade_WispGateUnderExogenousParent(t *testing.T) {
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := testContext(t)
defer cancel()
createPerm(t, ctx, store, "pcwm-x")
createPerm(t, ctx, store, "pcwm-q")
createWisp(t, ctx, store, "pcwm-p")
createWisp(t, ctx, store, "pcwm-c")
addDependencyWithMeta(t, ctx, store, "pcwm-q", "pcwm-x", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcwm-p", "pcwm-q", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pcwm-p", "pcwm-c", types.DepBlocks, "")
addDepSkippingCycleCheck(ctx, t, store, "pcwm-c", "pcwm-p", types.DepParentChild)
createWisp(t, ctx, store, "pcwm-q2")
createPerm(t, ctx, store, "pcwm-p2")
createPerm(t, ctx, store, "pcwm-c2")
addDependencyWithMeta(t, ctx, store, "pcwm-q2", "pcwm-x", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcwm-p2", "pcwm-q2", types.DepParentChild, "")
addDependencyWithMeta(t, ctx, store, "pcwm-p2", "pcwm-c2", types.DepBlocks, "")
addDepSkippingCycleCheck(ctx, t, store, "pcwm-c2", "pcwm-p2", types.DepParentChild)
// The rows live in two tables, so read both; the ids do not collide.
check := func(when string, want map[string]bool) {
t.Helper()
got := readIsBlockedFlags(t, ctx, store, "issues")
for id, blocked := range readIsBlockedFlags(t, ctx, store, "wisps") {
got[id] = blocked
}
for id, expected := range want {
if blocked, ok := got[id]; !ok {
t.Errorf("%s: %s not found in issues or wisps", when, id)
} else if blocked != expected {
t.Errorf("%s: %s is_blocked = %v, want %v", when, id, blocked, expected)
}
}
}
step := func(name string, want map[string]bool) {
t.Helper()
check(name+" (write path)", want)
assertConvergedAndStable(ctx, t, store)
check(name+" (after full repair)", want)
}
gated := map[string]bool{
"pcwm-x": false,
"pcwm-q": true, "pcwm-p": true, "pcwm-c": true,
"pcwm-q2": true, "pcwm-p2": true, "pcwm-c2": true,
}
step("wisp gate under an exogenously blocked epic", gated)
for _, table := range []string{"issues", "wisps"} {
if _, err := store.db.ExecContext(ctx, "UPDATE "+table+" SET is_blocked = 1 - is_blocked"); err != nil {
t.Fatalf("invert %s flags: %v", table, err)
}
}
if changed := recomputeAll(ctx, t, store.db); changed == 0 {
t.Fatal("repair reported 0 corrections over an inverted plane")
}
check("after repairing an inverted plane", gated)
if n := countInconsistencies(ctx, t, store.db); n != 0 {
t.Errorf("after repair: want 0 inconsistencies, got %d", n)
}
if err := store.CloseIssue(ctx, "pcwm-x", "lockstep", "tester", ""); err != nil {
t.Fatalf("close pcwm-x: %v", err)
}
step("outside blocker closed", map[string]bool{
"pcwm-x": false,
"pcwm-q": false, "pcwm-p": true, "pcwm-c": false,
"pcwm-q2": false, "pcwm-p2": true, "pcwm-c2": false,
})
if err := store.ReopenIssue(ctx, "pcwm-x", "lockstep", "tester"); err != nil {
t.Fatalf("reopen pcwm-x: %v", err)
}
step("outside blocker reopened", gated)
createWisp(t, ctx, store, "pcwm-q3")
createPerm(t, ctx, store, "pcwm-p3")
createPerm(t, ctx, store, "pcwm-c3")
addDependencyWithMeta(t, ctx, store, "pcwm-q3", "pcwm-p3", types.DepBlocks, "")
addDependencyWithMeta(t, ctx, store, "pcwm-p3", "pcwm-c3", types.DepBlocks, "")
addDepSkippingCycleCheck(ctx, t, store, "pcwm-p3", "pcwm-q3", types.DepParentChild)
addDepSkippingCycleCheck(ctx, t, store, "pcwm-c3", "pcwm-p3", types.DepParentChild)
nested := map[string]bool{"pcwm-q3": true, "pcwm-p3": true, "pcwm-c3": false}
step("two explained gates under a wisp", nested)
createPerm(t, ctx, store, "pcwm-y")
addDependencyWithMeta(t, ctx, store, "pcwm-c3", "pcwm-y", types.DepBlocks, "")
nested["pcwm-y"] = false
nested["pcwm-c3"] = true // by Y, for now
step("a blocker of its own on the grandchild", nested)
if err := store.CloseIssue(ctx, "pcwm-y", "done", "tester", ""); err != nil {
t.Fatalf("close pcwm-y: %v", err)
}
nested["pcwm-c3"] = false
step("its own blocker closed", nested)
}

View file

@ -17,7 +17,7 @@ package dolt
//
// What the differential half proves narrowed in gastownhall/beads#6291: both
// sides — the ground-truth producer (batched templates) and the system under
// test (full repair) — now render from the one shouldBeBlockedIDsUnionSQL
// test (full repair) — now render from the one shouldBeBlockedIDsUnionCoreSQL
// builder, so write-path-vs-full-repair agreement no longer cross-checks two
// independently written SQL forms. It validates scope-splicing (the batched
// legs' `AND d.issue_id IN (batch)`) and the waits-for DISTINCT barrier, which

View file

@ -0,0 +1,291 @@
package dolt
import (
"context"
"fmt"
"testing"
"time"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/types"
)
// Blocked-state cost at store scale (gastownhall/beads#6506, #6288).
//
// The timings below are the ones the parent-child cascade can regress, and the
// ones a change to shouldBeBlockedIDsUnionCoreSQL must be measured against
// BEFORE it ships:
//
// - RecomputeAllIsBlockedInTx — the full repair. It runs on every dolt
// pull/merge (versioncontrolops/blocked_recompute.go), on federation sync,
// and behind `bd doctor --fix`, so its cost is paid by every clone on
// every sync, not only by an operator running doctor.
// - CountIsBlockedInconsistenciesInTx — the read-only detection behind the
// doctor "Blocked State" check, over an ALREADY CONSISTENT store, which is
// the case an operator actually hits.
// - RecomputeIsBlockedForIDsInTx at 2, 20 and 200 ids — the batched
// mark/unmark the incremental write path runs, isolated from the dolt
// commit around it. 200 is queryBatchSize (the bulk/import shape); 2 is the
// small-batch floor where per-statement cost cannot be amortized.
// - CloseIssue x100 — the incremental write path end to end, which runs the
// same union scoped to a batch.
//
// The fixture matters as much as the numbers: the parent-child legs
// short-circuit on `p.is_blocked = 1`, so a plane with no blocked rows
// measures nothing at all. This one carries a realistic mix — exogenously
// blocked epics that really do cascade, close-gate epics that (since #6506)
// must not, one depth-2 gate over a grandchild, and a large population of
// ordinary leaf-to-leaf blocks edges.
//
// Skipped under -short. It is a measurement, not an assertion: it has no
// pass/fail threshold, because a threshold in wall-clock time on shared CI
// hardware is a flake. Run it on both sides of a change and compare:
//
// BEADS_TEST_EMBEDDED_DOLT=1 go test ./internal/storage/dolt/ \
// -run TestBlockedStateScaleTiming -count=1 -v
const (
scaleEpics = 100 // bs-0000..bs-0099; 0..9 are roots, 10..99 hang under them
scaleIssues = 1000
// Leaves bs-0100..bs-0699 carry blocks edges; bs-0700..bs-0999 are their
// targets, of which all but the last scaleOpenTargets are closed.
scaleFirstTarget = 700
scaleOpenTargets = 60
scaleEdgesPerLeaf = 3
scaleClosesTimed = 100
scaleBenchTimeout = 20 * time.Minute
scaleBenchIDFormat = "bs-%04d"
)
func scaleID(n int) string { return fmt.Sprintf(scaleBenchIDFormat, n) }
func TestBlockedStateScaleTiming(t *testing.T) {
if testing.Short() {
t.Skip("scale timing: skipped under -short")
}
store, cleanup := setupTestStore(t)
defer cleanup()
ctx, cancel := context.WithTimeout(context.Background(), scaleBenchTimeout)
defer cancel()
built := time.Now()
deps := buildScaleFixture(ctx, t, store)
t.Logf("fixture: %d issues, %d dependency edges, built in %s",
scaleIssues, deps, time.Since(built).Round(time.Millisecond))
// Converge once, untimed: every timing below is over a CONSISTENT store,
// which is the state a real clone is in when doctor or a pull runs.
recomputeAllScale(ctx, t, store)
t.Logf("blocked rows: %d of %d issues", scaleBlockedCount(ctx, t, store), scaleIssues)
start := time.Now()
n := scaleCountInconsistencies(ctx, t, store)
t.Logf("TIMING doctor-count %s (reported %d inconsistencies)",
time.Since(start).Round(time.Millisecond), n)
start = time.Now()
changed := recomputeAllScale(ctx, t, store)
t.Logf("TIMING full-repair %s (changed %d rows)",
time.Since(start).Round(time.Millisecond), changed)
// The write path's SQL on its own: the batched mark/unmark the incremental
// recompute runs, over a spread of ids, with no commit. CloseIssue below is
// the honest end-to-end number, but most of it is a dolt commit, which on a
// loaded box swings further than the thing being measured.
//
// THREE batch sizes, because the batched path's cost per statement and its
// cost per id pull in opposite directions and one size hides the other. 200
// is queryBatchSize — one statement pair, the bulk/import shape an
// AffectedBy* fan-out reaches. 20 is an ordinary epic-sized fan-out. 2 is
// the floor: the per-STATEMENT overhead (a scoped exogeneity read, the
// union built once) with almost no ids to amortize it over, which is where
// a fixed per-statement cost shows up worst.
for _, bs := range []struct {
size, reps int
}{{2, 50}, {20, 20}, {200, 5}} {
batchIDs := make([]string, 0, bs.size)
for i := 0; i < bs.size; i++ {
batchIDs = append(batchIDs, scaleID((i*5)%scaleIssues))
}
txb, err := store.db.BeginTx(ctx, nil)
if err != nil {
t.Fatalf("begin batched-recompute tx: %v", err)
}
start = time.Now()
for k := 0; k < bs.reps; k++ {
if err := issueops.RecomputeIsBlockedForIDsInTx(ctx, txb, batchIDs); err != nil {
_ = txb.Rollback()
t.Fatalf("RecomputeIsBlockedForIDsInTx(%d ids): %v", bs.size, err)
}
}
elapsedBatch := time.Since(start)
_ = txb.Rollback()
t.Logf("TIMING batched-recompute-%03d %s for %d x %d ids (%s each)",
bs.size, elapsedBatch.Round(time.Millisecond), bs.reps, len(batchIDs),
(elapsedBatch / time.Duration(bs.reps)).Round(time.Microsecond))
}
// The write path end to end: close scaleClosesTimed open leaves one at a
// time, each through the store's own CloseIssue, which recomputes the
// affected batch and commits.
targets := make([]string, 0, scaleClosesTimed)
for i := scaleFirstTarget + scaleOpenTargets; len(targets) < scaleClosesTimed && i < scaleIssues; i++ {
targets = append(targets, scaleID(i))
}
for i := 100; len(targets) < scaleClosesTimed && i < scaleFirstTarget; i++ {
targets = append(targets, scaleID(i))
}
start = time.Now()
for _, id := range targets {
if err := store.CloseIssue(ctx, id, "scale timing", "tester", ""); err != nil {
t.Fatalf("close %s: %v", id, err)
}
}
elapsed := time.Since(start)
t.Logf("TIMING write-path %s for %d CloseIssue (%s each)",
elapsed.Round(time.Millisecond), len(targets),
(elapsed / time.Duration(len(targets))).Round(time.Microsecond))
}
// buildScaleFixture writes the graph and returns the number of dependency
// edges it added.
//
// Every blocks edge goes in BEFORE any parent-child edge, and the parent-child
// edges go in with SkipCycleCheck. That is not a convenience: the close-gate
// shape is refused once both edges exist, in either order
// (CheckBlockingHierarchyInTx rejects a blocker that is a descendant,
// CheckDependencyCycleInTx rejects the parent-child edge that closes the
// loop), so one-edge-at-a-time is how the shape arrives in a real store too.
func buildScaleFixture(ctx context.Context, t *testing.T, store *DoltStore) int {
t.Helper()
issues := make([]*types.Issue, 0, scaleIssues)
for i := 0; i < scaleIssues; i++ {
issues = append(issues, &types.Issue{
ID: scaleID(i),
Title: "scale " + scaleID(i),
Status: types.StatusOpen,
Priority: 2,
IssueType: types.TypeTask,
})
}
if err := store.RunInTransaction(ctx, "scale fixture: issues", func(tx storage.Transaction) error {
return tx.CreateIssues(ctx, issues, "tester")
}); err != nil {
t.Fatalf("create scale issues: %v", err)
}
type edge struct {
src, tgt string
ty types.DependencyType
}
var edges []edge
openTarget := func(k int) string {
return scaleID(scaleFirstTarget + scaleOpenTargets - 1 - (k % scaleOpenTargets))
}
// Roots 0..4: a close gate over a GRANDCHILD — leaf 110+r hangs under epic
// 10+r, which hangs under root r. This is the depth-2 lock: it must not
// darken anything under the root.
for r := 0; r < 5; r++ {
edges = append(edges, edge{scaleID(r), scaleID(110 + r), types.DepBlocks})
}
// Roots 5..9: exogenously blocked, so their whole subtree inherits.
for r := 5; r < 10; r++ {
edges = append(edges, edge{scaleID(r), openTarget(r), types.DepBlocks})
}
// Epics 10..49: exogenously blocked. Epics 50..79: a close gate on one of
// their own direct children. Epics 80..99: clear.
for e := 10; e < 50; e++ {
edges = append(edges, edge{scaleID(e), openTarget(e), types.DepBlocks})
}
for e := 50; e < 80; e++ {
edges = append(edges, edge{scaleID(e), scaleID(100 + e), types.DepBlocks})
}
// Ordinary leaf-to-leaf blocks edges: sources 100..699, targets 700..999,
// most of which are closed below, so only some sources end up blocked.
strides := []int{13, 29, 47}
for i := 100; i < scaleFirstTarget; i++ {
for k := 0; k < scaleEdgesPerLeaf; k++ {
tgt := scaleFirstTarget + (i*strides[k]+7*k+3)%(scaleIssues-scaleFirstTarget)
edges = append(edges, edge{scaleID(i), scaleID(tgt), types.DepBlocks})
}
}
// The hierarchy, last: leaves under epics, epics under roots.
for i := scaleEpics; i < scaleIssues; i++ {
edges = append(edges, edge{scaleID(i), scaleID(i % scaleEpics), types.DepParentChild})
}
for e := 10; e < scaleEpics; e++ {
edges = append(edges, edge{scaleID(e), scaleID(e % 10), types.DepParentChild})
}
// One transaction per chunk: a single transaction over every edge holds the
// whole fixture in the uncommitted overlay, which is the pathological read
// shape #6288 documents and would measure the fixture rather than the code.
const chunk = 250
for start := 0; start < len(edges); start += chunk {
end := start + chunk
if end > len(edges) {
end = len(edges)
}
batch := edges[start:end]
if err := store.RunInTransaction(ctx, "scale fixture: edges", func(tx storage.Transaction) error {
for _, e := range batch {
if err := tx.AddDependencyWithOptions(ctx,
&types.Dependency{IssueID: e.src, DependsOnID: e.tgt, Type: e.ty},
"tester", storage.DependencyAddOptions{SkipCycleCheck: true}); err != nil {
return fmt.Errorf("add %s -> %s (%s): %w", e.src, e.tgt, e.ty, err)
}
}
return nil
}); err != nil {
t.Fatalf("scale fixture edges: %v", err)
}
}
// Close most of the target pool so the blocked fraction is a fraction.
for i := scaleFirstTarget; i < scaleIssues-scaleOpenTargets; i++ {
if err := store.CloseIssue(ctx, scaleID(i), "scale fixture", "tester", ""); err != nil {
t.Fatalf("close scale target %s: %v", scaleID(i), err)
}
}
return len(edges)
}
func recomputeAllScale(ctx context.Context, t *testing.T, store *DoltStore) int64 {
t.Helper()
tx, err := store.db.BeginTx(ctx, nil)
if err != nil {
t.Fatalf("begin scale recompute tx: %v", err)
}
changed, err := issueops.RecomputeAllIsBlockedInTx(ctx, tx)
if err != nil {
_ = tx.Rollback()
t.Fatalf("scale RecomputeAllIsBlockedInTx: %v", err)
}
if err := tx.Commit(); err != nil {
t.Fatalf("commit scale recompute tx: %v", err)
}
return changed
}
func scaleCountInconsistencies(ctx context.Context, t *testing.T, store *DoltStore) int64 {
t.Helper()
n, err := issueops.CountIsBlockedInconsistenciesInTx(ctx, store.db)
if err != nil {
t.Fatalf("scale CountIsBlockedInconsistenciesInTx: %v", err)
}
return n
}
func scaleBlockedCount(ctx context.Context, t *testing.T, store *DoltStore) int {
t.Helper()
var n int
if err := store.db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM issues WHERE is_blocked = 1").Scan(&n); err != nil {
t.Fatalf("count blocked rows: %v", err)
}
return n
}

View file

@ -163,13 +163,14 @@ const statementCountToleranceDolt = 2
// lower these; a change for any other reason should be re-measured and
// re-pinned deliberately, not adjusted to make a failure go away.
//
// Re-pinned for the batch-create round-trip work (was 7008 / 14014 / 846);
// see pinnedEmbeddedStatementCounts in internal/storage/embeddeddolt for the
// Re-pinned for the batch-create round-trip work (was 7008 / 14014 / 846),
// then for the parent-child cascade (classic was 786); see
// pinnedEmbeddedStatementCounts in internal/storage/embeddeddolt for the
// per-change breakdown. The two backends now agree to within the tolerance.
var pinnedDoltStatementCounts = map[string]int64{
"356 (mol 1x)": 6396,
"712 (mol 2x)": 12791,
"40 (classic)": 786,
"40 (classic)": 835,
}
// BenchmarkLargeBatchApply_Dolt is gated by setupBenchStore's own

View file

@ -37,6 +37,11 @@ func (r *statementRecorder) ExecContext(ctx context.Context, q string, a ...any)
return r.DBTX.ExecContext(ctx, q, a...)
}
func (r *statementRecorder) QueryContext(ctx context.Context, q string, a ...any) (*sql.Rows, error) {
r.stmts = append(r.stmts, recordedStatement{q, a})
return r.DBTX.QueryContext(ctx, q, a...)
}
func explainPlan(t *testing.T, tx *sql.Tx, st recordedStatement) string {
t.Helper()
rows, err := tx.QueryContext(context.Background(), "EXPLAIN PLAN "+st.query, st.args...)
@ -62,14 +67,36 @@ func explainPlan(t *testing.T, tx *sql.Tx, st recordedStatement) string {
return plan
}
// assertRecursiveMembersAreLookups checks a walk built on reachabilityMembers:
// one recursive member per dependency table, each a lookup join from the
// frontier into that table's issue_id index.
func assertRecursiveMembersAreLookups(t *testing.T, plan string) {
t.Helper()
for _, want := range []string{"IndexedTableAccess(dependencies)", "IndexedTableAccess(wisp_dependencies)"} {
if !strings.Contains(plan, want) {
t.Errorf("plan has no %s:\n%s", want, plan)
}
}
if got := strings.Count(plan, "LookupJoin"); got != 2 {
t.Errorf("plan has %d LookupJoin, want 2 (one per recursive member):\n%s", got, plan)
}
if got := strings.Count(plan, "keys: r.node"); got != 2 {
t.Errorf("plan probes the edge tables by the frontier node %d times, want 2:\n%s", got, plan)
}
if strings.Contains(plan, "InnerJoin") || strings.Contains(plan, "HashJoin") || strings.Contains(plan, "MergeJoin") {
t.Errorf("a recursive member is not a lookup join:\n%s", plan)
}
}
// TestGraphWalkPlansHonorJoinHints proves the engine honors the
// JOIN_ORDER/LOOKUP_JOIN hints the per-edge reachability walks and the batched
// JOIN_ORDER/LOOKUP_JOIN hints the per-edge reachability walks, the ancestor
// walk that seeds a parent-child edge's recompute, and the batched
// blocked-state recompute carry — not just that the text is there. Without
// them the embedded planner already plans the cycle walk's (statistics-less)
// wisp_dependencies member as a scan join, and the recompute's legs can be
// driven from an index scan of every open row (the sql-server flip described
// at shouldBeBlockedIDsUnionScopedSQL). A deleted, misspelled, or no longer
// resolvable hint (e.g. a renamed alias) fails here.
// at shouldBeBlockedIDsUnionScopedPrecomputedSQL). A deleted, misspelled, or
// no longer resolvable hint (e.g. a renamed alias) fails here.
func TestGraphWalkPlansHonorJoinHints(t *testing.T) {
skipUnlessEmbeddedDolt(t)
fixture := newPristineEmbeddedDoltFixture(t, "walkplan")
@ -114,24 +141,29 @@ func TestGraphWalkPlansHonorJoinHints(t *testing.T) {
t.Fatalf("recorded %d statements, want 3 (one cycle walk, two ancestor walks)", len(rec.stmts))
}
for _, st := range rec.stmts {
plan := explainPlan(t, tx, st)
// One recursive member per dependency table, each a lookup join
// from the frontier into that table's issue_id index.
for _, want := range []string{"IndexedTableAccess(dependencies)", "IndexedTableAccess(wisp_dependencies)"} {
if !strings.Contains(plan, want) {
t.Errorf("plan has no %s:\n%s", want, plan)
}
}
if got := strings.Count(plan, "LookupJoin"); got != 2 {
t.Errorf("plan has %d LookupJoin, want 2 (one per recursive member):\n%s", got, plan)
}
if got := strings.Count(plan, "keys: r.node"); got != 2 {
t.Errorf("plan probes the edge tables by the frontier node %d times, want 2:\n%s", got, plan)
}
if strings.Contains(plan, "InnerJoin") || strings.Contains(plan, "HashJoin") || strings.Contains(plan, "MergeJoin") {
t.Errorf("a recursive member is not a lookup join:\n%s", plan)
assertRecursiveMembersAreLookups(t, explainPlan(t, tx, st))
}
})
// ancestorChainInTx runs on every parent-child write and merge replay; it
// is reached here the way a write reaches it.
t.Run("ancestor-chain walk", func(t *testing.T) {
rec := &statementRecorder{DBTX: tx}
if _, _, err := issueops.AffectedByDepChangeInTx(ctx, rec, id(n-7), id(n/3), types.DepParentChild); err != nil {
t.Fatal(err)
}
var walks []recordedStatement
for _, st := range rec.stmts {
if strings.Contains(st.query, "WITH RECURSIVE ancestors") {
walks = append(walks, st)
}
}
if len(walks) != 1 {
t.Fatalf("recorded %d ancestor walks, want 1", len(walks))
}
// The same members as the reachability walks; the reason filter is
// the outer SELECT's, so it adds no join.
assertRecursiveMembersAreLookups(t, explainPlan(t, tx, walks[0]))
})
t.Run("blocked-state recompute", func(t *testing.T) {

View file

@ -419,10 +419,25 @@ func TestLargeBatchApplyWallClock_Embedded(t *testing.T) {
//
// Net: 356 -613 (incl. the documented 1-statement jitter), 712 -1224,
// classic -60.
//
// Re-pinned for the parent-child cascade (gastownhall/beads#6506; classic
// was 786). A parent-child edge now carries only a parent's exogenous
// blockedness, and in this shape each parent-child dep add pays seven more
// reads:
// - appendSiblingsUnderAncestorsInTx reseeds the parent's other children:
// one ancestorChainInTx walk and four child probes (both dependency
// tables, both parent columns), +5.
// - The batched mark/unmark reads its exogeneity set first
// (batchExogeneity): the parent-kind probe and the issue-parent read,
// +2. The probe skips the wisp-parent read, and the ancestry prune costs
// nothing while that read is empty.
//
// Net: classic +49 (seven distinct parent-child adds; the eighth repeats a
// pair and is a no-op). 356 and 712 are unchanged.
var pinnedEmbeddedStatementCounts = map[string]int64{
"356 (mol 1x)": 6396,
large712ShapeName: 12790,
"40 (classic)": 786,
"40 (classic)": 835,
}
// BenchmarkLargeBatchApply_Embedded benchmarks issueops.ApplyBatchInTx on

View file

@ -284,15 +284,21 @@ func recomputeIsBlockedCounting(ctx context.Context, tx DBTX, issueIDs, wispIDs
var total int64
for {
var changed int64
// One exogeneity read per PASS, shared by every statement in it. It is
// a property of the PARENT, so the same answer serves both tables.
explained, err := subtreeExplainedParentsInTx(ctx, tx)
if err != nil {
return total, err
}
if len(issueIDs) > 0 {
n, err := recomputeAllIsBlockedPassInTx(ctx, tx, "issues", "i", "dependencies")
n, err := recomputeAllIsBlockedPassInTx(ctx, tx, "issues", "i", "dependencies", explained)
if err != nil {
return total, err
}
changed += n
}
if len(wispIDs) > 0 {
n, err := recomputeAllIsBlockedPassInTx(ctx, tx, "wisps", "w", "wisp_dependencies")
n, err := recomputeAllIsBlockedPassInTx(ctx, tx, "wisps", "w", "wisp_dependencies", explained)
if err != nil {
return total, err
}
@ -312,9 +318,12 @@ func recomputeIsBlockedCounting(ctx context.Context, tx DBTX, issueIDs, wispIDs
// table using the unbatched semi-join statements, returning rows corrected.
// Error wrapping mirrors runMarkUnmarkBatchedInTx verbatim — downstream
// consumers (wh-bridge-sync among them) match on these strings.
func recomputeAllIsBlockedPassInTx(ctx context.Context, tx DBTX, table, alias, depTable string) (int64, error) {
func recomputeAllIsBlockedPassInTx(
ctx context.Context, tx DBTX, table, alias, depTable string, explained subtreeExplainedParents,
) (int64, error) {
var changed int64
res, err := tx.ExecContext(ctx, markAllBlockedSQL(table, alias, depTable))
res, err := tx.ExecContext(ctx,
markAllBlockedSQL(table, alias, depTable, explained), explained.args()...)
if err != nil {
return changed, fmt.Errorf("recompute is_blocked (mark): %w", err)
}
@ -324,7 +333,8 @@ func recomputeAllIsBlockedPassInTx(ctx context.Context, tx DBTX, table, alias, d
}
changed += n
res, err = tx.ExecContext(ctx, unmarkAllBlockedSQL(table, alias, depTable))
res, err = tx.ExecContext(ctx,
unmarkAllBlockedSQL(table, alias, depTable, explained), explained.args()...)
if err != nil {
return changed, fmt.Errorf("recompute is_blocked (unmark): %w", err)
}
@ -340,54 +350,554 @@ func recomputeAllIsBlockedPassInTx(ctx context.Context, tx DBTX, table, alias, d
// same predicate, restructured so the engine can execute it as a semi-join.
//
//nolint:gosec // G201: table, alias, and depTable are constants from the only two callers.
func markAllBlockedSQL(table, alias, depTable string) string {
func markAllBlockedSQL(table, alias, depTable string, explained subtreeExplainedParents) string {
return fmt.Sprintf(`
UPDATE %[1]s %[2]s SET %[2]s.is_blocked = 1, %[2]s.updated_at = %[2]s.updated_at
WHERE %[2]s.is_blocked = 0
AND %[2]s.status <> 'closed' AND %[2]s.status <> 'pinned'
AND %[2]s.id IN (%[3]s)
`, table, alias, shouldBeBlockedIDsUnionSQL(depTable))
`, table, alias, shouldBeBlockedIDsUnionPrecomputedSQL(depTable, explained))
}
// unmarkAllBlockedSQL is the full-table analog of the batched unmark
// template. NOT IN is null-hostile — one NULL in the subquery result makes the
// predicate UNKNOWN for every row and the unmark silently stops firing — which
// is why every leg of shouldBeBlockedIDsUnionSQL carries an explicit
// is why every leg of the union carries an explicit
// d.issue_id IS NOT NULL guard.
//
//nolint:gosec // G201: table, alias, and depTable are constants from the only two callers.
func unmarkAllBlockedSQL(table, alias, depTable string) string {
func unmarkAllBlockedSQL(table, alias, depTable string, explained subtreeExplainedParents) string {
return fmt.Sprintf(`
UPDATE %[1]s %[2]s SET %[2]s.is_blocked = 0, %[2]s.updated_at = %[2]s.updated_at
WHERE %[2]s.is_blocked = 1
AND ( %[2]s.status = 'closed' OR %[2]s.status = 'pinned'
OR %[2]s.id NOT IN (%[3]s) )
`, table, alias, shouldBeBlockedIDsUnionSQL(depTable))
`, table, alias, shouldBeBlockedIDsUnionPrecomputedSQL(depTable, explained))
}
// shouldBeBlockedIDsUnionSQL selects, UNCORRELATED with any outer row, every
// depTable issue_id that currently has a reason to be blocked: one UNION leg
// per reason — an open blocks/conditional-blocks target (issue or wisp), a
// blocked parent-child parent (issue or wisp), a blocking waits-for gate.
// subtreeExplainedParents is one read of parentsExplainedBySubtreeSQL, pruned
// by pruneExplainedByAncestryInTx: the ids of blocked parents whose
// blockedness is solely explained by their own subtree, with nothing blocked
// above them that is not, and which therefore may not darken their children
// (gastownhall/beads#6506). The ids are split by the parent's kind.
//
// WHY THE IDS AND NOT THE SQL. A spliced `p.id NOT IN (deriving query)` is
// evaluated once per leg per statement, and a full repair pass is four
// statements over two parent kinds. Measured at 1000 issues / 2870 deps:
// 46 ms on origin/main, 420 ms with the set spliced into all four, 80 ms read
// once per pass and bound as ids. The read itself is the same query either
// way; what changes is how many times a pass pays for it. The BATCHED path
// makes the same trade against its own scope rather than the whole store
// (scopedExplainedParentsInTx) — spliced there, it measured 6.9x.
//
// The set is small by construction — blocked parents that carry a blocking
// reason of their own, all of whose reasons point inside their own subtree —
// so binding it as an IN-list is bounded by the blocked-parent count, not by
// the store.
type subtreeExplainedParents struct {
issueParents []string
wispParents []string
}
// args returns the bound ids in the order the union's legs consume them: the
// issue-parent leg (3) before the wisp-parent leg (4). A caller that splices
// the union more than once (countStaleIsBlockedSQL) repeats this list once per
// splice, in the same order.
func (e subtreeExplainedParents) args() []any {
out := make([]any, 0, len(e.issueParents)+len(e.wispParents))
for _, id := range e.issueParents {
out = append(out, id)
}
for _, id := range e.wispParents {
out = append(out, id)
}
return out
}
// notInPlaceholders renders the leg's exclusion for n bound ids — and nothing
// at all for none, because `NOT IN ()` is a syntax error and "exclude nothing"
// is the right reading of an empty set.
func notInPlaceholders(n int) string {
if n == 0 {
return ""
}
return "AND p.id NOT IN (" + strings.TrimSuffix(strings.Repeat("?,", n), ",") + ")"
}
// subtreeExplainedParentsInTx reads both kinds' sets, once, for every
// unbatched statement that needs them.
//
// It is ONE read per kind and not one per dependency table because exogeneity
// is a property of the PARENT — of its own blocking reasons and its own
// subtree — and says nothing about which table the children asking about it
// live in. The candidate restriction inside the query therefore spans both
// tables (unbatchedCandSources): a parent that is a parent anywhere is
// computed once and excluded everywhere. Halving the reads is the difference
// between the doctor count landing at 1.7x origin/main and at 1.4x.
func subtreeExplainedParentsInTx(ctx context.Context, tx DBTX) (subtreeExplainedParents, error) {
var read subtreeExplainedParents
var err error
read.issueParents, err = queryIDs(ctx, tx,
parentsExplainedBySubtreeSQL("issues", "dependencies", "depends_on_issue_id", unbatchedCandSources()))
if err != nil {
return read, fmt.Errorf("read issue parents explained by their own subtree: %w", err)
}
read.wispParents, err = queryIDs(ctx, tx,
parentsExplainedBySubtreeSQL("wisps", "wisp_dependencies", "depends_on_wisp_id", unbatchedCandSources()))
if err != nil && !isTableNotExistError(err) {
return read, fmt.Errorf("read wisp parents explained by their own subtree: %w", err)
}
// Every parent in the store was a candidate, so a blocked parent this read
// did not return is one its own reasons do not explain.
return pruneExplainedByAncestryInTx(ctx, tx, read, true)
}
// scopedExplainedParentsInTx is subtreeExplainedParentsInTx confined to ONE
// BATCH: the same two reads, but the candidate parents are read only from the
// batch's own dependency rows, so the set is derived for the handful of
// parents above the batch instead of for every parent in the store.
//
// WHY A READ AND NOT A SPLICE. The batched mark/unmark templates used to
// splice the deriving query into both of their parent-child legs,
// batch-scoped, which reads as the cheap option: the scope confines cand to
// the batch, so the subquery is small. It is not cheap. A spliced
// `p.id NOT IN (subquery)` is paid once per leg per STATEMENT, and a batched
// pass is a mark and an unmark, so one 200-id recompute derived the set four
// times. Measured at 1000 issues / 2870 deps on embedded dolt, one pass over
// 200 ids: 53 ms on origin/main, 422 ms with the query spliced into both legs
// of both statements (6.9x, against a 1.5x bar), 61 ms read once per batch and
// bound as ids. It is the same trade the unbatched statements make one level
// up (see subtreeExplainedParents): the read is the same query either way,
// and what changes is how many times a pass pays for it.
//
// scope is the batch predicate already rendered to placeholders
// ("AND d.issue_id IN (?,?,…)") and args its ids. Both parent kinds are read
// with the same scope, because exogeneity is a property of the PARENT and the
// batch is only what bounds which parents can be asked about.
func scopedExplainedParentsInTx(
ctx context.Context, tx DBTX, depTable, scope string, args []any,
) (subtreeExplainedParents, error) {
var read subtreeExplainedParents
hasIssueParent, hasWispParent, err := batchParentKindsInTx(ctx, tx, depTable, scope, args)
if err != nil {
if isTableNotExistError(err) {
return read, nil
}
return read, err
}
cands := []candSource{{depTable: depTable, scope: scope}}
if hasIssueParent {
read.issueParents, err = queryIDs(ctx, tx,
parentsExplainedBySubtreeSQL("issues", "dependencies", "depends_on_issue_id", cands), args...)
if err != nil {
return read, fmt.Errorf("read batch issue parents explained by their own subtree: %w", err)
}
}
if hasWispParent {
read.wispParents, err = queryIDs(ctx, tx,
parentsExplainedBySubtreeSQL("wisps", "wisp_dependencies", "depends_on_wisp_id", cands), args...)
if err != nil && !isTableNotExistError(err) {
return read, fmt.Errorf("read batch wisp parents explained by their own subtree: %w", err)
}
}
// Only the batch's own parents were candidates, so the parents above them
// are read by id as the walk climbs.
return pruneExplainedByAncestryInTx(ctx, tx, read, false)
}
// parentRef is one parent in pruneExplainedByAncestryInTx's walk: its id and
// its kind, which picks the tables its rows live in. The two id spaces are kept
// apart rather than trusted not to collide.
type parentRef struct {
id string
wisp bool
}
// parentEdge is one parent-child edge from child up to a parent that is
// blocked.
type parentEdge struct {
child, parent parentRef
}
// pruneExplainedByAncestryInTx drops from a read of parentsExplainedBySubtreeSQL
// every parent that is blocked FROM ABOVE: one with a blocked parent that is
// not itself explained, at any depth. It is the contract's recursive clause
// ("…or P's own parent-child parent is exogenously blocked") for parents that
// carry reasons of their own.
//
// WHY THE FIXPOINT DOES NOT CARRY IT. The read decides from a parent's OWN
// reason rows. The legs' p.is_blocked = 1 conjunct carries inheritance only
// for a parent with no reason row: such a parent is dark exactly while
// something above it cascades into it. A parent with a subtree reason is dark
// either way, so its bit says nothing about what is above it. Q blocks on
// outside X, pc P -> Q, P blocks on its own child C: the read explains P, so C
// comes bright while the epic above it is still gated by X. A one-level check
// ("no blocked parent outside the set") is not enough either. Put a second
// close gate between them (R blocks on X, pc Q -> R, Q blocks on P, pc P -> Q,
// P blocks on C): P's blocked parent Q is in the set, so P stays explained and
// R's gate never reaches C. Whether a parent is explained depends on the whole
// chain above it, and no is_blocked bit records that chain.
//
// So this resolves it here, over the few parents the read returned:
//
// - the edges: each explained parent's blocked parents
// (blockedParentEdgesInTx);
// - membership of every blocked parent the walk meets. A complete read
// already decides it: every parent in the store was a candidate, so a
// blocked parent it did not return is not explained. A batch-scoped read
// decided only the batch's parents, so the walk reads the rest by id
// (explainedAmongInTx) and keeps climbing while it finds explained ones;
// - the set itself (dropBlockedFromAbove): the GREATEST set of read parents
// all of whose blocked parents are in it. Start from everything read, and
// drop any parent with a blocked parent outside the set, until nothing
// drops. Greatest, because "exogenously blocked" needs a finite chain up
// to a real outside reason, so a malformed hierarchy cycle of close gates
// with nothing exogenous above it stays explained. The greatest set is
// unique, so the order of the drops cannot change it.
//
// COST. Nothing when the read is empty, which covers every store with no close
// gate and every batch with none at or above it. Otherwise it is one indexed
// edge read per level of nested close gates, plus a by-id read per level on
// the scoped path, over parents that are few by construction.
func pruneExplainedByAncestryInTx(
ctx context.Context, tx DBTX, read subtreeExplainedParents, complete bool,
) (subtreeExplainedParents, error) {
if len(read.issueParents) == 0 && len(read.wispParents) == 0 {
return read, nil
}
refs := read.refs()
// explained holds every blocked parent the walk has met: true while it is
// explained by its own subtree, false once it is known not to be.
explained := make(map[parentRef]bool, len(refs))
for _, p := range refs {
explained[p] = true
}
above, err := climbBlockedParentsInTx(ctx, tx, refs, explained, complete)
if err != nil {
return read, err
}
dropBlockedFromAbove(explained, above)
var out subtreeExplainedParents
for _, p := range refs {
switch {
case !explained[p]:
case p.wisp:
out.wispParents = append(out.wispParents, p.id)
default:
out.issueParents = append(out.issueParents, p.id)
}
}
return out, nil
}
// refs lists the read's parents with their kinds, issue parents first.
func (e subtreeExplainedParents) refs() []parentRef {
out := make([]parentRef, 0, len(e.issueParents)+len(e.wispParents))
for _, id := range e.issueParents {
out = append(out, parentRef{id: id})
}
for _, id := range e.wispParents {
out = append(out, parentRef{id: id, wisp: true})
}
return out
}
// climbBlockedParentsInTx is pruneExplainedByAncestryInTx's walk: up from the
// read's parents through their blocked parents, returning the edges (child to
// its blocked parents) and entering every blocked parent it meets in
// explained. On a complete read it stops after one level; on a scoped read it
// climbs on from each new parent explainedAmongInTx finds explained.
func climbBlockedParentsInTx(
ctx context.Context, tx DBTX, frontier []parentRef, explained map[parentRef]bool, complete bool,
) (map[parentRef][]parentRef, error) {
above := make(map[parentRef][]parentRef)
for len(frontier) > 0 {
edges, err := blockedParentEdgesInTx(ctx, tx, frontier)
if err != nil {
return nil, err
}
var unknown []parentRef
for _, e := range edges {
above[e.child] = append(above[e.child], e.parent)
if _, met := explained[e.parent]; !met {
explained[e.parent] = false
unknown = append(unknown, e.parent)
}
}
if complete || len(unknown) == 0 {
break
}
frontier, err = explainedAmongInTx(ctx, tx, unknown)
if err != nil {
return nil, err
}
for _, p := range frontier {
explained[p] = true
}
}
return above, nil
}
// dropBlockedFromAbove shrinks explained to the greatest set the contract
// allows: it drops any parent with a blocked parent outside the set, until
// nothing drops.
func dropBlockedFromAbove(explained map[parentRef]bool, above map[parentRef][]parentRef) {
for dropped := true; dropped; {
dropped = false
for p, ok := range explained {
if ok && !allExplained(above[p], explained) {
explained[p] = false
dropped = true
}
}
}
}
func allExplained(parents []parentRef, explained map[parentRef]bool) bool {
for _, q := range parents {
if !explained[q] {
return false
}
}
return true
}
// blockedParentEdgesInTx reads the parent-child edges from children up to a
// parent that is blocked. An edge lives in the child's dependency table and
// names the parent in the column for the parent's kind, so this is one lookup
// per child kind and parent kind, each keyed on the child's issue_id. The hints
// pin that drive, as on the batched legs.
//
//nolint:gosec // G201: table and column names are constants; the ids are ? placeholders.
func blockedParentEdgesInTx(ctx context.Context, tx DBTX, children []parentRef) ([]parentEdge, error) {
var issueIDs, wispIDs []string
for _, c := range children {
if c.wisp {
wispIDs = append(wispIDs, c.id)
} else {
issueIDs = append(issueIDs, c.id)
}
}
var out []parentEdge
for _, side := range []struct {
depTable string
ids []string
wisp bool
}{
{"dependencies", issueIDs, false},
{"wisp_dependencies", wispIDs, true},
} {
for start := 0; start < len(side.ids); start += queryBatchSize {
end := min(start+queryBatchSize, len(side.ids))
placeholders, args := buildSQLInClause(side.ids[start:end])
for _, parent := range []struct {
table, col string
wisp bool
}{
{"issues", "depends_on_issue_id", false},
{"wisps", "depends_on_wisp_id", true},
} {
edges, err := queryParentEdges(ctx, tx, fmt.Sprintf(`
SELECT /*+ JOIN_ORDER(d, p) LOOKUP_JOIN(d, p) */ d.issue_id, p.id
FROM %[1]s d
JOIN %[2]s p ON p.id = d.%[3]s
WHERE d.issue_id IN (%[4]s)
AND d.type = 'parent-child'
AND p.is_blocked = 1
`, side.depTable, parent.table, parent.col, placeholders), args, side.wisp, parent.wisp)
if err != nil {
return nil, fmt.Errorf("read blocked parents of explained parents: %w", err)
}
out = append(out, edges...)
}
}
}
return out, nil
}
// queryParentEdges runs one of blockedParentEdgesInTx's lookups and tags each
// (child, parent) row with the kinds the lookup was for. A table that does not
// exist has no edges.
func queryParentEdges(
ctx context.Context, tx DBTX, query string, args []any, childWisp, parentWisp bool,
) ([]parentEdge, error) {
rows, err := tx.QueryContext(ctx, query, args...)
if err != nil {
if isTableNotExistError(err) {
return nil, nil
}
return nil, err
}
defer func() { _ = rows.Close() }()
var out []parentEdge
for rows.Next() {
var childID, parentID string
if err := rows.Scan(&childID, &parentID); err != nil {
return nil, fmt.Errorf("scan: %w", err)
}
out = append(out, parentEdge{
child: parentRef{id: childID, wisp: childWisp},
parent: parentRef{id: parentID, wisp: parentWisp},
})
}
return out, rows.Err()
}
// explainedAmongInTx is parentsExplainedBySubtreeSQL for the named parents, by
// id. pruneExplainedByAncestryInTx uses it for the blocked parents above a
// batch that the batch-scoped read was not asked about.
//
// Unlike the wisp reads above, it does not tolerate a missing table, because it
// cannot meet one: the prune runs only on a non-empty read, and that read came
// from the same core query, which reads both item tables and both dependency
// tables whichever kind it asks about.
func explainedAmongInTx(ctx context.Context, tx DBTX, parents []parentRef) ([]parentRef, error) {
var out []parentRef
for _, kind := range []struct {
table, depTable string
wisp bool
}{
{"issues", "dependencies", false},
{"wisps", "wisp_dependencies", true},
} {
var ids []string
for _, p := range parents {
if p.wisp == kind.wisp {
ids = append(ids, p.id)
}
}
for start := 0; start < len(ids); start += queryBatchSize {
end := min(start+queryBatchSize, len(ids))
placeholders, args := buildSQLInClause(ids[start:end])
found, err := queryIDs(ctx, tx,
parentsExplainedBySubtreeCoreSQL(kind.table, kind.depTable, candByIDSQL(kind.table, placeholders)),
args...)
if err != nil {
return nil, fmt.Errorf("read explained parents above the batch: %w", err)
}
for _, id := range found {
out = append(out, parentRef{id: id, wisp: kind.wisp})
}
}
}
return out, nil
}
// batchParentKindsInTx reports which parent KINDS the batch can possibly ask
// about: whether any parent-child row in the batch names an issue parent, and
// whether any names a wisp parent.
//
// It is the cheap guard in front of the two reads above, and it is what keeps
// the SMALL-batch case honest. A kind with no parent-child row in the batch
// has an empty candidate set, hence an empty exogeneity set, so its read is
// pure overhead — and that overhead is not small: the read is a ten-join walk
// under two CTEs, which dolt spends ~8 ms planning whether it returns rows or
// not. Most batches name no wisp parent at all, and a batch of roots (or of
// rows with no hierarchy) names neither. Measured at 1000 issues / 2870 deps,
// per pass: a 2-id batch of unparented rows 37 ms with both reads run blind
// against 20 ms with this guard, which is origin/main's own number.
//
// One indexed aggregate, both columns at once, because COUNT ignores NULLs and
// a parent-child row carries exactly one of the two.
//
//nolint:gosec // G201: depTable is a constant from the caller; scope carries only ? placeholders.
func batchParentKindsInTx(
ctx context.Context, tx DBTX, depTable, scope string, args []any,
) (bool, bool, error) {
query := fmt.Sprintf(`
SELECT COUNT(d.depends_on_issue_id), COUNT(d.depends_on_wisp_id)
FROM %[1]s d
WHERE d.issue_id IS NOT NULL %[2]s
AND d.type = 'parent-child'
`, depTable, scope)
rows, err := tx.QueryContext(ctx, query, args...)
if err != nil {
return false, false, err
}
defer func() { _ = rows.Close() }()
var issueParents, wispParents int64
if rows.Next() {
if err := rows.Scan(&issueParents, &wispParents); err != nil {
return false, false, fmt.Errorf("scan batch parent kinds: %w", err)
}
}
if err := rows.Err(); err != nil {
return false, false, err
}
return issueParents > 0, wispParents > 0, nil
}
// candSource names one dependency table the candidate-parent set is read from,
// with the batch scope that confines it there (empty for the unbatched reads).
type candSource struct {
depTable string
scope string
}
// unbatchedCandSources is every table a parent-child edge can live in. The
// batched templates pass their own single scoped source instead.
func unbatchedCandSources() []candSource {
return []candSource{{depTable: "dependencies"}, {depTable: "wisp_dependencies"}}
}
func queryIDs(ctx context.Context, tx DBTX, query string, args ...any) ([]string, error) {
rows, err := tx.QueryContext(ctx, query, args...)
if err != nil {
return nil, err
}
defer func() { _ = rows.Close() }()
var out []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
out = append(out, id)
}
return out, rows.Err()
}
// shouldBeBlockedIDsUnionPrecomputedSQL is the union for the UNBATCHED
// statements: no batch scope, and the two parent-child legs subtract an
// already-read id list instead of re-deriving it (see
// subtreeExplainedParents). The caller binds explained.args() once per splice.
//
//nolint:gosec // G201: depTable is constant; the exclusions are ? placeholders.
func shouldBeBlockedIDsUnionPrecomputedSQL(depTable string, explained subtreeExplainedParents) string {
return shouldBeBlockedIDsUnionCoreSQL(depTable, "",
notInPlaceholders(len(explained.issueParents)),
notInPlaceholders(len(explained.wispParents)))
}
// shouldBeBlockedIDsUnionScopedPrecomputedSQL selects, UNCORRELATED with any
// outer row, every depTable issue_id that currently has a reason to be
// blocked: one UNION leg per reason — an open blocks/conditional-blocks target
// (issue or wisp), an EXOGENOUSLY blocked parent-child parent (issue or wisp;
// see parentsExplainedBySubtreeSQL), a blocking waits-for gate.
// Semantically, `<alias>.id IN (this select)` ≡ the disjunction of those
// reasons correlated on d.issue_id = <alias>.id, which is what the batched
// mark/unmark templates in blocked_state.go once spelled out as five
// correlated EXISTS — but the engine evaluates this form once as hash
// lookups instead of per outer row (bd-t9ypt). Full repair, doctor count and
// the batched templates (scoped, see below) now share this one definition.
// the batched templates share the one core below.
//
//nolint:gosec // G201: depTable is constant; waitsForGateBlockedSQL is a constant template.
func shouldBeBlockedIDsUnionSQL(depTable string) string {
return shouldBeBlockedIDsUnionScopedSQL(depTable, "")
}
// shouldBeBlockedIDsUnionScopedSQL is shouldBeBlockedIDsUnionSQL with an
// It carries an
// extra predicate on the dependency row appended to every leg's WHERE — "" for
// the whole table (full repair, doctor count), batchScopeSQL for the batched
// mark/unmark templates in blocked_state.go, which confine each leg to
// d.issue_id IN (batch) so the union stays index-sized per statement. scope is
// spliced verbatim: a %s inside it survives this Sprintf for the batch runner.
//
// Like the unbatched form above, and unlike the shape this round replaced, the
// two parent-child legs SUBTRACT AN ALREADY-READ id list rather than the query
// that derives it: the batch runner reads the set once per batch
// (scopedExplainedParentsInTx) and binds it into both statements. The caller
// binds explained.args() once per splice, interleaved with the batch ids in
// text order — expandBatchTemplate does that bookkeeping.
//
// The waits-for leg wraps its dependency rows in a DISTINCT derived table
// BEFORE the gate predicate (waitsForGateBlockedSQL, six correlated EXISTS
// over the parent-child children and the spawner) is applied, so the engine
@ -418,36 +928,53 @@ func shouldBeBlockedIDsUnionSQL(depTable string) string {
// (embedded) pins the hinted plans.
//
//nolint:gosec // G201: depTable and scope are constants; waitsForGateBlockedSQL is a constant template.
func shouldBeBlockedIDsUnionScopedSQL(depTable, scope string) string {
func shouldBeBlockedIDsUnionScopedPrecomputedSQL(
depTable, scope string, explained subtreeExplainedParents,
) string {
return shouldBeBlockedIDsUnionCoreSQL(depTable, scope,
notInPlaceholders(len(explained.issueParents)),
notInPlaceholders(len(explained.wispParents)))
}
// shouldBeBlockedIDsUnionCoreSQL is the five legs themselves. The two
// parent-child legs take their exogeneity condition as text: both callers
// pass an already-read IN-list rendered to placeholders (notInPlaceholders),
// saying "unless this parent is explained by its own subtree", and the empty
// string says "no parent is", which is what an empty set means.
//
//nolint:gosec // G201: depTable and scope are constants; the conditions come from the two callers above.
func shouldBeBlockedIDsUnionCoreSQL(depTable, scope, issueParentCond, wispParentCond string) string {
targetHint, parentHint := "", ""
if scope != "" {
targetHint = "/*+ JOIN_ORDER(d, t) LOOKUP_JOIN(d, t) */ "
parentHint = "/*+ JOIN_ORDER(d, p) LOOKUP_JOIN(d, p) */ "
}
return fmt.Sprintf(`
SELECT %[4]sd.issue_id FROM %[1]s d
SELECT %[6]sd.issue_id FROM %[1]s d
JOIN issues t ON t.id = d.depends_on_issue_id
WHERE d.issue_id IS NOT NULL %[3]s
AND (d.type = 'blocks' OR d.type = 'conditional-blocks')
AND t.status <> 'closed' AND t.status <> 'pinned'
UNION
SELECT %[4]sd.issue_id FROM %[1]s d
SELECT %[6]sd.issue_id FROM %[1]s d
JOIN wisps t ON t.id = d.depends_on_wisp_id
WHERE d.issue_id IS NOT NULL %[3]s
AND (d.type = 'blocks' OR d.type = 'conditional-blocks')
AND t.status <> 'closed' AND t.status <> 'pinned'
UNION
SELECT %[5]sd.issue_id FROM %[1]s d
SELECT %[7]sd.issue_id FROM %[1]s d
JOIN issues p ON p.id = d.depends_on_issue_id
WHERE d.issue_id IS NOT NULL %[3]s
AND d.type = 'parent-child'
AND p.is_blocked = 1
%[4]s
UNION
SELECT %[5]sd.issue_id FROM %[1]s d
SELECT %[7]sd.issue_id FROM %[1]s d
JOIN wisps p ON p.id = d.depends_on_wisp_id
WHERE d.issue_id IS NOT NULL %[3]s
AND d.type = 'parent-child'
AND p.is_blocked = 1
%[5]s
UNION
SELECT d.issue_id FROM (
SELECT DISTINCT d.issue_id, d.depends_on_issue_id, d.depends_on_wisp_id, d.metadata
@ -456,7 +983,236 @@ func shouldBeBlockedIDsUnionScopedSQL(depTable, scope string) string {
AND d.type = 'waits-for'
) d
WHERE (%[2]s)
`, depTable, waitsForGateBlockedSQL, scope, targetHint, parentHint)
`, depTable, waitsForGateBlockedSQL, scope, issueParentCond, wispParentCond, targetHint, parentHint)
}
// subtreeWalkDepth is how far parentsExplainedBySubtreeSQL walks up from a
// blocking reason's target looking for the parent that holds the reason — the
// depth to which "inside my own subtree" is decided.
//
// The walk is spelled as this many pairs of LEFT JOINs rather than as WITH
// RECURSIVE, and that is a measured choice, not a portability one (dolt runs
// recursive CTEs elsewhere in this package). At 1000 issues / 2870 deps the
// seeded recursive walk cost 45 ms per evaluation against 6 ms for four levels
// of joins: a recursive CTE pays a whole query's overhead per ITERATION, so
// its cost tracks hierarchy depth even when it returns a hundred rows. A fixed
// join chain pays once, and each level is an index probe.
//
// The price is a horizon. A reason target more than this many parent-child
// edges below its blocker reads as exogenous, which is the pre-#6506 behavior
// for that one pair: it propagates MORE, never less, so nothing is hidden that
// the old code showed, and the permanent lock this fix removes needs the
// blocker and the target in the SAME subtree within the horizon. Four covers
// epic -> sub-epic -> leg -> task, which is deeper than any dotted-id
// hierarchy this store has carried, and it is the knob to turn first if a
// real store is ever found below it.
//
// BOTH SIDES OF THE HORIZON ARE PINNED, by a pair of dolt tests that differ by
// one edge: TestParentChildCascade_ReasonAtTheWalkHorizon (a reason exactly
// this many levels down; the subtree comes bright) and
// _ReasonPastTheWalkHorizon (one level further; the subtree stays dark, which
// is origin/main's own answer, so that test passes unchanged on origin/main).
// Raising this constant makes the second one fail and name the rows that
// changed. Measured cost of raising it, at 1000 issues / 2870 deps: see the
// commit that introduced the pin.
const subtreeWalkDepth = 4
// parentsExplainedBySubtreeSQL is the parent-child legs' exogeneity test
// (gastownhall/beads#6506): the set of blocked parents whose blockedness is
// SOLELY explained by their own subtree, which the legs subtract with
// `p.id NOT IN (...)`. It decides once per statement, uncorrelated with any
// outer row, which blocked parents may not darken their children.
//
// The bug it closes: the legs used to cascade on p.is_blocked = 1 alone,
// WHATEVER set that bit. A parent that is blocked only by its own children —
// the "close gate on the epic" idiom, where P carries blocks edges onto C1 and
// C2 so P cannot close before them — therefore darkened C1 and C2, the very
// children it was waiting for. Neither child could ever appear in bd ready, so
// neither could be worked, so neither could close: a permanent lock. P's OWN
// is_blocked stays 1 (it really does depend on open children); only the
// propagation to its children changes.
//
// The contract: a parent-child edge propagates only EXOGENOUS blockedness. P
// counts as blocked FOR CHILD C iff P has a blocking reason — an open
// blocks/conditional-blocks target, or a blocking waits-for gate — whose
// target is OUTSIDE P'S OWN SUBTREE, or P's own parent-child parent is
// (recursively) exogenously blocked.
//
// SUBTREE, not direct children. A depth-1 reading of "own child" leaves the
// permanent lock alive one level down: P blocks on grandchild G, pc C -> P,
// pc G -> C. G is not P's direct child, so P reads exogenous, so P darkens C,
// so C darkens G — and G is the only row whose close can free P. The walk up
// the ancestor chain (subtreeWalkDepth) is what makes this a descendant test.
//
// SHAPE. The caller's condition is a subtraction:
//
// AND p.id NOT IN (this select, pruned by pruneExplainedByAncestryInTx)
//
// A parent cascades unless it appears here. A parent with NO blocking reason
// row of its own is absent from this set, because the set is built from reason
// rows, so it cascades. For such a parent that is the contract's recursive
// clause, carried by the EXISTING fixpoint and not by a second pass of SQL
// recursion: it is dark because ITS parent is, so its children inherit.
// Because the legs keep the p.is_blocked = 1 conjunct, a chain converges the
// way it always did, one level per fixpoint pass, and the single-pass doctor
// COUNT stays the documented lower bound. A parent that DOES carry a reason is
// dark whatever is above it, so the fixpoint cannot carry the clause for it:
// pruneExplainedByAncestryInTx drops it from the set when a blocked parent
// above it is not itself explained.
//
// Every reason row is either inside the parent's subtree or outside it, so
// "no reason row is exogenous" (MAX(...) = 0 below) is the whole membership
// test; the two bits the shape started with collapse to one.
//
// COST (gastownhall/beads#6288, wy-pyrus4, and B2 of the #6506 review). Three
// earlier cuts were built and measured at 1000 issues / 2870 deps, and all
// three are in the rejected pile:
//
// - a per-row correlated EXISTS whose body held two more correlated EXISTS,
// the exact shape #6288 removed from the waits-for leg: 3.3-3.7x on the
// full repair, 5.2x on the doctor count (the review's own numbers);
// - the set as a derived table LEFT JOINed on p.id: ~30x. A join operand is
// a join operand — dolt planned it as the inner side of a nested loop and
// rebuilt it per outer row, the correlated shape again in a derived
// table's clothes;
// - the same set built with WITH RECURSIVE: it blew the driver read timeout
// on the full repair (see subtreeWalkDepth).
//
// What is left is uncorrelated by construction. Nothing below references the
// outer row — that is what lets the engine build the set once per statement
// and probe it, the form this file already relies on for
// `<alias>.id IN (the should-be-blocked union)` (bd-t9ypt) — and it is the
// property to preserve in any future edit here. Measured against a leg with
// no exogeneity test at all: 3 ms -> 18 ms per evaluation, flat in the number
// of parent-child rows the leg returns.
//
// The set is also confined to the statement's own work:
//
// - cand is the set of parents the caller can possibly ask about. A batched
// mark/unmark passes ONE source — its own dependency table, carrying its
// own batch scope (see scopedExplainedParentsInTx) — so it computes the
// handful of parents above the batch. The unbatched read passes both
// tables unscoped, which is every row that is a parent at all and already
// prunes most of the table. The alias must be d: batchScopeSQL is written
// against d.issue_id and is spliced verbatim. It binds to this subquery's
// own FROM, not an enclosing leg's d — the set is not lateral and
// correlates with nothing outside it.
// - rsn is those parents' blocking reason rows, and is where the reason
// predicate is paid: once per candidate parent row, not once per
// parent-child edge in the leg.
//
// blockingReasonSQL mirrors union legs 1, 2 and 5 row by row; keep the three
// in step or the "solely explained by its own subtree" reading silently
// drifts.
//
// parentTable/parentDepTable/parentCol describe the PARENT's kind (issues +
// dependencies, or wisps + wisp_dependencies); cands name the tables the
// parent-child edges are read from, which are the CHILDREN's. A parent-child
// edge lives in the child's table and names the parent in parentCol.
//
//nolint:gosec // G201: every argument is a constant from the two call sites; scope is batchScopeSQL.
func parentsExplainedBySubtreeSQL(parentTable, parentDepTable, parentCol string, cands []candSource) string {
var cand strings.Builder
for i, c := range cands {
if i > 0 {
cand.WriteString("\n\t\t UNION")
}
fmt.Fprintf(&cand, `
SELECT DISTINCT d.%[1]s FROM %[2]s d
WHERE d.issue_id IS NOT NULL %[3]s
AND d.type = 'parent-child'
AND d.%[1]s IS NOT NULL`, parentCol, c.depTable, c.scope)
}
return parentsExplainedBySubtreeCoreSQL(parentTable, parentDepTable, cand.String())
}
// candByIDSQL is a cand body naming parents by id: placeholders is a rendered
// IN-list, for explainedAmongInTx.
//
//nolint:gosec // G201: parentTable is a constant; placeholders carries only ? placeholders.
func candByIDSQL(parentTable, placeholders string) string {
return fmt.Sprintf(`
SELECT c.id FROM %s c WHERE c.id IN (%s)`, parentTable, placeholders)
}
// parentsExplainedBySubtreeCoreSQL is parentsExplainedBySubtreeSQL over a cand
// body the caller has already built: the parent-child rows of a batch or of the
// whole store (parentsExplainedBySubtreeSQL), or parents named by id
// (candByIDSQL).
//
//nolint:gosec // G201: parentTable and parentDepTable are constants; cand is built from constants and ? placeholders.
func parentsExplainedBySubtreeCoreSQL(parentTable, parentDepTable, cand string) string {
// The ancestor walk: one pair of indexed LEFT JOINs per level, on the
// dependency tables directly. Materializing the parent-child edges into a
// CTE first was measured at 6.6x on the batched write path — the whole
// edge set of the store, built per statement, to answer a question about a
// handful of rows. Joined straight onto idx_dependencies_issue it is a
// probe per level per reason row instead.
var joins, match strings.Builder
child := "r.tgt"
for i := 1; i <= subtreeWalkDepth; i++ {
ia, wa := fmt.Sprintf("i%d", i), fmt.Sprintf("w%d", i)
fmt.Fprintf(&joins, `
LEFT JOIN dependencies %[1]s ON %[1]s.issue_id = %[3]s AND %[1]s.type = 'parent-child'
LEFT JOIN wisp_dependencies %[2]s ON %[2]s.issue_id = %[3]s AND %[2]s.type = 'parent-child'`,
ia, wa, child)
// The level's parent, whichever table and column carries it. A child
// has one parent-child edge, so at most one of these four is set.
parent := fmt.Sprintf(
"COALESCE(%[1]s.depends_on_issue_id, %[1]s.depends_on_wisp_id, %[2]s.depends_on_issue_id, %[2]s.depends_on_wisp_id)",
ia, wa)
if i > 1 {
match.WriteString(" OR ")
}
fmt.Fprintf(&match, "%s = r.pid", parent)
child = parent
}
return fmt.Sprintf(`
WITH
cand(pid) AS (%[3]s
),
rsn(pid, tgt) AS (
SELECT pr.issue_id, COALESCE(pr.depends_on_issue_id, pr.depends_on_wisp_id)
FROM %[2]s pr
JOIN %[1]s pp ON pp.id = pr.issue_id
JOIN cand ON cand.pid = pr.issue_id
WHERE pp.is_blocked = 1
AND pr.issue_id IS NOT NULL
AND (%[4]s)
),
bits(pid, exogenous) AS (
SELECT r.pid, MAX(CASE WHEN %[6]s THEN 0 ELSE 1 END)
FROM rsn r%[5]s
GROUP BY r.pid
)
SELECT b.pid FROM bits b WHERE b.exogenous = 0
`, parentTable, parentDepTable, cand,
blockingReasonSQL("pr"), joins.String(), match.String())
}
// blockingReasonSQL is the row-level spelling of "this dependency row is,
// right now, a reason for its own issue to be blocked" — union legs 1 and 2
// (an open blocks/conditional-blocks target, issue or wisp) and leg 5 (a
// blocking waits-for gate), for a dependency row aliased rowAlias.
//
// It is a per-ROW predicate, but its one caller evaluates it in a set: inside
// parentsExplainedBySubtreeSQL's aggregate, over the reason rows of the parents
// that statement can be asked about, once. It is never applied per row of an
// enclosing leg — that is the correlated shape #6288 removed and B2 measured.
//
//nolint:gosec // G201: rowAlias is a constant; waitsForGateBlockedSQLFor rewrites a constant template.
func blockingReasonSQL(rowAlias string) string {
return fmt.Sprintf(`
( ( (%[1]s.type = 'blocks' OR %[1]s.type = 'conditional-blocks')
AND ( EXISTS (SELECT 1 FROM issues bt
WHERE bt.id = %[1]s.depends_on_issue_id
AND bt.status <> 'closed' AND bt.status <> 'pinned')
OR EXISTS (SELECT 1 FROM wisps bt
WHERE bt.id = %[1]s.depends_on_wisp_id
AND bt.status <> 'closed' AND bt.status <> 'pinned') ) )
OR ( %[1]s.type = 'waits-for' AND (%[2]s) ) )
`, rowAlias, waitsForGateBlockedSQLFor(rowAlias))
}
// CountIsBlockedInconsistenciesInTx reports how many issue and wisp rows carry a
@ -464,7 +1220,7 @@ func shouldBeBlockedIDsUnionScopedSQL(depTable, scope string) string {
// detection behind the bd doctor "Blocked State" check (bd-6dnrw.37); the
// repair is RecomputeAllIsBlockedInTx.
//
// Both derive their membership test from the one shouldBeBlockedIDsUnionSQL
// Both derive their membership test from the one union
// builder above, and they are pinned together by the blocked-consistency
// lockstep test: a converged database counts 0, and any row this counts is one
// a recompute pass changes. The count is a single-pass lower bound — a
@ -474,13 +1230,20 @@ func shouldBeBlockedIDsUnionScopedSQL(depTable, scope string) string {
func CountIsBlockedInconsistenciesInTx(ctx context.Context, tx DBTX) (int64, error) {
var total int64
n, err := countRows(ctx, tx, countStaleIsBlockedSQL("issues", "i", "dependencies"))
// One exogeneity read for both statements; each splices the union twice,
// so the ids are bound twice per statement.
explained, err := subtreeExplainedParentsInTx(ctx, tx)
if err != nil {
return 0, fmt.Errorf("count stale is_blocked issues: %w", err)
}
args := append(explained.args(), explained.args()...)
n, err := countRows(ctx, tx, countStaleIsBlockedSQL("issues", "i", "dependencies", explained), args...)
if err != nil {
return 0, fmt.Errorf("count stale is_blocked issues: %w", err)
}
total += n
n, err = countRows(ctx, tx, countStaleIsBlockedSQL("wisps", "w", "wisp_dependencies"))
n, err = countRows(ctx, tx, countStaleIsBlockedSQL("wisps", "w", "wisp_dependencies", explained), args...)
if err != nil {
if isTableNotExistError(err) {
return total, nil
@ -502,14 +1265,14 @@ func CountIsBlockedInconsistenciesInTx(ctx context.Context, tx DBTX) (int64, err
// set-membership form of De Morgan over the shouldBeBlocked disjunction).
//
// is_blocked is 0 or 1, so the branches are mutually exclusive and the count is
// their sum. Built on the uncorrelated shouldBeBlockedIDsUnionSQL for the same
// their sum. Built on the uncorrelated union for the same
// reason as the full repair (bd-t9ypt): the correlated disjunction form ran per
// outer row and cost whole-table scans per branch. table/alias/depTable are
// hardcoded constants supplied by the only two callers.
//
//nolint:gosec // G201: table, alias, and depTable are constant; only the constant gate SQL is interpolated.
func countStaleIsBlockedSQL(table, alias, depTable string) string {
union := shouldBeBlockedIDsUnionSQL(depTable)
func countStaleIsBlockedSQL(table, alias, depTable string, explained subtreeExplainedParents) string {
union := shouldBeBlockedIDsUnionPrecomputedSQL(depTable, explained)
return fmt.Sprintf(`
SELECT COUNT(*) FROM %[1]s %[2]s
WHERE
@ -524,8 +1287,8 @@ func countStaleIsBlockedSQL(table, alias, depTable string) string {
}
// countRows runs a single COUNT(*) query and returns the scalar.
func countRows(ctx context.Context, tx DBTX, query string) (int64, error) {
rows, err := tx.QueryContext(ctx, query)
func countRows(ctx context.Context, tx DBTX, query string, args ...any) (int64, error) {
rows, err := tx.QueryContext(ctx, query, args...)
if err != nil {
return 0, err
}

View file

@ -4,7 +4,7 @@ import (
"context"
"database/sql"
"fmt"
"strings"
"regexp"
"github.com/steveyegge/beads/internal/types"
)
@ -129,6 +129,25 @@ const waitsForGateBlockedSQL = `
)
`
// waitsForGateRowAliasRE matches the dependency-row alias references in
// waitsForGateBlockedSQL — `d.` at a word boundary, which never matches the
// gate's own inner `cd.` rows (no boundary between c and d).
var waitsForGateRowAliasRE = regexp.MustCompile(`\bd\.`)
// waitsForGateBlockedSQLFor returns waitsForGateBlockedSQL with its
// dependency-row alias rewritten from d to alias, for callers that evaluate
// the gate somewhere the row cannot be called d — blockingReasonSQL applies it
// to a parent's own dependency rows inside a union leg whose outer row is
// already d (gastownhall/beads#6506). Rewriting beats shadowing: an inner d
// would resolve correctly by scope rules but reads as a bug at every later
// glance.
func waitsForGateBlockedSQLFor(alias string) string {
if alias == "d" {
return waitsForGateBlockedSQL
}
return waitsForGateRowAliasRE.ReplaceAllString(waitsForGateBlockedSQL, alias+".")
}
// RecomputeIsBlockedResult reports which issue tables had rows changed while
// the blocked-state fixpoint converged.
type RecomputeIsBlockedResult struct {
@ -166,27 +185,25 @@ func recomputeIsBlockedInTxWithResult(
if err != nil {
return result, err
}
issuePlan, err := planRecomputeInTx(ctx, tx, "issues", "dependencies",
markBlockedTemplateForIssues(), unmarkBlockedTemplateForIssues(), issueIDs, splitEdgeless)
issuePlan, err := planRecomputeInTx(ctx, tx, issuesBlockedSpec, issueIDs, splitEdgeless)
if err != nil {
return result, err
}
wispPlan, err := planRecomputeInTx(ctx, tx, "wisps", "wisp_dependencies",
markBlockedTemplateForWisps(), unmarkBlockedTemplateForWisps(), wispIDs, splitEdgeless)
wispPlan, err := planRecomputeInTx(ctx, tx, wispsBlockedSpec, wispIDs, splitEdgeless)
if err != nil {
return result, err
}
for pass := 0; ; pass++ {
var changed int64
n, err := issuePlan.runPassInTx(ctx, tx, pass == 0)
n, err := runRecomputePassInTx(ctx, tx, issuePlan, pass == 0)
if err != nil {
return result, err
}
changed += n
result.IssueRowsChanged = result.IssueRowsChanged || n > 0
n, err = wispPlan.runPassInTx(ctx, tx, pass == 0)
n, err = runRecomputePassInTx(ctx, tx, wispPlan, pass == 0)
if err != nil {
return result, err
}
@ -240,7 +257,7 @@ func markIsBlockedPassForIssuesInTx(ctx context.Context, tx DBTX, ids []string)
if len(ids) == 0 {
return 0, nil
}
return runMarkBatchedInTx(ctx, tx, markBlockedTemplateForIssues(), ids)
return runMarkBatchedInTx(ctx, tx, issuesBlockedSpec, ids)
}
// The mark/unmark templates explicitly assign updated_at to itself:
@ -251,38 +268,60 @@ func markIsBlockedPassForIssuesInTx(ctx context.Context, tx DBTX, ids []string)
// makes stale-guard/conflict-guard consumers treat the row as user-edited.
// An explicit assignment suppresses the ON UPDATE clause.
//
// Both templates decide membership through shouldBeBlockedIDsUnionScopedSQL,
// the same uncorrelated union the full repair and the doctor count use
// (blocked_consistency.go), scoped to the batch: one derived blocked set per
// batch, computed once and probed by hash. The previous shape — five
// correlated EXISTS per outer row — was re-executed per row by the engine:
// ~4 s per 200-id batch on committed, indexed data, and unbounded (>69 min
// observed on dolt 2.1.8) over a large uncommitted working set, where every
// probe re-read the uncommitted overlay (gastownhall/beads#6288).
// Both templates decide membership through
// shouldBeBlockedIDsUnionScopedPrecomputedSQL, the same uncorrelated union the
// full repair and the doctor count use (blocked_consistency.go), scoped to the
// batch: one derived blocked set per batch, computed once and probed by hash.
// The previous shape — five correlated EXISTS per outer row — was re-executed
// per row by the engine: ~4 s per 200-id batch on committed, indexed data, and
// unbounded (>69 min observed on dolt 2.1.8) over a large uncommitted working
// set, where every probe re-read the uncommitted overlay
// (gastownhall/beads#6288).
//
// The parent-child legs' exogeneity set (gastownhall/beads#6506) is READ once
// per batch and BOUND into both statements as ids, never spliced as the query
// that derives it: spliced, a 200-id recompute paid for the derivation four
// times and measured 6.9x origin/main (scopedExplainedParentsInTx has the
// numbers). Which is why the templates are built per batch rather than once:
// the number of bound ids is a property of the batch.
//
// The batch IN-list therefore appears more than once per statement — the
// outer row filter plus one per union leg; expandBatchTemplate repeats the
// placeholders and the bound ids to match.
// outer row filter and one per union leg (six occurrences today) — and the
// exogeneity ids appear as already-rendered ? placeholders inside two of those
// legs. expandBatchTemplate binds both groups in TEXT order.
// batchScopeSQL is the per-leg predicate that confines the should-be-blocked
// union to the batch (see shouldBeBlockedIDsUnionScopedSQL); its %s is filled
// with the batch placeholders by expandBatchTemplate, never by fmt here.
// union to the batch (see shouldBeBlockedIDsUnionScopedPrecomputedSQL); its %s
// is filled with the batch placeholders by expandBatchTemplate, and by fmt
// here ONLY for the batch-scoped exogeneity read, which binds its own ids.
const batchScopeSQL = "AND d.issue_id IN (%s)"
func markBlockedTemplateForIssues() string {
return markBlockedTemplate("issues", "i", "dependencies")
// blockedTableSpec names the one table a batched mark/unmark pass runs over.
// The batched statements are now built per batch (the exogeneity ids are), so
// the runner takes the table rather than a finished template.
type blockedTableSpec struct {
table, alias, depTable string
}
func unmarkBlockedTemplateForIssues() string {
return unmarkBlockedTemplate("issues", "i", "dependencies")
var (
issuesBlockedSpec = blockedTableSpec{table: "issues", alias: "i", depTable: "dependencies"}
wispsBlockedSpec = blockedTableSpec{table: "wisps", alias: "w", depTable: "wisp_dependencies"}
)
func markBlockedTemplateForIssues(explained subtreeExplainedParents) string {
return markBlockedTemplate(issuesBlockedSpec, explained)
}
func markBlockedTemplateForWisps() string {
return markBlockedTemplate("wisps", "w", "wisp_dependencies")
func unmarkBlockedTemplateForIssues(explained subtreeExplainedParents) string {
return unmarkBlockedTemplate(issuesBlockedSpec, explained)
}
func unmarkBlockedTemplateForWisps() string {
return unmarkBlockedTemplate("wisps", "w", "wisp_dependencies")
func markBlockedTemplateForWisps(explained subtreeExplainedParents) string {
return markBlockedTemplate(wispsBlockedSpec, explained)
}
func unmarkBlockedTemplateForWisps(explained subtreeExplainedParents) string {
return unmarkBlockedTemplate(wispsBlockedSpec, explained)
}
// markBlockedTemplate is the batched mark statement for one table: the
@ -290,64 +329,106 @@ func unmarkBlockedTemplateForWisps() string {
// batch's own dependency rows, so `<alias>.id IN (union)` is exactly the
// old correlated disjunction for every id in the batch.
//
//nolint:gosec // G201: table, alias, and depTable are constants from the four callers above.
func markBlockedTemplate(table, alias, depTable string) string {
//nolint:gosec // G201: the spec's fields are constants from the four callers above.
func markBlockedTemplate(spec blockedTableSpec, explained subtreeExplainedParents) string {
return fmt.Sprintf(`
UPDATE %[1]s %[2]s SET %[2]s.is_blocked = 1, %[2]s.updated_at = %[2]s.updated_at
WHERE %[2]s.id IN (%%s)
AND %[2]s.is_blocked = 0
AND %[2]s.status <> 'closed' AND %[2]s.status <> 'pinned'
AND %[2]s.id IN (%[3]s)
`, table, alias, shouldBeBlockedIDsUnionScopedSQL(depTable, batchScopeSQL))
`, spec.table, spec.alias,
shouldBeBlockedIDsUnionScopedPrecomputedSQL(spec.depTable, batchScopeSQL, explained))
}
// unmarkBlockedTemplate is the batched unmark statement for one table: the
// batch-scoped analog of unmarkAllBlockedSQL. NOT IN is null-hostile; the
// union's d.issue_id IS NOT NULL guards keep it total.
//
//nolint:gosec // G201: table, alias, and depTable are constants from the four callers above.
func unmarkBlockedTemplate(table, alias, depTable string) string {
//nolint:gosec // G201: the spec's fields are constants from the four callers above.
func unmarkBlockedTemplate(spec blockedTableSpec, explained subtreeExplainedParents) string {
return fmt.Sprintf(`
UPDATE %[1]s %[2]s SET %[2]s.is_blocked = 0, %[2]s.updated_at = %[2]s.updated_at
WHERE %[2]s.id IN (%%s)
AND %[2]s.is_blocked = 1
AND ( %[2]s.status = 'closed' OR %[2]s.status = 'pinned'
OR %[2]s.id NOT IN (%[3]s) )
`, table, alias, shouldBeBlockedIDsUnionScopedSQL(depTable, batchScopeSQL))
`, spec.table, spec.alias,
shouldBeBlockedIDsUnionScopedPrecomputedSQL(spec.depTable, batchScopeSQL, explained))
}
// expandBatchTemplate fills every %s in a batched template with the same
// IN-list placeholders and repeats the bound ids once per occurrence, in
// order. Templates carry the batch list in the outer row filter and in each
// leg of the scoped union (six occurrences today); a template with a single
// %s degrades to the plain Sprintf it always was. The count is textual, so a
// template must contain no other percent sign (no LIKE 'x%' pattern, no %%)
// and at least one %s — a template with none is a programmer error that
// surfaces as an %!(EXTRA …) syntax error at exec.
// expandBatchTemplate finishes a batched template: every %s takes the batch's
// IN-list placeholders, and the args come out in the order the finished
// statement reads them — the batch ids once per %s, and the already-rendered
// ? placeholders the template carries (the two parent-child legs' exogeneity
// ids, see notInPlaceholders) interleaved where they actually appear.
//
// TEXT ORDER is the whole contract, and it is why this walks the template
// instead of counting: the batch scope and the exogeneity ids alternate inside
// the union (leg 3's scope, leg 3's NOT IN, leg 4's scope, leg 4's NOT IN), so
// no amount of counting says where a bound group goes. A template with one %s
// and no ? degrades to the plain Sprintf it always was.
//
// The count is textual, so a template must contain no percent sign that is not
// one of the %s verbs (no LIKE 'x%' pattern, no %%) and no ? of its own beyond
// the bound group — both pinned by blocked_state_template_test.go. A mismatch
// between the ? count and the bound ids is a programmer error and is returned
// as one rather than sent to the engine, where it would surface as an opaque
// argument-count error.
//
//nolint:gosec // G201: tmpl is a constant template; only IN-clause placeholders are formatted in.
func expandBatchTemplate(tmpl, placeholders string, args []interface{}) (string, []interface{}) {
n := strings.Count(tmpl, "%s")
if n <= 1 {
return fmt.Sprintf(tmpl, placeholders), args
func expandBatchTemplate(
tmpl, placeholders string, args, bound []interface{},
) (string, []interface{}, error) {
var fills []interface{}
out := make([]interface{}, 0, len(args)+len(bound))
next := 0
for i := 0; i < len(tmpl); i++ {
switch {
case tmpl[i] == '%' && i+1 < len(tmpl) && tmpl[i+1] == 's':
fills = append(fills, placeholders)
out = append(out, args...)
i++
case tmpl[i] == '?':
if next < len(bound) {
out = append(out, bound[next])
}
next++
}
}
fills := make([]interface{}, n)
expanded := make([]interface{}, 0, n*len(args))
for k := range fills {
fills[k] = placeholders
expanded = append(expanded, args...)
if next != len(bound) {
return "", nil, fmt.Errorf(
"batched template carries %d ? placeholders, want %d bound ids", next, len(bound))
}
return fmt.Sprintf(tmpl, fills...), expanded
return fmt.Sprintf(tmpl, fills...), out, nil
}
func markIsBlockedPassForWispsInTx(ctx context.Context, tx DBTX, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
return runMarkBatchedInTx(ctx, tx, markBlockedTemplateForWisps(), ids)
return runMarkBatchedInTx(ctx, tx, wispsBlockedSpec, ids)
}
func runMarkUnmarkBatchedInTx(ctx context.Context, tx DBTX, markTmpl, unmarkTmpl string, ids []string) (int64, error) {
// batchExogeneity reads the batch-scoped exogeneity set for one chunk of ids:
// ONE read for the chunk, shared by the chunk's mark and its unmark.
//
// It is read before the mark, so the unmark sees the set as it was at the top
// of the chunk. That is the same staleness the unbatched full repair accepts
// (one read per PASS, see recomputeIsBlockedCounting), and it is safe for the
// same reason: both callers are fixpoint loops that stop only on a pass that
// changes NO rows, and on that pass the set was read against the state the
// pass ends in. So the state the loop converges to satisfies both predicates
// under the set derived from that very state; a stale read can only cost an
// extra pass, never a wrong fixpoint.
func batchExogeneity(
ctx context.Context, tx DBTX, spec blockedTableSpec, placeholders string, args []interface{},
) (subtreeExplainedParents, error) {
return scopedExplainedParentsInTx(ctx, tx, spec.depTable,
fmt.Sprintf(batchScopeSQL, placeholders), args)
}
func runMarkUnmarkBatchedInTx(ctx context.Context, tx DBTX, spec blockedTableSpec, ids []string) (int64, error) {
var changed int64
for start := 0; start < len(ids); start += queryBatchSize {
end := start + queryBatchSize
@ -355,8 +436,16 @@ func runMarkUnmarkBatchedInTx(ctx context.Context, tx DBTX, markTmpl, unmarkTmpl
end = len(ids)
}
placeholders, args := buildSQLInClause(ids[start:end])
explained, err := batchExogeneity(ctx, tx, spec, placeholders, args)
if err != nil {
return changed, fmt.Errorf("recompute is_blocked (exogeneity): %w", err)
}
stmt, stmtArgs := expandBatchTemplate(markTmpl, placeholders, args)
stmt, stmtArgs, err := expandBatchTemplate(
markBlockedTemplate(spec, explained), placeholders, args, explained.args())
if err != nil {
return changed, fmt.Errorf("recompute is_blocked (mark): %w", err)
}
res, err := tx.ExecContext(ctx, stmt, stmtArgs...)
if err != nil {
return changed, fmt.Errorf("recompute is_blocked (mark): %w", err)
@ -367,7 +456,11 @@ func runMarkUnmarkBatchedInTx(ctx context.Context, tx DBTX, markTmpl, unmarkTmpl
}
changed += n
stmt, stmtArgs = expandBatchTemplate(unmarkTmpl, placeholders, args)
stmt, stmtArgs, err = expandBatchTemplate(
unmarkBlockedTemplate(spec, explained), placeholders, args, explained.args())
if err != nil {
return changed, fmt.Errorf("recompute is_blocked (unmark): %w", err)
}
res, err = tx.ExecContext(ctx, stmt, stmtArgs...)
if err != nil {
return changed, fmt.Errorf("recompute is_blocked (unmark): %w", err)
@ -395,19 +488,19 @@ type recomputeChunk struct {
// RecomputeIsBlockedInTxWithResult call: the fixpoint loop writes only
// is_blocked, never dependency rows, so the split holds for every pass.
type recomputePlan struct {
table, markTmpl, unmarkTmpl string
chunks []recomputeChunk
spec blockedTableSpec
chunks []recomputeChunk
}
// planRecomputeInTx chunks ids exactly as runMarkUnmarkBatchedInTx would and
// splits each chunk by whether depTable holds a row with that id as its
// splits each chunk by whether spec.depTable holds a row with that id as its
// issue_id.
//
// Every leg of the should-be-blocked union the mark/unmark templates probe is
// scoped to `d.issue_id IN (batch)` on depTable
// (shouldBeBlockedIDsUnionScopedSQL), so an id with no depTable row of its own
// is never in that union: mark cannot select it, and unmark reduces to
// "is_blocked = 1 -> 0". Those ids — every freshly created issue without
// scoped to `d.issue_id IN (batch)` on spec.depTable
// (shouldBeBlockedIDsUnionScopedPrecomputedSQL), so an id with no row of its
// own there is never in that union: mark cannot select it, and unmark reduces
// to "is_blocked = 1 -> 0". Those ids — every freshly created issue without
// edges, which is most of what a large create or apply-batch recomputes —
// need not ride the two union statements, whose cost grows with the table.
//
@ -416,14 +509,14 @@ type recomputePlan struct {
// the create paths) or when the chunk is large; the single-issue recomputes of
// close, reopen, update and dependency edits keep exactly the statements they
// ran before the shortcut existed.
func planRecomputeInTx(ctx context.Context, tx DBTX, table, depTable, markTmpl, unmarkTmpl string, ids []string, splitEdgeless bool) (recomputePlan, error) {
plan := recomputePlan{table: table, markTmpl: markTmpl, unmarkTmpl: unmarkTmpl}
func planRecomputeInTx(ctx context.Context, tx DBTX, spec blockedTableSpec, ids []string, splitEdgeless bool) (recomputePlan, error) {
plan := recomputePlan{spec: spec}
for start := 0; start < len(ids); start += queryBatchSize {
end := min(start+queryBatchSize, len(ids))
chunk := recomputeChunk{withEdges: ids[start:end]}
if !createFastPathsDisabled.Load() && (splitEdgeless || end-start >= recomputeSplitMinChunk) {
var err error
chunk.withEdges, chunk.edgeless, err = splitByOwnDependencyRowsInTx(ctx, tx, depTable, ids[start:end])
chunk.withEdges, chunk.edgeless, err = splitByOwnDependencyRowsInTx(ctx, tx, spec.depTable, ids[start:end])
if err != nil {
return plan, err
}
@ -433,20 +526,27 @@ func planRecomputeInTx(ctx context.Context, tx DBTX, table, depTable, markTmpl,
return plan, nil
}
// runPassInTx is one fixpoint pass over the plan, statement-for-statement
// equivalent to runMarkUnmarkBatchedInTx over the plan's full ids. Per chunk:
// the mark and unmark templates run over the ids with edges — the union, scoped
// to them, holds exactly what it held scoped to the whole chunk — and then, on
// the first pass only, a plain unmark clears the edgeless ids. It runs after
// both templates so their unions see the edgeless rows' pre-pass is_blocked,
// as they did when those rows were cleared by the chunk's own unmark
// statement; on later passes it would match no row (nothing marks an edgeless
// id), so it is skipped.
func (p recomputePlan) runPassInTx(ctx context.Context, tx DBTX, firstPass bool) (int64, error) {
// runRecomputePassInTx is one fixpoint pass over the plan,
// statement-for-statement equivalent to runMarkUnmarkBatchedInTx over the
// plan's full ids. Per chunk: the mark and unmark templates run over the ids
// with edges — the union, scoped to them, holds exactly what it held scoped to
// the whole chunk, and so does the batch's exogeneity read (an edgeless id
// names no parent) — and then, on the first pass only, a plain unmark clears
// the edgeless ids. It runs after both templates so their unions see the
// edgeless rows' pre-pass is_blocked, as they did when those rows were cleared
// by the chunk's own unmark statement; on later passes it would match no row
// (nothing marks an edgeless id), so it is skipped.
//
// It is a function rather than a method on recomputePlan because the
// structural write guards follow bare-identifier calls only: as a method, the
// mark/unmark DML it reaches would be invisible from the exported recompute
// entry points, and their versionExemptions entries would read as stale
// (TestEveryBeadMutatorMintsOrIsExempt).
func runRecomputePassInTx(ctx context.Context, tx DBTX, p recomputePlan, firstPass bool) (int64, error) {
var changed int64
for _, chunk := range p.chunks {
if len(chunk.withEdges) > 0 {
n, err := runMarkUnmarkBatchedInTx(ctx, tx, p.markTmpl, p.unmarkTmpl, chunk.withEdges)
n, err := runMarkUnmarkBatchedInTx(ctx, tx, p.spec, chunk.withEdges)
changed += n
if err != nil {
return changed, err
@ -460,7 +560,7 @@ func (p recomputePlan) runPassInTx(ctx context.Context, tx DBTX, firstPass bool)
res, err := tx.ExecContext(ctx, fmt.Sprintf(`
UPDATE %[1]s t SET t.is_blocked = 0, t.updated_at = t.updated_at
WHERE t.id IN (%[2]s) AND t.is_blocked = 1
`, p.table, placeholders), args...)
`, p.spec.table, placeholders), args...)
if err != nil {
return changed, fmt.Errorf("recompute is_blocked (unmark): %w", err)
}
@ -507,7 +607,7 @@ func splitByOwnDependencyRowsInTx(ctx context.Context, tx DBTX, depTable string,
return withEdges, withoutEdges, nil
}
func runMarkBatchedInTx(ctx context.Context, tx DBTX, markTmpl string, ids []string) (int64, error) {
func runMarkBatchedInTx(ctx context.Context, tx DBTX, spec blockedTableSpec, ids []string) (int64, error) {
var changed int64
for start := 0; start < len(ids); start += queryBatchSize {
end := start + queryBatchSize
@ -515,8 +615,16 @@ func runMarkBatchedInTx(ctx context.Context, tx DBTX, markTmpl string, ids []str
end = len(ids)
}
placeholders, args := buildSQLInClause(ids[start:end])
explained, err := batchExogeneity(ctx, tx, spec, placeholders, args)
if err != nil {
return changed, fmt.Errorf("mark is_blocked (exogeneity): %w", err)
}
stmt, stmtArgs := expandBatchTemplate(markTmpl, placeholders, args)
stmt, stmtArgs, err := expandBatchTemplate(
markBlockedTemplate(spec, explained), placeholders, args, explained.args())
if err != nil {
return changed, fmt.Errorf("mark is_blocked: %w", err)
}
res, err := tx.ExecContext(ctx, stmt, stmtArgs...)
if err != nil {
return changed, fmt.Errorf("mark is_blocked: %w", err)
@ -582,6 +690,9 @@ func AffectedByDepChangeInTx(ctx context.Context, tx DBTX, source, target string
if err := loadWaitersOnSpawnerIDsInTx(ctx, tx, []string{target}, &issueSeed, issueSeen, &wispSeed, wispSeen); err != nil {
return nil, nil, err
}
if err := appendSiblingsUnderAncestorsInTx(ctx, tx, target, &issueSeed, issueSeen, &wispSeed, wispSeen); err != nil {
return nil, nil, err
}
}
return expandByParentChildDescendantsInTx(ctx, tx, issueSeed, wispSeed, issueSeen, wispSeen)
default:
@ -600,6 +711,9 @@ func AffectedByDepChangeForWispInTx(ctx context.Context, tx DBTX, source, target
if err := loadWaitersOnSpawnerIDsInTx(ctx, tx, []string{target}, &issueSeed, issueSeen, &wispSeed, wispSeen); err != nil {
return nil, nil, err
}
if err := appendSiblingsUnderAncestorsInTx(ctx, tx, target, &issueSeed, issueSeen, &wispSeed, wispSeen); err != nil {
return nil, nil, err
}
}
return expandByParentChildDescendantsInTx(ctx, tx, issueSeed, wispSeed, issueSeen, wispSeen)
default:
@ -607,6 +721,120 @@ func AffectedByDepChangeForWispInTx(ctx context.Context, tx DBTX, source, target
}
}
// appendSiblingsUnderAncestorsInTx seeds every existing parent-child child of
// parent AND of each of parent's own ancestors (of either kind) for recompute.
//
// A parent-child edge onto a parent that BLOCKS on the new child reclassifies
// that blocking reason from exogenous to subtree-derived, and the cascade legs
// propagate only exogenous blockedness (gastownhall/beads#6506,
// parentsExplainedBySubtreeSQL). So writing or removing one edge can flip the
// answer for the parent's OTHER children, which no other seed in this walk
// reaches: the close-gate epic is built one child at a time, and the second
// edge is what un-darkens the first child.
//
// The walk goes all the way UP because the exogeneity test is over the whole
// SUBTREE, not over direct children. Hanging G under C changes nothing about C
// but everything about any ANCESTOR of C that blocks on G: that reason just
// moved inside its subtree, so the ancestor stops darkening its own children.
// The reviewer's depth-2 lock is exactly this shape (P blocks G; pc C -> P,
// C2 -> P, G -> C), and only P's sibling set — not C's — carries the flip.
// Seeding each ancestor's whole sibling set, which
// expandByParentChildDescendantsInTx then expands to their subtrees, is what
// keeps the incremental write path agreeing with the full repair.
func appendSiblingsUnderAncestorsInTx(
ctx context.Context, tx DBTX,
parent string,
issueSeed *[]string, issueSeen map[string]bool,
wispSeed *[]string, wispSeen map[string]bool,
) error {
parents, err := ancestorChainInTx(ctx, tx, parent)
if err != nil {
return err
}
if len(parents) == 0 {
return nil
}
// The parents' own kinds are not known here, so probe both target columns;
// the one that does not match a parent simply returns no rows.
for _, w := range []struct {
depTable, parentCol string
seed *[]string
seen map[string]bool
}{
{"dependencies", "depends_on_issue_id", issueSeed, issueSeen},
{"wisp_dependencies", "depends_on_issue_id", wispSeed, wispSeen},
{"dependencies", "depends_on_wisp_id", issueSeed, issueSeen},
{"wisp_dependencies", "depends_on_wisp_id", wispSeed, wispSeen},
} {
if err := appendChildrenInTx(ctx, tx, w.depTable, w.parentCol, parents, w.seen, w.seed); err != nil {
return err
}
}
return nil
}
// ancestorChainInTx returns id, plus every parent-child ancestor of id THAT
// CARRIES A BLOCKING REASON OF ITS OWN — a dependency row that is not
// parent-child. Order is not meaningful.
//
// The filter is what keeps the walk from being quadratic. Only an ancestor
// that holds a blocking reason can have that reason reclassified by an edge
// below it, so only such an ancestor's children can flip; an ancestor with
// nothing but hierarchy edges contributes seeds that can never change. Without
// the filter a 70-deep chain reseeds every ancestor's subtree on every edge
// written into it, which is O(depth x size) per write and times out.
//
// It walks with isAncestorInTx's own recursive members (reachabilityMembers in
// dependencies.go): one per dependency table, each an index probe on issue_id
// pinned by join hints, merged by UNION distinct, so a malformed diamond or
// cycle in the hierarchy terminates by unique reachable node rather than
// running away. The members project NULL for a row that is not parent-child,
// which the outer filter drops. Joining the frontier to a derived UNION of the
// two tables instead, this walk's first shape, is the scan #7224 removed from
// isAncestorInTx (minutes per walk at 100k edges), and it runs here on every
// parent-child write and merge replay. TestGraphWalkPlansHonorJoinHints
// (embedded) pins the lookup plan.
//
//nolint:gosec // G201: the members are built from constant table names and type filters.
func ancestorChainInTx(ctx context.Context, tx DBTX, id string) ([]string, error) {
if id == "" {
return nil, nil
}
query := fmt.Sprintf(`
WITH RECURSIVE ancestors(node) AS (
SELECT ?
UNION
%s
)
SELECT node FROM ancestors
WHERE node IS NOT NULL
AND ( node = ?
OR EXISTS (SELECT 1 FROM dependencies x
WHERE x.issue_id = ancestors.node AND x.type <> 'parent-child')
OR EXISTS (SELECT 1 FROM wisp_dependencies x
WHERE x.issue_id = ancestors.node AND x.type <> 'parent-child') )
`, reachabilityMembers("ancestors", cycleDetectionTables(), "d.type = 'parent-child'"))
rows, err := tx.QueryContext(ctx, query, id, id)
if err != nil {
return nil, fmt.Errorf("load ancestor chain of %s: %w", id, err)
}
defer func() { _ = rows.Close() }()
var out []string
for rows.Next() {
var node string
if err := rows.Scan(&node); err != nil {
return nil, fmt.Errorf("scan ancestor chain of %s: %w", id, err)
}
out = append(out, node)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("ancestor chain rows for %s: %w", id, err)
}
return out, nil
}
func loadBlockingDependersInTx(
ctx context.Context, tx DBTX,
targetCol, id string,

View file

@ -25,6 +25,19 @@ func newBlockedStateResultMock(t *testing.T) (sqlmock.Sqlmock, DBTX) {
return mock, db
}
// expectBatchExogeneityRead is the batch-scoped exogeneity read every batched
// pass now runs before its statements: first the parent-kind guard, and then
// one read per kind the batch actually names. Answering the guard with no
// parents of either kind is also the pin on the guard itself — a batch with no
// parent-child row must not run either read.
//
// Ordered before the execs because sqlmock is ordered and the runner reads
// once per batch, then binds the ids into both statements.
func expectBatchExogeneityRead(mock sqlmock.Sqlmock, depTable string) {
mock.ExpectQuery(regexp.QuoteMeta("SELECT COUNT(d.depends_on_issue_id), COUNT(d.depends_on_wisp_id)")).
WillReturnRows(sqlmock.NewRows([]string{"issue_parents", "wisp_parents"}).AddRow(0, 0))
}
// expectOwnEdgesProbe expects planRecomputeInTx's probe of depTable and
// answers that every id in withEdges has a dependency row of its own.
func expectOwnEdgesProbe(mock sqlmock.Sqlmock, depTable string, withEdges ...string) {
@ -37,12 +50,20 @@ func expectOwnEdgesProbe(mock sqlmock.Sqlmock, depTable string, withEdges ...str
}
func expectBlockedStatePass(mock sqlmock.Sqlmock, table, alias string, mark, unmark driver.Result) {
expectBatchExogeneityRead(mock, blockedSpecFor(table).depTable)
mock.ExpectExec(regexp.QuoteMeta("UPDATE " + table + " " + alias + " SET " + alias + ".is_blocked = 1")).
WillReturnResult(mark)
mock.ExpectExec(regexp.QuoteMeta("UPDATE " + table + " " + alias + " SET " + alias + ".is_blocked = 0")).
WillReturnResult(unmark)
}
func blockedSpecFor(table string) blockedTableSpec {
if table == "wisps" {
return wispsBlockedSpec
}
return issuesBlockedSpec
}
func TestRecomputeIsBlockedInTxWithResult(t *testing.T) {
t.Run("issue rows changed", func(t *testing.T) {
mock, db := newBlockedStateResultMock(t)
@ -94,6 +115,7 @@ func TestRunMarkUnmarkBatchedInTxPropagatesRowsAffectedErrors(t *testing.T) {
for _, phase := range []string{"mark", "unmark"} {
t.Run(phase, func(t *testing.T) {
mock, db := newBlockedStateResultMock(t)
expectBatchExogeneityRead(mock, "dependencies")
if phase == "mark" {
mock.ExpectExec(regexp.QuoteMeta("UPDATE issues i SET i.is_blocked = 1")).
WillReturnResult(sqlmock.NewErrorResult(sentinel))
@ -105,7 +127,7 @@ func TestRunMarkUnmarkBatchedInTxPropagatesRowsAffectedErrors(t *testing.T) {
}
_, err := runMarkUnmarkBatchedInTx(
context.Background(), db, markBlockedTemplateForIssues(), unmarkBlockedTemplateForIssues(), []string{"issue-1"},
context.Background(), db, issuesBlockedSpec, []string{"issue-1"},
)
if !errors.Is(err, sentinel) {
t.Fatalf("err = %v, want rows-affected error", err)
@ -126,6 +148,7 @@ func TestRecomputeIsBlockedSkipsUnionForIDsWithoutEdges(t *testing.T) {
expectOwnEdgesProbe(mock, "dependencies", "issue-2")
// Pass 1: the union statements over the id with edges, then the plain
// unmark over the edgeless ids.
expectBatchExogeneityRead(mock, "dependencies")
mock.ExpectExec(regexp.QuoteMeta("UPDATE issues i SET i.is_blocked = 1")).
WillReturnResult(sqlmock.NewResult(0, 1))
mock.ExpectExec(regexp.QuoteMeta("UPDATE issues i SET i.is_blocked = 0")).
@ -134,6 +157,7 @@ func TestRecomputeIsBlockedSkipsUnionForIDsWithoutEdges(t *testing.T) {
WithArgs("issue-1", "issue-3").
WillReturnResult(sqlmock.NewResult(0, 1))
// Pass 2: only the id with edges.
expectBatchExogeneityRead(mock, "dependencies")
mock.ExpectExec(regexp.QuoteMeta("UPDATE issues i SET i.is_blocked = 1")).
WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectExec(regexp.QuoteMeta("UPDATE issues i SET i.is_blocked = 0")).

View file

@ -6,35 +6,51 @@ import (
)
// The batched mark/unmark templates reach the engine through
// expandBatchTemplate, which decides how many IN-lists to splice by counting
// "%s" textually. Two things are therefore load-bearing and invisible to the
// sqlmock suite next door — its expectations match an unanchored statement
// expandBatchTemplate, which fills the batch IN-lists by walking the template
// for "%s" and interleaves the exogeneity ids the template already carries as
// "?" placeholders. Three things are therefore load-bearing and invisible to
// the sqlmock suite next door — its expectations match an unanchored statement
// prefix and never assert bound args, so it stays green against a miscounted
// template: the occurrence count itself, and the absence of any other percent
// sign in the finished template (a future LIKE 'x%' or %d in a union leg would
// shift the count or corrupt the statement). Today a break surfaces only in the
// container-backed dolt suite; these tests pin it at the fast tier
// (gastownhall/beads#6291).
// template: the %s occurrence count, the absence of any other percent sign in
// the finished template (a future LIKE 'x%' or %d in a union leg would shift
// the count or corrupt the statement), and the absence of any "?" the caller
// did not bind. Today a break surfaces only in the container-backed dolt
// suite; these tests pin it at the fast tier (gastownhall/beads#6291).
// batchTemplateOccurrences is the number of IN-lists a batched template
// carries: the outer row filter, plus one per leg of the scoped
// batchTemplateOccurrences is the number of batch IN-lists a batched template
// carries: the outer row filter, and one per leg of the scoped
// should-be-blocked union.
//
// It was briefly 8: the parent-child legs' exogeneity set was spliced as the
// query that derives it, and that query carried the batch scope twice more.
// The set is now read once per batch and bound as ids
// (scopedExplainedParentsInTx), which is both 6.9x faster on a 200-id batch
// and why the ? group below exists.
const batchTemplateOccurrences = 6
func batchedTemplates() map[string]string {
// explainedFixture is a non-empty exogeneity set of each kind, so the bound-?
// plumbing is exercised rather than degenerating to the empty case.
func explainedFixture() subtreeExplainedParents {
return subtreeExplainedParents{
issueParents: []string{"parent-1", "parent-2"},
wispParents: []string{"wisp-parent-1"},
}
}
func batchedTemplates(explained subtreeExplainedParents) map[string]string {
return map[string]string{
"markBlockedTemplateForIssues": markBlockedTemplateForIssues(),
"unmarkBlockedTemplateForIssues": unmarkBlockedTemplateForIssues(),
"markBlockedTemplateForWisps": markBlockedTemplateForWisps(),
"unmarkBlockedTemplateForWisps": unmarkBlockedTemplateForWisps(),
"markBlockedTemplateForIssues": markBlockedTemplateForIssues(explained),
"unmarkBlockedTemplateForIssues": unmarkBlockedTemplateForIssues(explained),
"markBlockedTemplateForWisps": markBlockedTemplateForWisps(explained),
"unmarkBlockedTemplateForWisps": unmarkBlockedTemplateForWisps(explained),
}
}
func TestBatchedTemplatePercentBudget(t *testing.T) {
for name, tmpl := range batchedTemplates() {
for name, tmpl := range batchedTemplates(explainedFixture()) {
t.Run(name, func(t *testing.T) {
if got := strings.Count(tmpl, "%s"); got != batchTemplateOccurrences {
t.Errorf("%%s count = %d, want %d — expandBatchTemplate binds one id group per occurrence, so a changed count must be a deliberate edit here", got, batchTemplateOccurrences)
t.Errorf("%%s count = %d, want %d — expandBatchTemplate binds one batch id group per occurrence, so a changed count must be a deliberate edit here", got, batchTemplateOccurrences)
}
// The stray-percent guard: every percent sign in the finished
// template must belong to one of the %s verbs counted above.
@ -45,21 +61,51 @@ func TestBatchedTemplatePercentBudget(t *testing.T) {
}
}
func TestWaitsForGateBlockedSQLCarriesNoPercent(t *testing.T) {
// TestBatchedTemplateQuestionMarksAreTheExogeneitySet pins the other half of
// expandBatchTemplate's contract: the ONLY ? a template carries are the two
// parent-child legs' bound exogeneity ids, so an empty set leaves none.
func TestBatchedTemplateQuestionMarksAreTheExogeneitySet(t *testing.T) {
explained := explainedFixture()
want := len(explained.issueParents) + len(explained.wispParents)
for name, tmpl := range batchedTemplates(explained) {
t.Run(name, func(t *testing.T) {
if got := strings.Count(tmpl, "?"); got != want {
t.Errorf("? count = %d, want %d (the bound exogeneity ids)", got, want)
}
})
}
for name, tmpl := range batchedTemplates(subtreeExplainedParents{}) {
t.Run(name+"/empty", func(t *testing.T) {
if got := strings.Count(tmpl, "?"); got != 0 {
t.Errorf("? count = %d over an empty exogeneity set, want 0", got)
}
})
}
}
func TestWaitsForGateBlockedSQLCarriesNoPercentOrPlaceholder(t *testing.T) {
// The gate is spliced into every template as a resolved constant, so any
// percent sign it grew would land in the counted text above.
// percent sign or placeholder it grew would land in the counted text above.
if got := strings.Count(waitsForGateBlockedSQL, "%"); got != 0 {
t.Errorf("waitsForGateBlockedSQL %% count = %d, want 0", got)
}
if got := strings.Count(waitsForGateBlockedSQL, "?"); got != 0 {
t.Errorf("waitsForGateBlockedSQL ? count = %d, want 0", got)
}
}
func TestExpandBatchTemplateFillsEveryOccurrence(t *testing.T) {
placeholders, args := buildSQLInClause([]string{"issue-1", "issue-2"})
wantArgs := batchTemplateOccurrences * len(args)
explained := explainedFixture()
bound := explained.args()
wantArgs := batchTemplateOccurrences*len(args) + len(bound)
for name, tmpl := range batchedTemplates() {
for name, tmpl := range batchedTemplates(explained) {
t.Run(name, func(t *testing.T) {
stmt, stmtArgs := expandBatchTemplate(tmpl, placeholders, args)
stmt, stmtArgs, err := expandBatchTemplate(tmpl, placeholders, args, bound)
if err != nil {
t.Fatalf("expandBatchTemplate: %v", err)
}
if strings.ContainsAny(stmt, "%") {
t.Errorf("expanded statement still contains %%: %s", stmt)
@ -70,21 +116,53 @@ func TestExpandBatchTemplateFillsEveryOccurrence(t *testing.T) {
if got := len(stmtArgs); got != wantArgs {
t.Fatalf("arg count = %d, want %d — placeholders and args must stay in lockstep", got, wantArgs)
}
// Each occurrence gets the same group, repeated in order.
// TEXT ORDER: the batch group for the outer filter and legs 1-2,
// then leg 3's batch group followed by the issue parents, then leg
// 4's batch group followed by the wisp parents, then leg 5's.
var want []interface{}
for k := 0; k < batchTemplateOccurrences; k++ {
want = append(want, args...)
if k == 3 {
want = append(want, "parent-1", "parent-2")
}
if k == 4 {
want = append(want, "wisp-parent-1")
}
}
for i, arg := range stmtArgs {
if want := args[i%len(args)]; arg != want {
t.Errorf("arg %d = %v, want %v", i, arg, want)
if arg != want[i] {
t.Errorf("arg %d = %v, want %v", i, arg, want[i])
}
}
})
}
}
// TestExpandBatchTemplateRefusesABoundMismatch is the guard that keeps a
// miscounted bound group from reaching the engine as an opaque argument-count
// error: the template's ? placeholders and the bound ids must agree exactly.
func TestExpandBatchTemplateRefusesABoundMismatch(t *testing.T) {
placeholders, args := buildSQLInClause([]string{"issue-1"})
tmpl := markBlockedTemplateForIssues(subtreeExplainedParents{issueParents: []string{"parent-1"}})
if _, _, err := expandBatchTemplate(tmpl, placeholders, args, nil); err == nil {
t.Error("want an error when the template carries a ? the caller did not bind")
}
if _, _, err := expandBatchTemplate(
tmpl, placeholders, args, []interface{}{"parent-1", "parent-2"}); err == nil {
t.Error("want an error when the caller binds more ids than the template carries")
}
}
func TestExpandBatchTemplateSingleOccurrenceDegrades(t *testing.T) {
// A template with one %s is the plain Sprintf it always was: the args pass
// through untouched rather than being repeated.
// A template with one %s and no bound ids is the plain Sprintf it always
// was: the args pass through untouched rather than being repeated.
placeholders, args := buildSQLInClause([]string{"issue-1", "issue-2"})
stmt, stmtArgs := expandBatchTemplate("SELECT id FROM issues WHERE id IN (%s)", placeholders, args)
stmt, stmtArgs, err := expandBatchTemplate(
"SELECT id FROM issues WHERE id IN (%s)", placeholders, args, nil)
if err != nil {
t.Fatalf("expandBatchTemplate: %v", err)
}
if want := "SELECT id FROM issues WHERE id IN (?,?)"; stmt != want {
t.Errorf("stmt = %q, want %q", stmt, want)
@ -98,10 +176,10 @@ func TestExpandBatchTemplateSingleOccurrenceDegrades(t *testing.T) {
// should-be-blocked union's four joined legs: without them the Dolt
// sql-server planner intermittently drives the issues legs from an index
// scan of every open issue, 6-10 s per statement over an import's
// uncommitted working set (see shouldBeBlockedIDsUnionScopedSQL). The
// unscoped union (full repair, doctor count) keeps the planner's choice.
// uncommitted working set (see shouldBeBlockedIDsUnionScopedPrecomputedSQL).
// The unscoped union (full repair, doctor count) keeps the planner's choice.
func TestBatchedTemplatesPinLookupJoins(t *testing.T) {
for name, tmpl := range batchedTemplates() {
for name, tmpl := range batchedTemplates(explainedFixture()) {
for _, hint := range []string{"JOIN_ORDER(d, t) LOOKUP_JOIN(d, t)", "JOIN_ORDER(d, p) LOOKUP_JOIN(d, p)"} {
if got := strings.Count(tmpl, hint); got != 2 {
t.Errorf("%s carries %q %d times, want 2 (one per joined issues/wisps leg)", name, hint, got)
@ -109,7 +187,7 @@ func TestBatchedTemplatesPinLookupJoins(t *testing.T) {
}
}
for _, depTable := range []string{"dependencies", "wisp_dependencies"} {
union := shouldBeBlockedIDsUnionSQL(depTable)
union := shouldBeBlockedIDsUnionPrecomputedSQL(depTable, explainedFixture())
if strings.Contains(union, "LOOKUP_JOIN(d, t)") || strings.Contains(union, "LOOKUP_JOIN(d, p)") {
t.Errorf("unscoped union over %s should not pin its legs' joins", depTable)
}

View file

@ -593,12 +593,6 @@ func cycleReachabilityQuery(depTables []string) string {
// The reachable set is the same as the single-member walk over the UNION:
// UNION distinct merges what every member produces per step.
func reachabilityQuery(cte string, depTables []string, typeFilter string) string {
members := make([]string, 0, len(depTables))
for _, t := range depTables {
members = append(members, fmt.Sprintf(
"SELECT /*+ JOIN_ORDER(r, d) LOOKUP_JOIN(r, d) */ CASE WHEN %s THEN %s END FROM %s r JOIN %s d ON d.issue_id = r.node",
typeFilter, depTargetExpr("d"), cte, t))
}
return fmt.Sprintf(`
WITH RECURSIVE %s(node) AS (
SELECT ?
@ -606,7 +600,22 @@ func reachabilityQuery(cte string, depTables []string, typeFilter string) string
%s
)
SELECT COUNT(*) FROM %s WHERE node = ?
`, cte, strings.Join(members, "\n\t\t\tUNION\n\t\t\t"), cte)
`, cte, reachabilityMembers(cte, depTables, typeFilter), cte)
}
// reachabilityMembers is reachabilityQuery's recursive members, one per
// dependency table and joined by UNION, for any walk over the same edges
// (ancestorChainInTx in blocked_state.go shares them). A NULL node, which is
// what a row of another type projects, is the caller's to filter out of a
// walk's result.
func reachabilityMembers(cte string, depTables []string, typeFilter string) string {
members := make([]string, 0, len(depTables))
for _, t := range depTables {
members = append(members, fmt.Sprintf(
"SELECT /*+ JOIN_ORDER(r, d) LOOKUP_JOIN(r, d) */ CASE WHEN %s THEN %s END FROM %s r JOIN %s d ON d.issue_id = r.node",
typeFilter, depTargetExpr("d"), cte, t))
}
return strings.Join(members, "\n\t\t\tUNION\n\t\t\t")
}
func cycleDetectionTables() []string {

View file

@ -57,6 +57,10 @@ func TestReopenIssueInTxRetriesConditionalUpdateWhenLatestStatusIsCustomDone(t *
mock.ExpectQuery(`(?s)SELECT id FROM events`).
WillReturnRows(sqlmock.NewRows([]string{"id"}))
mock.ExpectExec(`(?s)INSERT INTO events`).WillReturnResult(sqlmock.NewResult(0, 1))
// The batched recompute's parent-kind guard: no parent-child row in the
// batch, so neither exogeneity read runs before the mark and unmark.
mock.ExpectQuery(`(?s)SELECT COUNT\(d\.depends_on_issue_id\)`).
WillReturnRows(sqlmock.NewRows([]string{"i", "w"}).AddRow(0, 0))
mock.ExpectExec(`(?s)UPDATE issues i SET`).WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectExec(`(?s)UPDATE issues i SET`).WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectRollback()

View file

@ -399,19 +399,29 @@ func (e *DependencyEndpointNotFoundError) Error() string {
func (e *DependencyEndpointNotFoundError) Unwrap() error { return e.Err }
// DependencyHierarchyConflictError is returned when a blocking dependency
// would gate an issue on one of its own ancestors or descendants. Either shape
// can never clear under the parent-child close/blocking semantics.
// would gate an issue on one of its own ancestors or descendants. Neither
// shape is how the hierarchy expresses "wait for what is under me".
type DependencyHierarchyConflictError struct {
IssueID string
BlockerID string
BlockerIsAncestor bool
}
// Error states the DESCENDANT refusal as a modeling refusal, not a
// consequence of the cascade.
//
// It used to read "blocked status cascades to descendants, so <blocker> would
// inherit the block and never close". That was true until
// gastownhall/beads#6506: a parent-child edge now propagates only the parent's
// EXOGENOUS blockedness, so a parent that blocks on something in its own
// subtree no longer darkens it, and the sentence described a deadlock that no
// longer happens. The refusal itself stands — a blocks edge is not the way to
// say "I close after my children" — so the reason has to say what is.
func (e *DependencyHierarchyConflictError) Error() string {
if e.BlockerIsAncestor {
return fmt.Sprintf("%s cannot be blocked by its ancestor %s: %s cannot close until its descendants finish, so the gate would never clear",
e.IssueID, e.BlockerID, e.BlockerID)
}
return fmt.Sprintf("%s cannot be blocked by its descendant %s: blocked status cascades to descendants, so %s would inherit the block and never close",
e.IssueID, e.BlockerID, e.BlockerID)
return fmt.Sprintf("%s cannot be blocked by its descendant %s: waiting on your own subtree is a close gate, not a blocks edge — use a waits-for gate over the children (bd gate)",
e.IssueID, e.BlockerID)
}

View file

@ -11,6 +11,7 @@ internal/testutil/fixtures/fixtures_test.go::TestXLargeDolt
internal/testutil/fixtures/fixtures_test.go::TestLargeFromJSONL
internal/storage/dolt/concurrent_test.go::TestHighContentionStress
internal/storage/dolt/concurrent_test.go::TestConcurrentWorkQueueDrain
internal/storage/dolt/blocked_scale_bench_test.go::TestBlockedStateScaleTiming
internal/storage/dolt/lease_test.go::TestConcurrentHeartbeatReclaimClose
internal/storage/uow/lostupdate_dolt_test.go::TestUOW_ConcurrentMergeOps_NoLostUpdate
internal/workapi/sweep_test.go::TestCandidateIDMatcherLargeFixture