fix(cook): drop duplicate blocks edge when waits_for infers from needs[0] (GH#3783) (#3875)
* fix(cook): drop duplicate blocks edge when waits_for infers from needs[0] (GH#3783)
collectDependencies emits one DepBlocks edge per entry in step.Needs and one
DepWaitsFor edge per step.WaitsFor. When waits_for has no explicit `from:`,
its spawner is inferred from step.Needs[0] — so both blocks fire on the same
(source, target) pair, and storage rightly rejects the duplicate with:
dependency <step> -> <X> already exists with type "blocks"
(requested "waits-for")
This silently broke the documented pattern where a step gates on the children
of a step it depends on (needs=[surveyor], waits_for=all-children).
Pre-compute the waits_for spawner before processing needs and skip the
DepBlocks edge for that specific target — the DepWaitsFor edge already
encodes the blocking semantics plus the gate metadata, so the blocks dep is
redundant. Adds an e2e regression test that pours a formula with the
collision shape and asserts pour succeeds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cook): dedupe waits_for spawner in depends_on loop too (GH#3783)
collectDependencies pre-computes the waits_for spawner step ID and skips
the redundant DepBlocks edge for it in the `needs` loop (the DepWaitsFor
edge subsumes the blocking semantics for that target). The same collision
was still possible from the `depends_on` loop: an explicit waits_for
`from:` spawner that also appears in a step's `depends_on` list would hit
the same "already exists with type blocks (requested waits-for)" storage
rejection GH#3783 fixed for `needs`. Move the spawner pre-computation above
both loops and apply the same skip in `depends_on`.
This completes PR gastownhall/beads#3875 per review, but does not resolve
the review's second, larger finding: waitsForGateBlockedSQL only blocks a
waiter while its spawner has an open parent-child child, never merely
because the spawner itself is still open, so dropping the blocks edge on
needs/depends_on opens a pre-fanout window where the waiter can become
ready before the spawner completes. Extending the gate to also block on
"spawner still open" was attempted and reverted here: it makes
TestIsBlocked_WaitsForDefaultGate fail, because that test documents the
opposite semantics for the default (all-children) gate — the waiter must
unblock once all children close, even while the spawner itself remains
open. Resolving the pre-fanout window requires a design decision on
whether that default-gate test's expectation is still correct, not a
mechanical SQL change.
Agent-Signature: claude-fable-5-medium on behalf of matt wilkie
* fix(storage): collapsed needs+waits_for edges also block while the spawner is open (GH#3783)
Design decision (superseding the reverted general-gate attempt in
52ba524a1): instead of changing waitsForGateBlockedSQL's semantics for
every waits-for edge, scope the spawner-open block to edges that
provably collapsed a redundant DepBlocks edge into their DepWaitsFor
edge.
- types.WaitsForMeta gains also_blocks (bool, omitempty). A new
types.NewWaitsForBlockingDependency wraps NewWaitsForDependency and
sets it; NewWaitsForDependency and every other existing caller are
unchanged, so plain waits-for edges keep exactly their prior
metadata shape.
- cmd/bd/cook.go collectDependencies now tracks whether it actually
skipped a depends_on/needs entry because it matched the waits_for
spawner (waitsForCollapsedBlocks); only then does it call
NewWaitsForBlockingDependency instead of NewWaitsForDependency. A
plain waits_for with no matching needs/depends_on entry never gets
the flag.
- waitsForGateBlockedSQL gets one more top-level OR clause: when
also_blocks is 'true' (COALESCEd to 'false' for legacy/plain rows,
so this is a no-op for them), the gate also blocks while the spawner
issue/wisp itself is open. It is a top-level OR specifically so it
overrides the any-children early-open carve-out: a collapsed edge
means the original needs/depends_on required the spawner itself to
close, so an early child-close under any-children must not unblock
the waiter while the spawner remains open. Confirmed by
TestIsBlocked_WaitsForAlsoBlocksOverridesAnyChildrenEarlyOpen.
- AffectedByStatusChangeInTx / AffectedByStatusChangeForWispInTx now
also seed direct waiters on the closing/reopening id as spawner
(loadWaitersOnSpawnerIDsInTx), so a spawner's own status change
triggers is_blocked recompute for a waiter connected only by a
DepWaitsFor edge (no DepBlocks edge to hang the old propagation
off of).
- cmd/bd/template.go cloneSubgraph: found while debugging the new
pour-level regression test — cloning a template's dependencies into
a poured mol never copied dep.Metadata, so every poured waits-for
edge silently reset to an empty/default gate with also_blocks unset
regardless of what collectDependencies produced. Now copies
Metadata through.
Existing TestIsBlocked_WaitsForDefaultGate and
TestIsBlocked_WaitsForAnyChildrenGate pass unmodified: general
waits-for gate semantics are untouched for every edge without
also_blocks.
New coverage: TestIsBlocked_WaitsForAlsoBlocksSpawnerOpen,
TestIsBlocked_WaitsForAlsoBlocksOverridesAnyChildrenEarlyOpen
(internal/storage/dolt/is_blocked_test.go), and
TestBdMolPour_NeedsAndWaitsForSameTarget_SpawnerOpenBlocksWaiter
(cmd/bd/cook_qnt_test.go, exercises the embedded-Dolt/GMS pour path
end to end).
JSON_UNQUOTE(JSON_EXTRACT(...)) on a JSON boolean true unquotes to the
string "true" (not "1") consistently on both the dolt-sql-server path
(internal/storage/dolt tests) and the embedded pure-Go GMS path
(internal/storage/embeddeddolt, probed directly) — the COALESCE(...,
'false') = 'true' predicate needed no engine-specific variant.
Agent-Signature: claude-fable-5-medium on behalf of matt wilkie
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
This commit is contained in:
parent
9146050b8a
commit
c81badec27
6 changed files with 450 additions and 40 deletions
|
|
@ -905,8 +905,41 @@ func cookFormula(ctx context.Context, s storage.DoltStorage, f *formula.Formula,
|
|||
func collectDependencies(step *formula.Step, idMapping map[string]string, deps *[]*types.Dependency) {
|
||||
issueID := idMapping[step.ID]
|
||||
|
||||
// Pre-compute the waits_for spawner so we can dedupe against depends_on
|
||||
// and needs below. When waits_for has no explicit `from:`, it infers its
|
||||
// spawner from needs[0] — and `depends_on`/`needs` would otherwise emit a
|
||||
// DepBlocks edge on the same (source, target) pair that `waits_for`
|
||||
// emits a DepWaitsFor edge on. Storage rejects the duplicate. The
|
||||
// DepWaitsFor edge subsumes the blocking semantics, so we skip the
|
||||
// redundant DepBlocks for that specific target (GH#3783).
|
||||
var waitsForSpec *formula.WaitsForSpec
|
||||
var waitsForSpawnerStepID string
|
||||
if step.WaitsFor != "" {
|
||||
waitsForSpec = formula.ParseWaitsFor(step.WaitsFor)
|
||||
if waitsForSpec != nil {
|
||||
waitsForSpawnerStepID = waitsForSpec.SpawnerID
|
||||
if waitsForSpawnerStepID == "" && len(step.Needs) > 0 {
|
||||
waitsForSpawnerStepID = step.Needs[0]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// waitsForCollapsedBlocks records whether we actually skipped emitting a
|
||||
// DepBlocks edge for the waits_for spawner below (i.e. the spawner step ID
|
||||
// really did appear in depends_on/needs and resolve to a known issue). If
|
||||
// so, the DepWaitsFor edge emitted below must carry also_blocks so it
|
||||
// does not silently drop the blocking semantics that edge collapsed away
|
||||
// (GH#3783 review gap).
|
||||
var waitsForCollapsedBlocks bool
|
||||
|
||||
// Process depends_on field
|
||||
for _, depID := range step.DependsOn {
|
||||
if depID == waitsForSpawnerStepID {
|
||||
// This target is also the waits_for spawner; the DepWaitsFor edge
|
||||
// emitted below subsumes the blocking semantics for this pair.
|
||||
waitsForCollapsedBlocks = true
|
||||
continue
|
||||
}
|
||||
depIssueID, ok := idMapping[depID]
|
||||
if !ok {
|
||||
continue // Will be caught during validation
|
||||
|
|
@ -921,6 +954,12 @@ func collectDependencies(step *formula.Step, idMapping map[string]string, deps *
|
|||
|
||||
// Process needs field - simpler alias for sibling dependencies
|
||||
for _, needID := range step.Needs {
|
||||
if needID == waitsForSpawnerStepID {
|
||||
// This target is also the waits_for spawner; the DepWaitsFor edge
|
||||
// emitted below subsumes the blocking semantics for this pair.
|
||||
waitsForCollapsedBlocks = true
|
||||
continue
|
||||
}
|
||||
needIssueID, ok := idMapping[needID]
|
||||
if !ok {
|
||||
continue // Will be caught during validation
|
||||
|
|
@ -934,24 +973,21 @@ func collectDependencies(step *formula.Step, idMapping map[string]string, deps *
|
|||
}
|
||||
|
||||
// Process waits_for field - fanout gate dependency
|
||||
if step.WaitsFor != "" {
|
||||
waitsForSpec := formula.ParseWaitsFor(step.WaitsFor)
|
||||
if waitsForSpec != nil {
|
||||
// Determine spawner ID
|
||||
spawnerStepID := waitsForSpec.SpawnerID
|
||||
if spawnerStepID == "" && len(step.Needs) > 0 {
|
||||
// Infer spawner from first need
|
||||
spawnerStepID = step.Needs[0]
|
||||
if waitsForSpec != nil && waitsForSpawnerStepID != "" {
|
||||
if spawnerIssueID, ok := idMapping[waitsForSpawnerStepID]; ok {
|
||||
// Spawner identity is the depends_on_id; metadata carries
|
||||
// the gate. A collapsed needs/depends_on edge additionally marks
|
||||
// also_blocks so the gate blocks while the spawner itself is
|
||||
// open, not only while it has an open child (GH#3783).
|
||||
var dep *types.Dependency
|
||||
var err error
|
||||
if waitsForCollapsedBlocks {
|
||||
dep, err = types.NewWaitsForBlockingDependency(issueID, spawnerIssueID, waitsForSpec.Gate)
|
||||
} else {
|
||||
dep, err = types.NewWaitsForDependency(issueID, spawnerIssueID, waitsForSpec.Gate)
|
||||
}
|
||||
|
||||
if spawnerStepID != "" {
|
||||
if spawnerIssueID, ok := idMapping[spawnerStepID]; ok {
|
||||
// Spawner identity is the depends_on_id; metadata carries
|
||||
// the gate.
|
||||
if dep, err := types.NewWaitsForDependency(issueID, spawnerIssueID, waitsForSpec.Gate); err == nil {
|
||||
*deps = append(*deps, dep)
|
||||
}
|
||||
}
|
||||
if err == nil {
|
||||
*deps = append(*deps, dep)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
185
cmd/bd/cook_qnt_test.go
Normal file
185
cmd/bd/cook_qnt_test.go
Normal file
|
|
@ -0,0 +1,185 @@
|
|||
//go:build cgo
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/types"
|
||||
)
|
||||
|
||||
// TestBdMolPour_NeedsAndWaitsForSameTarget is a regression test for GH#3783:
|
||||
// a step that has both `needs = [X]` and `waits_for = "all-children"` (with
|
||||
// no explicit spawner, so the spawner is inferred from needs[0] == X)
|
||||
// causes bd mol pour to fail with a dep-type collision —
|
||||
//
|
||||
// dependency <step> -> <X> already exists with type "blocks"
|
||||
// (requested "waits-for")
|
||||
//
|
||||
// collectDependencies in cmd/bd/cook.go emits two edges on the same
|
||||
// (source, target) pair: a DepBlocks edge from `needs`, then a DepWaitsFor
|
||||
// edge from `waits_for` (which infers its spawner from needs[0]). Storage
|
||||
// rightly rejects the duplicate.
|
||||
//
|
||||
// This blocks the documented pattern where a step gates on the children of
|
||||
// the step it depends on (i.e. "wait for the fanout from the surveyor").
|
||||
//
|
||||
// After the fix, the DepWaitsFor edge subsumes the DepBlocks edge for that
|
||||
// specific target — the waits-for dep already carries the blocking
|
||||
// semantics plus the gate metadata, so the blocks dep is redundant.
|
||||
func TestBdMolPour_NeedsAndWaitsForSameTarget(t *testing.T) {
|
||||
bd := buildEmbeddedBD(t)
|
||||
|
||||
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "qnt")
|
||||
|
||||
formulaDir := filepath.Join(beadsDir, "formulas")
|
||||
if err := os.MkdirAll(formulaDir, 0o755); err != nil {
|
||||
t.Fatalf("mkdir formulas dir: %v", err)
|
||||
}
|
||||
|
||||
const formulaTOML = `formula = "qnt-repro"
|
||||
version = 1
|
||||
type = "workflow"
|
||||
|
||||
[[steps]]
|
||||
id = "survey-workers"
|
||||
title = "Survey"
|
||||
type = "task"
|
||||
|
||||
[[steps]]
|
||||
id = "wait"
|
||||
title = "Wait for fanout"
|
||||
type = "task"
|
||||
needs = ["survey-workers"]
|
||||
waits_for = "all-children"
|
||||
`
|
||||
formulaPath := filepath.Join(formulaDir, "qnt-repro.formula.toml")
|
||||
if err := os.WriteFile(formulaPath, []byte(formulaTOML), 0o644); err != nil {
|
||||
t.Fatalf("write formula: %v", err)
|
||||
}
|
||||
|
||||
// Pour should succeed. Before the fix, this exits 1 with:
|
||||
// Error: pouring proto: failed to create dependency:
|
||||
// dependency qnt-mol-XXX -> qnt-mol-YYY already exists with type "blocks"
|
||||
// (requested "waits-for")
|
||||
out, err := bdRunWithFlockRetry(t, bd, dir, "mol", "pour", "qnt-repro")
|
||||
if err != nil {
|
||||
t.Fatalf("bd mol pour should succeed for a step with both needs and waits_for; got error: %v\noutput:\n%s", err, out)
|
||||
}
|
||||
|
||||
outStr := string(out)
|
||||
if strings.Contains(outStr, "already exists with type") {
|
||||
t.Errorf("pour output contains the dep-collision error this test was meant to prevent:\n%s", outStr)
|
||||
}
|
||||
}
|
||||
|
||||
// pourJSONResult mirrors the anonymous pourResult wrapper `bd mol pour --json`
|
||||
// emits (cmd/bd/pour.go): the embedded InstantiateResult fields plus attached
|
||||
// count and phase.
|
||||
type pourJSONResult struct {
|
||||
NewEpicID string `json:"new_epic_id"`
|
||||
IDMapping map[string]string `json:"id_mapping"`
|
||||
Created int `json:"created"`
|
||||
}
|
||||
|
||||
// readyIDs runs `bd ready --json` and returns the set of ready issue IDs.
|
||||
func readyIDs(t *testing.T, bd, dir string) map[string]bool {
|
||||
t.Helper()
|
||||
out, err := bdRunWithFlockRetry(t, bd, dir, "ready", "--json", "--limit", "0")
|
||||
if err != nil {
|
||||
t.Fatalf("bd ready --json: %v\noutput:\n%s", err, out)
|
||||
}
|
||||
var ready []types.IssueWithCounts
|
||||
if err := json.Unmarshal(out, &ready); err != nil {
|
||||
t.Fatalf("parse ready --json: %v\noutput:\n%s", err, out)
|
||||
}
|
||||
ids := make(map[string]bool, len(ready))
|
||||
for _, r := range ready {
|
||||
ids[r.ID] = true
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
// TestBdMolPour_NeedsAndWaitsForSameTarget_SpawnerOpenBlocksWaiter is the
|
||||
// regression test for the review gap in GH#3875: the fix for GH#3783 drops
|
||||
// the DepBlocks edge from `needs` onto the waits_for spawner on the premise
|
||||
// that the DepWaitsFor edge subsumes its blocking semantics. also_blocks
|
||||
// metadata (set on exactly this collapsed edge, see cmd/bd/cook.go) closes
|
||||
// the resulting pre-fanout window: waitsForGateBlockedSQL now additionally
|
||||
// blocks while the spawner itself is open, not only while it has an open
|
||||
// parent-child child.
|
||||
//
|
||||
// This asserts the waiter stays blocked immediately after pour (spawner
|
||||
// open, zero children spawned yet), and unblocks once the spawner closes.
|
||||
func TestBdMolPour_NeedsAndWaitsForSameTarget_SpawnerOpenBlocksWaiter(t *testing.T) {
|
||||
bd := buildEmbeddedBD(t)
|
||||
|
||||
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "qnw")
|
||||
|
||||
formulaDir := filepath.Join(beadsDir, "formulas")
|
||||
if err := os.MkdirAll(formulaDir, 0o755); err != nil {
|
||||
t.Fatalf("mkdir formulas dir: %v", err)
|
||||
}
|
||||
|
||||
const formulaTOML = `formula = "qnw-repro"
|
||||
version = 1
|
||||
type = "workflow"
|
||||
|
||||
[[steps]]
|
||||
id = "survey-workers"
|
||||
title = "Survey"
|
||||
type = "task"
|
||||
|
||||
[[steps]]
|
||||
id = "wait"
|
||||
title = "Wait for fanout"
|
||||
type = "task"
|
||||
needs = ["survey-workers"]
|
||||
waits_for = "all-children"
|
||||
`
|
||||
formulaPath := filepath.Join(formulaDir, "qnw-repro.formula.toml")
|
||||
if err := os.WriteFile(formulaPath, []byte(formulaTOML), 0o644); err != nil {
|
||||
t.Fatalf("write formula: %v", err)
|
||||
}
|
||||
|
||||
out, err := bdRunWithFlockRetry(t, bd, dir, "mol", "pour", "qnw-repro", "--json")
|
||||
if err != nil {
|
||||
t.Fatalf("bd mol pour --json: %v\noutput:\n%s", err, out)
|
||||
}
|
||||
|
||||
var result pourJSONResult
|
||||
if err := json.Unmarshal(out, &result); err != nil {
|
||||
t.Fatalf("parse pour --json: %v\noutput:\n%s", err, out)
|
||||
}
|
||||
|
||||
spawnerID, ok := result.IDMapping["qnw-repro.survey-workers"]
|
||||
if !ok {
|
||||
t.Fatalf("id_mapping missing qnw-repro.survey-workers: %#v", result.IDMapping)
|
||||
}
|
||||
waiterID, ok := result.IDMapping["qnw-repro.wait"]
|
||||
if !ok {
|
||||
t.Fatalf("id_mapping missing qnw-repro.wait: %#v", result.IDMapping)
|
||||
}
|
||||
|
||||
// The spawner is open with zero children spawned yet — the waiter must
|
||||
// be blocked, not ready. Before the also_blocks fix this regressed:
|
||||
// dropping the blocks edge left the waits-for gate blind to a still-open,
|
||||
// childless spawner.
|
||||
if readyIDs(t, bd, dir)[waiterID] {
|
||||
t.Fatalf("waiter %s is ready while its spawner %s is still open with no children spawned yet (pre-fanout window)", waiterID, spawnerID)
|
||||
}
|
||||
|
||||
if _, err := bdRunWithFlockRetry(t, bd, dir, "close", spawnerID, "--reason", "no fanout needed"); err != nil {
|
||||
t.Fatalf("bd close %s: %v", spawnerID, err)
|
||||
}
|
||||
|
||||
// The spawner is now closed (and never spawned any children) — the
|
||||
// waiter must unblock.
|
||||
if !readyIDs(t, bd, dir)[waiterID] {
|
||||
t.Fatalf("waiter %s did not become ready after its childless spawner %s closed", waiterID, spawnerID)
|
||||
}
|
||||
}
|
||||
|
|
@ -633,6 +633,12 @@ func cloneSubgraph(ctx context.Context, s storage.DoltStorage, subgraph *Templat
|
|||
IssueID: newFromID,
|
||||
DependsOnID: newToID,
|
||||
Type: dep.Type,
|
||||
// Carry template metadata through to the clone (GH#3875):
|
||||
// waits-for edges store their gate (and, for a collapsed
|
||||
// needs/depends_on edge, also_blocks) here. Dropping it
|
||||
// silently reset every poured waits-for edge to the
|
||||
// default all-children gate with also_blocks unset.
|
||||
Metadata: dep.Metadata,
|
||||
}
|
||||
if err := tx.AddDependency(ctx, newDep, opts.Actor); err != nil {
|
||||
return fmt.Errorf("failed to create dependency: %w", err)
|
||||
|
|
|
|||
|
|
@ -328,6 +328,106 @@ func TestIsBlocked_WaitsForAnyChildrenGate(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestIsBlocked_WaitsForAlsoBlocksSpawnerOpen covers the GH#3783/#3875
|
||||
// also_blocks provenance flag: a waits-for edge collapsed from a redundant
|
||||
// needs/depends_on blocks edge (cmd/bd/cook.go collectDependencies) must
|
||||
// additionally block while the spawner itself is open, not only while it
|
||||
// has an open parent-child child — closing the pre-fanout window where the
|
||||
// waiter could become ready before the spawner ever produced (or finished)
|
||||
// its fanout.
|
||||
func TestIsBlocked_WaitsForAlsoBlocksSpawnerOpen(t *testing.T) {
|
||||
store, cleanup := setupTestStore(t)
|
||||
defer cleanup()
|
||||
ctx, cancel := testContext(t)
|
||||
defer cancel()
|
||||
|
||||
createPerm(t, ctx, store, "isb-wf-ab-waiter")
|
||||
createPerm(t, ctx, store, "isb-wf-ab-spawner")
|
||||
|
||||
if err := store.AddDependency(ctx, &types.Dependency{
|
||||
IssueID: "isb-wf-ab-waiter", DependsOnID: "isb-wf-ab-spawner",
|
||||
Type: types.DepWaitsFor, Metadata: `{"gate":"all-children","also_blocks":true}`,
|
||||
}, "tester"); err != nil {
|
||||
t.Fatalf("waits-for also_blocks: %v", err)
|
||||
}
|
||||
|
||||
// Spawner open, zero children spawned yet: the waiter must be blocked.
|
||||
// Without also_blocks this would be unblocked (no children => no
|
||||
// children-open EXISTS) — exactly the pre-fanout regression.
|
||||
if !getIsBlocked(t, ctx, store, "issues", "isb-wf-ab-waiter") {
|
||||
t.Fatal("expected waiter blocked: spawner still open with zero children spawned (also_blocks)")
|
||||
}
|
||||
|
||||
if err := store.CloseIssue(ctx, "isb-wf-ab-spawner", "no fanout needed", "tester", ""); err != nil {
|
||||
t.Fatalf("CloseIssue spawner: %v", err)
|
||||
}
|
||||
if getIsBlocked(t, ctx, store, "issues", "isb-wf-ab-waiter") {
|
||||
t.Fatal("expected waiter unblocked after its childless spawner closed")
|
||||
}
|
||||
|
||||
if err := store.ReopenIssue(ctx, "isb-wf-ab-spawner", "", "tester"); err != nil {
|
||||
t.Fatalf("ReopenIssue spawner: %v", err)
|
||||
}
|
||||
if !getIsBlocked(t, ctx, store, "issues", "isb-wf-ab-waiter") {
|
||||
t.Fatal("expected waiter re-blocked after spawner reopened")
|
||||
}
|
||||
}
|
||||
|
||||
// TestIsBlocked_WaitsForAlsoBlocksOverridesAnyChildrenEarlyOpen covers the
|
||||
// override case explicitly called out in the design: also_blocks is a
|
||||
// top-level OR outside the any-children early-open carve-out, so a
|
||||
// collapsed edge on an any-children gate must stay blocked while the
|
||||
// spawner itself is open even after one child closes (the normal
|
||||
// any-children early-open behavior is preserved only for edges WITHOUT
|
||||
// also_blocks — see TestIsBlocked_WaitsForAnyChildrenGate above).
|
||||
func TestIsBlocked_WaitsForAlsoBlocksOverridesAnyChildrenEarlyOpen(t *testing.T) {
|
||||
store, cleanup := setupTestStore(t)
|
||||
defer cleanup()
|
||||
ctx, cancel := testContext(t)
|
||||
defer cancel()
|
||||
|
||||
createPerm(t, ctx, store, "isb-wf-ab-any-waiter")
|
||||
createPerm(t, ctx, store, "isb-wf-ab-any-spawner")
|
||||
createPerm(t, ctx, store, "isb-wf-ab-any-child-1")
|
||||
createPerm(t, ctx, store, "isb-wf-ab-any-child-2")
|
||||
|
||||
for _, child := range []string{"isb-wf-ab-any-child-1", "isb-wf-ab-any-child-2"} {
|
||||
if err := store.AddDependency(ctx, &types.Dependency{
|
||||
IssueID: child, DependsOnID: "isb-wf-ab-any-spawner", Type: types.DepParentChild,
|
||||
}, "tester"); err != nil {
|
||||
t.Fatalf("parent-child %s: %v", child, err)
|
||||
}
|
||||
}
|
||||
if err := store.AddDependency(ctx, &types.Dependency{
|
||||
IssueID: "isb-wf-ab-any-waiter", DependsOnID: "isb-wf-ab-any-spawner",
|
||||
Type: types.DepWaitsFor, Metadata: `{"gate":"any-children","also_blocks":true}`,
|
||||
}, "tester"); err != nil {
|
||||
t.Fatalf("waits-for any-children also_blocks: %v", err)
|
||||
}
|
||||
if !getIsBlocked(t, ctx, store, "issues", "isb-wf-ab-any-waiter") {
|
||||
t.Fatal("expected waiter blocked: no children closed yet")
|
||||
}
|
||||
|
||||
// Close one child: under a plain any-children gate this alone would
|
||||
// unblock (see TestIsBlocked_WaitsForAnyChildrenGate). With also_blocks
|
||||
// the waiter must STAY blocked because the spawner itself is still open.
|
||||
if err := store.CloseIssue(ctx, "isb-wf-ab-any-child-1", "done", "tester", ""); err != nil {
|
||||
t.Fatalf("CloseIssue first child: %v", err)
|
||||
}
|
||||
if !getIsBlocked(t, ctx, store, "issues", "isb-wf-ab-any-waiter") {
|
||||
t.Fatal("expected waiter STILL blocked: also_blocks overrides the any-children early-open carve-out while the spawner is open")
|
||||
}
|
||||
|
||||
// Now close the spawner too: the gate is satisfied (a child closed under
|
||||
// any-children) AND the spawner itself is closed, so the waiter unblocks.
|
||||
if err := store.CloseIssue(ctx, "isb-wf-ab-any-spawner", "done", "tester", ""); err != nil {
|
||||
t.Fatalf("CloseIssue spawner: %v", err)
|
||||
}
|
||||
if getIsBlocked(t, ctx, store, "issues", "isb-wf-ab-any-waiter") {
|
||||
t.Fatal("expected waiter unblocked once the spawner closes with the any-children gate already satisfied")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsBlocked_AddClosedChildUnblocksAnyChildrenGate(t *testing.T) {
|
||||
store, cleanup := setupTestStore(t)
|
||||
defer cleanup()
|
||||
|
|
|
|||
|
|
@ -21,40 +21,73 @@ type DBTX interface {
|
|||
|
||||
const waitsForGateBlockedSQL = `
|
||||
(
|
||||
EXISTS (
|
||||
SELECT 1 FROM dependencies cd JOIN issues child ON child.id = cd.issue_id
|
||||
WHERE cd.type = 'parent-child'
|
||||
AND ((d.depends_on_issue_id IS NOT NULL AND cd.depends_on_issue_id = d.depends_on_issue_id)
|
||||
OR (d.depends_on_wisp_id IS NOT NULL AND cd.depends_on_wisp_id = d.depends_on_wisp_id))
|
||||
AND child.status <> 'closed' AND child.status <> 'pinned'
|
||||
)
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM wisp_dependencies cd JOIN wisps child ON child.id = cd.issue_id
|
||||
WHERE cd.type = 'parent-child'
|
||||
AND ((d.depends_on_issue_id IS NOT NULL AND cd.depends_on_issue_id = d.depends_on_issue_id)
|
||||
OR (d.depends_on_wisp_id IS NOT NULL AND cd.depends_on_wisp_id = d.depends_on_wisp_id))
|
||||
AND child.status <> 'closed' AND child.status <> 'pinned'
|
||||
)
|
||||
)
|
||||
AND NOT (
|
||||
-- COALESCE: metadata without a gate key (legacy '{}' rows) means the
|
||||
-- all-children default; a NULL here would poison the AND/NOT chain
|
||||
-- and unblock the gate as soon as any child closes.
|
||||
COALESCE(JSON_UNQUOTE(JSON_EXTRACT(d.metadata, '$.gate')), 'all-children') = 'any-children'
|
||||
AND (
|
||||
(
|
||||
EXISTS (
|
||||
SELECT 1 FROM dependencies cd JOIN issues child ON child.id = cd.issue_id
|
||||
WHERE cd.type = 'parent-child'
|
||||
AND ((d.depends_on_issue_id IS NOT NULL AND cd.depends_on_issue_id = d.depends_on_issue_id)
|
||||
OR (d.depends_on_wisp_id IS NOT NULL AND cd.depends_on_wisp_id = d.depends_on_wisp_id))
|
||||
AND child.status = 'closed'
|
||||
AND child.status <> 'closed' AND child.status <> 'pinned'
|
||||
)
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM wisp_dependencies cd JOIN wisps child ON child.id = cd.issue_id
|
||||
WHERE cd.type = 'parent-child'
|
||||
AND ((d.depends_on_issue_id IS NOT NULL AND cd.depends_on_issue_id = d.depends_on_issue_id)
|
||||
OR (d.depends_on_wisp_id IS NOT NULL AND cd.depends_on_wisp_id = d.depends_on_wisp_id))
|
||||
AND child.status = 'closed'
|
||||
AND child.status <> 'closed' AND child.status <> 'pinned'
|
||||
)
|
||||
)
|
||||
AND NOT (
|
||||
-- COALESCE: metadata without a gate key (legacy '{}' rows) means the
|
||||
-- all-children default; a NULL here would poison the AND/NOT chain
|
||||
-- and unblock the gate as soon as any child closes.
|
||||
COALESCE(JSON_UNQUOTE(JSON_EXTRACT(d.metadata, '$.gate')), 'all-children') = 'any-children'
|
||||
AND (
|
||||
EXISTS (
|
||||
SELECT 1 FROM dependencies cd JOIN issues child ON child.id = cd.issue_id
|
||||
WHERE cd.type = 'parent-child'
|
||||
AND ((d.depends_on_issue_id IS NOT NULL AND cd.depends_on_issue_id = d.depends_on_issue_id)
|
||||
OR (d.depends_on_wisp_id IS NOT NULL AND cd.depends_on_wisp_id = d.depends_on_wisp_id))
|
||||
AND child.status = 'closed'
|
||||
)
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM wisp_dependencies cd JOIN wisps child ON child.id = cd.issue_id
|
||||
WHERE cd.type = 'parent-child'
|
||||
AND ((d.depends_on_issue_id IS NOT NULL AND cd.depends_on_issue_id = d.depends_on_issue_id)
|
||||
OR (d.depends_on_wisp_id IS NOT NULL AND cd.depends_on_wisp_id = d.depends_on_wisp_id))
|
||||
AND child.status = 'closed'
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
OR (
|
||||
-- also_blocks (GH#3783/GH#3875): a waits-for edge collapsed from a
|
||||
-- redundant needs/depends_on blocks edge onto this same spawner
|
||||
-- (cmd/bd/cook.go collectDependencies) additionally carries classic
|
||||
-- blocking semantics — it must block while the spawner itself is
|
||||
-- open, not only while the spawner has an open parent-child child.
|
||||
-- This closes the pre-fanout window where the waiter could become
|
||||
-- ready before the spawner (and its fanout) ever completed. Legacy
|
||||
-- rows and plain (non-collapsed) waits-for edges lack the
|
||||
-- also_blocks key, so COALESCE defaults to 'false' and this branch
|
||||
-- is a no-op for them (zero behavior change).
|
||||
--
|
||||
-- This is a top-level OR, deliberately outside (and overriding) the
|
||||
-- any-children early-open carve-out above: a collapsed edge means
|
||||
-- the caller's needs/depends_on required the spawner itself to
|
||||
-- close, so an early-open child close must NOT unblock the waiter
|
||||
-- while the spawner remains open.
|
||||
COALESCE(JSON_UNQUOTE(JSON_EXTRACT(d.metadata, '$.also_blocks')), 'false') = 'true'
|
||||
AND (
|
||||
EXISTS (
|
||||
SELECT 1 FROM issues sp
|
||||
WHERE sp.id = d.depends_on_issue_id
|
||||
AND sp.status <> 'closed' AND sp.status <> 'pinned'
|
||||
)
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM wisps sp
|
||||
WHERE sp.id = d.depends_on_wisp_id
|
||||
AND sp.status <> 'closed' AND sp.status <> 'pinned'
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
@ -396,6 +429,14 @@ func AffectedByStatusChangeInTx(ctx context.Context, tx DBTX, id string) ([]stri
|
|||
if err := loadWaitersWhoseSpawnerIsParentOfInTx(ctx, tx, id, false, &issueSeed, issueSeen, &wispSeed, wispSeen); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
// id's own status just changed, and an also_blocks waits-for edge blocks
|
||||
// while the spawner itself is open (pre-fanout window, GH#3783/GH#3875).
|
||||
// A waiter with only a DepWaitsFor edge on id (no DepBlocks edge — the
|
||||
// blocking semantics were collapsed into also_blocks) would otherwise
|
||||
// never get recomputed when its spawner closes.
|
||||
if err := loadWaitersOnSpawnerIDsInTx(ctx, tx, []string{id}, &issueSeed, issueSeen, &wispSeed, wispSeen); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return expandByParentChildDescendantsInTx(ctx, tx, issueSeed, wispSeed, issueSeen, wispSeen)
|
||||
}
|
||||
|
||||
|
|
@ -411,6 +452,12 @@ func AffectedByStatusChangeForWispInTx(ctx context.Context, tx DBTX, id string)
|
|||
if err := loadWaitersWhoseSpawnerIsParentOfInTx(ctx, tx, id, true, &issueSeed, issueSeen, &wispSeed, wispSeen); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
// See the issue-id sibling above: id's own status just changed, and a
|
||||
// waiter that waits directly on this wisp id as spawner needs to be
|
||||
// recomputed too.
|
||||
if err := loadWaitersOnSpawnerIDsInTx(ctx, tx, []string{id}, &issueSeed, issueSeen, &wispSeed, wispSeen); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return expandByParentChildDescendantsInTx(ctx, tx, issueSeed, wispSeed, issueSeen, wispSeen)
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -984,6 +984,16 @@ type WaitsForMeta struct {
|
|||
// SpawnerID identifies which step/issue spawns the children to wait for.
|
||||
// If empty, waits for all direct children of the depends_on_id issue.
|
||||
SpawnerID string `json:"spawner_id,omitempty"`
|
||||
// AlsoBlocks marks a waits-for edge that was collapsed from a redundant
|
||||
// depends_on/needs blocks edge onto the same spawner (GH#3783): the
|
||||
// caller skipped emitting a separate DepBlocks edge because it collided
|
||||
// with this DepWaitsFor edge, so this edge must additionally carry
|
||||
// classic blocking semantics — it blocks while the spawner itself is
|
||||
// open, not only while the spawner has an open child. Omitted (and thus
|
||||
// COALESCEd to false by readers) for a plain waits_for with no matching
|
||||
// needs/depends_on entry, which must retain the original fanout-only
|
||||
// semantics.
|
||||
AlsoBlocks bool `json:"also_blocks,omitempty"`
|
||||
}
|
||||
|
||||
// WaitsForGate constants
|
||||
|
|
@ -1043,6 +1053,32 @@ func NewWaitsForDependency(issueID, spawnerID, gate string) (*Dependency, error)
|
|||
return NewGraphEdgeDependency(issueID, spawnerID, DepWaitsFor, gate, "", "", "", nil)
|
||||
}
|
||||
|
||||
// NewWaitsForBlockingDependency builds a waits-for dependency that also
|
||||
// carries classic blocking semantics (GH#3783): set also_blocks in the
|
||||
// metadata so waitsForGateBlockedSQL additionally blocks while the spawner
|
||||
// itself is open, not only while it has an open parent-child child. Use this
|
||||
// instead of NewWaitsForDependency exactly when the caller is collapsing a
|
||||
// would-be DepBlocks edge (from needs/depends_on) into this waits-for edge
|
||||
// because the two would otherwise collide on the same (source, target) pair
|
||||
// — never for a plain waits_for with no matching needs/depends_on entry.
|
||||
func NewWaitsForBlockingDependency(issueID, spawnerID, gate string) (*Dependency, error) {
|
||||
dep, err := NewWaitsForDependency(issueID, spawnerID, gate)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var meta WaitsForMeta
|
||||
if err := json.Unmarshal([]byte(dep.Metadata), &meta); err != nil {
|
||||
return nil, fmt.Errorf("parsing waits-for metadata to set also_blocks: %w", err)
|
||||
}
|
||||
meta.AlsoBlocks = true
|
||||
raw, err := json.Marshal(meta)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("serializing waits-for also_blocks metadata: %w", err)
|
||||
}
|
||||
dep.Metadata = string(raw)
|
||||
return dep, nil
|
||||
}
|
||||
|
||||
// NewGraphNodeDependency builds the dependency record for a graph-plan node's
|
||||
// inline dep, shared by the embedded and domain apply paths so their
|
||||
// resolution semantics cannot drift: an empty type defaults to blocks, and
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue