* test(storage): measure ApplyBatchInTx SQL statement counts (B0) Adds a driver.Connector-level counting/latency-injecting wrapper (internal/storage/sqlcount) plus shared plan-request fixtures (internal/storage/batchfixtures: mol-normalized 356/712-item shapes and a 40-item classic shape) and backend-specific pinning tests + benchmarks (internal/storage/embeddeddolt, internal/storage/dolt) that measure the ACTUAL number of SQL statements issueops.ApplyBatchInTx issues for each shape, on both the embedded and Dolt (TCP) backends. Measured totals (Prepare+Exec+Query+StmtExec+StmtQuery inside one transaction): 356 items ~7008-7009, 712 items 14014-14015, 40 items 846, identical on both backends (expected: statement issuance is driven by Go application logic, not backend-specific SQL planning). Pinned with a small (+/-2) tolerance to absorb an observed, host-contention-driven +/-1 Query drift (Exec is deterministic on every run). This is the regression baseline slice B2 (a later, lighter fast path) must demonstrably lower. batchfixtures.UncappedPlan bypasses issueops.MaxApplyBatchItems so this measurement does not depend on slice B1's cap raise. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(httpapi,issueops): raise batchApply envelope to 1000 items with bounded large-apply handling POST /v0/beads/issues:batchApply now accepts up to 1000 items (up from 100); issueops.MaxApplyBatchItems moves accordingly and the request body cap rises to 16 MiB. The whole plan still applies in exactly ONE transaction. A request over 100 items runs under a separate, flat, operator-configured whole-run budget (`bd serve --large-apply-ceiling`, 5 minutes by default) instead of the ordinary 60s request deadline; a request at or under 100 items is unaffected and keeps exactly the 60s deadline and 15s commit-retry ceiling it always had. Concurrency and shutdown are bounded, not just documented: - Oversized requests serialize through a dedicated one-wide semaphore, and at most one caller may wait for that semaphore at a time (a CAS-guarded single waiter) — a burst of oversized requests is shed with 503 rather than queuing and starving ordinary traffic behind the general semaphore. - The run deadline is stamped only once the slot is actually acquired, not at the time the request arrived. - Graceful shutdown closes new admissions before computing its drain budget, then waits out an in-flight large apply up to its own remaining deadline (plus a few seconds' grace for the response to write); any large apply still queued for the semaphore when shutdown begins is refused rather than extending the drain further. A queued caller that disconnects gets a clean 503, never a misclassified context error. `bd serve`'s orchestrator stop grace must be at least `--large-apply-ceiling` — see engdocs/SERVE_RUNBOOK.md. - ApplyBatch's commit-retry budget only extends past the ordinary 15s ceiling on this explicit large-apply path (via a context marker), with headroom reserved so the final retry attempt can't be killed mid-commit by the context's own expiry. A new `issues.batchApplyLarge` capability token on `GET /v0/beads/context` lets a client learn whether a given server accepts the raised envelope. Wall-clock measurement (embedded Dolt, whole-transaction build+commit) backs the 5-minute ceiling default: 356 items ~12.6s (23.8x headroom), 1000 items ~57.5s (5.2x headroom). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(httpapi,serve): wire the large-apply retry budget; repair stale pins Review fixes for the large batchApply envelope, on top of the test repairs the PR needed against main. The extended commit-retry budget was never switched on. internal/storage/uow's retryTxBudget lets a large apply keep retrying a commit-time conflict past the ordinary 15s ceiling only when its context carries issueops.WithExtendedRetryBudget's marker, and the docs on both sides say admitLargeApply sets it, but nothing did: the marker's only callers were the uow tests, which mark their contexts by hand. A 1000-item apply that hit a conflict still gave up after 15s with most of its 5-minute ceiling unused. admitLargeApply now marks the run context it builds, and TestApplyBatchMarksOnlyALargeApplyForTheExtendedRetryBudget pins the seam over HTTP: marked at 101 and 1000 items, unmarked at 100. Removing the mark fails the 101 and 1000 cases; marking every request in handleApplyBatch fails the 100 case. Smaller fixes from the same review: - `bd serve --large-apply-ceiling` repeated the 5m default as a literal of its own, and since bd serve always passes a positive ceiling, that literal, not httpapi's, was the default every deployment ran. Export httpapi.DefaultLargeApplyCeiling, default the flag to it, and pin the flag's 5m0s default in TestServeFlags. - admitLargeApply's release cancels the run context before clearing s.largeApplyDeadline, the order acquireLargeApply's doc states; it cleared the deadline first. - Drop the batchfixtures test's dead t.Skipf, which waited for the cap to reach 1000, and correct a graph-counter comment that still cited the 100-item cap and the TestServeFlags doc line that called the no-argument server unchanged. The PR raises issueops.MaxApplyBatchItems from 100 to 1000 and adds `bd serve --large-apply-ceiling`, but two base-owned tests still pinned the old surface and went red on every lane that runs them (Bazel / test, Test (macos-latest)): - TestServeFlags pins the exact `bd serve` flag list; add large-apply-ceiling. - TestPlanApplyBatchRefusesAnUnusableRequest/too_many_items built 101 items and expected "at most 100 items", a request the 1000 cap now plans successfully. Size the over-cap request and its message from issueops.MaxApplyBatchItems, as TestPlanApplyBatchAcceptsTheBoundary already does for the other side of the cap; the cap's value stays pinned by TestCapBatchApplyLargeTiesAllThreeLimits. The PR's own TestLargeApplyBurstNeverStarvesSmallRequestsForSemTimeout also went red in PR Core (-race, -p 4 on a 4-vCPU runner): all eight small requests took 160-165ms against a 100ms bound. The stall it guards against lasts semTimeout (5s), while a small request can still queue briefly behind the refused large requests, each holding a general sem slot while its body decodes. Bound it at a second, the "want well under a second" its sibling TestLargeApplyQueueDoesNotStarveOrdinaryRequests already uses. Pinned to four cores beside 24 busy loops, small requests measured 183ms mean and 338ms max, and the 1s bound passed 80/80 runs; with the one-waiter gate in acquireLargeApply disabled the test still fails 3/3, on its 2s never-completed check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(uow): stop a doomed large-apply retry before ctx cuts it off retryTxBudget sizes maxElapsed once at ApplyBatch entry, but says nothing about how long an individual attempt takes. A slow first attempt (e.g. 157s of a large apply's commit) left little of the budget for a retry, which then got killed mid-transaction by ctx's own deadline instead of finishing and reporting its own outcome -- surfacing as context.DeadlineExceeded (isUnavailable excludes it -> a generic 500) instead of the exhausted-write-conflict result this loop already reports correctly (-> 503 + Retry-After). RunTxResultWithin now tracks the previous attempt's duration and, before starting a new one, stops early with the last real serialization error when time left on ctx is less than that duration plus a fixed headroom -- but only when publicops.HasExtendedRetryBudget(ctx) is set (the same marker retryTxBudget itself checks). Applying the look-ahead unconditionally regressed TestApplyBatchHonorsAnExtendedRetryBudgetFromContext's unmarked case: ordinary requests have cheap attempts and are meant to run until their own ctx deadline ends the loop, and the large-apply-sized 5s headroom was tripping that for any short-ctx caller with a near-instant failing attempt. Adds TestRunTxResultWithin_SlowAttemptStopsBeforeADoomedRetry (a 1.2s commit against a 2s marked ctx must stop after one attempt with its own serialization error, never DeadlineExceeded, and never touch a second unit of work) and its control, TestRunTxResultWithin_ShortAttemptStillRetries (a 50ms attempt against the same shape must still retry and succeed). Both mutation-checked against the look-ahead condition and its gate. Also updates the stale "items accepts at most 100 entries" prose on the applyBatch endpoint description (openapi.v0.yaml) to the large-envelope wording already used elsewhere in the spec: 1000-item cap behind the issues.batchApplyLarge capability, large-apply-ceiling deadline extension, <=100 unaffected. dependencies:count's own "at most 100" lines are unrelated and untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(ci,serve): fit the 1000-item cap's tests in the storage shard budget Review fixes for the relaunched #7051, iteration 1. Raising issueops.MaxApplyBatchItems from 100 to 1000 made the base-owned TestBatchApplyContract take 604s under -race, against 97s at the old cap (main run 35862663577). It hashed onto the legacy test-embedded-storage shard 2/5, beside ~5m of other tests, and that shard hit its 15m budget twice (PR Risk run 36807191489). PR Risk no longer runs the legacy tier on same-repo PRs (#7071), but Main's push run and fork PRs still do, so the merge would have carried the timeout to Main. Raising the jobs' timeout-minutes alone would not fix it: the shard script passes the binary its own -test.timeout=15m. So: - embedded-storage-test-shard.sh passes -test.timeout=20m (and its go test fallback -timeout 20m), as embedded-test-shard.sh does for cmd/bd. - test-embedded-storage in main.yml and pr-risk.yml: timeout-minutes 15 -> 25, the script's 20m plus setup. - embeddeddolt_embedded_test appends -test.timeout=19m, as bd_embedded_test does, so Go's timeout, which dumps every goroutine, fires before rbe-west's 1200s action kill, which does not. TestBazelEmbeddedJobMirrorsEmbeddedTier now checks the override on both sharded targets instead of only cmd/bd's, and the PR Risk coverage test pins the storage target's new args. - embedded-storage-test-shards.txt places TestBatchApplyContract on shard 4, which has ~3m of other tests, and the PR's two large-batch tests, TestLargeBatchApplyStatementCounts_Embedded (247s) and TestLargeBatchApplyWallClock_Embedded (261s), on shards 2 and 5. From the PR run's per-test times, the heaviest shard projects to ~13m, down from shard 2's 15m-plus. Also correct the stop-grace guidance. For an in-flight large apply, Serve's drain waits out the apply's remaining deadline plus drainGrace (5s), so the orchestrator's stop grace must be at least --large-apply-ceiling plus 5s, as Serve's own doc says. The runbook, the flag help, the LargeApplyCeiling field doc and the CHANGELOG entry asked for the ceiling alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
568 lines
20 KiB
Go
568 lines
20 KiB
Go
package storage
|
|
|
|
import (
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"reflect"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/steveyegge/beads/internal/types"
|
|
"github.com/steveyegge/beads/issueops"
|
|
)
|
|
|
|
// These pin the DATABASE-FREE half of issueops.BatchApplier: what a request
|
|
// means, which refs it can resolve, which guards it may carry, and how a
|
|
// waits-for gate is spelled. Every one of them is a promise the leaf makes and
|
|
// a conformance case would otherwise need three live backends to observe — so
|
|
// they run here, in milliseconds, and the contract is left to assert what only
|
|
// a real backend can show.
|
|
|
|
// planApplyItem builds one item of the given kind with a minimal valid payload.
|
|
func planApplyCreate(key string) issueops.ApplyItem {
|
|
return issueops.ApplyItem{
|
|
Kind: issueops.ItemCreate,
|
|
Create: &issueops.CreateItem{Key: key, Issue: &issueops.Issue{Title: "t"}},
|
|
}
|
|
}
|
|
|
|
func planApplyUpdate(target issueops.Ref, expected *int64) issueops.ApplyItem {
|
|
return issueops.ApplyItem{
|
|
Kind: issueops.ItemUpdate,
|
|
Update: &issueops.UpdateItem{Target: target, ExpectedVersion: expected},
|
|
}
|
|
}
|
|
|
|
func planApplyClose(target issueops.Ref, expected *int64) issueops.ApplyItem {
|
|
return issueops.ApplyItem{
|
|
Kind: issueops.ItemClose,
|
|
Close: &issueops.CloseItem{Target: target, ExpectedVersion: expected},
|
|
}
|
|
}
|
|
|
|
func planApplyDep(source, target issueops.Ref, depType types.DependencyType, metadata string) issueops.ApplyItem {
|
|
return issueops.ApplyItem{
|
|
Kind: issueops.ItemDepAdd,
|
|
DepAdd: &issueops.DepAddItem{Source: source, Target: target, Type: depType, Metadata: metadata},
|
|
}
|
|
}
|
|
|
|
func planApplyRequest(items ...issueops.ApplyItem) issueops.ApplyBatchRequest {
|
|
return issueops.ApplyBatchRequest{Actor: "planner", Items: items}
|
|
}
|
|
|
|
func int64Ptr(v int64) *int64 { return &v }
|
|
|
|
// TestPlanApplyBatchRefusesAnUnusableRequest pins the request-level shape rules:
|
|
// every one of them is knowable from the request alone, so a request refused
|
|
// here has provably touched no database.
|
|
func TestPlanApplyBatchRefusesAnUnusableRequest(t *testing.T) {
|
|
overCap := make([]issueops.ApplyItem, issueops.MaxApplyBatchItems+1)
|
|
for i := range overCap {
|
|
overCap[i] = planApplyCreate("")
|
|
}
|
|
|
|
for _, test := range []struct {
|
|
name string
|
|
request issueops.ApplyBatchRequest
|
|
want string
|
|
}{
|
|
{"no actor", issueops.ApplyBatchRequest{Items: []issueops.ApplyItem{planApplyCreate("")}}, "requires an actor"},
|
|
{"no items", issueops.ApplyBatchRequest{Actor: "planner"}, "at least one item"},
|
|
{"too many items", issueops.ApplyBatchRequest{Actor: "planner", Items: overCap}, fmt.Sprintf("at most %d items", issueops.MaxApplyBatchItems)},
|
|
{"no payload", planApplyRequest(issueops.ApplyItem{Kind: issueops.ItemCreate}), "exactly one payload"},
|
|
{
|
|
"two payloads",
|
|
planApplyRequest(issueops.ApplyItem{
|
|
Kind: issueops.ItemCreate,
|
|
Create: &issueops.CreateItem{Issue: &issueops.Issue{}},
|
|
Close: &issueops.CloseItem{Target: issueops.Ref{ID: "bd-1"}},
|
|
}),
|
|
"exactly one payload",
|
|
},
|
|
{
|
|
"kind and payload disagree",
|
|
planApplyRequest(issueops.ApplyItem{
|
|
Kind: issueops.ItemCreate,
|
|
Close: &issueops.CloseItem{Target: issueops.Ref{ID: "bd-1"}},
|
|
}),
|
|
"carries another kind's payload",
|
|
},
|
|
{
|
|
"unknown kind",
|
|
planApplyRequest(issueops.ApplyItem{
|
|
Kind: "reopen",
|
|
Create: &issueops.CreateItem{Issue: &issueops.Issue{}},
|
|
}),
|
|
`unknown kind "reopen"`,
|
|
},
|
|
{
|
|
"create with no issue",
|
|
planApplyRequest(issueops.ApplyItem{Kind: issueops.ItemCreate, Create: &issueops.CreateItem{}}),
|
|
"requires an issue",
|
|
},
|
|
{
|
|
"create carrying inline dependencies",
|
|
planApplyRequest(issueops.ApplyItem{Kind: issueops.ItemCreate, Create: &issueops.CreateItem{
|
|
Issue: &issueops.Issue{Dependencies: []*types.Dependency{{IssueID: "a", DependsOnID: "b"}}},
|
|
}}),
|
|
"edges are their own items",
|
|
},
|
|
{
|
|
"duplicate create key",
|
|
planApplyRequest(planApplyCreate("root"), planApplyCreate("root")),
|
|
`reuses key "root"`,
|
|
},
|
|
{
|
|
"ref names neither member",
|
|
planApplyRequest(planApplyUpdate(issueops.Ref{}, nil)),
|
|
"must name a key or an id",
|
|
},
|
|
{
|
|
"ref names both members",
|
|
planApplyRequest(planApplyUpdate(issueops.Ref{Key: "k", ID: "bd-1"}, nil)),
|
|
"exactly one",
|
|
},
|
|
{
|
|
"edge with no type",
|
|
planApplyRequest(planApplyDep(issueops.Ref{ID: "bd-1"}, issueops.Ref{ID: "bd-2"}, "", "")),
|
|
"requires a dependency type",
|
|
},
|
|
{
|
|
"edge metadata that is not JSON",
|
|
planApplyRequest(planApplyDep(issueops.Ref{ID: "bd-1"}, issueops.Ref{ID: "bd-2"}, types.DepBlocks, "not json")),
|
|
"not well-formed JSON",
|
|
},
|
|
{
|
|
"metadata_ref with an empty key",
|
|
planApplyRequest(issueops.ApplyItem{Kind: issueops.ItemCreate, Create: &issueops.CreateItem{
|
|
Issue: &issueops.Issue{},
|
|
MetadataRefs: map[string]issueops.Ref{"": {ID: "bd-1"}},
|
|
}}),
|
|
"empty key",
|
|
},
|
|
} {
|
|
t.Run(test.name, func(t *testing.T) {
|
|
_, err := PlanApplyBatch(test.request)
|
|
if err == nil {
|
|
t.Fatalf("PlanApplyBatch accepted %s", test.name)
|
|
}
|
|
if !errors.Is(err, issueops.ErrValidation) {
|
|
t.Fatalf("error = %v, want it to match ErrValidation so every front door classifies it", err)
|
|
}
|
|
if !strings.Contains(err.Error(), test.want) {
|
|
t.Fatalf("error = %q, want it to name %q", err, test.want)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchAcceptsTheBoundary pins the OTHER side of the item cap.
|
|
// Without it the cap check could be off by one in the refusing direction and
|
|
// nothing above would notice.
|
|
func TestPlanApplyBatchAcceptsTheBoundary(t *testing.T) {
|
|
items := make([]issueops.ApplyItem, issueops.MaxApplyBatchItems)
|
|
for i := range items {
|
|
items[i] = planApplyCreate("")
|
|
}
|
|
if _, err := PlanApplyBatch(planApplyRequest(items...)); err != nil {
|
|
t.Fatalf("PlanApplyBatch(%d items) = %v, want the cap to be inclusive", issueops.MaxApplyBatchItems, err)
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchSelfDependencyIsItsOwnRefusal pins that an edge from a row
|
|
// to itself is ErrSelfDependency rather than a cycle, the way every other
|
|
// dependency path types it — a scheduling self-edge would otherwise trip the
|
|
// cycle probe and report the wrong refusal.
|
|
func TestPlanApplyBatchSelfDependencyIsItsOwnRefusal(t *testing.T) {
|
|
for _, test := range []struct {
|
|
name string
|
|
ref issueops.Ref
|
|
}{
|
|
{"by id", issueops.Ref{ID: "bd-1"}},
|
|
{"by key", issueops.Ref{Key: "root"}},
|
|
} {
|
|
t.Run(test.name, func(t *testing.T) {
|
|
_, err := PlanApplyBatch(planApplyRequest(
|
|
planApplyCreate("root"),
|
|
planApplyDep(test.ref, test.ref, types.DepBlocks, ""),
|
|
))
|
|
if !errors.Is(err, issueops.ErrSelfDependency) {
|
|
t.Fatalf("error = %v, want ErrSelfDependency", err)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchResolvesKeysBackwardOnly pins the ref rule the whole item
|
|
// vocabulary rests on, and — the part that earns the type — that the TWO
|
|
// failures are told apart. A key declared later is an ORDERING mistake a caller
|
|
// fixes by moving an item; a key nothing declares is a typo. Collapsing them
|
|
// would send a caller hunting for the wrong thing.
|
|
func TestPlanApplyBatchResolvesKeysBackwardOnly(t *testing.T) {
|
|
t.Run("backward is accepted", func(t *testing.T) {
|
|
plan, err := PlanApplyBatch(planApplyRequest(
|
|
planApplyCreate("root"),
|
|
planApplyUpdate(issueops.Ref{Key: "root"}, nil),
|
|
))
|
|
if err != nil {
|
|
t.Fatalf("PlanApplyBatch = %v, want a backward key to resolve", err)
|
|
}
|
|
if got := plan.KeyIndex["root"]; got != 0 {
|
|
t.Fatalf("KeyIndex[root] = %d, want 0", got)
|
|
}
|
|
})
|
|
|
|
for _, test := range []struct {
|
|
name string
|
|
items []issueops.ApplyItem
|
|
member string
|
|
index int
|
|
declaredLater bool
|
|
}{
|
|
{
|
|
"forward target",
|
|
[]issueops.ApplyItem{planApplyUpdate(issueops.Ref{Key: "root"}, nil), planApplyCreate("root")},
|
|
"target", 0, true,
|
|
},
|
|
{
|
|
"a target naming its own item's key",
|
|
[]issueops.ApplyItem{
|
|
planApplyCreate("root"),
|
|
planApplyDep(issueops.Ref{Key: "later"}, issueops.Ref{ID: "bd-2"}, types.DepBlocks, ""),
|
|
planApplyCreate("later"),
|
|
},
|
|
"source", 1, true,
|
|
},
|
|
{
|
|
"unknown target",
|
|
[]issueops.ApplyItem{planApplyClose(issueops.Ref{Key: "ghost"}, nil)},
|
|
"target", 0, false,
|
|
},
|
|
{
|
|
"forward edge source",
|
|
[]issueops.ApplyItem{
|
|
planApplyDep(issueops.Ref{Key: "root"}, issueops.Ref{ID: "bd-2"}, types.DepBlocks, ""),
|
|
planApplyCreate("root"),
|
|
},
|
|
"source", 0, true,
|
|
},
|
|
{
|
|
"unknown edge target",
|
|
[]issueops.ApplyItem{
|
|
planApplyCreate("root"),
|
|
planApplyDep(issueops.Ref{Key: "root"}, issueops.Ref{Key: "ghost"}, types.DepBlocks, ""),
|
|
},
|
|
"target", 1, false,
|
|
},
|
|
} {
|
|
t.Run(test.name, func(t *testing.T) {
|
|
_, err := PlanApplyBatch(planApplyRequest(test.items...))
|
|
var refErr *issueops.RefError
|
|
if !errors.As(err, &refErr) {
|
|
t.Fatalf("error = %v, want *issueops.RefError", err)
|
|
}
|
|
if !errors.Is(err, issueops.ErrValidation) {
|
|
t.Fatal("a *RefError must match ErrValidation, so a front door classifies it without knowing the type")
|
|
}
|
|
if refErr.Index != test.index || refErr.Member != test.member {
|
|
t.Fatalf("RefError{Index: %d, Member: %q}, want {Index: %d, Member: %q}",
|
|
refErr.Index, refErr.Member, test.index, test.member)
|
|
}
|
|
if refErr.DeclaredLater != test.declaredLater {
|
|
t.Fatalf("RefError.DeclaredLater = %v, want %v: a key declared later and a key nothing declares "+
|
|
"are different diagnoses", refErr.DeclaredLater, test.declaredLater)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchMetadataRefsReachAnyDirection pins the ONE exception to the
|
|
// backward rule, and it is the measured shape rather than a generalization: a
|
|
// plan whose first step records the id of a row a later step mints (the retry
|
|
// that stamps the original's id onto its replacement). Every id exists before
|
|
// any splice runs, so the direction cannot matter for a VALUE.
|
|
func TestPlanApplyBatchMetadataRefsReachAnyDirection(t *testing.T) {
|
|
forward := issueops.ApplyItem{Kind: issueops.ItemCreate, Create: &issueops.CreateItem{
|
|
Key: "first",
|
|
Issue: &issueops.Issue{Title: "first"},
|
|
MetadataRefs: map[string]issueops.Ref{"gc.retry_of": {Key: "second"}, "gc.self": {Key: "first"}},
|
|
}}
|
|
if _, err := PlanApplyBatch(planApplyRequest(forward, planApplyCreate("second"))); err != nil {
|
|
t.Fatalf("PlanApplyBatch = %v, want a forward and a self metadata_ref to resolve", err)
|
|
}
|
|
|
|
unknown := issueops.ApplyItem{Kind: issueops.ItemCreate, Create: &issueops.CreateItem{
|
|
Issue: &issueops.Issue{Title: "first"},
|
|
MetadataRefs: map[string]issueops.Ref{"gc.retry_of": {Key: "ghost"}},
|
|
}}
|
|
_, err := PlanApplyBatch(planApplyRequest(unknown))
|
|
var refErr *issueops.RefError
|
|
if !errors.As(err, &refErr) {
|
|
t.Fatalf("error = %v, want *issueops.RefError for a metadata_ref naming no item", err)
|
|
}
|
|
if refErr.Member != "metadata_ref gc.retry_of" {
|
|
t.Fatalf("RefError.Member = %q, want it to name WHICH ref failed", refErr.Member)
|
|
}
|
|
if refErr.DeclaredLater {
|
|
t.Fatal("a metadata_ref has no forward/backward distinction to report, so DeclaredLater must be false")
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchRefusesAVersionGuardOnARowThisRequestWrote is the F4 rule,
|
|
// and it is a REQUEST-SHAPE refusal rather than a race: the token is minted by
|
|
// the write, so mid-request there is no value a caller could send. Letting it
|
|
// through would answer every such request with ErrVersionMismatch and send the
|
|
// caller looking for a concurrent writer that does not exist.
|
|
func TestPlanApplyBatchRefusesAVersionGuardOnARowThisRequestWrote(t *testing.T) {
|
|
for _, test := range []struct {
|
|
name string
|
|
items []issueops.ApplyItem
|
|
}{
|
|
{
|
|
"guard on a row an earlier item created",
|
|
[]issueops.ApplyItem{planApplyCreate("root"), planApplyUpdate(issueops.Ref{Key: "root"}, int64Ptr(7))},
|
|
},
|
|
{
|
|
"guard on a row an earlier item updated",
|
|
[]issueops.ApplyItem{
|
|
planApplyUpdate(issueops.Ref{ID: "bd-1"}, nil),
|
|
planApplyUpdate(issueops.Ref{ID: "bd-1"}, int64Ptr(7)),
|
|
},
|
|
},
|
|
{
|
|
"guard on a row an earlier item closed",
|
|
[]issueops.ApplyItem{
|
|
planApplyClose(issueops.Ref{ID: "bd-1"}, nil),
|
|
planApplyUpdate(issueops.Ref{ID: "bd-1"}, int64Ptr(7)),
|
|
},
|
|
},
|
|
{
|
|
"close guard on a row an earlier item updated",
|
|
[]issueops.ApplyItem{
|
|
planApplyUpdate(issueops.Ref{ID: "bd-1"}, nil),
|
|
planApplyClose(issueops.Ref{ID: "bd-1"}, int64Ptr(7)),
|
|
},
|
|
},
|
|
{
|
|
"guard on a row an earlier item created by explicit id",
|
|
[]issueops.ApplyItem{
|
|
{Kind: issueops.ItemCreate, Create: &issueops.CreateItem{Issue: &issueops.Issue{ID: "bd-1"}}},
|
|
planApplyClose(issueops.Ref{ID: "bd-1"}, int64Ptr(7)),
|
|
},
|
|
},
|
|
} {
|
|
t.Run(test.name, func(t *testing.T) {
|
|
_, err := PlanApplyBatch(planApplyRequest(test.items...))
|
|
if !errors.Is(err, issueops.ErrValidation) {
|
|
t.Fatalf("error = %v, want ErrValidation", err)
|
|
}
|
|
if !strings.Contains(err.Error(), "cannot be known mid-request") {
|
|
t.Fatalf("error = %q, want it to say why the guard is unanswerable", err)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchAcceptsAVersionGuardTheRequestCanAnswer is the positive
|
|
// half, and it covers the three shapes the rule must NOT refuse. The dep_add
|
|
// row is the one that is a decision rather than an omission: an edge write is a
|
|
// change to the GRAPH, and this role promises nothing about whether it moves
|
|
// either endpoint's version token, so a later guard on that source is left to
|
|
// the substrate where a real mismatch is an honest ErrVersionMismatch.
|
|
func TestPlanApplyBatchAcceptsAVersionGuardTheRequestCanAnswer(t *testing.T) {
|
|
for _, test := range []struct {
|
|
name string
|
|
items []issueops.ApplyItem
|
|
}{
|
|
{
|
|
"guard on a row this request has not written",
|
|
[]issueops.ApplyItem{
|
|
planApplyUpdate(issueops.Ref{ID: "bd-1"}, nil),
|
|
planApplyUpdate(issueops.Ref{ID: "bd-2"}, int64Ptr(7)),
|
|
},
|
|
},
|
|
{
|
|
"guard on the first item",
|
|
[]issueops.ApplyItem{planApplyUpdate(issueops.Ref{ID: "bd-1"}, int64Ptr(7))},
|
|
},
|
|
{
|
|
"guard after an edge write on the same source",
|
|
[]issueops.ApplyItem{
|
|
planApplyDep(issueops.Ref{ID: "bd-1"}, issueops.Ref{ID: "bd-2"}, types.DepBlocks, ""),
|
|
planApplyUpdate(issueops.Ref{ID: "bd-1"}, int64Ptr(7)),
|
|
},
|
|
},
|
|
{
|
|
"a key and an id that read alike are two rows",
|
|
[]issueops.ApplyItem{
|
|
planApplyCreate("bd-1"),
|
|
planApplyUpdate(issueops.Ref{ID: "bd-1"}, int64Ptr(7)),
|
|
},
|
|
},
|
|
} {
|
|
t.Run(test.name, func(t *testing.T) {
|
|
if _, err := PlanApplyBatch(planApplyRequest(test.items...)); err != nil {
|
|
t.Fatalf("PlanApplyBatch = %v, want the guard accepted", err)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchOtherGuardsAreNeverStaticallyRefused pins the difference
|
|
// between the version token and the other two: a caller CAN know it wants the
|
|
// status its own earlier item set, and as-modified evaluation gives it exactly
|
|
// that. Refusing those the way the version guard is refused would delete the
|
|
// role's most useful shape.
|
|
func TestPlanApplyBatchOtherGuardsAreNeverStaticallyRefused(t *testing.T) {
|
|
status := issueops.StatusInProgress
|
|
assignee := "worker"
|
|
items := []issueops.ApplyItem{
|
|
planApplyCreate("root"),
|
|
{Kind: issueops.ItemUpdate, Update: &issueops.UpdateItem{
|
|
Target: issueops.Ref{Key: "root"},
|
|
ExpectedStatus: &status,
|
|
ExpectedAssignee: &assignee,
|
|
}},
|
|
}
|
|
if _, err := PlanApplyBatch(planApplyRequest(items...)); err != nil {
|
|
t.Fatalf("PlanApplyBatch = %v, want an as-modified status/assignee guard accepted", err)
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchNormalizesWaitsForGates pins the migration-0059 invariant:
|
|
// a stored waits-for row must be SELF-DESCRIBING, because readers that predate
|
|
// the gate do not default a missing one. Empty metadata on a waits-for edge is
|
|
// a row those readers get wrong, so it is never what is stored.
|
|
func TestPlanApplyBatchNormalizesWaitsForGates(t *testing.T) {
|
|
allChildren := fmt.Sprintf(`{"gate":%q}`, types.WaitsForAllChildren)
|
|
|
|
for _, test := range []struct {
|
|
name string
|
|
depType types.DependencyType
|
|
metadata string
|
|
want string
|
|
}{
|
|
{"absent gate", types.DepWaitsFor, "", allChildren},
|
|
{"blank gate", types.DepWaitsFor, " ", allChildren},
|
|
{"empty object", types.DepWaitsFor, "{}", allChildren},
|
|
{"explicit all-children", types.DepWaitsFor, allChildren, allChildren},
|
|
{
|
|
"any-children survives",
|
|
types.DepWaitsFor,
|
|
fmt.Sprintf(`{"gate":%q}`, types.WaitsForAnyChildren),
|
|
fmt.Sprintf(`{"gate":%q}`, types.WaitsForAnyChildren),
|
|
},
|
|
{
|
|
"the spawner and also-blocks members survive",
|
|
types.DepWaitsFor,
|
|
`{"spawner_id":"bd-9","also_blocks":true}`,
|
|
fmt.Sprintf(`{"gate":%q,"spawner_id":"bd-9","also_blocks":true}`, types.WaitsForAllChildren),
|
|
},
|
|
{"a blocking edge is untouched", types.DepBlocks, "", ""},
|
|
{"other metadata passes through", types.DepRelated, `{"score":0.5}`, `{"score":0.5}`},
|
|
} {
|
|
t.Run(test.name, func(t *testing.T) {
|
|
plan, err := PlanApplyBatch(planApplyRequest(
|
|
planApplyDep(issueops.Ref{ID: "bd-1"}, issueops.Ref{ID: "bd-2"}, test.depType, test.metadata),
|
|
))
|
|
if err != nil {
|
|
t.Fatalf("PlanApplyBatch = %v", err)
|
|
}
|
|
got := plan.Items[0].DepAdd.Metadata
|
|
if test.depType != types.DepWaitsFor {
|
|
if got != test.want {
|
|
t.Fatalf("metadata = %q, want %q unchanged", got, test.want)
|
|
}
|
|
return
|
|
}
|
|
var gotMeta, wantMeta types.WaitsForMeta
|
|
if err := json.Unmarshal([]byte(got), &gotMeta); err != nil {
|
|
t.Fatalf("normalized metadata %q is not a gate object: %v", got, err)
|
|
}
|
|
if err := json.Unmarshal([]byte(test.want), &wantMeta); err != nil {
|
|
t.Fatalf("want %q is not a gate object: %v", test.want, err)
|
|
}
|
|
if gotMeta != wantMeta {
|
|
t.Fatalf("normalized metadata = %+v, want %+v", gotMeta, wantMeta)
|
|
}
|
|
if gotMeta.Gate == "" {
|
|
t.Fatal("a stored waits-for row with no gate is exactly what migration 0059 forbids")
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchRefusesAnUnknownWaitsForGate pins the other half: a gate
|
|
// this workspace could never evaluate is refused rather than stored.
|
|
func TestPlanApplyBatchRefusesAnUnknownWaitsForGate(t *testing.T) {
|
|
for _, metadata := range []string{`{"gate":"first-tuesday"}`, `{"gate":`} {
|
|
_, err := PlanApplyBatch(planApplyRequest(
|
|
planApplyDep(issueops.Ref{ID: "bd-1"}, issueops.Ref{ID: "bd-2"}, types.DepWaitsFor, metadata),
|
|
))
|
|
if !errors.Is(err, issueops.ErrValidation) {
|
|
t.Fatalf("PlanApplyBatch(%q) = %v, want ErrValidation", metadata, err)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchDoesNotMutateTheCallerRequest is the promise every role
|
|
// makes, and the plan is the one place this role could break it: it normalizes
|
|
// gate metadata, and the item payloads it copies are the caller's POINTERS. The
|
|
// case would go green on a plan that wrote the normalized gate straight through
|
|
// the pointer, so it compares the whole request rather than the one member.
|
|
func TestPlanApplyBatchDoesNotMutateTheCallerRequest(t *testing.T) {
|
|
request := issueops.ApplyBatchRequest{
|
|
Actor: "planner",
|
|
Items: []issueops.ApplyItem{
|
|
planApplyCreate("root"),
|
|
planApplyDep(issueops.Ref{Key: "root"}, issueops.Ref{ID: "bd-2"}, types.DepWaitsFor, ""),
|
|
},
|
|
}
|
|
before := issueops.ApplyBatchRequest{
|
|
Actor: request.Actor,
|
|
Items: []issueops.ApplyItem{
|
|
planApplyCreate("root"),
|
|
planApplyDep(issueops.Ref{Key: "root"}, issueops.Ref{ID: "bd-2"}, types.DepWaitsFor, ""),
|
|
},
|
|
}
|
|
|
|
plan, err := PlanApplyBatch(request)
|
|
if err != nil {
|
|
t.Fatalf("PlanApplyBatch = %v", err)
|
|
}
|
|
if plan.Items[1].DepAdd.Metadata == "" {
|
|
t.Fatal("the plan did not normalize the gate at all; this case would be vacuous")
|
|
}
|
|
if !reflect.DeepEqual(request.Items[0].Create, before.Items[0].Create) {
|
|
t.Fatalf("PlanApplyBatch wrote through to the caller's create item: %+v", request.Items[0].Create)
|
|
}
|
|
if !reflect.DeepEqual(request.Items[1].DepAdd, before.Items[1].DepAdd) {
|
|
t.Fatalf("PlanApplyBatch wrote the normalized gate through to the caller's edge item: %+v",
|
|
request.Items[1].DepAdd)
|
|
}
|
|
}
|
|
|
|
// TestPlanApplyBatchCarriesTheRequestForward pins that the plan is the whole
|
|
// request rather than a filtered one: a field dropped here is a field no
|
|
// backend can honor, and nothing else in the tree would notice.
|
|
func TestPlanApplyBatchCarriesTheRequestForward(t *testing.T) {
|
|
plan, err := PlanApplyBatch(issueops.ApplyBatchRequest{
|
|
Actor: "planner",
|
|
Provenance: "bd: apply plan.md",
|
|
ForceIDPrefix: true,
|
|
SkipPerEdgeCycleCheck: true,
|
|
Items: []issueops.ApplyItem{planApplyCreate("root")},
|
|
})
|
|
if err != nil {
|
|
t.Fatalf("PlanApplyBatch = %v", err)
|
|
}
|
|
if plan.Actor != "planner" || plan.Provenance != "bd: apply plan.md" ||
|
|
!plan.ForceIDPrefix || !plan.SkipPerEdgeCycleCheck {
|
|
t.Fatalf("plan = %+v, want every request field carried", plan)
|
|
}
|
|
if len(plan.Items) != 1 {
|
|
t.Fatalf("plan.Items = %d, want 1", len(plan.Items))
|
|
}
|
|
}
|