From a1bc167b54922b7a3953b23d4427aa7fb46409fc Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Fri, 25 Sep 2026 13:53:47 -0700 Subject: [PATCH] fix(ready): keep the ReadyCounter span through externaldeps, and announce the interface break (#6748) Maintainer review fixups for #6731. The review found no blocker and no major; these are its two gate-impact minors plus two nearby nits. - externaldeps.Store.ReadyCounter built a fresh store-backed counter, so the documented storage.ReadyCounter.CountReady span stopped being emitted for every text-mode `bd ready` in the cmd/bd chain (hooks -> externaldeps -> telemetry -> store). PR-introduced: at base this decorator had no ReadyCounter at all, so the embedded passthrough reached the instrumented layer -- and this PR edits the very otel document that lists the span. The accessor cannot simply recurse, because the exclusions live on this store, so it now hands the finished counter to the inner store's WrapReadyCounter, which telemetry already exported for exactly this case and had no caller for. Pinned by TestExternalDepsReadyCounterSurvivesThisLayer, which fails with the old body (verified by reverting it). - CHANGELOG: GetReadyWorkWithCountsAndTotal is a new REQUIRED storage.Storage method with no optional-interface fallback, so an out-of-tree `backends` registrant stops compiling on upgrade. The entry names the method, the contract an implementer owes, and why the compile-time posture is deliberate. - The ready probe's facts now record each index as its column is appended. Two conditional columns were read back through a mix of literals and a computed `next`, so the two conventions disagreed inside one function and a future column inserted above would silently mis-read a different fact. Behavior-identical; the four excludeDeferred x wantCollision combinations are covered by the embedded filter matrix. - The combined family count wraps as `count ready work: issues+wisps:`, so it is distinguishable from the issues-only retry beneath it, and storereadycounter's package doc no longer claims only two importers. Validation: go build ./...; go vet and go test on issueops, externaldeps, telemetry; BEADS_TEST_EMBEDDED_DOLT=1 TestReadyWorkPageTotal, TestReadyWorkPageTotalFilterMatrix, TestReadyWorkStatementBudget (embeddeddolt) and TestEmbeddedReady (cmd/bd). Co-authored-by: Claude Opus 5 --- CHANGELOG.md | 28 +++++++++++ internal/storage/externaldeps/decorator.go | 25 +++++++++- .../storage/issueops/ready_work_counts.go | 5 +- internal/storage/issueops/ready_work_probe.go | 46 ++++++++++++------- .../telemetry/role_accessor_decorator_test.go | 31 +++++++++++++ internal/workapi/storereadycounter/counter.go | 13 ++++-- 6 files changed, 126 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 55232ef9d..8e7fd405f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -253,6 +253,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 of its own. Unsetting a key that is not set remains a successful no-op in every shape. +### Changed + +- **`bd ready --json` answers a capped page and its total in one pass, and + `storage.Storage` gains a required `GetReadyWorkWithCountsAndTotal` method** + ([#6731](https://github.com/gastownhall/beads/pull/6731)). A full capped page + used to trigger a second counting pass in its own transaction, re-running the + expired-defer sweep and every wisp/deferred probe; the total now comes from + `COUNT(*) OVER ()` on the page's own query inside the same read transaction. + Two behaviors improve with it: the `--json` total honors external-dependency + exclusions (the old second pass counted externally blocked issues, so text and + `--json` could report different numbers), and the page and its total now come + from one snapshot instead of two, so "Showing N of M" can no longer be + internally inconsistent under concurrent writes. One contract change: a total + that fails now fails the command, where it previously could not — the total is + taken from the page's own query, so the page would have failed anyway. + + **Out-of-tree storage backends must implement the new method to compile.** + Every in-tree implementer is updated, but `internal/storage/backends` is an + extension seam whose registrants are supplied by a downstream distribution and + return `storage.DoltStorage` directly. A registrant must implement + `GetReadyWorkWithCountsAndTotal(ctx, types.WorkFilter) ([]*types.IssueWithCounts, int, error)`: + the page identical to `GetReadyWorkWithCounts`, and the total identical to the + number `CountReadyWork` answers for the same filter (which ignores `Limit`), + resolved in the page's own read transaction. There is + deliberately no optional-interface fallback — a store that cannot size the + ready set should fail to compile rather than silently fall back to an + unbounded query. + ## [1.3.0] - 2026-09-15 The first tested release off `main` since the 1.1 line. [1.2.2] was a recovery diff --git a/internal/storage/externaldeps/decorator.go b/internal/storage/externaldeps/decorator.go index b9ac2c545..721974eeb 100644 --- a/internal/storage/externaldeps/decorator.go +++ b/internal/storage/externaldeps/decorator.go @@ -236,8 +236,31 @@ func (s *Store) CountReadyWork(ctx context.Context, filter types.WorkFilter) (in // GetReadyWorkWithCountsAndTotal. It must be overridden here: the embedded // passthrough would hand back the inner store's counter, which counts // externally blocked issues as ready. +// +// Overriding costs the layers beneath their turn, though, and this accessor +// cannot simply recurse the way IssueLifecycle above does: the exclusions live +// on THIS store, so the counter has to be built over it. The inner store gets +// its layer back by wrapping the finished counter — which is what +// telemetry.InstrumentedStorage.WrapReadyCounter exists for, and why the +// documented storage.ReadyCounter.CountReady span still appears for text-mode +// `bd ready` in the cmd/bd chain (hooks -> externaldeps -> telemetry -> store). func (s *Store) ReadyCounter() (publicops.ReadyCounter, error) { - return storereadycounter.New(s) + counter, err := storereadycounter.New(s) + if err != nil { + return nil, err + } + if wrapper, ok := s.inner.(readyCounterWrapper); ok { + return wrapper.WrapReadyCounter(counter), nil + } + return counter, nil +} + +// readyCounterWrapper is how a decorator beneath this one adds its layer to a +// ready counter it did not construct. The shape is asserted rather than +// imported so this package stays independent of which layers are wired below +// it — telemetry's wrapper is absent entirely when telemetry is disabled. +type readyCounterWrapper interface { + WrapReadyCounter(publicops.ReadyCounter) publicops.ReadyCounter } // ClaimReadyIssue resolves external blockers before using the existing local diff --git a/internal/storage/issueops/ready_work_counts.go b/internal/storage/issueops/ready_work_counts.go index 9b6e318d8..a54aab812 100644 --- a/internal/storage/issueops/ready_work_counts.go +++ b/internal/storage/issueops/ready_work_counts.go @@ -333,7 +333,10 @@ func CountReadyWorkInTx(ctx context.Context, tx DBTX, filter types.WorkFilter) ( issueCount, wispCount, err := countReadyFamiliesInTx(ctx, tx, issuePreds, wispPreds) if err != nil { if !missingOptionalWispTable(err) { - return 0, fmt.Errorf("count ready work: %w", err) + // Both families ride one statement now, so name it: the + // issues-only retry below reports `issues:`, and a reader + // otherwise cannot tell the combined statement from it. + return 0, fmt.Errorf("count ready work: issues+wisps: %w", err) } // A wisp plane the database may legitimately lack: issues-only, and // the issues count has to be taken on its own. diff --git a/internal/storage/issueops/ready_work_probe.go b/internal/storage/issueops/ready_work_probe.go index 9ed77af0c..cd8d48988 100644 --- a/internal/storage/issueops/ready_work_probe.go +++ b/internal/storage/issueops/ready_work_probe.go @@ -58,21 +58,33 @@ func probeReadyWorkInTx(ctx context.Context, tx DBTX, filter types.WorkFilter, w p := &readyWorkProbe{} excludeDeferred := !filter.IncludeDeferred - cols := []string{ - "EXISTS (SELECT 1 FROM wisps)", - // Only the table REFERENCE matters here: the statement fails with a - // missing-table error when wisp_dependencies is absent, and an empty - // table still exists (optionalTableExistsInTx's contract). The value - // is not read. - "EXISTS (SELECT 1 FROM wisp_dependencies)", + // Which columns this statement carries depends on the filter, so each fact + // records the index it landed on as it is appended rather than the reader + // below counting appends back. Two of these columns are conditional, so a + // hardcoded index is a fact read from the wrong column the first time a + // new one is inserted above it — and the fact being misread would be the + // ready predicate. + cols := make([]string, 0, 5) // the two unconditional facts plus the three conditional ones + appendFact := func(sql string) int { + cols = append(cols, sql) + return len(cols) - 1 } + + iWispsPresent := appendFact("EXISTS (SELECT 1 FROM wisps)") + // Only the table REFERENCE matters here: the statement fails with a + // missing-table error when wisp_dependencies is absent, and an empty + // table still exists (optionalTableExistsInTx's contract). The value + // is not read, so this fact keeps no index. + appendFact("EXISTS (SELECT 1 FROM wisp_dependencies)") + + iIssuesDeferred, iWispsDeferred := -1, -1 if excludeDeferred { - cols = append(cols, - "EXISTS (SELECT 1 FROM issues WHERE "+futureDeferredPredicate+")", - "EXISTS (SELECT 1 FROM wisps WHERE "+futureDeferredPredicate+")") + iIssuesDeferred = appendFact("EXISTS (SELECT 1 FROM issues WHERE " + futureDeferredPredicate + ")") + iWispsDeferred = appendFact("EXISTS (SELECT 1 FROM wisps WHERE " + futureDeferredPredicate + ")") } + iCollision := -1 if wantCollision { - cols = append(cols, "EXISTS (SELECT 1 FROM issues JOIN wisps ON wisps.id = issues.id)") + iCollision = appendFact("EXISTS (SELECT 1 FROM issues JOIN wisps ON wisps.id = issues.id)") } flags := make([]bool, len(cols)) dest := make([]any, len(cols)) @@ -84,16 +96,16 @@ func probeReadyWorkInTx(ctx context.Context, tx DBTX, filter types.WorkFilter, w err := tx.QueryRowContext(ctx, "SELECT "+strings.Join(cols, ", ")).Scan(dest...) switch { case err == nil: - p.wispsPresent = flags[0] + p.wispsPresent = flags[iWispsPresent] // The statement ran, so every table it names exists. p.wispDepsExist = true - next := 2 if excludeDeferred { - hasDeferredParent = flags[2] || flags[3] - next = 4 + hasDeferredParent = flags[iIssuesDeferred] || flags[iWispsDeferred] + } + if iCollision >= 0 { + // Without a wisp row there is nothing to collide with. + p.idCollision = p.wispsPresent && flags[iCollision] } - // Without a wisp row there is nothing to collide with. - p.idCollision = wantCollision && p.wispsPresent && flags[next] if hasDeferredParent { // Every table the child scan reads is known to exist, so its // four legs can go out as one statement. diff --git a/internal/telemetry/role_accessor_decorator_test.go b/internal/telemetry/role_accessor_decorator_test.go index cd5609f4c..0d7caece4 100644 --- a/internal/telemetry/role_accessor_decorator_test.go +++ b/internal/telemetry/role_accessor_decorator_test.go @@ -10,6 +10,7 @@ import ( "testing" "github.com/steveyegge/beads/internal/storage" + "github.com/steveyegge/beads/internal/storage/externaldeps" "github.com/steveyegge/beads/issueops" "github.com/steveyegge/beads/memoryops" ) @@ -387,6 +388,36 @@ func TestInstrumentedStorageInstrumentsEveryRoleAccessor(t *testing.T) { } } +// TestExternalDepsReadyCounterSurvivesThisLayer pins the span across the one +// decorator above this one that CANNOT recurse. cmd/bd wires +// hooks -> externaldeps -> telemetry -> store, and externaldeps has to build +// its own ready counter over itself so the total honors its external-dependency +// exclusions — recursing into the inner counter the way the accessors above do +// would count externally blocked issues as ready. That leaves handing the +// finished counter to WrapReadyCounter as the only way this layer keeps its +// turn, so an externaldeps override that skips it silently drops the +// storage.ReadyCounter.CountReady span for every text-mode `bd ready`. +// +// This is the only test that wires a real decorator above the instrumented +// store, which is why WrapReadyCounter's sole caller outside this package is +// the thing being pinned here. +func TestExternalDepsReadyCounterSurvivesThisLayer(t *testing.T) { + t.Setenv("BD_OTEL_STDOUT", "true") + inner := &roleAccessorStore{surface: &roleAccessorSentinel{}} + instrumented, ok := WrapStorage(inner).(*InstrumentedStorage) + if !ok { + t.Fatal("WrapStorage did not instrument the store; telemetry is disabled in this environment") + } + + counter, err := externaldeps.New(instrumented, nil, nil).ReadyCounter() + if err != nil { + t.Fatalf("externaldeps ReadyCounter() error = %v", err) + } + if _, ok := counter.(*instrumentedReadyCounter); !ok { + t.Fatalf("externaldeps ReadyCounter() = %T, want it wrapped by this layer; the storage.ReadyCounter.CountReady span stops being emitted for text-mode bd ready", counter) + } +} + // TestInstrumentedStorageRoleAccessorsPropagateInnerErrors pins the other half // of the recursion: an accessor that recurses must not swallow the inner // store's refusal, nor hand back a wrapper around nothing. diff --git a/internal/workapi/storereadycounter/counter.go b/internal/workapi/storereadycounter/counter.go index 87a30bffc..d049b5425 100644 --- a/internal/workapi/storereadycounter/counter.go +++ b/internal/workapi/storereadycounter/counter.go @@ -5,9 +5,16 @@ // It is a package of its own for the reason internal/workapi/storereader and // internal/workapi/storecounter are: a constructor sitting in internal/workapi // would be a one-line drop-in for store.ReadyCounter() that silently skips the -// decorators. Down here the only importers are the two Dolt store packages, and -// the cmd-bd-role-constructors depguard rule in .golangci.yml makes a front -// door importing it a lint failure rather than a review comment. +// decorators. Down here the importers are the two Dolt store packages and +// internal/storage/externaldeps, and the cmd-bd-role-constructors depguard rule +// in .golangci.yml makes a front door importing it a lint failure rather than a +// review comment. +// +// The external-dependency decorator needs its own construction site because its +// exclusions live on the decorator, not beneath it: recursing into the inner +// store's counter the way its other accessors do would count externally blocked +// issues as ready. It hands the counter it builds to the inner store's +// WrapReadyCounter so the layers below still get their turn. package storereadycounter import (