Commit graph

39 commits

Author SHA1 Message Date
Jim Wordelman
2db739e8b0
perf(storage): SearchIssueSummaries narrow-projection foundation (be-nu4.3) (#3458)
* feat(search): re-cut SearchIssueSummaries atop Lite as searchProjection[T]

Re-cuts release/be-nu4.3-stack onto current main, resolved against the
merged Lite foundation (#3906) instead of the stale pre-Lite conflict set.

SearchIssueSummaries moves from a standalone search_summary.go into a
summaryProjection literal alongside issueProjection/issueLiteProjection/
idProjection in issueops/search.go, sharing the generic searchInTx core
instead of a parallel top-level function — retiring the anti-pattern
coffeegoddd's CHANGES_REQUESTED flagged. LessSummary shares lessFields
with Less via a summarySortFields adapter so the two comparators can't
drift on NULL-handling or tie-break rules.

Adds parity coverage (SearchIssues vs SearchIssueSummaries agree on ID
set/order and on every IssueSummary field) and the three-way wide/lite/
summary A/B benchmark, extending dolt_benchmark_test.go — which still
only compiles together with the projection method, the fact that
retired the earlier "land the bench file first" plan.

* test(feat): red — fix parity test to compare order directly + add comparator/wisp-limit coverage (refs be-be3pj)

- TestSearchIssuesAndSearchIssueSummaries_Parity now compares
  SearchIssues/SearchIssueSummaries IDs order-sensitively (slices.Equal)
  instead of via the sort-then-compare equalStringSlices helper, which
  would silently pass an ordering/top-N regression.
- Add TestLessSummaryMatchesLessAcrossSortKeys: Less vs LessSummary
  produce identical order across every SortDefs key + "id", both
  directions.
- Add TestSearchIssueSummariesInTx_MergesAcrossIssuesAndWisps: be-yrtwi
  regression pattern for summaryProjection specifically, covering
  wisp-merge top-N truncation under a limit.

* fix(types): give IssueSummary the same JSON tags as Issue

IssueSummary carried no struct tags, so encoding/json emitted Go's default
field names ({"ID":…,"Title":…}). The type exists to back list-shaped
rendering and `bd list --json` is one of that command's primary modes, so
the wiring follow-up would have silently changed the wire shape for every
consumer parsing bd output. Zero production callers today makes this a
zero-risk moment to fix it.

Tags mirror the same-named fields on types.Issue exactly, including
priority carrying no omitempty (0 is a valid P0). Three tests pin it: a
reflection test comparing every tag against Issue's so the two cannot
drift, an exact-key-set marshal test, and an omitempty test.

Reviewer item 1 (maphew, 2026-08-03).

* docs(storage): pin the --sort id contract on SearchIssueSummaries

The interface said SortBy/SortDesc are honored identically to SearchIssues,
which is true but left the id case looking like a gap in the summary path.
It is not a gap, and the doc now says why: sqlbuild.OrderBy emits no ORDER BY
for id (sqlbuild.IsGoSideSort), so *neither* path gets a SQL id order, and
both carry a comparator used only where the issues+wisps merge re-sorts
(sqlbuild.Less / sqlbuild.LessSummary).

The user-facing id order is cmd/bd's own, applied after the call via
utils.NaturalCompareIDs (cmd/bd/list.go sortIssues -> compareIssuesBy) —
numeric-aware, so bd-2 sorts before bd-10. Adding a lexicographic
storage-side summary sorter would therefore diverge from SearchIssues and
still not match what bd list renders.

Reviewer question (maphew, 2026-08-03): the contract no longer depends on
an unfiled follow-up to be readable.

* fix(backend): alias IssueSummary into the public backend contract

#5321 landed the public backend surface and its reflection drift guard on
2026-08-03, about two hours after this branch's last push, so the 08-11
maintenance rebase brought the branch under a gate that did not exist when
the engine interface was widened. TestPublicSurfaceComplete fails on both
platforms at the rebased head:

    internal type github.com/steveyegge/beads/internal/types.IssueSummary
    is reachable from the public backend contract but has no alias in
    package backend

The gate is right and the rebase broke nothing. SearchIssueSummaries
returns []*types.IssueSummary on storage.Storage, which backend.DoltStorage
aliases whole, so the row type is now part of the contract an out-of-tree
backend writes against — and without the alias that backend cannot name it.

Verified this is ours rather than base breakage: the same test passes on
origin/main and fails only on this branch.

* ci: retrigger after infra-flake red on dc92c3c3

Run 34381971686 went red on two failures, neither of them in this PR's
15-file diff:

- Test (storage domain + uow): internal/tracker lost its Dolt
  testcontainer mid-suite after a Ryuk reaper handshake reset
  (54 failed, 1994 passed). Known parallel-binary reaper flake; the
  fix is PR #5770, open and unmerged upstream since 2026-08-15. Third
  recorded recurrence, same package and same mapped port as the prior
  two.

- PR Core (wrapper timing): TestDoltServer_LogFile_CapturesOutput in
  internal/storage/dbproxy/server, a package this PR does not touch.
  Not reproducible locally in 55 attempts against clean main, 30 of
  them under 2x CPU load with the CI-pinned dolt 2.2.0.

We are pull-only on the upstream repo and cannot rerun jobs, so an
empty commit is the only retrigger lever available.

* test(issueops): tie the summary column list to its scanner, delete a third copy

Review point (minor 2): IssueSummaryColumns, ScanIssueSummaryFrom's dest list
and search_summary_merge_test.go's summaryColumnNames were three hand-synced
copies of the same list, with only the real-database field-parity test to
catch a drift between them.

Two changes rather than one guard over three lists:

  - summaryColumnNames now derives from IssueSummaryColumns instead of
    restating it, so the fixture cannot drift at all. summaryMergeRow places
    its values by column NAME against that list rather than by literal
    position, so adding a column widens the row automatically instead of
    silently shifting every value one slot left.

  - TestIssueSummaryColumnsMatchScanner pins the remaining pair by count,
    using a counting scanner that needs no database. This is the summary
    counterpart of TestIssueSelectColumns_LitePlusHeavyEqualsFull.

TestIssueSummaryColumnsAreRealIssueColumns covers what a count cannot: a
misspelled column keeps the widths equal and fails only at query time against
a real database, as an opaque unknown-column error from whichever caller runs
first. Requiring membership in IssueSelectColumns catches it in a unit test.

Membership, not order: unlike IssueSelectColumnsLite, IssueSummaryColumns is
deliberately not a subsequence of the full list, so there is no analogue of
TestIssueSelectColumnsLite_IsFullMinusHeavyInOrder.

Both guards verified red-before-green by mutation: adding a column without a
dest fails the first, and misspelling one fails the second.

* fix(types): carry the wisp-plane markers through the summary projection

Review point (MAJOR): IssueSummary's doc comment promises a summary-backed
list serializes to the same wire shape a full-Issue-backed one does, and
issueops.searchInTx merges wisp rows into every result whose filter does not
set SkipWisps. But IssueSummary had no Ephemeral, NoHistory, WispType or
StorageClass, so a wisp rendered through it dropped ephemeral, no_history,
wisp_type and storage_class from `bd list --json` while every other key stayed
identical -- a wisp and a durable bead became indistinguishable on the wire.
maphew raised this on 2026-08-03 and it was still open.

Adding the fields, not excluding wisps. Excluding them would break the
same-wisp-merge-semantics claim that is this PR's central design argument, and
would make the wiring follow-up silently drop wisps from `bd list` entirely --
a strictly worse outcome than the missing keys.

Correcting our own 2026-08-03 reasoning, which argued the other way. That
comment observed that no list *renderer* reads Ephemeral or WispType back off
the row, which is true and beside the point: `--json` does not read fields,
it marshals the struct. It also treated the cost as the one D3 exists to
remove. These four are narrow scalars -- two TINYINT(1), two short VARCHARs,
already present on both the issues and wisps tables and already scanned by
ScanIssueFrom -- not the multi-KB TEXT bodies and JSON parse the projection is
narrow to avoid. Zero callers today makes this free; after the wiring lands it
is a wire-format change.

Verified red-before-green at both levels:

  - TestIssueSummaryCarriesWispMarkers failed on all four fields before the
    change (reflection against Issue, asserting tag AND type -- WispType and
    StorageClass are named string types that a plain string would impersonate
    under a tag-only check).

  - TestIssueSummaryWispKeysMatchIssue marshals the same marker values through
    both types and compares the four keys byte for byte, including a durable
    row that must emit none of them.

  - TestSearchIssueSummaries_FieldsMatchSearchIssues now seeds an ephemeral
    wisp and a no-history wisp (mutually exclusive, so one row each) and
    compares all four markers against SearchIssues on a real Dolt database. It
    refuses to pass vacuously: fewer than two wisp rows in the result is a
    fatal, since zero-vs-zero marker comparisons would otherwise succeed if
    the merge stopped returning wisps. Mutating ScanIssueSummaryFrom to stop
    mapping wisp_type fails it on both seeded rows.

ScanIssueSummaryFrom maps the four exactly as ScanIssueFrom does, including a
NULL storage_class staying empty for EffectiveStorageClass to resolve per
plane rather than being pinned to a literal.

* perf(sqlbuild): compare sortFields through pointers, not two copies per call

Review point (minor 3): the shared-comparator refactor made Less build two
sortFields values per comparison and pass them by value, a small regression on
the pre-existing issues+wisps merge sort inside a perf PR.

lessFields and sortKeyCompareFields now take *sortFields. sortFields is a
~150-byte struct, so passing by value copied it twice into lessFields and
twice more into sortKeyCompareFields on every comparison; that is now zero
copies after the adapters run. The adapters still build one value per row --
that normalization is exactly what buys Less and LessSummary a single shared
body, and giving it up would reintroduce the drift this PR exists to remove.

No allocation either way, before or after: escape analysis reports nothing in
Less/LessSummary/lessFields escaping to heap (the only escapes in sort.go are
the pre-existing SortDefs map and OrderBy's string formatting), and both arms
measure 0 B/op, 0 allocs/op. A throwaway A/B in one binary, -benchtime=3s
-count=3 on the priority key, put the by-value median at 26.33 ns/op and the
pointer median at 25.35 ns/op, with non-overlapping ranges -- a consistent but
small ~4% win.

Stating that plainly rather than claiming more: the value here is removing four
struct copies per comparison from the hot path, not the wall-clock number,
which is within a couple of nanoseconds. Behavior is unchanged; the sqlbuild
suite, including the parity and LessID-honors-sortDesc tests, passes.

* test(dolt): vary SortBy across the summary parity table

Review point (minor 4): none of the nine parity cases set SortBy, so
LessSummary-versus-SQL ORDER BY agreement was proven by unit test alone
(TestLessSummaryMatchesLessAcrossSortKeys) and never against a real database.
Every case ran at the default order, priority ASC.

Nine rows added, covering created, title, status, updated and assignee -- the
first two in both directions -- plus id in both directions. Each carries a
Limit, so the merge's sort-before-trim actually decides which rows survive
rather than the ordering being cosmetic: an ordering bug that only shows up in
which rows a top-N keeps is the failure mode this table exists to catch.

id is included deliberately even though sqlbuild.OrderBy emits no ORDER BY for
it. It is the one Go-side key, so these two rows pin that both paths still
agree on the order the merge comparator produces where SQL contributes none.

All nine pass against a real Dolt database.

* docs(storage): document the summary projection's caller contract

Two review points, both about the contract being legible to a caller who has
only the interface and the extension guide.

Minor 1 -- IncludeDependencies is a silent no-op on the summary path, because
types.IssueSummary has no Dependencies field to populate. hydrateSummaryLabels
says why in its own comment; the interface a caller actually reads did not.
SearchIssueSummaries' doc now states it, alongside the neighbouring facts that
SkipLabels IS honored identically and that wisps are not excluded.

Minor 5 -- engdocs/EXTENDING.md documented the Lite projection's contract and
never mentioned the third one. It now carries a parallel section: what a caller
may read, why there is no IsLitePartial equivalent (a narrower type makes a
missing field a compile error rather than a silent zero value, which is the
substantive difference from Lite), the dependency and wisp rules above, the
ordering guarantee, and where each is enforced -- including the note that
IssueSummaryColumns is deliberately not an ordered subsequence of the full
list, so it has no analogue of the lite path's in-order guard.

The Lite section is left intact, its own Backend coverage subsection included;
the new section follows it rather than interleaving.

* chore(bazel): enumerate the two new test files in their go_test srcs

The PR adds internal/types/issue_summary_test.go and
internal/storage/issueops/search_summary_merge_test.go. Both are untagged,
and both packages carry a base-added BUILD.bazel whose go_test enumerates
srcs explicitly, so bazel silently would not have compiled or run them:
measured on //internal/types:types_test, the unrepaired binary lists 0 of
the 5 new tests (68 total) and the repaired one lists 5 of 5 (73), with
both builds exiting 0 -- a fake green, since bazel reports "1 test passes"
either way.

Base parity at 43a526b828 is clean (102 packages / 1192 untagged test files
enumerated, 0 srcs gaps), so the omission is PR-introduced rather than an
unenforced list. make bazel-sync-check and tools/bazel/go_srcs.py --check
both exit 0 on the broken head; generated with make bazel-sync (gazelle),
which is idempotent here.

Maintainer fixup applied during adopt-pr rebase (PR #3458).

* docs: announce the required SearchIssueSummaries method and inventory its span

Review follow-through for #3458 (maintainer fixup, iteration 1).

CHANGELOG.md: backend/backend.go promises that a required method added to
the engine interface is called out in CHANGELOG.md. SearchIssueSummaries is
one and had no entry. Adds an [Unreleased] "Changed" bullet in the #4911
form, next to the #6731 GetReadyWorkWithCountsAndTotal announcement: the
method and its signature, that callers are unaffected while implementers
must add it to compile, and its contract (SortBy/SortDesc and SkipLabels as
in SearchIssues, IncludeDependencies a silent no-op, wisps merged with their
markers set).

engdocs/design/otel/otel-data-model.md: the direct-operations block calls
itself a complete inventory and was exact at the PR base. The PR's new
storage.SearchIssueSummaries span was the one missing name. Inserted in
alphabetical order and bumped the count to 74. A set-diff against the
s.op(ctx, "...") names in internal/telemetry/storage.go now finds 74 == 74
with no drift in either direction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 01:19:24 -07:00
Alex
1764637093
perf(ready): recurse the --parent descendant walk off indexed base tables (#6130)
* perf(ready): recurse the descendant walk off indexed base tables (be-qfm)

`bd ready --parent` and `bd blocked --parent` compute the parent's
transitive descendants with a recursive CTE whose edge relation was a
non-recursive `parent_edges` CTE over `dependencies UNION ALL
wisp_dependencies` with the typed target columns COALESCE'd into one.
Dolt cannot index through that wrapper: it materializes the relation and
re-scans it once per recursion row, so the walk costs (parent-child rows)
x (descendants) — 7.46 s for 483 descendants on a 4.6k-edge database,
about 14 ms per descendant, and past the shared pool's read deadline on
a busy server.

Build the walk in sqlbuild.DescendantWalkQuery instead: one anchor and
one recursive member per (dependency table x typed target column), each
joining the base table on a bare indexed column, with per-column IS NULL
guards reproducing the COALESCE precedence and the LOCATE cycle guard,
depth guard and parent-child filter carried onto every member. The
recursive members carry JOIN_ORDER(d,e) LOOKUP_JOIN(d,e): the cycle guard
references both join sides, and without the hints Dolt's planner keeps
the recursive working table on the inner side and scans the edge table
per iteration, which measured no faster than the wrapper shape and
flipped plans between runs. With them the frontier is the outer side and
every member is an index probe.

issueops.GetDescendantIDsInTx and domain/db.getDescendantIDs call the
builder; they had carried the same query text separately.

Measured on a local Dolt 2.2.4 sql-server with a generated forest of
4,541 issues, 4,500 parent-child rows and 500 descendants under the
probed parent (BenchmarkDescendantWalk, new): 1.26–2.28 s/op on main,
32–36 ms/op with this change. Same rows on every target kind, cycle
and depth limit (TestDescendantWalkReachesEveryTargetKindAndStopsOnCycles,
new; the conformance ReadyParentScopesToItsTransitiveDescendants cases on
all three backends).

Refs #6128

Co-authored-by: qcore/syl <q@withq.ai>
Agent-Signature: claude-code-claude-fable-5-max on behalf of a3ackerman

* test(embeddeddolt): gate the descendant-walk reference test on cgo (be-qfm)

The embedded-Dolt test tier is cgo-only and every sibling file carries
the build tag; this one did not, so the CGO_ENABLED=0 boundary job
compiled it against a package whose helpers were excluded and failed on
an undefined newTestEnv.

Refs #6128

Agent-Signature: claude-code-claude-fable-5-max on behalf of a3ackerman

* fix(review): sync BUILD.bazel srcs and resolve every dep target column in the descendants CTE

Maintainer fixup from the PR #6130 adoption review, iterations 1 and 2.

Bazel srcs gap (major, claude+codex). The four new .go files were absent from
the gazelle-managed go_library/go_test srcs lists, so
//internal/storage/sqlbuild did not contain DescendantWalkQuery and both
consumers failed to compile under Bazel:

    internal/storage/issueops/blocked.go:170:13: undefined: sqlbuild.DescendantWalkQuery
    internal/storage/issueops/blocked.go:171:13: undefined: sqlbuild.DescendantWalkArgs

`make bazel-sync` adds exactly the four srcs rows the review measured; the
fifth BUILD.bazel row is a deps entry this commit's own new test requires.
A second sync leaves the tree clean. Note that `make bazel-sync-check` and
tools/bazel/go_srcs.py --check pass falsely on a srcs gap -- they manage only
the go_srcs glob filegroups -- and bazel-autofix cannot absorb this one
because bazel.yml:313 guards the sync step on steps.bazel.outcome ==
'success', which a compile break never reaches.

buildDescendantsCTE target columns (major, codex). GetDescendants' walk
resolved a row's parent through COALESCE(depends_on_issue_id,
depends_on_wisp_id), omitting depends_on_external, while the classic ParentID
filter it mirrors resolves all three and the reference test this PR adds makes
a cross-prefix parent a pinned contract for the sibling walk. ck_dep_one_target
(migration 0041) leaves exactly one target column set, so an external-target
row coalesced to NULL and could never match an edge: adding the column is
strictly additive, and it closes a gap where the proxied-server
bd list --parent tree (and --watch) could not see a cross-prefix child or
its subtree, while the direct route's per-level ParentID filter listed them.
TestBuildDescendantsCTEResolvesEveryDepTargetColumn derives the expected
column set from sqlbuild.DepTargetExpr, so the two cannot drift again.

That finding's other half -- migrating these members to the per-(table,
column) shape -- is deliberately deferred to bd-tztam and recorded at the
site. buildDescendantsCTE is a different query ((id, src, via) projection, a
dotted-ID fallback recursion, hoisted per-level filter predicates, no depth
cap, no path cycle guard), sqlbuild exports no member-fragment API, and
per-column members would triple its branch count, which the predBundle
comment above it records as tripping the dolt 2.1.6 analyzer.

nullGuards precedence and column binding (minor, claude+codex; minor, codex).
TestDescendantWalkQueryReproducesDepTargetPrecedence pins the per-column
IS NULL guards, which no existing test could see: the shape test never asserts
on a guard and the reference test seeds exactly one target column per edge, so
emptying nullGuards left both new test files green. The expected counts are
derived from DepTargetExpr's column list rather than restated, so the same
test binds descendantWalkTargetCols to it. The review's prescribed
`DepTargetExpr = "COALESCE(" + strings.Join(descendantWalkTargetCols, ", ")
+ ")"` does not compile -- issueops/dependencies.go:58 and
domain/db/dependency.go:35,:37 consume it in const declarations -- so this
takes the unit-assertion arm the same finding offers.

LOOKUP_JOIN warning multiplicity (nit, codex). The doc comment now records
that the ignored-hint warning is raised once per hint comment, six per
includeWisps=true query.

Iteration 2 (two minors, gate impact none, on a 981/850 approve).
TestGetDescendantsCrossPrefixParent pins the widened COALESCE against real
rows, where the test above pins only query text. Its edges go through the
repository's own classifier, which routes any cross-prefix target to
depends_on_external even when both ends are local, and each of its four
edges is matched by a different edge member (issues and wisps, anchor and
recursive): reverting the column at any one member reds the case with a
different set of children missing. CHANGELOG.md gains a Fixed bullet for
the user-visible half of this change, the proxied-server bd list --parent
tree.

Validation: sqlbuild 31 PASS / 0 FAIL / 0 SKIP; full domain/db package green
against the real Dolt container (915 PASS / 0 FAIL / 0 SKIP, 49s),
including TestGetDescendantsCrossPrefixParent,
TestGetDescendantsDottedOrphans, TestGetDescendantsFilteredByStatus and
TestParityReadyBlockedDescendants; golangci-lint 2.10.1 reports 0 issues on
the three changed packages; go test -run Bazel ./scripts/ is 13 PASS / 0 FAIL,
including TestBazelTestTagsFollowTaxonomy; bazel test
//internal/storage/sqlbuild:sqlbuild_test PASSED and bazel build
//internal/storage/domain/db:db_test succeeds; a final make bazel-sync
leaves the tree unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: qcore/syl <q@withq.ai>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 20:22:43 -07:00
Julian Knutsen
f66375459f
test(bazel): runfiles helpers + declared inputs for scan-guard tests (#6828)
* test(bazel): add internal/testutil/bazeltest runfiles helpers

Repository-path resolution for tests that works identically under plain
go test (package cwd, full checkout), bazel test locally, and remote
execution (declared runfiles only):

- IsBazel, RepoRoot (BEADS_TEST_REPO_ROOT, then the runfiles workspace
  root, then a go.mod walk), OverrideRoot (drop-in condition for existing
  helpers; empty under plain go test), CallerDir
- Runfile for $(rlocationpath) values, GOROOT for type-checking tests
- ShardFreeEnv for helpers that re-exec a test binary

Every helper reduces to the pre-Bazel behavior under go test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): declare source/data reads for 14 scan-guard targets

Source-scanning and repo-file tests read files their Bazel targets never
declared, so they failed under `bazel test` (and could not run on remote
workers at all). Declare narrow inputs as `data` and resolve paths through
bazeltest where the test located the tree via runtime.Caller:

- tools/bazel/go_srcs.py (run by `make bazel-sync` after gazelle) keeps a
  managed `go_srcs` filegroup in the packages whose Go source tests scan,
  plus `//internal/storage:tree_go_srcs` for the storage-tree walk. Bazel
  globs stop at package boundaries, so the aggregate needs one filegroup
  per package; the script keeps it in sync and gazelle-stable.
- Hand-written narrow inputs: //internal/storage/schema:migrations,
  //:docsync_files, //plugins/beads:plugin_files, and exports of
  .gitattributes, .claude-plugin/marketplace.json, cmd/bd list.go and
  ready.go, testutil/testdoltcommon.go, scripts README.md and the release
  catalog, each visible only to its consumer.
- runtime.Caller-based roots in backend, backend/conformance,
  internal/storage, internal/formula and test/docsync now go through
  bazeltest.CallerDir / OverrideRoot; relative-path tests need data only.
- integrations/ leaves .bazelignore (it is still gazelle-excluded) so the
  docsync link check can declare integrations/beads-mcp/PYPI.md.

Under --config=prcore every target reports the same PASS/SKIP/FAIL test
set as `go test -race -short` for its package, locally and remotely.
Plain `go test` behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): guard generated go_srcs and harden bazeltest root override

- tools/bazel/go_srcs.py: add --check (print diff, exit 1 on stale blocks);
  new `make bazel-sync-check` target runs it.
- scripts/bazel_policy_test.go: under go test, assert every tree_go_srcs
  (//internal/storage) lists exactly the BUILD.bazel packages below its
  root, run go_srcs.py --check, and forbid BUILD files under docs/ and
  engdocs/ (they would cut subtrees out of //:docsync_files).
- bazeltest: honor BEADS_TEST_REPO_ROOT only when it holds go.mod
  (RepoRoot fails loudly otherwise); drop the unused GOROOT() helper,
  whose compiled-in branch could never fire under rules_go.
- scripts/ci/lib/test-env.sh: unset BEADS_TEST_REPO_ROOT.
- Document the hand-listed out-of-tree link targets in //:docsync_files.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): re-sync BUILD files with main

Regenerated with make bazel-sync after commits that landed while the
scaffold PR was in CI (new internal/execenv, gitenv, gitignore packages
and new sources/deps).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): declare init_safety_help.go for docsync

TestInitSafetyCLIHelpCoversReCloneGotchas (#6431) reads
cmd/bd/init_safety_help.go; export it to //test/docsync only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 08:24:08 -07:00
Julian Knutsen
57ef364501
build(bazel): side-by-side Bazel scaffold + policy tests (#6798)
* build(bazel): add side-by-side Bazel scaffold (module + gazelle)

Add a Bzlmod Bazel build alongside the Go toolchain. `go build` and
`go test` are unchanged; nothing in CI invokes Bazel yet.

- MODULE.bazel: rules_go 0.63.0, gazelle 0.53.0, Go SDK 1.26.7 (matches
  the go.mod toolchain directive), go_deps from go.mod.
- ICU policy (engdocs/ICU-POLICY.md): a go-mysql-server gazelle_override
  selects the pure-Go regex backend and excludes regex_cgo.go, and
  .bazelrc builds with --@rules_go//go/config:tags=gms_pure_go, so no
  target depends on go-icu-regex or links libicu.
- third_party/patches: three module_override patches for dolt deps that
  ship WORKSPACE-era BUILD files, adapted from the gascity project.
- .bazelrc: prcore/ci test configs (inert for now; they match pr-core's
  test selection, not yet its environment isolation) and a generic
  remote-exec config with a bounded remote timeout. Executor endpoints and
  TLS credentials come only from a gitignored .bazelrc.local/user.bazelrc
  or a CI rc outside the workspace.
- .bazelversion pins Bazel 9.2.0; .bazelignore skips separate modules,
  non-Go trees, nested agent worktrees (.claude, .worktrees) and local
  data/output dirs (.beads, .dolt, dist, .runtime, tmp), so a primary
  checkout neither builds nor gazelle-rewrites nested worktree copies.
- Gazelle-generated BUILD.bazel files for every Go package.
- Makefile: `make bazel-sync` (gazelle + bazel mod tidy).
- .gitignore: bazel-* output symlinks and machine-local rc files.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): policy tests for the Bazel configuration

Add scripts/bazel_policy_test.go, plain Go tests that run under both
`go test ./scripts` and `bazel test //scripts:scripts_test`:

- MODULE.bazel go_sdk.download version equals the go.mod toolchain.
- ICU policy: .bazelrc sets gms_pure_go for build, the go-mysql-server
  gazelle_override excludes regex_cgo.go, and the root BUILD.bazel
  carries `# gazelle:build_tags gms_pure_go`.
- .bazelversion pins an exact Bazel release.
- .bazelrc.local and user.bazelrc are gitignored (parsed from .gitignore
  and cross-checked with git check-ignore when in a checkout).
- No remote-execution endpoints or credentials in committed bazelrc-like
  files, Markdown docs or .github/**: remote/BES/TLS flags
  (--remote_executor, --remote_cache, --bes_backend, --remote_header,
  --tls_client_certificate, ...) with a literal value, with or without a
  URL scheme, and non-loopback gRPC URLs in rc/.github files. Placeholders,
  $VARIABLES, and loopback are allowed; Markdown gRPC URLs (OTel exporter
  docs) are not flagged. Skipped outside a git checkout, e.g. in the Bazel
  sandbox.

Each check is a pure function also exercised against fixtures that break
the invariant. The files the tests read are declared as data via
//:bazel_policy_files so they resolve from runfiles under Bazel.

scripts/check-build-tags.sh now treats Bazel invocations as governed by
.bazelrc: the shell command segment that invokes Bazel (split on &&, ||,
;, |, $( and backticks) is exempt from the per-line `go` tag check, while
a bare `go` in another segment of the same line is still flagged. When
any scanned file invokes Bazel (or .bazelrc exists) .bazelrc must set
--@rules_go//go/config:tags=gms_pure_go for build.
scripts/check_build_tags_test.go covers both in a temp repository.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(scripts): split bazel lines in pure bash so check-build-tags works with BSD sed

BSD sed (macOS) does not expand \n in s/// replacements, so the segment
split never happened and the Bazel-exemption tests failed on macos-latest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(scripts): skip check-build-tags tests when bash lacks mapfile

macOS runners use /bin/bash 3.2, which has no mapfile; check-build-tags.sh
has always required bash >= 4, so the new tests are skipped there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 03:41:02 -07:00
Vishnu J
66aa73bb99
fix(storage): include custom active-category statuses in bd ready (#5831) (#5918)
bd ready pins WorkFilter.Status to StatusOpen. BuildReadyWorkWhere treated
that pin as an exact match, so issues in a custom status with category
active never appeared — even though bd statuses --help, the ready_issues
view (migration 0025), and ClaimableSourceStatusesInTx all treat that
category as ready-eligible.

Widen the singular StatusOpen clause to the same predicate the view uses:

  status = ? OR status IN (SELECT name FROM custom_statuses WHERE category = 'active')

Non-open singular statuses stay exact. Built-in open still matches. Custom
wip/done/frozen/unspecified statuses stay out, matching claim's GH-3570
anti-steal set.

Reported by Justin Clemens (@justin-candor).

Addresses #5831
2026-09-02 18:11:45 -07:00
Denis Redozubov
5f56ab2a53
fix(deps): enforce external capability blockers across ready/claim/close (#4769) (#4753)
External (cross-project) dependency blockers are now enforced everywhere a bead's readiness or closability is decided: `bd ready` / `--claim`, `bd blocked`, `bd close` / done-status `bd update` (direct and proxied), `bd dep tree`, and served HTTP ready/claim. The policy is a storage-boundary decorator (`internal/storage/externaldeps`) applied on both the DoltStorage chain and the UOW provider, with full capability-forwarding parity, `Unwrap` at every level, and the backend's advisory defer-wake preserved on the claim path.

Review trail: maphew rounds 1–4 (Jul–Aug), bee rounds 1–4 (Sep 2): D1 recording-parity exemption, D2 ready-filter goldens, D3 UOW-level Unwrap for the proxied importer, D4 claim-path defer wake. Verified at head 3468d62f on a clean merge with current main; CI fully green at that head.

Fixes #4769.
2026-09-02 16:56:23 -07:00
Julian Knutsen
8aaefe2d4f
feat(httpapi): serve bd list's order with sort on listIssues (#5666)
* feat(httpapi): serve `bd list`'s order with `sort` on listIssues

The issue listing served one order, `(created_at DESC, id ASC)`, because the
cursor is a keyset position in it. The spec stated the constraint in its own
words: "the sort order is welded to the cursor contract, and a new order needs
new surface." That welding put the cost on every client wanting `bd list`'s
ordering — the only way to get it was to page the whole result set and sort
locally, which is the one cost on this surface that grows with the project.

Measured against a real `bd serve` over 1400 rows at the 200-row page an HTTP
client actually uses: walk-and-sort is 7 requests plus a client-side
comparator, `?sort=priority` is 1. Both are checked against `bd list --json`
row for row, because the claim that matters is not "fewer requests" but "the
same order".

`sort` takes `created` and `priority` and nothing else. Each value is a keyset
contract — a position shape, a strictly-after predicate, an index — not a
display preference, and both keys are provably total (priority and created_at
are non-null, id is unique) so a page boundary inside a run of equal keys
resolves on id. The seven other orders `bd list --sort` takes have no such key:
`id` is natural-numeric and no database expresses it, `updated` moves on every
write, `closed` is nullable, and status/title/type/assignee are mutable and
unindexed. `priority` here is (priority ASC, created_at DESC, id ASC), which is
both `bd list`'s flagless order and what `--sort priority` produces — a stable
priority sort over the created order is the same sequence — so one served order
retires the walk for both.

Absent `sort` still means `created`, permanently: it is the compatibility
contract for every client written before the parameter existed.

THE CURSOR NOW CARRIES ITS ORDER. This is the half that makes the parameter
safe rather than merely useful, and the obvious implementation gets it wrong.
The token was base64 of `{t,i}` with no binding, so a created-order cursor
replayed under `sort=priority` decoded perfectly and was read as a position in
a different total order — a page that skips and duplicates rows, served with a
200, undetectable by the client. Tokens are now `v2` with an explicit order
member, and decode takes the wanted order as a parameter so a mismatch is
unrepresentable at the seam rather than merely detectable past it. Outstanding
`v1` tokens stay readable as the created-order positions they are, so no
traversal in flight has to restart. This does not contradict "the token carries
no filters": filters select the set, the order decides what the position means.

Storage grows one field. IssueFilter.AfterPriority extends the existing
position; the predicate nests the created one rather than restating it, because
the created/id comparison is only reachable at the cursor's own priority.
Flattening it the way the created predicate's leading-bound trick invites looks
equivalent and is not — it re-delivers rows at the cursor's priority created
after it, which the new conformance walk catches against real Dolt.

NO INDEX MIGRATION, by measurement rather than by omission. A composite
(priority, created_at, id) index was tried on Dolt 2.2.0 against the production
predicate, in both the plain ascending and the mixed-direction spelling Dolt's
DDL accepts: the plan is byte-identical in all three cases, keeping the
single-column priority seek and the TopN. It would have bought write
amplification on every insert and every priority update for no measured change.
The sargability guard pins what is achieved — an IndexedTableAccess range on
idx_issues_priority — and records the measurement to redo if Dolt learns to
stream the composite.

Agent-Signature: claude-opus-5
Co-authored-by: CI Bot <ci@beads.test>

* fix(changelog): drop leftover diff3 conflict marker in sort entry

The rebased adopt-PR head left a stray '|||||||' merge-base marker on the first line of the new `sort` CHANGELOG entry (CHANGELOG.md:381), which `git diff --check` flagged as a leftover conflict marker. Remove only the marker line; the resolved entry content is unchanged. Maintainer fixup for PR #5666 adoption review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: CI Bot <ci@beads.test>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-27 09:21:12 -07:00
Bee
8301974ecd
fix(sweep): purge/prune select candidates by tier — a typed wisp is ephemeral however it was minted (#5995)
* fix(sweep): select purge/prune candidates by tier, not the raw ephemeral flag

A typed wisp minted without the ephemeral flag was invisible to bd purge:
the candidate filter matched ephemeral=1 alone, and the Ephemeral=true
search route only looks at the wisps plane — so legacy typed wisps in the
issues table (858 rows / 7.3 MB in one production DB) survived every sweep
that existed to delete them, while still reporting 'No closed ephemeral
beads to purge'.

IssueFilter gains EphemeralTier: ephemeral-tier is ephemeral=1 OR a
non-empty wisp_type; the durable tier is the complement, so bd prune stops
silently owning rows purge should. The field deliberately does not take
the wisps-plane-only fast path — a tier query merges both planes, because
the rows that motivated it live in the issues table. NoHistory beads
(wisps plane, ephemeral=0, no wisp_type) stay durable-tier (GH#3649).

The list/ready golden filters gain the new field as null.

* fix(create): a wisp_type implies the ephemeral tier at mint

All three create paths (ExecuteCreate, the batch body, and the dolt
store's own createIssue routing) now infer ephemeral from a non-empty
wisp_type, the same way they already infer it from a configured infra
type — so no new mints produce the flag-less typed-wisp shape the sweep
fix has to catch. NoHistory keeps its own retention mode, as before.

* test(dolt): pin the tier boundary on the legacy typed-wisp shape

Three cases: a manufactured issues-plane row with wisp_type but no
ephemeral flag is taken by the ephemeral sweep and left by the durable
one; a NoHistory bead stays durable-tier; and a create carrying only a
wisp_type lands in the wisps plane marked ephemeral. The first and third
were verified red against their halves of the fix reverted.

* test(sweep): conformance tier-boundary cases + the remaining mint sites

Adds two cases to the sweeper conformance kit, wired in all three backends:

- RunSweeperTreatsALegacyTypedWispAsEphemeralTier — the row shape that
  motivated the fix: a closed issues-plane row carrying wisp_type but not
  the ephemeral flag (858 such rows / 7.3 MB survived every purge in one
  production DB). Manufactured with a raw UPDATE through a new optional
  Exec fixture hook, because every current create path now infers the flag.
- RunSweeperLeavesNoHistoryBeadsToTheDurableTier — the other edge: a
  NoHistory bead (wisps plane, ephemeral=0, no wisp_type) is durable-tier
  (GH#3649) and must survive the widened ephemeral sweep. Seeded through
  CreateWisp because plane routing of NoHistory is the create verb's
  concern and the uow kit's CreateIssue calls the domain-level create.

Also carries the wisp_type→ephemeral mint inference to the five remaining
create sites (uow issue_operations/batch_creator/batch_applier,
embeddeddolt create_issue) and slims the dolt-local typed-wisp test to the
mint case now that the sweep half lives in the conformance kit.

Verified: TestSweeperContract green on dolt (91s), uow (87s), and
embeddeddolt with BEADS_TEST_EMBEDDED_DOLT=1 (76s, 13/13 subtests, no
skips).

* test(sweep): dispatch the two new sweeper cases in roleContractCases

The conformance package's own coverage tripwire
(TestRoleContractCasesCoverEveryContractCase) failed in CI: a role contract
case that no roleContractCases row dispatches is invisible to an external
backend running RunRoleContracts. The legacy typed-wisp case still skips
cleanly on a bundle whose SweeperFixture carries no Exec hook.
2026-08-27 05:06:50 -07:00
Julian Knutsen
7e7dce8177
fix(storage): stop reporting a broken wisp plane as an empty one in search (re-land of #5805) (#5932)
* fix(storage): stop reporting a broken wisp plane as an empty one in search

A wisp search folded any table-not-exist error into "there are no wisps". The
tolerance is there for pre-migration databases, which have no wisp plane at
all, but the wisp query reads more tables than the wisp plane owns: its FROM
clause carries sqlbuild.LeaseJoin and its hydration reads wisp_labels. So a
database that has wisps but has lost wisp_labels is reported as having no wisps
-- with a nil error, from an ordinary list.

Reproduced end-to-end against embedded Dolt, with the row present in the same
transaction that is told it is not:

  RENAME TABLE wisp_labels TO wisp_labels_renamed_away
  SearchIssues(ephemeral)  before: 1 row   after: 0 rows, nil error
  SearchIssues(no filter)  before: 2 rows  after: 1 row,  nil error

GetIssueInTx already surfaces a missing wisp_labels as an error (pinned by the
existing embedded-Dolt wisp_label_table_error_propagates case), so the two read
paths disagreed today about whether that database is broken or empty.

Narrow the tolerance to the tables the wisp plane may legitimately not have.
sqlbuild.OptionalWispTable becomes the one place that set is written down --
issueops.optionalBlockedTable now reads it, so the classic and domain/db stacks
cannot drift on what a missing table means, which is the same reason the query
text itself lives in sqlbuild. The check is gated on the existing
IsTableNotExist classification via dberrors.MissingTableName, so nothing that
was previously an error becomes tolerated.

The leases half of the same over-broad check is included for the same reason
but is not, on its own, a reachable wrong answer today: the ephemeral branch
falls through to an issues-plane query over the same join, which re-raises the
missing table, so the caller does get an error. That was verified rather than
assumed, and the tests pin it at the guard rather than claiming a user-visible
repro.

Tests are red-first: sqlmock cases on both stacks and an end-to-end
RENAME TABLE against real embedded Dolt. Both directions have controls -- a
pre-migration database missing wisps must still search to an empty result, and
a missing wisp_dependencies must stay invisible -- because over-tightening to
"never tolerate" passes every must-error assertion and only those controls
catch it.

* test(storage): cover the union hydrate-wisps guard, which nothing reached

The review council mutated all four guarded call sites and found one that
no test in the repository reaches: the domain/db searchUnion hydrate-wisps
check. Reverting it to the old blanket table-not-exist test survived the
full domain tree and the nine-minute uow contract suite with embedded Dolt
enabled.

It is the site that matters most. The ephemeral branch needs an explicit
filter; the union is what a plain `bd list` runs. And the union is exactly
where the two stacks diverge: the leg's FROM clause is plan.FromSQL, with
no lease join and no label read, so a database missing leases or
wisp_labels answers the union query successfully and returns wisp ids --
the breakage surfaces only in fetchIssuesByIDs, which adds
sqlbuild.LeaseJoin and hydrates labels. Tolerating it there sends every
wisp through reassembleBySrc as a miss and drops it behind a nil error.

Also adds the fail-open control the domain/db file was missing. The
council's other surviving mutant made the guard return true for any error
it could not parse a table name out of; issueops caught it, domain/db was
green across the whole package.

Red before green (real edits, restored by trap):

  union guard -> blanket IsTableNotExist   TestSearchUnionBrokenWispPlane
                                           IsAnError, both subtests; every
                                           other test stays green
  fail open on unparseable errors          both UnrelatedErrorPropagates
                                           tests, ephemeral and union

The e2e wisp_dependencies control keeps its name but loses its overclaim:
a search that never asks for dependencies cannot see that table missing,
so it stays green whatever the tolerance set is. The sqlmock control is
what pins that entry; this one proves the query still answers end to end.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-21 16:58:43 -07:00
Robin Carswell
734701824a
feat(stale,blocked): add label filtering to bd stale and bd blocked (#5822)
* feat(stale,blocked): add label filtering to bd stale and bd blocked

bd stale and bd blocked were the last two issue-listing commands with no way
to scope output by label; bd list, bd ready, bd orphans, bd count and bd
search all have it. Add -l/--label (AND), --label-any (OR) and
--exclude-label (NOT) to both, reusing the existing help text so the flags
mean the same thing wherever a user meets them.

bd blocked was also silently discarding label filters it already accepted:
types.WorkFilter has carried Labels/LabelsAny/ExcludeLabels all along and
blockedCmd builds a WorkFilter, but GetBlockedIssuesInTx only ever read
ParentID, so a caller setting them got unfiltered results back and no error.

New sqlbuild.LabelSetClauses is the IN-subquery form of the three predicates,
for callers whose FROM clause is fixed and cannot take the JOIN that
BuildLabelDrivenSearch builds. A test pins it against the equivalent blocks in
BuildIssueFilterClauses so the two cannot drift: --label should mean the same
thing whichever command it is passed to.

On stale the predicates go in the WHERE clause rather than onto the hydrated
rows, because filter.Limit is this query's LIMIT and post-filtering would make
--label x --limit 10 return fewer than ten matching issues. On blocked they go
in the per-table scan, so issues filter against labels and wisps against
wisp_labels, which also spares the blocker-dependency and hydration passes any
work on rows that cannot survive.

Both the direct and the proxied-server path now build the blocked filter
through one blockedFilterFromFlags helper rather than assembling it twice.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(stale,blocked): normalize the new label filters, as every sibling does

Review feedback on #5822. The new flags built exact `label = ?` clauses from
raw pflag values, while every other label filter in bd normalizes first --
list_input.go:293-295, search.go:106, orphans.go:56, workapi/ready.go:56-58.

That gap is the read-side twin of #5812, and #5813 does not cure it:
normalize-on-write only protects labels a fixed bd wrote, whereas a filter
input still needs trimming wherever it comes from.

Two shapes, both verified on the embedded engine:

- `--label ' theme:a'` returned 0 rows while `bd list --label ' theme:a'`
  returned the issue. This is the everyday case, not an exotic one: pflag's CSV
  split leaves the space on the second value of `--label 'a, b'`.
- `--label 'a,,b'` ANDed in a `label = ''` clause, which nothing can satisfy,
  turning a valid filter into an empty result.

Both fail silently -- a filtered list short of rows reads exactly like a
complete one -- and both cut against the promise LabelSetClauses is documented
to keep, that --label means the same thing whichever command you reach for.

Normalization sits at the two shared choke points, so no route is missed:
blockedFilterFromFlags (direct + proxied blocked) and stale's RunE, which
builds the StaleFilter both its routes use.

Six new embedded subtests across bd stale and bd blocked pin trimming, the
empty-element case, and that both apply to --label-any and --exclude-label.
Verified failing with this commit's normalization reverted (6 fail, the nine
pre-existing label subtests still pass) and passing with it.

CHANGELOG: the flag addition, plus the WorkFilter silent-drop fix found en
route -- bd blocked accepted Labels/LabelsAny/ExcludeLabels and read only
ParentID, so programmatic callers got unfiltered results and no error.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-20 15:12:04 -07:00
Steve Yegge
96388ea019
fix(search): order Go-side-sort pages before every trim; honor sortDesc for id (#5639)
* fix(search): order Go-side-sort pages before every trim; honor sortDesc for id

Two live defects, both named-but-deferred in #5587's commit message and
both reachable from bd query '<expr>' --sort id (DefaultQueryLimit means a
bound is always set):

1. sqlbuild.Less ignored sortDesc for "id" — the one Go-side sort key —
   so every merge sort feeding a reversed id page kept the byte-FIRST
   rows. idSrcPage.sortGoSide's doc named this sibling bug explicitly.

2. #5587's fix (a page bound is only ever pushed under an order the query
   can express) covered the union seam only. The per-table seams
   (searchTable/scanFilterIDs, and the counts twin's
   finishSearchCountsPage) received the full matching set — searchWindowFor
   already withholds the page bound — but trimmed it with no sort anywhere:
   measured, 10 rows sort id desc limit 3 answered [000 001 002] where
   [009 008 007] was owed. The store-backed seam (issueops.searchTableInTxT)
   was worse off: it still pushed LIMIT into the leg SQL with no ORDER BY,
   so each leg contributed an arbitrary engine-ordered subset.

Fixes, one per seam, all converging on the same rule:

- sqlbuild.Less: sortDesc now flips the id comparison (byte order, matching
  the SQL the key would render if it could).
- domain/db: new sortRowsGoSide helper (the per-table copy of
  idSrcPage.sortGoSide) runs before finishWindow at searchTable's wide
  path, scanFilterIDs, and finishSearchCountsPage.
- issueops.searchTableInTxT: a Go-side sort leg no longer carries a SQL
  LIMIT; the leg scans the complete matching set, orders it in Go, and
  applies the same EffectiveSearchLimit bound after the order exists
  (goSideSortAndTrim), so cap accounting is unchanged and Pattern B
  hydrates only surviving rows.
- idProjection gains a comparator for the one key its payload can order
  ("id"), removing the merged-id-search landmine the old nil documented.

Tests: red-before-green verified on all three seams. sqlbuild unit test
pins Less; sqlmock tests pin the store-backed leg shape (no LIMIT under a
Go-side sort, bound still pushed under a SQL sort) and the ordered+trimmed
page; the domain/db suite adds per-table membership cases (asc, the
measured desc repro, and a four-route parity check: wisps-empty probe,
SkipWisps, wide path, counts twin must answer one page) plus a reversed
union page fence.

(bd-jao3t, bd-69c1a)

* fix(search): extend the Go-side-sort bound rule to the store-backed counts seam; one LessID comparator

Review round (adversarial pass on PR #5639):

F1 — runFilterSearchQueryInTx (issueops/search_counts.go) had the identical
defect the first commit fixed in searchTableInTxT: LIMIT eff pushed
unconditionally while sqlbuild.OrderBy renders no ORDER BY for "id". This
is the seam bd query '<expr>' --sort id actually reaches on the
store-shaped backends (storequerier → SearchIssuesWithCounts →
SearchIssuesWithCountsInTx), where BuildQueryPlan always pushes a bound —
so the first commit's repro claim held only for the uow/domain-db bodies.
Now the counts leg scans the complete matching set under a Go-side sort
and applies the same eff bound after ordering (goSideSortAndTrim),
leaving merge/trim/cap accounting exactly what the SQL LIMIT handed them.
bd list was already safe here (workapi.SQLLimit zeroes the pushed limit
for --sort id); only the querier path pushed a bound.

F2 — the byte-order id comparison existed in five hand-rolled copies
(Less, idProjection.less, goSideSortAndTrim, sortRowsGoSide,
idSrcPage.sortGoSide) — the exact duplication that let Less drift from
sortGoSide in the first place, and that hid F1's sixth site. All five now
call the one exported comparator, sqlbuild.LessID.

F3 — the ascending per-table suite case is annotated as the weak witness
it is (engine order for a PK-clustered prefix scan tends to be
byte-ascending already); the DESC cases are the red-before-green fences.

New shape test pins the counts legs: under --sort id neither the issues
nor the wisps counts query may carry a LIMIT (red on the previous commit:
the leg query carried LIMIT 3).

(bd-jao3t, bd-69c1a)
2026-08-11 00:24:28 -07:00
Stephanie Jarmak
7e825441b4
feat(workapi,storage): opt-in Brief projection for the list and ready pages (#5554)
STORAGE LAYER ONLY, per CONTRIBUTING_PR_GUIDELINES.md. issueops.ListRequest
and issueops.ReadyRequest gain Brief; no CLI flag and no HTTP parameter sets
it yet, so no front door's payload changes. The wiring stacks on top.

WHAT IT DROPS is the free-form text: description, design, acceptance_criteria,
notes, payload and waiters are not selected, and the row comes back carrying
types.Issue.IsLitePartial. #5546 measured a 16,051-issue store where
`bd ready --json` returned 454,021 bytes and notes plus description were 76% of
it, on the call an agent makes to decide what to work on. Measured through
issueops.Reader on 100 rows sized from those per-row ratios: 362,301 bytes
full, 24,001 brief, a 93.4% reduction.

THE PRIMITIVE WAS HALF-BUILT AND UNREACHABLE. #3906 landed
types.IssueFilter.Lite, IssueSelectColumnsLite, HeavyDropList,
ScanIssueLiteFrom and Issue.IsLitePartial, then wired filter.Lite into exactly
one place: SearchIssuesInTx, the UNCOUNTED store-backed search. Every read that
answers with IssueWithCounts, which is every --json route of both commands,
both implementations of issueops.Reader, and both HTTP reads, runs the counts
mega-query instead and ignored the field. A request that set it would have come
back correct, fully hydrated, and with nothing on it to say the projection had
not happened. That silent no-op is what this closes.

THE APPLICATION POINT IS THE COUNTS MEGA-QUERY, and it is one point rather than
four because sqlbuild.SearchCountsSQL and issueops.ScanReadyWorkRowWithCounts
are already the pair every counted read passes through: the store seam's list
(search_counts.go) and ready (ready_work_counts.go), and the unit-of-work
seam's list (issue_search_counts.go) and ready (ready_work_union.go, via
fetchCountsByIDs). sqlbuild.CountsHydration is already the value that carries
hydration opt-outs across that pair on both seams, so Lite is a third field on
it rather than a new channel.

IT CHANGES THE PROJECTION'S SHAPE, which SkipLabels and SkipCounts do not, and
that is why ScanReadyWorkRowWithCounts now takes the whole hydration instead of
a bool: the value that chose the SELECT list is the value that drives the scan,
so the two cannot be given different answers. Substituting empty strings in the
heavy positions would have kept one scan path and was rejected, because the row
would carry blank text with IsLitePartial=false, which is indistinguishable
from a genuinely textless row, the ambiguity ga-clgh and CommentsOmitted
already record.

WHERE IS INDEPENDENT OF SELECT, so this bounds the size of a row and never
which rows match: DescContains, NotesContains and EmptyDesc keep selecting
exactly what they select today, and no sort key, keyset cursor or wisp-merge
dedup reads a dropped column (CompareIssuesBy orders on nine fields, none of
them heavy).

THE CLAIM REFUSES IT, which is not the same call the page makes and is the
sharper case of the rule Limit and Offset already state. A claim does not read
its row through the page's query at all: ExecuteClaimNext refetches the winning
row whole with GetIssueInTx and hydrates its counts itself, so the projection
has nothing to apply to. Accepting the field would answer a MUTATING request
with a fully hydrated row carrying IsLitePartial=false, the caller's one signal
that it did not get what it asked for, reporting that it did. The check lives
in issueops.ValidateClaimNextRequest, which is the shared rule both ready
claimers and BatchCloser's claim already run, so all four backends refuse it on
the same line.

BuildReadyCountFilter CARRIES it, which is the opposite treatment and the
opposite reason. A count reads no field of any row, so the projection cannot
make the number wrong, and the count is not always cheap enough for that to be
the end of it: the unit-of-work seam has no COUNT(*) over the ready predicate
and sizes the set by running the unbounded page and taking its length
(uow/ready_counter.go), so dropping the field there would hydrate every heavy
column of the whole ready set to answer `bd ready --brief`, a larger read than
the page it was asked for. Carrying it also keeps that builder what its doc
says it is: the listing's filter with the PAGE removed and nothing else.

TWO ASYMMETRIES on the page path, both deliberate and both stated in code:

  - Brief IS carried onto the --ready arm (ReadyFilterFromIssueFilter) where
    SkipLabels and SkipCounts are dropped. Those two hide a number the ready
    renderings print; this drops a body no listing prints, so carrying it is
    what makes `bd list --ready` and `bd ready` answer one request one way.
  - types.WorkFilter carries Lite and neither of the other two, so the ready
    paths get readyHydrationFor rather than hydrationFor. Before this, both
    uow ready call sites passed a hardcoded CountsHydration{}, a literal that
    could not carry a filter's answer at all, which is how that path would have
    been missed.

CONFORMANCE at all four registered sites, one Test per case, for the two page
cases and for the claim refusal. Every field the cases assert stripped is
seeded heavy first and the premise is checked, so a backend that never
round-trips one of the six fails loudly instead of turning that clause into one
that cannot fail. Identity is asserted too (ID, Title, Status, IssueType,
Priority), or a body that stripped the row to its id would satisfy every
emptiness clause. IsLitePartial is asserted in both directions.

THE LITE COLUMN LIST IS PINNED BY ORDER, not only by membership. Both lists are
scanned positionally, so two columns of the same SQL type can be transposed
with no membership change at all: the pre-existing set-based parity test stays
green, the scan succeeds, and every row silently carries one column's value in
another's field. TestIssueSelectColumnsLite_IsFullMinusHeavyInOrder derives the
expected list by deleting HeavyDropList from IssueSelectColumns in place, which
makes the lite list a derivation rather than a second hand-maintained copy.

RED BEFORE GREEN, five sabotages, each verified and reverted:

  - deleting the lite branch from CountsHydration.IssueColumns fails both page
    cases with "expected 58 destination arguments in Scan, not 52";
  - accepting Brief without plumbing it to either filter, the silent no-op this
    change exists to close, fails both page cases on all six text fields and on
    IsLitePartial, which is the run that proves the assertions are
    load-bearing;
  - transposing created_by and owner in IssueSelectColumnsLite fails only the
    new order test, on IssueSelectColumnsLite[9] = owner, want created_by,
    while the set-based test stays green, which is the gap it was added for;
  - dropping the claim's refusal fails the claimer contract with "ClaimNext
    with brief: error = <nil>, want ErrValidation"; and
  - clearing Brief in BuildReadyCountFilter fails
    TestBuildReadyCountFilterIsTheUnboundedReadyFilter, which already existed to
    catch a ReadyRequest field the count builder forgets to carry and gains
    Brief in its fixture here.

The recorded ready-filter golden gains "Lite": false in 62 cases because
types.WorkFilter gained the field and the test compares marshaled JSON
byte-for-byte. False is what a request that did not ask for the projection
resolves that field to, so the replay stays exact; the test comment says so.

Gates: go build ./..., go vet ./..., gofmt, internal/storage/... (dolt,
embeddeddolt with BEADS_TEST_EMBEDDED_DOLT=1, uow, issueops, sqlbuild,
domain/db), internal/workapi/..., backend/conformance/..., issueops/...,
make api-check.

Refs: #5546, #3906

Co-authored-by: sjarmak <t@t.co>
2026-08-10 10:45:23 -07:00
CI Bot
67d38d2620 Trim this series' comments toward the repo's own density
Ten agents over 216 files, one directory-balanced group each, working to a
rubric that named what to cut (duplication narration, restatement of the next
line, the governing rule echoed in every command file, throat-clearing) and
what to keep (traps, invariants, security, tracker ids, behaviour changes, and
the "why a naive simplification destroys this assertion" rationale).

Net 1,523 lines. Density against this repo's own per-directory norm on main:

  cmd/bd              33.6% -> 27.3%   (main 10.7%)
  internal/storage    26.1% -> 23.1%   (main 14.1%)
  backend/conformance 34.0% -> 29.6%   (main 22.8%)
  internal/httpapi    30.9% -> 27.8%   (main 30.6%, now below)
  overall             34.7% -> 31.6%

Every agent landed at 18-25% rather than the 30% asked for, and every one gave
the same reason independently: what remained was on the keep list. Ten separate
judgements agreeing is the most useful result here — the prose is dense, but it
is mostly load-bearing, and the shortfall is the honest answer rather than a
failure to try.

TWO REVERTS, both to protect something worth more than the lines.

issueops/ is back to its pre-trim state, giving up 610 lines. The conformance
contracts cite the leaf docs BY LINE — "pins treewalker.go:52-66" — 155 times.
Trimming the cited files silently repointed those at whatever text moved into
those numbers, across agent boundaries so no agent could see it. A semantic
remapper recovered 141 of them, but 23 needed judgement and several landed on
struct fields. The leaf docs are the spec the contracts are written against;
they are the one place in this change where dense prose IS the deliverable, and
keeping them stable is worth more than the 610 lines. tree_walker_contract.go
and version_reconciler_contract.go went back for the same reason: an agent had
deleted a paragraph containing a citation, so positional restoration could not
align them.

After the reverts, exactly 5 citations point into a still-trimmed file and 2 of
those are weak — both in cmd/bd/config_guard_test.go, which this series never
touched and whose citations were already wrong on main (init.go:361 pointed at
a closing brace; the guard it names is at init.go:1499).

VERIFICATION. A guard strips every comment and all whitespace from each of the
193 modified files at HEAD and in the worktree and requires the two to be
byte-identical, so a code change cannot hide in a comment diff. It handles `//`
inside string and raw-string literals, and was tested against a deliberate code
change before being trusted. gofmt clean, go build, go vet, golangci-lint 0
issues, and the two AST-based tests that PARSE trimmed files — the timed
provider binding pin over claim.go and the serve database-source gate over
serve.go — both pass.

One real defect found while trimming: checkDatabaseSource's doc block ran into
`// sourceRoles is ...` with no blank line, so godoc attached thirty-one lines
of the wrong prose to func sourceRoles and left checkDatabaseSource
undocumented. Fixed by separating them.
2026-08-06 08:06:59 +00:00
CI Bot
aa10a725e5 feat(list): put bd list's row cap and counts hydration on ListRequest (bd-b34o7)
STAGE ONE OF TWO. Fields, not methods, and no route flips — those are stage 2
(bd-5k21g). `issueops.ListRequest` gains MaxRows, MaxRowsSource and SkipCounts,
each with a conformance contract running at all three backends and zero parks.

MAXROWS IS THE EXACT INVERSE OF OFFSET, which is why it needed no new shape.
Offset is honored by the unit-of-work body — it renders OFFSET — and refused by
the store-backed one with a typed *ErrUnsupported. The cap points the other way:
the store search path enforces it after the scan (EnforceMaxRowsCap), and the
unit-of-work query path (internal/storage/domain/db) reads no cap at all, so a
request carrying one would come back as the FULL uncapped answer with nothing on
it to say so — a caller that set a circuit breaker handed exactly the runaway
query it was guarding against. It refuses instead, naming the operation and the
backend. `bd list --max-rows --proxied-server` has always refused one layer up
in the CLI for that reason; this is the same refusal for the callers that are
not the CLI. RunReaderListMaxRowsIsHonoredOrRefused mirrors
RunReaderOffsetIsHonoredOrRefused, including the never-silently-ignored clause,
and asserts the complement too: a cap the result set FITS inside must still come
back as an ordinary page, or a body that refused every cap out of hand would
pass.

MAXROWSSOURCE is the third field and the bead named two. It is attribution
only — it decides no answer — but `handleMaxRowsError` falls back to
"BEADS_MAX_ROWS" when it is empty, so a request that carried the number without
the source would make stage 2's route flip start telling operators to raise the
wrong knob. The contract asserts it comes back on the cap error.

SKIPCOUNTS is the SkipLabels precedent one column over: the three cardinality
joins are dropped and DependencyCount/DependentCount/CommentCount come back
zero, to be read as unknown rather than as none. It has to land WITH MaxRows —
stage 2 moves `bd list`'s TEXT path onto the role, and that path calls
SearchIssues today and projects no counts at all, so arriving on a counted page
without a way to opt out is a straight perf regression. The expensive member is
the reverse-blocker aggregate, which joins on a COALESCE key the pure-Go
analyzer cannot auto-index — the known embedded sore point, and the reason the
knob is worth having at all.

The contract's second half is the load-bearing one: NOTHING ELSE MOVES. Same
rows, same order, same has-more, same Parent — asserted by running the identical
request both ways rather than against a written-out expectation. Parent is the
tripwire: it rides the same mega-query and is not a cardinality, so an
implementation that made an aggregate join inner, or suppressed the parent join
too, would answer a strict subset and still look like it had "skipped the
counts".

WHAT THE SQL DOES. sqlbuild.SearchCountsSQL's two trailing hydration bools
become one CountsHydration value, so a path cannot pick up one opt-out and drop
the other, and both stores plus the unit-of-work repository read the pair
through one helper. A suppressed aggregate is projected as a constant IN PLACE
rather than removed: the scan reads the six extra columns positionally, so the
shape is invariant. Ready work passes the zero value at every call site —
types.WorkFilter carries neither knob, which is what "SkipLabels and SkipCounts
are not carried onto the ReadyFlag arm" means, now stated where a caller reads
it. MaxRows IS carried there; reader_ready_scope.go already claimed so, and this
is the commit that makes the claim true, since until now the field existed only
on the filter.

ONE HAND-ROLL DELETED. `bd list` built the filter through the shared builder and
then reached in and stamped filter.MaxRows/MaxRowsSource onto it — the
"build it, then change it" half-step this programme exists to make unreachable,
and the reason the cap was invisible to every implementation of Reader.List. The
cap is resolved in gatherListInput onto the request, BuildListFilter is now the
only writer of those two filter fields, and the proxied refusal reads the
already-resolved value (rejectResolvedMaxRowsUnderProxiedServer) so a malformed
BEADS_MAX_ROWS still warns exactly once.

PROVABLY INERT AT THE CLI, measured rather than argued: 22 `bd list` / `bd
children` invocations — every output mode, both max-rows knobs, the malformed
and negative cases — run against a binary built from integration/facade-wave-2
and one built from this commit, over ONE shared workspace so both read identical
data, differ on exactly one line of 617. `bd list --max-rows -1 --status bogus`
now reports the malformed flag instead of the invalid status; both exit 1, it
needs two bad arguments to reach, and it is recorded as A-lst1-3.

NO OPENAPI OPERATION AND NO NEW QUERY PARAMETER, deliberately, recorded as
A-lst1-1. `bd list`'s operation (GET /v0/beads/issues) already exists and is
already on the role, and neither field can be published on it yet: `bd serve`
always builds a provider-backed server, which REFUSES max_rows, so the parameter
would be one the reference server can never honor; and skip_counts would set
three unconditional integers on a PINNED response component to zero, leaving a
client unable to tell "no dependencies" from "not computed" — the failure
comments_omitted exists to prevent on the detail view. `make api-check`
regenerates nothing.

Verified: build, vet, gofmt clean; golangci-lint 0 issues; the two new cases at
dolt, embeddeddolt (BEADS_TEST_EMBEDDED_DOLT=1) and uow, plus the whole Reader
contract at all three; the builder golden (35 recorded cases plus two new ones
for the new fields) and the counts-SQL shape test; internal/httpapi and
make api-check; internal/storage/{issueops,domain/...}, internal/workapi,
internal/types, internal/telemetry, the root package; the embedded
`bd list`/`bd children`/max-rows CLI suites, TestProxiedServerList under
BEADS_TEST_PROXIED_SERVER=1 (including reject_max_rows_flag,
reject_max_rows_env and allow_max_rows_zero), TestProxiedLookup*, and the
protocol corpus package.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 16:36:31 +00:00
Julian Knutsen
b92442d1a3
feat(storage): one guarded issue-lifecycle surface for the CLI and linked consumers (#5191)
Adds issueops.Lifecycle — Create, Update, Close, Reopen — reached from the storage handle via
store.IssueLifecycle(), implemented once and shared by the CLI and linked library consumers. Three backends
implement it and a shared conformance suite asserts them against one spec. The bd CLI's create, update,
close and reopen now route through it.

Fixes several defects that predate this change and became visible once the write paths were forced to
agree: swallowed metadata-edit failures, a claim applied before field validation, missing value validation
on title/priority/estimate, silently dropped create edges to missing targets, infra-typed creates landing in
the wrong table, a dropped SourceRepo, closed_by_session written but never read, and a compound update
writing twice.

Behavior changes needing release notes: create with an occupied explicit id now fails instead of silently
overwriting; a compound update is one atomic operation with one hook and one version commit; reparenting
replaces all parent edges; reopening something not closed reports nothing to do.
2026-07-31 11:26:38 -07:00
Steve Yegge
b0c2286bff
feat(storage): storage-class plumbing — schema marker, --storage-class flag, per-type config defaults (bd-8rifr) (#5149)
* feat(storage): storage_class marker column + types plumbing (bd-8rifr)

Protocol v0.1 §C (bd-7vb13): types.StorageClass (versioned|unversioned|
ephemeral, empty = unset), Issue.StorageClass with C1.2 defaulting via
EffectiveStorageClass, Validate coherence with the wisp-plane flags,
migration 0060 (nullable marker on issues+wisps, no row rewrite), and the
column threaded through IssueBaseColumns/Lite, both scans, and both
insert stacks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LxHUGXk944MDbtQ9xCNygJ

* feat(create,config): --storage-class flag + storage-class.<type> defaults (bd-8rifr)

Create-time resolution per C1.3: explicit flag > per-type config default >
unset; versioned normalizes to the omitted/NULL marker form (C2.4);
--storage-class ephemeral routes to the wisp plane as the spelled-out
--ephemeral (C1.4). storage-class.<type> is a yaml-plane key (dolt.auto-commit
precedent) validated at set time in both config set and set-many. Unit tests
plus an embedded end-to-end contract test (flag, config default, override,
at-rest NULL rule, wisp routing, set-time validation).

Docs: deferred to the cli-docs.pin bump per docs policy (docs describe the
pinned release; #5052/#5089 precedent) — flag help text regenerates into the
CLI reference then.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LxHUGXk944MDbtQ9xCNygJ

* fix(graph): per-node storage_class in graph plans (bd-8rifr)

TestGraphApplyNodeCoversCreateIssueParams caught the gap on CI:
createIssueParams grew StorageClass but graph plans could not address it.
Wire a per-node storage_class with single-issue create's exact C1.3
precedence (explicit node value > storage-class.<type> config > unset),
including the ephemeral spelling routing to the wisp plane with the same
mutual-exclusion errors, and extend the graph_full_fields round-trip to
pin the at-rest value.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(config): validate the storage-class.<type> key suffix at set time (bd-8rifr)

Lion's #5149 should-fix: the value was validated but the key suffix was
not, so an alias (storage-class.feat) or typo (storage-class.taks) passed
set-time validation and silently never matched — create-time lookup keys
on the Normalize()d type. Reject non-canonical suffixes with the canonical
key as the hint, and unknown types with a types.custom pointer. Also the
ephemeral-constant doc nit: class-ephemeral is typically TTL-bounded, not
definitionally (no-history rows are TTL-exempt).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 03:20:07 -07:00
John Zook
21822ad725
perf(search): filter the main table before the counts aggregate joins (#4480)
* perf(search): filter the main table before the counts aggregate joins (#5)

* perf(search): filter the main table before the counts aggregate joins

sqlbuild.SearchCountsSQL (the counts mega-query behind bd query/search --counts,
bd list --counts, and ready-work counts) joined six aggregate derived tables
(labels, dep/rdep/comment counts, parent, deps JSON) onto every main-table row
and only then applied the WHERE filter. For the narrow ephemeral-work searches
that dominate the hot path, Dolt materialized those aggregates across the full
tables before pruning -- the dominant cost for that access pattern.

Wrap the main table in an inner subquery that applies the WHERE filter first, so
the aggregate LEFT JOINs drive off the already-narrowed set:

    FROM ( SELECT i.* FROM <main> i <whereSQL> ) i
    LEFT JOIN ( ... ) ...

The WHERE clause references only issue columns, never the aggregates, and the
joins are all LEFT JOINs that preserve every main row, so filtering before vs.
after the joins yields the identical row set. ORDER BY and LIMIT deliberately
stay in their original position after the joins: the ready-work counts path
passes a parameterized ORDER BY, so it must render exactly once to keep the
placeholder/arg counts aligned, and callers that do not re-sort in Go (the
domain/db stack) depend on a post-join ORDER BY for the final row order.

Tests:
- TestSearchCountsFilterBeforeJoinParity (real Dolt): the new shape returns
  byte-identical rows (ids, order, dep/rdep/comment counts, parent, labels, deps
  JSON) to the prior filter-after-join shape across 11 filter/order/limit/skip
  scenarios. The old shape is kept in test code as the reference.
- TestSearchCountsSQLShape: asserts the WHERE sits inside the derived subquery
  (before the first LEFT JOIN) and that ORDER BY / LIMIT stay after the joins and
  appear exactly once.
- Full sqlbuild, issueops, dolt counts/search/ready, and domain/db suites green.

* test(search): enforce WHERE-never-references-aggregates invariant

Add TestWhereClausesNeverReferenceAggregates plus contract comments on
both WHERE producers (BuildIssueFilterClauses, BuildReadyWorkWhere) and
SearchCountsSQL, pinning the load-bearing invariant behind the
filter-before-join shape: WHERE may reference only issue columns, never
the six counts aggregate aliases (out of scope in the pre-join subquery).

Drop the stale ORDER BY/LIMIT doc paragraph (described unchanged
behavior; the arg-count invariant is already pinned by
TestSearchCountsSQLShape).

* test(search): drop the maximally-populated WHERE-aggregate test

The enumeration test only checked fields it happened to populate, so a
new filter field referencing an aggregate would pass unnoticed — false
confidence plus a standing sync cost. The invariant is already guarded
structurally: TestSearchCountsSQLShape pins the WHERE inside the pre-join
subquery, which is exactly what makes a violation an out-of-scope column
reference that Dolt errors on (fail-loud). Reword the builder/contract
comments to lean on that rather than the deleted test.

* fix(search): keep plain driver on empty WHERE; by-IDs hydration in domain/db

Review follow-ups for the filter-before-joins reshape:

- SearchCountsSQL: an empty whereSQL keeps the plain "FROM <main> i"
  driver instead of wrapping it in a no-op derived table, so the
  unfiltered list/counts path keeps its pre-existing shape and plans.
  Pinned by a new TestSearchCountsSQLShape assertion.

- domain/db fetchCountsByIDs: hydrate via the by-IDs form instead of
  hand-building "WHERE i.id IN (...)" for the predicate form - that
  clause now renders inside the derived subquery, silently coupling the
  caller to the subquery's internal alias. The by-IDs form also
  constrains every aggregate subquery to the page; IDs are chunked
  (queryBatchSize) to stay within placeholder limits, mirroring
  issueops.runReadyCountsInTx.

---------

Co-authored-by: Zook Bot <275398848+zook-bot@users.noreply.github.com>
2026-07-26 13:04:52 -07:00
Remus Cazacu
9208279712
feat(ready): WorkFilter.Statuses — multi-status ready work in one query (#4752)
Callers that need ready work across several statuses (e.g. Gas City's
native store Ready(), which spans open/blocked/deferred/pinned/hooked/
review/testing) currently issue one GetReadyWork per status. Each call
re-pays the deferred-parents pre-query, the wisps arm, and the
per-transaction round trips — measured at 30-70ms per call on a live
server-mode store, ~7x per Ready().

WorkFilter.Statuses (OR semantics) renders a single status IN (...)
clause through BuildReadyWorkWhere and propagates to the wisp-side
IssueFilter.Statuses, so multi-status callers pay the fixed per-call
work once. Singular Status keeps precedence; both empty keeps the
legacy ('open', 'in_progress') default.

Review round 2: a conformance audit case (ReadyMultiStatusFilter) pins
the contract at the result level against both backends — real issues
and wisps across several statuses, OR semantics in one call, singular-
Status precedence, the empty default, a custom status through the same
IN clause, and the ephemeral exclusion of wisps without
IncludeEphemeral. The library guide documents Statuses, and
BuildReadyWorkWhere computes the status clause and args in one pass.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-26 12:53:23 -07:00
Athos Bernardes
83a5858044
fix(storage): wire --label-pattern into the WHERE builder (#4882)
* fix(storage): wire --label-pattern/--label-regex into the WHERE builder (ga-hqchm)

IssueFilter.LabelPattern/LabelRegex and WorkFilter.LabelPattern/LabelRegex
were parsed from `bd list --label-pattern`/`--label-regex` (and threaded
through to `bd list --ready`) but never read by BuildIssueFilterClauses or
BuildReadyWorkWhere — the SQL query builder silently ignored both fields, so
a non-matching pattern returned every issue instead of an empty/filtered set.

Wire both fields into a label-subquery clause, mirroring the existing
Labels/LabelsAny/ExcludeLabels idiom: LabelPattern converts glob syntax (*, ?)
to a SQL LIKE pattern (escaping literal %, _, and \ in the input first);
LabelRegex passes through to SQL REGEXP directly.

Confirmed via direct binary execution, not just unit tests: built the fixed
bd, ran `bd list --label-pattern 'warrant*'` against a scratch workspace with
no matching label — now returns [] instead of 5 arbitrary unrelated issues,
matching the exact repro reported on ga-hqchm.

Also strengthens the existing `label_pattern` CLI test
(cmd/bd/list_embedded_test.go), which only asserted the matching issue was
present, never that a non-matching issue was absent — which is why it kept
passing despite the dead filter — and adds a sibling `label_regex` case.

Note: the standalone `bd ready` subcommand doesn't expose --label-pattern/
--label-regex as flags at all (separate gap from this bug; `bd list --ready`,
which shares WorkFilter/BuildReadyWorkWhere, works correctly with this fix).
Filed separately, not fixed here.

* fix(storage): drop LabelRegex/SQL REGEXP wiring, narrow to --label-pattern

engdocs/ICU-POLICY.md is explicit: "bd does not use SQL REGEXP functions,
so [excluding ICU] has zero practical impact" — that sentence is the stated
justification for shipping without libicu at all. The previous commit's
LabelRegex wiring used `WHERE label REGEXP ?`, which is real SQL REGEXP
usage and directly contradicts that invariant, even though it happens to
still work under gms_pure_go (RE2 via Go's regexp instead of ICU).

--label-regex needs a different implementation (Go-side regexp.MatchString
against fetched candidates, not a SQL REGEXP push-down) to stay consistent
with the policy, and ga-hqchm's reported bug only covers --label-pattern
(glob/LIKE) in the first place. Per CONTRIBUTING.md's "one issue per PR"
rule, splitting it out rather than doing the redesign inline here.

Rebuilt and re-verified end to end with the project's own canonical build
(`go build -tags gms_pure_go`, confirmed ICU-free via `otool -L`): both
`bd list --label-pattern` and `bd list --ready --label-pattern` now filter
correctly, matching-pattern issues included, non-matching issues excluded.

* fix(ready): wire --label-pattern and --label-regex into BuildReadyWorkWhere and bd ready

Upstream #3971 landed LabelPattern/LabelRegex for the list path; the
ready path still silently ignored both. Add the LIKE ESCAPE and REGEXP
subquery clauses to BuildReadyWorkWhere mirroring filter.go, and register
both flags on the standalone bd ready subcommand.

The bd ready flag registration absorbs the surviving scope of #4884 —
credit to athosmartins for both the pattern (this PR) and regex (#4884)
ready-path intent.

Addresses the 2026-07-23 review on gastownhall/beads#4882.

Agent-Signature: claude-sonnet-5-medium on behalf of maphew

* fix(ready): honor --label-pattern/--label-regex under the proxied path

Reviewer finding: flag reads happened after the proxied early-return and
gatherReadyInput never copied them into WorkFilter, so both filters were
silently ignored under --proxied-server. Hoist the reads, wire them
through readyInput, and pin with a gatherReadyInput unit test plus a
proxied integration subtest covering both filters.

Agent-Signature: codex-gpt-5.6-terra-medium on behalf of maphew

---------

Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-25 19:05:27 -07:00
Steve Yegge
966ddac5af feat(leases): replica-aware leases — record the granting node, refuse cross-replica reclaim (wy-jpd3.7)
A claim lease is only meaningful on the replica that granted it. Every other
replica's view of the holder's liveness is stale by up to one sync interval,
so a reaper over there can revert a unit that is very much alive here. Until
now the only defense was operator discipline: partition every supervisor's
reclaim by label and hope nobody runs an unscoped sweep.

Leases now record WHICH replica granted them (leases.granted_node, ignored
migration 0016, stamped from config.NodeID(): BEADS_NODE_ID / node_id config,
falling back to the hostname). bd reclaim skips a lease that positively names
another replica — on both the snapshot SELECT and the per-row DELETE
re-check — and names each skip on stderr, since silence there is
indistinguishable from "nothing to recover" while a genuinely dead remote
worker's unit sits in_progress forever. bd reclaim --any-replica is the
operator escape hatch for a replica that is gone for good.

The guard is deliberately fail-open on unknown provenance: granted_node ''
(a row predating the column, or a deployment whose NodeID is empty) is
treated as local, so the upgrade itself can never strand a stale lease the
reaper could previously recover. A heartbeat backfills an unknown row with
the local node and never overwrites a known one.

The one path that can materialize a FOREIGN lease row locally is
RestoreLeaseOnImportInTx — lease fields round-trip the JSONL interchange — so
the granting replica rides types.Issue.LeaseGrantedNode and a restored lease
keeps the replica the snapshot names, never this node. A snapshot from an
older binary carries none, restores as unknown, and behaves exactly as before.

What the guard cannot enforce is documented instead, in both bd reclaim --help
and docs/multi-agent/federation.md: grace window > sync interval, and lease
TTL > sync interval. A TTL shorter than the cadence at which replicas exchange
state is meaningless across the bridge — the remote view is a full interval
old by construction.

doc/beads-upstream-asks.md section 5.
2026-07-24 23:53:34 -07:00
Steve Yegge
c38582b7e6 feat(reclaim): scope filters --label/--label-any/--exclude-label/--assignee/--id (wy-jpd3.3)
bd reclaim was global-only: on a federated deployment one machine's reaper
can revert a stale-looking unit that is alive on the machine that granted its
lease. Add scope filters mirroring the claim-side label surface
(bd ready --claim) so a supervisor can point its reaper at exactly the
partition it claims from.

- types.ReclaimFilter (IDs/Assignees/Labels/LabelsAny/ExcludeLabels) + IsEmpty.
- sqlbuild.BuildReclaimScopeClauses/ReclaimScopeSQL: same SQL shapes the
  claim/ready path emits, alias-qualified for the reclaim JOIN. Fails closed
  (1=0) on a degenerate all-empty field so a scoped sweep can never silently
  widen to global.
- issueops.ReclaimExpiredLeasesInTx threads the filter into the snapshot SELECT
  only; the per-row revert re-checks by id from the already-scoped snapshot.
- Threaded ReclaimExpiredLeases through the Store/domain interfaces and every
  backend (dolt, embedded dolt, sql repo, use case) and both CLI paths
  (direct + proxied server). A zero ReclaimFilter reproduces the exact prior
  global behavior.
- CLI hard-errors a scope flag supplied with no usable value (e.g. --label
  "$LANE" with LANE unset) instead of degrading to a global sweep; --json now
  reports a "scoped" boolean.
- Cross-backend conformance coverage (testReclaimScoped) + sqlbuild and
  flag-parsing unit tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 05:31:23 -07:00
Jim Wordelman
be402a022a
fix(filters): Dolt label-filter regression fix — LabelPattern + LabelRegex + JSON truncation hint suppression (#3971)
* fix(list): suppress truncation hint in --json mode + lock-in label filter tests (be-acnquj)

Filter logic for --label/--label-any/--label-pattern/--title-contains was
already correct in tree (BuildIssueFilterClauses handles each case), so the
bug PM observed was a stale-binary artifact. To prevent regression, add a
unit test in TestListLabelFiltersAcnquj that exercises every acceptance
criterion:

  AC#1 -l X returns ONLY beads with label X
  AC#2 --label-any A,B (OR semantics)
  AC#3 --label-pattern glob
  AC#4 --title-contains case-insensitive
  AC#5 -l NONEXISTENT returns empty
  AC#6 AND/OR composition with multiple flags

For AC#7, gate printTruncationHint on jsonOutput so the trailing
"Showing N issues; more results matched..." line is suppressed in
--json mode. TestPrintTruncationHintJSONSuppression exercises both paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(filters): wire LabelPattern/LabelRegex through SQL builder (be-ucslk4)

Both fields had been on IssueFilter and set by the cobra layer for
`bd list`, but BuildIssueFilterClauses never consumed them — so
--label-pattern and --label-regex silently returned the full result
set instead of filtering. This was the live regression in be-ucslk4
(every agent's intake hook saw the entire open queue).

Fix:

- LabelPattern: convert glob (* / ?) to a SQL LIKE pattern, escape
  literal %/_/| with ESCAPE '|', and AND-join an `id IN (SELECT
  issue_id FROM <labels-table> WHERE label LIKE ? ESCAPE '|')`
  subquery — same shape as the existing single-label clause.
- LabelRegex: pass through verbatim to a `label REGEXP ?` subquery.
- Helper globToLikePattern is unit-tested for stars, question marks,
  literal wildcards, and the escape char itself.

Tests:

- filters_test.go: TestBuildIssueFilterClauses_LabelPattern (issues +
  wisps tables), TestBuildIssueFilterClauses_LabelRegex,
  TestGlobToLikePattern table test. These run without Dolt — they
  pin the SQL emission and arg shape so the regression cannot recur.
- list_embedded_test.go: strengthened the existing label_pattern
  subtest with negative-case assertions (feature/chore must NOT
  match `back*`). The prior assertion only checked positive
  inclusion, so it would pass even when the filter was a no-op.

Verified end-to-end on live city: --label-pattern, --label-regex,
--label, --label-any, --title-contains all return the correct counts
after the fix; trailing 'Showing N issues; more results matched...'
hint stays off stdout under --json.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: release gate PASS for be-ucslk4-dolt-ship (be-qd2vmt)

* docs(regression): mark BUG-28 label-pattern filter as fixed (PR #3971)

TestDiscovery_LabelPatternFilterDeadCode's assertions already described the
correct (filtered) behavior, so no assertion changes were needed — it now
passes against the candidate binary where it previously failed and
documented the dead-code bug. Update the test docstring and the DISCOVERY.md
BUG-28 entry/status table/index list to reflect the fix landing in this PR
rather than describing an open bug.

Agent-Signature: claude-sonnet-medium on behalf of matt wilkie

* fix(filters): dedupe globToLikePattern, pin test against live impl

TestGlobToLikePattern was pinning a byte-identical but unreachable copy of
globToLikePattern in internal/storage/issueops/filters.go — the production
caller is internal/storage/sqlbuild/filter.go:325 via BuildIssueFilterClauses.
An escaping regression in the sqlbuild copy (%, _, | handling) could have
shipped unseen since only the dead copy was tested.

Delete the issueops copy (and its now-unused "strings" import) and move
TestGlobToLikePattern into internal/storage/sqlbuild, in-package, so it
exercises the function BuildIssueFilterClauses actually calls.

Flagged by cross-vendor paired review (claude-opus + codex-gpt-5.6-sol) on
PR #3971.

Agent-Signature: claude-sonnet-medium on behalf of matt wilkie

* test(list): add store-level --label-regex coverage

The label REGEXP ? path was previously pinned only at the clause-string
level (TestBuildIssueFilterClauses_LabelRegex); no committed test exercised
it end-to-end against the embedded store, though a reviewer's ad-hoc probe
had confirmed it works. Add a label_regex subtest beside the existing
label_pattern case in TestEmbeddedList, following the same build-tag/env
gating (BEADS_TEST_EMBEDDED_DOLT=1), covering both a single-label regex and
an alternation across unrelated labels to confirm REGEXP semantics (not just
substring/glob matching).

Flagged by cross-vendor paired review (claude-opus + codex-gpt-5.6-sol) on
PR #3971.

Agent-Signature: claude-sonnet-medium on behalf of matt wilkie

* chore: drop contributor release-gate artifact from PR

release-gates/be-ucslk4-dolt-ship-gate.md is a contributor-process artifact
that doesn't belong in the upstream tree.

Flagged by cross-vendor paired review (claude-opus + codex-gpt-5.6-sol) on
PR #3971.

Agent-Signature: claude-sonnet-medium on behalf of matt wilkie

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-24 00:17:02 -07:00
Julian Knutsen
873e3040bf
feat(storage): keyset reads — comments page, issue-list cursor, IsBlocked filter (#4922)
* feat(comments): keyset-paged comments read (GetIssueCommentsPage)

The comments read surface was whole-thread only: GetIssueComments materializes
every comment, so a consumer that wants a page of a long thread must load it
all and page in memory (and cap it, serving long threads incomplete). Add a
keyset-paged read with a stable order and an index-backed resume predicate.

- Storage.GetIssueCommentsPage(ctx, issueID, after CommentPageCursor, limit):
  (created_at ASC, id ASC) order; resume via the sargable predicate
  created_at >= ? AND ((created_at > ?) OR (id > ?)); limit defaults to 100 and
  caps at 500 (documented, with the len(page)<limit idiom hazard called out —
  terminate on an empty page). Cursor comes from a previously read row; the
  forward-only-walk semantics for backdated inserts are documented. Wisp-routed
  like the whole-thread read. Page-walk equals the full read exactly, including
  same-second id tie-breaks (comment ids are lowercase UUIDv7, so the CHAR(36)
  collation cannot reorder them).
- Migration 0056 adds idx_comments_issue_created_id (issue_id, created_at, id)
  on comments and — table-guarded, since wisp tables are clone-local and main
  migrations run before the ignored chain materializes them (the 0035/0037/0053
  precedent, with a matching no-op pin test) — on wisp_comments. Fresh clones
  get the wisp index from the canonical clone-local schema (ignored/0001), the
  same two-path design earlier wisp indexes use. An EXPLAIN guard pins the
  composite index by its column-list signature, single-sourced from the exported
  production query.
- AddIssueCommentInTx now truncates the returned CreatedAt to seconds so the
  returned comment matches what the DATETIME(0) column stored and is usable as
  a page cursor (the stored value is unchanged).
- Implemented on DoltStore and EmbeddedDoltStore; InstrumentedStorage wraps it;
  beads re-exports the cursor type. Tests: page-walk==full-read (same-second
  groups), tie-break, first/past-end/empty/default-limit, wisp routing, the
  migration no-op pin, and the EXPLAIN guard — on both stores.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(storage): (created_at, id) keyset issue-list predicate on IssueFilter

`bd list` and the library's own callers over the public Storage surface need
true keyset paging over issues with second-granularity timestamps, which the
surface lacked: IssueFilter had CreatedBefore + Limit but no (created_at, id)
tuple cursor, so a same-second group larger than one page could not page without
dropping or duplicating its overflow.

Add an additive keyset to types.IssueFilter — AfterCreatedAt *time.Time +
AfterID string — meaning "rows strictly after the (created_at DESC, id ASC)
position (AfterCreatedAt, AfterID)". id is the primary key, so the tie-break is
total. It composes with every existing filter (CreatedBefore keeps working) and
is honored by BOTH SearchIssues and SearchIssuesWithCounts on every backend,
because it is emitted through the shared sqlbuild.BuildIssueFilterClauses path
that every storage backend goes through. Pair it with SortBy="created" for the
matching ORDER BY created_at DESC, id ASC.

The predicate is single-sourced as sqlbuild.KeysetCreatedAtIDPredicate:
(created_at <= ? AND ((created_at < ?) OR (id > ?))). This is the keyset
form the planner can drive through an index seek rather than a scan; a dolt
EXPLAIN guard pins the plan against the exact production constant, a sqlbuild
unit test pins clause/args emission, and same-second-overflow paging
regressions run on both the server and embedded Dolt backends through the
public SearchIssues.

(cherry picked from commit 167e25b3af3002939fe27d4bbfc1598fff21c586)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(storage): IssueFilter.IsBlocked filter — index-backed on every backend

`bd list`/`bd ready` and the library's own callers could not filter on the
denormalized is_blocked column server-side: the column and the
idx_issues_is_blocked(is_blocked, status) index already exist, but the
projection column was not a filter — callers had to over-fetch and post-filter
in Go. Make it one.

Purely additive: an optional IssueFilter.IsBlocked *bool (nil = any; no
signature change, no new interface method). Wired into the shared sqlbuild
filter clause as `is_blocked = ?` (bound int 0/1; the column is NOT NULL
DEFAULT 0 so no IS NULL arm), so BOTH SearchIssues(WithCounts) AND
CountIssues/CountIssuesByGroup honor it on every backend (dolt, embeddeddolt)
through the one BuildIssueFilterClauses path.

Tests: clause/arg unit test (true→1, false→0, nil→no clause); multi-backend
conformance — CountIssuesByGroup(IsBlocked=true) returns only blocked beads per
status, false the complement, nil the whole population; and a dolt EXPLAIN
guard that the filter drives the is_blocked index.

(cherry picked from commit 96c0b03024f0df08fcada9f312669f970500eea8)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(schema): move the wisp comments keyset index to a new ignored-chain migration

CI's migration-hygiene gate froze ignored/0001 (it exists on main; clones have
recorded its content hash), so the wisp twin of the 0056 comments keyset index
cannot be baked into the canonical wisp schema there. Add it as ignored/0014
instead — guarded exactly like ignored/0006 (table exists AND index absent) —
which is strictly better: the ignored chain runs per clone with its own
cursor, so fresh clones AND already-materialized workspaces both pick the
index up; 0056's guarded half remains for workspaces whose wisp tables
existed when the main chain ran. ignored/0001 reverts to its main content.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: CI Bot <ci@beads.test>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Test User <test@test.com>
2026-07-21 22:29:09 -07:00
Julian Knutsen
607df586d1
feat(engine): guarded write ops — atomic metadata merge, row-version CAS, typed dep errors + events, bd close delegation (#4911)
* feat(metadata): MergeMetadata — atomic single-key metadata merge

SlotSet did a read-modify-write across TWO transactions: GetIssue (tx 1),
metadata[key]=value in memory, then UpdateIssue (tx 2). Two concurrent SlotSet
calls — even on DIFFERENT keys — both read the same base metadata and each
wrote back its whole blob, so the second clobbered the first's key. SlotClear
had the same bug. Add MergeMetadata, which does the read-modify-write inside
ONE transaction (so a concurrent conflict is retried and re-reads), stores a
JSON value (nested objects/arrays, not just strings), and reimplement SlotSet
and SlotClear on top of it so they inherit the atomicity.

- issueops.MergeMetadataInTx / DeleteMetadataInTx: read the metadata (routed
  issues/wisps; missing issue -> ErrNotFound), merge/delete the one key, and
  write the whole object back THROUGH UpdateIssueInTx — so the operation keeps
  everything the old SlotSet got from UpdateIssue (the EventUpdated history
  event with actor attribution, the configured metadata-schema validation, and
  updated_at), now atomic. The read and write share the caller's transaction,
  so a concurrent merge of a different key is retried and re-read, never
  clobbered.
- Storage.MergeMetadata implemented on DoltStore (perm withRetryTx +
  DOLT_ADD(issues,events) + DOLT_COMMIT; wisp path no commit) and
  EmbeddedDoltStore (withConn); forwarded on InstrumentedStorage; auto-promoted
  on HookFiringStore; no-op on the configStore mock. SlotSet/SlotClear rebuilt
  on it (a string value marshals to a JSON string, byte-compatible with the
  historical rewrite).
- Cross-backend conformance covers MergeMetadata + atomic SlotClear; the
  8-goroutine concurrent-no-clobber test proves the B1 fix on real Dolt, and
  a test asserts the EventUpdated event and schema validation are preserved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(concurrency): expose a per-row version on Issue reads (RowVersion)

The engine's internal row_lock column — a random non-zero int64 the engine
rewrites on every status/ownership-mutating write and on the generic update
path — was not surfaced on the public types.Issue. Consumers that wanted a
fine-grained optimistic-concurrency token had only updated_at, which is stored
at second granularity, so two same-second writes were indistinguishable. Expose
row_lock read-only as types.Issue.RowVersion so a caller can tell same-second
writes apart.

- types.Issue gains RowVersion int64 (json:"-": a Go-only concurrency token,
  never on the CLI/export JSON wire — it is opaque and equality-only). Added
  row_lock to the canonical sqlbuild.IssueBaseColumns and a matching scan target
  in issueops.ScanIssueFrom; both engine write stacks, wisps, and the counts /
  dependents parallel scans share those, so the field hydrates everywhere from a
  single-point change. Read-only — no CAS/write behavior (that is a follow-up).
- Doc is precise about coverage: RowVersion changes on claim/close/unclaim and
  the generic update path; it does NOT change on some direct-UPDATE paths
  (restore, compaction text), which bump updated_at — so a complete change key
  combines RowVersion with updated_at, status, and labels. A created row is
  already non-zero (create stamps row_lock); 0 only appears on legacy rows
  backfilled by migration 0054's DEFAULT 0.
- Tests: RowVersion hydrates on read and on the list path, changes on a mutating
  write, distinguishes two same-second writes (identical updated_at, distinct
  RowVersion), and never appears on any JSON surface.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(deps): return a typed error on dependency type-conflict

Adding a dependency between a pair that already has an edge of a different type
is a deterministic rejection, but the two engine write paths disagreed on how
they surfaced it: one returned a typed *DependencyTypeConflictError, the other a
plain fmt.Errorf string. So a caller could errors.As the conflict on one path
but had to string-match the message on the other. Make both paths return the
same typed error, and re-export the two dependency-conflict types on the public
beads package so `bd` (and the library's own call sites) classify by errors.As
instead of parsing message text.

- issueops.AddDependencyInTx now returns *domain.DependencyTypeConflictError for
  the type-conflict case (byte-identical message — the struct's Error() is the
  exact former string, so no string-matcher changes). The hierarchy/cross-type
  case already returned *domain.DependencyHierarchyConflictError on both paths;
  self-dependency and cycle are already typed sentinels.
- beads.go re-exports DependencyTypeConflictError and
  DependencyHierarchyConflictError as type aliases, so errors.As against the
  public type matches the value the engine returns.
- Tests assert errors.As + the four fields + the byte-identical message on both
  write paths (permanent issues->issues and the wisp-source BeginTx/Commit seam),
  and lock the re-exports; the deterministic conflict is returned immediately
  (never retried as a transient serialization error).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(deps): emit dependency_added / dependency_removed events

`bd dep add` and `bd dep remove` recorded no event, so the events feed never
surfaced dependency changes even though the event types (types.EventDependencyAdded
/ EventDependencyRemoved) have existed since the original SQLite backend, which
emitted them on every add/remove. The emission was dropped when that backend was
removed for the Dolt-only migration and never ported to the current write paths.
Restore it on both Dolt write plumbings, matching the original shape (an events
row on the source issue: "Added dependency: <src> <type> <target>" /
"Removed dependency on <target>").

- issueops.AddDependencyInTx / RemoveDependencyInTx record the event via
  RecordEventInTable on the source's event table (wisp-routed), on the genuine
  add/remove only — the idempotent same-type re-add and the no-op remove of a
  missing edge record nothing.
- The event is committed with the edge: DoltStore.AddDependency/RemoveDependency
  DOLT_ADD `events`, and the transaction-path (RunInTransaction / batch /
  graph-apply) methods mark the source's event table dirty (events or wisp_events)
  so StageAndCommit commits it — without this the event row would dangle in the
  working set.
- Emission is gated to the explicit dep-add/remove verb on both plumbings: the
  proxied-server repo only records when the use-case sets DepInsertOpts.EmitEvent,
  which create-with-deps (implicit parent-child / --deps / waits-for, via a direct
  repo Insert) does not — so `bd create --parent` produces the same history on
  either backend (the embedded create path via PersistDependencies likewise emits
  nothing).
- Tests on both plumbings: add/remove emit exactly once and commit (verified via
  `events AS OF 'HEAD'`), idempotent re-add and no-op remove emit nothing, wisp
  sources route to wisp_events, and create-with-deps emits no dependency_added.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(close): optional ExpectedVersion CAS on CloseIssueChecked

CloseIssueChecked guards a close on is_blocked in-transaction, but a caller that
read an issue and then closes it has no way to say "only if it hasn't changed
since I read it" — a lost-update window. Add an optional compare-and-swap on the
row's version (the RowVersion / row_lock token): when the caller supplies an
ExpectedVersion, the close proceeds only if the row still has that version, else
it refuses with a typed ErrVersionMismatch. The version read and the close share
one transaction, so it is a true CAS with no read-then-write gap.

- storage.ErrVersionMismatch (re-exported as beads.ErrVersionMismatch) and
  CloseIssueOptions.ExpectedVersion *int64 (nil disables the check; a pointer so
  nil "no check" is distinct from "require version 0").
- issueops.CheckVersionInTx reads row_lock (wisp-routed; ErrNotFound on a missing
  row) and returns ErrVersionMismatch on divergence. CloseIssueCheckedInTx runs
  it FIRST — before the is_blocked guard and before the Force short-circuit, so
  Force bypasses only the guard, never the CAS (the version check is an
  orthogonal precondition). A mismatch returns before any write, so the tx rolls
  back leaving the issue open with no `closed` event (atomic refuse).
- The CAS has two limbs: the read-side check catches a writer that committed
  before the close began, and on the retry-wrapped permanent path a commit-time
  row_lock conflict is replayed by withRetryTx so the re-read refuses.
- Tests on both stores: match closes, stale refuses atomically (still open, zero
  closed events), a committed concurrent write invalidates a captured version,
  nil is unchanged behavior, Force does not bypass, missing id → ErrNotFound,
  wisp sources route to the wisps table, and an already-closed re-close with the
  post-close version stays idempotent (Unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(close): document RowVersion CAS coverage boundary on ExpectedVersion

Both adoption reviewers flagged that the public ExpectedVersion godoc and the
shared CloseIssueCheckedInTx doc could be misread as a full-row "unchanged"
check. RowVersion (row_lock) only tracks lifecycle/ownership writes
(status/assignee/started_at), so concurrent label, dependency, rename, or
is_blocked writes leave it untouched and are intentionally outside this CAS
boundary. Document that on both sites, grounded in the freshRowLock invariant.
Doc-only; no behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(cli): bd close delegates to the engine's guarded close

The embedded `bd close` path ran its own is_blocked pre-check and then closed in
a separate call — a read-then-write TOCTOU that duplicated a guard the engine
provides atomically. The verb now builds CloseIssueOptions and delegates to
CloseIssueChecked, which runs the guard and the close in ONE transaction; the
CLI's duplicated guard is deleted. --force maps to Options.Force; a blocked
close refuses with the blockers named and the --force hint.

Behavior-preserving on purpose (verified against the previous binary):

- The engine guard now uses the exact historical CLI predicate — refuse only on
  a LIVE direct blocker (blocked && len(blockers) > 0), not on the bare
  denormalized is_blocked column. A transitively-blocked child (is_blocked
  inherited from a blocked parent, zero direct blockers) still closes without
  --force, and a stale is_blocked with since-closed blockers self-heals, exactly
  as before.
- An already-closed issue stays an idempotent success: it still appears in the
  --json array and the text report (same output shape), still exits 0, but no
  longer produces a spurious closed->closed audit entry, a no-op commit, or the
  real-close side effects (molecule auto-close, newly-unblocked, claim-next).

Tests cover: direct blocker refuses atomically (still open) and closes with
--force; transitively-blocked and stale-is_blocked beads close without --force;
already-closed emits in --json alone and in a mixed batch; idempotent exit 0 —
on the engine (both stores) and through the bd binary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(update): UpdateIssueChecked — optional ExpectedVersion CAS on the update

UpdateIssue applies a field-map update with no optimistic-concurrency option, so
a caller that wants "update only if unchanged since I read it" had to
read-then-write — a lost-update window. Add UpdateIssueChecked: when the caller
supplies an ExpectedVersion (the RowVersion / row_lock token), the version read
and the update share ONE transaction, refusing with the typed ErrVersionMismatch
if the row has moved — a true compare-and-swap. Nil disables the check and is
byte-identical to UpdateIssue, which is left untouched on the hot path.

- storage.UpdateIssueOptions{ExpectedVersion *int64} (aliased as
  beads.UpdateIssueOptions) + Storage.UpdateIssueChecked, mirroring the
  CloseIssueChecked pattern: CheckVersionInTx runs first inside the same
  transaction on every route — the permanent withRetryTx path, the wisp path,
  AND the demote route (an update carrying no_history/wisp routes through the
  demote flow; a pure demoteToWispInTx extraction lets the check compose
  atomically with the row move — verified byte-identical to the old
  DemoteToWisp).
- Implemented on DoltStore and EmbeddedDoltStore; InstrumentedStorage wraps it;
  HookFiringStore fires on_update only on success (never on a refused update).
- Tests on both stores: match updates and bumps RowVersion; stale refuses
  atomically (field unchanged, zero updated events); a committed concurrent
  write invalidates a captured version; nil behaves exactly like UpdateIssue;
  wisp match/stale; demote-route match (migrates + applies) and stale (still in
  issues, unchanged); missing id → ErrNotFound; the hook decorator fires once on
  success and never on refusal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(cli): proxied bd close delegates to a library checked close

The proxied-server `bd close` path duplicated the close guard in the CLI:
it queried blockers via the dependency use-case, refused with its own message,
and then closed in a separate use-case call. Move the guard into the library —
IssueUseCase gains CloseIssueChecked / CloseWispChecked, which run the same
live-direct-blocker predicate (refuse only on blocked && len(blockers) > 0)
and then the untouched unchecked close — and thin the CLI verb to a single
delegated call. Both `bd close` paths now enforce the guard in the library and
emit the byte-identical refusal: the same storage.ErrCloseBlocked sentinel
("cannot close blocked issue: <id> is blocked by [...]") plus the
"(use --force to override)" hint.

- Guard and close run on the one pinned unit-of-work connection (the whole
  close batch is a single transaction), and the guard is read-only and refuses
  before any write — a refused id leaves the shared batch transaction clean,
  and an all-refused batch skips the commit entirely, exactly as before.
- The unchecked CloseIssue / CloseWisp are untouched: internal closes (molecule
  auto-close, gate closes, bd todo done) intentionally stay unguarded.
- Behavior-parity preserved: guard ordering (validate → epic-child → gate →
  blocked), exit codes, JSON/batch shape, already-closed handling, audit fields,
  and commit-message contents are unchanged; a transitively-blocked bead
  (is_blocked with no direct open blocker) still closes without --force, and
  --force still bypasses. The guard-failure diagnostic now uses the embedded
  path's generic wording (further convergence; nothing matched the old text).
- Tests: use-case level on real Dolt (direct blocker refuses with blockers
  named + still open, transitive-only closes, force closes, already-closed
  parity, wisp refuse/force) plus the proxied CLI integration suite.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(review): cross-backend dependency-event + RowVersion parity

Maintainer review fixups for PR #4911 (adopt-pr review loop):

- RowVersion: the proxied/domain create (insertIssueRow) and generic Update
  paths never stamped or rewrote row_lock, so an issue written through the
  proxied-server backend read back RowVersion 0 on create and unchanged on
  update -- a stale CAS token that violated the types.Issue.RowVersion
  contract on that backend. Stamp a fresh non-zero row_lock on insert (and the
  duplicate-key rewrite) and append RowLockClause() to the generic update,
  mirroring the classic issueops insert/update invariant. Adds domain/db
  regression tests proving CreateIssue/CreateWisp are non-zero and
  UpdateIssue/UpdateWisp change it.

- Dependency events: embedded issueops AddDependencyInTx/RemoveDependencyInTx
  recorded history unconditionally, so structural create-with-deps
  (bd create --parent/--deps/--waits-for) and reparent (bd update --parent)
  emitted dependency_added/removed on embedded but were silent on the proxied
  backend, diverging issue history for identical commands. Gate the embedded
  emit on EmitEvent and thread it through AddDependencyWithOptions /
  RemoveDependencyWithOptions on both the store and transaction interfaces:
  the plain AddDependency/RemoveDependency are the no-event structural default
  (create-with-deps, reparent), while the explicit dep verbs (bd dep add/remove,
  bd link, bd relate/unrelate) pass EmitEvent -- matching the proxied
  DepInsertOpts.EmitEvent gate. DoltStore stages the events table only when an
  event was actually written (GH#2455). Corrects the false
  "parity with PersistDependencies" comments and adds symmetric embedded/proxied
  dep-event coverage for create-with-parent/deps and structural removal.

- Retain iteration-1 fixups: heal molecules on already-closed re-close,
  idempotent proxied checked-close, and documented public Storage growth.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(storage): drop always-constant event param from fireDependencyHookByID (unparam)

golangci-lint (latest, v2.12.2 in CI) unparam flags fireDependencyHookByID:
its `event` parameter always receives hooks.EventUpdate. The parameter was
already effectively constant before this PR (both base call sites passed
EventUpdate); this PR adds two more call sites, all EventUpdate. Remove the
dead parameter and hardcode hooks.EventUpdate at the single runner.Run call.
All four call sites pass hooks.EventUpdate, so behavior is unchanged; go
build, go vet, and a scoped golangci-lint run pass clean.

Maintainer fixup during adoption finalize (PR #4911).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Test User <test@test.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <ci@beads.test>
2026-07-21 18:13:03 -07:00
Test User
0d7cf0022d Reapply "Simplify the supported backends: roll back the direct PostgreSQL and MySQL adapters" (#4857)
This reverts commit 0cf3b3ca09.
2026-07-18 02:43:56 +00:00
Steve Yegge
59f8d6ae97 feat(leases): move leases to ephemeral dolt_ignored leases table (bd-lrgn1)
Claims/heartbeats no longer mint dolt commits: lease state (holder,
granted_at, lease_expires_at, heartbeat_at) moves off the issues table
into a leases table registered in dolt_ignore. Migration 0055 +
ignored/0012 move live leases over; all full-issue query sites hydrate
via LEFT JOIN; import never clobbers a live local lease.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 01:02:29 -07:00
Julian Knutsen
0cf3b3ca09
Revert "Simplify the supported backends: roll back the direct PostgreSQL and MySQL adapters" (#4857) 2026-07-16 16:35:51 -07:00
Test User
cafb6888bd refactor(storage): remove direct PostgreSQL and MySQL adapters (bd-sadcd)
Roll back the direct adapters, dialect layers, credential handling, dependencies, CI services, and conformance profiles for general-purpose server databases. That support adds operational and testing complexity at odds with our goal of keeping Beads as simple as possible and consuming as few resources as possible.

Keep the Storage and DoltStorage interfaces unchanged. Embedded Dolt, Dolt server, and SQLite remain supported; the MySQL wire driver remains because Dolt server uses it.

Retain legacy metadata identifiers and hidden init flags only as compatibility tombstones. Removed, unknown, or corrupt backend metadata now fails safely across CLI, SDK, bootstrap, doctor, factories, and Dolt administration before storage is opened or modified, with guidance to export using a supporting build and reinitialize via bd help init-safety.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Test User
2026-07-16 03:35:18 +00:00
matt wilkie
fd03f7a9c9
Merge pull request #4095 from quad341/maintainer/pr-4065-ready-fifo
fix(ready): use FIFO ordering within same priority tier (maintainer cherry-pick from PR #4065)
2026-07-13 22:13:38 -07:00
matt wilkie
288e3128f0 Merge remote-tracking branch 'upstream/main' into pr-4347-fix 2026-07-13 20:39:48 -07:00
matt wilkie
fbdf7badbc Merge remote-tracking branch 'upstream/main' into pr-4095-fix 2026-07-13 20:39:27 -07:00
Steve Yegge
67ccdbfd44 fix(ready): honor --label-any on the ready/claim path (wy-jpd3.2)
BuildReadyWorkWhere emitted clauses for --label (AND-set) and
--exclude-label but none at all for --label-any, so the OR-set filter was
silently dropped by `bd ready` and `bd ready --claim` on every backend
(with or without --parent), while `bd list`/`bd search` honored it.

On an atomic claim the silent drop is dangerous rather than merely wrong:
a worker fencing itself to its own lane
(`bd ready --claim --label-any lane-a --parent epic-1`) would claim
another lane's issue and believe it was fenced. Reproduced on current
main: the new conformance case claims a lane-b issue when lane-a was
requested.

Emit an OR-set membership clause (same shape as the list path in
sqlbuild/filter.go) that AND-combines with Labels, ExcludeLabels and the
parent filter. All claim paths (Dolt, embedded-Dolt, sqlkit SQL family)
funnel through ClaimReadyIssueInTx -> GetReadyWorkInTx ->
BuildReadyWorkWhere, so the single seam fixes every backend.

Tests: a SQL-shape unit test in sqlbuild, plus a behavioral conformance
case (ClaimReadyIssueLabelFilters) asserting the claim is fenced by the
OR-set alone, AND-combined with --label and --parent, and — the safety
property — that an exhausted lane claims NOTHING rather than falling back
to unfenced ready work.
2026-07-12 07:58:22 -07:00
Test User
1fc38ba778 feat: choose your own storage backend (Postgres / MySQL / SQLite) — rebased on main
Rebase of PR #4601 (feat/choose-your-own-backend) onto current origin/main,
squashed to a single commit because the original 73-commit history built and then
removed the uowStore derisk spike incrementally, which does not replay cleanly
over the 125 commits main has since advanced (notably #4679, the ready-work
counts fix, which merged to main and is now inherited rather than re-applied).
The full pre-rebase history remains on the original branch tip for reference.

Contents (multi-backend feature + the six review follow-ups):
- bd init --backend=postgres|mysql|sqlite provisioning behind the Dolt-parity
  seam; the three SQL backends share *sqlkit.Store over pluggable dialects and
  carry a generated typed-unsupported shell for the non-core surface.
- Passwords in init DSNs are redacted before persisting to metadata.json (driver
  parser + re-parse verification), with a negative-test matrix proving no secret
  survives (pgdialect + mysql).
- The uowStore derisk spike and its report artifacts are removed; the shell
  generator now lives in internal/storage/unsupportedgen (neutral name).
- Compile-time guard `var _ storage.DoltStorage = (*Store)(nil)` on each backend
  plus a regen-idempotence test keep the generated skip lists honest.
- sqlkit exposes CountReadyWork so every SQL backend satisfies
  storage.ReadyWorkCounter and inherits the shared page-pushed ready-counts path.
- Commercial/product names scrubbed from the feature's own code for vendor
  neutrality.

Rebase reconciliations: adopted main's metadata-load hardening in the store
factories alongside the backend dispatch; combined the config interface
(GetMetadata + ReconcileVersion) and the conformance registrations (ReadyCounts +
Claim/lease); took main's versions of the shared-infra concurrency (uow
doltserver_tx) and delete-cascade (domain/issue_delete) evolutions; implemented
the interface's new UnclaimIssue on sqlkit (delegating to issueops) and added it
to the skip lists.

Verified: go build (cgo + CGO_ENABLED=0) and go vet clean; unit tests
(sqlbuild/issueops/pgdialect/mysql/unsupportedgen) and the SQLite conformance
suite green (RunAll, incl. ReadyCounts equivalence and Claim/lease on the real
sqlkit backend).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 17:44:37 +00:00
Julian Knutsen
fb527d034a
perf(ready): page-push ready-work counts and size the total with cheap COUNT(*)s (#4679)
`bd ready --json` had two O(N) hotspots on main. The counts mega-query's
reverse-blocker `rc` subquery keys on COALESCE(depends_on_issue_id, …), an
expression the pure-Go GMS analyzer cannot auto-index, so it re-scanned its
whole materialization once per candidate — O(candidates × blockers). And the
"Showing X of N" total re-ran that entire mega-query UNBOUNDED just to size N.

Changes:
- Generalize one counts builder: sqlbuild.SearchCountsSQL now takes an optional
  `ids []string`. Empty ids = the predicate form (unchanged callers); non-empty
  ids = the page-pushed form that constrains the driver and every count
  subquery to the page, bounding rc by the page instead of the full graph.
  Removes the drifted SearchCountsSQL/SearchCountsByIDsSQL duplication; the
  projection order is defined once and DepTargetExpr is used throughout.
- issueops.runReadyCountsInTx resolves the ready-ID page with the cheap indexed
  ID query, then hydrates counts constrained to those IDs, CHUNKED by
  sqlbuild.QueryBatchSize so large pages stay within backend placeholder limits
  (no quadratic fallback, no wasted ID pass). The page order is restored so the
  result is byte-identical to the unbounded ORDER BY … LIMIT.
- issueops.CountReadyWorkInTx sizes N with indexed COUNT(*)s over the ready
  predicate for the issues AND wisps families, minus the wisp-wins overlap —
  identical to len(GetReadyWorkWithCounts(Limit=0)) even when wisps exist, with
  no mega-query fallback.
- Add ReadyWorkCounter to internal/storage beside the other optional
  capabilities; DoltStore and EmbeddedDoltStore implement CountReadyWork.
  cmd/bd/ready.go zeroes the limit at the call site and unwraps to the
  capability via storage.UnwrapStore, falling back to the unbounded query only
  when a store predates it.
- Extract the shared scan/dedupe loop (scanCountsRowsInTx).
- Conformance tests: page counts byte-identical to unbounded, and
  CountReadyWork == len(unbounded), with fixtures for wisps and a page that
  spans the chunk boundary.

Co-authored-by: Test User <test@test.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 18:07:41 -07:00
matt wilkie
8ad064a961 Merge upstream/main into PR 4095
Resolve ready-work ordering against current main after upstream moved the shared ordering into sqlbuild. Priority ready work now breaks same-priority ties with created_at ASC, and regression coverage checks both the shared helper and ClaimReadyIssue.

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie
2026-07-08 12:59:23 -07:00
lumaks-redox
ee98454dcd feat(list): add --external-ref exact-match filter
Add --external-ref flag to `bd list` for exact-match lookup by external
reference value. Also adds --external-contains to list for parity with
bd search.

Use case: AI coding agent hooks that need to find the bead associated
with a specific GitHub branch or Jira ticket without fetching all beads
and filtering client-side.

Closes #4341

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-06 16:23:58 -06:00
Steve Yegge
e97839a2e1
feat(lease): land the leasing stack + schema v54 (claim-TTL, heartbeat, reclaim) (#4537)
* fix(dolt): retry mutate+commit methods on serialization conflict (1213/1205)

ClaimIssue, ClaimReadyIssue, UpdateIssue and CloseIssue each hand-rolled
their own tx (BeginTx + DOLT_ADD/DOLT_COMMIT + Commit) outside
withRetryTx, so a concurrent writer that lost Dolt's optimistic
commit-time merge surfaced MySQL 1213/1205 as a hard failure instead of
retrying. Dolt has no real row locking — FOR UPDATE / SKIP LOCKED are
parse-only no-ops (https://www.dolthub.com/blog/2023-10-23-hold-my-beer/)
— so retry is the only safety net under N concurrent workers. These four
are the work-queue hot paths: `bd ready --claim`, claim, update, close.

Move each body into a withRetryTx closure (which owns BeginTx and the
final Commit and retries serialization failures with backoff), preserving
the single-tx CAS/mutation + DOLT_COMMIT atomicity exactly. On retry,
ClaimReadyIssue re-scans the ready front from a fresh snapshot and claims
the next available issue.

Tested: TestEmbeddedReadyClaimConcurrent, TestEmbeddedReadyConcurrent,
TestEmbeddedCloseConcurrent, TestEmbeddedUpdateConcurrent and the
non-concurrent embedded Update/Close/Ready tests pass; build + vet clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(dolt): add concurrent work-queue drain test (Gas Station claim invariant)

N workers concurrently dequeue one shared ready-front via ClaimReadyIssue
until empty; asserts every issue claimed exactly once, none stranded, no
double-claims, and no serialization error ever surfaces (withRetryTx absorbs
the 1213/1205 from the claim-CAS cell collision). This is the regression guard
for the multi-agent claim queue that #4491 makes safe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(lease): claim-TTL + heartbeat + reclaim for dead-worker recovery

Gas Station v1.1 (wy-5r9j). A claim was previously permanent: a worker that
died mid-task stranded its issue in_progress forever. This adds a lease so a
claim can expire and be recovered.

Schema (migration 0054): add lease_expires_at, heartbeat_at (DATETIME) and
row_lock (BIGINT) to issues (and wisps, for uniform issueops routing), plus
idx_issues_lease for the reaper scan. assignee doubles as the lease owner.

Lifecycle (shared issueops, so both server-mode DoltStore and EmbeddedDoltStore
get it):
  - Claim stamps lease_expires_at = now + TTL (default 5m, WithLeaseTTL to
    override), heartbeat_at = now, and a fresh row_lock.
  - bd heartbeat <id> — owner-only; pushes the lease forward and rewrites
    row_lock. Fails (ErrNotClaimable/ErrAlreadyClaimed) once the lease is gone.
  - bd reclaim --older-than <dur> — reverts in_progress issues whose lease
    expired more than <dur> ago back to ready (clears assignee/started_at,
    records a lease_reclaimed event). Default grace 2×TTL.

The row_lock trick: Dolt has no row locking and merges concurrent commits
cell-by-cell, so a heartbeat (writes heartbeat_at) racing a reclaim (writes
status) would silently cell-merge into a zombie — an open/unassigned issue that
still carries the worker's fresh heartbeat. Every mutating path (claim, close,
update, heartbeat, reclaim) now rewrites the shared row_lock cell, forcing the
loser to a 1213/1205 serialization conflict that withRetryTx replays. All lease
paths are withRetryTx/withConn wrapped.

Tests (real Dolt): TestRowLockForcesConflictOnDisjointCellWrites proves the bug
without row_lock (silent merge to zombie) and correctness with it (conflict);
TestConcurrentHeartbeatReclaimClose is the integration race (live workers
heartbeat+close vs a continuous reaper vs dead workers) asserting exactly-once,
no lost close, no zombie; plus claim/heartbeat/reclaim unit coverage on both
backends.

Note: lease columns are second-granular DATETIME (Dolt rounds), so sub-second
TTLs are meaningless — production TTL is minutes. Heartbeat writes a Dolt
commit, so cadence must stay ≪ claim cadence to avoid history bloat.

Fixes the test coupling in TestMigration0053PromotesRigWisps, which assumed the
rig-wisp repair was always LatestVersion().

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci+docs: allowlist lease stress tests; scope the row_lock invariant comment

- Add TestConcurrentWorkQueueDrain and TestConcurrentHeartbeatReclaimClose
  to the check-testing-short.sh allowlist; both are real-Dolt runtime/stress
  skips and were failing ci-pr-policy.
- freshRowLock doc said "EVERY mutating path" rewrites row_lock, which
  overstates it. Scope it to status/ownership/lease-mutating paths (the set
  the reclaim/heartbeat races care about) and document the invariant + why
  orthogonal-cell writers (is_blocked, compaction, deps, rename, reopen) are
  intentionally exempt, so a future writer does not silently reintroduce the
  zombie-merge bug.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(show): surface lease_expires_at/heartbeat_at in bd show (--json + text) (wy-9cdw)

Migration 0054 added lease_expires_at/heartbeat_at (and internal row_lock) to
the issues/wisps tables for the claim/heartbeat/reclaim leasing feature, but
bd show could not observe lease state — you had to drop to raw SQL.

- types.Issue: add LeaseExpiresAt/HeartbeatAt (*time.Time, omitempty), matching
  the StartedAt nullable-timestamp convention. row_lock stays unexported.
- sqlbuild.IssueSelectColumns: append lease_expires_at, heartbeat_at to the
  canonical hydration list (flows to every stack via the shared constant).
- issueops.ScanIssueFrom: scan + map the two nullable columns (before the
  trailing extra dests, keeping positional parity across all scan sites).
- bd show text view: emit a muted 'Lease: expires <rel> (heartbeat <rel> ago)'
  line only for in_progress issues holding an active lease; add formatTimeUntil
  as the forward-looking mirror of formatTimeAgo.
- types_test: assert lease keys appear when set, are omitted when nil, and that
  row_lock is never surfaced.

* fix(lease): settle v54 review blockers

Address #4537 review blockers before landing v54: keep heartbeat off wisps in both backends, keep direct status/assignee updates from leaving stale leases, guard the down migration, and emit reclaim JSON as snake_case.

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie

* fix(lease): cover lease fields in tests and hooks

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-03 23:28:55 -07:00
silvanshade
4d3a552424
Fix rig bead durability (#4373)
* Fix rig bead durability (bds-vwz)

* fix: keep rig beads out of infra migration

* fix: repair durable rig wisps

Restore migration 0035 to its frozen upstream shape and add a forward repair migration for existing type=rig rows that were routed into wisps.

Keep type=rig durable while excluding it from ready work, and cover the repair path with CLI-backed schema tests plus Dolt-server tests where available.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of matt wilkie

* fix: make rig wisp repair migration upgrade-safe

Guard migration 0053 so older workspaces without local wisp tables no-op instead of failing, while keeping the fresh-Dolt CLI migration bundle able to exercise rig wisp repair fixtures.

Regenerate CLI reference docs for the updated infra type help text.

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie

---------

Co-authored-by: silvanshade <silvanshade@users.noreply.github.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
2026-06-22 17:06:35 -07:00
Steve Yegge
1f8f1fff31 refactor(storage): extract shared SQL builders into internal/storage/sqlbuild (bd-6dnrw.46)
Move all pure SQL-text builders shared by issueops (classic) and
domain/db (proxied) into one package; both stacks now delegate, so
ready/filter/counts query semantics can no longer drift (Seam A).
Execution orchestration stays per-stack per the extract-don't-unify
decision. Canonical SQL text is classic's; the only proxied-side text
deltas are IN-list whitespace.

Adversarial re-review (bd-6dnrw.47) verified all 10 attack items with
no production divergence, and adds an import-boundary test pinning the
layering (domain must never import issueops/sqlbuild) plus a single-use
constraint note on LabelSearchPlan.MergeInto. Follow-ups filed:
bd-emrs2 (+1-overfetch convention), bd-kzeme (IN-helper consolidation).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 15:56:23 -04:00