* feat(httpapi): wire-shape signal — wire_revision, floor header, golden digest, param tokens
S0 of the HTTP role-parity program: give a client a single place to learn
whether this build's wire shape has moved under it, and give CI a gate that
catches an undocumented move.
- GET /v0/beads/context reports wire_revision (bumps only on a non-additive
response-shape change) and min_client_wire_revision. An optional
Bd-Wire-Revision request header lets a client declare the revision it was
built for; checkWireRevision refuses a declared revision below the floor
with 400 invalid_argument/reason=wire_revision_unsupported and a
min_wire_revision field, on every route but /healthz (wireRevisionExempt) —
including the identity handshake, where catching the mismatch is cheapest.
Absent header is today's behavior; malformed value is invalid_value, never
wire_revision_unsupported. The floor lives in Server.minClientWireRevision
(defaulting to the MinClientWireRevision constant) so a test can raise it.
- internal/httpapi/wireshape: a golden digest over every response's every
content-type schema (including application/problem+json) across the whole
spec, keyed by schema+member. TestWireShapeDigest fails closed when a
member's JSON name or type changes without wire_revision bumping to match;
regenerate with `go run ./internal/httpapi/wireshape/cmd/gendigest`.
- GET /v0/beads/issues' `sort` parameter now carries the issues.listSort
capability token, the same convention `capabilities` entries use.
TestNewParameterOnExistingOperationHasABehaviorToken requires every new
parameter on an existing operation to carry one, checked against a
write-once grandfather baseline (testdata/pretoken_parameters.json) of
parameters that predate the rule; TestUntokenedParameterRuleFires proves
the checker itself against a synthetic fixture.
- Docs: engdocs/design/bd-serve-v0.md gets a "The wire-shape signal" section;
CHANGELOG.md gets an Added entry.
Live-mutated and reverted three RED demonstrations during development:
retyping api_version in the spec failed TestWireShapeDigest; short-circuiting
checkWireRevision's floor comparison failed
TestWireRevisionBelowFloorRefusesEveryEnforcedRoute; adding an untokened
query parameter to listIssues failed
TestNewParameterOnExistingOperationHasABehaviorToken. All three now pass
against the unmutated tree.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi): rename issues.listSort to issues.list.sort, harden token rule
Opus review of 12c5144ca (HIGH-1, MEDIUM-5):
- Rename the capability token issues.listSort -> issues.list.sort to match
bd-enterprise's client (httpstore/wire/handshake.go:121, DESIGN.txt S4e),
and widen capabilityToken to accept an optional third segment.
- untokenedParameters() now requires the extracted token to be one
Capabilities() actually serves, not just backtick-shaped text. This
catches real false positives that existed in the spec already
(workapi.DefaultReadyLimit, status.custom) which only shape-matched the
old regex; they're added to the pretoken baseline as genuinely
pre-existing untokened parameters.
- Extend the rule to request body members (new
testdata/pretoken_request_body_members.json baseline) and exempt
brand-new operations via a frozen testdata/pretoken_operations.json
snapshot.
- Add TestPretokenBaselinesNeverGrow pinning all three baseline sizes so
a future change can't hand-add an exemption instead of a token.
- Update spec prose for the widened rule scope.
go test ./internal/httpapi/... and the reviewer's pmut.py all pass as
expected: plain/bogus modes go RED, reuse (real-but-unrelated token) stays
green, matching the review's literal ask ("a token must be one advertised
by Capabilities()").
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(wireshape): close four digest-coverage gaps, add gendigest write guard
Opus review of 12c5144ca (HIGH-2, MEDIUM-4). The digest only tracked
Type/Format/Enum/Required on direct properties, missing:
- nullable: true (never read at all)
- array item shape (Issue.labels items string->integer was invisible;
only object-shaped items got recursed into)
- additionalProperties value shape (IssueCount.groups)
- request bodies entirely (Compute() only walked responses, so
DeleteIssuesRequest.expected_version string->integer, the other half
of #6053, was invisible)
Also added defensive oneOf flattening alongside the existing allOf merge
(dormant today -- grep confirms this spec uses oneOf nowhere -- but no
longer silently invisible if that changes).
Extracted the golden-vs-computed comparison into wireshape.Compare(),
shared by TestWireShapeDigest and the new wireshape.SafeToWrite(), which
gendigest now calls before writing: it refuses to overwrite golden.json
with changed/removed entries unless the candidate's wire_revision is
strictly greater than the existing golden's. Pure additions always write.
TestSafeToWrite exercises both guard directions.
golden.json is regenerated. This single regen is a one-time exception to
the new guard, done by writing the digest directly rather than through
gendigest: confirmed via a throwaway diff (not checked in) that every one
of the 57 "changed" entries differs only in the NEW fields the wider walk
now populates (ItemType/ItemFormat/Nullable/AdditionalProps*) -- every
pre-existing Type/Format/Required/Enum value is byte-identical. Nothing on
the wire changed; the measurement got more precise. Bumping
CurrentWireRevision for this would have written a false "the protocol
changed here" entry into the revision history table. Going forward, any
REAL change the guard now reports uses the normal bump-then-regenerate
path, same as before.
Verified against the reviewer's mut.py: nested_type, nullable,
nullable_issue, array_items, addlprops, request_body and format all go RED
on TestWireShapeDigest, each reverted after confirming.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi): make bd_version wire_revision inference advisory
Opus review of 12c5144ca (HIGH-3). The spec told a client inferring
wire_revision from bd_version to use the release cutoff >= 1.3.0, and said
nothing about what happens when that inference is wrong -- which it
demonstrably is for real builds already in the field:
- main builds up to commit b3ef65c85 report bd_version 1.2.2 (below any
1.3.0 cutoff) while already sending the string-typed revision/
expected_version shape (#6053).
- 1.3.0-rc.N pre-release builds sort BELOW the release cutoff 1.3.0 under
plain string comparison despite already carrying the string shape.
- The bd-enterprise compatibility line at d3ab32773462 reports bd_version
1.1.0 while also sending strings.
Moved the cutoff to >= 1.3.0-rc.1 (compared as semver-with-prerelease,
which at least captures the rc builds), documented both named exceptions
inline, and made the whole inference explicitly ADVISORY: a client must
accept revision/expected_version as either a string or a number whenever
wire_revision is absent (or probe directly) rather than let the inferred
value choose a decoder, and must never refuse solely because the inferred
revision is 0. Mirrored into engdocs/design/bd-serve-v0.md.
This is spec/doc prose only -- there is no server-side inference code to
change, since this server always sends wire_revision directly; the
inference is guidance for clients reading a pre-signal server's response
(this server, past builds, or an unrelated implementation).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi): LOW-7/8/9/10 review follow-ups for wire-revision signal
LOW-7: WireRevisionUnsupported now also sets Problem.wire_revision (this
server's CurrentWireRevision) and Problem.bd_version on the
wire_revision_unsupported refusal, alongside the existing
min_wire_revision, so a client logging the refusal has this server's
full self-description without a second /v0/beads/context call.
LOW-8: gazelle + make bazel-sync add the BUILD.bazel entries the prior
commits were missing: wire_revision.go/wire_revision_test.go in
internal/httpapi's srcs, new BUILD.bazel files for wireshape and
wireshape/cmd/gendigest, and a testdata glob on httpapi's go_test so
the pretoken_*.json baselines are visible under `bazel test`. Verified
with `bazel build //internal/httpapi/...` and
`bazel test //internal/httpapi/... --test_env=BEADS_TEST_SKIP=dolt`
(both green) and `python3 tools/bazel/go_srcs.py --check` (clean).
LOW-9: narrowed CHANGELOG.md and engdocs/design/bd-serve-v0.md to
describe the digest's actual final coverage (format/enum/required/
nullable, array items, additionalProperties, request bodies, $ref/
allOf/oneOf recursion, the SafeToWrite revision-bump guard) and the
token rule's actual final shape (three pinned baselines, request-body
members, Capabilities()-membership validation, new-operation
exemption) instead of the original, narrower claims.
LOW-10: Bd-Wire-Revision now parses via a strict decimal-only
parseStrictNonNegativeInt (no leading '+', no leading zero except a
bare "0") instead of strconv.Atoi alone, so two different header
bytes can no longer name the same revision. New malformed-header test
cases: "+1", "01", "007".
Regenerated apigen/types.gen.go (make api-gen) for the new Problem
fields and wireshape/testdata/golden.json (go run
.../wireshape/cmd/gendigest) for the two new additive Problem members;
the latter wrote cleanly through SafeToWrite itself, since both are
pure additions.
go test ./... and bazel test //internal/httpapi/... both green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi): re-review LOW follow-ups — retirement test, gendigest -init, docs
1. TestRetiredRevisionsStayBelowTheFloor pins the PRESENCE guarantee the
wire_revision property doc relies on: CurrentWireRevision >= 2 (so 0
and 1 stay permanently retired) and MinClientWireRevision <=
CurrentWireRevision (so this build's own floor never refuses every
request against itself).
2. gendigest now refuses to create a golden.json that does not exist yet
unless run with -init. Previously a missing file (moved testdata,
wrong path, accidental delete) was indistinguishable from a real first
run and silently wrote a fresh golden with no revision history checked
against anything. writeGolden() is extracted out of main() so this is
covered directly: TestWriteGoldenRequiresInitWhenMissing,
TestWriteGoldenInitCreatesWhenMissing,
TestWriteGoldenIgnoresInitWhenGoldenExists (the existing-golden path is
still governed only by SafeToWrite, -init or not).
3. Narrowed the two remaining stale claims that still described the
digest's original, narrower coverage: the wire_revision property's
prose in openapi.v0.yaml (regenerated into apigen/types.gen.go via
make api-gen) and wireshape/doc.go now both list nullable, array item
shape, additionalProperties value shape, request bodies, and
$ref/allOf/oneOf recursion, plus the new -init guard.
go test ./internal/httpapi/... and make api-check both green;
bazel test //internal/httpapi/... (httpapi, wireshape, and the new
gendigest test target) green; python3 tools/bazel/go_srcs.py --check
clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(wireshape): cover operation parameters in the wire-shape digest
The digest previously walked only response and request-body schemas, so a
type, enum, required, or removal change on one of the spec's 138 operation
parameters (123 query, 14 path, 1 header) passed the gate silently — the
open item from PR review on #7182.
Each parameter is now recorded as its own Entry, keyed by
"param:"+operationId / location+":"+name (operation-scoped, not
schema-scoped, since two operations' same-named parameter are independent
wire contracts), capturing type, array item shape, enum, required, style,
explode and default. Adding a parameter stays additive; removing,
retyping, narrowing its enum, or changing required/style/explode/default
now trips SafeToWrite exactly like a response or request-body change.
Verified against the live spec: retyping listIssues' `limit` query
parameter from integer to string now fails TestWireShapeDigest (confirmed
red, then reverted — not part of this commit). golden.json regenerated via
gendigest; the new parameter entries are a pure addition, no wire_revision
bump needed. Added TestParameterMutationsAreCaught (remove/retype/narrow-
enum against real golden entries) and updated wireshape's doc.go, the
`wire_revision` spec prose, and the CHANGELOG to describe the new
coverage.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(wireshape): review fixes — stale-golden and lowered-revision arms, effective param serialization, digest boundary
Maintainer fixup from the adopt-pr review loop (iterations 1 and 2), on
top of e17ba924f.
- goldenDrift (A, I): TestWireShapeDigest's comparison moves into a
goldenDrift helper. An addition alongside a moved wire_revision no
longer passes with a stale golden: it asks for a regenerate, as a
changed or removed entry after a bump already did. Every message now
names response/request-body members AND parameters, and TestGoldenDrift
pins each arm. This takes only the test half of the stranded fixup
(adopt-pr/bd-47kdge/stranded-fixup-528cd6095); its doc half, which says
parameters are outside the digest, is false since e17ba924f and is not
replayed (J).
- Lowered wire_revision (D): a CurrentWireRevision LOWER than the
golden's is a wrong constant, not a stale golden, so goldenDrift says to
restore it instead of regenerating, and SafeToWrite (so gendigest)
refuses any write at a lower revision, whatever the entries.
- Effective style/explode (B): walkParam records style and explode with
OpenAPI 3.0's defaults filled in (form for query/cookie, simple for
path/header; explode defaults to style == form). Adding `explode: false`
to a query array that relied on form's implicit explode is now a
changed entry, while spelling out a default changes nothing.
testdata/golden.json is regenerated with -init: 111 parameter entries
change only in style/explode (96 to form/true, 15 to simple), 828
entries before and after, wire_revision stays 2. The golden is new in
this PR and unreleased, so this needs no bump.
- TestWalkParam (H): a package-internal table test
(wireshape_internal_test.go) pins every field walkParam extracts,
through a computeFrom(document, revision) seam that Compute now wraps:
per-location defaults, $ref'd parameters and items, format, nullable and
default, content parameters, and the nil returns.
TestEffectiveSerializationIsCompared pins B end to end through Compare
and SafeToWrite. BUILD.bazel embeds the library in the go_test, which is
gazelle's own output.
- Required inputs and the digest boundary (C, E, F; docs only): all five
prose surfaces (openapi.v0.yaml's wire_revision description and the
regenerated apigen/types.gen.go, wireshape/doc.go, CHANGELOG.md,
engdocs/design/bd-serve-v0.md) now say that a new REQUIRED request
input is recorded as an ordinary addition and needs its own review,
that value constraints and member-level composition are outside the
digest, and that object and content parameters are recorded only as
their container. bd-serve-v0.md also gains the parameter coverage it
was missing.
- G: a brand-new operation is permanently exempt from the
capability-token rule; its own review decides its capability story.
- Iteration 2 (nit): walkParam's comment now says that the location
default it records as a `content` parameter's style and explode is a
placeholder, since OpenAPI 3.0 applies style and explode only to a
parameter described by `schema`. All 138 operation parameters in the
spec use `schema`, so the golden does not change.
Validated: go test ./internal/httpapi/... (gms_pure_go), make api-check,
go vet for linux/windows/darwin, gofmt, git diff --check, gazelle on the
package (no drift), make bazel-sync-check, and bazel test
//internal/httpapi/wireshape/... in the local processwrapper sandbox
(2 of 2 pass). Overlay mutation probes: dropping the added-after-bump
arm, the lowered-revision arm, SafeToWrite's lowered check, or the
effective-default fill each turns the new tests red.
Iteration 2 changes only that comment; go test
./internal/httpapi/wireshape/... (gms_pure_go), go vet, gofmt and git
diff --check pass again.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(count): add Counter scope fields (S8) — ParentID, NoParent, ExcludeTypes, ExcludeStatus
Extends issueops.CountRequest with ParentID, NoParent, ExcludeTypes and
ExcludeStatus, closing the gap where bd count could not express the
filters bd list already has, forcing downstream consumers like gc to
fall back to listing everything just to get a count.
- issueops.Counter: four new CountRequest fields, documented against the
List equivalents they mirror (ParentID/ExcludeTypes) or newly define
(NoParent, ExcludeStatus has no List counterpart). Status "all" is
pinned as equivalent to unset, matching BuildListFilter's own "all" and
preserving count/list cardinality parity (the role's reason to exist).
- internal/workapi.BuildCountFilter: maps the four fields onto the one
types.IssueFilter builder all three storage legs (dolt, embeddeddolt,
uow) already share via internal/storage/issueops and sqlbuild, so no
per-leg storage code was needed.
- cmd/bd: --parent, --no-parent, --exclude-type, --exclude-status flags.
- HTTP: adds parent/no_parent/exclude_type/exclude_status to GET
/v0/beads/issues:count, behind the new issues.count.scope behavior
token (routes.go), wired in reads.go's countFilters, documented in
openapi.v0.yaml with per-parameter tokens and an extended capabilities
description. gendigest confirms a byte-identical golden.json: query
parameters are outside the wire-shape digest's scope, so this is a
zero-diff, no-revision-bump change as expected.
- backend/conformance: four new Counter contract cases (ParentID scoping,
NoParent's edge-only semantics vs ParentID's edge-or-dotted-id match,
ExcludeTypes composing with include_infra's own exclusions, and
ExcludeStatus), run identically against all three storage legs via a
new CounterFixture.AddDependency hook shared with ReaderFixture.
Verified: go build ./...; go test ./internal/httpapi/... ./internal/workapi/...
./issueops/... and the Counter contract on all three legs (dolt via
testcontainers, embeddeddolt, uow via local dolt binary); make api-gen
(additive-only apigen diff) and make api-check; go vet and golangci-lint
clean; gofmt clean; gazelle diff empty (no new files/imports). Mutation
testing: dropping issues.count.scope from behaviorCapabilities turns
TestCapabilitiesAdvertiseEveryImplementedOperation,
TestSpecCapabilityVocabularyMatchesTheRouteTable and
TestNewParameterOnExistingOperationHasABehaviorToken red; no-op'ing
ExcludeStatus in BuildCountFilter turns
TestBuildCountFilterExcludeStatusTakesNamesAsWritten and the
embeddeddolt ExcludeStatusNarrowsThePredicate conformance case red. Both
reverted and reconfirmed green.
Deviation: DESIGN.txt's literal text would make count --status all an
ErrValidation; shipped behind a test and a doc comment instead, since
"all" already means "every status" on both count and list and rejecting
it would break the count/list parity this role exists to guarantee.
S3/S4 note: no OSS HTTP client exists yet to enforce capability-token
skew. A doc comment on CapIssuesCountScope in routes.go specifies the
required behavior once one lands: check issues.count.scope against the
cached handshake capabilities and refuse locally with a typed
ErrUnsupported{Capability} before sending any of the four new params to
an older server.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(count): S8 review follow-ups — refusal parity, exclude-status validation, doc fixes, parity tests
Addresses the Opus review's six required changes for S8 (bd count scope
fields):
1. BuildCountFilter now refuses ParentID+NoParent together as ErrValidation,
matching bd list's CLI refusal wording; wired through the role, the CLI,
and invalid_argument classification in internal/httpapi/reads.go.
2. Status:"all" is documented in DESIGN.txt as every status, matching List's
no-filter reading (count/list cardinality parity) — a coordinator decision
recorded in beads-design/DESIGN.txt, outside this worktree.
3. ExcludeStatus names are now validated against the workspace vocabulary
(built-in statuses plus workspace-custom ones); an unknown name is
ErrValidation instead of a silent no-op, since a typo here would overcount.
4. Fixed openapi.v0.yaml and issueops/reader.go doc claims about bd list
--parent's default tree-walk vs. the one-level role answer that bd count
--parent agrees with (confirmed by tracing list_show_filter_modes.go); fixed
an Oxford-comma nit in routes.go and the spec.
5. Added storage-level count == len(List) parity tests for ParentID, NoParent
and ExcludeTypes (List has no ExcludeStatus counterpart), plus wisp-child
and closed-child contract cases, across all three backends (dolt,
embeddeddolt, uow). CounterFixture gained a List hook for this.
6. Extended the routes.go skew note: gc's ErrCountUnsupported fallback to List
is now named explicitly, with a note that S3/S4 must add a skew test that
masks the token and asserts the fallback fires.
Also fixed RunCounterExcludeStatusNarrowsThePredicate's now-stale "unrecognized
status excludes nothing" assertion to expect ErrValidation instead.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(count): PR #7199 review follow-ups — exclude-status config load, ParentID doc swap
Addresses bee's REQUEST-CHANGES on gastownhall/beads#7199 (S8):
1. BLOCKER: storecounter.filter and uow.countFilter only loaded the
workspace list config when IncludeInfra was set, so BuildCountFilter's
ExcludeStatus validation refused a workspace's own custom status unless
the caller also passed the unrelated IncludeInfra flag. Both legs now
load the config when IncludeInfra OR len(ExcludeStatus) > 0, with
comments corrected to say so. Added leg-level regression tests (the
conformance fixture cannot drive a workspace-configured custom status)
on embeddeddolt and uow, reproducing the reviewer's CLI repro directly,
plus an HTTP-level probe in cmd/bd's real-Dolt proxied count test.
2. issueops.ReadyRequest.ParentID's comment had been overwritten with the
one-level direct-children wording that belongs to ListRequest.ParentID;
Ready actually walks all descendants via GetDescendantIDsInTx (GH#3396).
Restored the descendants comment on ReadyRequest.ParentID and moved the
one-level explanation to ListRequest.ParentID, which had none.
Verified: make api-check; dolt-server, embeddeddolt and uow Counter
contract legs (BEADS_TEST_DOLT_SERVER=local with a pinned dolt binary for
uow; Docker for the dolt-server leg); the new proxied HTTP exclude_status
probe (BEADS_TEST_PROXIED_SERVER=1).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* chore(wireshape): add the countIssues scope parameters to the wire-shape golden
#7182 now digests operation parameters. The four new issues:count parameters
(parent, no_parent, exclude_type, exclude_status) are additive entries, so no
wire_revision bump is needed; regenerated with gendigest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(conformance): register the S8 count-scope cases with RunRoleContracts (adopt-pr #7199)
The PR adds nine RunCounter* contract cases to counter_contract.go and wires
them into the three in-tree TestCounterContract legs, but not into the
Counter row of roleContractCases. TestRoleContractCasesCoverEveryContractCase
derives the expected set from source, so it fails on the PR head, and an
external backend running RunRoleContracts would never see the new cases.
Register them in source-declaration order (after
RunCounterAnUnknownStatusMatchesNothing), which the same guard enforces.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(count): correct the S8 scope docs the review found wrong (adopt-pr #7199)
Comment and spec-description fixes only; no behavior change. Review-loop
iterations 1 and 2 of the #7199 relaunch.
- ListRequest.ParentID (issueops/reader.go) said "DIRECT children only
... the same one-level clause". The sqlbuild clause has two arms and
neither walks: the parent-child edge arm stops at one level, but the
dotted-prefix arm (`id LIKE CONCAT(?, '.%')` for rows with no
parent-child edge) reaches every edge-less dotted id at any depth, which
RunReaderListParentReachesEveryDescendantAndOnlyItsOwn pins with an
edge-less X.1.1. State both arms, and say the CLI tree view re-runs the
clause against the store once per level (findAllDescendants calls
SearchIssues, not the role). Document ListRequest.NoParent, which had no
doc. The contributor's ReadyRequest.ParentID text is kept; only its
cross-reference to this field changes from "one-level" to "non-walking".
- CountRequest.NoParent (issueops/counter.go) and the count flag comment
(cmd/bd/count.go) said NoParent counts rows with "neither" an edge nor a
dotted prefix. Its SQL checks only the edge table, so an edge-less X.1
is counted by both --parent X and --no-parent, as bd list lists it.
- RunCounterExcludeStatusNarrowsThePredicate's header said an
unrecognized name "excludes nothing rather than failing" while its body
asserts ErrValidation. The header now matches the body.
- The OpCountIssues entry in problem.go's operationCodes said the role has
"exactly one reachable refusal". It has three: an invalid metadata key,
an unrecognized exclude_status name, and parent with no_parent (named
no_parent), all classified by invalidFilterParam. The code set is
unchanged.
- BuildCountFilter's header said cfg is "only read under IncludeInfra".
It is also read to validate ExcludeStatus against the workspace's
custom statuses, which is why both callers load it when ExcludeStatus is
non-empty.
- The countIssues `parent` description in openapi.v0.yaml carried the
same one-level claim as ListRequest.ParentID; it now states both arms.
types.gen.go is regenerated (make api-gen); make api-check passes.
Review fixes (iteration 2), still comment and spec text only:
- The iteration-1 problem.go rewrite called all three count refusals
ErrValidation. Only the exclude_status and parent-with-no_parent
refusals wrap issueops.ErrValidation (internal/workapi/count.go). The
metadata-key refusal is ValidateMetadataFilters' plain fmt.Errorf
chain, so errors.Is(err, ErrValidation) is false for it; it reaches the
wire as a 400 only because invalidFilterParam matches message prefixes.
The comment now says that, so a later move to typed errors cannot trust
it and demote the metadata-key refusal to a 500. The sentinel itself is
not added here: ValidateMetadataFilters is shared with the list and
ready builders.
- invalidFilterParam's comment on the count rows pointed at the
metadata-key rows as "above"; they come after it in the switch.
- The countIssues `parent` description said the CLI tree mode walks
"every edge-linked descendant". findAllDescendants re-runs the full
two-arm predicate per level, so it also descends through edge-less
dotted ids; it now says "every descendant reachable through either
arm". types.gen.go is regenerated; make api-check passes.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(wireshape): re-record the count scope params under the base's param digest (adopt-pr #7199)
The base branch (feat/wire-revision-signal, #7182be1f0bad72) now digests
the effective style/explode of scalar query parameters. This PR's golden
entries for countIssues query:parent and query:no_parent were written by
the old algorithm, so after the base refresh TestWireShapeDigest reported
them as "changed without a wire_revision bump".
No bump is owed: all four count-scope entries (parent, no_parent,
exclude_type, exclude_status) are additions this PR introduces and are
absent from the base golden. golden.json was reset to the base copy and
regenerated with `go run ./internal/httpapi/wireshape/cmd/gendigest`. The
result is four pure additions against the base (+35/-0); parent and
no_parent gain "style": "form" and "explode": true, and wire_revision
stays 2.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* ci: place the PR's new embedded counter test in the shard manifest
TestCounterExcludeStatusAcceptsAWorkspaceCustomStatusWithoutIncludeInfra,
added to internal/storage/embeddeddolt in f23bf5ddbf, is a new top-level
embedded storage test. The 15-shard block of
.github/scripts/embedded-storage-test-shards.txt must list every
discovered test exactly once. Without the entry,
TestStorageEmbeddedShardManifestGeneratorNotStale (scripts/) fails, both
on the reviewed head 89a387de6c and on the merge. CI never ran the check
on this PR: pr.yml runs only for PRs based on main, release/** or
hotfix/**, and this PR was based on feat/wire-revision-signal until the
retarget.
Generated with
`python3 scripts/ci/gen_embedded_storage_shard_manifest.py 15 --weights=duration --write`.
The incremental write adds this one line, in shard 4, and moves no other
test.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.local>
* perf(issueops): index each step of the per-edge cycle and hierarchy walks
The single-edge cycle and hierarchy probes (AddDependencyInTx, i.e. bd dep
add and apply-batch dep_add items, the domain stack's HasCycle /
ValidateBlockingHierarchy, and apply-batch's end-gate hierarchy check) joined
the recursive frontier against a UNION of dependencies and wisp_dependencies.
Dolt cannot index that derived table, so every recursion step rescanned every
matching edge: on a 50k-issue / 100k-edge database one dep add took minutes
(an apply-batch of 1 create + 2 dep adds ~10 min).
Give each dependency table its own recursive member joined directly on
d.issue_id, so each step is an index lookup, and pin the lookup plan with
JOIN_ORDER/LOOKUP_JOIN hints: without them the Dolt sql-server planner picks
a type-index scan join while table statistics are not yet available (right
after a commit), back to minutes. Same reachable set (UNION distinct merges
the members per step), same edge types, same error paths, and still one
statement per probe.
Adds a "depadd" createbatchequiv scenario with a golden digest recorded from
the pre-change code: every cycle/hierarchy probe over 17 nodes x 3 table
sets (mixed tables, every target column, external and missing ids), then 20
AddDependencyInTx calls covering cycles through both tables, hierarchy
conflicts in both directions, self, missing endpoints, type conflict,
external and non-scheduling edges.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(issueops): pin lookup joins in the batched blocked-state recompute
The batched mark/unmark UPDATEs decide membership through the scoped
should-be-blocked union. On the Dolt sql-server its planner intermittently
(depending on when the background statistics refresh lands) turned the
union's issues legs around: it drove them from an index scan of every open
issue through (is_blocked, status) and probed dependencies per row. Over an
import's few thousand uncommitted issues that is ~3 s per leg and 6-10 s per
mark/unmark statement for each 200-id chunk, where the lookup plan takes
~0.1 s; an import of 5000 issues with edges took ~28 min and came close to
the 10 s read timeout. EXPLAIN on the server engine showed the two plans;
two of three identical 2000-issue imports hit the slow one (4-5 min vs 14 s).
Pin each joined leg of the scoped union to drive from the batch's own
dependency rows and look the target/parent up by primary key
(JOIN_ORDER/LOOKUP_JOIN hints, ignored by MySQL). The SQL is otherwise
unchanged, so results are identical (createbatchequiv golden digests,
including is_blocked, pass on both engines) and the statement count is the
same. The unscoped union used by full repair and the doctor count walks
whole tables and keeps the planner's choice.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): pin the hinted graph-walk and blocked-state plans with EXPLAIN
The JOIN_ORDER/LOOKUP_JOIN hints on the per-edge reachability walks and the
batched blocked-state recompute were only pinned textually: the suite passed
with them deleted or naming a wrong alias. TestGraphWalkPlansHonorJoinHints
(embedded) records the statements the production entry points send
(WouldCreateSchedulingCycleInTx, CheckBlockingHierarchyInTx,
RecomputeIsBlockedInTx) through a DBTX wrapper, EXPLAINs exactly those, and
asserts each recursive member is a lookup join keyed by the frontier node and
each joined union leg a primary-key lookup, never an is_blocked/status scan.
Deleting a hint or misspelling its alias fails it.
The test found a remaining planner trap in the ancestor walk: with
"d.type = 'parent-child'" in the join condition, the embedded planner
satisfied the wisp_dependencies member through the type index keyed by the
constant, a lookup returning every parent-child row of the table per
frontier node. The type filter now sits in each member's projection (a row
of another type yields a NULL node, which joins nothing and never equals the
goal), so issue_id is the join's only predicate. The depadd golden, recorded
from the pre-change code, still matches.
Also corrects the hint comments: MySQL honors JOIN_ORDER (the order it forces
is the wanted one) and ignores the unknown LOOKUP_JOIN with a warning.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(createbatchequiv): check run scenarios against the golden only; probe a stored parent cycle
A run scenario (depadd) drives code the create fast paths do not gate, so
its fast-vs-per-row comparison compared a body with itself. Apply it once,
check it against its golden digest (the pre-change code's answers), and fail
if the run recorded nothing.
depadd now plants a parent-child 2-cycle straight into dependencies, as
legacy or merged data can hold even though every write path refuses it, and
probes across it plus three adds touching it, so termination of the walks on
stored cycles is covered. Golden re-recorded from a93498c81 (pre-change
walks); the current code matches it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(issueops): index the waits-for gate's child lookups; waits-for bench fixture
The large-graph benchmark gains a formula-shaped fan-out
(batchfixtures.WaitsForInserts: one spawner, 1000 parent-child children,
20 waiters over every waits-for gate shape) and two shapes over it:
create+waits-for and closing a spawner child (which recomputes every waiter
through the waits-for gate).
It was not a statistics-dependent plan flip but a consistently bad plan:
each "spawner has an open/closed child" EXISTS matched both spawner columns
with one OR, which no index serves, so every correlated evaluation scanned
the edge table. On 50k issues / 100k edges, closing one child took ~39 s
per close on embedded, and on the server create+waits-for hit the 30-minute
read timeout. The gate now splits each EXISTS per spawner column (one per
dependency table and column, each served by its target-column index) and
pins the child lookup by primary key (JOIN_ORDER/LOOKUP_JOIN).
EXISTS(a OR b) is EXISTS(a) OR EXISTS(b), so the result is unchanged. The
template is shared with the unscoped full repair, where a per-row correlated
lookup is the right plan too.
Close spawner child per close: emb 39.5 s -> 3.3 s; create+waits-for emb
2.1 s -> 0.46 s, srv >30 min (read timeout) -> 0.29 s; srv close spawner
child 3.2 s (not reached before).
Adds a "waitsfor" createbatchequiv scenario, golden recorded from
a93498c81 (pre-change gate): issue and wisp spawners with issue and wisp
children, waiters in all five gate shapes, then closes, reopens, a scoped
recompute and the full repair, snapshotting every is_blocked. A mutation
dropping the wisp-spawner legs fails it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(createbatchequiv): compare dependency metadata canonicalized across engines
The embedded engine and the Dolt sql-server serialize a JSON metadata value
differently (key order, spacing), so the waitsfor golden recorded on embedded
failed on the server for its gate metadata alone. Re-serialize each
dependency row's metadata through encoding/json before comparing. Rows with
'{}' metadata are unchanged, so the other goldens keep their digests;
waitsfor is re-recorded from a93498c81 and now passes on both engines.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37272437653
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* perf(storage): cut batch-create round trips (import, bulk create, apply-batch)
A multi-issue create (import, bulk create, CreateIssuesWithFullOptions) ran
~20 SQL statements per issue, and its dependency pass ran a recursive cycle
CTE per edge over the union of both dependency tables, which Dolt cannot
index — every recursion step rescanned the edge tables, so an import whose
edges form chains went superlinear (400 issues: 73 s of 82 s in that CTE).
issueops batch create, same transaction and same stored outcome:
- createBatchCache: one IN-list read of every explicit id's presence in both
planes (replacing two COUNT probes per issue), one read of the batch's
stored labels (each issue's new labels then land in one multi-row INSERT),
created/label_added events buffered and written in one pass under the ids
InsertDerivedEvent would mint (same digest, same lowest free ordinal), and
runs of brand-new rows written with multi-row INSERTs (same ODKU clause)
then finished in order.
- Dependency pass: wisp routing and target presence from batch reads; the
cycle and hierarchy checks walk an in-memory copy of the edge tables,
loaded once and kept in step with every edge the pass writes (a pair it
already holds is re-read, since a clientFoundRows connection reports a
matched duplicate as written).
- createBlockedRecomputeIDs seeds all created edges and expands once.
Shared paths:
- Blocked-state recompute splits ids once per call by whether they own a
dependency row; edgeless ids (every freshly created issue) skip the two
union UPDATEs for one plain unmark, run after the chunk's union statements
on the first pass so the unions see what they saw before.
- appendChildrenInTx reads children per IN-list chunk, not per parent.
- ExecuteCreate hands its BatchContext to the batch body instead of having
it re-read the same config (-6 statements per apply-batch create item).
Not changed: one version-history row per created issue (the version
contract, version_history.go), journal order and content, and the per-row
bodies, which stay the path for singular creates.
Equivalence: internal/storage/createbatchequiv runs an import scenario
(upserts, duplicate ids, wisps, generated ids, cycles, hierarchy conflicts,
dangling/external targets, stale is_blocked, comments, journal and
versioned history on) through the fast and per-row bodies on the embedded
and server engines and compares every written table and the skip reports.
Re-pinned TestLargeBatchApplyStatementCounts*: embedded 7009/14014/846 ->
6396/12790/793, server 7008/14014/846 -> 6396/12791/793 (breakdown in
pinnedEmbeddedStatementCounts).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): batch-create review fixes — lazy edge graph, opt-in recompute probe, insert budget
Review of the batch-create round-trip change (S1-S3 and nits):
- The dependency pass's in-memory graph no longer loads both edge tables.
It reads a node's outgoing edges through the issue_id index the first
time a walk reaches it: the batch's own endpoints up front, then one
IN-list read per BFS level. Memory and reads scale with the subgraph
the batch's walks touch, not the edge tables, so a two-edge create on a
large database no longer materialises every edge. A written edge whose
source was never loaded loads that source (row included).
- The blocked-state no-edge probe runs only when a create asks for it or
when a chunk holds at least 32 ids. Single-issue recomputes on close,
reopen, update and dependency edits issue exactly the statements they
did on main (the reopen sqlmock expectation is back to main's).
- Multi-row issue INSERTs are bounded by an estimated 16 MiB of bound
text as well as 100 rows. When a multi-row statement fails, its rows are
replayed one at a time, and the batch continues if every row lands
instead of returning the original error.
- Nits: document the binary-collation assumption behind Go string
equality; DisableCreateFastPathsForTest documents that it is
process-global and non-parallel, and panics on an overlapping caller;
drop the unreachable skip branch; explain why wisp targets need no
presence read.
Re-pinned TestLargeBatchApplyStatementCounts* classic 40: 793 -> 786
(the +7 probes on dep-add recomputes are gone; net vs main is -60, all of
it ExecuteCreate's config reuse). 356/712 unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): search the batch dependency graph from both ends
The lazy per-node load (previous commit) still walked forward from the
check's start, so on a large stored graph a single create's cycle check
read every node reachable from its target — about 800 statements per
create on a 2000-issue fixture. The cycle and hierarchy questions a batch
asks almost always end at a row the batch just created, whose only
incoming edges are the batch's own.
depGraph now answers each reachability question with a bidirectional
search: forward from the start along outgoing edges (issue_id index),
backward from the goal along incoming edges (the three target-column
indexes, read as one UNION ALL per table), always expanding the smaller
frontier and loading a level's nodes in that direction once. The batch's
endpoints are loaded in both directions up front, so a create's checks
usually finish with no further read (a chain import, too: each new
edge's goal has no incoming edge yet). Rows are keyed by (table, source,
target), so a row read from both ends counts once. An incoming read
ignores a row whose matching column is not its first set target column,
as the CTE's COALESCE does. A written pair that neither loaded end can
vouch for is re-read instead of assumed new.
On a 50k-issue / 100k-edge database, a create with two blocks edges
now runs 27 statements, against the full-scan or per-node walks before
(numbers in the review-fix summary).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): hold batch create to golden digests from the pre-change code
The batch-create equivalence test compared the fast paths with the same
code's per-row bodies, so changes the fast-path switch does not gate —
the recompute seed merge, the IN-list children read, ExecuteCreate's
context reuse, the prepare/finish split — were checked only by older
contract tests. Its batch was also too small to cross a chunk boundary
and never set RejectStaleUpserts.
createbatchequiv now runs four scenarios, each through the fast and
per-row bodies, and checks both against a per-table row count and
SHA-256 recorded from ebe3b6bcb (before the fast paths existed;
CREATEBATCHEQUIV_GOLDEN_DIR re-records them):
- small: the original scenario;
- import458 and import458-reject-stale: the review's 458-issue batch
(every chunk boundary, wisps and cross-plane edges, cycles, hierarchy
conflicts, duplicate ids across deferred runs, generated and
hierarchical ids, stale upserts rejected and accepted), plus new edges
whose blocked-state fallout reaches stored rows outside the batch (a
spawner's waiter, an upserted parent's existing child);
- apply: an apply-batch through ExecuteCreate, dep adds and an update.
The recorded digests match on the embedded and server engines (the
server test now sets the fixture's prefix, so generated ids agree).
Dropping the spawner-waiter seeding is invisible to the fast-vs-per-row
comparison and caught by the golden check.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): large-graph create benchmark
batchbench measures creates against a database already holding a large
dependency graph (batchfixtures.LargeGraphInserts: N issues, ~2N
acyclic edges with wide reachable sets and a deep parent-child forest,
committed as a Dolt version): ExecuteCreate under an existing parent with
an existing blocker, a store create with two blocks edges, and an
apply-batch of creates each with a parent-child and a blocks dep_add.
It logs wall time, statements and Go allocation per shape on the
embedded and server engines. Opt in with BEADS_BENCH_LARGE_GRAPH=<issues>
(and BEADS_BENCH_LARGE_GRAPH_CREATES); it asserts nothing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(storage): read incoming dependency edges one target column at a time
The bidirectional graph search read a node's incoming edges with one
UNION ALL over the three target columns per table. Dolt plans that as a
scan: 0.7 s per read on a 100k-edge table, where each leg alone is a 1 ms
lookup on its own index (EXPLAIN shows IndexedTableAccess per leg). Issue
the three legs as separate reads into one row set; a row matched through
two columns still counts once.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): run the 458-issue equivalence scenarios outside the race lane
The 458-issue golden scenarios took the embedded tier's shard over its
20-minute limit under -race (the in-process engine's own instrumentation;
about 35 s each without it). They move to
TestCreateBatchFastPathsMatchPerRowLarge_Embedded, skipped under -race and
added to nightly.yml's non-race embedded step, following the precedent of
the 712-item statement-count and 1000-item wall-clock cases. The light
scenarios (small, apply) stay in TestCreateBatchFastPathsMatchPerRow_Embedded
on every lane. The server backend (built without -race in its full
suite) runs both: TestCreateBatchFastPathsMatchPerRow_Dolt and, as its own
top-level test so the shard manifest's hash places it on another shard
(about four minutes there), TestCreateBatchFastPathsMatchPerRowLarge_Dolt.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): reset cobra flags between in-process runs; keep them out of parallel tests
runBDInProcess and runBDInProcessAllowError execute the package-level
rootCmd tree again and again in one process, and cobra never resets a
flag between Execute calls: a value one invocation parsed (create's
--title, --labels, list's --status, ...) and its Changed bit stayed on the
FlagSet, so the next invocation saw it, and a slice flag that had been
set appended to the stale value instead of replacing it. With a Dolt test
server (the only setup in which these tests run) that failed about 20
TestCLI_* tests depending on suite order.
TestMain now snapshots every flag of the tree (snapshotCommandFlags,
after all init()s registered them, before any test runs) and both runners
put the tree back before and after each run (resetCommandFlags): the
pointee of each flag.Value is restored (for pflag's scalar types that is
the bound variable; for slice types it also clears the internal "changed"
bit, so the next Set replaces), slice contents via SliceValue.Replace,
and Changed=false. TestResetCommandFlagsRestoresEveryFlag sets every flag
of the real tree and checks the reset. The ad-hoc --title reset in
TestCLI_CreateRejectsEmptyTitle_ProxiedServerMode is no longer needed.
Seven TestCLI_Comment* tests called t.Parallel() around the runners.
inProcessMutex serializes the runners only against each other, so their
chdir, stdio swap, env and config re-initialization raced every other
parallel test (data races on config globals; TestCobraParallelPolicyGuard
failing to read its own source files after a concurrent chdir). They run
serially now, and the guard treats the two runners like rootCmd.Execute.
Also two stale TestCLI_Create* cases: DryRunWithRigPrefix exercised a
create --prefix flag that no longer exists, and the flag-like-title cases
passed "--help" or "--foo-bar" where cobra parses them as flags; they now
pass the title after "--", the only way it reaches create's own guard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix the ADO round-trip and bulk-delete tests that rotted unseen
Both need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd (BEADS_TEST_SKIP=dolt everywhere), so they failed unnoticed:
- TestADORoundTrip* stored ado.pat in the database. ado.pat is a secret
read only from config.yaml or AZURE_DEVOPS_PAT (config.IsYamlOnlyKey),
so the tracker found no PAT. The fixture sets AZURE_DEVOPS_PAT.
- TestBulkDeleteNoResurrection expected GetIssue to return (nil, nil) for
a deleted issue; the Dolt store reports storage.ErrNotFound.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix parity, completion and notion tests that only fail with a Dolt server
These are Dolt-gated (integration tag + test server), so no CI lane ran
them and they rotted:
- TestParityCreate*/TestParityReopenErrorExits1Silently: bd create
auto-routes via routing.DetectUserRole("."), which warns "beads.role
not configured (GH#2950)" on stderr when the working directory's git
config has no beads.role. The suite pins the configured case, so the
parity env now sets beads.role=maintainer in a test-owned global git
config ($XDG_CONFIG_HOME/git/config; the role lookup scrubs every
GIT_CONFIG_* variable, so those cannot be used).
- TestIssueIDCompletion: issueIDCompletion resolves a database path
before looking at the open store and returns nothing when there is
none; the test now sets dbPath to its store's path.
- TestIssueIDCompletion_UsesWorktreeFallbackWhenStoreNil and
TestGetNotionConfigReadsDBPathWhenStoreUnset: the code under test
reopens the store from metadata.json, which names a database but not
the per-test branch of the shared database that newTestStore uses, so
the reopened store saw none of the test's rows. They now use their own
database (newTestStoreIsolatedDB).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): fix stale doctor/e2e/autocommit/readonly Dolt-gated tests
These need the integration tag and a Dolt test server, which no CI lane
gave cmd/bd, so they rotted unseen:
- TestE2E_DoctorFixMetadataRoundtrip, TestE2E_MigrateDoltMetadata,
TestDoctorCheckHealthReportsVersionMismatchOnRepoLocalPort: bd init now
defaults to embedded mode, where `bd sql` and `bd dolt start` are
refused. Init with --server on a repo-local server of their own
(repoLocalServerEnv: the suite's shared-server port vars cleared, server
stopped at cleanup) and start it explicitly, since BEADS_TEST_MODE=1 never
auto-starts one. MigrateDoltMetadata's DELETE also needed `key` quoted.
- TestDoctor{Conventions,Pollution}MaxRows_EnvOnly_Exits2: bdRunRaw drops
every BEADS_* variable, so the subprocess hit dolt.New's test-database
firewall (exit 1) before the row cap; pass BEADS_TEST_SERVER=1.
- TestDoltAutoCommit_On_WritesAdvanceHead: the embedded engine keeps its
data under .beads/embeddeddolt, not .beads/dolt; findDoltRepoDir looks
in both.
- TestValidateCheck_AllClean: bd doctor validate runs five checks since
the cross-table duplicates check was added.
- TestConfigValidateReadOnlyIsHermetic: readonly commands take a shared
workspace gate, whose lock file (.beads/dolt.gate.lock) workspacegate
creates on first use by design; compare the .beads tree without gate
lock files.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): undo each in-process execution's env and storage-mode leaks
Every in-process run of the bd command tree (runBDInProcess and the many
tests that call rootCmd.Execute directly) leaves process state behind:
PersistentPreRun exports the selected workspace as BEADS_DIR, loads its
.beads/.env, and records the workspace's mode in serverMode /
proxiedServerMode. With a Dolt test server, later tests in the same shard
then resolved a deleted fixture ("no active beads workspace found",
"migration sidecar is missing"), ran subprocess bd against the wrong
workspace ("recorded as proxied-server in .beads/metadata.json"), or took
the proxied path in RunE-level tests ("proxied-server UOW provider not
initialized", a nil-UOW panic in reopenProxiedResolve that killed a shard).
TestMain now registers a cobra initializer/finalizer pair
(installExecuteIsolation) that snapshots the environment and the two mode
globals when an execution starts and restores them when it ends; the
runners' restoreEnvSnapshot shares the helper. TestDoltConfigEnvironmentOverrides
used os.Setenv + defer os.Unsetenv, which deleted the suite's
BEADS_DOLT_SERVER_PORT (the test server's port) for every later test; it
uses t.Setenv now.
Full integration-tagged cmd/bd suite against a hermetic dolt (16 shards):
18 order-dependent failures and the shard panic gone; the 27 remaining
failures all fail alone too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(init): keep --quiet quiet through the Claude/Codex/Cursor auto-setup
`bd init --quiet` ("Suppress output") still printed the project
integration installers' progress to stdout ("Installing Claude hooks for
this project...", "Claude Code integration installed", ...): the
installers write to os.Stdout unconditionally. Point stdout at the null
device around them while quiet; their failures keep going to stderr as
before. Found by TestInitCommand's quiet case, which asserts no output and
had not run in CI (Dolt-gated).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(init): bring the Dolt-gated init tests up to the current init contract
These tests need a Dolt test server, so no CI lane ran them and they
drifted from the product:
- Embedded is init's default mode, so the server-mode assertions
(.beads/dolt, opening the database on the test server, dolt_mode
"server" in metadata.json) now pass --server (TestInitCommand,
TestInitAlreadyInitialized, TestInitRedirect, TestInitBEADS_DIR,
TestInitDoltMetadata, TestInitDoltMetadataNoGit, TestInitDatabaseFlag's
metadata subtests). The in-process ones reset rootCmd's flags around
themselves so --server cannot leak into later in-process inits.
- Each server init gets its own --database: the suite shares one test
server, and a "test" database another test created belongs to another
project (init refuses: PROJECT IDENTITY MISMATCH).
- The marker tests seed a current-era version witness: the legacy-upgrade
guard refuses a non-empty .beads/dolt without one.
- The redirect test provisions the canonical target with config.yaml: bd
ignores a redirect to a directory with no project files (#4692).
- Shared-server inits get a free port of their own
(sharedServerTestEnv): the inherited BEADS_DOLT_SERVER_PORT is the
suite's test server, which a shared-server start refuses as another
project's, and the default shared port is 3308, a developer's live
server.
- --shared-server's config.yaml is now nested YAML (dolt: shared-server:
true); parse it instead of matching the old flat line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): run the Dolt-gated tests that skipped on a bare dolt.New
setupTestDB (config, kv and Linear round-trip tests), setupGatedTestDB
(mol ready --gated) and the rename-prefix and bd sql tests opened stores
with dolt.New(&dolt.Config{Path: ...}): no server port, no
CreateIfMissing, so against a running test server they got "database ...
not found" and skipped as "Dolt server not available". They now use the
suite's newTestStore helpers (skip only when no test server runs), and
what that exposed is fixed test-side:
- TestCustomStatusConfig: custom_statuses is read back ORDER BY name;
look statuses up by name / expect name order.
- TestLinearRoundTripCoreFields, TestLinearPullMilestonesCreatesEpicHierarchy:
linear.api_key is a yaml-only secret (LINEAR_API_KEY), push now needs an
explicit linear.state_map, and batch push uses TeamLabels and
issueBatchCreate, which the mock server now answers.
- TestSqlCommand: bd sql refuses in the default embedded mode; set
serverMode for the server-backed store, and stop discarding RunE errors.
- TestE2E_AutoStartedRepoLocalServerPersistsAcrossCommands skipped on
this process's embedded mode although every bd it runs is a subprocess
on a --server workspace; gate on the dolt binary instead and stop the
auto-started server at cleanup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): no fixed "unused" port, no swallowed JSON, no leaked front-door server
Three more things the Dolt-backed cmd/bd run exposed:
- TestTestServerConnection, TestSharedServerCfgNilHonorsSharedServer and
the two TestLegacyGuard* e2e tests pointed bd at 127.0.0.1:59999 as a
port nothing listens on. Any process on the host may hold it; a
listener that accepts and resets turns the expected "connection
refused" / "Dolt server unreachable" into another error. They take a
just-freed port from closedLoopbackPort (testutil.FindFreePort) now.
- TestDoltSetConfigJSONOutput and TestDoltSetConfigWithUpdateConfig
created .beads 0755, so bd's permissions warning landed in the JSON
under test and the tests skipped ("output not pure JSON") every run.
The fixture uses 0700, and non-JSON output fails instead of skipping.
- TestMigrateDoltModeFrontDoor's closing show/list auto-start a
repo-local server after the reverse migration; nothing stopped it, so
the suite's leak check failed (when the opt-in
BEADS_TEST_MIGRATION_FRONTDOOR suite runs). It stops it at cleanup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): advisory cmd/bd Dolt-server tier (bazel-cmd-dolt)
No CI lane ran cmd/bd's Dolt-gated tests (TestCLI_*, the init and
store-backed suites, the doctor/e2e tests): main.yml's integration shards,
nightly's Full Test Suite, and Bazel's --config=prcore/ci and
--config=integration all pass BEADS_TEST_SKIP=dolt, so cmd/bd's TestMain
never starts a test server and those tests skip; the proxied and embedded
tiers run only their manifests' tests. They rotted unseen.
//cmd/bd:bd_dolt_server_test (tag dolt-server-cmd) runs the whole
integration-tagged cmd/bd suite, the integration build's bd_test, against
a hermetic dolt sql-server: BEADS_TEST_DOLT_SERVER=local starts it from
the pinned dolt in the test wrapper's runfiles, and
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 fails the target instead of skipping
into a cached pass. No test selection: the Go binary shards itself over
every top-level test (32 shards), so each runs exactly once, and
check_testcases.py fails a shard that ran none. --config=doltserver-cmd
has exactly --config=integration's build flags, so the two lanes share
bd_test's compile; prcore/ci and integration exclude the tag.
bazel.yml's new bazel-cmd-dolt job runs it exactly where bazel-integration
runs (same if: remotely on the farm for same-repo PRs, fork-ro/fork-rw,
and locally with the read-only cache in mode cache for forks). It is
advisory: job-level continue-on-error keeps a failure out of the call's
aggregate (BAZEL), and pr.yml's ci-gate reads its result (BAZEL_CMD_DOLT)
but requires it only while the committed top-level flag
BAZEL_CMD_DOLT_REQUIRED is "true" (now "false"; flipping it is the one
change needed). bazel-gate.sh accepts its skip in modes skip and local.
Policy tests: a third lane category in TestBazelLaneIsGatedAlongsideLegacy
(bazelFlagGatedLanes: gate env wired, never statically required, added by
the run script exactly when the pinned flag is "true"), the gate
simulation over every mode with the flag off (advisory: nothing it
reports reddens the gate) and on (exactly a gated lane), the cache-mode
and execution-mode checks, the tag taxonomy and prcore/integration
exclusions, and TestBazelCmdDoltJob (job steps, timeouts, .bazelrc
config equal to integration's build with no dolt skip or test selection,
the target's dolt-server rules, and bd_test's env repeated).
Measured locally (all fixes in this series, 32 shards, 16 at a time on a
shared host): 3267 top-level tests, all shards pass, max shard 313s (one
5001-issue export test, 214s, sets it), mean 100s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(init): hand the project installers a writer instead of swapping os.Stdout
The --quiet fix earlier in this series pointed the process-global
os.Stdout at the null device around the Claude/Codex/Cursor installers.
That works, but it mutates a global from inside a cobra Run: anything else
writing to stdout meanwhile (a goroutine, a future concurrent installer)
would vanish, and a concurrent read of os.Stdout is a data race.
The installers already write through env structs with stdout/stderr
fields. setup.ProjectInstallOutput carries the two writers, and
InstallClaudeProjectTo / InstallCodexProjectTo / InstallCursorProjectTo
use them (the old entry points keep the process streams). init passes
QuietProjectInstallOutput() under --quiet: progress discarded, error lines
still on stderr. The proxied-server init path (domain adapter
InstallClaudeProject now takes quiet) does the same, so its
"Installing Claude hooks..." is quiet too.
Consistent error policy under --quiet: errors still reach stderr. init's
own "Warning: failed to setup <installer>" lines were suppressed under
--quiet while the installers' "Error:" lines were not, and some installer
failures (Cursor rules) print nothing themselves; the warnings now print
whether or not quiet, like the installers' errors.
Coverage in required lanes: TestProjectInstallersReportToTheGivenOutput
(cmd/bd/setup) and TestEmbeddedInitA/basic, which now asserts that
`bd init --quiet` with agents enabled installs the Claude hooks and prints
none of the installers' progress (verified to fail with the quiet output
disabled).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(export): exercise the incremental-export fallback with a small threshold
TestTryIncrementalExport_ThresholdExceededFallsBack seeded
incrementalExportThreshold+1 = 5001 issues to cross the threshold. Batch
create is one transaction but still several statements per issue, so under
-race against the shared test server it took ~214 s, the longest test in
cmd/bd's Dolt-backed suite by far. The behaviour under test is only
"more changed issues than the threshold -> full export fallback".
incrementalExportThreshold becomes a var (same value), and the test, which
is not parallel, sets it to 20 and seeds 21 issues: 214 s -> 0.6 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): tighten the flag-reset self-test, guards and canaries
Review follow-ups:
- TestResetCommandFlagsRestoresEveryFlag no longer skips a flag it cannot
set: a flag type without a sample value, a Set error, or a non-pointer
Value (which resetCommandFlags could not restore) is an error, so a new
flag type cannot go untested. resetCommandFlags documents that it also
resets flag-bound package globals.
- TestCobraParallelPolicyGuard treats .ExecuteC/.ExecuteContext/
.ExecuteContextC like .Execute (the per-execution isolation writes the
mode globals; a parallel execution would race).
- TestConfigValidateReadOnlyIsHermetic ignores only the gate a readonly
command takes (.beads/dolt.gate.lock and its .info sidecar), not any
"*.gate.lock*" file, so a new unexpected gate still trips the canary.
- TestCLI_CreateRejectsFlagLikeTitles: a bare `bd create --foo-bar` must
fail and create no issue (bd-2c0's intent), next to the "--" cases.
- TestCustomStatusConfig cites why statuses come back in name order: the
normalized custom_statuses table (#2961, 8537d943b) has no ordinal and is
read ORDER BY name, like custom_types; the test predates it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the cmd/bd Dolt-server tier in 16 shards
Each shard pays ~30-40 s of fixed cost (TestMain plus its own dolt
sql-server), so 32 shards spent roughly 40% of the lane's executor time on
setup and added 32 farm actions to every PR touching cmd/bd. With the
5001-issue export test cut to under a second, 16 shards measure (locally,
--nocache_test_results): max 186 s, mean 137 s, 2,187 executor-seconds
(vs ~2,800-3,200 at 32), all 3,267 top-level tests, every shard passing.
TestBazelCmdDoltJob pins the count; bazel.yml's timing comment is
restated (cache mode ~20-30 minutes).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(proxy): fail fast with a MySQL error when the external upstream is unreachable
A proxied-server command against a stopped external Dolt upstream stalled
~20-30s before failing with a bare "invalid connection". The local db proxy
stays up, so on a refused backend dial (or a front such as stunnel/socat
that accepts and closes because its own upstream is gone) it simply closed
the client connection. The MySQL driver reports that as invalid
connection / unexpected EOF, which the uow bootstrap ping (#6003) classifies
as a transient drop and retries with exponential backoff for its whole 30s
budget. Retrying cannot help: the proxy already proved the upstream at its
own startup, and the outage is the upstream's.
The proxy now answers such a connection itself with a MySQL ERR packet
(2003, HY000) in place of the server greeting, naming the upstream and the
cause, exactly as a real server refuses a connection before the handshake.
The driver surfaces it as a *mysql.MySQLError, which the bootstrap retry
already treats as permanent, so the command fails in ~0.1s with an
actionable message. Only shapes that prove the upstream is not serving
qualify: ECONNREFUSED/ENOENT/EHOSTUNREACH/ENETUNREACH on dial, or a backend
that reaches EOF having sent zero bytes (a MySQL server always speaks
first). A dial timeout or a drop after the greeting keeps the plain close
and the client's transient retry.
External backends only: a managed (local sidecar) backend is owned by the
proxy and its dial can be refused transiently while bd drives it (the
full proxied lane caught TestProxiedServerInitExplicitBeadsDirIsAuthoritative
hitting exactly that mid-init when this applied to every backend), so
managed backends keep the bare close and the client's bootstrap retry.
TestExternalProxyOutageAndRestoreReDialsTCPAndUnix asserted the old
silent close during an outage; it now asserts the ERR packet.
With the latency fixed, TestProxiedOutageReconnectAcceptanceMatrix is
renamed into the proxied discovery pattern as
TestProxiedServerOutageReconnectAcceptanceMatrix and placed in the Bazel
30-shard block by gen_proxied_shard_manifest.py's incremental --write
(shard 8). The frozen legacy 15-shard block is untouched; the test
hash-distributes there like every other test added since it was frozen.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(proxy,uow): retry the proxy's upstream-outage report for ~1s; hedge the zero-byte case
Review follow-up to the fail-fast change.
A zero-byte close from the upstream is not proof it is down: Dolt closes
queued connections without a byte when max_connections/back_log are full,
when a queued connection times out, and while it shuts down for a
restart; an L4 front does the same while its target is replaced. Its
report now says "closed the connection before the MySQL greeting (down,
restarting, or at its connection limit)" instead of "unreachable", and
the comment no longer claims proof.
Refusals and endpoint churn also lost #6003's retry entirely, so an
external Dolt restart or a Service with no ready endpoints failed a
command at once. The client now retries the proxy's report
(proxy.IsUpstreamOutageError: 2003 with the "beads db proxy: " prefix)
for upstreamOutageRetryWindow (1s) from the first time it sees it, with
the backoff clamped so the last attempt lands at the window's end; other
transient errors keep the 30s budget. A real outage still fails in about
a second, inside the acceptance test's 3s bound.
Also: the dial report drops Go call-site wrapping ("dial tcp H:P:
connection refused"), truncation keeps UTF-8 intact, the comment states
how the packet differs from a real server's pre-handshake ERR (keeps the
'#'+SQLSTATE marker for vitess; 2003 is a client code chosen on purpose),
a Stats counter for reported outages, and the isTransientPingError
paragraph is reflowed and corrected.
Tests: pingWithRetry retries the report inside the window and gives up
at its end without using the full budget; openDB against a front that
plays the proxy (its real packet) and comes back 300ms after the first
connection succeeds in one call, and fails in about the window when it
never does; NewProxyServer enables the report for external backends
only; the dial message carries no Go type names; rune-safe truncation.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(proxied): fail, not skip, socat-bridged subtests in the Bazel lane
The external-unix front-door matrix subtest and both outage/reconnect
matrix subtests bridge an external endpoint with socat and skipped when
it was missing, so they could stop running without anyone noticing (the
parent test still passes). requireSocat now fails them when
BEADS_TEST_REQUIRE_SOCAT=1, which //cmd/bd:bd_proxied_test sets (its
remote workers have socat; a recent main run's external-unix subtest
passed there). The legacy GitHub 15-shard jobs have no socat and leave
it unset, so the subtests keep skipping there. The check now runs before
the upstream Dolt server is started, so a skip no longer pays for one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(proxied): measured weight for the outage matrix; pin it to legacy shard 1
Give TestProxiedServerOutageReconnectAcceptanceMatrix a duration entry
(80s: local runs measured 55-79s alone, 60-70s in its shard, and review
68-109s under load) and re-place it with the generator's incremental
--write: it moves from 30/8 to 30/11, one line, nothing else moved. The
heaviest estimated shard is now 435s against a ~357s median; a repack
would even that out but moves most assignments, so it is left to a
separate CI-only change that refreshes all durations together.
Legacy 15-shard block: the test would hash onto shard 14, already ~13.5
of the 15 min timeout on main. It is pinned to shard 1 (~6.5 min in
recent main runs) under the block's reviewed-pin rule; it skips there
today (no socat), and the pin keeps shard 14 safe if that changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Main's "Test (proxied-server cmd 5/15)" has hit the 15m -test.timeout in
6 of the 9 Main runs since #7129 where the shard ran to completion.
That PR renamed 12 proxied tests into TestProxiedServer* discovery.
They are not in the frozen 15-shard legacy block, so they hash-place,
and two landed on shard 5:
TestProxiedServerStorageClass (nine parallel inits, the heaviest of the
12) and TestProxiedServerServeRefusesAStaleCheckpointWithGone. Shard 5
went from 430-620s of job time to 800-950s. In those runs the timeout
fired while TestProxiedServerMolReadyGated's parallel subtests were still
queued behind the rest of the shard. That test is the victim, not a
pathological one. Shard 3 (also hash-loaded) runs 810-870s, within about
a minute of the same limit.
Measured locally (4 pinned CPUs, GOMAXPROCS=4), shard 5 took 897s with
its current test set and 524s with the pre-#7129 set.
Pin those two tests, plus shard 3's two heaviest hash-placed tests
(TestProxiedServerListComments and
TestProxiedServerServeRefusesAStreamOnADisabledJournal), onto the legacy
shards with the lowest measured job time: 14, 7, 4 and 1. Every test
already listed in the block stays where it is, so shard 5 is back to its
pre-#7129 test set. pr-risk.yml's fork jobs read the same block, so they
get the same relief. The script's 15m timeout, the job and step
timeouts, and the Bazel 30-shard block are unchanged. The block header
and the generator's legacy header now say a reviewed pin is allowed.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
TestSharedServerModeServeGlobalReportsTheServedDatabase was parked outside
the proxied discovery pattern because `bd serve --global` exited 1 on the
#5920 shared-store migration guard (v65 -> v69). The guard was right; the
test setup was wrong.
The workspace was a plain `bd init --server --server-port <test server>`
that only flipped BEADS_DOLT_SHARED_SERVER=1 afterwards. Shared-server
mode deliberately ignores the workspace's own dolt_server_port and
resolves the machine-wide shared server (BEADS_DOLT_SERVER_PORT, the
shared port file, ..., then the 3308 default), so every --global command
dialled 127.0.0.1:3308 rather than the hermetic test server. On CI nothing
listens there; on a developer machine running a real shared server, the
test read that server's own beads_global (created by an older bd at v65)
and serve asked to migrate it. beads_global was also never provisioned on
the test server, because only `bd init --shared-server` does that.
Set the workspace up the way an operator joins an externally-managed
shared server: `bd init --shared-server --external` (provisions
beads_global on the test server at the current schema) plus
BEADS_DOLT_SERVER_PORT naming that server. No product change: the guard
correctly refuses an un-consented migration of a shared database, and a
freshly created one (version 0) passes it.
Rename the test into the TestServerMode* discovery pattern and add it to
the Bazel 30-shard block via the generator's incremental --write. The
frozen legacy 15-shard block is untouched; the name hash-distributes there
(shard 12/15).
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): embedded-Dolt tier 20/5 -> 40/14 duration-balanced shards (F1)
Give the Bazel-only bazel-embedded job its own duration-balanced manifest
blocks for //cmd/bd:bd_embedded_test (20 -> 40 shards) and
//internal/storage/embeddeddolt:embeddeddolt_embedded_test (5 -> 14
shards), decoupled from PR Risk's/main.yml's legacy frozen 20/5-shard
fork/push jobs, mirroring F2's bd_proxied_test precedent.
- Measured every TestEmbedded* / embeddeddolt test's wall time in
isolation; split 21 cmd-tier and 1 storage-tier single top-level test
over 180s into disjoint sub-tests (3 irreducible exceptions left:
TestEmbeddedUpdateConcurrent 475s, TestEmbeddedCloseConcurrent 308s,
TestLargeBatchApplyStatementCounts712_Embedded 261s).
- New scripts/ci/gen_embedded_{cmd,storage}_shard_manifest.py +
_embedded_shard_manifest_lib.py generate/check duration-balanced
manifest blocks (LPT packing, incremental --write/--repack, exact-once
--check), chosen at N=40/M=14 where the heaviest shard reaches the
irreducible-test floor.
- Fixed 18 stale legacy 20-shard cmd manifest entries (and 1 storage
entry) left over from earlier test splits, which would have hard-failed
every shard of PR Risk's real test-embedded-cmd jobs on their next run.
- New scripts/embedded_shard_count_test.go policy tests tie the
Bazel-only block's shard count to bazel.yml's check_shard_coverage.py
argument and catch generator staleness; generalized
TestBazelRetiredLanesCheckListedTestsRan's live-shard-count dispatch
from bd_proxied_test-only to a map covering both embedded targets;
removed TestBazelEmbeddedJobMirrorsEmbeddedTier's now-invalid
BUILD.bazel-shard-count-equals-PR-Risk-matrix assertion.
- Verified: go test ./scripts/..., bazel test //scripts:scripts_test,
actionlint, make ci-pr-policy, make bazel-sync-check all pass; full
local bazel test of both targets at 40/14 shards passes 2/2; mutation
testing confirms each new/changed policy test fails exactly when it
should.
See ~/beads-bazel-plan/f1/impl-report.md for full detail.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(embedded-dolt): skip 712-item statement-count assertion under -race (F1)
TestLargeBatchApplyStatementCounts712_Embedded genuinely costs ~260-300s
under -race (confirmed by solo re-measurement: 298.60s) -- 14014 real SQL
statement round-trips through the race-instrumented in-process Dolt engine,
not an artifact and not splittable (one indivisible ApplyBatchInTx
transaction). It was the storage tier's single-test floor for CI shard
packing.
Apply the exact precedent TestLargeBatchApplyWallClock_Embedded already set
for its 1000-item shape: skip this assertion under raceEnabled (the cost is
race-instrumentation overhead on the Dolt engine's own internal
goroutine/lock machinery, not this test's own logic) and extend nightly.yml's
"Embedded Dolt batch-apply suite (non-race)" step's -run regex so the full
pinned 14014-statement baseline still runs, non-race, every night. Verified
directly: under -race the assertion now skips in 0.00s; without -race it
still runs and asserts Total=14015 (within the existing +/-2 tolerance) in
36.76s.
Updates scripts/ci_workflow_test.go's
TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite to match the extended
regex.
Storage tier's new single-test ceiling under the per-PR -race lane drops
from ~260.7s to 152.38s (TestBatchApplyContract).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): correct cmd-tier long-pole durations, re-pack embedded manifolds (F1)
Investigated coordinator item 2 (single-test long poles): TestEmbeddedUpdateConcurrent
(recorded 475.39s) and TestEmbeddedCloseConcurrent (recorded 308.06s), plus
the two other >180s cmd-tier entries from the same capture
(TestEmbeddedInitA 196.94s, TestEmbeddedEpicCloseEligibleReason 192.47s).
Root cause: NOT host contention, NOT backoff/retry sleeps (bdRunWithFlockRetry
contributes negligibly), NOT race-instrumented-subprocess overhead alone.
These numbers were captured via `bazel --nohome_rc test --config=embedded`
with the then-current 20-shard legacy manifest (~10-11 tests/shard) and
`-test.parallel=4`. Go's t.Parallel() starts each parallel test's elapsed
clock when its function is invoked, before the Parallel() call blocks it
waiting for a free one-of-4 slot -- so a test queued behind several other
heavy shard-mates has that queueing wait baked into its own recorded
`--- PASS: TestX (Ys)` time. This is shard-composition-dependent scheduling
delay, not a cost intrinsic to the test.
Confirmed two ways in an isolated clone (not this worktree, per the
no-plain-go-test-from-a-beads-checkout rule):
- Solo (no rivals at all): 151.83s / 106.48s / 63.58s / 66.08s.
- All four together with the real -race -test.parallel=4 flags and exactly
4 tests requested (so nobody queues -- parallel slots == test count):
141.34s / 113.80s / 69.79s / 74.63s.
Both agree closely and both are far below the capture-time numbers.
F1's LPT packer already isolates the heaviest tests into shards with few or
no other heavy rivals (TestEmbeddedUpdateConcurrent lands alone in its own
40-shard-block shard), so under the new manifest these tests see little to
no queueing delay in real CI. None of the four needed splitting, parallel
subtests, or raceEnabled-scaled N: once correctly measured, none exceeds the
180s single-test guideline (new corrected weights: 155.0 / 115.0 / 72.0 /
77.0s). Corrected scripts/ci/embedded_cmd_test_durations.json accordingly
(with a doc-comment explaining the methodology and both confirmations) and
re-packed the 40-shard Bazel-only block (--repack --write); --check passes
(exact-once coverage of all 241 discovered tests). The frozen legacy 20-shard
block is untouched and independently re-verified exact-once by simulating
embedded-test-shard.sh across all 20 shards (241/241 tests, no
duplicates/misses/errors).
New cmd-tier single-test ceiling: 179.47s (TestEmbeddedCloseAlreadyClosed,
likely itself somewhat inflated by the same mechanism but already under the
180s line as recorded, so no action needed).
Also updates scripts/ci/embedded_storage_test_durations.json: the prior
commit's raceEnabled-skip for TestLargeBatchApplyStatementCounts712_Embedded
means it now costs ~0s in the per-PR -race lane bazel-embedded actually
packs for (not its real ~260-300s non-race cost, which nightly.yml still
exercises) -- weight set to 1.0 with a doc comment tying it to the skip in
large_batch_apply_measure_test.go. Re-packed the 14-shard Bazel-only block
accordingly; --check passes (exact-once coverage of all 387 discovered
tests, excluding TestMain/TestConformance per that file's existing
convention). The frozen legacy 5-shard block is untouched and independently
re-verified exact-once via embedded-storage-test-shard.sh simulation
(388/388 tests, no duplicates/misses/errors).
Verification: go test ./scripts/... (includes the nightly.yml policy test),
make ci-pr-policy, make bazel-sync-check, actionlint all pass. Real local
`bazel --nohome_rc test --config=embedded //cmd/bd:bd_embedded_test
//internal/storage/embeddeddolt:embeddeddolt_embedded_test`
(--local_test_jobs=8, both targets together, on a shared/noisy sandbox host)
passes both targets; per-shard wall time from each shard's test.xml: cmd/bd
max 218.9s (shard 23/40, min 20.6s), storage max 251.1s (shard 5/14, min
0.98s) -- down from the prior oversubscribed sweep's 783.41s / 351.94s and
well under the single-test-floor-based ~475s projection this slice started
from. Projected tier lane time (max of the two targets, real dedicated CI
runners should do at least this well): ~251s, under the 5-minute target.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(embedded): un-vacuous TestEmbeddedDepB edges, split epic-close test, trim concurrency stress N (F1 review)
Coordinator review follow-ups on the F1 embedded-Dolt resharding slice:
- TestEmbeddedDepB (one of TestEmbeddedDep's two CI-speed split halves) was
asserting its tree_* subtests against an edgeless graph, passing
vacuously; restores the same parent-child/blocks edges TestEmbeddedDepA
builds so those subtests exercise real multi-node traversal (review S2).
- TestEmbeddedEpicCloseEligibleReason split into
TestEmbeddedEpicCloseEligibleReasonFlag/...Output for CI shard balance,
matching the project's established single-concern split pattern.
- TestEmbeddedUpdateConcurrent/TestEmbeddedCloseConcurrent's
issuesPerWorker reduced 5->2: under real concurrent-shard-load
measurement these were genuine (not queueing-artifact) serialized-write
costs; 2 preserves every invariant the test checks (duplicate-ID
detection across all workers, one non-decreasing-list-count check per
worker) at a fraction of the serialized work.
- large712ShapeName: the 712-shape's race skip and its zero-match guard now
share one named constant instead of two independent literal shape-name
strings, so a future rename of the shape can't silently skip the pinned
statement-count assertion under race (review N2).
- Comment-only fixes (review N5) in files referencing test names since
renamed/split by this slice's earlier work: close_proxied_integration_test.go,
graph_apply_test.go, storage_class_proxied_integration_test.go,
tests/regression/discovery_test.go, plus doc-comment path cleanups (N1:
drop unverified ~/beads-bazel-plan paths/bead ids in favor of the
committed duration JSON files and engdocs/TESTING.md).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): re-derive embedded-Dolt durations under concurrent load; cap cmd shards at Bazel's 50-shard limit (F1 round 2)
Coordinator review required re-deriving every per-test duration weight from
a single, consistent, CI-realistic measurement (all shards running
concurrently, -test.parallel=4, race-enabled) instead of the prior pass's
mix of crowded-shard captures and solo/4-together spot-corrections, and
required measuring per-shard wall time via BEP testAttemptDuration rather
than a shard's outer test.xml <testsuites time> (which can include a
different, already-finished shard-mate's setup/teardown).
- scripts/ci/embedded_cmd_test_durations.json (242 tests) and
scripts/ci/embedded_storage_test_durations.json (387 tests) rewritten in
full from one consistent sweep. Every entry is now a fresh measurement;
none is a carried-over capture-time value or an unverified estimate. Both
files' _comment blocks retract the earlier "queueing delay baked into a
test's own reported elapsed time" theory (testing.go's (*T).Parallel does
not add wait time to a blocked test's own clock) and document the real
methodology.
- Only one cmd-tier test remains over 200s (TestEmbeddedListFiltering,
239.70s), already under the ~240s ceiling; no further top-level splits
were needed this round.
- New finding: Bazel hard-caps shard_count at 50 per sh_test-like rule
("Having more than 50 shards is indicative of poor test organization";
a native analysis-time limit, not configurable). The previous 88-shard
plan for //cmd/bd:bd_embedded_test (chosen as the LPT-packed floor before
this limit was discovered) is reverted to 50, Bazel's real ceiling.
Repacking the fresh weights at 50 shards gives a heaviest shard of
409.6s -- above the single-test floor of 239.7s, because 50 bins is not
enough headroom over the suite's ~20145 measured test-seconds to flatten
every bin to that floor. This makes the <=5 min lane target unreachable
for the cmd tier via resharding alone within Bazel's native per-rule
limit; see ~/beads-bazel-plan/f1/impl-report.md section 3c for the full
writeup and the two remaining levers (suite-wide speedups or a
multi-Bazel-rule split), neither attempted in this slice.
- internal/storage/embeddeddolt gets 15 Bazel-only shards (unaffected by
the 50-shard cap): LPT-packed heaviest shard 225.3s, within target.
- cmd/bd/BUILD.bazel, internal/storage/embeddeddolt/BUILD.bazel,
.github/workflows/bazel.yml, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md,
scripts/embedded_shard_count_test.go, scripts/pr_risk_bazel_coverage_test.go,
scripts/ci/gen_embedded_cmd_shard_manifest.py updated to the corrected
50/15 shard counts (and, for the generator, N3: --write without an
explicit total_shards now errors instead of silently regenerating the
frozen legacy 20-shard block).
- .github/scripts/embedded-{cmd,storage}-test-shards.txt: re-packed
Bazel-only blocks at 50/15; frozen legacy 20/5 blocks untouched except for
N5 fixes to stale entries left by this slice's earlier test splits
(TestShardScriptsListOnlyRealTests catches this class of bug; both
legacy blocks reconfirmed exact-once).
- .github/workflows/nightly.yml / scripts/ci_workflow_test.go (N4): the
embedded-dolt batch-apply nightly step now runs with `if: ${{
!cancelled() }}` and its own explicit `-timeout 20m`, independent of the
job-level timeout, with both halves now asserted by
TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite.
Verified: gofmt -l clean, go vet ./scripts/... clean, go test ./scripts/...
green, both generators' --check pass at 50/15 (and legacy 20/5 remain
exact-once via the frozen-block invariant).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): rename embedded manifest staleness tests out of the ^TestEmbedded skip
The allowlisted go-test leg runs with -skip ^TestEmbedded, so tests named
TestEmbedded*ShardManifestGeneratorNotStale never ran there and the leg
failed with "matched no test that ran".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): fork and Dependabot PRs execute remotely on rbe-fork when rbe-west's mint is open (F9)
Fork and Dependabot pull_request runs get no secrets, no OIDC token and no
repository variables, so rbe-west decides for them. bazel.yml's rbe job asks
rbe-fork-mint's /v1/status (rbe-mint.ops.gascity.com:8444) once per run:
- open, tier ro: mode fork-ro (instance oss-fork, the fork pool, nothing
cached for anyone);
- open, tier rw (allowlisted author's own fork, allowlisted trigger): mode
fork-rw (instance oss, the OSS pool);
- closed, refusing, unreachable or anything else: mode cache, exactly as
today (local execution plus the anonymous read-only cache).
In the fork modes every lane runs, remotely, from a GitHub-hosted runner:
setup-bazel makes an EC P-256 key and a CSR, uploads the CSR as an artifact
of this run (the mint's proof), gets a ~2 h certificate
(fork-credential.sh, a byte copy of gascity's tools/rbe/fork-credential.sh
from F6) and writes an rc for grpcs://rbe-fork.ops.gascity.com:8444 with
the mint's instance. A lane whose certificate names another tier than the
rbe job fails ("re-run all jobs"). Fork lanes add --config=sole-run like
mode remote.
BAZEL_COVERS_FORKS ships "false" in pr.yml and pr-risk.yml: forks and
Dependabot keep their legacy tiers, and the Bazel lanes run beside them
(remotely while rbe-fork is open). F13 commits "true"; then a covered fork
run in mode cache is red, like a covered same-repo PR without the secret.
Safe while rbe-fork is dark: with the gate closed the mint's port resets
connections and the status probe falls back to mode cache in seconds; with
the mint up but disabled, canaried or rw off it answers open false or 403,
also mode cache.
Not in this change (later slices): bazel-farm.yml and its inputs stay (F12),
bazel.yml keeps its push trigger and pr.yml/pr-risk.yml gain none (F14).
keep their go build fallback on GitHub-hosted runners.
Tests: the rbe decision against a stubbed mint (ro, rw, closed, rw closed,
canary, busy, garbage, unknown tier, unreachable); the coverage and gate
matrices learn the fork modes, BAZEL_COVERS_FORKS and mint answers that
change between runs; TestBazelCacheModeReachesTheRC drives setup-bazel's
fork steps against a stand-in mint and checks each lane's rc;
fork-credential.sh's client contract and write-bazelrc.sh's fork mode and
exclusivity; the endpoint pin is allowed as exactly its one line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): fork-credential.sh retries the mint's 429 and never sleeps after the last try
A 429 (per-minute rate, live-certificate quota) may pass on its own like a
502, so it is retried with the same 10/20/30 s backoff (one minute, the
rate window); the loop no longer sleeps after the fourth failed attempt.
The cert request and the rbe job's /v1/status probe give up connecting
after 5 s, so a closed gate (dropped SYNs) costs seconds, not minutes.
Byte-identical to gascity's tools/rbe/fork-credential.sh.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: gascity-agent <agent@gascity.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): raise proxied-server tier from 15 to 30 shards (F2)
bazel.yml's bazel-proxied lane (//cmd/bd:bd_proxied_test, run only with
remote execution) was sharing PR Risk's legacy 15-shard, frozen
bd-init-cost-proxy manifest block, so its longest shard was the long
pole of the Bazel-only lanes. Give it its own 30-shard block in
.github/scripts/proxied-cmd-test-shards.txt, bin-packed
longest-processing-time-first by measured wall-time
(scripts/ci/proxied_test_durations.json, captured from two bazel-proxied
job failures' testlogs) instead of the bd-init count proxy, via
scripts/ci/gen_proxied_shard_manifest.py's new --weights=duration mode.
Coverage is proven equal to the full discovered test set by two
independent methods (sorted-name diff and an actual 30-shard invocation
of proxied-test-shard.sh in list-only mode), both exact matches with no
duplicates.
cmd/bd:bd_proxied_test's shard_count moves to 30; PR Risk's and
main.yml's own test-proxied-cmd jobs keep their 15-shard matrix and
frozen manifest block untouched, per D2's retirement design (the Bazel
lane only needs to run the same tests, not the same split). Updated the
two coupled policy tests (scripts/ci_workflow_test.go's
TestBazelDoltServerTiersMirrorPRRisk, scripts/pr_risk_bazel_coverage_test.go's
TestBazelRetiredLanesCheckListedTestsRan) to allow the Bazel target's
shard_count to differ from its legacy job's own matrix size, and
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md / engdocs/TESTING.md to document
the split.
Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint on bazel.yml; make bazel-sync-check;
make ci-pr-policy; 3 representative shards (1, 15, 30 of 30) run
directly against Bazel-built race bd_test/bd_for_tests binaries and the
pinned dolt 2.2.0 release, all green, 55-81s wall time each (the
duration data's absolute seconds are from a contention-era sample and
run far faster here; the LPT balance itself, not the absolute SLA, is
the property that matters).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): close fork-PR gap in proxied 30-shard manifest validation (review B1/S1)
TestShardScriptsListOnlyRealTests only ran proxied-test-shard.sh at PR
Risk's legacy 15-shard matrix, never at the Bazel-only bazel-proxied
lane's 30-shard manifest block. bazel-proxied only runs in mode=remote
(skipped on fork PRs), so a fork PR could rename, duplicate, or corrupt
a TestProxiedServer*/TestServerMode* manifest entry in the 30-block and
merge green via go test ./scripts/... (which runs on every PR), only
breaking the required lane afterward on main and every same-repo PR.
Fix: the test now computes every shard total it must validate per
script (this job's own PR Risk matrix size, every distinct total found
in that script's own committed manifest, and - for proxied-test-shard.sh
- cmd/bd:bd_proxied_test's live shard_count from BUILD.bazel) and runs
the full per-shard validation loop at each one, relying on the script's
own duplicate/stale-name guards. It also sums each total's "manifest: N"
counts across all shards and fails if they sum to zero, catching a
silently-deleted block that would otherwise fall back to 100% hashing
undetected.
Also fold in S1: replace the independently hard-coded `30` overrides in
ci_workflow_test.go and pr_risk_bazel_coverage_test.go with
bazelProxiedShardCount(t), a new shared helper that reads
cmd/bd:bd_proxied_test's shard_count live from cmd/bd/BUILD.bazel. This
turns the check_shard_coverage.py argument comparison in
pr_risk_bazel_coverage_test.go into a genuine cross-file pin (BUILD.bazel
vs bazel.yml) instead of two literals that could drift together
unnoticed. ci_workflow_test.go's tiers loop no longer asserts
BUILD.bazel's shard_count against a value read from the same file
(tautological); it instead asserts pr-risk.yml's and main.yml's legacy
test-proxied-cmd matrices equal each other, restoring the pin S1 found
removed on this branch relative to base.
Verified against the reviewer's mutations (duplicate entry, bogus
entry, whole-block deletion, stale rename, matrix divergence, BUILD
shard_count drift, bazel.yml arg drift): each now fails go test
./scripts/..., where it previously passed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): add generator --write/--check and freeze legacy manifest block (review S3/N1/N4/N5)
gen_proxied_shard_manifest.py:
- Add --write (rewrite only the target total_shards block in place,
leaving every other block's header/body text untouched) and --check
(regenerate in memory and diff against the committed block, exiting
1 with the exact --write command needed if stale). Both operate via
new split_blocks()/write_block()/check_block() helpers that round-trip
the real manifest file losslessly.
- N1: fix the duration-weights header's false claim about the
fallback ratio.
- N5: warn on stderr when proxied_test_durations.json has duration
captures for tests that no longer exist in cmd/bd/*_test.go (renamed,
split, or deleted since the last capture).
proxied-cmd-test-shards.txt:
- N4: mark the legacy 15-shard block's header as frozen for
pr-risk.yml's/main.yml's legacy fork/push jobs: it is deliberately
not kept in sync with newly-added tests (those hash-fallback
instead), so regenerating it is not routine maintenance. Only the
header text changed; all 149 explicit assignment lines and the
"known blind spots" callouts are untouched (verified: a fresh
--write would pack all 164 currently-discovered tests explicitly,
silently changing which shard runs which test for this block, which
is exactly what "frozen" means to avoid).
- Regenerate the Bazel-only 30-shard block's header via
`gen_proxied_shard_manifest.py 30 --weights=duration --write` to
describe --write/--check instead of the old manual-copy-paste
instructions. proxied_test_durations.json has not changed yet (S2
is still in progress), so this run only touched header text; the 164
assignment lines are unchanged.
pr_risk_bazel_coverage_test.go:
- Add TestProxiedShardManifestGeneratorNotStale: runs
gen_proxied_shard_manifest.py 30 --weights=duration --check under
plain `go test` (skipped under `bazel test`, whose runfiles hold
neither the generator's sources nor cmd/bd) so a stale 30-block
(e.g. committed by hand, or left behind after adding tests without
regenerating) fails scripts-go-checks on every PR, including forks.
tools/bazel/equivalence_allowlist.txt: allowlist the new test's guarded
Bazel skip.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): rename 14 undiscovered proxied-gated tests; N2/N3/N6 doc fixes
cmd/bd/*_test.go: 14 tests call requireProxiedServerEnv/
requireSharedProxiedServer but their names did not match the shard
scripts' discovery regex ^Test(ProxiedServer|ServerMode), so no CI lane
ever set BEADS_TEST_PROXIED_SERVER=1 for them and they silently skipped
everywhere (pre-existing gap, not caused by F2; list was in
~/beads-bazel-plan/f2/review/undiscovered_gated.txt).
Fixed by renaming rather than widening discovery: 162 of the 164
already-discovered tests call one of the two gate functions in the first
few lines of their body, so name and gate already correlate almost
perfectly, but the 14 tests' names are each styled differently with no
single regex term that would cover all of them without a hand-maintained
list. Renaming lets the existing discovery mechanism, hash-fallback
behavior, and both shard-coverage policy tests pick them up with zero
other changes. Checked each of the 14 names for cross-references first;
each appeared only in its own file plus at most one doc-comment in a
sibling file, both updated alongside the rename. No collisions with
existing test names.
Discovery went from 164 to 178 tests. Regenerated the Bazel-only 30-shard
block (gen_proxied_shard_manifest.py 30 --weights=duration --write) to
pack the 14 new tests; this is the first --write this round to actually
change assignments, not just header text. The frozen legacy 15-shard
block (N4) was deliberately left alone: the 14 renamed tests now
hash-fallback into it, exactly like every other test added since it was
last generated.
cmd/bd/BUILD.bazel (N6): restore the bd_proxied_test comment sentence
bazel.yml's mirror comment kept but BUILD.bazel's had dropped: PR Risk
builds its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd
twin uses build-artifacts' non-race bd-cmd-test, so this lane is the
stricter of the two.
.github/workflows/bazel.yml, .bazelrc (N2/N3): remove stale timing
language ("30 shards of ~5-10 minutes" / ".bazelrc's leftover "15 shards
of ~10 minutes") never reconciled with this task's own local
representative-shard measurements (55-81s/shard); replaced with the
concurrency argument (30 simultaneous dolt sql-servers don't fit a
GitHub-hosted runner regardless of any single shard's wall time). Also
removes the "within 5s of each other" balance claim (S2): the underlying
duration weights are contaminated by -test.parallel overlap in the
top-level <testsuite time> figures they were derived from; a corrected
re-measurement is in progress (separate commit once it lands).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied duration weights without overlap double-counting (review S2)
scripts/ci/proxied_test_durations.json's 178 entries are now each a
clean, single-job (no concurrent heavy bazel/dolt job on the box),
-test.parallel=1 measurement: a fresh manifest-aligned 30-shard sweep's
own <testsuite time> per top-level test where errors="0" (including
failing-but-complete suites, whose elapsed time is still trustworthy),
plus, for the handful too slow to finish inside Bazel's default
per-target timeout, a `bazel run` leaf-subtest-only sum that never adds
a parallel group's own PASS/FAIL line on top of its already-overlapping
children's lines. An earlier pass at this data summed every PASS/FAIL
line indiscriminately, double-counting parallel subtest groups.
A first attempt at re-measuring the slow stragglers overlapped in
wall-clock time with a second, unrelated heavy sweep and came out 2-3x
inflated versus the same tests measured cleanly; that contaminated data
was discarded in favor of isolated re-runs. TestProxiedServerDep2
initially looked hung during its isolated re-run (15+ minutes with no
PASS/FAIL output and no CPU growth in the top-level test binary) but
was not: each of its subtests does real `bd init --proxied-server`
work in a short-lived child process, and Go buffers a parallel
subtest group's output until the group's last member finishes. Letting
it run to completion (2357.72s) rather than guessing avoided reporting
bad data.
seconds_per_init_fallback is recomputed as the weighted average
seconds-per-init across all 178 measured tests (sum of durations / sum
of inits), for use only when a brand-new test has no entry yet.
Re-packing the 30-shard manifest with this corrected data changes the
shard-balance picture: TestProxiedServerDep2 and TestProxiedServerClose
are each large enough, alone, to dominate their own shard (~2358s and
~1105s locally) while every other shard clusters around 741-876s. The
"within 5s" claim this replaces was never earned by the underlying
data; .github/workflows/bazel.yml now documents the real, uneven
spread and flags that the two outlier shards' local numbers already
exceed the job's 25m step timeout on their own merits (unverified
against RBE's actual per-shard throughput, since this job has only
ever run remote).
Verified: go vet ./scripts/..., go test ./scripts/... (includes the
manifest/coverage policy tests), actionlint, make bazel-sync-check,
make ci-pr-policy all pass. Exact-once coverage re-confirmed: 178
manifest assignments for 178 discovered tests, zero duplicates, zero
gaps.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): apply re-review candidate fixes; revert 2 unready renames (R-B1)
Re-review found the prior round's 14 renames would turn the required
Bazel + legacy 15-shard lanes red once discovered:
- TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix/external-unix and
TestProxiedServerOutageReconnectAcceptanceMatrix/external-unix-socket
built a dolt.sock path under t.TempDir(), which nests Bazel's hermetic
tmp-root plus the (64-byte-truncated) test name deep enough to exceed
sockaddr_un's 108-byte sun_path limit. Both now use a new shortSocketPath
helper (a short os.MkdirTemp("", "bds") directory) instead.
- runProxiedDeadline raced a deadline-watcher goroutine against
exec.Cmd.Start()/Run()'s write to cmd.Process, caught by -race. Replaced
with cmd.Cancel, which exec.CommandContext only invokes after Start, so
the read is ordered after Start's write.
With those two fixes applied, TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix
passes outright. TestProxiedServerOutageReconnectAcceptanceMatrix's race is
fixed, but both its subtests still fail a real "exceeded 3s bound" assertion
against a stopped external upstream -- a genuine latency issue, not a test
bug -- so it is reverted back to its original name
(TestProxiedOutageReconnectAcceptanceMatrix) with a comment explaining why,
pending a bead against the underlying latency.
TestServerModeSharedServeGlobalReportsTheServedDatabase also reverted back
to TestSharedServerModeServeGlobalReportsTheServedDatabase: `bd serve
--global` refuses to auto-apply 4 pending schema migrations (v65->v69) to a
shared/proxied server database and exits 1 with a consent-required usage
message instead of starting, per the #5920 migration-consent guard. Also a
genuine product/test-fixture question, not fixed here; reverted with a
comment, pending a bead.
The other 12 of the original 14 renames are kept; all 12 verified passing
in both lanes (see commits that follow for the manifest update and full
verification).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied weights from parallel=4 CI-identical sweep (R-S1)
The prior round's duration data (committed in 6be9605d9f) was captured at
-test.parallel=1 (serial). That fixed a real double-counting bug, but
serial timings don't model this lane, which always runs -test.parallel=4
(--config=doltserver-proxied's own setting). Re-review measured near-zero
correlation (r=0.08) between the serial-derived predicted shard load and
actual parallel=4 wall time -- e.g. TestProxiedServerRecomputeBlocked was
weighted 812s serially but measured 33s actual at parallel=4.
Replaced with a full fresh capture: `bazel --nohome_rc test
--config=doltserver-proxied //cmd/bd:bd_proxied_test` (local execution
only, the exact config bazel.yml's proxied-server tier uses), 30/30 shards
passing, 0 failures, 0 races. Every one of the 176 discovered tests'
top-level <testsuite time> was captured directly from this run's
bazel-testlogs test.xml files -- no leaf-summation workaround was needed
this time, since none of the 176 hit Bazel's per-target default timeout
running alongside the rest of its shard.
Re-ran `gen_proxied_shard_manifest.py 30 --weights=duration --repack
--write` with these corrected weights: a deliberate full reweight, not an
incremental add, since every weight changed. The new packing clusters all
30 shards within a few seconds of ~355-360s each (176 tests summing to
10712s / 30), a dramatic improvement over the serial data's wrongly
implied two-outlier-shard spread (TestProxiedServerDep2 alone at ~2358s,
TestProxiedServerClose alone at ~1105s -- both artifacts of serial
timing, not real per-shard cost). Updated bazel.yml's paragraph to report
this honestly instead of the prior, false "exceeds 25-minute timeout"
claim, which was itself downstream of the serial/parallel mismatch.
Verified: the 12 tests kept renamed in the previous commit, plus
TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix's socket-path fix,
all pass under the new 30-shard Bazel manifest (this capture run) and
under the legacy 15-shard hash-fallback script (ran the 7 shards their
hash lands in: 2, 3, 4, 5, 8, 10, 11 -- all pass).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): incremental --write, --repack, name-coverage --check (R-S2)
The prior --check compared the committed 30-shard block byte-for-byte
against a fresh from-scratch LPT pack, and --write always did a full
repack. That meant any weight drift (even from proxied_test_durations.json
alone, with no test added/renamed/removed) failed --check, and two
unrelated PRs each adding one proxied test would both force a full
repack and conflict on unrelated shard lines -- re-review demonstrated
one new test moving 73 assignment lines under the old full-repack
behavior.
Redesigned:
- --check now verifies exact-once NAME coverage of the currently
discovered TestProxiedServer*/TestServerMode* set against the
committed block -- missing, stale, or duplicated names fail with an
actionable message -- but does NOT compare shard assignments, so a
weights-only change no longer forces --check to fail.
- --write is now incremental by default: every already-assigned test
keeps its current shard, stale (no longer discovered) names are
dropped, and only newly-discovered names are LPT-placed onto whichever
shard is currently lightest. A full from-scratch LPT pack now requires
the explicit --repack flag (used in the preceding commit for the
deliberate full reweight; ordinary new-test additions should use the
incremental default).
Mutation-tested directly (not just reviewed): verified --check correctly
flags a stale name as "stale/unknown" and a newly added test as "missing"
with the exact --write command to fix it; verified incremental --write
touches only the lines for the added/removed names; verified --repack
--write produces a materially different (full rebalance) diff from the
incremental default.
Updated TestProxiedShardManifestGeneratorNotStale's comment in
pr_risk_bazel_coverage_test.go to describe the new name-coverage-only
--check semantics (it previously described the old byte-identical-packing
behavior; the test's own assertion -- exit code only -- was already
correct and unaffected).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(lint): drop unparam findings in primeGitCmd and expandStep
primeGitCmd's error return was always nil (GH#4927 made every path
infallible but left the signature); drop it and update both call sites.
expandStep's depth parameter was always called with 0 from every caller;
recursion depth is tracked internally by expandStepWith via its own
depth+1 calls on children, so the parameter was dead. Hardcode the 0
start inside expandStep and drop the parameter.
Fixes golangci-lint unparam findings:
cmd/bd/prime.go:306:67: primeGitCmd - result 1 (error) is always nil
internal/formula/expand.go:197:49: expandStep - depth always receives 0
make ci-pr-lint is clean (native + windows/amd64 + darwin/arm64 non-CGO).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(dolt): give each TestBatchApplyContract subtest its own context
TestBatchApplyContract's subtests all shared one 90s context built once
in newDoltBatchApplyFixture. BoundsTheItemCount legitimately needs most
of that budget against the server-Dolt tier (ApplyBatchInTx's create
path calls ExecuteCreate once per item, ~5-8 round trips each, for all
1000 items at the MaxApplyBatchItems bound raised by #7051). Once
earlier subtests had spent part of the shared budget, BoundsTheItemCount
would blow the deadline and every subtest that ran after it in the same
parent would fail too with "context deadline exceeded", masking the
real failure behind unrelated red subtests (nightly run 37089445362,
shard 2/16: BoundsTheItemCount at 78.9s, deadline exceeded).
Give each t.Run its own fresh testContext(t) instead of sharing one,
mirroring openStoreWithOwnBudget's established rationale for the same
bug class (be-gvnsq). The store and its copy-on-write branch stay
shared across subtests, and subtests still run in sequence; only the
per-subtest deadline is decoupled. No production code changed: the
underlying per-item round-trip cost is genuine, documented, deliberate
behavior (see applyCreate's doc comment), and batching it safely would
be a larger, riskier change than this CI flake warrants.
Verified via bazel --nohome_rc test --config=doltserver-integration
//internal/storage/dolt:dolt_server_full_test --test_filter=TestBatchApplyContract
-t- --test_output=all: shard 2/16 (same shard as the original failure)
now passes, TestBatchApplyContract in 77.68s with BoundsTheItemCount at
66.08s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(embeddeddolt): skip redundant 1000-item wall-clock case under -race
TestLargeBatchApplyWallClock_Embedded's 1000-item shape was taking
597-651s under --config=embedded (-race), inflated by race-instrumenting
the in-process Dolt engine's own internal goroutine/lock machinery on
every statement issueops.ApplyBatchInTx issues. The Dolt server (TCP)
tier doesn't pay this cost: its sql-server runs in a separate,
non-raced subprocess (dolt_race_off), but the embedded backend runs
Dolt entirely inside this test binary, so -race instruments all of it.
This pushed the embedded-storage CI lane's shard 5 over its 19-minute
per-shard test timeout (panic: test timed out after 19m0s).
The 1000-item case carries no pass/fail assertion on duration or count
(the test is explicitly a logging-only measurement, not a regression
gate), so skipping it under race removes no coverage. Real 1000-item
(issueops.MaxApplyBatchItems) correctness coverage is unaffected: it
still runs and is asserted by TestBatchApplyContract/BoundsTheItemCount
in this same package, under the same race lane.
Adds the established raceEnabled build-tag pair (see
internal/workapi and cmd/bd for the sibling idiom) to this package so
the skip can be conditioned on -race.
Verified via Bazel (--config=embedded, --nohome_rc):
TestLargeBatchApplyWallClock_Embedded: 651.23s -> 86.79s
(1000 now SKIPped; 356 still runs and passes)
Shard 5 of embeddeddolt_embedded_test: FAIL -> PASS
Not fixed in this commit: TestBatchApplyContract/BoundsTheItemCount's
embedded instantiation still takes ~903-945s under -race and leaves
shard 4 of embeddeddolt_embedded_test over its 19-minute budget. Unlike
the wall-clock test, that subtest carries real correctness assertions
via the shared backend/conformance.RunBatchApplyBoundsTheItemCount, and
shrinking it would require a cross-backend conformance-test design
decision (e.g. a per-fixture item-count override) that trades away real
MaxApplyBatchItems boundary coverage for one backend - judged out of
scope for an isolated test-only fix here. Flagging for a deliberate
follow-up rather than weakening the assertion to make it fast.
TestLargeBatchApplyStatementCounts_Embedded and the dolt-tier
TestLargeBatchApplyStatementCounts_Dolt were checked and left
untouched: neither includes the 1000-item shape, neither showed any
slowness or flakiness, and both pin exact statement-count regression
baselines that should not be disturbed without cause.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cmd/bd): fence test-binary beads/git discovery against ancestor .beads
cmd/bd's test binary ambiently discovers its beads workspace the same way
production bd does: via internal/beads.FindBeadsDir's worktree-fallback and
ancestor-walk logic. When this package's checkout is itself a linked git
worktree (e.g. a sibling checkout like beads-bazel-y next to beads), the
first cached git-context call made anywhere in the test binary - before any
individual test has had a chance to chdir into its own isolated fixture -
permanently answers isWorktree=true/a real --git-common-dir for the rest of
the process. beads.FindBeadsDir then treats that as license to read (and
tests that don't isolate properly could write) the main checkout's shared
.beads database.
Confirmed via direct reproduction in a disposable scratch main+worktree pair
under /data/tmp (never /data/projects/beads): running a correctly-built `bd
--version` from the worktree checkout's cmd/bd directory resolves and warns
about the fake main checkout's .beads permissions, demonstrating the
fallback is reachable from plain ambient discovery, not just from a
contrived unit test.
internal/git.PinNoRepositoryForTesting seeds the process-wide git context
cache with a "not a git repository" sentinel as the first statement of
testMainInner, making the worktree-fallback vector inert by default. Tests
that need real git behavior for a fixture they control already call
git.ResetCaches/beads.ResetCaches after chdir'ing there (the existing
cmd/bd/git_test_helpers.go idiom), which overrides the pin for their own
scope - this is unchanged.
Separately, testMainInner now appends (never overwrites) a
BEADS_CEILING_DIRECTORIES entry for the package's own repo root, bounding
any literal ancestor-directory walk that starts at or under this checkout -
closing a second vector (an ancestor walk escaping past a nested worktree
root into a directory-with-.beads further up) that the pin alone doesn't
address. The existing pollution guard is extended to also watch the
worktree-fallback .beads directory (resolved via a direct, cache-independent
git shell-out) as a defense-in-depth detection backstop.
Verified via two full before/after `go test -tags "cgo integration" -v .`
runs of cmd/bd in a disposable linked-worktree scratch clone: identical
pass/fail sets before and after (no regressions), and zero filesystem
writes observed (inotify-monitored) in the scratch main checkout's .beads
in either run - the existing subprocess test helpers already discipline
themselves to set an explicit cmd.Dir/BEADS_DIR, so no currently-live test
was caught actively polluting the fallback directory in this repro. This
change is defense-in-depth against the mechanism itself (and against
regressions reintroducing it), rather than a fix for an empirically
isolated failing test in the current suite.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): make embedded-dolt and go:embed tests discoverable by their CI shards
embedded-test-shard.sh discovers cmd/bd's embedded-dolt tier by grepping
`^func TestEmbedded` out of cmd/bd/*_embedded_test.go — a filename-glob
match, not a package-wide one. 11 TestEmbedded* functions lived in files
that didn't match that glob (auto_import_upgrade_test.go,
dep_list_cross_db_test.go, and four files mixing TestEmbedded* with
unrelated TestProxiedServer*/TestServerMode*/TestWithFetchOneExtra_*
siblings: max_rows_test.go, store_factory_test.go,
purge_wisps_plane_integration_test.go, types_create_agreement_test.go).
They compiled and would run under a plain `go test`, but were invisible
to every CI lane that drives this tier off that discovery (the 20-shard
embedded cmd matrix), so they never ran in CI at all.
Fixed by relocating them into *_embedded_test.go files: a straight
`git mv` for the two files that were purely TestEmbedded* content, and a
function-level split for the four mixed files (moving just the
TestEmbedded* function itself; shared helpers/types stay put since
Go doesn't care which file in a package defines a symbol). The newly
discoverable tests aren't in embedded-cmd-test-shards.txt — consistent
with the ~55 pre-existing discovered-but-unlisted tests already in this
suite, they fall back to the manifest's documented hash(name)%shards
assignment. Verified via `BEADS_TEST_SHARD_LIST_ONLY=1` across all 20
shards: all 212 tests (201 + 11) are now listed exactly once, no shard
comes up empty. Also verified for real: built the cmd/bd test binary
and ran all 11 against a prebuilt bd binary (BEADS_TEST_BD_BINARY) from
the repo root, matching how CI invokes the prebuilt CMD_BINARY — all 11
pass.
Separately, 5 functions had a `TestEmbedded` prefix for an unrelated
reason — go:embed'd content (agents_test.go's AGENTS.md template tests,
schema_test.go's migration-source duplicate-version check) — and so were
being silently caught and skipped by the repo-wide `-skip '^TestEmbedded'`
pattern PR Core/main CI lanes use to exclude the Dolt-embedded tier from
./... runs. Renamed to a TestGoEmbed* prefix so they run where every
other unit test in their packages does; no other references to the old
names exist in the repo. Confirmed with `-skip '^TestEmbedded'` locally:
all 5 now run (previously would have been silently skipped).
Repo-wide grep for stray `^func TestEmbedded` outside cmd/bd,
internal/storage/embeddeddolt and internal/storage/dolt now returns only
internal/tracker's TestEmbeddedDoltExternalRefChangedAfterUsesHistoryFastPath,
which is genuinely about the Dolt-embedded backend and correctly named.
cmd/bd/BUILD.bazel regenerated via `bazel run //:gazelle` to reflect the
renames/new files; it also picked up internal/ceiling as a go_test dep,
a sync gap left over from test_repo_beads_guard_test.go's import in the
prior commit on this branch.
Verified: go build/vet (default, -tags cgo) clean; gofmt clean; go test
./scripts/...; bazel --nohome_rc test //scripts:scripts_test; bazel
--nohome_rc build //cmd/bd:bd_embedded_test; bazel --nohome_rc run
//:gazelle (cmd/bd/BUILD.bazel diff only); python3 tools/bazel/go_srcs.py
--check (make bazel-sync-check); make ci-pr-policy; make ci-pr-lint
(gofmt + golangci-lint native/windows/darwin, 0 issues). cmd/bd test
binary runs were built from cmd/bd (not the module root) and executed
with BEADS_TEST_BD_BINARY set to a binary built the same way, to avoid
buildEmbeddedBD's `go build -o X .` resolving "." against the wrong
package when invoked outside `go test`'s own working directory — not a
bug in this change, reproduced identically against an untouched,
already-correctly-named test (TestEmbeddedVersion). /data/projects/beads/.beads
checked before and after: no change beyond the pre-existing 166 untracked
.beads/formulas/*.toml files already present at session start.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(embeddeddolt): shrink BoundsTheItemCount's real apply under -race
TestBatchApplyContract/BoundsTheItemCount's embedded instantiation
measured 903-945s under -race (shard 4's single biggest long pole),
because the in-process cgo Dolt engine's own goroutine/lock machinery
runs inside the race-instrumented binary for a full 1000-item apply.
Every backend's ApplyBatch validates via the same storage.PlanApplyBatch
before opening any transaction, so the refusal half of this subtest is
already O(1) regardless of count, and the exact 1000/1001 boundary is
already pinned, for real and sub-millisecond, by
TestPlanApplyBatchAcceptsTheBoundary in internal/storage/batch_apply_test.go.
That leaves only "does this backend's own write body durably land N
accepted rows" as the expensive, backend-specific thing this subtest's
"at bound" half needs N real items for - and N does not need to be 1000
to prove that.
Split RunBatchApplyBoundsTheItemCount into a thin wrapper (unchanged,
still used by internal/storage/dolt and internal/storage/uow at the
full MaxApplyBatchItems) plus a new RunBatchApplyBoundsTheItemCountAtScale
taking an explicit atBoundItems. Embedded's subtest now applies 150 items
under -race (65.33s locally, down from 749.22s unmodified on the same
host) and the full 1000 otherwise, via the existing raceEnabled idiom.
internal/storage/dolt's non-race BoundsTheItemCount still covers the
shared inner write body at the full 1000-item count every PR. No CI lane
covers embedded's own post-commit wrapper at 1000 items anymore after
this change (the only non-race embeddeddolt build is compile-only and
never executed) - a known, reported, deliberate gap, documented in both
the updated large_batch_apply_measure_test.go comment and the task's
impl-report.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(git): make test-repository pin survive ResetCaches
PinNoRepositoryForTesting wrote its "no repository" sentinel into the
same gitCtx variable ResetCaches() unconditionally clears, so any test
fixture that calls ResetCaches() after the pin is set (runInDir's
cleanup, resetRepoCachesForTest, 25+ call sites) silently un-pins the
whole test binary for every later test. A reviewer's probe test proved
the bypass: GetRepoRoot() returned the real checkout path after a single
runInDir round-trip.
Replace it with PinNoRepositoryUnderForTesting(root), a separate,
directory-scoped pin checked live against os.Getwd() on every
getGitContext() call, before the gitCtxOnce/gitCtx one-shot cache is
touched at all. The pin survives ResetCaches() for any working directory
at-or-under the pinned root, while fixtures that chdir OUTSIDE that root
(the normal t.TempDir() pattern) still get real git detection.
Add TestPinNoRepositoryUnderForTestingSurvivesResetCaches, reproducing
the reviewer's exact repro and proving the fix: pins a real repo root,
confirms unpinned after chdir+ResetCaches outside the root, and confirms
STILL pinned after chdir+ResetCaches back inside the root (mimicking
runInDir's cleanup).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): widen guard fallback coverage, document -C/doctor gap
Two review follow-ups on the test-repo .beads guard:
1. Honest scope documentation: FindBeadsDirFrom/ResolveBeadsDirForRepo's
-C/doctor paths shell out to git directly with the ceiling env var
stripped, so PinNoRepositoryUnderForTesting (see previous commit) does
not fence them. Document this residual gap explicitly rather than
implying full coverage; covering it would need those call sites to
consult the pin too, which is out of scope here.
2. The guard's fallback-.beads watch list only watched metadata.json and
skipped watching entirely when the main .beads directory was absent —
exactly the layout where the worktree fallback is actually in use.
Restructure to check the main and fallback directories independently,
add config.yaml/routes.jsonl to the flat watch list, and add a
recursive snapshot of dolt/ (snapshotDoltTree) so a real pollution
write anywhere under the fallback's Dolt-backed state is caught.
diffSnapshots is generalized to also report newly-appeared keys, since
a recursive tree snapshot can legitimately gain entries between
before/after in ways the old fixed-name flat list never could.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): pin MaxRows embedded tests off their shared race shard
TestEmbeddedMaxRowsNonListPaths and TestEmbeddedMaxRowsList both hashed
to embedded cmd shard 16, which measured 746s wall under -race
(605s + 214s) against a 19-20 minute budget on a 192-core host with
effectively unbounded test.parallel; CI's 4-way parallelism would be
worse. Neither test had ever run under race in CI before this branch
surfaced them to their own shards.
Both tests spawn many independent bd subprocesses per subtest
(bdInit + exec.Command), so their cost is dominated by subtest/subprocess
count rather than a single large-N parameter; buildEmbeddedBD also never
passes -race to the spawned bd binary, so there is no raceEnabled-style
"shrink N" lever analogous to BoundsTheItemCount's that would apply here
without just deleting coverage. Pin them to shards 11 and 13 instead,
the two lightest shards by current manifest+hash-fallback test count
(verified via BEADS_TEST_SHARD_LIST_ONLY=1, which lists shard
composition without executing any tests).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(embeddeddolt): correct coverage comments, range-check, document build tags
- large_batch_apply_measure_test.go's doc comment above
TestLargeBatchApplyWallClock_Embedded had three factual errors: the
150-item sibling subtest does not always run at 150 "race or not" (it's
1000 without -race); the 356-item shape is not "non-race only" (it runs
under race too, as the same comment's next line said); and the
server-Dolt full suite does not run on "every PR" (full_embedded in
.github/scripts/ci-embedded-tier.sh is unconditional on merge_group/push
but conditional on PRs, per pr-risk.yml's gating). Rewrote the comment to
be self-contained (no out-of-repo citation) and accurate, and reworded
the related t.Skip message. Fixed the same "every PR" overstatement in
batch_apply_contract_test.go's BoundsTheItemCount race-branch comment.
- Added a range-check to
conformance.RunBatchApplyBoundsTheItemCountAtScale: atBoundItems must be
1..publicops.MaxApplyBatchItems, since that's the production bound this
contract pins.
- Documented why race_detector_on_test.go/race_detector_off_test.go use
`cgo && race`/`cgo && !race` instead of the plain `race`/`!race` pair
used by internal/workapi and cmd/bd: every consumer of raceEnabled here
is itself cgo-gated, so the matching cgo tag keeps this file out of
non-cgo builds too, where it would otherwise compile with no consumer
and trip an unused-lint finding.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(dolt): give BoundsTheItemCount 3x timeout budget
The server-Dolt tier's BoundsTheItemCount subtest measured 66s against
the package's shared 90s testTimeout — a thin margin on a loaded CI
runner. Give it a dedicated 3x (270s) context.WithTimeout instead of
raising the shared testTimeout constant, since that constant also scales
every other subtest's 5*testTimeout cleanup bound package-wide; a local
override keeps the blast radius to the one expensive subtest.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(ci): run embedded-Dolt batch-apply suite non-race in nightly
The nightly full-test job's main suite (go test -race ... ./...) never
sets BEADS_TEST_EMBEDDED_DOLT, so every skipUnlessEmbeddedDolt-gated test
in internal/storage/embeddeddolt — including TestBatchApplyContract and
the 1000-item case of TestLargeBatchApplyWallClock_Embedded — is skipped
on every CI lane, not just this one. Add a dedicated non-race step so a
real 1000-item apply through the embedded backend's own
version-commit-published-after-the-tx wrapper gets exercised somewhere:
non-race because the embedded engine's own goroutine/lock machinery runs
inside this test binary and the race detector dramatically inflates its
wall-clock (see race_detector_on_test.go), which would defeat the wall-
clock test's purpose and reintroduce the kind of shard-timeout pressure
the MaxRows shard-pin commit just worked around elsewhere.
Verified locally from a fresh /tmp clone (outside any .beads ancestor,
per this task's safety rules):
BEADS_TEST_EMBEDDED_DOLT=1 go test -tags gms_pure_go -run '^(TestBatchApplyContract|TestLargeBatchApplyWallClock_Embedded)$' ./internal/storage/embeddeddolt
PASS, 178.3s total (BoundsTheItemCount 81.06s, WallClock/1000 63.14s),
comfortably inside the job's 45-minute timeout.
Add TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite to
scripts/ci_workflow_test.go, pinning the step's exact run command, its
BEADS_TEST_EMBEDDED_DOLT=1 env, that it runs after the main suite step,
and that it stays non-race.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): retry Go module fetches and scale download timeouts in every mode
A fork run failed Bazel / test on one proxy.golang.org TLS handshake
timeout. gazelle's go_repository fetches modules with the go command
(fetch_repo), which never retries, and Bazel's downloader retry flags do
not reach it. setup-bazel's generated rc now sets
--repo_env=GOPROXY=https://proxy.golang.org|https://proxy.golang.org|direct:
with "|" the go command moves to the next entry on any error, so the proxy
is tried twice and then the module's origin. fetch_repo checks every
module against its go.sum hash whatever the source. Verified with Bazel
9.2.0: an unreachable first entry still fetches all 438 go_deps repos,
and without a fallback the fetch fails.
--http_timeout_scaling=2.0 doubles the timeouts of Bazel's own
downloader (the Go SDK, http_archive), which already retries.
Both flags are key neutral (repository fetching only) and apply in every
mode, because repository fetching always runs on the runner.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the integration lane for fork and Dependabot PRs in cache mode
bazel-integration was remote-only, so fork and Dependabot PRs (mode
cache) had no PR-time integration run at all: main.yml's integration jobs
run only after merge. With rbe-west's anonymous read-only cache, every
action trusted CI already ran is a hit, so the GitHub-hosted runner builds
and tests only what the PR changed.
Simulated in an ubuntu:24.04 container with 4 CPUs and only the generated
rc (no user rc). This setup's action keys match CI's: the ci lane
reproduced the fork run's 6550 processes with 4847 cache hits.
- Main HEAD: 143 s (4656 hits, 12 local, 113 of 115 tests cached).
- A cmd/bd change: 170-290 s.
- Cache closed (cold, all local): about 30 minutes.
- bazel.yml: the lane's if is now mode remote or cache, never local
(rbe=off). The step timeout goes from 45 to 75 minutes (job 85), so a
closed cache (the farm's kill switch) still leaves fork PRs green.
- bazel-gate.sh: mode cache no longer allows BAZEL_INTEGRATION to skip.
Mode local still does.
- New TestBazelCacheModeReachesTheRC runs, end to end, the real decision
step for 22 starts:
- fork, Dependabot and same-repo PRs, push, rbe=cache/off dispatches
and bazel-farm, each with the variable on and unset;
- same-repo runs with and without the secret;
- a fork PR that "has" the secrets, and one passing fork-farm.
For each start it evaluates every lane's if and setup-bazel env, runs
write-bazelrc.sh, and checks bazel-gate.sh's skips. A cache-mode rc
carries --config=fork-cache and the fetch hardening, and no executor,
TLS file, instance, upload, header, BES or masked secret.
- ci_workflow_test.go: the integration if, the run modes, and a
closed-cache floor for the step timeout.
- Comments in bazel.yml, pr.yml and bazel-farm.yml, and the topology
engdoc. The engdoc also covers:
- fork-cache's slow-blob behaviour: 15 s read deadlines trip the
breaker, then "Lost inputs" and one build restart, still green;
- why the endpoint's FindMissingBlobs denial never affects cache-only
clients.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(storage): measure ApplyBatchInTx SQL statement counts (B0)
Adds a driver.Connector-level counting/latency-injecting wrapper
(internal/storage/sqlcount) plus shared plan-request fixtures
(internal/storage/batchfixtures: mol-normalized 356/712-item shapes and a
40-item classic shape) and backend-specific pinning tests + benchmarks
(internal/storage/embeddeddolt, internal/storage/dolt) that measure the
ACTUAL number of SQL statements issueops.ApplyBatchInTx issues for each
shape, on both the embedded and Dolt (TCP) backends.
Measured totals (Prepare+Exec+Query+StmtExec+StmtQuery inside one
transaction): 356 items ~7008-7009, 712 items 14014-14015, 40 items 846,
identical on both backends (expected: statement issuance is driven by Go
application logic, not backend-specific SQL planning). Pinned with a small
(+/-2) tolerance to absorb an observed, host-contention-driven +/-1 Query
drift (Exec is deterministic on every run). This is the regression baseline
slice B2 (a later, lighter fast path) must demonstrably lower.
batchfixtures.UncappedPlan bypasses issueops.MaxApplyBatchItems so this
measurement does not depend on slice B1's cap raise.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(httpapi,issueops): raise batchApply envelope to 1000 items with bounded large-apply handling
POST /v0/beads/issues:batchApply now accepts up to 1000 items (up from 100);
issueops.MaxApplyBatchItems moves accordingly and the request body cap rises
to 16 MiB. The whole plan still applies in exactly ONE transaction. A request
over 100 items runs under a separate, flat, operator-configured whole-run
budget (`bd serve --large-apply-ceiling`, 5 minutes by default) instead of
the ordinary 60s request deadline; a request at or under 100 items is
unaffected and keeps exactly the 60s deadline and 15s commit-retry ceiling it
always had.
Concurrency and shutdown are bounded, not just documented:
- Oversized requests serialize through a dedicated one-wide semaphore, and at
most one caller may wait for that semaphore at a time (a CAS-guarded single
waiter) — a burst of oversized requests is shed with 503 rather than
queuing and starving ordinary traffic behind the general semaphore.
- The run deadline is stamped only once the slot is actually acquired, not
at the time the request arrived.
- Graceful shutdown closes new admissions before computing its drain budget,
then waits out an in-flight large apply up to its own remaining deadline
(plus a few seconds' grace for the response to write); any large apply
still queued for the semaphore when shutdown begins is refused rather than
extending the drain further. A queued caller that disconnects gets a clean
503, never a misclassified context error. `bd serve`'s orchestrator stop
grace must be at least `--large-apply-ceiling` — see
engdocs/SERVE_RUNBOOK.md.
- ApplyBatch's commit-retry budget only extends past the ordinary 15s ceiling
on this explicit large-apply path (via a context marker), with headroom
reserved so the final retry attempt can't be killed mid-commit by the
context's own expiry.
A new `issues.batchApplyLarge` capability token on `GET /v0/beads/context`
lets a client learn whether a given server accepts the raised envelope.
Wall-clock measurement (embedded Dolt, whole-transaction build+commit) backs
the 5-minute ceiling default: 356 items ~12.6s (23.8x headroom), 1000 items
~57.5s (5.2x headroom).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpapi,serve): wire the large-apply retry budget; repair stale pins
Review fixes for the large batchApply envelope, on top of the test
repairs the PR needed against main.
The extended commit-retry budget was never switched on.
internal/storage/uow's retryTxBudget lets a large apply keep retrying a
commit-time conflict past the ordinary 15s ceiling only when its context
carries issueops.WithExtendedRetryBudget's marker, and the docs on both
sides say admitLargeApply sets it, but nothing did: the marker's only
callers were the uow tests, which mark their contexts by hand. A
1000-item apply that hit a conflict still gave up after 15s with most of
its 5-minute ceiling unused. admitLargeApply now marks the run context
it builds, and TestApplyBatchMarksOnlyALargeApplyForTheExtendedRetryBudget
pins the seam over HTTP: marked at 101 and 1000 items, unmarked at 100.
Removing the mark fails the 101 and 1000 cases; marking every request in
handleApplyBatch fails the 100 case.
Smaller fixes from the same review:
- `bd serve --large-apply-ceiling` repeated the 5m default as a literal
of its own, and since bd serve always passes a positive ceiling, that
literal, not httpapi's, was the default every deployment ran. Export
httpapi.DefaultLargeApplyCeiling, default the flag to it, and pin the
flag's 5m0s default in TestServeFlags.
- admitLargeApply's release cancels the run context before clearing
s.largeApplyDeadline, the order acquireLargeApply's doc states; it
cleared the deadline first.
- Drop the batchfixtures test's dead t.Skipf, which waited for the cap
to reach 1000, and correct a graph-counter comment that still cited
the 100-item cap and the TestServeFlags doc line that called the
no-argument server unchanged.
The PR raises issueops.MaxApplyBatchItems from 100 to 1000 and adds
`bd serve --large-apply-ceiling`, but two base-owned tests still pinned
the old surface and went red on every lane that runs them (Bazel / test,
Test (macos-latest)):
- TestServeFlags pins the exact `bd serve` flag list; add
large-apply-ceiling.
- TestPlanApplyBatchRefusesAnUnusableRequest/too_many_items built 101
items and expected "at most 100 items", a request the 1000 cap now
plans successfully. Size the over-cap request and its message from
issueops.MaxApplyBatchItems, as TestPlanApplyBatchAcceptsTheBoundary
already does for the other side of the cap; the cap's value stays
pinned by TestCapBatchApplyLargeTiesAllThreeLimits.
The PR's own TestLargeApplyBurstNeverStarvesSmallRequestsForSemTimeout
also went red in PR Core (-race, -p 4 on a 4-vCPU runner): all eight
small requests took 160-165ms against a 100ms bound. The stall it
guards against lasts semTimeout (5s), while a small request can still
queue briefly behind the refused large requests, each holding a general
sem slot while its body decodes. Bound it at a second, the "want well
under a second" its sibling TestLargeApplyQueueDoesNotStarveOrdinaryRequests
already uses. Pinned to four cores beside 24 busy loops, small requests
measured 183ms mean and 338ms max, and the 1s bound passed 80/80 runs;
with the one-waiter gate in acquireLargeApply disabled the test still
fails 3/3, on its 2s never-completed check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(uow): stop a doomed large-apply retry before ctx cuts it off
retryTxBudget sizes maxElapsed once at ApplyBatch entry, but says nothing
about how long an individual attempt takes. A slow first attempt (e.g.
157s of a large apply's commit) left little of the budget for a retry,
which then got killed mid-transaction by ctx's own deadline instead of
finishing and reporting its own outcome -- surfacing as
context.DeadlineExceeded (isUnavailable excludes it -> a generic 500)
instead of the exhausted-write-conflict result this loop already reports
correctly (-> 503 + Retry-After).
RunTxResultWithin now tracks the previous attempt's duration and, before
starting a new one, stops early with the last real serialization error
when time left on ctx is less than that duration plus a fixed headroom --
but only when publicops.HasExtendedRetryBudget(ctx) is set (the same
marker retryTxBudget itself checks). Applying the look-ahead
unconditionally regressed TestApplyBatchHonorsAnExtendedRetryBudgetFromContext's
unmarked case: ordinary requests have cheap attempts and are meant to run
until their own ctx deadline ends the loop, and the large-apply-sized 5s
headroom was tripping that for any short-ctx caller with a near-instant
failing attempt.
Adds TestRunTxResultWithin_SlowAttemptStopsBeforeADoomedRetry (a 1.2s
commit against a 2s marked ctx must stop after one attempt with its own
serialization error, never DeadlineExceeded, and never touch a second
unit of work) and its control, TestRunTxResultWithin_ShortAttemptStillRetries
(a 50ms attempt against the same shape must still retry and succeed).
Both mutation-checked against the look-ahead condition and its gate.
Also updates the stale "items accepts at most 100 entries" prose on the
applyBatch endpoint description (openapi.v0.yaml) to the large-envelope
wording already used elsewhere in the spec: 1000-item cap behind the
issues.batchApplyLarge capability, large-apply-ceiling deadline extension,
<=100 unaffected. dependencies:count's own "at most 100" lines are
unrelated and untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,serve): fit the 1000-item cap's tests in the storage shard budget
Review fixes for the relaunched #7051, iteration 1.
Raising issueops.MaxApplyBatchItems from 100 to 1000 made the base-owned
TestBatchApplyContract take 604s under -race, against 97s at the old cap
(main run 35862663577). It hashed onto the legacy test-embedded-storage
shard 2/5, beside ~5m of other tests, and that shard hit its 15m budget
twice (PR Risk run 36807191489). PR Risk no longer runs the legacy tier
on same-repo PRs (#7071), but Main's push run and fork PRs still do, so
the merge would have carried the timeout to Main.
Raising the jobs' timeout-minutes alone would not fix it: the shard
script passes the binary its own -test.timeout=15m. So:
- embedded-storage-test-shard.sh passes -test.timeout=20m (and its go
test fallback -timeout 20m), as embedded-test-shard.sh does for
cmd/bd.
- test-embedded-storage in main.yml and pr-risk.yml: timeout-minutes 15
-> 25, the script's 20m plus setup.
- embeddeddolt_embedded_test appends -test.timeout=19m, as
bd_embedded_test does, so Go's timeout, which dumps every goroutine,
fires before rbe-west's 1200s action kill, which does not.
TestBazelEmbeddedJobMirrorsEmbeddedTier now checks the override on
both sharded targets instead of only cmd/bd's, and the PR Risk
coverage test pins the storage target's new args.
- embedded-storage-test-shards.txt places TestBatchApplyContract on
shard 4, which has ~3m of other tests, and the PR's two large-batch
tests, TestLargeBatchApplyStatementCounts_Embedded (247s) and
TestLargeBatchApplyWallClock_Embedded (261s), on shards 2 and 5. From
the PR run's per-test times, the heaviest shard projects to ~13m, down
from shard 2's 15m-plus.
Also correct the stop-grace guidance. For an in-flight large apply,
Serve's drain waits out the apply's remaining deadline plus drainGrace
(5s), so the orchestrator's stop grace must be at least
--large-apply-ceiling plus 5s, as Serve's own doc says. The runbook, the
flag help, the LargeApplyCeiling field doc and the CHANGELOG entry asked
for the ceiling alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): fork PRs use rbe-west's anonymous read-only cache (mode cache)
Fork PRs (unless bazel-farm.yml authorized them), rbe=cache dispatches and
secret-less same-repo runs (Dependabot) now run in mode "cache": local
execution on ubuntu-latest as before, plus .bazelrc's new fork-cache config,
rbe-west's credential-free rbe-cache endpoint (instance oss, reads only).
rbe=off keeps the old local mode without the cache.
- .bazelrc: build:fork-cache. Only key-neutral flags (--jobs, --remote_*), so
fork actions hash like trusted ones. --noremote_upload_local_results, and
--remote_local_fallback plus
--incompatible_remote_local_fallback_for_remote_cache: without both, a
closed endpoint (farm kill switch) fails every action in GetCapabilities.
- setup-bazel: BAZEL_FORK_CACHE=true appends --config=fork-cache; it is
mutually exclusive with the RBE secrets; new output cache.
- bazel.yml: RBE_INPUT_CACHE, mode cache, dispatch option cache; every
setup-bazel step gets BAZEL_FORK_CACHE. Remote and fork-farm paths
unchanged.
- bazel-gate.sh: cache/false is valid and skips what local skips.
- Policy tests: the two public endpoint lines are allowlisted in .bazelrc
only; TestBazelForkCacheConfig pins the fork-cache contents and checks both
fork-cache and remote-exec stay key neutral; decision, gate and rc-writer
tests cover cache mode.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): fork cache gives up on a slow endpoint and uploads no BEP files
Review fixes for the fork read-only cache (mode cache).
- .bazelrc fork-cache: --remote_timeout 60 -> 15 and
--experimental_circuit_breaker_strategy=failure, so a slow (not closed)
endpoint stops being called once too many lookups fail instead of
costing every lookup 60s x retries; --experimental_build_event_upload_strategy=local,
because bazel.yml's --build_event_json_file and --profile otherwise make
Bazel upload test logs and the profile over ByteStream. equivalence.py
and check_testcases.py read test.xml from the local testlogs dir, not
from BEP URIs, so they are unaffected.
- Policy test: parseBazelrcOptions now scans plain command lines too
(lane checks still ignore them), so `build|common --config=fork-cache`
is caught. Outside fork-cache and remote-exec, no line (test: lines
included) may turn uploads on, turn either fallback off, or set a
non-local BEP upload strategy. fork-cache must end with the breaker,
the local BEP strategy and a --remote_timeout of 1-15s; keyNeutralFlag
accepts the breaker and BEP-strategy flags.
- engdoc: the closed-endpoint fallback sentence now also describes the
slow-endpoint path (timeout, breaker). bazel-farm.yml and pr.yml
comments say fork PRs run in mode cache, not plain local.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): reconcile fork cache (mode cache) with the legacy embedded retirement
After the rebase onto D2 step 1 (#7071), two policy tests failed against
mode cache.
- evalRBEExpr learns `inputs.rbe == 'cache'` (bazel.yml's RBE_INPUT_CACHE).
The decision itself is unchanged and right for mode cache: covered never
depends on the mode, forks and Dependabot (now mode cache) keep the legacy
tier, and a covered same-repo PR without the secret (now mode cache, not
local) still turns pr.yml's gate red. The named cases now pin the mode
too, and the "mode alone makes it red" check includes cache.
- TestBazelEmbeddedLaneCannotBeNarrowed treats fork-cache like remote-exec:
a config setup-bazel's rc enables for every command. Its lines (and any
config they reference) are now in use for the lane, so the rc narrowing
regex applies to them, including a forbidden nested --config=. The
setup-bazel scan accepts exactly --config=remote-exec and
--config=fork-cache.
- Comments in bazel.yml, pr.yml, pr-risk.yml and the engdoc that said
missing-secret, fork or Dependabot runs take mode local now say cache.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pr.yml's bazel.yml call now passes integration: "on", and its ci-gate
requires BAZEL_INTEGRATION (bazel-integration's job.status output, a
missing result reading as skipped). The lane stays remote-only:
bazel-gate.sh allows it to skip in modes local (fork and Dependabot PRs,
until the farm has a read-only cache for them) and skip, never in remote.
bazel-farm.yml's advisory call runs it too.
The integration test step gets 45 minutes (job 55) for cold PR compiles;
the policy test keeps it at or above a 20-minute cold compile plus
.bazelrc's test:integration --test_timeout (the 1200s per-action cap).
The integration input (default "on") stays for callers that want the lane
off; pr.yml's call inputs are pinned, and the gate simulation now covers
the lane skipped, failed and cancelled in each mode.
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(dolt): red — defer auto-wake must run for classified-read-only server-mode opens (be-vbhpf)
TestServerModeDeferAutoWake exercises the same defer/wake contract as the
embedded and proxied-server variants, against a real dolt sql-server. Today
the sweep (DoltStore.wakeExpiredDefers) unconditionally skips whenever the
store is opened read-only — including when that read-only-ness comes purely
from command classification (bd ready/bd list are read-only for the CURRENT
project), which is indistinguishable, before the fix, from a strict
--readonly or foreign-project open that must never sweep.
RED: expired_dated_defer_wakes_on_ready and expired_dated_defer_wakes_on_list_ready
fail (issue never returns to open). dateless_defer_never_wakes and
future_dated_defer_stays_hidden pass today too, trivially (no sweep runs at
all pre-fix) — kept as regression guards for the GREEN step, which is where a
unit test proving classifiedRead=false still does NOT sweep belongs, since
that distinction doesn't exist yet at RED.
* feat: green — defer auto-wake runs for classified-read-only server-mode opens (refs be-vbhpf)
bd ready/bd list on a dolt-sql-server-backed store open with the store
classified read-only for command purposes; the lazy defer-wake sweep
silently early-returned before ever running, so expired defers never
woke unless something else opened the store writable first.
Threads a new ClassifiedRead flag through dolt.Config -> DoltStore,
set at the call site to exactly "read-only because of command
classification, not strict --readonly/preview/foreign-project"
(policy.readOnly && !readonlyMode && !previewMode in main.go).
wakeExpiredDefers's guard now delegates to a small extracted
predicate, deferWakeSweepEligible(readOnly, classifiedRead), so the
truth table can be unit-tested without a live store.
* chore: release gate PASS for be-6iglh-gate
* docs(release-gates): record the be-5fjqh rebase onto current main
origin/main advanced 6 commits past this branch's base and PR #6082 went
CONFLICTING; hourly pr-audit filed be-5fjqh. The branch is now rebased onto
0efe0adf5.
The only conflict was cmd/bd/main.go: this branch and e05000d64 (wy-sbgucn)
each append a field to the same dolt.Config literal at the same anchor line.
Resolved keep-both -- PoolReadTimeoutFallback and ClassifiedRead drive
disjoint code paths.
The addendum records the new base and SHAs, patch-id equivalence for the five
files that did not conflict, what was re-verified at the new base (build plus
the diff-owned tests, container-backed), and what was NOT re-run (the
full-suite and ci-pr-policy lanes still refer to the old base).
Agent-Signature: claude-code-claude-opus-5[1m]-unknown-reasoning on behalf of Jim Wordelman
* docs(release-gates): record the be-qjern re-propose rebase and full re-gate
PR #6082 was closed and its branch deleted 2026-09-01 by an unknown actor
under the shared quad341 identity. Operator ruling 2026-09-08: accidental,
not a rejection — re-proposing. This addendum records a second self-rebase
onto current origin/main (zero conflicts this time), the full-suite and
ci-pr-policy re-gate the 09-01 addendum left outstanding (0 FAIL, policy
FAIL attributed to pre-existing be-mrl6j), and re-verification of both
diff-owned container-backed tests at the new HEAD.
refs be-qjern
* test(feat): red — add strict --readonly server-mode subtest to TestServerModeDeferAutoWake (refs be-acsyx)
* docs(release-gates): green — reword container-backed evidence claims, add nit-3 comment (refs be-acsyx)
bee-ghosttrack nit 2: TestDeferWakeSweepEligibleHonorsClassifiedRead is a
pure-function table test, not container-backed like TestServerModeDeferAutoWake
— reworded release-gates/be-6iglh-gate.md criteria 2(4) and 3 accordingly, and
attributed the preview/foreign-project claim to code structure at
cmd/bd/main.go:1485 rather than to that test. PR #6424 body reworded to match
(not part of this commit; edited via gh pr edit).
bee-ghosttrack nit 3 (optional): one-line comment on the unreachable
"classifiedRead without readOnly" table case explaining it documents totality,
not a reachable state.
Full suite: 0 FAIL. Targeted re-verification (TESTCONTAINERS_RYUK_DISABLED=true
to dodge host-load reaper flakiness): TestServerModeDeferAutoWake 5/5 subtests
PASS (12.427s), TestDeferWakeSweepEligibleHonorsClassifiedRead 4/4 subtests
PASS (5.957s).
* chore: remove leaked internal release-gate doc from PR branch
release-gates/be-6iglh-gate.md
Tracked our internal gc-management/beads-rig release-gate checklist and
should never have been committed to this upstream branch -- it's process
residue from our deploy tooling, not part of the change itself.
Per maintainer bee-ghosttrack's request during review of PR #6571. The
root cause (our deploy tooling auto-committing this file) is already fixed
as of gm-qatxl1; this branch predates that fix, so this commit is cleanup
only -- no other content on this branch is touched.
Ref: be-lk7o3
* ci(proxied-shard): discover TestServerMode* entry points
Review finding (major, Test Evidence Quality): TestServerModeDeferAutoWake —
this PR's only end-to-end guard for the classified-read defer-wake fix — was
selected by no CI lane. The two lanes that set BEADS_TEST_PROXIED_SERVER=1
both shard through proxied-test-shard.sh, whose discovery grep matched only
`^func TestProxiedServer`, so the wiring lines at cmd/bd/main.go:1675 and
internal/storage/dolt/store.go:2136 had zero CI-executed coverage: the one
CI-reachable new test pins the pure predicate and cannot catch a mis-wired
flag.
Widen discovery to `^func Test(ProxiedServer|ServerMode)` in the shard script
and in scripts/ci/gen_proxied_shard_manifest.py so the two discovery patterns
stay in step, and widen the shard script's manifest-name validation to match.
That last part is load-bearing, not cosmetic: without it a regenerated
manifest containing a TestServerMode* row is rejected outright ("Invalid
shard manifest line") and the whole lane exits 1.
This also rescues the two pre-existing TestServerModeServe* suites, which
share the identical requireSharedProxiedServer gate and the same shared dolt
sql-server harness and were equally invisible — the unclosed class this PR
was following.
Verified: discovery goes 151 -> 154; a 15-shard BEADS_TEST_SHARD_LIST_ONLY=1
dry run assigns each of the 154 to exactly one shard with no pre-existing
test moved (the three land on shards 4/13/15 via hash fallback); the
pre-change script selects none of the three.
Also clarifies the reviewers' nit: the "foreign-project" exclusion named in
the new comments is not tested at the guard or in the policy expression —
those opens build their own dolt.Config and leave ClassifiedRead false, so
the guarantee is fail-closed by default.
Iteration-2 review (two minors, both non-gating): the fixup's own new prose
asserted two universals that measurement falsifies — the same class as the
nit above, reintroduced in the CI files this fixup added.
* proxied-test-shard.sh claimed "Both families gate on
requireSharedProxiedServer", contradicted two lines earlier by its own
mention of the TestProxiedServerPathHelpers unit test. Transitive gate
census over the 154 discovered entry points: 143 reach
requireSharedProxiedServer, 10 reach the different helper
requireProxiedServerEnv, and TestProxiedServerPathHelpers
(cmd/bd/proxied_server_test.go, bare `package main`, three assert.Equal
calls on path helpers) has no gate and no build tag, so it also runs in
every ordinary `go test ./cmd/bd/` lane. Now scoped to the
TestServerMode* family, which does gate uniformly (3/3), and the
TestProxiedServer* side is described as mixed rather than generalized
over.
* main.yml claimed TestServerMode* discovery means "neither family can be
silently left out of CI". Discovery is a non-recursive cmd/bd/*_test.go
glob: 154 top-level matches versus 156 recursive, the difference being
cmd/bd/doctor/database_test.go:60 and cmd/bd/doctor/dolt_test.go:126.
Now scoped to the top-level entry points, and the comment names the
residual hazard (a container-gated suite added under cmd/bd/<subpkg>/ is
still never selected here) instead of implying it cannot happen.
Both edits are comment-only and behaviorally inert: non-comment lines are
byte-identical in both files, and the 15-shard BEADS_TEST_SHARD_LIST_ONLY=1
sweep is byte-identical before and after (154 assignments, 0 duplicates,
assigned set == discovered set, the three TestServerMode* still at 4/13/15).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* ci(bazel): add defer_wake_server_test.go to cmd/bd go_test srcs
Maintainer CI repair, generated by `make bazel-sync` (gazelle).
`cmd/bd/BUILD.bazel`'s `go_test` srcs is an explicit list, not a glob, so the
new `cmd/bd/defer_wake_server_test.go` was never compiled or run by the Bazel
lane. `Bazel / test` failed two required steps on the pushed head
`0a3f5e0047`:
Go test equivalence:
go test: 105 packages, 8988 top-level tests
bazel: 107 go_test targets, 106 tested; 8987 top-level tests seen
tests go test runs that Bazel did not (1):
cmd/bd TestServerModeDeferAutoWake
EQUIVALENCE: FAIL
BUILD files in sync (gazelle, go_srcs, MODULE.bazel): out of sync
`BAZEL_TEST` is listed in `CI_GATE_REQUIRED` for `CI Gate / Required`
(.github/workflows/pr.yml), so this is a required-lane red, not advisory.
That the missing test is this PR's own headline guard makes the repair squarely
in scope: the PR exists to close exactly this "no CI lane runs it" gap, and its
review fixup closed it for the proxied shard lane. The Bazel lane had the same
gap for a different reason.
Attribution: main's Bazel run at this PR's exact base 43a526b828 concluded
success, so the red is this branch's, not inherited.
Validation:
- `make bazel-sync` produces exactly this one line; a second run is a no-op
(gazelle idempotent, md5 7d1d0549fcd466159451c02181661c78).
- Byte-identical to CI's own `bazel-sync-patch` artifact for this head
(blob pair efec974..6dd76e3).
- srcs-only change: gazelle added no deps entry, so every import the new file
needs was already declared.
- `bazel build //cmd/bd:bd_test` succeeds, and the generated
`bd_test_/testmain.go` now enumerates TestServerModeDeferAutoWake, which is
the row the equivalence check reported missing.
- `tools/bazel/go_srcs.py --check` was already clean, and `bazel mod tidy`
produced no MODULE.bazel drift, so this single line is the whole repair.
No contributor commit is modified and no Go source changes.
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* ci(bazel): remote Bazel lanes for allowlisted fork authors (advisory)
Add bazel-farm.yml, a pull_request_target caller of bazel.yml. It gives the
RBE farm to fork PRs to main whose author and triggering user (sender: the
pusher on synchronize) are both on .github/bazel-farm-allowlist.txt, which
is read from the base commit. The list has julianknutsen, quad341, sjarmak
and csells, the same as gascity's blacksmith allowlist.
- authorize job: sparse checkout of github.sha that holds only the
allowlist and .github/scripts/bazel-farm-authorize.sh; no token on disk,
no secrets, event facts through env only. The script also checks fork vs
base repo, the default branch, the action, and a 40-hex head SHA.
- farm job: calls bazel.yml with contents: read, exactly the four RBE
secrets, checkout-sha pinned to the event's head SHA, and
fork-farm: authorized.
- bazel.yml: new checkout-sha and fork-farm inputs (no-ops for pr.yml and
nightly.yml). Every lane checks out inputs.checkout-sha with
persist-credentials: false. The rbe job runs a fork remotely only for an
authorized pull_request_target call, and then remote or skip, never
local. PR concurrency also cancels on pull_request_target. The
bazel-sync-patch upload is skipped on pull_request_target.
Advisory only: pr.yml is unchanged, so every fork PR still runs the local
Bazel lanes inside the enforced CI Gate / Required.
Policy tests pin the trigger and types, both jobs' conditions, the
allowlist source and contents, the pinned checkout, read-only permissions,
where secrets appear, no expressions in run scripts, per-PR concurrency,
the timeout, and that no workflow_run consumer can pick up a farm run.
TestBazelLaneIsGatedAlongsideLegacy now allows pull_request_target on a
bazel.yml caller for bazel-farm.yml only. 26 mutations were each caught
by these tests (sender check dropped, allowlist read from the PR head,
write permission added, event data interpolated into run, SHA pin dropped,
and others).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): trusted-author fork PR farm design and threat model
Document bazel-farm.yml in the required-check topology:
- the design, and why it is advisory rather than part of the enforced
gate;
- the gate options considered;
- the threat model: allowlist spoofing, sender vs. author, TOCTOU with the
pinned SHA, workflow-file edits, script, label and comment injection,
token scope, artifact poisoning, and farm and Actions cache poisoning;
- what can only be verified on GitHub.
The one capability a listed author gets beyond a same-repo PR is called
out as an accepted risk, with mitigations: pull_request_target runs use
the default branch's Actions cache scope.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): opt the farm's lane checkouts into fork-head checkout (F1)
actions/checkout@3d3c42e5 is v7.0.1, not v6, and v7 refuses to check out
a fork PR's head SHA on pull_request_target unless
allow-unsafe-pr-checkout is true. Every lane of an authorized farm run
therefore failed at checkout.
- Each bazel.yml lane checkout now sets allow-unsafe-pr-checkout to
inputs.fork-farm == 'authorized' && github.event_name ==
'pull_request_target'. That is false for every other caller and event.
- TestBazelWorkflowForkFarmInputs pins that exact expression. It also
fails if the key appears anywhere else in the workflows (so never a
literal true, and never on the authorize checkout).
- Relabel the checkout pin as # v7.0.1 in bazel.yml and bazel-farm.yml.
- Add the opt-in to the engdoc's design section and to the
verify-on-GitHub list.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): verify everything restored from caches in Bazel lanes (F3)
A farm run executes PR code in the default branch's cache scope. GitHub's
own cache is read-only for pull_request_target: GitHub issues a read-only
token, and cache-mode defaults to read. Whether Blacksmith's colocated
cache enforces the same is unverified. So trusted runs no longer execute
anything they restore without verifying it first:
- setup-bazel: Bazelisk's home moves out of the runner cache to a fresh
$RUNNER_TEMP/bazelisk-home, so the Bazel binary is never restored from
a cache. BAZELISK_VERIFY_SHA256 is set to the release sha256 pinned for
the .bazelversion version on each arch (9.2.0 amd64 and arm64). The
wrapper step then re-checks every Bazel binary in that home.
- write-bazelrc.sh: add common --repo_contents_cache=. The repo contents
cache holds extracted repos that Bazel never re-verifies. The
content-addressable repository cache stays, since Bazel re-hashes it on
every hit.
- Runner cache key bumped to bazel-repo-v3 (the content changed).
- bazel-test runs `go mod verify` right after the Go module cache restore.
- Policy tests:
- cache-mode may only be read or none in bazel-farm.yml, bazel.yml and
setup-bazel;
- bazel.yml permissions are exactly contents: read;
- the Bazel sha pin matches .bazelversion, and Bazelisk's home is
outside the cache;
- the binary is verified after download;
- go mod verify follows every module cache restore;
- the rc disables the repo contents cache.
- Engdoc and bazel-farm.yml header: corrected about GitHub's read-only
cache, noted Blacksmith as unknown, and listed the full blast radius
(release.yml hardening tracked separately). The canary design is in the
rollout notes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): farm runs only the author's own fork, on opened/synchronize (F4)
"The sender is listed" did not prove that a listed user chose the head
commit:
- A listed author could open a PR whose head is a third party's fork, and
that fork's owner could push between the author's review and "Create".
- A collaborator on a listed author's fork could push while the PR was
closed or a draft (that push is skipped), and the author's later
"Reopen" or "Ready for review" click would run it with secrets.
Changes:
- The authorize script requires the head repository owner's numeric id to
equal the PR author's (HEAD_OWNER_ID, PR_AUTHOR_ID, passed via env).
- Drop the reopened and ready_for_review triggers, and the script now
refuses those actions too. Drafts already run on opened and
synchronize.
- Tests pin the trigger types and the new env, and add script cases:
- cross-fork head, missing or non-numeric owner id;
- reopened and ready_for_review refused;
- unanchored action (xopenedx);
- base-repo prefix (gastownhall/beads-evil). This covers two mutations
the reviewer found surviving.
- Engdoc: the new checks, and the residual that stays the listed author's
responsibility (commits pushed by fork collaborators).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): match the farm allowlist on numeric user ids (F5)
Logins are mutable: after a rename, anyone can register the old login,
and before this change they would then be listed as both author and
sender.
- The allowlist now holds one numeric user id per line, with the login as
a comment. The ids were looked up with gh api users/<login> --jq .id.
- The authorize script matches PR_AUTHOR_ID and SENDER_ID
(github.event.pull_request.user.id, github.event.sender.id). Logins are
now used only in the log. An allowlist entry that is not a numeric id
fails the job.
- Tests pin the id-to-login table and the SENDER_ID env. Script cases:
- a renamed login re-registered with a new id (author and sender);
- id prefix, suffix and leading zero;
- an id that appears only in a comment;
- listed ids with unlisted logins;
- a PR from another listed user's fork;
- broken allowlists (a login entry, a signed id).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): farm re-run semantics and the add/remove-user procedure (F6)
A re-run replays the original event. It checks out the same pinned head
SHA, which is fine. It also reads the allowlist at the original base
commit, so a user removed from the list still gets the RBE secrets when
one of their old runs is re-run. Only users with write access can
re-run.
- Engdoc: how to add a user.
- Engdoc: how to remove a user. Merge the removal, then cancel and delete
their farm runs (a gh api query by head-repository owner id). For
cause, rotate RBE_TLS_CERT/KEY and have the farm revoke the old
certificate; rotating also closes the re-run window.
- bazel-farm.yml header: points to the procedure.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): name the farm call "Bazel Farm", not "Bazel" (F7)
The farm's check runs were named "Bazel / <lane>", the same as pr.yml's.
Farm results are PR-controlled: PR code runs setup-bazel and can make a
lane exit 0. If a ruleset ever required a "Bazel / ..." check, a farm run
could satisfy it. The farm's checks are now "Bazel Farm / <lane>", and a
test pins the name and requires it to differ from pr.yml's bazel job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): run the farm authorize script in the C locale (F8)
Bash bracket ranges and ${v,,} depend on the locale. In en_US.UTF-8,
[0-9], [0-9a-f] and [A-Za-z] match some non-ASCII characters, and the
Kelvin sign lowercases to "k". This is not exploitable (GitHub ids,
logins and SHAs are ASCII, and the allowlist is maintainer-controlled),
but the script now exports LC_ALL=C.
The script tests now run under en_US.UTF-8 and add cases with non-ASCII
digits in ids and a non-ASCII character in the SHA. Without the export,
the SHA case fails, so the tests catch its removal.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): farm authorize script runs on bash 3.2 (macOS)
${v,,} is bash >= 4; macOS runners' /bin/bash is 3.2, so the script's
tests failed there. Lowercase with tr under LC_ALL=C (ASCII only, as before).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): gate the Bazel lane alongside the legacy jobs (D1)
pr.yml now calls bazel.yml (workflow_call) as its `bazel` job, and its
CI Gate / Required requires the call's aggregate (BAZEL) plus each gated
lane's result (BAZEL_TEST, BAZEL_PURE, BAZEL_EMBEDDED, BAZEL_DOLTSERVER).
No legacy job is removed: PR Core, build-artifacts, the pure-Go/js-wasm
check, domain+uow, contract corpus and pr-risk.yml's embedded tier stay
required (D2 is separate).
- bazel.yml drops its own pull_request/merge_group triggers, so a PR runs
the lane once; push to main, workflow_dispatch and workflow_call stay.
- Every lane records job.status in an always() last step and exports it
as a workflow_call output named after the job, next to the rbe job's
rbe-mode / rbe-enabled.
- .github/scripts/bazel-gate.sh reads the call's rbe-mode/rbe-enabled and
never re-derives the variable/fork/secret condition: mode skip allows
every Bazel id to skip, local only the remote-only BAZEL_EMBEDDED,
remote none; a missing or inconsistent mode fails the gate.
- bazel-integration runs in the PR call but stays advisory (its legacy
counterparts in main.yml run only on push to main): no gate id, and the
gate script excuses the aggregate failure only when that lane reported
failure or cancelled in remote mode; the gated lanes are still checked
on their own ids.
- Callers (pr.yml, nightly.yml) pass exactly the four RBE secrets that
bazel.yml now declares, never `secrets: inherit`.
- Inside pr.yml's run the Bazel bd artifact is bazel-ci-build-artifacts,
so it cannot collide with build-artifacts' ci-build-artifacts upload.
- bazel-autofix.yml now triggers on workflow "PR": the bazel-sync-patch
artifact is uploaded inside the PR run, and "Bazel" runs are no longer
PR events. Its security model is unchanged (base-branch checkout only,
artifact read from the triggering run id, data-only patch).
- Policy tests (review D1 F3/F4/F7):
- TestBazelLaneIsGatedAlongsideLegacy: every bazel.yml job is the rbe
job, a gated lane (bazelLaneGateIDs) or an advisory lane with a
reason; gate ids, env expressions and outputs are derived from that
map; no re-derivation in the gate step or script.
- TestBazelGateSimulation runs pr.yml's actual gate step over mode in
{remote, local, skip} (pull_request and merge_group): the script's
skips equal the ids whose job `if:` is false, and for each lane that
should run, it alone skipped, failed or cancelled turns the gate red
whatever the aggregate says; advisory, aggregate and invalid-mode
cases too.
- TestBazelArtifactNamesUniqueInCallerRuns: no artifact name collides
within a pr.yml or nightly.yml run, following local workflow calls.
- Autofix: workflow_run on "PR", which must upload bazel-sync-patch
itself or through an unconditional local call.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): keep integration out of the PR call; pin the call's inputs (D1 review v2)
- N1: bazel.yml gains a string workflow_call input `integration`
(default "on"); bazel-integration runs only when remote and
`inputs.integration != 'off'`. A string, not a boolean: on push and
dispatch the input is null, and null == false would silently stop the
lane on main. pr.yml passes integration: "off" (main.yml's integration
jobs, its legacy counterpart, run only on push to main). The gate no
longer reads bazel-integration and bazel-gate.sh no longer excuses an
aggregate failure: BAZEL is the call's result, red on a missing or
inconsistent rbe mode.
- N3: pr.yml's call passes exactly build-artifact-name and
integration: "off" (no rbe or other override).
- N4: bazel.yml's concurrency group never uses github.workflow (the
caller's name in a call) and differs from each caller's group for every
event, so the call cannot deadlock against pr.yml.
- N5: no caller of bazel.yml (pr.yml, nightly.yml) may have a
pull_request_target or workflow_run trigger.
- Simulation: lanes' run modes are evaluated with pr.yml's call inputs;
advisory lanes must not run in the PR call, gated lanes must run when
remote.
- engdocs: runbook for main BUILD drift, a farm outage or the RBE cert
expiry (~2027-09-27): fix main / renew, or unset RBE_WEST_WORKERS.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): record the enforced required-gates ruleset and PR Risk's renamed gate
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): gate the proxied-server and server-Dolt storage lanes
bazel-proxied and bazel-server-storage (remote-only, #6940) join the
gated lanes like bazel-embedded: pr.yml's ci-gate requires
BAZEL_PROXIED and BAZEL_SERVER_STORAGE, each read from the lane's
job.status output (always() recorder step, workflow_call output named
after the job). Both run in pr.yml's call (no integration switch).
bazel-gate.sh allows them to skip in mode local (forks, Dependabot:
pr-risk.yml's legacy tiers cover those PRs) and in mode skip, never in
mode remote. The gate simulation derives this from the jobs' ifs, so
each lane skipped, failed or cancelled in remote mode turns the gate
red. Jobs are renamed "proxied-server tier" / "server-Dolt storage tier"
(shown as "Bazel / ..." under pr.yml's call).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(dolt): verify shared cloud auth fixture without a time limit (saa-cgh.107)
Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell
* test(dolt): require cloud routing server in the opted-in full suite (saa-cgh.107)
Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell
* test(dolt): count successfully registered cloud auth cases (saa-cgh.107)
Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell
* test: describe current shared cloud-auth shard fixture (saa-cgh.119)
Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell
* ci(proxied-cmd): green — give the proxied-shard job its own step timeout, refresh manifest (refs be-bruke)
The job-level timeout-minutes was the only backstop for the "Test
proxied-server cmd shard" step, and it counts the "Download binaries"
step too — so a slow download alone could exhaust the job budget and
kill an otherwise-healthy shard's test run with no way to tell the two
failure modes apart. Give the test step its own 16m timeout inside the
existing 30m job budget, and regenerate the bin-packed shard manifest
via scripts/ci/gen_proxied_shard_manifest.py so it stays in sync with
the current TestProxiedServer* set (0 drift, all 15 shards fallback:0).
* ci(proxied-cmd): mirror the shard timeouts onto the main lane, regen the manifest at the adoption base
Adoption review of #6689, iteration 1 — three minor findings, applied here
rather than deferred.
main.yml's `test-proxied-cmd` is the same job as pr-risk.yml's twin — same
proxied-test-shard.sh, same manifest, same total 15 — behind a
download/verify step inside a 20m budget with no step timeout, so the exact
ambiguous-timeout failure mode this PR removes from the PR lane survived on
the post-merge lane, which is where triage happens with nobody watching.
Mirror both edits (job 20m -> 30m, step 16m) and record the ordering at the
step, since it is not obvious from the site that 16m is deliberately below
the script's own -test.timeout=15m plus setup.
The manifest's only record of the cost proxy's two known blind spots lived
in a hand-written comment block the regeneration deleted:
TestProxiedServerListSortRetiresTheWalk seeds 1400 rows and walks the
listing three ways off a single init, and TestProxiedServerServeReadParity
is the CLI/HTTP read-parity oracle whose presence in this gate-required lane
is what makes the anti-drift claim enforced. Re-adding prose to a generated
file would just be deleted again, so it moves into the generator's emitted
header.
Base drift since authoring left the refreshed manifest one test stale —
#6584 added TestProxiedServerBackupRefusedOnExternalTopology, which
hash-distributed to shard 14 — so "0 drift, all 15 shards fallback:0" was
already false on the merge base. Benign (unlisted tests hash-distribute and
still run, and drift removed no name, so the strict parser's stale-entry
hard-fail was unreachable), but the stated deliverable did not hold.
Regenerated at the adoption base: 149 of 149 discovered tests listed,
fallback:0 on every shard, peak load unchanged at 39 estimated inits. That
figure is a point-in-time measurement, not an invariant: nothing in CI diffs
the committed manifest against a regeneration, so it re-drifts the moment
the next TestProxiedServer* lands. A freshness step
(`gen_proxied_shard_manifest.py 15 | diff - <manifest>`) on a lane already
triggered by .github/scripts/* would make it one; that is follow-up scope.
Verified: the generator reproduces the committed manifest byte-identically;
BEADS_TEST_SHARD_LIST_ONLY=1 across all 15 shards assigns all 149 discovered
TestProxiedServer* functions exactly once with no duplicates or orphans;
actionlint clean on both workflows.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* feat(httpapi): serve `bd list`'s order with `sort` on listIssues
The issue listing served one order, `(created_at DESC, id ASC)`, because the
cursor is a keyset position in it. The spec stated the constraint in its own
words: "the sort order is welded to the cursor contract, and a new order needs
new surface." That welding put the cost on every client wanting `bd list`'s
ordering — the only way to get it was to page the whole result set and sort
locally, which is the one cost on this surface that grows with the project.
Measured against a real `bd serve` over 1400 rows at the 200-row page an HTTP
client actually uses: walk-and-sort is 7 requests plus a client-side
comparator, `?sort=priority` is 1. Both are checked against `bd list --json`
row for row, because the claim that matters is not "fewer requests" but "the
same order".
`sort` takes `created` and `priority` and nothing else. Each value is a keyset
contract — a position shape, a strictly-after predicate, an index — not a
display preference, and both keys are provably total (priority and created_at
are non-null, id is unique) so a page boundary inside a run of equal keys
resolves on id. The seven other orders `bd list --sort` takes have no such key:
`id` is natural-numeric and no database expresses it, `updated` moves on every
write, `closed` is nullable, and status/title/type/assignee are mutable and
unindexed. `priority` here is (priority ASC, created_at DESC, id ASC), which is
both `bd list`'s flagless order and what `--sort priority` produces — a stable
priority sort over the created order is the same sequence — so one served order
retires the walk for both.
Absent `sort` still means `created`, permanently: it is the compatibility
contract for every client written before the parameter existed.
THE CURSOR NOW CARRIES ITS ORDER. This is the half that makes the parameter
safe rather than merely useful, and the obvious implementation gets it wrong.
The token was base64 of `{t,i}` with no binding, so a created-order cursor
replayed under `sort=priority` decoded perfectly and was read as a position in
a different total order — a page that skips and duplicates rows, served with a
200, undetectable by the client. Tokens are now `v2` with an explicit order
member, and decode takes the wanted order as a parameter so a mismatch is
unrepresentable at the seam rather than merely detectable past it. Outstanding
`v1` tokens stay readable as the created-order positions they are, so no
traversal in flight has to restart. This does not contradict "the token carries
no filters": filters select the set, the order decides what the position means.
Storage grows one field. IssueFilter.AfterPriority extends the existing
position; the predicate nests the created one rather than restating it, because
the created/id comparison is only reachable at the cursor's own priority.
Flattening it the way the created predicate's leading-bound trick invites looks
equivalent and is not — it re-delivers rows at the cursor's priority created
after it, which the new conformance walk catches against real Dolt.
NO INDEX MIGRATION, by measurement rather than by omission. A composite
(priority, created_at, id) index was tried on Dolt 2.2.0 against the production
predicate, in both the plain ascending and the mixed-direction spelling Dolt's
DDL accepts: the plan is byte-identical in all three cases, keeping the
single-column priority seek and the TopN. It would have bought write
amplification on every insert and every priority update for no measured change.
The sargability guard pins what is achieved — an IndexedTableAccess range on
idx_issues_priority — and records the measurement to redo if Dolt learns to
stream the composite.
Agent-Signature: claude-opus-5
Co-authored-by: CI Bot <ci@beads.test>
* fix(changelog): drop leftover diff3 conflict marker in sort entry
The rebased adopt-PR head left a stray '|||||||' merge-base marker on the first line of the new `sort` CHANGELOG entry (CHANGELOG.md:381), which `git diff --check` flagged as a leftover conflict marker. Remove only the marker line; the resolved entry content is unchanged. Maintainer fixup for PR #5666 adoption review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: CI Bot <ci@beads.test>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* test(ci): red — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)
* feat(ci): green — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)
Add test-server-storage-full as a sibling job to test-server-storage in
pr-risk.yml. build-embedded already compiles every top-level test in
internal/storage/dolt into one binary (embedded-test-binaries); the
existing job only ever runs -test.run '^TestConformance$' against it.
The new job reuses that same artifact and live-server setup with no new
build step, and runs everything else via -test.skip '^TestConformance$'
so previously-stranded tests (TestCreateGuard_*, federation/concurrent
suites, and diff-owned tests such as TestBenchDBPurgeDoesNotLeak on PR
#5792) get a real PASS/FAIL instead of a silent SKIP. Sets
BEADS_TEST_ENV_RUN_DOLT=1 so federation_test.go's own live-server check
fails loudly rather than skipping if this job's setup ever regresses.
Wired into ci-gate: added to needs, CI_GATE_REQUIRED, and the
tier-gated skipped_ok fallback (mirrors test-server-storage's existing
treatment in each).
Also updates two pre-existing meta-tests over pr-risk.yml that pinned
exact snapshots of the file this job necessarily changes:
scripts/ci_capability_selector_test.go's ci-gate authority snapshot,
and scripts/pull_dolt_image_test.go's retry-helper call count
(pr-risk.yml: 2 -> 3, the new job's own Dolt-image-pull step).
* chore: release gate PASS for be-pp7e (PR-CI lane for TestBenchDBPurgeDoesNotLeak)
Deploy gate record for be-pp7e, gating be-aiy5's PR-CI lane addition
(runs TestBenchDBPurgeDoesNotLeak against a live Dolt server instead of
letting checkDolt SKIP it). Review: be-w90n, verdict pass. All 7 gate
criteria PASS; two pre-existing/unrelated failures encountered during
evaluation (be-jy56, be-vf95) were attributed and excluded from the
verdict.
* test(ci): red — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)
TestPRRiskGateReachesFullServerDoltStorageSuite now asserts
pr-risk.yml's test-server-storage-full job is a fail-fast:false matrix
over shard 1..16, with the Test step invoking
server-storage-test-shard.sh, instead of pinning the old single-job
unsharded command. Fails against the current workflow, which still
runs all 1126 tests unsharded in one 15m job (be-extn).
* feat(ci): green — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)
Adds .github/scripts/server-storage-test-shard.sh (mirrors
embedded-storage-test-shard.sh: discovers internal/storage/dolt/*_test.go,
excludes TestConformance, manifest-plus-hash-fallback shard assignment) and
its shard manifest .github/scripts/server-storage-test-shards.txt, pinning
TestCloudAuthCLIRouting (~9.5m) alone on shard 1 and spreading the other
known 10-20s-heavy tests across the rest.
pr-risk.yml's test-server-storage-full job becomes a fail-fast:false matrix
over shard 1..16, Test step now runs server-storage-test-shard.sh instead of
the old unsharded single-job command that died at 256/1126 tests after
15m28s with zero failures (job 95616282508).
Documents the new lane in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md alongside the
existing embedded matrix description.
TestPRRiskGateReachesFullServerDoltStorageSuite now passes. Offline shard
partition verified disjoint and complete across all 1125 discovered tests
(BEADS_TEST_SHARD_LIST_ONLY=1 for i=1..16). (be-extn)
* test(feat): red — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)
* feat: green — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)
* chore: release gate PASS for be-23f3
Extends deploy/be-pp7e-gate (PR gastownhall/beads#5836) with be-gepv's
fix rather than cutting a fresh isolated branch: be-gepv's own bead
text and be-extn's precedent both establish this fix belongs on the
already-open PR. All 7 gate criteria PASS; two repo-wide lint/policy
failures attributed to already-tracked pre-existing issues (be-vf95,
be-jy56), confirmed untouched by this PR lineage.
be-gepv, be-6yo1, be-23f3
* fix(dolt): federation test isolation + panic (be-3c78s)
TestFederationDatabaseIsolation and TestFederationVersionControlAPIs are
federation-prototype tests that predate the suite's shared-server,
branch-per-test isolation architecture (setupTestStore/StartTestBranch)
and were never updated to account for it.
- setupFederationStore hardcoded Database: "beads" for both towns. New()
always connects in server mode against one shared Dolt server for the
whole test binary (TestMain), so Path alone doesn't isolate two stores
— Database does. Both towns silently shared one database, so Alpha
could read Beta's issues and vice versa. Give each town its own
database (beads_test_federation_<prefix>, matching the beads_test AD-01
cleanup-firewall prefix).
- TestFederationDatabaseIsolation's not-found assertions assumed GetIssue
returns (nil, nil) on a miss. It never does — issueops.GetIssueInTx
always returns a wrapped storage.ErrNotFound. Switched to
errors.Is(err, storage.ErrNotFound).
- TestFederationVersionControlAPIs hardcoded Checkout(ctx, "main") to
return to its "starting" branch, but setupTestStore already checks out
a private branch (test-<name>-<hex>) — "main" is a branch shared by
every parallel test in the binary and never has vc-001. That mismatch
is what produced the nil-pointer panic (GetIssue's error was discarded
via `_`, and the resulting nil issue was dereferenced). Capture the
real starting branch via CurrentBranch() right after setupTestStore()
and check both previously-discarded GetIssue errors before
dereferencing.
Neither bug is a production security issue: New(), Checkout(), and
GetIssue() all behave exactly as specified. Both bugs are confined to
this prototype test file's own setup/assertions, which hadn't caught up
to the isolation model the rest of the suite already uses correctly.
Verified via `go build`/`go vet`/`go test -c` (all clean) and running the
compiled binary, which now reaches and cleanly SKIPs both tests (no
Docker locally) instead of the previous isolation failure / panic
surfacing at compile-adjacent runtime. Full GREEN confirmation is via
CI, which has a working Dolt/Docker environment.
Refs be-3c78s
* chore: release gate PASS for be-42tlj
RunSearchPaging now owns these behaviors in the shared suite, and both
backends run it through their own TestConformance -> RunAll. Keeping the
backend-local copies would leave the same scenario asserted at two layers.
The deletions ride the same PR as the block that supersedes them, so no
window exists where a behavior has zero owning proofs.
The five retired embeddeddolt tests also leave the CI shard manifest
(.github/scripts/embedded-storage-test-shards.txt), which is why that file
moves in this commit rather than a follow-up: embedded-storage-test-shard.sh
validates every manifest entry against the discovered test set and exits 1
on a miss, and it does so for every shard, not only the shards owning the
stale names. Nothing is re-added — the script hash-distributes tests that
are not listed, and the manifest's own comment stops citing the retired
TestSearchIssues_MaxRows_* family as a spreading example.
Every retired assertion and its new owner, all under
backend/conformance RunSearchPaging:
internal/storage/dolt/keyset_search_test.go (partial; file kept)
TestSearchIssuesKeysetSameSecondOverflow
one-shot (created_at DESC, id ASC) order
-> KeysetWalkLosesNothingAndRepeatsNothing, one-shot leg
page-size-2 walk: nothing dropped, no id twice, page within Limit
-> KeysetWalkLosesNothingAndRepeatsNothing, walk leg
resume after k-a3 = [k-a4 k-a5 k-older]
-> KeysetResumeAnswersTheStrictlyAfterRows, mid-group leg
ksIDs / ksEqual helpers -> conformance orderedIDs + reflect.DeepEqual
TestSearchIssuesKeysetPlanIsIndexed -> KEPT. EXPLAIN sargability is Dolt
planner residue, outside the observable contract, and cannot move.
internal/storage/embeddeddolt/keyset_search_test.go (file deleted)
TestSearchIssuesKeysetEmbedded — the same two assertions as the dolt
twin above, same owners, reached on this backend through RunAll.
internal/storage/dolt/search_max_rows_test.go (file deleted)
MaxRows_NotExceeded (rows under cap -> all rows, nil)
-> MaxRowsAnswersTheTypedRefusal/UnderTheCapIsAnOrdinaryResult
Exceeded/FlagSource (errors.As, Found>3, Cap=3, Source "--max-rows")
-> .../OverTheCapAnswersTheTypedRefusal/cap3
Exceeded/EnvSource (Cap=2, Source "BEADS_MAX_ROWS")
-> .../OverTheCapAnswersTheTypedRefusal/cap2
MaxRows_Zero_NoCap -> .../ZeroDisablesTheCap
WithLimit/LimitUnderCap_NoError
-> .../ALimitAtOrUnderTheCapCannotFireIt/limit3_cap100
WithLimit/LimitEqualsCap_NoError
-> .../ALimitAtOrUnderTheCapCannotFireIt/limit3_cap3
WithLimit/LimitOverCap_FiresCap -> .../ALimitOverTheCapStillFiresIt
WithLimit/OnlyCap_FiresOnOverage
-> .../OverTheCapAnswersTheTypedRefusal/cap4
internal/storage/embeddeddolt/search_max_rows_test.go (file deleted)
The same eight assertions as its dolt twin, same owners, reached on this
backend through RunAll.
The shared cases are stronger than what they retire on three counts: the
retired cap tests asserted only len(results), where the new legs assert the
whole id set; the at-the-cap boundary (cap == matching rows) was never
covered; and the AfterID:"" group start and filter composition had no
backend-local copy at all.
Co-Authored-By: Claude Fable <noreply@anthropic.com>
Slice 1 of a test-deduplication pass against the conformance contracts. Net
-295 LOC: 8 test functions deleted, 4 files removed, 3 cases ported into
contracts, and — after adversarial review — one residue kept that the deletion
should not have taken.
Every deletion is backed by an assertion-targeted mutation showing the
contract goes red on the same break: blocker refusal, open-children refusal,
CheckVersionInTx on close and update, assignee guard, status guard, LabelPatch
ordering, ParentID replace-all, configured-done resolution, priority-range
validation. update_guards_test.go was additionally found partly cannot-fail —
its only ptr("") leg runs against a fresh unassigned issue and asserts
success, so it could not fail if ptr("") degraded to no guard.
Three cases ported to three legs. The first is the find of the slice: the
Lifecycle contract was green on three backends while EVERY case seeded a
DIRECT blocker, so a guard reading the denormalized is_blocked column instead
of the live blocker list passed all of them. That is 903cbe031, the Deleter
defect, verbatim in another contract.
An adversarial review council then found three things no gate could see — go
build, go vet, go test and golangci-lint were green over all of them:
The deletion orphaned five entries in the CI shard manifest, and the shard
script hard-fails on any entry that no longer matches a discovered test.
All five embedded-storage shard jobs would have exited 1 at startup.
The deletion removed the only observers of a public-API wrapper route.
EmbeddedDoltStore.{Update,Close}IssueChecked are a separate composition of
the shared bodies; disabling the wrapper's CAS branch failed the deleted
test and passed all 136 contract cases. A narrow wrapper-routing residue is
restored, verified red against that mutant.
The metadata port did not pin the Merge<Set order its own name leads with.
Its only Merge/Set collision was also in Unset, so a reordered body produced
an identical document and passed all 74 Update cases. Fixed with one key
written by both stages and removed by neither.
Kept deliberately: comments_page (live seam with no role — a uow provider is
not a DoltStorage), counts_wisp (Counter asks a different question),
reopen_category (seeds the wisp plane and the normalized custom_statuses
table), create_issue (defines newTestEnv/skipUnlessEmbeddedDolt for 58
siblings including all 26 conformance runners), and
issue_operations_persistence (the only file with no skipUnlessEmbeddedDolt
gate).
From lion's #5361 review: formatMemoriesForPrime called
ensureStoreActiveForPrime with no proxied gate — the bd-m7zzd direct-store
seam class, in a read-only limb. prime sits in noDbCommands, so no UOW
provider was ever opened; the lazy direct open then hit the store factory's
proxied refusal and the silent-skip contract swallowed it, meaning proxied
prime never carried memories at all.
The memory read now routes through the proxied plane (routing chosen over
skipping — the surface exists: ConfigUseCase().GetAllConfig on one
read-only UOW, same verb as the bd memories proxied dual), via a provider
opened scoped to the read and closed after. Prime's contracts are
preserved: silent skip when the plane is unavailable, the timeout banner on
a deadline, and memories rendered through a tail now shared with the
classic path (renderPrimeMemoriesFromAllConfig) so the two cannot drift.
Tests: TestProxiedServerPrime journey suite on the shared testcontainers
harness — memories-only and full prime carry the remembered key/content
(red before this fix: proxied prime printed 'No memories stored'), the read
lands zero Dolt commits, no embedded store dir materializes, and a corrupt
sidecar degrades to exit-0 silent skip. Shard manifest entry added at the
hash-fallback shard (6).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(unclaim): port --if-assignee to proxied-server mode (bd-tb0yi)
`bd unclaim --if-assignee` was refused outright under
BEADS_DOLT_PROXIED_SERVER ("not supported in proxied-server mode"); it now
runs the compare-and-swap release, and runs the SAME one the embedded
backend runs.
No new SQL is written for this port, and that is what holds the bar. The
conditional release reaches the classic implementation exactly as the
unconditional one already does: domain.IssueUseCase.UnclaimIfAssignee ->
IssueSQLRepository.UnclaimIssueIfAssignee ->
issueops.UnclaimIssueIfAssigneeInTx, the twin of the Unclaim ->
UnclaimIssue -> UnclaimIssueInTx chain beside it. So a conditional release
applies the same transition (assignee cleared, status reopened, started_at
cleared, lease row dropped, row_lock rewritten), records the same
"unclaimed" event, routes a wisp's claim to the wisp tables the same way,
and refuses with the same storage.ErrAssigneeMismatch naming the current
holder. Routing it instead through Lifecycle.Update + ExpectedAssignee
would have written the row correctly but recorded a generic update event
in place of the release, and diverged from the unconditional path inside
the same command.
The CLI passes the expectation as a plain string because the flag has only
two states here: `--if-assignee ""` is rejected in unclaim.go before any
issue is touched (an unset shell variable must never silently downgrade a
CAS to an unconditional release), so empty unambiguously means
unconditional.
EXIT CONTRACT, unchanged and deliberately not "improved": a mismatched
holder writes nothing and exits 1 on BOTH paths. `bd unclaim` has never had
`bd update`'s ExitGuardMismatch (13) verdict — its help says so outright
("1 when any release failed (including an --if-assignee mismatch)") — and
minting one on the proxied path only would be the exact divergence this
lane exists to prevent. If unclaim should adopt 13, it adopts it in both
modes, as its own change.
TestProxiedServerUnclaimIfAssigneeParity is the cross-mode oracle: six
steps (stale guard, matching guard, repeat release, `--if-assignee ""`
rejection, --force/--if-assignee rejection) run against an embedded
workspace and a proxied one, and every observable — exit code, post-step
row state, whether the refusal names the holder — must match, with each
mode also checked against the contract itself so parity with a shared bug
still fails. It rides a new cross-mode harness
(cross_mode_parity_harness_test.go): one initialized workspace per backend,
the same argv against both, outcomes compared field by field and never by
id.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014zUQg5iF119KbGeYuhXQ5Y
* feat(batch): port bd batch to proxied-server mode (bd-tb0yi)
`bd batch` was refused outright under BEADS_DOLT_PROXIED_SERVER; it now
runs, with the property that is its entire reason to exist intact: N
operations, ONE transaction, ONE commit, and whole-batch rollback on the
first failing line.
The split between the modes is exactly the command's own seam. The
grammar, the tokenizer, the arity checks, the dry-run echo, the
empty-input no-op, the default commit message and every byte of output
stay shared in batch.go — a batch script means the same thing whichever
backend runs it, and a script the parser rejects is rejected before either
backend is reached. What is per-mode is the transaction primitive and the
verbs the ops dispatch against: uow.RunTxResult + the unit of work's use
cases here, `transact` + storage.Transaction there. runBatchOpUOW mirrors
runBatchOp op for op, including its error strings.
RunTxResult, not RunTxEphemeral: a batch writes versioned tables, and the
ephemeral form commits with no Dolt commit (its whole point for the leases
table), which would persist these writes while bypassing Dolt history. The
retry it brings redoes the WHOLE batch in a fresh unit of work on a
serialization failure — re-executed against the winner's committed rows,
never re-committed on a session the server already rolled back — and the
per-attempt result slice means a retried batch reports the attempt that
actually committed.
Plane routing is explicit because it has to be: the classic transaction
methods route an id to the issues or wisps tables themselves, while the
domain exposes the planes as separate verbs. close/update resolve through
workapi.GetIssueOrWisp inside the transaction and call the matching verb;
dep add/remove use the source-routed AddDependencies /
RemoveDependencyBySource, which land each edge in the plane its own source
lives in. `close` is deliberately the UNCHECKED close, matching the classic
batch: `close <id>` in a batch applies no close policy, while an `update`
that crosses into done does — an asymmetry the command's help documents.
TestProxiedServerBatchParity is the cross-mode oracle, on the F0-style
harness: the same script runs against an embedded workspace and a proxied
one, and exit codes, the summary line, the per-line report, and the rows
left behind must match — including the two transactional cases that are
the point (a failing last line undoing the successful lines before it, and
a grammar error writing nothing at all).
TWO KNOWN DIVERGENCES are named at their sites and left for their own
change, because neither changes the rows a batch lands and both are
classic-batch inconsistencies with the CLI verbs rather than artifacts of
the port: an out-of-range `priority=5` is written by the classic batch and
refused by the proxied one (the domain update funnel validates the range,
issueops' transaction path does not; `bd update --priority 5` is refused on
both), and a batched `dep add`/`dep remove` records a dependency history
event here and not there (every domain edge verb emits, the classic batch's
bare transaction method does not — and `bd dep add` at the CLI emits on
both backends, which is what makes the classic batch the outlier).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014zUQg5iF119KbGeYuhXQ5Y
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(recompute-blocked): port to proxied-server mode (bd-04vav)
`bd recompute-blocked` stops refusing proxied-server mode. It is the
mode-independent repair for a stale is_blocked column (bd-6dnrw.37) and
wh-bridge-sync's remedy after a cross-machine merge, so a constellation
running on the proxied plane without it has no way to repair the flag
`bd ready` trusts.
Parity is by construction rather than by assertion: the whole
in-transaction body (dirty-graph guard + fixpoint recompute), the commit
message, and the staging set now live in ONE place,
versioncontrolops.GuardedRecomputeAllBlockedInTx /
BlockedRecomputeCommitMsg / BlockedRecomputeStagedTables, which the
embedded store, the server store and the new proxied route all call. The
CLI's two rendered branches are likewise one function shared by both
routes. Nothing about the repair's semantics is restated per mode; what
differs is only how each mode gets its connection.
The proxied route runs on a PINNED non-transactional connection
(uow.MaintenanceProvider, as compact/gc already do) rather than through
uow.RunTx*, for two reasons specific to this command: the post-commit
DOLT_ADD/DOLT_COMMIT are stored procedures Dolt refuses inside an
explicit transaction, and the uow commit path stages with '-Am' while
this repair must stage `issues` ALONE — a commit whose entire content is
a derived flag has no business sweeping an unrelated dirty working set,
which is the very thing the read-side guard exists to prevent.
StageAndCommit grows an empty-author form for that call: proxied-mode
commits are unauthored by design (uow.Tx.Commit passes no --author), and
an empty '--author' argument would be malformed rather than
unattributed.
admin cleanup stays gated. The 2026-08-04 recon on bd-04vav found it has
no wheelhouse call sites, so porting it would be untested surface area
bought for nobody.
TestProxiedServerRecomputeBlocked is a cross-mode oracle: the same
corrupt-then-repair scenario is replayed against a classic embedded
workspace and a proxied one, and every observable — both rendered
branches and the corrected-row count — must match, with the shared
values additionally pinned so a symmetric regression in both modes still
fails. It also covers what wh-bridge-sync actually depends on
(idempotency, honest exit codes), that a no-op repair mints no Dolt
commit, and that the dirty-graph refusal survived the port.
Part of bd-vxavz (proxied-server capability parity).
* test(delete/prune/purge): cross-mode cascade parity, and a data-loss divergence (bd-04vav)
The verification half of F7. Proxied `bd delete` "always cascades"
(--cascade is refused outright) while classic cascades only when asked,
so the question the constellation needs answered before cutover is not
"is the flag there" but "does the cascade reach anything classic would
have left alone" — on the shapes the wheelhouse actually runs.
TestProxiedServerCascadeParity replays three of them against a classic
embedded workspace AND a proxied one and compares the outcomes, so a
difference is reported as a difference rather than as one mode's test
failing alone: wh-gate-sweep's delivery-ledger purge, the wisp-decay
close + `purge --pattern '*' --force` pair, and a reference-aware durable
`prune` sweep.
FINDING — proxied delete of a gate destroys the bead it gated.
`bd gate create --blocks <target>` records the edge as target
DEPENDS-ON gate, so the gated bead is a DEPENDENT of the gate row. On the
proxied plane wh-gate-sweep's `bd delete <gate> --force` therefore
deletes the target and everything downstream of it; classic leaves both
standing. Measured, not inferred: proxied {gateGone:true,
targetSurvives:false, siblingSurvives:false} vs classic {true, true,
true}. This is a shared-server cutover blocker, and the fix is a design
call on what --cascade should mean in proxied mode (three existing
proxied delete suites deliberately pin today's unconditional cascade), so
it is filed rather than smuggled in here.
The subtest CHARACTERIZES the defect: the classic half pins the correct
behavior, the proxied half pins exactly today's blast radius, and the
comparison is logged loudly. The suite stays green while the divergence
is open and fails the moment the damage changes in EITHER direction —
including when it is fixed, at which point the expectation is flipped to
match classic and the paragraph explaining it goes away.
Wisp decay and prune agree across modes with no divergence: the closed
wisp is reclaimed while a durable closed bead and an open wisp survive
(purge is ephemeral-only and closed-only), and prune protects a closed
bead an open bead still cites.
Part of bd-vxavz (proxied-server capability parity).
Two P1-1 residual proxied-mode seams from the bd-6dnrw.44 re-verification,
both PORTED (each was genuinely small):
- bd dep relate/unrelate: no usesProxiedServer() branch meant an opaque
"cannot resolve issue ID: storage is nil" death. Now routed through new
runRelate/UnrelateProxiedServer duals: one RunTx with a real commit
message per invocation (both directed relates-to edges in one Dolt
commit for relate; both removals in one for unrelate, empty-message
no-op when neither edge existed), classic "issue not found" /
self-relate refusals preserved. Exact canonical ids only, like every
other proxied dep verb.
- bd human list/respond/dismiss/stats: each called ensureStoreActive(),
which in proxied mode lazily opened a DIRECT store, silently bypassing
the proxy. Root cause: "human" sits in noDbCommands for its bare help
screen, so its DB-backed subcommands skipped store init entirely and
direct mode papered over it lazily. The subcommands now fall through to
store init (needsStoreHumanSubcommands, dolt-subcommand pattern) and
branch to proxied duals before the direct-store path: reads on one
read-only UOW, respond/dismiss each ONE RunTxResult (existence +
already-closed preflight, comment + close in the same transaction)
under "bd: human respond/dismiss <id>" commit messages. human moves to
the forbidigo proxied-twin family in .golangci.yml (same filter built
on both routes).
Tests: TestProxiedServerRelate + TestProxiedServerHuman journey suites on
the shared testcontainers harness, incl. dolt_log assertions that each
write invocation lands exactly one commit with its own message, and that
the "storage is nil" death is gone; shard manifest entries added at the
hash-fallback shards (7, 11).
Claude-Session: https://claude.ai/code/session_01JtAkNTf8DGzP1AF3FMuGmm
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Both commands are pure computed reads (epic progress; ready fronts +
cycle detection), so they route through one read-only unit of work via
uow.RunTxRead — no commit message, no Dolt commit, no commandDidWrite —
with the existing uowMolReader adapter satisfying the SwarmStorage seam
(and utils.PartialIDResolverStore) over the UOW use-cases. All traversal
and rendering code is shared with the classic route unchanged.
swarm create and swarm list stay gated with their existing refusals
(no downstream consumer).
TestProxiedServerSwarm builds an identical explicit-ID epic DAG (one
closed, one in_progress+assignee, one ready, one blocked child) in a
proxied project and a classic embedded project, and asserts status +
validate human output byte-for-byte and --json deep-equal across modes,
a planted-cycle validate failure (proxied-only: classic embedded has no
bd sql and dep add cycle-gates, so there is no classic door to plant
one), the create/list refusals, and that the reads advance no Dolt HEAD.
Claude-Session: https://claude.ai/code/session_01JtAkNTf8DGzP1AF3FMuGmm
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(export): port bd export to proxied-server mode (bd-o58fz)
Remove the proxied-server gate from bd export and route the command's
storage reads through an exportSource seam:
- storeExportSource preserves the classic (embedded/direct/server) call
pattern over the store global, verbatim.
- uowExportSource reads through the proxied unit of work's domain use
cases, with the whole read set inside ONE uow.RunTxRead transaction
(read-only, never committed) so the export is a consistent snapshot.
Plane-pinned use cases are re-partitioned to match the classic bulk
loaders: labels/comments partition by the row's own Ephemeral/NoHistory
flags; dependency records already partition internally; dependency
counts sum both planes for every id, as classic does.
Everything downstream of the reads (filtering, sanitizeZeroTime, record
shaping, marshal order, atomicfile, stderr summary) is shared code, which
is what holds the byte-identical cross-mode output bar. Ordering parity
holds because both stacks render ORDER BY from the same sqlbuild.SortDefs
(priority ASC, created_at DESC, id ASC; labels by issue_id,label;
comments by created_at,id; memories sort.Strings).
TestProxiedServerExport builds a rich fixture through the proxied CLI and
asserts flag semantics plus the acceptance oracle: proxied export,
imported into a fresh classic embedded workspace and re-exported, is
byte-identical (default shape and --include-memories shape).
buildOwnerExcludeSet's export_auto.go call site is adapted to the new
seam signature (classic source over the same store global); auto-export
behavior is unchanged and its proxied port stays a separate bead.
export_source.go joins the bulk-movement forbidigo family in
.golangci.yml (it carries export.go's filter, it does not build one).
The import half is NOT in this change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018YmpYVCa5eN6JtG26dJLMs
* fix(export): classify relation planes by table membership, not row flags (bd-o58fz)
Review items on #5351:
1. LoadExportRelations queried the wisp plane for any row carrying
Ephemeral or NoHistory — but a promoted no-history wisp is a durable
issues-table row still flagged NoHistory (promote clears only
Ephemeral), so its durable-plane labels/comments were silently
dropped. Both planes are now queried with the full ID set and merged
(ID-disjointness per the GH#4455 cross-table guard), matching what
the dep-counts leg already did; wisp-plane reads tolerate
IsTableNotExist like that leg too.
2. The byte-identity oracle is blind to symmetric omissions, so the
suite now asserts seeded content directly (labels, comment texts,
comment_count, dependencies, blocks-only dep counts) and adds a
promoted no-history fixture row in its own project. That shape stays
out of the round-trip fixture deliberately: classic IMPORT routes by
the same flag heuristic (issueops.IsWisp) and re-planes the row into
the wisps table, a pre-existing export/import infidelity filed as
bd-r9uce.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NvU6bUda8oQjvG3zoa4uyX
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
bd promote stops refusing proxied-server mode and routes through the UOW
plane via a new cross-plane domain verb: IssueUseCase.PromoteWisp ->
IssueSQLRepository.PromoteFromEphemeral -> the exact classic
issueops.PromoteFromEphemeralInTx body, widened from *sql.Tx to DBTX so
both modes share one implementation (same id preserved, wisp_type
retained, labels/deps/events/comments carried across, inbound
wisp-targeted edges retargeted, blocked state recomputed). The widening
replaces the NewBatchContext call (which needs *sql.Tx) with the same
two config reads it performed internally (ResolveCustomStatusesDetailedInTx
+ CustomStatusNames, ResolveCustomTypesInTx) — verified semantically
identical composition.
Commit semantics: exactly ONE Dolt commit per invocation ("bd: promote
<id>", the classic message), pinned by the journey test. The wisp-side
deletes touch dolt_ignored tables; the one transaction still makes the
wisp delete and issues insert atomic.
One deliberate tightening vs classic (documented divergence): the
promotion comment shares the promote's transaction, so a failed comment
write rolls the promote back instead of degrading to a stderr warning —
atomic beats warn-and-continue.
Success output (text + --json best-effort re-read) extracted into shared
printPromoteResult/promotionComment so the two routes cannot drift.
Journey test TestProxiedServerPromote (4 subtests, sharded 15/8):
row-rewritten-in-place with edges/comments preserved and dolt-commit
count == 1; --json shape vs classic; one-way (purge no longer reclaims,
prune does); non-wisp and missing-id error contracts.
Verification note: TestRecomputeAllBlocked_AllowsDirtyWisps
(internal/storage/dolt) fails identically on the unpatched origin/main
baseline on this machine (FAIL-set diff empty; filed bd-7xtsq) — not
attributable to this diff.
Claude-Session: https://claude.ai/code/session_01JtAkNTf8DGzP1AF3FMuGmm
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
bd heartbeat stops refusing proxied-server mode and routes through the UOW
plane. The port preserves the two properties that define heartbeat:
- ZERO Dolt commits per invocation (bd-lrgn1). The lease write lands in the
dolt_ignored leases table under a new uow.RunTxEphemeral helper, which
commits the transaction with a plain SQL COMMIT (Tx.Commit with an empty
message) instead of CALL DOLT_COMMIT — the proxied analog of the classic
DoltStore.withRetryTx commit discipline. RunTx/RunTxResult semantics are
unchanged (they never call Commit on an empty message); serialization
losers (heartbeat vs reclaim/close on the same lease row) replay exactly
like RunTxResult.
- Timestamp/classification parity with classic mode. The new repo verb
IssueSQLRepository.HeartbeatIssue delegates to the identical classic body,
issueops.HeartbeatIssueInTx — same clock (time.Now().UTC()), same TTL
resolution (issueops.LeaseTTL), same only-current-owner classification
(ErrAlreadyClaimed / ErrNotClaimable), same lease re-arm for hand-doled
claims, same wisp refusal — so classic bd reclaim staleness semantics see
proxied heartbeats identically.
Success output (text + the {"id","status":"heartbeat","owner"} --json shape)
is extracted into renderHeartbeatSuccess and shared by both routes, so the
worker-facing contract cannot drift between modes. Exit code remains the
contract: rc 0 = lease refreshed, nonzero = the lease is gone and the worker
should stop. --actor overrides the ownership-check source in both modes.
Journey test TestProxiedServerHeartbeat (shared testcontainers server,
database-per-test) covers: stale-lease rescue with lease_expires_at /
heartbeat_at advanced and dolt_log count unchanged; reclaim ignoring a
just-heartbeaten lease; --json shape; wrong-owner / unclaimed / closed /
missing refusals; and the lease re-arm INSERT path, also under the
zero-Dolt-commit assertion. Sharded at 15/7 in the proxied test manifest.
No behavioral divergences from classic mode. (Failure-message prefixes gain
the domain-layer wrapping other proxied ports carry; classification
sentinels and exit codes are identical.)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(gate): port create/add-waiter/resolve/show to proxied-server mode (bd-v50ru)
Four gate subcommands stop refusing proxied-server mode and route through
the UOW provider, following the conventions gate check/list already set in
cmd/bd/gate_proxied_server.go:
- gate create: ONE RunTxResult doing target lookup + CreateIssue +
AddDependency under a single commit message ("bd: create gate <id>
blocking <target>") — the direct route's create + add-dep + explicit
store.Commit collapse into one Dolt commit per invocation.
- gate add-waiter: one transaction; an already-registered waiter returns an
empty commit message (deliberate no-commit no-op). Fires the on_update
hook post-commit.
- gate resolve: one transaction (type check + CloseIssue), then audit
LogFieldChange + close hooks post-commit, mirroring gate check's closes.
- gate show: read-only UOW; not-found and read-failure exit nonzero with
distinct messages (fail-closed: an unreachable server is not "no gate").
Output parity is enforced structurally: the direct route's renderers and
gate-create input/issue construction are extracted (renderGateShow,
renderGateCreated, renderGateResolved, renderGateWaiterAdded/Already,
gatherGateCreateInput, buildGateIssue) and both routes call the same code,
so the text downstream scripts parse cannot drift.
gate discover keeps its proxied-mode refusal: it interleaves GitHub CLI
queries with per-gate writes and incremental output, so it is not the pure
routing this port covers.
Also fixes the latent domain/db bug the new journey test caught: the
proxied repository's normalizeUpdateValue never serialized "waiters", so
any waiters update through the proxied backend failed at the SQL driver
("unsupported type []string"). It now JSON-marshals the value exactly as
the embedded path (issueops.updateIssueInTx) always has, with a unit test
pinning the behavior.
New journey test TestProxiedServerGateLifecycle covers the wheelhouse
consumer shapes (gate-id parsing from create output, wh-park bead-gate
create, show markers, list --all --json fields, rc-only resolve,
fail-closed error paths) and is listed in the proxied shard manifest.
Part of bd-vxavz (proxied-server capability parity).
* fix(ci): drop stray duplicate TestProxiedServerDelete shard line (bd-v50ru)
Review blocker on #5346: the duplicate tripped proxied-test-shard.sh's
duplicate check and failed all 15 Proxied Dolt Cmd shards, so
TestProxiedServerGateLifecycle never ran in CI.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NvU6bUda8oQjvG3zoa4uyX
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Remove the "not supported in proxied-server mode" gates from bd kv
(set/get/clear/list) and the memory surface (remember/memories/forget/
recall). Both are thin prefix layers (kv.* / kv.memory.*) over the config
table, so the handlers mirror config_proxied_server.go: one RunTx per
write invocation with a real commit message ("bd: kv set <key>",
"bd: kv clear <key>", "bd: remember <key>", "bd: forget <key>"), reads
via RunTxRead. The proxied path never touches ensureDirectMode; the
classic path keeps it.
Presentation and namespace filtering are extracted into helpers shared
by both paths (printKV*, kvPairsFromConfig, printRemember*/printForget*/
printRecall*, memoriesFromConfig, rememberBareKeyPath), so output —
including the kv list --json shape that the wyvern wheelhouse mail
transport parses — cannot drift between modes. Key validation and the
remember desire-path/command-word guards run before dispatch and behave
identically in both modes. Storage errors pass through %v unmodified so
callers can keep retrying on the 'Merge conflict detected' /
'constraint violation, transaction rolled back' substrings.
Journey tests: TestProxiedServerKV (byte-preserving multi-line/JSON
envelope round-trip, list --json shape + classic-vs-proxied comparison,
idempotent clear, invalid-key refusal, one-Dolt-commit-per-write) and
TestProxiedServerMemory (remember → memories → recall → forget journey,
kv.memory.<key> storage-key check, bare-key desire path, command-word
guard, one-commit-per-remember). Manifest entries added at the shards
the hash fallback already assigns (13/14), so nothing reshuffles.
Refs bd-dnzjy, program bd-vxavz.
Record fail-closed candidate lane selection without changing which PR Risk jobs execute. This creates replayable evidence for capability-owned admission while branch protection and historical validation are still pending.
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot
Co-authored-by: CI Bot <ci@beads.test>
The audit read TestProxiedServerServeReadParity as unenforced on PRs because it
does not appear in pr.yml. It is enforced, in the other PR-time workflow:
pr-risk.yml's test-proxied-cmd job runs `on: pull_request`, discovers every
`^func TestProxiedServer` in cmd/bd/*_test.go, and feeds TEST_PROXIED_CMD into
that workflow's "CI Gate / Required". Its concurrency group is keyed on the PR
number, so it is not the perpetually-cancelled Main workflow. Verified by running
the lane's own discovery across all fifteen shards: the oracle appears in exactly
one.
What was true is that it appeared there by HASH FALLBACK. The manifest header
asks for a regenerate after adding a TestProxiedServer* function, and neither
`bd serve` suite was ever added, so the assignment was incidental rather than
reviewed and nothing in the file said the parity oracle was in the lane at all.
Both suites are listed at the shard the hash was already choosing (8), which is
zero behaviour change and avoids reshuffling the other 100 entries — a full
regenerate rewrites 202 lines across tests this branch never touched. Verified:
shard 8 now reports `manifest: 8, fallback: 0`.
Two further tests remain unlisted (TestProxiedServerCleanDatabasesPurgeDropped,
TestProxiedServerReassignFence). They are not from this branch and still
distribute by hash, which is the supported path; a cost-rebalancing regenerate
is the change that should pick them up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first slice that runs a server. Proxied-server workspaces go live; the
read and claim operations are registered as transitional 501 stubs so route
and spec parity can be proved all at once instead of growing a hole per slice.
`bd serve --addr 127.0.0.1:0 [--allow-non-loopback]`, and no other flags in
v0. The bound address is stdout's first line, because under the ephemeral
default it is the only way to find the port. Shutdown rides the signal context
the root command already installs. No lock file, pid file or discovery file:
serve is operator-invoked and the TCP bind IS the mutual exclusion — dbproxy
needs that machinery only because clients race to auto-spawn it.
The mode gate says two different things, deliberately. Embedded is a PERMANENT
typed storage.ErrUnsupported: there is no unit-of-work provider for that
backend. Dolt server mode is STAGED — the provider construction exists and is
tested, it just is not wired in yet — so the text names what works today and
says this does not, rather than reading as an architectural limit that would
send an operator off to change their topology for nothing.
One unit of work per request, closed through a DETACHED context. The
transaction layer poisons the pinned connection when a ROLLBACK send fails
(doltserver_tx.go), so closing with the request's own canceled context would
burn a session on every client disconnect.
Front-tier hardening lands here, not later, because without it a hung Dolt
becomes a silently growing black hole: slots fill with immortal queries,
/healthz stays green because it never touches the database, and recovery
produces a thundering herd. So: a Host-header allowlist (DNS-rebinding defense
for an unauthenticated, browser-reachable loopback service), a
netutil.LimitListener connection cap (Go spawns a goroutine per connection, so
the database semaphore alone does not bound what is accepted), a database
semaphore with a BOUNDED 10s wait mapping to the already-documented 503 busy,
a 60s per-request deadline, and explicit pool limits on the provider's *sql.DB
via a new uow.PoolTuner knob — openDB set none, and database/sql's default is
unlimited.
/v0/beads/context serializes an explicit ten-field allowlist and nothing else.
SyncRemote is hard-excluded in this and every future version: it is populated
unconditionally from config sync.remote, and remote URLs routinely embed
credentials. The database bind endpoint and the remaining absolute host paths
are excluded too. `capabilities` is derived from IMPLEMENTED handlers only, so
a release cut between this slice and the endpoint slices advertises only what
actually works.
The observability floor is an acceptance criterion, not a convention: one
structured stderr line per request (method, path, status, problem code,
duration, semaphore wait, unit-of-work acquire), startup and shutdown lines, a
semaphore-saturation event, and a request_id on every problem response. That
last one required documenting request_id on the spec's Problem schema — the
only spec edit here, additive, with types regenerated.
Decisions implemented as given: hooks do not fire (a user-controlled
subprocess per mutation is an unbounded latency multiplier and an orphaned
child at shutdown); the per-command auto-commit globals are never touched
(nothing to exclude — proxied PersistentPostRunE only closes the provider);
and actor on an HTTP request stays caller-asserted provenance, which the
context and capabilities wording does not contradict.
Tests: TestSpecRouteParity asserts set equality between the route table and
the document's paths, and builds the real mux so a pattern ServeMux would
refuse fails here rather than on the first request.
TestSpecStatusCodesMatchHandlerTable carries a commented, enumerated 501
exemption that is self-deleting — it fails if the route table and the list
disagree, and fails again if the list outlives the last stub. The context
allowlist is enforced from the handler out, over a body built from a snapshot
with every field populated and a credential-bearing sync remote.
TestWithUOWClosesWithADetachedContext records the close context's state AT
close time, which is the only moment it says anything.
Verified: build, vet and golangci-lint clean (the two gosec findings in
cmd/bd/main.go pre-date this change); internal/httpapi green under -race;
protocol corpus byte-identical with BEADS_PROTOCOL_REQUIRE_DOLT=1; and a real
server started against a proxied workspace, answered both endpoints, refused
a foreign Host, an unknown query key and a second bind, and drained to exit 0
on SIGTERM.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Test (Embedded Dolt Storage) ran the ~84-function internal/storage/embeddeddolt
suite (minus TestConformance) as a single unsharded job, ~24.3min and the
longest pole in every Main/PR-Risk run per engdocs/CI_CLEANUP_PLAN.md and
mybd-t8l8; it timeout-red'd upstream main at its 30m suite budget twice on
2026-07-28 (mybd-jobt).
Extends the existing test-embedded-cmd manifest-driven scheme
(embedded-test-shard.sh + embedded-cmd-test-shards.txt, already 20-way) to the
storage job: new embedded-storage-test-shard.sh discovers top-level Test*
functions in internal/storage/embeddeddolt/*_test.go (excluding
TestConformance, which stays in its own job), consults a committed manifest
(embedded-storage-test-shards.txt) for balance, and hash-falls-back for
newly-added tests. No per-test timing data exists for this suite, so the
manifest assigns the 83 discovered tests round-robin over the alphabetically
sorted list for 5 shards (17/17/17/16/16), which spreads related families
(the 7 TestEmbeddedMergeAndSettle*, ~11 TestEmbeddedMigrat(e|ion)*, 4
TestSearchIssues_MaxRows_*) across shards instead of clustering them.
test-embedded-storage becomes a 5-way matrix job in both main.yml and
pr-risk.yml, per-shard timeout-minutes: 15 and -test.timeout=15m (half the
old 30m budget, matching that each shard carries roughly 1/5 of the tests and
~2-3 of the ~11 migration-chain tests that motivated the original 20m/30m
budget per bd-lrgn1). Job id test-embedded-storage and its BEADS_TEST_EMBEDDED_DOLT
env stay the same, so pr-risk.yml's ci-gate needs/CI_GATE_REQUIRED wiring
(TEST_EMBEDDED_STORAGE) is unaffected by the matrix split; only the job's
display name changes shape, from "Test (Embedded Dolt Storage)" to
"Test (Embedded Dolt Storage N/5)". Updated the two required-check-name
references to that in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md and
engdocs/CI_TEST_SURFACE_AUDIT.md.
Conformance (core/audit, ~14min each, mybd-ae0n) is left alone: TestConformance
is sharded by sub-test path via -test.run '^TestConformance$/^Audit$', a
different mechanism from this manifest's exact top-level function-name
matching, so it does not extend naturally in this commit.
Agent-Signature: claude-fable-5-high on behalf of maphew
Adds the 14 new TestProxiedServer{MolShow,MolCurrent,MolProgress,
MolLastActivity,MolStale,MolReadyGated,MolDistill,Pour,MolBond,
MolSquash,MolBurn,WispCreate,WispList,WispGC} entry points to the
committed 15-shard manifest via scripts/ci/gen_proxied_shard_manifest.py,
so they run with pinned, cost-balanced shard assignments in the
test-proxied-cmd CI job (main.yml and pr-risk.yml) instead of falling
back to hash-based distribution. Verified via
`proxied-test-shard.sh <N> 15` with BEADS_TEST_SHARD_LIST_ONLY=1 across
all 15 shards: every test resolves via the manifest with zero fallback
and no validation errors.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The proxied cmd lane is init-bound: each subtest's bd init runs the full
migration chain, so a shard's wall-time tracks its total init count. The six
heaviest suites (Update 56 inits, Show 47, Create/Close 41, Dep 36, Ready 34)
were each a single top-level function, and a function is the atomic unit of
sharding — so no shard split could pull the worst shard below Update's ~169s
(local), which is what kept brushing the 15m budget on CI.
Split each of the six into 2-3 top-level functions (Update/Show ->3, the rest
->2), balanced by init count, by inserting function boundaries between the
already-independent per-subtest blocks — no test logic moves, every t.Run set
is byte-identical to before. Regenerate the 15-shard manifest by greedily
bin-packing every TestProxiedServer* function by init count
(scripts/ci/gen_proxied_shard_manifest.py); the heaviest shard drops from ~56
to ~31 estimated inits, roughly halving the long pole.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The proxied suites are init-bound (each per-subtest project runs the full
migration chain), so the heaviest shards were brushing the 15m budget and
occasionally timing out. Widen the matrix from 8 to 15 shards in both the
main and pr-risk workflows and rebalance the committed manifest, greedily
bin-packing by estimated cost so the six heaviest suites (Update, Show,
Create, Close, Dep, Ready) each own a shard and the rest pack evenly.
Estimated worst shard ~600s, comfortably under the 900s timeout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-subtest project split multiplied bd init calls (each runs the full
migration chain on a fresh DB): epic 2->8, set-state 1->4, unclaim 2->9,
defer 1->7. That is added, not saved, wall-clock on already-saturated
shards and pushed shard 4/8 over the 15m budget. Revert those five suites
to one shared project per test function (the read-suite pattern), and move
the pre-existing heaviest suite (Ready: 36 subtests / 34 inits) off the
overloaded shard 4 onto the near-idle shard 8.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- beads-release formula: drop the snapshot-docs step (ran the deleted
scripts/snapshot-release-docs.sh on the critical needs-chain); stamp-
changelog now needs update-vendorhash
- update-versions.sh: remove the mangled --skip-docs/docs-snapshot logic
and garbled usage text left by mid-line deletions
- pre-push hook: drop the snapshot-script remediation hint
- release.yml: collapse the prerelease branch; BEADS_REQUIRE_RELEASE_DOCS
has no consumer since check-docs-version.sh was deleted
- RELEASING.md / scripts/docs.md: describe the current pipeline (no
--versioned, no website/ outputs, no check-docs-version.sh)
- docs-mintlify.yml: paths filter now includes engdocs/** and the curated
root markdown files the docsync guard validates
- docs-render-check.sh: pin mint@4.2.687, fail closed when mint dies
without producing a broken-links report, parse report lines only, drop
dead extract_page_links(), fix the stale jq requirement; mint.sh pins
the same version
- nightly.yml: remove the deleted website measurement suite option
- CODEOWNERS: init-safety ADR gate follows the file to engdocs/adr/
- beads-docs skill: align with settled decision 6 — no pointer stubs,
moved pages get docs.json redirects; bd's printed paths are fixed at
the source
- CHANGELOG (edited line), FEDERATION-SETUP, README, examples, oracle-a,
.buildflags, gh-issue-to-pr formula: retarget moved-file references;
anchor the build/ gitignore pattern
- mint.sh + make docs-dev: local preview of the docs/ Mintlify site
(Node 22 handling modeled on the Gas City wrapper).
- test/docsync: Go guard keeping docs/ and docs.json in exact
correspondence — every nav entry has a file, every page is in the nav
(tiny stub allowlist for paths released bd binaries print), published
pages use root-relative extensionless links that resolve, and engdocs/
plus curated root markdown keep exact GitHub-style paths. Wired into
make check-docs.
- .github/workflows/docs-mintlify.yml: docsync + baseline-aware
mint broken-links check on PRs touching docs/**.
- Retire scripts/sync-website-docs.sh and its mirror-drift gate: the six
mirrored files now live once in the Mintlify site; the committed
Docusaurus mirrors stay frozen so the parallel-run site keeps building.
- Repoint inbound links to moved docs across README, guides, examples,
npm-package, plugin resources, and Go comments; add docs/SYNC_CONCEPTS.md
pointer stub (bd's agent templates print that path). Runtime strings in
cmd/bd (prime.go, store_factory_nocgo.go, init_git_hooks.go) still print
old paths — covered by stubs, flagged in the PR.
Converges the sprinkle of -tags=gms_pure_go fixes into a single source plus a source-time CI guard. .buildflags is now canonical (exports GOFLAGS, CGO_ENABLED), scripts source it, scripts/check-build-tags.sh runs in CI to catch bare go commands. Fixes 5 latent bugs the guard caught (regression.yml, Makefile bench/test-regression, devcontainer, embedded-test-shard fallback, smoke-test scripts).