Commit graph

74 commits

Author SHA1 Message Date
Gas City Adopt-PR
567bd8483b ci(pr-risk): retire the embedded tier by committed flag; pr.yml gate fails closed
Review F1. PR Risk's coverage decision read the mutable RBE_WEST_WORKERS
variable in a different run from pr.yml's rbe job, so unsetting it and
re-running one workflow could leave both required checks green with
neither the legacy tier nor the Bazel lane having run the tier.

- Both workflows now commit BAZEL_RETIRES_LEGACY_EMBEDDED (policy-tested
  equal) and run the identical bazel-embedded-coverage job:
  covered = flag && pull_request && !fork && executor secret present.
  RBE_WEST_WORKERS is no longer read by PR Risk at all.
- pr.yml's ci-gate requires the job's result (BAZEL_EMBEDDED_COVERAGE)
  and BAZEL_EMBEDDED_RETIRED, which is red when covered but the Bazel
  embedded lane did not run remotely and pass (mode skip included).
- Revert is a commit of the flag to "false" in both workflows; in-flight
  PRs pick it up on their next push. The kill-switch runbook now flips
  the flag first.

Tests run both decision steps, bazel.yml's rbe step and pr.yml's real
gate step over every event/var/secret/fork/flag combination, with the
variable differing between the two runs, and require no double-green
without the tier.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 07:18:21 +00:00
Gas City Adopt-PR
ff49befd87 docs(ci): record the retired legacy embedded tier in the check topology
Who keeps the legacy tier, how the decision and gate work, the drift
guard, the residual cross-run race, the conformance.yml duplicate, and
how to revert (unset RBE_WEST_WORKERS).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 07:18:21 +00:00
Julian Knutsen
7a21446e6c
ci(bazel): require the integration lane on same-repo PRs (#7069)
pr.yml's bazel.yml call now passes integration: "on", and its ci-gate
requires BAZEL_INTEGRATION (bazel-integration's job.status output, a
missing result reading as skipped). The lane stays remote-only:
bazel-gate.sh allows it to skip in modes local (fork and Dependabot PRs,
until the farm has a read-only cache for them) and skip, never in remote.
bazel-farm.yml's advisory call runs it too.

The integration test step gets 45 minutes (job 55) for cold PR compiles;
the policy test keeps it at or above a 20-minute cold compile plus
.bazelrc's test:integration --test_timeout (the 1200s per-action cap).
The integration input (default "on") stays for callers that want the lane
off; pr.yml's call inputs are pinned, and the gate simulation now covers
the lane skipped, failed and cancelled in each mode.

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 23:56:33 -07:00
ecuthiell
6899813df6
fix(preflight): share checkout lint authority (#5780)
* fix(preflight): share checkout lint authority

Move the Beads repository's native and Windows non-CGO lint policy into a checkout-owned Go driver. The supported Make/Bash wrapper and bd preflight now execute that same authority, while generic Go repositories retain their direct lint contract and formatting is rooted at the detected project.

Fixes #5300.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell

* docs(lint): clarify shared driver scope and timeout limits

Address PR #5780 review 5191123668. Document existing inherited-tag semantics and distinguish the wrapper/preflight driver from standalone workflow and pre-commit callers.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell

* fix(lint): prefer the selected Windows Go toolchain (#6865)

Keep per-pass toolchain requests and deadlines while avoiding the extra auto-selection launcher under the linter. Nonstandard SDKs retain their original PATH.

Agent-Signature: codex-tui-gpt-6-astra-max on behalf of Ewen Cuthiell

* fix(lint): retain PATH for SDK directories containing separators (#6865)

Agent-Signature: codex-tui-gpt-6-astra-max on behalf of Ewen Cuthiell

* test(preflight): declare lint fixture executable for Bazel (#5300)

Agent-Signature: codex-tui-gpt-6-astra-max on behalf of Ewen Cuthiell

* fix(bazel): keep cmd/bd/testdata out of gazelle's package scan

PR #5780 adds the compilable fixture cmd/bd/testdata/preflight-go.go. Left to
itself gazelle generates a BUILD.bazel for cmd/bd/testdata, which makes it its
own package and then (a) empties cmd/bd's data = glob(["testdata/**"]) under the
default allow_empty = False and (b) puts testdata/preflight-go.go across a
package boundary from the preflight_go_fixture target that lists it.

bazel.yml's "BUILD files in sync" step runs `make bazel-sync` and fails on any
resulting drift, and fork PRs always run that job, so this was a live red whose
prescribed remedy ("commit the result") would have broken the build further.
Exclude the directory instead, and take gazelle's ordering for the data list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(preflight): bound the lint deadline and report each check's scope

Review iteration 1 for PR #5780. Synthesis and scorecard both approve (975/1000,
no blocker or major), so this addresses non-gating findings only.

Finding 4 (Resource Lifecycle) - the 13-minute deadline could not bound the call
it guards. CombinedOutput collects through a pipe every descendant inherits, so
Wait kept blocking on a surviving golangci-lint after the context killed the
direct `go run` child. Measured on the same process shape (direct child killed at
1s, descendant holding the pipe for 8s): without WaitDelay the call returned in
8.00s, with WaitDelay in 1.50s. Sets cmd.WaitDelay so the deadline is a real
bound and the "lint check exceeded" line can actually print.

Also gates that line on err != nil, which retires the claude lane's nit (a): a
child that exits cleanly as the deadline fires is a pass, not a timeout. Measured
at the crossover, 68 of 300 passing runs would have carried a false "exceeded"
line; 0 after the gate.

Finding 1 (Change Impact) and nit 7 (Debuggability), one remedy - the PR re-roots
lint and fmt at preflightProjectRoot() but leaves runTestCheck in the caller's
cwd, so a --check run from a subdirectory mixes a repo-wide verdict with a
subtree-only one "with nothing in the output distinguishing them", and the
reported command is only reproducible from the root. CheckResult now carries the
directory each check resolved to, and the report prints it.

Took the synthesis's second arm (state the scope split) rather than re-rooting
runTestCheck, because re-rooting is a behaviour change neither lane priced: it
widens a subdirectory user's test run from 5 packages to 128, including every
Dolt-dependent package, on a repo whose full test gate the PR body already
discloses as red. The stated hazard is that the scopes are indistinguishable,
which reporting the directory discharges directly.

Finding 2 (Architectural Consistency), second clause - adds
TestBuildFlagsContractMatchesShellSource, which parses .buildflags and pins the
driver constants to it. The existing mirror test asserts today's values from the
Go side only, so the files could drift silently; buildEnvironment overrides
rather than merges -tags, so a tag added to .buildflags alone would be dropped
with every test still green. Each of the three legs was mutation-proved to red.

Declined, with the measurements that decided it:

- Finding 2's first clause (route the key comparisons through
  internal/execenv.KeyEqualForOS). The two are provably equivalent today -
  execenv's keyIdentityForWindows is strings.ToLower for the same documented
  reason - so this is de-duplication, not a defect, and the hazard it closes
  (os/exec key-identity semantics changing) is far less live than the
  .buildflags drift the second clause pins. The faithful swap means threading
  goos through five driver functions and ~27 test call sites in this PR's
  centerpiece file, and it would put a non-stdlib dependency into pr-lint_lib,
  which the binary currently does not have. Worth doing on its own footing, not
  inside a non-gating cleanup.

- Nit 6 (parse GOFLAGS into fields instead of a substring test). .buildflags:22
  guards its own append with `!= *gms_pure_go*`, a substring test, and the Go
  code is a deliberate mirror of it. Making only the Go side stricter would
  manufacture exactly the drift finding 2 warns about: on
  GOFLAGS=-tags=gms_pure_go_x the shell would skip its append while the driver
  performed one. The fix belongs to .buildflags and the driver together, and
  .buildflags is outside this PR. The new mirror test now pins that agreement.

- Nit 5 (probe golangci-lint as well as `go`). The claude lane examined the
  LookPath target change and recorded it as deliberate, with the driver's own
  missing-linter error as the mitigation. The suggested alternative - have the
  driver emit preflight's `--skip-lint` hint - puts a bd-specific flag into a
  driver that CI also invokes, where that advice is wrong.

- Finding 3 (two false PR-body claims). Correcting the body is outward-facing
  and belongs to finalize; stamped for it. Swept the in-tree copies: the PR's own
  LINTING.md sentence "Toolchain requests, formatting, and lint deadlines are
  unchanged" is scoped to the Windows SDK-preference paragraph it sits in and is
  true there, so it is graded rather than repaired. CHANGELOG.md contains no
  pr-lint entry, so the remedy's CHANGELOG half has no target in this PR.

Validation: go build, go vet and gofmt clean on the changed packages;
./scripts/pr-lint/ 12 pass / 2 Windows-only skip; ./scripts/prlintmake/ pass;
cmd/bd preflight selector pass; scripts/check-doc-freshness.sh PASSED;
make bazel-sync idempotent and bazel test //scripts/pr-lint:pr-lint_test PASSED.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(prlintmake): inject the pr-lint wrapper's gofmt shim through GOFMT

The merge with #6294 (be-gx8) left runPRLintWrapper relying on a
code path #6294 made deliberately loud. The helper put its gofmt shim
on PATH. Since #6294, fmt-check.sh resolves gofmt through
scripts/ci/gofmt-bin.sh, which ignores a PATH gofmt by design.

Both TestPRLintWrapper* tests still passed, but only via the
resolver's fallback. The go shim answers `go env GOVERSION` and
`go env GOROOT` with nothing, so gofmt-bin.sh warns "could not resolve
a gofmt from go.mod's go1.26.7 toolchain" and falls back to whatever
gofmt is first on PATH, which happened to be the shim. With GOFMT set
in the environment, which is #6294's documented override, the
resolver uses that value instead. Both tests then fail: exit status
127 from fmt-check.sh, and a missing aggregate failure diagnostic
because pr-lint.sh stops before `go run`. At the pre-merge head
(7f37ba5165) the same ambient GOFMT=/nonexistent run passed all 5
tests.

Pass the shim through GOFMT, as main's runFmtCheck does, and keep it
off PATH so only the go shim is found there. gofmt-bin.sh now returns
the override immediately, and the go shim sees only `version <shim>`
and `run ... ./scripts/pr-lint`. Both wrapper assertions, on the
driver arguments and on the failure diagnostic, are unchanged.

go test ./scripts/prlintmake/: 9/9 pass by default and 9/9 pass with
GOFMT=/nonexistent in the environment. Before this change the second
run was 7/9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 22:41:55 -07:00
Jim Wordelman
da89d21813
docs(testutil): document migration-0032 hang and Ryuk-disabled container-leak caveats (#6001)
* docs(testutil): document migration-0032 hang and container-leak caveats (refs be-r7j5x)

Adds a comment near DoltDockerImage documenting two known limitations
of the podman-rootless + testcontainers-go real-server test harness:

- Schema migration 0032 hangs indefinitely over the sql-server wire
  protocol in this harness only; not reachable on real GitHub Actions
  CI or a bare-host-process dolt sql-server (be-j3szz).
- Containers can leak because Ryuk is disabled for this harness, so
  there is no reap safety net if a test process skips its cleanup
  (be-5kkk6).

No production code changes.

* docs(testutil): fix Ryuk-disabled citation and causal claim (refs be-ohrc0)

Round-1 review (be-ohrc0) flagged that the container-leak bullet cited
be-hsa9t for the TESTCONTAINERS_RYUK_DISABLED=true rationale and claimed
it was a workaround for this host's broken dockerd/containerd. Neither
is accurate: be-hsa9t is an unrelated git-remote E2BIG fix that never
mentions Ryuk or dockerd/containerd.

Corrects the citation to be-w3n2m (open, undecided) and the causal claim
to match its actual finding: under rootless podman, Ryuk frequently
cannot start because it wants the runtime socket.

No production code changes.

* docs(testing): move the Dolt container caveats out of DoltDockerImage

Review #6001 asked for this to leave the source file, on three grounds
that all check out.

CONTRIBUTING_PR_GUIDELINES.md line 33 is explicit — "Minimize inline code
comments … let the commit message and PR description carry the why" — and
a 23-line comment-only diff is the shape it names. DoltDockerImage is also
the wrong hook: it is the image pin guarded by test/docsync/doltpin_test.go
and read by container_provider.go and testdoltserver.go, so harness
lifecycle lore attached to it reaches every caller including CI.

The Ryuk sentence was also wrong as written. It claimed Ryuk "is disabled
for this harness (TESTCONTAINERS_RYUK_DISABLED=true)". This repository sets
that variable nowhere — not in Go, scripts/, the Makefile, or
.github/workflows/; the only tracked occurrences are release-gates/*.md,
which are records of operators' local podman runs. CI therefore runs with
Ryuk enabled, and the original text described a local shell as if it were
the repo's harness. Reworded to say what is true: the podman-rootless flow
generally requires disabling Ryuk by hand, and doing so is what removes the
safety net.

Both caveats now live in engdocs/TESTING.md as a sibling of "Test
Environment and Readiness", the section that already covers
BEADS_TEST_SKIP=dolt and BEADS_TEST_ENV_RUN_DOLT. Content is otherwise
preserved, including the be-j3szz / be-w3n2m / be-5kkk6 evidence chain, plus
a stray-container check and a pointer to testMainInner in beads_test.go.
engdocs/TESTING.md is not in check-doc-freshness.sh's DOCS list, so this
needs no inventory or freshness marker.

internal/testutil/testdoltcommon.go is now byte-identical to main.

Addresses the doc half of #6171. Refs be-8n9ns.

* docs(testing): scope the Dolt caveats to podman-rootless and fix the repro

Maintainer review fixes for PR #6001 (adopt-pr review iterations 1 and 2).
All edits are phrase-level and confined to engdocs/TESTING.md.

- Scope the causal claim to the podman-rootless path. CI runs the same
  containerized wire path green on every risk-tier PR -- pr-risk.yml's
  test-server-storage* jobs pull the pinned image precisely to defeat the
  self-skip and run with no BEADS_TEST_SKIP=dolt -- so "the container
  port-forwarding path" as a general cause is refuted by the repo's own CI,
  and as written invited a Docker developer to dismiss a real 0032/initSchema
  hang as a known harness limitation. Softened "Neither is reachable ... on
  GitHub Actions" to "Neither failure mode manifests ...", which is what is
  actually true.

- State every way a suite reaches a real dolt sql-server. The entry condition
  named one mechanism (testcontainers-go) and one fallback (self-skip), but
  the uow package the section uses as its repro also has suites that call
  testutil.RequireDoltBinary and start a server from the local dolt CLI
  (proxy.BackendLocalServer) with no container involved, and lanes that set
  BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 fail rather than skip. The sentence now
  covers all three and scopes the two limitations to the containerized path.
  The testcontainers leg says "by default" so it stays true once main's
  BEADS_TEST_DOLT_SERVER=local backend section merges above it, without
  forward-referencing a section this branch does not have.

- Attribute the two failure shapes separately. The fenced banner could not be
  produced by the repro printed beside it: the uow entry points pass
  context.Background(), so context deadline exceeded has no producer there;
  the only 60s in the uow package is bo.MaxElapsedTime, a backoff budget that
  backoff.Retry consults only between attempts and which therefore cannot
  bound a hang inside one. The banner's only producer is store.go's
  "failed to initialize schema: %w" inside newServerMode, the server-mode
  store open, which runs initSchema with the caller's context -- so the doc
  names that path. (Iteration 1 labelled it the embedded-store open, which
  contradicted the section's own statement that the embedded/CLI engine does
  not hang.) Kept the uow "sits until the package timeout" sentence (the
  accurate half), dropped the unverifiable ~60s figure, and dropped the
  "failed to create embedded DoltStore: " prefix, which has zero occurrences
  in the tree, so the quoted line is greppable.

- Make the stray-container check runnable under the flow the section is about.
  A bare docker ps talks to the CLI's default endpoint and returns a false
  all-clear on exactly the leaked-container incident the paragraph cites; it
  now carries DOCKER_HOST=unix:///run/user/$(id -u)/podman/podman.sock, the
  form all six in-repo records of this flow use.

- Close the image-pin duplication without touching the guard, and make the
  read fail loudly. The command read the tag from a hand-copy outside
  test/docsync/doltpin_test.go's reach; it now reads it from
  internal/testutil/testdoltcommon.go, resolved from the repository root with
  git rev-parse --show-toplevel so it works from any directory in the tree.
  An empty read would reproduce the false all-clear (docker ps --filter
  ancestor= exits 0 listing nothing), so the tag is expanded as
  ${dolt_image:?...}, which refuses to run docker without it. A trailing
  "[ -n ... ] || { ...; return 1; }" guard does not do that when pasted at an
  interactive prompt: a top-level return stops neither bash nor zsh, and the
  next line still runs docker with an empty filter. Deliberately NOT added to
  doltpin_test.go's scanned set: doltPinImageRE requires the assignment form
  DoltDockerImage = "..." which cannot appear in prose, and find() t.Fatalf's
  on a miss, so that would red a currently green test.

Validation: git diff --check clean; go test ./test/docsync/ 8 PASS / 0 FAIL /
0 SKIP with TestDoltVersionPinsAgree green; scripts/check-doc-freshness.sh
exit 0; the snippet extracted from the doc passes bash -n and zsh -n, resolves
dolthub/dolt-sql-server:2.2.0 from the repository root and from a
subdirectory in interactive bash and zsh, and outside a repository prints the
error without running docker (checked against a stub docker).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 20:17:56 -07:00
Jim Wordelman
7d545486e5
fix(init): refuse to silently recreate a missing server-side database as empty (#5791)
* test(init): red — existing project with missing server DB must refuse init

be-5up5: server mode never has a local dolt/ directory, whether this is a
genuine fresh clone or an existing project whose database was lost from the
server — doltDirExists==false alone (GH#2433's signal) can't distinguish
them. Adds TestInitGuard_ExistingProjectMissingServerDB_Refuses, which spins
up a real dolt sql-server with no database and a metadata.json carrying a
non-empty project_id (proof of a prior real init). Currently fails: the
guard returns nil and lets init silently recreate the database as empty,
which is the 2026-08-11 fleet-wide data loss.

RED only — checkExistingBeadsDataAt is not yet fixed.

* fix(init): refuse to recreate a missing server-mode database (be-5up5)

bd init's server-mode "already initialized" check treated a missing or
unreachable database the same as a genuine fresh clone, silently
returning nil and letting init proceed to create an empty database —
even when metadata.json's project_id proved this was an existing
project whose database had gone missing (the 2026-08-11 fleet-wide
data loss).

Guard on cfg.ProjectID != "" to distinguish the two cases. An existing
project now gets a refusal (initGuardMissingServerDBMessage) that
names bd doctor / bd dolt status / bd backup restore as the recovery
path and explicitly does not recommend bd bootstrap, since bootstrap
has its own related mode-blind create bug (tracked separately). A new
--recreate-missing flag is required to opt into creating fresh in this
case — never implied by --force, config, or env. Genuine fresh clones
(empty project_id) are unaffected.

Also:
- doctor/gitignore.go: drop the "bd init (safe to re-run)" fix hint,
  since re-running bd init is exactly the unsafe action this fix
  prevents in the case that matters.
- storage/dolt/errors.go: the general "database not found" hint no
  longer claims bd bootstrap is a safe first step for "an existing
  project" — only for genuine fresh-clone / shared-server-recovery,
  which is what the existing tests actually cover.

* test(init): pin exit_contract's bootstrap-not-recommended requirement (be-5up5)

TestInitGuard_ExistingProjectMissingServerDB_Refuses checked the refusal
and the absence of --force, but not the exit_contract's explicit "does
not recommend bd bootstrap for a server-mode workspace" requirement —
that was only verified by hand during self-review. Pin it directly, and
assert the message names an actual recovery path.

* test(init): cover --recreate-missing positive path (be-ab3b)

be-5up5's guard already correctly permits init to proceed when
--recreate-missing is set; only the negative path (flag absent -> refuse)
had test coverage. Adds TestInitGuard_ExistingProjectMissingServerDB_
RecreateMissingAllows, covering both guard sites the flag gates:
init.go:2566 (server reachable, database confirmed missing) and
init.go:2578 (server unreachable / errored during the check).

refs be-3mvz (review round 1 request-changes), closes be-ab3b

* chore: release gate PASS for bd init missing-server-DB guard fix (be-udx2)

* chore: record full pre-PR contributor check results for be-udx2 gate

* test(init): red — bd init --force recreates a missing server DB as empty

--force is a deprecated alias for --reinit-local (init.go ~395), and the
existing-data guard is gated on !reinitLocal (~763), so --force skips
checkExistingBeadsData — and with it the be-5up5 missing-database refusal
this PR added. The reinit path's own typed confirmation does not cover the
gap either: it keys on countExistingIssues, which returns 0 or an error in
exactly the case where the server-side database is missing, so no prompt
and no destroy token are ever required.

Net effect at this commit, measured against a real dolt sql-server
(2.2.3) holding no "myproject" database, with metadata.json carrying a
project_id:

    $ bd init --force --prefix myproject --skip-hooks
    ...
      Database: myproject
      Issue prefix: myproject
    Run bd quickstart to get started.
    $ echo $?
    0
    $ ls <server data-dir>
    config.yaml  .dolt  .doltcfg  myproject      <-- created, empty

An empty database at the name the real one used to occupy, exit 0, no
prompt. That is the 2026-08-11 fleet-wide data-loss reflex, reached
through the flag an operator in a panic is most likely to reach for.

This test drives the real binary because that is the only way to exercise
the --force -> reinitLocal -> "skip the guard" path end to end; a unit
test on the guard function cannot see that the caller never runs it.
Both subtests fail at this commit.

Reported in review of PR #5791 by bee-ghosttrack, item 1.

* fix(init): green — --reinit-local/--force cannot bypass the missing-DB guard

Add guardMissingServerDatabaseAt, a narrowed form of the be-5up5 refusal
that answers only one question — is this an already-initialized
server-mode project whose configured database is missing or
unconfirmable? — and run it on the reinitLocal path, which
checkExistingBeadsData never reaches.

The distinction the guard encodes: --reinit-local/--force authorize
destroying a database that EXISTS. They do not authorize inventing an
empty one where the configured database has gone missing. Only the
explicit, per-invocation --recreate-missing does that, which is what makes
that flag's "never implied by --force" help text an honest guarantee
rather than an aspiration. The review suggested softening that help text
instead; making the claim true seemed better than documenting that it
isn't.

Deliberately narrow. It returns nil for every state except the missing
database — no metadata, non-Dolt backend, non-server mode, empty
project_id, or a local dolt/ directory all fall through — so it never
resurrects the "already initialized" refusal that --reinit-local is
legitimately meant to bypass. The "stays silent on a fresh clone with no
project_id" subtest pins that boundary.

Same scenario as the previous commit, now:

    $ bd init --force --prefix myproject --skip-hooks
    Error:
    (warn) Database "myproject" not found on server at 127.0.0.1:34813 ...
    bd init will NOT create an empty database here — that would strand any
    existing issue data behind a new, empty database of the same name.
    $ echo $?
    1
    $ ls <server data-dir>
    config.yaml  .dolt  .doltcfg            <-- nothing created

Addresses review of PR #5791 by bee-ghosttrack, item 1.

* test(init): cover the unreachable-server refuse branch

Review of PR #5791, item 2. The PR claimed
TestInitGuard_ExistingProjectMissingServerDB_Refuses covered both guard
sites — server reachable with the database missing, and server unreachable
or errored while checking. It does not: that test starts a live dolt
sql-server, so it only ever exercises the first. The unreachable site had
coverage on the PERMIT side only (the BEADS_DOLT_SERVER_PORT=1 subtest of
_RecreateMissingAllows), which would not notice if its refusal were
dropped.

TestInitGuard_UnreachableServerRefuses closes that: port 1 is unreachable
by construction, so it needs no dolt binary and runs in the fast lane.

Also drops a comment reference to scripts/repro-bd-init-recreates-empty-db.sh,
which is not in the repo or in this diff (item 3, third bullet), and
describes the state the test sets up instead.

Addresses review of PR #5791 by bee-ghosttrack, item 2.

* docs(init-safety): document --recreate-missing and its known limits

Review of PR #5791, item 3 (doc drift), plus the non-blocking note.

The flag's own help pointed users at `bd help init-safety`, whose FLAG
SURFACE section did not list it — so the one place the help sends you was
the one place it was missing. Added there, stating plainly that it is
per-invocation only and never implied by --force/--reinit-local, config,
or env.

docs/recovery/init-safety.md gains an init-missing-server-db section (and
a TOC entry) covering the refusal users will actually hit: why the
fresh-clone and lost-database cases are indistinguishable from the local
filesystem in server mode, why the old "assume fresh clone" answer caused
the 2026-08-11 loss, and the three ways out in the order they should be
tried — diagnose the server first (a stopped server or wrong --data-dir
presents identically and needs no re-init), then restore from an export,
then --recreate-missing as the deliberate last resort.

Both limits the review flagged as non-blocking are recorded there and in
the guard comment rather than left as folklore:

- The guard keys on project_id, minted by GH#2372. A pre-GH#2372 workspace
  carries none and is indistinguishable from a fresh clone, so it FAILS
  OPEN. Accepted deliberately — failing closed would block legitimate
  first inits on every such clone, and this guard's job is to stop a
  silent recreate where prior initialization can be PROVEN, not to guess
  where it cannot.
- Server mode only. Proxied-server mode returns early on a missing root
  and never reaches the guard, and bd bootstrap has its own separate
  mode-blind create path.

Addresses review of PR #5791 by bee-ghosttrack, item 3 and the
non-blocking notes.

* docs(changelog): record the --force missing-database bypass fix

Under Unreleased/Fixed. This is a behavior change to bd init --force on
every server-mode workspace whose database has gone missing, and it was
not recorded anywhere users read.

* fix(init): make the missing-server-database guard fire in shared-server mode

Review round 2, item 1 (blocking): the guard was a no-op in shared-server mode
— the topology of the 2026-08-11 data loss, and so the primary case.

guardMissingServerDatabaseAt statted doltserver.ResolveDoltDir(beadsDir) and
returned early when it was a directory, reading that as "there is local data to
protect". In shared mode ResolveDoltDir returns the machine-global
~/.beads/shared-server/dolt, and SharedDoltDir() MkdirAlls it
(internal/doltserver/doltserver.go:281). The stat therefore always succeeded and
checkDatabaseOnServer was never reached, while every per-project test kept
passing.

Fix per the reviewer's first option: test for THIS project's database directory
rather than the data directory that holds it. A Dolt data dir contains one
subdirectory per database, each with its own .dolt — verified on disk — so
<doltDir>/<dbName>/.dolt is the correct probe in BOTH modes, and it is strictly
more precise in per-project mode too: a data dir that exists but holds no
database for this project no longer counts as local data to protect.

Two new tests, both against a real dolt sql-server rather than an unreachable
port, covering the two cases the reviewer noted were uncovered:

  TestInitGuard_SharedServerMode_MissingServerDB_Refuses
      reachable server, database absent -> must refuse.  This is the actual
      loss shape. Reverting the one-line guard change turns it red with
      "got nil error"; the pre-existing per-project tests stay green, so it
      pins the shared-mode gap specifically.
  TestInitGuard_SharedServerMode_PresentServerDB_Allows
      reachable server, database present -> must NOT refuse, so --reinit-local
      and ordinary reinit keep working. The reviewer verified this by hand and
      flagged it as untested.

Also from the same review:
- docs/recovery/init-safety.md overstated the proxied limit. The proxied path
  does not reach THIS guard (dispatch at init.go:619 precedes the
  initAllowRecreateMissing assignment at :767), but it does reach the older
  checkExistingBeadsData (init_proxied_server.go:89) — and because the flag is
  never assigned there, --recreate-missing is inert under --proxied-server.
  Documented as pre-existing rather than silently fixed.
- Shared-server mode now named explicitly in the same Limits section.
- checkExistingBeedsData typo corrected.
- The "### 3." heading wrapped onto a second line and rendered broken.

Verified: full TestInitGuard suite 15/15 PASS, 0 SKIP (12.1s), including all
pre-existing guard tests. go build ./... clean, gofmt clean, go vet clean.

Refs be-ok1tt, be-udx2.

* test(init): redirect the shared-mode fixtures with BEADS_SHARED_SERVER_DIR

Round-3 MINOR-2, unaddressed through round 4. The two shared-server tests
redirected HOME so SharedDoltDir() would resolve inside the test tree, but
SharedServerPath() reads BEADS_SHARED_SERVER_DIR first and only then falls
back to os.UserHomeDir(). An ambient BEADS_SHARED_SERVER_DIR — the override
that function's own comment advertises for testing — silently won, and the
pair then wrote into and asserted against the developer's real
~/.beads/shared-server/dolt, failing spuriously if a real "myproject" lived
there. HOME is also a no-op on Windows, where os.UserHomeDir() reads
%USERPROFILE%.

Set the variable the resolver actually consults first. No behavior change.

* test(init): make the present-database control pin the early return

Round-3 NIT-4, unaddressed through round 4, and demonstrated there: with the
local-database probe reverted, _SharedServerMode_PresentServerDB_Allows stayed
green, so it pinned nothing of the early return that commit rewrote. It
returned nil via the server check alone, because the fixture never gave this
project a database directory under the shared data dir.

Create that directory, the way a real shared-mode install has one, and then
assert the early return specifically: with the server made unreachable, the
local database directory alone must still satisfy the guard. Removing the
early return now turns this test red.

* test(init): red — the documented --recreate-missing recovery cannot reach its opt-in

Round-3 MAJOR-1 / round-4 finding 1. checkExistingBeadsDataAt derives "is this
project's database here?" from os.Stat(ResolveDoltDir(beadsDir)) — the DATA
directory, which exists on every initialized workspace (shared-server mode
MkdirAlls the machine-global one in SharedDoltDir; per-project mode MkdirAlls
it in ensureDoltInit). So the FR-010 missing-vs-present branch never runs and
both --recreate-missing opt-ins inside it are unreachable.

Four surfaces tell the operator to run `bd init --recreate-missing --prefix
<p>`: the flag help, `bd help init-safety`, docs/recovery/init-safety.md, and
the guard's own refusal text. That command exits 1 with a false "already
initialized" message pointing back at --reinit-local, which is the destructive
path. The documented recovery is a dead end and the only thing that works,
--reinit-local --recreate-missing, is named nowhere.

The existing _Refuses/_RecreateMissingAllows fixtures cannot see this: they
omit the data directory entirely, so their branch condition is true for the
wrong reason. These two add it, which is the state of every real workspace
whose server-side database was dropped.

Both fail at this commit.

* fix(init): green — probe this project's database directory, not the data dir holding it

Round-3 MAJOR-1 / round-4 finding 1. Test for <dolt-data-dir>/<database>/.dolt
instead of the data directory itself, so the FR-010 missing-vs-present branch
is reachable on an initialized workspace and `bd init --recreate-missing`
reaches its own opt-in. This is the same probe, for the same reason, that
guardMissingServerDatabaseAt already uses on the reinit path.

Making that branch reachable also exposes states the data-dir stat used to
shadow, and permitting them would be a new fail-open policy rather than a fix.
The clearest is a workspace that predates GH#2372: it carries no project_id,
so `existingProject` is false, and the branch would permit a silent empty
recreate on exactly the installed base this guard exists to protect. The
data-dir stat therefore stays — it stops being the gate and becomes the permit
condition, admitting only what the old probe already admitted, plus the
explicit per-invocation opt-in. Net behavior change is the opt-in becoming
reachable, and the false "already initialized" message being replaced by the
accurate missing-database refusal where the database is genuinely gone.

Choosing a better proof of prior local init is deliberately NOT done here.
project_id comes from git-tracked .beads/metadata.json, so a fresh clone
inherits one (round-4 finding 2); the replacement signal needs a different
answer in per-project and shared-server mode, and is tracked as its own
architecture decision on be-5pjhd. The known limit is now recorded in the
guard comment and in docs/recovery/init-safety.md rather than left implicit.

Tests: the two red tests from the previous commit now pass, and two more pin
the narrowing — a pre-project_id workspace with a local data dir stays blocked,
and --recreate-missing still opens it.

* docs: record the --recreate-missing fix and the fresh-clone project_id limit

Round-3 NIT-3 / round-4 finding 4: the round-3 probe change (data dir to
<dolt-data-dir>/<database>/.dolt) was never written down, and it is
user-visible wherever the data dir exists but the database is gone. Both that
and this round's --recreate-missing fix are now under [Unreleased] / ### Fixed.
The stray double blank line is gone with the section's move.

Also records, in docs/recovery/init-safety.md's "Limits worth knowing", that
project_id is inherited by a fresh clone because .beads/metadata.json is
git-tracked by default — the round-4 finding-2 gap this series deliberately
does not close. It fails in the safe direction and --recreate-missing is the
supported way through it; the replacement signal is tracked separately.

* docs(adr): ADR-0004 — proof-of-prior-init signal for the missing-database guards

Round-4 finding 2 asked what should replace project_id as the proof that a
workspace was initialized locally, and what the fail-open/fail-closed policy
should be per server mode. It needed a design decision rather than a
mechanical fix, so it was escalated; this records the answer.

The decision is to formalize what the code already does for local hosts and
keep project_id where nothing local can ever help. Presence of this project's
own <dolt-data-dir>/<database>/.dolt is unconditional proof, because the path
is gitignored and only a real local process creates it. Its ABSENCE, however,
cannot be promoted to proof in either direction: it looks identical whether
the workspace was never initialized here or was initialized here and lost its
storage — and the second is the shape of the 2026-08-11 incident. For a
genuinely remote host the probe is not merely silent today, it is structurally
silent forever, since the only code that creates that directory runs on local
server-management paths.

So project_id stays as the tier-2 fallback with its git-inheritance weakness
accepted, and --recreate-missing is the permanent resolution UX rather than a
stopgap. A local-only init marker and repurposing .local_version are recorded
as rejected, with reasons, so neither is re-proposed each review round.

Extends ADR 0002, which set the init safety invariants but left the evidence
question open.

* test(init): red — the reinit guard waves through a pre-project_id workspace

ADR-0004 follow-up 1. guardMissingServerDatabaseAt is the safety net for
--reinit-local and --force, which bypass checkExistingBeadsDataAt entirely. It
returns nil the moment cfg.ProjectID is empty, BEFORE it ever reaches the
per-database .dolt probe eighteen lines below.

checkExistingBeadsDataAt does the opposite: it probes first, and only falls
back to project_id — narrowed by the coarse-directory permit condition — when
that probe comes up empty. So a workspace initialized before GH#2372 minted
project_id, whose local Dolt storage has since been lost, is protected under
plain `bd init` and waved straight through under the two flags an operator
reaches for in a panic. The weaker path is the one behind --force.

Three tests through guardMissingServerDatabaseAt, mirroring the plain-init
trio: no project_id with a local data dir present must refuse; no project_id
with nothing local at all is a genuine fresh clone and must be allowed; and
--recreate-missing must still open the refused case.

The first fails at this commit. The other two pass, and pin the directions the
fix must not break.

* fix(init): green — probe the local database before consulting project_id

ADR-0004 follow-up 1. Move the per-database <dolt-data-dir>/<database>/.dolt
probe above the project_id check in guardMissingServerDatabaseAt, and run it
unconditionally, so this guard reasons in the same order as the plain-init
guard it backstops.

When the probe comes up empty and project_id is also absent, apply the same
coarse-directory permit condition checkExistingBeadsDataAt uses: allow only
when there is no local Dolt data directory either. A data directory with no
database in it still says this workspace has touched local Dolt storage, and
that is exactly the pre-GH#2372 lost-storage case the old early return waved
through under --reinit-local and --force.

One cell of the behavior matrix changes: no project_id, no local database, but
a local data directory present now refuses instead of returning nil.
Everything else is untouched, --recreate-missing still returns nil at the top
of the function, and a workspace with nothing local at all is still treated as
a fresh clone.

The comment records the shared-server asymmetry ADR-0004 accepts — there the
coarse directory is machine-global, so that cell is more conservative — as
settled rather than as an open defect, since it fails toward refuse and its
population only shrinks.

* docs: cite ADR-0004 from the plain-init guard, the recovery doc and CHANGELOG

The comment at checkExistingBeadsDataAt's project_id check said the limit was
"OPEN and tracked by be-5pjhd" and that the replacement signal was
"deliberately not made here". It is made now, so the comment says settled
instead of open, and points at the ADR rather than at a bead — with a note to
read it before reopening, since the two obvious alternatives are recorded
there as rejected rather than unconsidered.

docs/recovery/init-safety.md said the same thing in operator language and is
updated the same way, plus a new bullet stating that the guard now applies
equally to bd init, --reinit-local and --force. That equality is the
user-visible half of the fix and was not previously stated anywhere.

Its "Last reviewed" marker moves to today. The doc is freshness-tracked
against cmd/bd/init.go, which this series changes, and leaving the marker at
2026-09-08 would claim text written today had been reviewed eight days before
it existed.

* chore: remove leaked internal release-gate doc from PR branch

  release-gates/be-udx2-init-missing-db-guard-gate.md

Tracked our internal gc-management/beads-rig release-gate checklist and
should never have been committed to this upstream branch -- it's process
residue from our deploy tooling, not part of the change itself.

Per maintainer bee-ghosttrack's request during review of PR #6571. The
root cause (our deploy tooling auto-committing this file) is already fixed
as of gm-qatxl1; this branch predates that fix, so this commit is cleanup
only -- no other content on this branch is touched.

Ref: be-lk7o3

* chore(bazel): enumerate init_guard_force_e2e_test.go in cmd/bd srcs

The PR adds cmd/bd/init_guard_force_e2e_test.go under a plain '//go:build cgo'
tag, which is the tag class cmd/bd/BUILD.bazel enumerates (the 23 unenumerated
test files in this package are all 'cgo && integration', 'scripttests' or
'chaos'). cmd/bd/BUILD.bazel is base-added, so the contributor's branch could
not have carried this row. Generated by 'make bazel-sync'; no hand edits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(init): honor --recreate-missing on the proxied route; scope the refusal's evidence claim

Maintainer review fixes for PR #5791 (iteration 1). Five review minors; the
three substantive ones:

1. --recreate-missing was inert under --proxied-server. runInit dispatches to
   runInitProxiedServer and returns before the assignment site, while
   runInitProxiedServer calls checkExistingBeadsData, which reads the global —
   so converting a server-mode workspace with a missing database to proxied
   mode was refused despite the explicit opt-in, and the refusal named the
   exact flag the operator had just passed. It failed closed, so there was no
   data hazard, only a recovery dead-end. The assignment now sits beside its
   flag read, upstream of every dispatch in runInit.

   The PR's recovery runbook documented this inertness as "pre-existing
   behaviour, not introduced here". The early dispatch is pre-existing, but
   --recreate-missing is introduced by this PR, so the flag could not have had
   pre-existing inertness. That paragraph is rewritten to the new behaviour.

2. initGuardMissingServerDBMessage asserted "metadata.json has a project_id"
   unconditionally, but the --reinit-local/--force guard also emits it for a
   workspace predating GH#2372 where only local Dolt storage proves prior use —
   the population the CHANGELOG bullet names. The evidence sentence is now
   selected by the caller, so the message never cites an artifact its caller
   does not have. Pinned from both sides: the project_id cell must cite that
   evidence, the no-project_id cell must not.

3. The new doc comment claimed initGuardServerMessage is "used when there is no
   evidence either way". It has no production caller, so the contrast was
   asserted as two live code paths. Reworded to intent. The dead message's
   surviving "existing project -> bd bootstrap" sentence is pre-existing, is
   pinned by four tests, and is left for a follow-up.

Also re-cited two stale path:line references by symbol and branch (the recovery
runbook's three, and a test comment's two), and expanded the else-branch comment
in checkExistingBeadsDataAt to cover the reachable-but-missing fall-through that
lands there — the branch's guards are correct for both cases only because their
conditions do not mention reachability.

Declined, with reasons, as out of scope for opportunistic cleanup: restructuring
the fall-through itself (a control-flow change in the diff's most complex
function, which both review lanes agree is behaviourally correct today);
rewording the caller-less message's bootstrap sentence (four pinned tests,
pre-existing); and adding --recreate-missing to the generated CLI reference
(the required check-doc-flags job passes at this head and seven other registered
initCmd flags are equally absent, so that page is not a flag enumeration).

Validation: go build ./... and go vet ./cmd/bd/ clean; gofmt clean;
go test ./cmd/bd/ -run 'TestInitGuard|TestCheckExistingBeadsData' ok (26 tests,
real Dolt servers); scripts/check-doc-freshness.sh PASSED; git diff --check
clean. The evidence pin was proved non-vacuous by mutation: forcing the reinit
call site back to an unconditional project_id claim reddens exactly the
no-project_id test and nothing else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(doctor): drop the bd init hint from #5634's permissions warning

1d7629213b on this branch removed "bd init (safe to re-run)" from the
.beads/.gitignore fix hints, because re-running bd init is the unsafe
action be-5up5 guards against. #5634 (c33738d209) reached main while
this PR was open. It added a CheckGitignore warning for unexpected file
permissions that still carries the old hint, so after the merge it was
the only gitignore check that suggested bd init.

Give it the same "Run: bd doctor --fix" hint as the four other
gitignore warnings. That is a complete repair here: FixGitignore
restores mode 0600, and #5634's test already checks that the warning
clears after it runs.

TestCheckGitignore_WarnsOnLoosePermsWhenPatternComplete only checked
that Fix was non-empty. It now pins the exact hint. With the old
gitignore.go line put back, the test fails.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(init): name the real gitignore template in ADR-0004 and init.go

ADR-0004 and the matching comment in init.go both cite
`defaultGitignoreContent` in cmd/bd/doctor/gitignore.go as the place
that says .beads/metadata.json is git-tracked by default. No symbol by
that name exists, in this branch or in main's history. The constant
that says it is GitignoreTemplate. Its closing lines read "Config files
(metadata.json, config.yaml) are tracked by git by default since no
pattern above ignores them".

Rename the reference in both places. The claim itself is unchanged.
Neither file changes outside that one identifier.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 20:07:49 -07:00
Julian Knutsen
a3b6dc9ac5
ci(bazel): remote-execution Bazel lanes for allowlisted fork authors (advisory) (#6994)
* ci(bazel): remote Bazel lanes for allowlisted fork authors (advisory)

Add bazel-farm.yml, a pull_request_target caller of bazel.yml. It gives the
RBE farm to fork PRs to main whose author and triggering user (sender: the
pusher on synchronize) are both on .github/bazel-farm-allowlist.txt, which
is read from the base commit. The list has julianknutsen, quad341, sjarmak
and csells, the same as gascity's blacksmith allowlist.

- authorize job: sparse checkout of github.sha that holds only the
  allowlist and .github/scripts/bazel-farm-authorize.sh; no token on disk,
  no secrets, event facts through env only. The script also checks fork vs
  base repo, the default branch, the action, and a 40-hex head SHA.
- farm job: calls bazel.yml with contents: read, exactly the four RBE
  secrets, checkout-sha pinned to the event's head SHA, and
  fork-farm: authorized.
- bazel.yml: new checkout-sha and fork-farm inputs (no-ops for pr.yml and
  nightly.yml). Every lane checks out inputs.checkout-sha with
  persist-credentials: false. The rbe job runs a fork remotely only for an
  authorized pull_request_target call, and then remote or skip, never
  local. PR concurrency also cancels on pull_request_target. The
  bazel-sync-patch upload is skipped on pull_request_target.

Advisory only: pr.yml is unchanged, so every fork PR still runs the local
Bazel lanes inside the enforced CI Gate / Required.

Policy tests pin the trigger and types, both jobs' conditions, the
allowlist source and contents, the pinned checkout, read-only permissions,
where secrets appear, no expressions in run scripts, per-PR concurrency,
the timeout, and that no workflow_run consumer can pick up a farm run.
TestBazelLaneIsGatedAlongsideLegacy now allows pull_request_target on a
bazel.yml caller for bazel-farm.yml only. 26 mutations were each caught
by these tests (sender check dropped, allowlist read from the PR head,
write permission added, event data interpolated into run, SHA pin dropped,
and others).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(ci): trusted-author fork PR farm design and threat model

Document bazel-farm.yml in the required-check topology:
- the design, and why it is advisory rather than part of the enforced
  gate;
- the gate options considered;
- the threat model: allowlist spoofing, sender vs. author, TOCTOU with the
  pinned SHA, workflow-file edits, script, label and comment injection,
  token scope, artifact poisoning, and farm and Actions cache poisoning;
- what can only be verified on GitHub.

The one capability a listed author gets beyond a same-repo PR is called
out as an accepted risk, with mitigations: pull_request_target runs use
the default branch's Actions cache scope.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): opt the farm's lane checkouts into fork-head checkout (F1)

actions/checkout@3d3c42e5 is v7.0.1, not v6, and v7 refuses to check out
a fork PR's head SHA on pull_request_target unless
allow-unsafe-pr-checkout is true. Every lane of an authorized farm run
therefore failed at checkout.

- Each bazel.yml lane checkout now sets allow-unsafe-pr-checkout to
  inputs.fork-farm == 'authorized' && github.event_name ==
  'pull_request_target'. That is false for every other caller and event.
- TestBazelWorkflowForkFarmInputs pins that exact expression. It also
  fails if the key appears anywhere else in the workflows (so never a
  literal true, and never on the authorize checkout).
- Relabel the checkout pin as # v7.0.1 in bazel.yml and bazel-farm.yml.
- Add the opt-in to the engdoc's design section and to the
  verify-on-GitHub list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): verify everything restored from caches in Bazel lanes (F3)

A farm run executes PR code in the default branch's cache scope. GitHub's
own cache is read-only for pull_request_target: GitHub issues a read-only
token, and cache-mode defaults to read. Whether Blacksmith's colocated
cache enforces the same is unverified. So trusted runs no longer execute
anything they restore without verifying it first:

- setup-bazel: Bazelisk's home moves out of the runner cache to a fresh
  $RUNNER_TEMP/bazelisk-home, so the Bazel binary is never restored from
  a cache. BAZELISK_VERIFY_SHA256 is set to the release sha256 pinned for
  the .bazelversion version on each arch (9.2.0 amd64 and arm64). The
  wrapper step then re-checks every Bazel binary in that home.
- write-bazelrc.sh: add common --repo_contents_cache=. The repo contents
  cache holds extracted repos that Bazel never re-verifies. The
  content-addressable repository cache stays, since Bazel re-hashes it on
  every hit.
- Runner cache key bumped to bazel-repo-v3 (the content changed).
- bazel-test runs `go mod verify` right after the Go module cache restore.
- Policy tests:
  - cache-mode may only be read or none in bazel-farm.yml, bazel.yml and
    setup-bazel;
  - bazel.yml permissions are exactly contents: read;
  - the Bazel sha pin matches .bazelversion, and Bazelisk's home is
    outside the cache;
  - the binary is verified after download;
  - go mod verify follows every module cache restore;
  - the rc disables the repo contents cache.
- Engdoc and bazel-farm.yml header: corrected about GitHub's read-only
  cache, noted Blacksmith as unknown, and listed the full blast radius
  (release.yml hardening tracked separately). The canary design is in the
  rollout notes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): farm runs only the author's own fork, on opened/synchronize (F4)

"The sender is listed" did not prove that a listed user chose the head
commit:
- A listed author could open a PR whose head is a third party's fork, and
  that fork's owner could push between the author's review and "Create".
- A collaborator on a listed author's fork could push while the PR was
  closed or a draft (that push is skipped), and the author's later
  "Reopen" or "Ready for review" click would run it with secrets.

Changes:
- The authorize script requires the head repository owner's numeric id to
  equal the PR author's (HEAD_OWNER_ID, PR_AUTHOR_ID, passed via env).
- Drop the reopened and ready_for_review triggers, and the script now
  refuses those actions too. Drafts already run on opened and
  synchronize.
- Tests pin the trigger types and the new env, and add script cases:
  - cross-fork head, missing or non-numeric owner id;
  - reopened and ready_for_review refused;
  - unanchored action (xopenedx);
  - base-repo prefix (gastownhall/beads-evil). This covers two mutations
    the reviewer found surviving.
- Engdoc: the new checks, and the residual that stays the listed author's
  responsibility (commits pushed by fork collaborators).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): match the farm allowlist on numeric user ids (F5)

Logins are mutable: after a rename, anyone can register the old login,
and before this change they would then be listed as both author and
sender.

- The allowlist now holds one numeric user id per line, with the login as
  a comment. The ids were looked up with gh api users/<login> --jq .id.
- The authorize script matches PR_AUTHOR_ID and SENDER_ID
  (github.event.pull_request.user.id, github.event.sender.id). Logins are
  now used only in the log. An allowlist entry that is not a numeric id
  fails the job.
- Tests pin the id-to-login table and the SENDER_ID env. Script cases:
  - a renamed login re-registered with a new id (author and sender);
  - id prefix, suffix and leading zero;
  - an id that appears only in a comment;
  - listed ids with unlisted logins;
  - a PR from another listed user's fork;
  - broken allowlists (a login entry, a signed id).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(ci): farm re-run semantics and the add/remove-user procedure (F6)

A re-run replays the original event. It checks out the same pinned head
SHA, which is fine. It also reads the allowlist at the original base
commit, so a user removed from the list still gets the RBE secrets when
one of their old runs is re-run. Only users with write access can
re-run.

- Engdoc: how to add a user.
- Engdoc: how to remove a user. Merge the removal, then cancel and delete
  their farm runs (a gh api query by head-repository owner id). For
  cause, rotate RBE_TLS_CERT/KEY and have the farm revoke the old
  certificate; rotating also closes the re-run window.
- bazel-farm.yml header: points to the procedure.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): name the farm call "Bazel Farm", not "Bazel" (F7)

The farm's check runs were named "Bazel / <lane>", the same as pr.yml's.
Farm results are PR-controlled: PR code runs setup-bazel and can make a
lane exit 0. If a ruleset ever required a "Bazel / ..." check, a farm run
could satisfy it. The farm's checks are now "Bazel Farm / <lane>", and a
test pins the name and requires it to differ from pr.yml's bazel job.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): run the farm authorize script in the C locale (F8)

Bash bracket ranges and ${v,,} depend on the locale. In en_US.UTF-8,
[0-9], [0-9a-f] and [A-Za-z] match some non-ASCII characters, and the
Kelvin sign lowercases to "k". This is not exploitable (GitHub ids,
logins and SHAs are ASCII, and the allowlist is maintainer-controlled),
but the script now exports LC_ALL=C.

The script tests now run under en_US.UTF-8 and add cases with non-ASCII
digits in ids and a non-ASCII character in the SHA. Without the export,
the SHA case fails, so the tests catch its removal.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): farm authorize script runs on bash 3.2 (macOS)

${v,,} is bash >= 4; macOS runners' /bin/bash is 3.2, so the script's
tests failed there. Lowercase with tr under LC_ALL=C (ASCII only, as before).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 14:54:06 -07:00
Julian Knutsen
e1b2e79484
ci: gate PRs on the Bazel lanes, alongside the legacy jobs (D1) (#6941)
* ci(bazel): gate the Bazel lane alongside the legacy jobs (D1)

pr.yml now calls bazel.yml (workflow_call) as its `bazel` job, and its
CI Gate / Required requires the call's aggregate (BAZEL) plus each gated
lane's result (BAZEL_TEST, BAZEL_PURE, BAZEL_EMBEDDED, BAZEL_DOLTSERVER).
No legacy job is removed: PR Core, build-artifacts, the pure-Go/js-wasm
check, domain+uow, contract corpus and pr-risk.yml's embedded tier stay
required (D2 is separate).

- bazel.yml drops its own pull_request/merge_group triggers, so a PR runs
  the lane once; push to main, workflow_dispatch and workflow_call stay.
- Every lane records job.status in an always() last step and exports it
  as a workflow_call output named after the job, next to the rbe job's
  rbe-mode / rbe-enabled.
- .github/scripts/bazel-gate.sh reads the call's rbe-mode/rbe-enabled and
  never re-derives the variable/fork/secret condition: mode skip allows
  every Bazel id to skip, local only the remote-only BAZEL_EMBEDDED,
  remote none; a missing or inconsistent mode fails the gate.
- bazel-integration runs in the PR call but stays advisory (its legacy
  counterparts in main.yml run only on push to main): no gate id, and the
  gate script excuses the aggregate failure only when that lane reported
  failure or cancelled in remote mode; the gated lanes are still checked
  on their own ids.
- Callers (pr.yml, nightly.yml) pass exactly the four RBE secrets that
  bazel.yml now declares, never `secrets: inherit`.
- Inside pr.yml's run the Bazel bd artifact is bazel-ci-build-artifacts,
  so it cannot collide with build-artifacts' ci-build-artifacts upload.
- bazel-autofix.yml now triggers on workflow "PR": the bazel-sync-patch
  artifact is uploaded inside the PR run, and "Bazel" runs are no longer
  PR events. Its security model is unchanged (base-branch checkout only,
  artifact read from the triggering run id, data-only patch).
- Policy tests (review D1 F3/F4/F7):
  - TestBazelLaneIsGatedAlongsideLegacy: every bazel.yml job is the rbe
    job, a gated lane (bazelLaneGateIDs) or an advisory lane with a
    reason; gate ids, env expressions and outputs are derived from that
    map; no re-derivation in the gate step or script.
  - TestBazelGateSimulation runs pr.yml's actual gate step over mode in
    {remote, local, skip} (pull_request and merge_group): the script's
    skips equal the ids whose job `if:` is false, and for each lane that
    should run, it alone skipped, failed or cancelled turns the gate red
    whatever the aggregate says; advisory, aggregate and invalid-mode
    cases too.
  - TestBazelArtifactNamesUniqueInCallerRuns: no artifact name collides
    within a pr.yml or nightly.yml run, following local workflow calls.
  - Autofix: workflow_run on "PR", which must upload bazel-sync-patch
    itself or through an unconditional local call.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): keep integration out of the PR call; pin the call's inputs (D1 review v2)

- N1: bazel.yml gains a string workflow_call input `integration`
  (default "on"); bazel-integration runs only when remote and
  `inputs.integration != 'off'`. A string, not a boolean: on push and
  dispatch the input is null, and null == false would silently stop the
  lane on main. pr.yml passes integration: "off" (main.yml's integration
  jobs, its legacy counterpart, run only on push to main). The gate no
  longer reads bazel-integration and bazel-gate.sh no longer excuses an
  aggregate failure: BAZEL is the call's result, red on a missing or
  inconsistent rbe mode.
- N3: pr.yml's call passes exactly build-artifact-name and
  integration: "off" (no rbe or other override).
- N4: bazel.yml's concurrency group never uses github.workflow (the
  caller's name in a call) and differs from each caller's group for every
  event, so the call cannot deadlock against pr.yml.
- N5: no caller of bazel.yml (pr.yml, nightly.yml) may have a
  pull_request_target or workflow_run trigger.
- Simulation: lanes' run modes are evaluated with pr.yml's call inputs;
  advisory lanes must not run in the PR call, gated lanes must run when
  remote.
- engdocs: runbook for main BUILD drift, a farm outage or the RBE cert
  expiry (~2027-09-27): fix main / renew, or unset RBE_WEST_WORKERS.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(ci): record the enforced required-gates ruleset and PR Risk's renamed gate

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): gate the proxied-server and server-Dolt storage lanes

bazel-proxied and bazel-server-storage (remote-only, #6940) join the
gated lanes like bazel-embedded: pr.yml's ci-gate requires
BAZEL_PROXIED and BAZEL_SERVER_STORAGE, each read from the lane's
job.status output (always() recorder step, workflow_call output named
after the job). Both run in pr.yml's call (no integration switch).

bazel-gate.sh allows them to skip in mode local (forks, Dependabot:
pr-risk.yml's legacy tiers cover those PRs) and in mode skip, never in
mode remote. The gate simulation derives this from the jobs' ifs, so
each lane skipped, failed or cancelled in remote mode turns the gate
red. Jobs are renamed "proxied-server tier" / "server-Dolt storage tier"
(shown as "Bazel / ..." under pr.yml's call).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 03:33:20 -07:00
Julian Knutsen
e7138e93af
ci(bazel): server-Dolt storage and proxied tiers on remote execution (#6940)
* test(integration): let SubprocessRunner reuse a prebuilt test binary

BEADS_TEST_SUBPROCESS_BINARY hands SubprocessRunner an already-built test
binary of the package instead of running `go test -c`, which no Bazel
action can do offline (it needs the module source tree and module cache).
A Bazel target passes the running go_test itself: the helper tests
(TestHelperSchemaInit, TestHelperMultiStore) live in the same package.
Unset, behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(dolt): keep a lane's REQUIRE switch out of the leak-probe child

TestSuiteFixtureLeakFailsOwningSuite re-runs the suite with
BEADS_TEST_SKIP=dolt on purpose. When the parent runs under
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 (a fail-closed server lane), the child
inherited it and TestMain exited FATAL before the leak sweep the test
checks. Clear it in the child like the other suite-scoped variables.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(bazel): proxied-server and server-Dolt storage tiers (D6d, D6e)

PR Risk's "Test (Proxied Dolt Cmd N/15)" (also main.yml's) and "Test
(Server Dolt Conformance)" / "Test (Server Dolt Full Suite N/16)" as
Bazel variants against local dolt sql-servers (internal/testutil local
backend, pinned dolt from the test wrapper), fail-closed like the
dolt-server lane:

- //cmd/bd:bd_proxied_test: 15 shards; shard k runs the job's
  proxied-test-shard.sh k+1 15 through go_test_manifest_shard.sh over the
  race bd_test, non-race bd_for_tests as the subprocess bd,
  BEADS_TEST_PROXIED_SERVER=1, GOMAXPROCS=4. Tag dolt-server-proxied,
  --config=doltserver-proxied (the doltserver lane's build).
- //internal/storage/dolt:dolt_server_{conformance,full}_test over
  dolt_race_off (go_test_race_off of dolt_test: the jobs' binary is not
  race); the full suite runs server-storage-test-shard.sh k+1 16 with
  BEADS_TEST_ENV_RUN_DOLT=1 and hands SubprocessRunner its own binary.
  Tag dolt-server-integration, --config=doltserver-integration, whose
  build flags are exactly --config=integration's, so the tier shares that
  lane's compile (race-off deps land where bd_for_tests already is)
  instead of compiling the tagged graph a third time.
- dolt_test gains //internal/storage/schema:migrations as data (the
  PR4107/migration replay tests read ../schema/migrations).
- test_env.sh: BEADS_TEST_GIT_IDENTITY=1 writes the jobs' global git
  identity ("Configure Git"); the git-remote tests commit in fresh repos.
- Both tags join the taxonomy and bazelPRCoreExcludedTags, so prcore/ci
  and integration exclude them.

Per-shard names and statuses equal a replay of the jobs' commands
(container backend) for all 32 jobs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the proxied and server storage tiers remotely (advisory)

bazel-doltserver gains a second, remote-only step, `bazel test //...
--config=doltserver-proxied` (15 shards of ~5-10 min with a busy dolt
sql-server each do not fit a GitHub-hosted runner; forks and rbe=off keep
the local lane unchanged). bazel-integration gains `bazel test //...
--config=doltserver-integration`, which shares its build flags, so the
server storage tier compiles nothing the integration step has not. Both
steps write their own BEP, run check_testcases.py on it, run even when the
job's first test step failed, and upload logs on failure.

Policy (TestBazelDoltServerTiersMirrorPRRisk): each variant runs its jobs'
shard script with their shard total and has script + manifest in data;
the conformance variant passes the job's flags; the jobs' race/non-race
binaries and switches (BEADS_TEST_PROXIED_SERVER, BEADS_TEST_ENV_RUN_DOLT)
match; doltserver-integration's tags and race equal --config=integration's;
the steps are pinned to their jobs. checkDoltServerRules now holds all
three dolt-server lane tags to the local-backend, fail-closed,
remote-capable rules.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the proxied and server storage tiers in jobs of their own

Review of D6d/e: as steps of bazel-doltserver and bazel-integration the
tiers inherited those jobs' gating (a caller switching the integration
lane off would drop PR Risk's PR-time server tier; a gate requiring the
dolt-server lane would silently require the proxied tier too) and
lengthened their critical paths.

bazel-proxied (--config=doltserver-proxied, 15 shards) and
bazel-server-storage (--config=doltserver-integration: conformance + 16
shards) are remote-only jobs that need only rbe, each with its own BEP,
check_testcases.py step and log upload, 25-minute test step inside a
30-minute job. bazel-doltserver and bazel-integration are back to their
previous content, including the dolt-server step's 80-minute timeout for
local runs. The server tier shares bazel-integration's compiles through
the remote cache (same build flags).

The proxied tier mirrors PR Risk's race bd-cmd-test; main.yml's twin
job runs build-artifacts' non-race one, so Bazel is the stricter of the
two. Comments now say so, and the policy test pins main.yml's build so a
change there is noticed. Policy: the new jobs are pinned (command,
config, remote-only if, BEP + testcase steps, timeouts, log upload), and
no tier step may run in bazel-integration or bazel-doltserver.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 23:01:40 -07:00
Julian Knutsen
9116068c54
ci: give PR Risk's gate a distinct check name (#6939)
pr.yml and pr-risk.yml both reported a check run named "CI Gate / Required".
GitHub resolves required checks (and pr-preflight its rollup) latest-per-name,
so a red gate in one workflow could be masked by a later green one in the
other. Rename PR Risk's aggregate to "PR Risk Gate / Required" so both can be
required independently.

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:31:43 -07:00
Donna Box
fa3edf78a5
BDP bead-graph plan and W-arch design docs (P-1 rulings complete; docs-only) (#6154)
* BDP bead-graph plan: seven-round converged design (v7, SOUND)

The plan for implementing BDP natively in this repo, driven through seven
adversarial review rounds at max effort. Headlines:

- Pinned to BDP aee075f5 (the owned-Links rulings); spec-first enforced.
- v0 Scope serves NATIVE beads/links only. The Issue/Dependency projection
  was withdrawn on proven counterexamples (rounds 3-5): no read-side
  mechanism can satisfy BDP's revision and identity laws over a store
  permitting timestamp ties, stale imports, arbitrary SQL, and identity
  resurrection. The counterexample record and §2b conflict inventory are
  preserved as the C-lane requirements list.
- §2b: fifteen-row inventory of where BDP and the current stack disagree.
- C lane sketched: funnel (C1 compat-shim-over-graph-storage → C3 cutover)
  vs observe (C2 storage-level trigger feed), all claims as verification
  tasks.
- Native design: graph model + graphops trait, GraphReadSource resolution
  honoring the tree's decorator/telemetry discipline, persisted Type
  Descriptor store with installation lifecycle, allocation/tombstone
  ledger, replication participation matrix, provable P-1..P3 exits.
- Twelve P-1 decisions enumerated for operator ruling.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan: the storage interfaces, concretely (operator request)

Member-by-member attachment: Storage unchanged (GraphCapable as separate
optional interface on DoltStore); no decorator forwarding (targeted-peel
resolution, telemetry re-wrapped); the snapshot lease as the one new
storage primitive (request-scoped transaction lifetime vs per-call
withReadTx, pool/timeout design named); normal migration series; serve
role-source entry with non-capable fallback; backend/ additive+gated;
issueops/journal/sync untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan: storage-interfaces round-8 fixes

UOW lease cannot ride RunTxRead (it closes its unit of work on return) —
OpenSnapshot owns a direct NewUOW, Close rolls back; telemetry wrapper
does not promote inner methods — resolver inspects through it and
explicitly re-wraps; capability contract closed (graphops.Store = read
surface for v0, ErrGraphUnsupported vs operational failure, in-tree-only
exposure); bd serve gets an optional graph field + conditional route
seam, not a mandatory role-table entry, with absent/broken/uninitialized
kept distinct; ScopeResolver named the one snapshot owner with detached
bounded close (cancelled-context rollback burns the connection);
conformance description corrected (RunUnsupportedContract proves
refusals; GraphCapable becomes a completeness-guard root if opened).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan v8: round-9 reconciliation

Resolver signatures unified on the error contract everywhere;
graphops.Store defined (v0 alias of GraphReadSource, package ownership
stated, UOW-to-Scope adaptation named); decision 12 kept open with the
uninitialized state explicitly represented under either branch; serve
assembly names both source-appropriate resolvers; header reflects the
nine-round record.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan v8: round-10 reconciliation — resolver pair named everywhere; package ownership cycle-free

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan v8: resolvers to neutral graphsource package — the actual cycle-free layout

uow already imports storage, so the provider-arm resolver cannot live in
package storage. Both resolvers move to internal/storage/graphsource, a
separate assembly package importing storage+uow+graphops, consumed only
by composition roots; GraphCapable stays in storage (leafward import of
graphops). Signatures fully qualified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan v8: align the remaining resolver sketch (package clause + qualified types)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan v9: ruling tranche 1 — terminology, charter, withdrawal, S1, Scope URL, authority model

- Terminology block: 'graph beads / graph links / the graph store' replace
  'native' (which reads as the existing implementation inside this repo);
  'the bead graph' names ruling 1's union concept. Swept throughout.
- Ruling 1 (charter): core; amendment follows working bits; scope = the
  bead graph expands to non-work-tracking information (wisps the example).
- Ruling 5: projection withdrawal ratified (wisps included).
- Ruling 2: substrate S1.
- Ruling 7a: Scope URL mirrors BDP's pinned startup contract.
- Ruling 9 (corrected model): the authority is the graph store as reached
  through the NORMALIZED STORAGE ABSTRACTION, whichever provider realizes
  it — not bd serve, not Dolt. Marker, serialized history, non-authority
  refusal, and the snapshot lease are contract obligations proven by the
  graph conformance suite; CLI verbs and the BDP handler are both clients.
  Consequence: the graph capability is part of the PUBLIC backend/
  contract from P1 (flips the in-tree-only hedge); §4 reframed with Dolt
  as reference realization.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan v10: ruling tranche 2 — restore, lifecycle commands, ledger, journal, wisps, listener

All twelve P-1 decisions are now ruled (one spelling open):
- 11 restore vs identity: layered — restorable append-only ledger with
  provider-declared survival; Scope-URL + epoch rotation when preservation
  cannot be guaranteed (spec: identity-losing restore = different Scope).
- 12 (replaces empty-at-birth): store/Scope/client — bd init initializes
  the graph store against the interfaces; bd serve mints the Scope on
  first serve under a configured URL and serves it honestly empty; a
  client-wiring command (spelling TBD) makes the CLI a BDP client of a
  designated server. New §4 'Lifecycle commands'.
- 3 ledger: append-only allocation record; ID test is a keyed O(1)/O(log n)
  lookup as a contract obligation.
- 8 journal/changefeed, 6 wisps (C-lane visibility), 7b listener (same
  middleware path; v0 view = whole Scope per bearer token): ratified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan v11: close P-1 — client wiring is a bd init target that reroutes above the storage abstraction

bd init --bdp-server <url> joins --server/--shared-server/--proxied-server/
--team-server/--backend as an init target; unlike those (which select the
provider/topology BELOW the normalized storage interfaces) it reroutes
ABOVE them, at the CLI: graph verbs become a BDP client of the designated
server. Config bdp.client store|server (explicit mode) + bdp.server;
env/credentials/precedence mirror dolt.mode. All twelve P-1 decisions
are now ruled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Plan v12: bd bdp-serve as the isolated bootstrap path; related workstreams (W-arch, W1, W2, W3)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* W-arch: graph architecture/design + detailed CLI and storage-interface spec

Two companion writeups to the plan (operator request):
- BDP_GRAPH_ARCHITECTURE.md: the shape — packages, imports, roles, a read
  end to end, where each ruling lands — and three corrections to the
  plan's §4 after reading the tree's role machinery: accessors on
  storage.Storage (house rule; promoted through decorators by interface
  embedding; breaking-change-with-changelog as Memories() was) replace
  GraphCapable + the graphsource resolver; one-role-call-one-transaction
  replaces the snapshot lease for P1 (cross-request continuation stays
  P2's ADR); accessors carry the layers, so no peel-and-rewrap policy.
- BDP_GRAPH_CLI_AND_STORAGE_SPEC.md: the diff — bd init graph bootstrap,
  bd init --bdp-server (config/metadata/env/precedence), bd bdp-serve
  flags and behavior, bd graph restore/promote, reserved verbs and the
  three-route accessor fork; Storage accessors, the graphops public leaf,
  bodies per leg, migrations, decorators, backend/ aliases, conformance
  contracts and gates, bdpapi + DTO generation; what does not change; open
  implementation questions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* W-arch v2: answer the three-reviewer council; six ruling amendments proposed (pending)

Architecture v2: mechanism corrections separated from ruling amendments;
six roles (Reader, DescriptorReader, TypeInstaller, IdentityReader,
ScopeBootstrapper, AuthorityRotator); DBTX-shaped shared bodies so the UOW
leg (bd serve's primary path) shares one body; replicated Scope row +
clone-local dolt-ignored authority half asserted inside every read
transaction; BDP rows mounted inside internal/httpapi behind route();
bd bdp serve as a thin strict command; bd bdp namespace; laws in graphops;
no backend/ aliases.

CLI/storage spec v2: config.yaml single source for bdp.*; BEADS_BDP_TOKEN_FILE
(no env token); credentials keyed by origin+scope path; identity state table;
promote CAS + fencing by expectation; bd bdp restore branching on
LedgerDurability; ledger export/import lane; full DDL with utf8mb4_bin
collation, Scope-relative paths, LONGBLOB canonical JSON; three censuses and
every embedding surface; conformance cases the council named; bts migration
coupling; Part C2 (merge funnel, federation, bd sql) replaces v1's
'does not change' claims.

Plan v13: superseded-mechanism banner on §4, amended lifecycle spellings,
and a PENDING 'Amendments proposed by W-arch v2' block under §9 (A1–A6 plus
the DML enforcement boundary and the replication/merge ADR).

* W-arch v3: answer council round 2; plan v14 reopens P-1 for amendments A1-A8

Round 2 (Gemini/Codex/Claude on v2) disproved v2's A5 premise with a live
Dolt probe: DOLT_BACKUP restore carries the working set, so a dolt-ignored
authority table survives restore. v3: the clone-local authority half is a
gitignored .beads/graph-authority.local.json carrying a ledger high-water
mark, asserted in every transaction (clone/pull arrive without it; a
restore keeps it but rewinds the ledger -> ErrStateRewound). Append-only
graph_ledger_events with one global sequence; graph_allocations derived;
bd bdp ledger snapshot|apply continuity lane. A7: promotion fenced at the
remote (CAS + push, non-fast-forward fails) with bdp.authority_heartbeat.
A8: constraint #1 scoped to behavior; out-of-tree backends stub six
accessors. Also: BeadGraph* accessor names (GraphCounter clash); Admin
role; OwnedLinks grouped by Type URL (the pinned schema's shape); opaque
Cursor from P1; expectation on descriptor reads/installs; bd bdp serve
inherits errServeReadonly; CommandPath-keyed root policy; --dev-local-test
removed; BDP_SCOPE_URL honored via BindEnv; no token key in config;
bd backup restore marks unverified; merge entry-point inventory corrected;
bts coupling stated as a release-parity gate; provenance columns; LIMIT
bound+1; statement counts pinned; wire artifact in future tense.

Plan v14: status reopens P-1; section 9 block A1-A8 + two decisions;
section 7 superseded-names note; section 8 W-arch bullet.

* W-arch v4: answer council round 3 — store-owned witness, per-topology fencing, hash-chained ledger; plan v15

Round 3 converged on the authority mechanism. v4: the authority witness is
STORE-OWNED (the accessor loads it from .beads/graph-authority.local.json
under an advisory lock; no request type carries authority fields — a
caller-supplied expectation was forgeable from replicated rows); the file is
bound to a workspace key (hostname + real path, not the shared project id),
written under an exclusive lock with monotone read-modify-write and
directory fsync, and only once .beads/.gitignore carries the entry (all
three doctor lists). The ledger is hash-chained (prev_hash/hash) with
mint/install/promote/rotate/allocate/tombstone/refuse_url events; the
witness records the head {seq, hash} so a restore or a different history is
refused by exact-prefix identity. A7 fencing per topology: shared SQL server
-> dolt-ignored graph_authority_lease (the leases/bd-lrgn1 precedent,
ephemeral heartbeats); remote-backed embedded -> Mint/Promote are fetch ->
compare -> CAS -> DOLT_COMMIT -> push with non-fast-forward reset, heartbeat
compares (authority_id, epoch) and fails closed after a grace, P3 writes
push-on-commit; no remote -> CAS alone. A8 presented as two options, not
pre-empted. A6: bdp.scope_url in tracked config.yaml; bdp.client/server/
insecure_http in untracked config.local.yaml via bd bdp client (config.yaml
is git-tracked by default). bd serve never refuses on account of the graph
(legacy up, rows absent on non-authority clones); bd init bootstraps only on
an unminted store or the authority workspace and is silent otherwise;
CommandPath-keyed policy authoritative at every leaf-name call site (list
may coincide); five statements per read with LIMIT remaining+1; validator
singleflight; tree corrections (14 files, ReadyClaimer CHANGELOG precedent,
-0.0 attribution, facade census file, provider-arm assembly).

Plan v15: operative lifecycle items 2-3 rewritten as amended (pending),
backup/restore matrix row, W2 bullet, section 9 block refreshed to v4.

* W-arch v5: answer council round 4 — fences by hazard on the serving leg, sequence counter, two-phase transitions; plan v16

Round 4 (Gemini/Codex/Claude on v4) converged on the authority mechanism.
v5 on the tree's precedents: a single-row graph_ledger_seq counter (the
journal's bd_events_seq shape; Dolt's FOR UPDATE is a no-op); fences
composed by hazard rather than storage mode — a shared database gets the
dolt-ignored lease that every mutation must UPDATE with one affected row
(a takeover is a serialization loser), a configured remote gets DOLT_FETCH
-> ancestor check (DOLT_MERGE_BASE) -> scoped commit (DOLT_ADD graph tables,
never -Am) -> DOLT_PUSH with a typed non-fast-forward classifier and reset
to the recorded pre-op HEAD; both when both; embedded no-remote not
promotable in place. Every serving workspace is a SQL-server workspace, so
the serving leg is the UOW provider and the fences live there (doltSQLProvider
gains beadsDir). Only bd bdp serve mints (tracked Scope URL); bd serve
converts every graph failure into rows-absent + notice. No ledger event
before mint: the built-in catalog is installed inside the Mint transaction;
bd init installs nothing. Two-phase transitions (pending record, recovery on
load) for mint/promote/rotate/ledger-apply; LedgerApply and restore carry
their own recovery predicate. Installation key (per-installation id under
the user config dir + canonical path) replaces the hostname. Graph-state
version = per-table DOLT_HASHOF_TABLE hashes (DOLT_HASHOF_DB moves on every
heartbeat); ErrStateChanged returned without validating in the held tx,
validated under singleflight by the accessor. config.local.yaml/witness/lock
in all three doctor lists; bd config set bdp.scope_url refused once minted;
BDP_SCOPE_URL read first. Lease table via doltIgnorePatterns + main-series
+ ignored-series (0055/ignored 0012 shape). update event kind; provenance on
allocations; per-method statement budgets; tree corrections from round 4.

Plan v16: witness wording in the section 4 banner, config split in the
lifecycle banner, mint trigger in lifecycle item 2, ruling 11 spelling,
section 9 block refreshed to v5.

* W-arch v6: answer council round 5 (Codex, Gemini); propose A9 simplification; plan v17

Lease predicate now carries the holder installation key, a per-process
nonce, the epoch, and an unexpired expires_at by database time; an expired
claim is never renewed. Every replicated graph mutation (mint, promote,
rotate, install, ledger apply, seeds, P3 writes) runs through one provider
primitive on hazard R: fetch -> ancestor check -> record remote graph roots
-> fenced transaction -> scoped commit (new RunTxScopedResult; the UOW commit
hardcodes -Am today) -> push classified by a typed lift of sync.go's
pushRacePattern; on a race every remote graph root is compared (any graph
delta fails closed; issue-plane-only divergence keeps the commit as
ErrSyncRequired; other failures keep it unpublished). Undo is soft reset +
per-table DOLT_CHECKOUT when HEAD is still the op commit (verified against
Dolt 2.1.8), DOLT_REVERT otherwise — never a hard reset. Transitions are
multi-phase (begun/local_committed/published/config_written) with resume-or-
undo recovery. Installation id created O_EXCL 0600 in the directory
UserConfigYamlPath chooses. v0 serves BDP only from SQL-server workspaces
(registered backends' store arm has no fence; deferred behind a
GraphPublication capability). bd bdp serve mints via a staged startup
(exclusive gates through a temporary source, then shared). config set/
set-many/unset refuse bdp.scope_url once a witness is held (file check);
rotate refused under an overriding BDP_SCOPE_URL. LedgerApply restores
anti-reuse history only (reserved allocations; counter set). Graph-state
version = ordered per-table hashes of the eight replicated tables
(DOLT_HASHOF_TABLE verified to exist); one singleflight coordinator per
provider; descriptor cache keyed by the descriptors table hash. A8's
method-set reasoning corrected (required methods promote through
interface-embedding wrappers; optional interfaces do not). A9 proposed:
v0 authority requires a shared database; hazard R deferred.

Plan v17: A7/A8 refreshed, A9 added, lifecycle item 2 notes registered
backends and the staged mint.

* W-arch v7: fold the Claude reviewer's probe-backed round 5 — fence cell, counter nonce, ledger-head comparisons; plan v18

Probed on Dolt 2.1.8: concurrent transactions merge cell by cell, so a lease
UPDATE fences a takeover only when both rewrite the same cell with different
values, and two 'next = next + 1' increments from the same value both commit
with one increment landing. v7: every lease write (grant, steal, expiry
reclaim, renewal, per-mutation fence) rewrites a random 'fence' cell and
predicates on the value it read; the ledger counter writes a random
allocation nonce ('next' is reserved; column is next_seq); the events PK is
the second guard (it converges on byte-identical rows, which the nonce
prevents). Push-race classification and the heartbeat compare the remote
ledger head via AS OF 'origin/main', never the (authority_id, epoch) tuple
(a VM-image twin shares it). No configured remote => not promotable in
place on every topology (a copied data directory under a second sql-server
is a different, unobservable database). Lease holder is the workspace
(installation key); admin verbs other than restore/ledger apply take the
shared gate and rely on the lease so they run beside a live server; plain
bd serve never runs transition recovery. Installation id: user_config_path.go
resolver, BEADS_INSTALLATION_ID_FILE, per-OS-user and ephemeral-home
residuals. Fresh-remote and ref-spelling rules; timedProvider getter;
domain/fs gitignore writer; check D warns without a base ref.

* W-arch v8: fold council round 6 — self-regrant, A9 arbiter, gate model, undo unstage step, ref spelling; plan v19

Round 6 (Gemini/Codex/Claude on v7): the Claude reviewer's live Dolt 2.1.8
probes CONFIRM the fence cell (1213 on the versioned and ephemeral paths,
read-then-mutate, and disjoint-column takeovers) and the counter nonce
(byte-identical events conflict). v8 closes the residuals: an expired lease
still naming this workspace self-regrants (renewal and mutation extend
expires_at; reads carry no expiry term; promotion only when holder or epoch
differ); the lease row Mint creates is the sole proof of 'minted here' and
in-place Promote requires it (A9's arbiter; clones promote only with
--rotate-url); promote/rotate/install take the shared gate, restore/ledger
apply the exclusive gate, stated identically everywhere; staged bd bdp
serve bypasses the generic pre-run gate and takes exclusive only when there
is no Scope row; transition recovery is by evidence (durable op id in the
ledger event; begun-with-committed-op detected; local-committed on a shared
DB finalizes; remote classified pre-head / contains op / foreign); Advance
is a descendant-aware compare-and-advance; the undo unstages before the
per-table checkout (a bare checkout after a soft reset reverts nothing,
probed) and restores the ignored lease row; push races compare the remote
ledger head (ORDER BY seq DESC LIMIT 1 — MAX(seq),hash errors under
ONLY_FULL_GROUP_BY) AND the graph tables' diff, against the workspace's
witness; the ref is remotes/<remote>/<branch>; the classifier lifts all of
isPushRaceErr; DOLT_HASHOF_TABLE('<name>') x 8 with a stated encoding;
renewal-interval bound on fenced transactions; A9 cut sheet; doctor-list
membership per list; tree corrections (uow_factory forwards beadsDir; no
example backend package; jira stub is a flat implementer; config unset
gains an explicit guard call; config.local.yaml merge wording).

* W-arch v9: fold council round 7 — honest A9 arbiter, read-path expiry, op_id column, phased undo, one staged sequence; plan v20

Round 7 (Gemini, Codex; Claude probes pending): a dolt-ignored lease row is
NOT proof of 'minted here' — DOLT_BACKUP restore and a directory copy carry
the working set, and Dolt's @@server_uuid is per machine (probed: identical
for a same-machine copy). v9: in-place promotion is self-regrant (the lease
names this workspace) or an operator --steal (an assertion of 'same
database', in the class of force-push); a foreign holder's expiry alone never
grants a takeover; promote --rotate-url is the bootstrap that creates the
lease row for clones, copies, and restores; physical database copies are the
stated operator-managed hazard; the lease row is bound to scope_url and
authority_id. Protected reads self-regrant ephemerally first and require an
unexpired lease inside the read; request deadline below the remaining lease
interval. op_id becomes an indexed, hash-covered ledger column; the SQL-free
manager asks an evidence provider; no-remote recovery executes config_intent
before finalizing; undo is phased (undo_started/versioned_undone/
lease_restored), pauses renewal, and restores the lease only on the reset
path and only if it still names this workspace; Advance runs a
provider-supplied prefix/ancestry verifier under the file lock; one
normative staged-startup sequence with an explicit shared release before
exclusive; every fenced transaction carries a deadline below TTL/3 with
jittered, serialized renewal; types install is uniformly exclusive;
hazard-R recovery reuses the delta classifier and the watcher recognizes a
pending local op; A9 topology matrix keeps the default-branch rule;
transaction-bound capabilities (StateVersioner, GraphPublication,
LeaseClaim) move to internal/storage/graphcap. Plan v20 mirrors A1/A7/A9.

* W-arch v9: op_id on the public LedgerEvent sketch

* W-arch v10: fold council round 8 — one lease predicate, read path never writes the lease, A9 propagated, transition lock held; plan v21; held for rulings

Round 8 (Gemini/Codex/Claude on v9): the residuals were text-level. v10:
one lease predicate everywhere (a self-held lease CASes regardless of
expiry; a foreign holder is replaced only by --steal; no row -> --rotate-url;
every 'expired or --steal' and 'lease Mint created here' formulation
removed, including the plan's unexpired-expires_at term); protected reads
SELECT first and require an unexpired lease inside the transaction, never
write it (the serving leg's watcher renews; a near-expiry read fails closed;
a CLI read on its own expired lease regrants once, ephemerally, before a
fresh read tx — probe-backed ordering); BDP rows carry their own per-row
deadline below TTL/3 (the shared route() deadline of 60s exceeds the TTL);
types install exclusive in every section and the TTL/3 deadline scoped to
the shared-gate context; the file lock is held from Begin through
Finalize/Abandon and recovery acquires it first; LedgerApply's crash
evidence is the manifest (imported op_ids are never rewritten); recovery
keyed on 'hazard R in force' (any remote under A9 takes the local path)
with the delta-classifier precedence incl. issue-plane-only; the undo
restores the lease on either path under the same-holder guard with a fresh
fence (a revert restores the Scope epoch and would orphan the lease —
probed); one staged-startup sequence everywhere; A9 propagated to A5's
bracket, the embedded leg, and 'CLI-only'; concurrent mint under one
tracked URL settled by the ADR (interim: earlier mint wins); ephemeral-home
services pin BEADS_INSTALLATION_ID_FILE; rotate beside a live server exits
3; graphcap references; INDEX (op_id); ledger snapshot shared;
commandNeedsExclusiveGate learns the new verbs; depguard allowlist gains
uow/graphcap; UnstageTables; dolt_log AS OF for the remote pre-head; shared
staging sweep stated; matching-head restore is not a rewind; metadata.

* Materialize rulings A1–A7 and A9 (2026-09-07); re-pin BDP to bdp#18 merge 0b7d86e7; plan v22, W-arch v11

Operator rulings taken one decision at a time on 2026-09-07:
A1 store-asserted witness replaces the snapshot lease (ruling 9); A2 BDP
rows inside httpapi, only the strict serve mints, bd serve never refuses,
intra-Scope references Scope-relative and a new base URL is a rotation
(7b/12); A3 the root flag --graph-mode link|dependency replaces the bd bdp
prefix (the mode names the graph by its edge kind; bd link creates the
selected graph's edge; policy keyed on mode + command path);
link-graph.route replaces bdp.client; A4 public graphops + BeadGraph*
accessors; A5 witness file + hash-chained ledger + ledger lane +
LedgerDurability (retained versions to the History lane); A6 config
placement (tracked scope_url, local per-workspace keys, file tokens); A7
the shared-database fence (fence cell); A9 v0 authority requires a shared
database, remote half of A7 deferred to the write-profile ADR.

Also folded: bdp#18 (attribution, two statuses) merged and re-pinned in §0,
attribution row restored in §2b; sjarmak's accepted inputs (typed incident
index; the two §2 plane statements); migration slots start at 0068 (0067 is
versioned-beads Phase 1) with a CLAIMED.md registration at P0; History's
store_epoch vs the graph authority epoch stated as distinct. A8 and the two
decisions remain pending.

* A2: record the remount-is-rotation statement in the architecture row

* Materialize A8 (option A, 2026-09-07): constraint #1 scoped to behavior; out-of-tree backends take the declared source break; plan v23, W-arch v12

* Materialize ruling 13 (D1, A+B, 2026-09-07): out-of-role DML out of contract; validator with ledger accounting; session-gated triggers on the eight replicated tables; plan v24, W-arch v13

* Materialize ruling 14 (D2, option B, 2026-09-07): replication/merge ADR gates P1 with a four-law charter; P0 open; plan v25, W-arch v14

* Docs: clear four stale proposed/pending markers left after rulings 13–14 (arch §2b heading; plan matrix header, wisps row, §8 W-arch bullet, ruling 11 spelling note)

* Docs: record the P0 results — 40 pinned scenarios, generator decision (Part D.3), verification rows (i)-(iii) answered with their rules; fence census over dolt_schemas queued for ruling (Part D.7)

* Docs (P0 council 2026-09-07): Link endpoints symmetric (external-source Links are in the pinned fixtures), pins variable-width, error home recorded, canonical-form and UTF-8 hashing rules stated

* Docs (P0 council): P0 wire wording, JCS interop caveat, conformsTo sorted in the descriptor canonical form

* Docs: migration slots 0069 and later (0068 claimed by versioned-beads Phase 2 attribution_status; CLAIMED.md row c53ef8810)

* Docs: P0 implementation notes from council 11's fold (Scope-URL law split, WHATWG origin model, symmetric endpoints, ledger bounds, descriptor canonical form, strict wire decoding); P0 exit met

* Docs: name the graph plane's epoch columns authority_epoch at P1 to keep them distinct from the History lane's store_epoch

* Docs: record the bdp#1 rulings of 2026-09-08 — wildcard ownership entry (B2/B4) and complete-or-refuse historical reads (History lane)

* Docs: wildcard ownership as implemented at P0 — reserved-key sum, max reading, present-only groups, pinned-wire tripwire

* Docs: OW1 ruled A — the wildcard max bounds the whole owned set; explicit max never exceeds it

* Docs: bdp#21 admission law in B4 — round-trip rule refuses at admission; ledger framing exempt

* Docs: B4 — CheckBeadRecord enforces both owned-set bounds at acceptance; P1's LIMIT covers the whole-set bound only

* Docs: A10 solo topology (ruled B), Part D.7 fence census (ruled A), authority_epoch confirmed — 2026-09-08

* engdocs: properties are opaque — references inside properties are data, never edges (bdp#1 item 5 as amended, ruled 2026-09-08)

* docs: align graph design with current BDP and versioned beads (#6154)

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Donna Box

* docs: wire graph alignment exits and clarify pinned sources (#6154)

Preserve historical P0 results, assign current-wire and serving gates, and clarify selected-profile pins and future storage delta review. Fold adjudicated source and History precision corrections without changing the ruled graph plane or adding runtime claims.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Donna Box

* Align graph architecture with ruled solo topology (#6154)

Carry A10 into the architecture and embedded-leg summaries, reconcile minting and registry references, and record merged Read/RU dependencies alongside the published TX draft and Jim's dated checks. Preserve the formal change-request hold and phase-specific implementation/evidence gates.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Donna Box

* Reconcile solo authority mechanisms and remaining graph review references

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Donna Box

* docs: complete A10 solo authority consistency sweep

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Donna Box

* docs: close approved graph review follow-up nits

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Donna Box

* docs: reconcile graph adoption and remaining phase gates (#6154)

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Donna Box

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-28 22:12:19 -07:00
Bùi Phan Viết Cường
5cc896c8f0
fix(lint): suppress two darwin-only gosec findings and cross-lint the darwin build in CI (#6243)
* fix(lint): suppress two darwin-only gosec findings and cross-lint the darwin build in CI

scripts/ci/pr-lint.sh lints natively on the ubuntu runner plus one
GOOS=windows CGO_ENABLED=0 cross-lint, so files behind //go:build darwin
(and the !windows && !linux fallbacks) were never linted in CI. Linting the
darwin build reports two gosec findings that are clean on linux:

- internal/config/permissions_open_nonlinux.go: G304 on os.Open(path). The
  file comment already explains why it is safe (fixBeadsDirPermissions
  re-checks descriptor identity before chmod); it just had no directive.
- internal/utils/path_case_darwin.go: G103 on unsafe.Pointer. A directive
  existed but sat on the first line of a multi-line unix.Syscall call,
  while gosec reports the argument line four lines down, so it never
  covered the finding. It now sits on its own line above the statement,
  the placement cmd/bd/nocow_linux.go already uses.

Add a GOOS=darwin GOARCH=arm64 CGO_ENABLED=0 cross-lint block to
pr-lint.sh mirroring the windows block (same flags, same lane scoping,
skipped when the native target already matches), and update the three
places that described the wrapper as two passes: engdocs/LINTING.md,
engdocs/CI_CLEANUP_PLAN.md and the comment in main.yml.

The darwin pass is compile-only on the Linux runner: no macOS runner, no
change to the pr-lint platform column. Like the windows pass it is
CGO_ENABLED=0, so darwin+cgo code stays outside its reach.

* docs(linting): bump the Last reviewed marker alongside the pr-lint change
2026-09-28 15:44:52 -07:00
Julian Knutsen
3d76cbd4e9
ci(bazel): dolt-server lane on remote execution (replaces the docker lane) (#6898)
* test(bazel): dolt-server lane for the storage domain + uow job

--config=doltserver runs pr.yml's "Test (storage domain + uow)" job as
remote-executable Bazel tests: each dolt-server target runs an existing
go_test binary with BEADS_TEST_DOLT_SERVER=local, so every Dolt server
is a `dolt sql-server` child of the test process, started from the
pinned dolt the test wrapper already ships. No docker, no
no-remote-exec, and BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 fails a target
whose server cannot start, so cached results are trustworthy.

- db_dolt_test, uow_dolt_test (8 shards), tracker_dolt_test,
  fix_dolt_test (4 shards, BEADS_FIX_REQUIRE_DOLT=1); GOMAXPROCS=4 like
  the job's runner.
- no -short twins of domain, domain/fs, domain/git and
  tracker/conformance, so the lane alone runs every package of the job.
- tag `dolt-server` in the taxonomy; test:prcore (and so test:ci)
  excludes it. The requires-docker variants stay as the A/B control.

Remote (default instance, --nocache_test_results): 9/9 PASS in 113 s
wall; longest shard 111 s (uow). Per-test statuses, subtests included,
equal the job's go test -json on the same tree: 0 differences.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): contract corpus on the dolt-server lane

protocol_dolt_test runs protocol_test as pr.yml's "Contract corpus" job
does (non-race via protocol_race_off, BEADS_PROTOCOL_REQUIRE_DOLT=1) with
the local dolt sql-server backend, in 8 shards, so the corpus golden,
determinism and conformance tests run on remote workers and cannot skip.

Remote: 8 shards, longest 53 s. 153 tests (151 PASS / 2 SKIP, subtests
included) equal the job's go test -json: 0 differences.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the dolt-server lane in place of the docker lane

bazel.yml's container-lane job becomes bazel-doltserver: `bazel test
//... --config=doltserver`, whose targets start their own dolt
sql-servers and so execute on the remote workers (locally for forks and
rbe=off) instead of against the runner's docker daemon. The docker lane
stays reachable as the A/B control: workflow_dispatch dolt-lane=docker
runs --config=docker in the same job, with the image pull that only it
needs. Still advisory; runner, rbe-west gate and secret handling are
unchanged. Policy test and TESTING.md follow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(bazel): reap processes left in a test's private root on exit

A Bazel test timeout signals the test's process group, which misses the
detached bd db-proxy-child processes (Setsid) and the dolt sql-servers they
start. A proxy that died without its idle timeout left its server orphaned on
a persistent remote worker. test_env.sh's EXIT trap now kills every process of
the current user whose command line names the action's private mktemp root
(pkill -f, two passes; a /proc scan when pkill is absent) before removing it.
The root is unique per action and never in the wrapper's own argv; cleanup
still cannot change the test's exit status.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): fail closed on every dolt-server target, checked per rule

The no-short twins (domain, domain/fs, domain/git, tracker/conformance) now
pin BEADS_TEST_DOLT_SERVER=local and BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 like
the lane's other targets, so a Dolt-server test added to those packages fails
instead of skipping into a cached pass.

TestBazelDoltJobMirrorsContainerJobs checked the env with a file-wide
substring search, so a docker variant in the same BUILD file covered for a
dolt-server target that lost it. It now parses each rule tagged dolt-server
and requires both variables, BEADS_FIX_REQUIRE_DOLT on fix_dolt_test,
BEADS_PROTOCOL_REQUIRE_DOLT on protocol_dolt_test, and no no-remote-exec.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 05:17:36 -07:00
Julian Knutsen
671715bb8b
test(testutil): opt-in local dolt sql-server backend for Dolt-backed tests (#6891)
* test(testutil): local dolt sql-server backend behind the Dolt test server API

Add a second backend to internal/testutil behind the unchanged container
API (EnsureDoltContainerForTestMain, RequireDoltContainer,
StartIsolatedDoltContainer[Handle], NewContainerProvider,
RestartSharedDoltContainer, DoltContainerCrashed, IsolatedDoltContainer
Stop/Start/CurrentPort/Exec): the test process spawns `dolt sql-server`
from the pinned dolt CLI instead of a dolthub/dolt-sql-server container.

- BEADS_TEST_DOLT_SERVER=local|container selects it; unset keeps plain
  `go test` on the container (no behavior change) and makes `bazel test`
  use the local backend. Other values fail instead of guessing.
- The CLI (BEADS_TEST_DOLT_BINARY, else PATH) must be the image's version.
- Provisioned like testcontainers' dolt module: root@% (DOLT_ROOT_HOST=%),
  database beads_test, user test/test. Clean env, TZ=UTC, GOMAXPROCS=4,
  own dolt global config with metrics and the version check disabled.
- Loopback port picked here (dolt rejects -P 0), never 3307, retried when
  another process wins the port; readiness requires the server's own
  "Server ready" line plus a bounded ping, so a foreign listener on a
  contested port cannot pass for it.
- State lives under the TMPDIR captured at package init, outside every
  PinSuiteTempRoot root, so suite leak sweeps never see it. The server
  stays in the test's process group (bazel's group kill reaps it) with
  Pdeathsig on Linux.
- BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 now also makes every TestMain that
  only warned about an unavailable server exit 1
  (DoltUnavailableForTestMain), plus the shared proxied harness.
- dolt_benchmark_purge_test finds .dolt_dropped_databases relative to
  Exec's working directory (the data dir in both backends).
- TestDoltServerFingerprint checks each available backend against one
  checked-in fingerprint (users, grants, databases, server variables);
  local-backend tests cover the bind-race retry, 3307, crash detection,
  Stop/Start, Exec cwd, state outside the suite root, and teardown.
- Bazel: the requires-docker variants pin BEADS_TEST_DOLT_SERVER=container
  so the docker lane stays the container A/B control; the test wrapper's
  dolt global config disables metrics and the version check.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(testutil): select the local Dolt test server only when asked for

Under Bazel an unset BEADS_TEST_DOLT_SERVER used to mean the local
backend, so every non-ci `bazel test` started servers for Dolt suites no
lane runs with one (//internal/storage/dolt:dolt_test went from a 16 s
skip to a timeout). Unset now means container everywhere: in a Bazel
action without docker that is the pre-existing warn+skip, and only a
target's env or --test_env=BEADS_TEST_DOLT_SERVER=local starts local
servers. Plain `go test` is unchanged.

The docker variants keep their container pin: a rule env beats
--test_env, so a local-backend invocation cannot flip the docker lane.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(testutil): harden the local Dolt test server's host footprint

- DOLT_DISABLE_EVENT_FLUSH=1 for every dolt the backend runs (and in the
  Bazel test wrapper): the metrics/versioncheck keys stop network calls
  but not the detached `dolt send-metrics` child each command forks on
  exit, which re-created the version probe's removed HOME (one leaked
  bdt-doltver-* per test process) and raced terminate's RemoveAll.
- Every bdt-dolt-*/bdt-doltver-* root records its owner PID, and the
  first use of the backend in a process removes this user's roots whose
  owner is dead (the SweepDeadSuiteRoots rule; a copy, since doltserver's
  tests import testutil). A panicking, timed-out or killed test binary,
  or a package whose TestMain never terminates the singleton, no longer
  leaks a 40-100 MB state dir per run.
- secure_file_priv is the state root's sfp/ directory, set through a
  per-launch --config file (dolt ignores other flags with --config): a
  local server runs as the test user on the host with a passwordless
  root@%, so LOAD_FILE / INTO OUTFILE must not reach the user's files.
  The fingerprint treats that path as the container's "".
- Fingerprint: character_set_server, collation_server,
  max_allowed_packet, transaction_isolation, lower_case_table_names,
  lower_case_file_system (compared on Linux only: APFS reports ON),
  wait_timeout, innodb_lock_wait_timeout.
- The local backend's own tests skip off Linux (no Pdeathsig, so a
  killed test binary would orphan servers on the macOS runners) unless
  BEADS_TEST_DOLT_SERVER=local asks for them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci: compare the container and local Dolt test servers in the domain+uow job

TestDoltServerFingerprint's container half never ran in CI: only the
test-domain-uow job has both the pulled image and the pinned dolt CLI,
and it never ran ./internal/testutil. Run the fingerprint there (with
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 so the container half cannot skip)
so an image bump that drifts the container side fails in the PR.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 04:06:49 -07:00
Tomasz Feliksik
33bed587e0
Accept uutils coreutils timeout in generated hooks, and check the shims against real implementations (#6561)
* Accept uutils coreutils timeout in generated hooks

The generated hook selects a timeout helper only when its --version
banner starts with "timeout (GNU coreutils) ". uutils coreutils reports
"timeout (uutils coreutils) " with the same command line and the same
exit 124 on expiry, so hosts that ship it (Ubuntu 25.10+, NixOS with
uutils-coreutils-noprefix) silently fell through to the Perl alarm, or
to the unbounded direct run when Perl is absent. Follow-up from #5522,
tracked in #5541.

- Widen the identity allowlist to uutils; the match stays by banner so
  Windows timeout.exe is still never probed (#5503).
- Add a backend-selection case using the GNU stub with the uutils
  banner: fails on the old shim (selects gtimeout), passes with the
  widening.
- Regenerate the managed sections in the tracked .githooks/* so
  TestTrackedManagedHookSectionsMatchGenerator stays green.
- Update the hook-timeout passages in docs/reference/git-integration.md
  and troubleshooting.md, which described the probe as GNU-only.

Verified on NixOS with uutils 0.10.0 as /run/current-system/sw/bin/timeout:
with Perl removed from PATH and a bd that hangs, the old shim runs
"without timeout" for the full duration; the new shim expires at
BEADS_HOOK_TIMEOUT and continues.

Agent-Signature: claude-code-claude-opus-5-unknown-reasoning on behalf of Tomasz Feliksik

* Regenerate tracked .githooks managed sections with make githooks-regen

TestTrackedManagedHookSectionsMatchGenerator holds the managed section in
each tracked .githooks/* byte-equal to the cmd/bd/hooks.go generator, but
nothing regenerated them: after a generator change a contributor had to
splice the new section in by hand, and the failure only dumped both
texts. The v1.2.0 bump reddened main this way (ce20a64bb), and #6561 shipped
its first revision with the five hooks stale.

- BD_UPDATE_GOLDEN=1 makes the test rewrite the section in place, keeping
  everything around the markers (the project's own hook logic) untouched.
  Same switch the package's events golden already uses.
- `make githooks-regen` wraps it, next to `corpus-regen`.
- The drift failure now names that command.
- The exactly-one-section check runs before any rewrite.

Verified: drifting one byte inside .githooks/pre-push fails the test with
the hint; `make githooks-regen` restores the file to the same blob SHA;
the test is green again.

Agent-Signature: claude-code-claude-opus-5-unknown-reasoning on behalf of Tomasz Feliksik

* Check the tracked hook shims against real timeout implementations

The Go tests pin the shim's helper selection with scripted stand-ins, so
they can only encode what the author believes each real `timeout` does.
That is where the last two defects lived: uutils prints a banner the
allowlist did not know (#5541), and busybox — which #5541 suggests
adding — exits 143 on expiry rather than 124, so allowlisting it would
turn every deadline into a failed hook and a blocked push.

checks.nix adds a flake check that runs the tracked .githooks/* managed
sections (held equal to the generator by
TestTrackedManagedHookSectionsMatchGenerator) against the binaries
themselves: GNU coreutils, uutils, busybox and toybox `timeout`, each
with and without perl, under dash, bash and busybox ash, for all five
hooks — 150 cases. Expectations come from a model of the contract (the
allowlist as data), so widening it is one line here and the binaries
either agree or they do not.

A fake bd on the shim's restricted PATH blocks on a held-open fifo until
a helper kills it, records the signal and re-raises it, so each case
asserts the backend that ran (TERM from timeout, ALRM from perl's alarm)
and not merely that the shim finished. No clock: a helper that fails to
kill the fake trips the harness's own `timeout 10`. Every case asserts
exit 0, since anything else blocks the user's push. Reading .githooks/*
instead of building bd keeps it at ~1 s and off the Go build's critical
path; CI's test-nix job runs it after its existing help-text smoke test.

Mutation-checked: restoring main's GNU-only sections fails all 30 uutils
cases (15 unbounded, 15 on the perl backend); allowlisting busybox with
a loosened probe fails all 30 busybox cases with hook exit 143.

Agent-Signature: claude-code-claude-opus-5-unknown-reasoning on behalf of Tomasz Feliksik

* Run the hook-timeout check under the gtimeout candidate name too

The shim probes `timeout` and then `gtimeout`, but the matrix only ever
put a binary named `timeout` on PATH, so the second candidate was never
run against a real implementation. Whether a helper is accepted under
that name depends on what its banner does with argv[0].

Measured: the uutils multicall dispatches on the argv[0] SUFFIX
(src/bin/coreutils.rs, `binary_as_util.ends_with(util)`) and then prints
the canonical utility name — invoked as gtimeout it reports
"timeout (uutils coreutils) 0.8.0" (this flake's nixos-25.11 pin; 0.11.0
identical), which the allowlist accepts. busybox and toybox know no
gtimeout applet and fail the probe outright, rejected as before. GNU is
immune by construction: Homebrew's gtimeout is a separate binary whose
banner is the fixed "timeout (GNU coreutils) 9.11"; nixpkgs' coreutils
is a single-binary multicall that refuses a foreign name ("unknown
program 'gtimeout'"), so it gets no variant here.

Each multicall is now also installed on a case PATH as `gtimeout` ALONE
(`installs`, derived from `implementations`; `backendFor` stays keyed on
the implementation, so the model's claim is "uutils is allowlisted
whatever it is called"). 240 cases on Linux, 80 on darwin (no busybox or
toybox there, and no busybox-ash shell). The shadowed
`out=` inside run_case is `case_out` now: it only survived because every
case is a backgrounded subshell, and a de-backgrounded case would have
died on the final `touch "$out"`.

Mutation-checked: installing the variant as `xtimeout` (a name the shim
never probes) fails exactly the 30 uutils-as-gtimeout cases — 15 with
nothing killing the fake bd, 15 on the perl backend (ALRM) instead of
coreutils; restored, 240 of 240 pass. The busybox/toybox variants cannot
distinguish (their expectation is rejection either way); their weight is
that the probe does not blow up on a foreign name.

* Run the flake checks on main too, by nix flake check; docs wording

pr-risk.yml's test-nix job ran the hook-timeout check on PRs only; the
byte-identical job in main.yml never got the step, so post-merge main
never ran it. Both jobs now run `nix flake check -L`: no hardcoded
`x86_64-linux` (the step beside it, `nix run .#default`, was already
system-agnostic), and a check added to checks.nix runs without a
workflow edit. checks.aarch64-darwin and .x86_64-darwin evaluate.

Measured on ubuntu-latest with a cold Nix store (fork run 35700791032,
this tree, 240 cases): `nix flake check -L` 3.6 s including substitution
of uutils, busybox, toybox, dash and perl; 0.08 s warm — beside the
4 min 9 s the existing `nix run .#default` step spends building bd in
the same job. The 10-minute timeout is not in question.

git-integration.md: on Git for Windows the preferred deadline backend is
"a coreutils timeout (GNU or uutils)", not GNU alone, since both now
qualify. troubleshooting.md: reflow a ragged paragraph.

* review: close the duplicated contracts around the uutils timeout accept

Maintainer review fixes for #6561. No change to the shim's behavior or to the
generated/tracked hook sections.

- cmd/bd/hooks_timeout_process_test.go: the uutils fixture was built with
  strings.Replace, which returns its input unchanged when the pattern is
  absent, so a later edit to the GNU stub's banner would silently degenerate
  the uutils case into a second copy of the GNU one — still green, no longer
  testing the new accept. Hoist the derivation and fail loudly when the banner
  no longer matches.
- cmd/bd/hooks_timeout_process_test.go: findHookProcessGNUTimeout held a second,
  un-widened copy of the identity allowlist, so on exactly the hosts this PR
  exists for (uutils timeout, no gtimeout) the only Go test that drives a real
  timeout binary skipped with a reason the fix had just made false. Widen the
  probe's case, rename it findHookProcessCoreutilsTimeout, and reword the skip
  and the subtest. Measured against stub PATHs: uutils goes rc=1 -> rc=0, GNU
  stays rc=0, a busybox banner stays rc=1.
- cmd/bd/init_hooks_test.go: TestGenerateHookSection_Timeout asserted only the
  GNU alternative, so deleting the new uutils one from the generator left the
  structural contract green (verified: it passes under that mutation before
  this change, and reports "section missing uutils coreutils identity check"
  after). Add the companion assertion.
- cmd/bd/init_hooks_test.go, Makefile: the regen arm rewrites tracked source
  files, so give it its own BD_UPDATE_HOOKS_GOLDEN rather than the package's
  shared BD_UPDATE_GOLDEN — an unscoped `BD_UPDATE_GOLDEN=1 go test ./cmd/bd/`
  used to turn genuine hook drift into a silent working-tree rewrite that
  reported PASS.
- cmd/bd/init_hooks_test.go: locate the managed section by marker prefix
  instead of the versioned marker line. Tracked hooks drift both from generator
  content and from the release version stamp; the exact-line lookup made the
  second class fatal ("tracked hook missing ...") before the regen arm was
  reachable, so `make githooks-regen` failed at exactly the moment a release
  engineer reaches for it. The versioned markers are still verified — they are
  part of the byte compare.
- checks.nix: the `rc -eq 124` safety-net verdict discarded both $case_out and
  the already-captured $signal, so the one failure mode that is intermittent
  and not locally reproducible was also the only one with no diagnostics.
  Report them like the `rc -ne 0` branch below it.
- docs/reference/troubleshooting.md, docs/reference/git-integration.md: mirror
  the coreutils wording across both docs this PR touches, and generalize the
  re-install guidance in both. Managed sections are never reported Outdated
  (getHookVersion returns IsShim for them and computeHookStatuses gates on
  !IsShim), so nothing tells an existing install it is stale; the sentence was
  still scoped to the older name-only check (#5503).
- CHANGELOG.md: add the [Unreleased] entry for #5541, worded to the actual
  scope — with Perl installed the shim still reached the Perl alarm arm, so the
  lost deadline bit uutils hosts without Perl — say that existing installs must
  run `bd hooks install` once, and drop the claim that stock NixOS ships uutils
  (this PR's own checks.nix takes GNU from pkgs.coreutils).

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-27 03:38:45 -07:00
ecuthiell
fd8da7b13d
test(cmd/bd): scope fresh-workspace BEADS_DIR isolation (#6519)
* test(cmd/bd): share fresh-workspace environment isolation (saa-cgh.120.22)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* test(cmd/bd): restore mode globals after isolation regression (saa-cgh.120.22)

Agent-Signature: unknown-model/unknown-reasoning on behalf of Ewen Cuthiell

* docs(test): scope environment restoration claim to BEADS_DIR

Address the wording nit in PR #6519 review 5189902686 without widening the helper.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Ewen Cuthiell
2026-09-26 18:49:44 -07:00
Julian Knutsen
3c30a28d07
perf(ready): answer the page and its total in one pass (#6731)
* perf(issueops): answer a ready page and its total in one pass

`bd ready --json --limit 1` against a remote SQL store (~55 ms per round
trip) took 5.4 s and ~100 sequential round trips; `--limit 0` (557 rows)
took 3.8 s. Asking for one row was slower than asking for all of them. The
data is small, so the cost was purely statement count:

- a capped page re-ran the whole ready predicate in a SECOND transaction
  (CountReadyWorkInTx, behind a second defer-wake sweep) just to print
  "Showing 1 of 556";
- every leg re-probed the same facts: wisp_dependencies existence, wisps
  emptiness, and the future-deferred-parent scan (2 probes + 4 child
  queries), once per table family and again in the count pass.

This change:

- Adds a per-transaction probe (probeReadyWorkInTx): one statement of
  EXISTS subqueries answers wisps-present, wisp_dependencies-exists,
  future-deferred-parent-exists and (when sizing) issues/wisps ID collision;
  the deferred-parent children are then read in one UNION ALL instead of
  four queries. The facts and the rendered ID sets are shared by both
  families. A missing-table error (pre-wisp database) falls back to the old
  per-table probes, so their tolerance and error classification are
  unchanged.
- Adds GetReadyWorkWithCountsAndTotalInTx and
  DoltStorage.GetReadyWorkWithCountsAndTotal: the capped page's ID query
  carries COUNT(*) OVER (), evaluated over the full predicate before LIMIT,
  so the total costs no extra statement. The overlap count only runs when
  the probe saw an ID in both tables. Window functions are supported by
  every engine this code runs on (Dolt/go-mysql-server server and embedded,
  MySQL 8, Postgres, SQLite >= 3.25). The externaldeps decorator overrides
  the new method so its exclusions apply to page and total alike, and now
  also overrides ReadyCounter() so the text-mode total honors the same
  exclusions (it used to pass through to the inner store and count
  externally blocked issues); telemetry instruments the new method.
- CountReadyWorkInTx (ReadyCounter role, text `bd ready`) uses the probe
  and counts both families in one statement.

Whole-command statements for `bd ready --json`, counted from a local dolt
sql-server query log (13 of each are startup/config reads this change does
not touch):

  store                          --limit 1   --limit 0
  issues only                    28 -> 18    20 -> 17
  issues + wisps                 34 -> 19    23 -> 18
  issues + wisps + deferred epic 46 -> 20    29 -> 19

TestReadyWorkStatementBudget holds the per-transaction statement count at
or under today's budget on embedded Dolt. TestReadyWorkPageTotal and
TestReadyWorkPageTotalFilterMatrix check the page and total against
hand-written expected ready sets (wisps, deferred parent, blocked, ID
collision; sorts, label/assignee/priority/type/parent/deferred/exclude
filters) and every capped page against the unbounded listing's prefix.

stdout (plain and BD_JSON_ENVELOPE=1) and the stderr hint were
byte-identical before/after for every case above. The one intended output
change: with an external dependency present, the --json total (and the
text "of N") now excludes externally blocked issues, as the page already
did; before, the total counted them.

Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(ready): take the --json total in-band instead of a second count pass

`bd ready --json --limit N` listed a page, then - when the page came back
full - opened a second store call (ReadyCounter.CountReady: a new read
transaction plus its own defer-wake sweep) only to print "Showing N of M"
on stderr and the pagination total in the JSON envelope. On a remote store
(~55 ms per round trip) that pass is why `--limit 1` (5.4 s) was slower
than `--limit 0` (3.8 s).

The --json branch now calls GetReadyWorkWithCountsAndTotal, which returns
the page and the size of the whole ready set from the same transaction.
The total is the same identity the ReadyCounter role promises, taken over
the listing's own filter, so the hint text, the envelope's pagination and
the stdout JSON are identical except that the total now honors
external-dependency exclusions (the externaldeps decorator applies them to
page and total alike; the old second pass counted externally blocked
issues). The text-mode branch still asks the ReadyCounter role, which the
previous commit made cheaper and made apply the same exclusions, so text
and --json report one number.

The embedded CLI test now pins the exact "Showing 2 of N" hint and the
envelope's pagination total against an unbounded listing.

Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 23:50:09 -07:00
Julian Knutsen
b3d17caa6d Merge release/1.3.0 into main (post-tag forward-port)
Forward-ports the 38 commits that shipped on release/1.3.0 (tagged v1.3.0 at
f45b249ce) into main, as RELEASING.md mandates after a tag. None of them had
ever been forward-ported. Recorded as a merge so main becomes a superset of
the release branch and the lineage is in the history.

Migration ordinal collision (the reason this needed a human-shaped merge):
both branches independently claimed clone-local ignored slot 0026 under
different filenames, so git merged both in with no conflict and left the
series with two migrations at ordinal 26 — a single-integer cursor cannot
address that, and migrationSource.list() panics on it.

Main's ignored/0026_add_wisps_current_revision is renumbered to 0027.
Release's 0026_dep_rekey_dedup_marker is in the shipped v1.3.0 tag, so field
stores have already recorded it at 26 and it can never move; main's twin has
never shipped in any release, so it is the one that is free to move. 0027 was
unused in the ignored series (both sides topped out at 26).

This trips check-migration-hygiene.sh Check C (frozen migrations), which has
no exemption mechanism. That is expected and accepted: breaking the collision
requires a rename, and the ignored series' cursor table is itself dolt-ignored,
so its content hashes are clone-local and never participate in the cross-clone
ContentHashSkew comparison that Check C exists to protect. Main's 0067 body is
deliberately NOT edited for the same reason in reverse — it is main-plane and
its hash IS compared across clones.

Conflicts resolved:

  CHANGELOG.md                       release's finalized [1.3.0] section kept
                                     byte-identical; main's two unreleased
                                     entries (#6023, #5859) moved up under
                                     [Unreleased] where they belong
  cmd/bd/init_safety.go              union — release's added paragraph is
  cmd/bd/init_safety_help.go         accurate against the merged code (three
                                     exit-12 sources, incl. init.go:3147)
  internal/storage/uow/dolt_sql_provider.go
                                     kept main's newProvider refactor AND
                                     release's readOnly field/assertion;
                                     newProvider gained readOnly: opts.readOnly
                                     so release's read-through-refused-migration
                                     behavior survives main's refactor
  .github/workflows/main.yml         single -timeout=30m per release's #6091
  scripts/ci_workflow_test.go        design (flag on the invocation, not in
                                     matrix.test-flags); the shadowed 25m in
                                     matrix.test-flags removed
  scripts/migration-test/lib/versions.sh
                                     union — main's darwin/arm64 corpus plus
                                     release's v1.2.2 linux/amd64 entry
  integrations/beads-mcp/uv.lock     ruff 0.16.5 (release, newer); block is
                                     internally consistent, uv lock --check ok
  default.nix                        vendorHash left at main's value and is
                                     KNOWN STALE — see below

Version stamp lands at 1.3.0 (main had been sitting at the previously-released
1.2.2, this project's between-releases pattern). check-versions.sh: 24/24.

Known follow-up: default.nix vendorHash. The merged go.mod/go.sum is a genuine
union (release's kin-openapi/oapi-codegen bumps + main's newer Dolt stack) and
matches neither side, so neither recorded hash is correct. It must be
regenerated with scripts/update-nix-vendorhash.sh. The repo's automatic fixer
(update-vendor-hash.yml) is dependabot-only and will not run on this PR.
2026-09-15 22:09:13 +00:00
Julian Knutsen
f45b249ce6
release: make release/1.3.0 tag-ready — revert #6281, stamp 1.3.0, date the changelog (#6564)
* revert: drop the journaled ownership handoff API from the release branch (#6281)

This reverts commit 74e25cba15, restoring the
release branch to the exact tree validated as v1.3.0-rc.2.

#6281 landed one commit past rc.2, so no release candidate ever exercised it.
Nothing imports `internal/ownershiphandoff` and no command registers it — the
package is unreachable from any `bd` entry point, so removing it cannot change
shipped behavior. It is superseded by a redesign that builds a fresh
`internal/ownershiphandoffv2` and deletes this package outright, and its
Hooks/Execute shape is the scaffolding of an architecture the owner rejected.

The commit never landed on main, so this is release-branch-only and nothing is
lost anywhere else. Reverting also returns
engdocs/design/ownership-handoff-contract.md to its prospective voice, which is
accurate again now that the implementation it described is gone.

* release: stamp 1.3.0 and date the changelog entry to the release day

`verify-version-consistency` in .github/workflows/release.yml compares the tag
name against the `Version = ` constant in cmd/bd/version.go and fails the job on
any mismatch, so a v1.3.0 tag against the 1.3.0-rc.2 stamp would abort the
pipeline before it built anything.

Bumped with ./scripts/update-versions.sh 1.3.0, which carries the stamp into all
seven version files, the two uv.lock entries, the ten githook managed-section
markers and the winres strings. The winres numeric file_version/product_version
and the manifest assemblyIdentity never carried the prerelease suffix and are
unchanged. ./scripts/check-versions.sh passes 24/24.

CHANGELOG.md's [1.3.0] heading carried 2026-08-28, the branch-cut date, which
predated both of its own release candidates. It and the matching
cmd/bd/info.go versionChanges entry now read 2026-09-15, the release date. The
section's reviewed content is untouched and [Unreleased] remains empty.

Dating the entry truthfully also forced its position. versionChanges is
reverse-chronological and getVersionsSince treats array index as recency, so
TestGetVersionsSinceOrder failed: the 1.3.0 entry still sat below its own rc.1
and rc.2, where the branch-cut date had made it look like the oldest of the
three. The entry moves to the head of the array, byte for byte unchanged. That
restores the strict date ordering and fixes the behavior it was hiding — an
rc.2 tester upgrading to the release now gets the [1.3.0] digest from
`bd upgrade review` instead of being told there is nothing new.
2026-09-15 12:13:59 -07:00
Julian Knutsen
74e25cba15
feat: add journaled direct-local ownership handoff API (#6281)
* feat: add journaled direct-local ownership handoff API

* fix: harden ownership handoff retries and identity

* fix(ownershiphandoff): durable checkpoints, stated retry contract, decomposed Execute

Maintainer review fixups for PR #6281, closing the two correctness majors and
both measured-complexity band majors raised by the quality scorecard.

save() now fsyncs the journal's parent directory after the atomic rename,
mirroring configfile.SaveProxiedServerClientInfo. Without it, power loss could
lose the commit reservation while the Commit hook's external effects survived,
so a retry would re-invoke Commit — exactly what the reservation's own comment
promises to prevent.

Hooks now states the retry contract it always relied on: Execute resumes from
the last durable phase, so Snapshot/Configure/StopLegacy/Verify must be
idempotent, an already-absent legacy owner is success rather than a refusal,
and Configure must not mutate the legacy-owned scope. Commit remains the sole
at-most-once hook. The stop leg gains a pre-invocation reservation
(legacy_stop_in_progress) so a partial or interrupted stop is reported as a
mutation instead of an untouched legacy owner, with a matching Load invariant.

Execute is decomposed into an unlocked preflight and a handoffRun phase
machine; ValidateRequest and Load split along their existing seams. Worst
changed-function complexity drops from cyclomatic 44 / cognitive 87 to 18 / 16.

Also folds in the non-gating cleanups: cumulative mutation reporting on both
identity-conflict paths, journal_unreadable typing, a truthful dry-run phase,
lock contention distinguished from lock I/O failure (lock_unavailable), a
bounded commit-recovery message, the ambiguous-commit contract sentence, the
dead root guard and empty-host message, and new tests for the corrupt journal,
mid-flight dry run, stop reservation and retry, and message growth.

The second review round reproduced three operator-surface defects, all fixed
here. commitRecoveryMessage is now a fixed point rather than regenerative: an
ambiguous-commit journal that already carries the explanation is returned
unchanged, so from the third retry onward the original commit failure survives
instead of being overwritten with bare boilerplate — it used to vanish from both
the journal and the returned error precisely when a recovery loop had set in.
resolveRoot splits its precondition branch: a root that exists but is not a
directory no longer wraps a nil error, which rendered the literal
"root must be an existing directory: %!w(<nil>)" on the public ValidateRequest
path, while a missing root still carries its real cause. Result.Mutates gains
the doc sentence its cumulative-on-refusal, attempt-scoped-on-no-op semantics
needed. Each behavior is pinned: the recovery test drives a fourth attempt and
asserts the cause survives in both the journal and the returned error, and the
rejection table gains a non-directory root case beside a message-asserting test.

The Windows directory-fsync exposure raised that round is deliberately not
addressed here. The identical unconditional os.Open+Sync shape already ships at
base in configfile.SaveProxiedServerClientInfo, so guarding only the new site
would fix an inert copy and leave the shipping one broken while making the two
diverge. It is tracked as one two-site follow-up to be settled on a real Windows
runner before a consumer makes this path reachable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 16:43:53 -07:00
Julian Knutsen
24a9da6cc7
test(proxy): topology-parameterized history and remote front-door matrix (#6269)
* feat(ga-p9iuv.22): add shared tracker proxy storage seam

* fix(ga-p9iuv.22): preserve tracker storage capabilities safely

* test(ga-p9iuv.22): cover tracker adapter label semantics

* test(ga-p9iuv.22): harden tracker adapter contracts

* fix(ga-p9iuv.22): reject invalid proxied tracker stores

* feat(ga-p9iuv.1): route Linear and Jira front doors through proxy store

* test(ga-p9iuv.1): exercise tracker mutations through UOW store

* fix(ga-p9iuv.1): preserve proxied Linear endpoint config

* test(ga-p9iuv.1): cover proxied Linear endpoint and direct config

* feat(ga-p9iuv.19): add typed proxied capability matrix

* feat(proxy): enforce typed capability matrix before provider

* test(proxy): cover command capability rows and claim caps

* test(proxy): check capability rows across topologies

* fix(proxy): validate capabilities before workspace side effects

* fix(proxy): refuse ready claim max rows caps

* feat(proxy): add shared maintenance capability policy

* fix(proxy): key maintenance refusals by command path

* test(proxy): cover nested maintenance refusals

* fix(proxy): enumerate nested maintenance command paths

* fix(proxy): refine compact and nested maintenance policy

* test(proxy): prove maintenance refusals preserve files

* fix(proxy): cover admin and remote maintenance refusals

* fix(proxy): refuse doctor before store initialization

* test(proxy): define history capability command matrix

* fix(proxy): wire history capability refusals to front door

* fix(proxy): refuse top-level sync history path

* fix(proxy): preserve maintenance refusal codes

* feat(proxy): classify history as supported UOW API

* feat(proxy): classify transform capability paths

* test(proxy): harden transform capability matrix

* fix(proxy): drive transform validation from exact rows

* fix(proxy): drive transform matrix from normalized flags

* fix(proxy): bound external backend dials per attempt

* test(proxy): cover bounded blocked backend dial

* fix(proxy): make transform capability policy row-driven

* fix(proxy): make transform capability policy row-driven

* fix(proxy): classify formula swarm and merge-slot refusals

* test(proxy): enforce workflow refusal contract

* fix(proxy): classify Dolt remote history boundaries

* test(proxy): enforce history refusal contract

* test(proxy): cover canceled backend dial cleanup

* test(proxy): cover real TCP blackhole cancellation

* test(proxy): cover external transport lifecycle matrix

* test(proxy): assert stable dial failure diagnostics

* test(proxy): add outage reconnect acceptance matrix

* test(proxy): make unix outage bridge restartable

* test(proxy): bound outage subprocess cleanup

* test(proxy): cover upstream restart with retained endpoint bridge

* test(proxy): stabilize managed-local outage contract

* fix(proxy): retire managed proxy when backend exits

* test(proxy): accept managed child retirement restart

* test(proxy): cover workflow refusals at bd front door

* test(proxy): cover history VC front doors

* test(proxy): pass valid workflow refusal arguments

* test(proxy): assert text and json refusal contracts

* test(proxy): assert exact workflow refusal codes

* test(proxy): guard refusal issue-row stability

* fix(proxy): expose non-mutating capability refusals

* test(proxy): match stable refusal text

* test(proxy): compare refusal artifact snapshots

* test(proxy): assert exact refusal messages

* test(proxy): retain direct workflow behavior

* test(proxy): match nested refusal messages

* test(proxy): cover direct-only front-door refusals

* test(proxy): provide valid direct-only command args

* fix(migrate): repair missing proxied sidecar on reentry

* fix: checkpoint Dolt mode migrations for safe retry

* fix: harden migration journal topology and retries

* fix: fsync migration journal removal

* fix: fail on migration journal directory sync errors

* fix: reject external ownership in every migration phase

* fix: enforce canonical shared migration topology

* fix: enforce live migration topology

* test: add migration front-door and topology regressions

* fix: require physical Dolt roots for migration

* test: strengthen migration front-door contracts

* fix: reject missing or invalid Dolt migration roots

* fix: keep shared migration dry-runs side-effect free

* fix: require Dolt repository identity for migrations

* fix: validate Dolt repository identity metadata

* fix: require typed Dolt repo state identity

* test: expand migration front-door CRUD assertions

* test: make migration JSON dry-run front-door safe

* test: sanitize Dolt environment for migration front door

* test: use cgo binary for migration front door

* test: omit empty Dolt overrides in migration front door

* test: isolate migration front door from test mode

* test: isolate home for migration front door

* test: stop supervised Dolt before reverse migration

* fix: reject empty Dolt head refs

* test: harden migration topology invariants

* test: harden external and shared topology refusals

* test: snapshot migration refusal artifacts

* test: cover external migration refusal phases

* fix: refuse stale proxied sidecars during migration

* test: strengthen migration artifact and external front door checks

* feat: type external migration refusals

* test: assert external refusal starts no proxy

* fix: validate migration journal sidecar topology

* fix: make sidecar recovery phase aware

* fix: honor persisted shared topology during migration

* test: exercise external journal migration front door matrix

* test(tracker): add shared conformance harness

* test: harden external migration matrix fixtures

* test: preserve durable dolt gate lock in matrix

* test: preseed gate lock for refusal matrix

* test: cover malformed migration state front door

* fix: type malformed migration config refusals

* fix: type migration preflight state errors

* test: separate migration JSON stdout from warnings

* test: reach typed malformed journal refusal

* test: assert migration retry phase and idempotence

* test: cover shared migration checkpoint retries

* fix: make reverse migration idempotent without sidecar

* test: cover proxied to shared retries

* fix: derive shared topology from persisted YAML

* fix: validate shared migration no-op state

* fix: allow completed proxied shared no-op

* fix: validate forward proxied no-op sidecar

* test: reject bad already-proxied sidecars

* fix: reject stale proxy controls on no-op

* fix: fail closed on malformed migration workspace yaml

* test: pin migration yaml topology precedence

* Cover forward migration no-op ownership refusals (ga-p9iuv.26.5.2)

Agent-Signature: unknown-model/unknown-reasoning

* Strengthen migration retry sentinel coverage (ga-p9iuv.26.1)

Agent-Signature: unknown-model/unknown-reasoning

* Expand migration fault retry artifact coverage (ga-p9iuv.26.1)

Agent-Signature: unknown-model/unknown-reasoning

* Add reverse migration frontdoor fault matrix (ga-p9iuv.26.1.1)

Agent-Signature: unknown-model/unknown-reasoning

* test: cover shared migration frontdoor fault matrices

* test: strengthen direct migration matrix assertions

* test: preserve migration dependency edges

* test: assert exact migration dependency edges

* test: parse migration dependency edges from batch list

* test: fix shared migration matrix compile

* test: verify shared dependencies before shutdown

* test(proxy): cover history remote front-door topology matrix

* docs: define direct-local ownership handoff contract

* fix(ci): repair pre-review CI failures on proxy parity branch

Five PR-introduced failures across the docs, PR Core, and Proxied Dolt Cmd
lanes. Each fix is scoped to the evidenced defect.

- test/docsync: TestEveryDocsPageIsPublished flagged
  docs/architecture/ownership-handoff-contract.md as unpublished. It is an
  internal Beads/Gas City handoff contract with no Mintlify frontmatter and
  no inbound links, so move it to engdocs/design/ beside its peers rather
  than publishing it.

- cmd/bd: TestProxyWorkflowRefusalContractAndNoMutation asserted on
  commandDidWrite, a process-wide atomic latch set by 71 non-test sites and
  never reset, so it reported an earlier test's write. Baseline and restore
  it around the assertion, matching the save/reset/restore idiom already
  used in proxied_lookup_failure_test.go and readonly_policy_test.go.

- cmd/bd: the new pre-provider gate short-circuited compact.go's refusal
  with new wording, breaking TestProxiedServerCompactDolt/non_dolt_mode_-
  rejected in an unchanged test. Restore the long-standing message; the
  proxy.compact.unsupported code is unchanged.

- internal/storage/dbproxy/proxy: the unix subtests bound sockets under
  t.TempDir(), whose embedded test name overflows macOS's ~104-byte
  sun_path limit ("bind: invalid argument"). Add unixSocketPath() to place
  them in a short temp dir.

- cmd/bd: migration topology compared recorded and canonical Dolt roots
  lexically, so macOS's /var -> /private/var symlink made equivalent paths
  compare unequal. Add sameMigrationRoot(), mirroring the existing
  resolveExistingPrefix idiom in cmd/bd/doctor/git.go.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(review): address blocker and major findings from adopt-pr review

Maintainer review fixup for the proxy parity branch, iterations 1-2.
Blockers and majors from the review synthesis and quality scorecard.

Blockers:

- cmd/bd/migrate_dolt_mode.go: resuming from migratePrepared rebuilt the
  migration plan from the current argv instead of replaying the recorded
  one. A resume that omitted or changed --idle-timeout wrote a sidecar the
  target_configured check then rejected, and because that check runs only
  after the phase advances past migratePrepared, the block never ran again
  — the workspace stayed wedged in proxied-server mode with no in-tool
  recovery. Replay j.Sidecar and warn when the supplied value is ignored.

- internal/tracker/store.go: NewStore double-wrapped already-adapted values.
  directStore embeds the storage.Storage interface, so *directStore also
  satisfies storage.Storage and was re-wrapped as
  &directStore{Storage: &directStore{...}}; the inner value can never
  satisfy storage.IssueLifecycleStore, which silently degraded
  ApplyIssueUpdate to a bare UpdateIssue — no labels, no lifecycle
  transaction. Match the tracker's own adapters by concrete type first.
  Testing value.(Store) first, as the review suggested, does not work:
  storage.Storage is a strict superset of Store, so a raw direct store
  satisfies Store too and would be returned un-adapted, relocating the same
  bug. The branch order is documented at the call site.

Majors:

- internal/tracker/store.go: directStore.ApplyIssueUpdate now calls
  syncIssueLabels, which normalizes both label sets (trim, drop empty) and
  restores the semantics the pull path had before this seam moved behind
  ApplyIssueUpdate. The un-normalized local copy churned remove+add on
  whitespace-only differences and let an empty label reach AddLabel; it is
  deleted rather than left as a second, divergent implementation.

- Both regressions were invisible to the suite. Adds
  TestNewEngineKeepsDirectStoreAdaptationIntact, TestNewStoreIsIdempotent,
  TestDirectStoreApplyIssueUpdateNormalizesLabels, and
  TestMigrateToProxiedServer_ResumeReplaysRecordedIdleTimeout (changed /
  omitted / never subtests, each also asserting the reverse-migration escape
  hatch still recovers). All four were verified to fail against the
  pre-fix production code with the predicted symptoms.

Non-gating follow-ups from the scorecard, fixed while nearby:

- cmd/bd/proxy_capability.go: treat ProxyOutcomeNA as "this command has no
  such flag" rather than a refusal, so an assert on an N/A row cannot return
  a non-nil error carrying an empty message.

- cmd/bd/errors.go: emit the rule's real Mutates value in the JSON refusal
  envelope instead of a hardcoded false.

- cmd/bd/main.go: the doctor pre-provider check returned unconditionally,
  so a nil result (every doctor subcommand) skipped the legacy-store guard
  and autocommit-mode resolution that other skipsStoreInit commands run.
  Return only on a real refusal.

- internal/tracker/engine.go: type NewEngine's store parameter as Store so
  the seam names its contract, and drop the now-dead applyPullIssueUpdate.

- CHANGELOG.md: record the proxied `bd ready --claim` row-cap refusal, which
  was a behavior change shipped without a release note.

Iteration 2 -- gating major (uowStore/directStore external_ref divergence):

- internal/tracker/uow_store.go: GetIssueByExternalRef resolved through a
  single merged search over both storage planes, which has no plane
  preference: it is a UNION ALL ordered by content and terminating in
  `id ASC`, so one external_ref present on both planes resolved to whichever
  row's id sorted first. The direct backend is explicitly issues-first --
  issueops.GetIssueByExternalRefInTx queries `issues` and falls through to
  `wisps` only on sql.ErrNoRows, and says why ("so that pushed ephemeral
  beads are found during pull dedup"). The consumer is the pull dedup, which
  updates whatever row comes back, so the divergence was a silent write to
  the wrong local bead under --proxied-server only. Query the issues plane
  first via SkipWisps, then fall back.

  The fallback is the merged search rather than the review's literal
  Ephemeral=true: that filter contributes an `ephemeral = 1` predicate the
  direct backend's `SELECT id FROM wisps WHERE external_ref = ?` does not
  have, and wisp-plane rows legitimately carry ephemeral=0 (NoHistory beads;
  see types.IssueFilter.EphemeralTier). Taking it literally would have traded
  one divergence for a narrower one. Verified: implementing the prescription
  fails the ephemeral_flag_clear subtest below.

- internal/tracker/uow_store_engine_test.go: the engine fake now models plane
  routing (SkipWisps / Ephemeral / merged) instead of one flat map, so the
  contract is expressible at all. Adds
  TestUOWStoreGetIssueByExternalRefPrefersIssuePlane, whose wisp id sorts
  first so "first merged hit" fails it, and
  TestUOWStoreGetIssueByExternalRefFallsBackToWispPlane over both ephemeral
  values. Verified to fail against the pre-fix uow_store.go.

- internal/tracker/conformance: the shared suite gains a Wisps plane and
  external_ref_resolution_prefers_issue_plane, so this is a contract both
  backends are held to rather than one adapter's unit test.

Iteration 2 -- non-gating follow-ups from the scorecard:

- cmd/bd/migrate_dolt_mode.go: scope the migratePrepared sidecar-equivalence
  check in runMigrateFromProxiedServer to reverse journals. A forward journal
  at that phase records the sidecar it is about to write, so a crash in the
  pre-write window leaves none on disk -- the correct state -- and the
  unscoped check refused the documented escape hatch by naming a missing
  file, stranding the workspace. Tested against TargetMode's forward value so
  the guard stays live for both reverse targets, "server" and
  "shared-server". Adds
  TestMigrateFromProxiedServer_ForwardJournalAtPreparedIsNotSidecarRefused,
  verified to fail against the unscoped guard.

- cmd/bd/linear.go: buildLinearClientAPIOnly returned the OAuth client inline
  and never reached the WithEndpoint block, so a configured
  linear.api_endpoint was honored for an API key and ignored for OAuth. Hoist
  the endpoint application after the branch, as Tracker.Init does, and add
  the OAuth mirror of the existing endpoint test.

- cmd/bd/ready_max_rows_proxied_test.go: pin the proxied --max-rows refusal
  on `bd ready --claim`. It lives in RunE above the dispatch and nothing
  below it can enforce a cap, but the existing coverage all sits below it.
  Covers both cap sources (--max-rows and BEADS_MAX_ROWS) and the negative
  case that the refusal is conditioned on a live cap, not on --claim.

- engdocs/design/ownership-handoff-contract.md: trim the trailing blank line
  at EOF flagged by `git diff --check`.

Iteration 3 -- non-gating residual on the iteration-2 fix:

- internal/tracker/uow_store.go: the issues-first fix above split one logical
  lookup across two read transactions. uow.RunTxRead opens a fresh unit of
  work per call, so calling s.SearchIssues twice left the wisp fallback's
  premise -- "the issues plane is already known empty" -- without a shared
  snapshot: a concurrent same-ref insert into the issues plane landing between
  the two reads could let the merged fallback return the wisp, re-creating the
  wrong-row write that ordering exists to prevent. The consumer is the pull
  dedup loop, so it runs per imported issue. Both plane reads now share a
  single RunTxRead, which is also the shape the direct backend uses (both
  planes inside one withReadTx). Plane ordering is unchanged.

- internal/tracker/uow_store_engine_test.go: the engine fake counts the units
  of work it hands out, and
  TestUOWStoreGetIssueByExternalRefUsesOneReadTransaction pins that count at 1
  across the two paths that actually perform both reads -- the wisp fallback
  and the miss. An issues-plane hit short-circuits after one read and would
  pass either way, so it is not a case. Verified to fail against the
  two-transaction version with "opened 2, want 1".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 08:25:49 -07:00
Denis Redozubov
5f56ab2a53
fix(deps): enforce external capability blockers across ready/claim/close (#4769) (#4753)
External (cross-project) dependency blockers are now enforced everywhere a bead's readiness or closability is decided: `bd ready` / `--claim`, `bd blocked`, `bd close` / done-status `bd update` (direct and proxied), `bd dep tree`, and served HTTP ready/claim. The policy is a storage-boundary decorator (`internal/storage/externaldeps`) applied on both the DoltStorage chain and the UOW provider, with full capability-forwarding parity, `Unwrap` at every level, and the backend's advisory defer-wake preserved on the claim path.

Review trail: maphew rounds 1–4 (Jul–Aug), bee rounds 1–4 (Sep 2): D1 recording-parity exemption, D2 ready-filter goldens, D3 UOW-level Unwrap for the proxied importer, D4 claim-path defer wake. Verified at head 3468d62f on a clean merge with current main; CI fully green at that head.

Fixes #4769.
2026-09-02 16:56:23 -07:00
Julian Knutsen
d530cddfa6
ci: add advisory cyclomatic complexity reporting (#6081)
* ci: add advisory cyclomatic complexity reporting (bd-7g21)

Add a pinned gocyclo report, checked-in baseline, stable-key PR diff, and opt-in guard for shipped Go code. Run the report on every PR as a non-required artifact while maintainers measure signal quality; preserve existing complexity as grandfathered debt.

Agent-Signature: unknown-model/unknown-reasoning

* test(protocol): refresh dependency removal corpus
2026-08-30 19:05:17 -07:00
Julian Knutsen
8aaefe2d4f
feat(httpapi): serve bd list's order with sort on listIssues (#5666)
* feat(httpapi): serve `bd list`'s order with `sort` on listIssues

The issue listing served one order, `(created_at DESC, id ASC)`, because the
cursor is a keyset position in it. The spec stated the constraint in its own
words: "the sort order is welded to the cursor contract, and a new order needs
new surface." That welding put the cost on every client wanting `bd list`'s
ordering — the only way to get it was to page the whole result set and sort
locally, which is the one cost on this surface that grows with the project.

Measured against a real `bd serve` over 1400 rows at the 200-row page an HTTP
client actually uses: walk-and-sort is 7 requests plus a client-side
comparator, `?sort=priority` is 1. Both are checked against `bd list --json`
row for row, because the claim that matters is not "fewer requests" but "the
same order".

`sort` takes `created` and `priority` and nothing else. Each value is a keyset
contract — a position shape, a strictly-after predicate, an index — not a
display preference, and both keys are provably total (priority and created_at
are non-null, id is unique) so a page boundary inside a run of equal keys
resolves on id. The seven other orders `bd list --sort` takes have no such key:
`id` is natural-numeric and no database expresses it, `updated` moves on every
write, `closed` is nullable, and status/title/type/assignee are mutable and
unindexed. `priority` here is (priority ASC, created_at DESC, id ASC), which is
both `bd list`'s flagless order and what `--sort priority` produces — a stable
priority sort over the created order is the same sequence — so one served order
retires the walk for both.

Absent `sort` still means `created`, permanently: it is the compatibility
contract for every client written before the parameter existed.

THE CURSOR NOW CARRIES ITS ORDER. This is the half that makes the parameter
safe rather than merely useful, and the obvious implementation gets it wrong.
The token was base64 of `{t,i}` with no binding, so a created-order cursor
replayed under `sort=priority` decoded perfectly and was read as a position in
a different total order — a page that skips and duplicates rows, served with a
200, undetectable by the client. Tokens are now `v2` with an explicit order
member, and decode takes the wanted order as a parameter so a mismatch is
unrepresentable at the seam rather than merely detectable past it. Outstanding
`v1` tokens stay readable as the created-order positions they are, so no
traversal in flight has to restart. This does not contradict "the token carries
no filters": filters select the set, the order decides what the position means.

Storage grows one field. IssueFilter.AfterPriority extends the existing
position; the predicate nests the created one rather than restating it, because
the created/id comparison is only reachable at the cursor's own priority.
Flattening it the way the created predicate's leading-bound trick invites looks
equivalent and is not — it re-delivers rows at the cursor's priority created
after it, which the new conformance walk catches against real Dolt.

NO INDEX MIGRATION, by measurement rather than by omission. A composite
(priority, created_at, id) index was tried on Dolt 2.2.0 against the production
predicate, in both the plain ascending and the mixed-direction spelling Dolt's
DDL accepts: the plan is byte-identical in all three cases, keeping the
single-column priority seek and the TopN. It would have bought write
amplification on every insert and every priority update for no measured change.
The sargability guard pins what is achieved — an IndexedTableAccess range on
idx_issues_priority — and records the measurement to redo if Dolt learns to
stream the composite.

Agent-Signature: claude-opus-5
Co-authored-by: CI Bot <ci@beads.test>

* fix(changelog): drop leftover diff3 conflict marker in sort entry

The rebased adopt-PR head left a stray '|||||||' merge-base marker on the first line of the new `sort` CHANGELOG entry (CHANGELOG.md:381), which `git diff --check` flagged as a leftover conflict marker. Remove only the marker line; the resolved entry content is unchanged. Maintainer fixup for PR #5666 adoption review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: CI Bot <ci@beads.test>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-27 09:21:12 -07:00
Julian Knutsen
9a1f086a41
docs: pin the Dolt version instead of steering users to releases/latest (ga-m0dlk) (#5899)
The install instructions told users to run the upstream install.sh from
releases/latest, which today resolves to Dolt 2.3.1. Dolt 2.3.0 regressed
CALL DOLT_RESET('--hard'): a few percent of freshly created databases come
up with that procedure answering "Error 1105 (HY000): context canceled" for
the life of the server process. bd flatten and the Dolt-history compaction
in bd admin compact both finish by hard-resetting main onto a temporary
branch, and the merge-settle path behind bd dolt pull falls back to a hard
reset when it abandons a merge — so the documented happy path installed a
Dolt that breaks those operations on a database that otherwise looks fine.

Point the docs at a pinned version instead, with the measurements and the
raise criterion, and tell anyone already on 2.3.x how to check a database
and what to do about it. The pin and its rationale live in
scripts/ci/install-dolt.sh; the docs name the version once, on the Dolt
page, and everything else links there.

Also note that releases/latest resolves to the most recently created
release rather than the highest version, so it can move backwards — v1.88.2
was created after v2.2.4 and v2.3.0 — which makes "install latest" unsafe
independently of this defect.

test/docsync/doltpin_test.go keeps the documented pin, the copy-paste
install snippet and testutil.DoltDockerImage from drifting apart, which is
how the docs ended up on a broken release in the first place.

This documents the pin that gastownhall/beads#5894 applies to CI; land
after it so the reference to scripts/ci/install-dolt.sh resolves.
2026-08-20 22:41:24 -07:00
swedeinasia-flow
857723a17a
fix(doctor): recommend 'bd prune' instead of nonexistent 'bd cleanup' (#5829)
* fix(doctor): recommend 'bd prune' instead of nonexistent 'bd cleanup'

The Large Database check, its agent enrichment, the stale-closed-issues
enrichment, the Dolt perf recommendations, and the doctor --fix advisory
all told users to run 'bd cleanup', which is not a root-level command in
this build (cleanup lives under 'bd admin cleanup'). The operator-facing
lifecycle command is 'bd prune', whose --older-than flag takes durations
like 90d and which previews by default (--force to delete) — a better fit
for advice the doctor explicitly refuses to auto-fix because it deletes
data.

Extract largeDatabaseWarning so the remedy string is pinned by a test,
and pin the two enrichments' command lists likewise.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(messaging): replace stale ephemeral cleanup commands

Update the remaining live messaging examples to use the current wisp-scoped bd purge command and duration syntax.

Agent-Signature: Codex-unknown-model-unknown-reasoning on behalf of Klas Hesselman

---------

Co-authored-by: Klas Hesselman <211416696+swedeinasia-flow@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 05:30:47 -07:00
Jim Wordelman
30f30ca95b
ci: PR-CI lane runs the full internal/storage/dolt suite against a live Dolt server (#5836)
* test(ci): red — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)

* feat(ci): green — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)

Add test-server-storage-full as a sibling job to test-server-storage in
pr-risk.yml. build-embedded already compiles every top-level test in
internal/storage/dolt into one binary (embedded-test-binaries); the
existing job only ever runs -test.run '^TestConformance$' against it.
The new job reuses that same artifact and live-server setup with no new
build step, and runs everything else via -test.skip '^TestConformance$'
so previously-stranded tests (TestCreateGuard_*, federation/concurrent
suites, and diff-owned tests such as TestBenchDBPurgeDoesNotLeak on PR
#5792) get a real PASS/FAIL instead of a silent SKIP. Sets
BEADS_TEST_ENV_RUN_DOLT=1 so federation_test.go's own live-server check
fails loudly rather than skipping if this job's setup ever regresses.
Wired into ci-gate: added to needs, CI_GATE_REQUIRED, and the
tier-gated skipped_ok fallback (mirrors test-server-storage's existing
treatment in each).

Also updates two pre-existing meta-tests over pr-risk.yml that pinned
exact snapshots of the file this job necessarily changes:
scripts/ci_capability_selector_test.go's ci-gate authority snapshot,
and scripts/pull_dolt_image_test.go's retry-helper call count
(pr-risk.yml: 2 -> 3, the new job's own Dolt-image-pull step).

* chore: release gate PASS for be-pp7e (PR-CI lane for TestBenchDBPurgeDoesNotLeak)

Deploy gate record for be-pp7e, gating be-aiy5's PR-CI lane addition
(runs TestBenchDBPurgeDoesNotLeak against a live Dolt server instead of
letting checkDolt SKIP it). Review: be-w90n, verdict pass. All 7 gate
criteria PASS; two pre-existing/unrelated failures encountered during
evaluation (be-jy56, be-vf95) were attributed and excluded from the
verdict.

* test(ci): red — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)

TestPRRiskGateReachesFullServerDoltStorageSuite now asserts
pr-risk.yml's test-server-storage-full job is a fail-fast:false matrix
over shard 1..16, with the Test step invoking
server-storage-test-shard.sh, instead of pinning the old single-job
unsharded command. Fails against the current workflow, which still
runs all 1126 tests unsharded in one 15m job (be-extn).

* feat(ci): green — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)

Adds .github/scripts/server-storage-test-shard.sh (mirrors
embedded-storage-test-shard.sh: discovers internal/storage/dolt/*_test.go,
excludes TestConformance, manifest-plus-hash-fallback shard assignment) and
its shard manifest .github/scripts/server-storage-test-shards.txt, pinning
TestCloudAuthCLIRouting (~9.5m) alone on shard 1 and spreading the other
known 10-20s-heavy tests across the rest.

pr-risk.yml's test-server-storage-full job becomes a fail-fast:false matrix
over shard 1..16, Test step now runs server-storage-test-shard.sh instead of
the old unsharded single-job command that died at 256/1126 tests after
15m28s with zero failures (job 95616282508).

Documents the new lane in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md alongside the
existing embedded matrix description.

TestPRRiskGateReachesFullServerDoltStorageSuite now passes. Offline shard
partition verified disjoint and complete across all 1125 discovered tests
(BEADS_TEST_SHARD_LIST_ONLY=1 for i=1..16). (be-extn)

* test(feat): red — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)

* feat: green — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)

* chore: release gate PASS for be-23f3

Extends deploy/be-pp7e-gate (PR gastownhall/beads#5836) with be-gepv's
fix rather than cutting a fresh isolated branch: be-gepv's own bead
text and be-extn's precedent both establish this fix belongs on the
already-open PR. All 7 gate criteria PASS; two repo-wide lint/policy
failures attributed to already-tracked pre-existing issues (be-vf95,
be-jy56), confirmed untouched by this PR lineage.

be-gepv, be-6yo1, be-23f3

* fix(dolt): federation test isolation + panic (be-3c78s)

TestFederationDatabaseIsolation and TestFederationVersionControlAPIs are
federation-prototype tests that predate the suite's shared-server,
branch-per-test isolation architecture (setupTestStore/StartTestBranch)
and were never updated to account for it.

- setupFederationStore hardcoded Database: "beads" for both towns. New()
  always connects in server mode against one shared Dolt server for the
  whole test binary (TestMain), so Path alone doesn't isolate two stores
  — Database does. Both towns silently shared one database, so Alpha
  could read Beta's issues and vice versa. Give each town its own
  database (beads_test_federation_<prefix>, matching the beads_test AD-01
  cleanup-firewall prefix).

- TestFederationDatabaseIsolation's not-found assertions assumed GetIssue
  returns (nil, nil) on a miss. It never does — issueops.GetIssueInTx
  always returns a wrapped storage.ErrNotFound. Switched to
  errors.Is(err, storage.ErrNotFound).

- TestFederationVersionControlAPIs hardcoded Checkout(ctx, "main") to
  return to its "starting" branch, but setupTestStore already checks out
  a private branch (test-<name>-<hex>) — "main" is a branch shared by
  every parallel test in the binary and never has vc-001. That mismatch
  is what produced the nil-pointer panic (GetIssue's error was discarded
  via `_`, and the resulting nil issue was dereferenced). Capture the
  real starting branch via CurrentBranch() right after setupTestStore()
  and check both previously-discarded GetIssue errors before
  dereferencing.

Neither bug is a production security issue: New(), Checkout(), and
GetIssue() all behave exactly as specified. Both bugs are confined to
this prototype test file's own setup/assertions, which hadn't caught up
to the isolation model the rest of the suite already uses correctly.

Verified via `go build`/`go vet`/`go test -c` (all clean) and running the
compiled binary, which now reaches and cleanly SKIPs both tests (no
Docker locally) instead of the previous isolation failure / panic
surfacing at compile-adjacent runtime. Full GREEN confirmation is via
CI, which has a working Dolt/Docker environment.

Refs be-3c78s

* chore: release gate PASS for be-42tlj
2026-08-19 20:39:38 -07:00
Julian Knutsen
4e7cc88180
feat(journalops): the mutation journal becomes the third facade namespace (#5552)
* feat(journalops): the mutation journal becomes the third facade namespace

The durable mutation journal's read side is now a role. `journalops` is a
leaf package beside `issueops` and `memoryops` holding Row, Page,
TruncatedError/TruncatedCode and `Journal`, whose doc comment is the
specification: since-exclusive, seq-ascending, limit 0 uncapped with the
wire's cap named as the handler's, a head that comes with its rows and never
decreases, a caught-up checkpoint, and a typed truncation naming a resumable
window.

NOTHING CHANGES BEHAVIOR. The four names in `internal/storage` become
ALIASES of the leaf — `type EventsJournalCursor = journalops.Journal` and its
three siblings — so every implementation, every errors.As site and every
caller compiles unchanged. The alias direction is what avoids the import
cycle: the leaf imports context and fmt and cannot name internal/storage, so
the canon goes down and the alias goes up. Outside storage.go and the
conformance tier, the only non-test edits in this change are COMMENTS.

The role brings the apparatus a role owes:

- backend/conformance/journal_contract.go, six cases over a fixture carrying
  the role, the activation switch, a prune, and a mutation kit that is
  exhaustive against the engine's closed op vocabulary.
- Wirings on all three legs (dolt, embeddeddolt, uow), so #5499's
  entrypoint-scoped lock passes with every leg running every case.
- journalops added to the conformance census's facadePackages. That is the
  #5459 demand side, and the role that needed it: `Journal` is handed out by
  NO accessor — a front door type-asserts for it — so only a source parse can
  see it. issueops.Importer is the precedent.

EventsJournalAccessor and EventsJournalConfigurer stay in internal/storage.
Retention and activation are operator surface, and a server documented to
publish the journal and never retain it must not hold a delete.

Net deletion on the bespoke side: TestEventsJournalPageReportsTheHead is
subsumed by the contract and is gone. TestEventsJournalRestartAndRetentionBoundary
stays, because it observes ReadEventsInTx — the accessor's second read body,
deliberately not on the role — and a mutation of one is invisible to the other.

Mutation-measured, per case, on the embedded leg: an inclusive checkpoint, a
head derived from the page, an untyped truncation, a head taken from MAX(seq)
of surviving rows, and a dropped engine-only op each redden exactly the arm
named for them, and a fixture whose activation hook is a no-op reddens all
six. Reading the head BEFORE the rows leaves all six green — that promise is
declared unpinnable in the contract header rather than faked.

Refs ga-fvya1, ga-2yaqp.4.

* docs(journalops): fix three comment-precision findings from review

All comments, plus one test-only compile-time pin block.

1. journal_facade_test.go told a misassembled causal story: it used the
   CURSOR de-aliasing as its example and then attributed the breakage to
   errors.As. Those are two different failures. Measured, both:
   de-aliasing the interface builds the WHOLE TREE clean and breaks nothing
   behavioral — what it costs is the claim that one role has one name — while
   the error's dangerous shape is not the bare defined type (which loses
   Error() and fails to build) but the COPY-PASTE TWIN, which compiles in
   silence and stops crossing errors.As for everything holding the leaf's
   spelling. Rewritten to say that straight.

2. The seq commit-order unpinnable declaration cited "the dolt package's own
   journal tests" for the server-mode half. Those are single-threaded. The
   actual pin is internal/storage/domain/db/journal_seq_test.go —
   TestEventsJournal_CommitOrderedGaplessSeq drives two OVERLAPPING
   transactions on independent connections, and
   TestEventsJournal_ConcurrentWritersGaplessNoDup a racing batch. Cited
   correctly, and the single-threaded ones named as NOT the pin.

3. EveryMutationKindLandsARow's runs-last ordering is load-bearing but held
   by convention: a reorder compiles, passes, and silently gives up the only
   coverage that a journal keeps recording after a full prune. It was stated
   in the dispatch table and the three leg wirings; it is now stated at the
   case itself, where somebody editing a leg file is actually looking.

And the compile-time half of the alias pin, which fires before any test runs:
a pointer conversion per name. The pointer form is not decoration — for the
three structs a plain assignment would do, but for the INTERFACE it would be
blind, since interface-to-interface assignment goes by method set and a
de-aliased cursor still accepts a journalops.Journal. Watched fail: the
de-aliased interface, which nothing else in the tree catches.

---------

Co-authored-by: Review Sim <ci@beads.test>
2026-08-10 10:58:05 -07:00
Julian Knutsen
ff75b45c91
docs: the ext-api-store documentation tranche — post-#5516 auth sweep, wave doc coherence, role checklist refresh (#5545)
Three beads (ga-uwix0.1, ga-2yaqp.5, ga-2yaqp.6) plus a fable review round.
Docs and spec prose only: all 19 touched .go files are gofmt-normalized,
comment-stripped byte-identical to main. The one exception is deliberate and
reviewed — scripts/check_doc_freshness_test.go gains a data row, because the
delegated gate ruling required it.

ga-uwix0.1 — #5516 gave bd serve optional bearer auth and made it MANDATORY
beyond loopback; fifteen prose sites still said the opposite. The three the
council named are reworded to the stronger true form (the actor is not the
authenticated principal even where a bearer is required; a shared token admits
a client to the WHOLE surface and grants no narrower right). The sweep for a
fourth found eight more in five spellings the audit's grep could not reach,
including claim.go's "any client can claim as any name", release.go's
no-AUTHORIZATION variant, and six memory/settings-plane sites where the true
statement is sharper: a bearer decides WHO may call, never what a caller who
is let in may read. The review round found four more in the CHANGELOG's
UNRELEASED section, which round one had misclassified as released history —
they would have shipped false in the same release as the auth entry.

The serve runbook's "Binding beyond loopback" section was actionably wrong: it
said the flag "prints a warning and changes nothing else", and its refusals
table quoted a string not in the binary. Rewritten against a build, with every
flag, error string, event name and reason value produced by running the
command. It gains the operational trap nobody had written down: /healthz is
auth-exempt and GET /v0/beads/ready is not, so an unauthenticated readiness
probe earns a 401 forever while liveness stays green. The runbook is now IN
scripts/check-doc-freshness.sh, pinned to the four files it is read against.

ga-2yaqp.5 — nine prose defects, every count re-derived rather than trusted;
three of the audit's own numbers had moved again. 38 route rows and 36
capability-bearing ones; thirteen pinned schemas, not twelve or seven; six
missing entries in the no-query-parameter list, not five; twenty-eight role
accessors, not twenty-seven; nineteen patch members, not thirteen. Two prose
"code counts" were counting paragraphs. CodeAlreadyClaimed claimed the holder
is always attached when only one of four producers always attaches it and
releaseIssue never does. MetadataCAS's leaf contradicted its own shipped wire.
The path/operationId inversion is now recorded where a spec author looks.

ga-2yaqp.6 — the role checklist refreshed against what this wave's additions
actually cost, traced to real commits. Step 10 sent people to a name list
#5460 deleted. The "four more edits" claim was stated twice and undercounted
both times: the two freshest wire slices share a nine-file core plus
roles_test.go. The step-with-no-number gained the fact that explains its
instances — write-role decorators wrap, read-role decorators recurse, so the
call lands on the stub, and both real instances are reads. The wave got its
first CHANGELOG entry: six new public operations, not three, and zero new role
accessors in the window.

Review verdict: FIX-REQUIRED then all items addressed. The reviewer
live-falsified the strong auth form and failed to break it, and confirmed
every recount re-derives exact.
2026-08-10 08:05:07 -07:00
ecuthiell
27c1fe7085
docs: clarify CI topology authority (saa-cgh.61) (#5501)
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 05:30:43 -07:00
Julian Knutsen
cff62bd631
Make the pre-commit lint gate trustworthy, and stop one main-side lint word from reddening every PR (#5538)
* fix(githooks): build golangci-lint with the repo's Go, not golangci-lint's

The pre-commit hook ran `go run golangci-lint@v2.10.1`. For a pkg@version
build the go command honours the TARGET module's toolchain directive, so
GOTOOLCHAIN=auto downgraded to go1.25.12 — and golangci-lint built by
go1.25 refuses this repo's `go 1.26.5` at config load:

    go: github.com/golangci/golangci-lint/v2@v2.10.1 requires go >= 1.25.0; switching to go1.25.12
    Error: can't load config: the Go language version (go1.25) used to build golangci-lint is lower than the targeted Go version (1.26.5)

Every Go commit failed there, so the hook taught six-plus contributors to
reach for --no-verify — the one habit a lint gate must never train.

CI never hit it because actions/setup-go installs the go.mod version and
exports GOTOOLCHAIN=local, so its `go install ...@v2.10.1` compiles the
linter with go1.26.5. Pinning GOTOOLCHAIN to the go directive reproduces
that resolution on any host, including one whose base toolchain is older
than the repo's target (where a bare GOTOOLCHAIN=local would pin the
linter to that older toolchain instead).

Verified on a real staged Go change: red before with the message above,
`0 issues.` after, and still exit 1 on a planted errcheck violation.

Bead: ga-2ltro.13

* ci: lint PRs for what they introduce, sweep the whole tree on main

Both PR lint gates linted the whole tree: the `lint` job left
only-new-issues at its false default, and pr-lint-wrapper ran
`make ci-pr-lint` over ./... . So a single lint word landing on main —
twice this wave, once a prose misspell — turned EVERY open PR red for a
change it did not make, and each author had to either fix someone else's
line or wait.

The tradeoff, recorded beside the job: a pre-existing issue in untouched
code no longer blocks a PR, and because "new" is measured off the diff, a
violation carried in by a MOVE or a rename does. What it buys is that a
main-side word now reds main's own run, where it belongs. main.yml
already runs both jobs unscoped on every push, so whole-tree coverage —
including the GOOS=windows cross-lint — is unchanged; only its blast
radius moves.

The wrapper takes its scope from BD_LINT_NEW_FROM_MERGE_BASE, set only
by pr.yml, so `make ci-pr-lint` run by hand still sweeps the tree. That
makes the local wrapper STRICTER than the PR gate rather than equal to
it, which is what LINTING.md and CI_CLEANUP_PLAN.md said before and no
longer would: both now state the two-lane contract and say the local
run's extra strictness is deliberate. Their freshness markers move with
the workflow and wrapper files they are pinned to.

Bead: ga-uwix0.2

---------

Co-authored-by: Julian Knutsen <ci@beads.test>
2026-08-10 04:55:55 -07:00
Julian Knutsen
06b0dffb22
Make the next serve role a compile error, and check the one role with a positional result (#5539)
* refactor(serve): make the next role a compile error in the serve stubs

serveIssueRoles took a whole storage.DoltStorage, so the test stubs that
stand in for a store satisfied it the only affordable way: embed the
interface and leave it nil. Every accessor a stub had not declared then
arrived as a promoted method on a nil interface — a segfault inside role
extraction, not a compile error.

It has landed twice in one PR cycle, both on GraphCounter. First in
serve_source_test.go, then in serve_store_identity_test.go, where no local
-run pattern names the test and only a full cmd/bd shard found it.

So the function now asks for serveRoleSource: the accessor subset it
actually reaches, which a whole store still satisfies. Both stubs declare
that subset and assert it, and the store they must still BE — the
decorator chain takes one, and backends.Backend.Open returns one — comes
from an embed one level deeper that their own accessors shadow. Adding a
role to the loop is now:

    cannot use (*serveRolesStore)(nil) ... does not implement
    serveRoleSource (missing method GraphCounter)

ADDING_AN_ISSUEOPS_ROLE.md's "step with no number" listed these two stubs
among the surfaces to grep for by hand; it now says this pair is taken
care of and the rest still are not.

Swept the embed pattern across cmd/bd for a third instance reaching this
path: none.

Bead: ga-c3g45

* fix(httpapi): check the batch closer, the one role with a positional result

internal/httpapi has checked wrappers for Reader, BatchCreator, Lifecycle,
Claimer and Releaser, but the batch closer went out unwrapped — on the
reasoning that CloseBatchResult is a value and its issue pointer is
forwarded rather than dereferenced. True, and it is why there is no panic
here to prevent. What it left unguarded is the thing this role owns that
none of the others do: an array the client reads POSITIONALLY.

Two shapes reached the wire as answers rather than faults, and they fold
at different scopes:

  - An outcome count that does not match the request cannot be walked
    against the caller's items at all. Projected anyway it reports one
    item's answer under another item's id for the rest of the array,
    silently, inside a 200. Whole-batch 500, never a partial projection —
    checkedBatchCreator's refusal exactly.

  - An outcome carrying neither a row nor a refusal has no honest wire
    shape: `code`'s absence promises `issue`, so it projects as a success
    with no row, which is the shape closeOutcome's `default` branch
    already refuses to produce. Folded into THAT item, because this batch
    is not all-or-nothing — the survivors have committed, and failing the
    request would hide durable closes.

The fold corrects a copy: the outcomes slice is the role's, and one that
answers from a prepared result would otherwise carry this server's
correction into everything it hands back afterwards.

No wire shape changes and no spec changes; make api-check is a no-op, and
the error classification order in failBatchClose is untouched.

Bead: ga-39tl5

---------

Co-authored-by: Julian Knutsen <ci@beads.test>
2026-08-10 04:34:15 -07:00
Julian Knutsen
3eab445b0a
Add issueops.GraphCounter: how many edges, in one named direction (ga-itny5) (#5485)
* Add issueops.GraphCounter: how many edges, in one named direction (ga-itny5)

The graph counts are the last read on the storage surface with no role behind
them. `storage.DoltStorage` publishes CountDependents, CountDependencies and
CountDependentsByStatus as three methods that differ only in a direction and a
narrowing, and every one of them is a raw method a front door can call with no
decorator, no probe and no plane story. This is their facade.

ONE METHOD, AND THE DIRECTION IS IN THE REQUEST. The raw seam spells the
direction in the METHOD NAME, which is the one thing a request field can get
wrong, so `EdgeCountRequest.Direction` is REQUIRED and an empty one is
ErrValidation before anything is read — including for a request that names no
ids at all, which is otherwise the empty answer EdgeReader gives. There is no
`both`: the two numbers are over different edge sets, summing them
double-counts an edge between two requested anchors, and a paired answer would
make Status — which narrows by a row only the inbound direction has — govern
half an answer silently. internal/workapi's detail assembly already makes two
calls for the pair today, so nothing pays a round trip it was not paying.

STATUS IS INBOUND-ONLY AND SAYS SO RATHER THAN BEING IGNORED. Narrowing by
status means joining the far end of an edge to the row holding its status, and
an OUTBOUND edge's far end may be a row this database does not hold — an
"external:" reference or a foreign-repository id, both of which
DependencyEditor accepts. A status filter there would silently drop every
dangling edge, which is the class of edge EdgeReader exists to return verbatim.
The raw seam has the same asymmetry: CountDependentsByStatus exists and
CountDependenciesByStatus does not.

A MISSING ANCHOR IS A FLAG, NOT ErrNotFound. 0 is the common answer — most
issues have no edges in at least one direction — so an id typo would never
surface, and a batch that failed on one absent id would discard the answers for
the ids that were found. That is EdgeReader's rule, and it is what forces the
existence probe and the tally into ONE transaction.

WHAT IS EXCLUDED, since the never-append rule makes each one a decision. The
COMMENT count is genuinely consumed — `bd show` prints it beside the two edge
counts — and is still out: a comment has no direction, no type and no far end,
so every field of the request below IDs would be meaningless to it. CountEvents
has no caller in the tree at all and follows the journal. GetDependencyCounts
is expressible as two calls with Types set to the blocking type, and naming it
as a method would have added a map-shaped answer no wire operation can carry.

THREE LEGS, ONE BODY. All three reach issueops.ExecuteEdgeCount — the two
stores wrap it in their own read transaction, the unit of work reaches it
through the domain repository, whose runner publishes exactly the DBTX method
set — so the contract header says one reading plus two engine checks and three
wrapper checks, not "three backends agree". Validation moved INSIDE that body
for the same reason: with one body there is no second implementation for a
per-leg check to belong to.

FACADE ONLY. No HTTP operation: the wire half follows in the graph-counts wire
slice per the program's council gating. No `bd` command either, and that is the
larger of the two decisions — the numbers are printed today through
internal/workapi's detail seam, which an HTTP handler shares, so moving it is a
change with its own parity argument. Both absences are written into the leaf
doc beside the promises, as VersionReconciler writes down its absent HTTP half
and BatchApplier its absent CLI.

Contract: 14 cases, wired on all three legs. Mutation-checked — a constant
count reddens 7 including both per-direction cases, ignoring the direction
reddens the 4 inbound ones, and dropping the ephemeral plane reddens the 2
cross-plane ones. The pure halves (the request vocabulary, the missing-anchor
rule, the SQL shape) are pinned without a database beside the body.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Review: reattach the Importer note, cite the collision, size the id list

Three corrections, none behavioral.

The four-line note about Importer's uow-only accessor was left stranded above
the GraphCounter row by this slice's insertion, where it asserted three things
that are false of it: GraphCounter IS a storage.DoltStorage accessor, a
store-built backend CAN fill its field, and importer_contract.go's header says
nothing about it. Moved back onto the row it describes.

"Nothing in this contract's reach can produce one" reads too easily as "it
cannot happen". A cross-table id collision is real and constructed:
TestDependentRecordsCrossTableCollision seeds one logical edge into both
dependency tables and names the two production paths that make them — a wisp
promoted to durable, and two Dolt clones merged. The leaf now cites it and says
what this role promises against it (the sum) versus what
CountDependentRecords' question is (the distinct total).

And EdgeCountRequest.IDs says out loud that it carries no size cap, which
EdgeReadRequest.IDs also does not: the bound belongs to the wire operation,
where the request arrives from somewhere else and the cost of an unbounded one
is a stranger's. ApplyBatchRequest.Items makes the same split explicitly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Review Sim <ci@beads.test>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:29:32 -07:00
Julian Knutsen
e9e2c49431
Add issueops.BatchApplier: one plan, one transaction (#5480)
* Add issueops.BatchApplier: one plan, one transaction

A heterogeneous, ORDERED list of creates, updates, closes and edges applied
as one durable act. Each of BatchCreator, BatchCloser and DependencyEditor is
one verb repeated, so a caller that needs two of them needs two transactions
with a window in between; this role's unit is a PLAN.

The role, the two bodies, both decorators and the accessors on every
implementer. The wire half and the conformance contract land beside this.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Close the empty-commit-message hole a no-op batch could open

ApplyBatchCommitMessage now reads the write as well as the result. The counts
are ItemResult.Changed, which is what a CALLER observes; a request can still
write a durable row with every count at zero — a same-type edge re-add
rewrites that edge row's metadata, a re-close can settle blocked state. An
empty message for one of those is an unlabelled Dolt commit on the store legs
and a rollback of a real write on the unit-of-work leg.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Give BatchApplier the contract its two bodies justify

Thirty-nine cases over three legs and TWO independent bodies — the store legs
share issueops.ApplyBatchInTx, the unit-of-work leg cannot reach it, and the
contract header says so rather than claiming three backends agree.

The end-gate case is the one that could not have been written before: every
edge in it is legal when it is written, and only a re-validation against the
closure the whole request produced refuses the pair.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Publish batchApply on v0, and delete batchCreate's false promise

The spec first, the types from it, the handler against the types. The item
union is a TAGGED single-shape object rather than a oneOf, because the
document's x-go-type doctrine forbids composition keywords; the create item
publishes sender, metadata, status, ephemeral and no_history, none of which
batchCreateIssues has — which is why no measured caller could use that one.

The wire result stays lean: ids, changed and a revision. The hydrated
snapshot on the Go contract is what the completion hooks consume, and hooks
never fire on this surface.

OQ2: batchCreateIssues' description claimed a dependency target could name an
earlier item of the same batch. It never could — the server mints the ids and
a wire item has no name — so the paragraph is deleted rather than qualified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Partition the two new dependency probes on the recorder's surface

The gate halves BatchApplier re-runs are repository PROBES — an ancestry walk
and a reachability walk — so recordingDepUC declares neither and the parity
test has to say so out loud rather than let embedding answer for it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Teach the proxied topology's capability golden about batchApply

The one expectation stored as data outside the package whose gates would
otherwise have caught it — which is what its own header warns about.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Restore the CAS hook case, mint 409 already_exists, and stop over-claiming

Three fixes from review.

The rebase that brought MetadataCAS in left `case *hookMetadataCAS:` with an
EMPTY BODY: a Go type switch does not fall through, so RoleFiresHooks answered
FALSE for it and #5473's Listen refusal was silently disarmed. The compiler,
vet, lint and CI all pass on that. The case is restored and — because it had no
precondition of its own, which is why nothing caught it — serve_source_test now
pins CAS the way it pins the other four, with its peel assertion.

An occupied explicit create id becomes 409 already_exists rather than 400. The
body is well-formed and stays well-formed; what refuses it is state the client
must READ, and recovery is to look at that state. A dead 409 retires for free
where a narrowed 400 breaks clients.

Two headers claimed all three legs share the store body — every clause false,
one citing the contract header as agreeing when it says the opposite — and the
plan file gave a placement reason that is MetadataCAS's, not this role's. The
end-gate claim in ADDING_AN_ISSUEOPS_ROLE.md is narrowed to the true thing: the
two LEAF checks are shared, the orchestration around them forks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Give the scheduling-edge set one definition, in types

Four packages walked {blocks, conditional-blocks, parent-child} under four
names — isSchedulingEdge, isSchedulingDep, isSchedulingDependency, and the
isApplySchedulingEdge this branch added as the fourth. Adding a fifth
scheduling type was four edits with nothing to catch a missed one.

It goes in internal/types rather than the issueops leaf the review named,
because internal/storage/issueops imports internal/storage/domain, so domain
cannot import back and two of the four copies would have survived there. Beside
the Dep* constants themselves, every walker reaches it: nine call sites, one
definition, one edit for a fifth type.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Review Sim <ci@beads.test>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 06:59:34 -07:00
Julian Knutsen
1a1431be3f
Add issueops.MetadataCAS: compare-and-set over one metadata key (#5473)
* Add issueops.MetadataCAS: compare-and-set over one metadata key

The one coordination primitive missing at every layer. Everything the
metadata plane holds is written through Lifecycle.Update, which has no way
to say "only if", so every caller that needs a guarded write composes a
transaction, a read and an update by hand — and each one has to get the
composition right. Gas City emulates it client-side with RunInTransaction +
GetIssue + UpdateIssue for exactly that reason, and the comment it left
behind names the gap: "the upstream storage layer exposes no
conditional-UPDATE ... WHERE primitive".

The role is a state transition stated as a pair. Expected is the key's value
before, Value is its value after, and nil on either side means the key is
ABSENT there: a conditional create is (nil -> value), a conditional delete
is (value -> nil), and a caller that can take a lease can return it to the
state its own acquire tests for. Equality is canonical, so a caller cannot
lose a swap to its own formatter; numbers compare as their source literal,
because decoding through float64 equates two int64s past 2^53 and that is
the silent false match a compare-and-set exists to prevent.

A MISMATCH IS AN ANSWER, not an error: Swapped false, Current carrying the
value that refused it, nil error. That is the one place this diverges from
PROPOSAL-cas-conditional-update.md, which specifies a typed error and a loud
non-zero exit for the assignee/status guards. Its argument is a CLI one — a
swallowed refusal in a shell script is the bug class it was written for —
and a non-zero exit is the front door's enforcement, which the HTTP handler
here still owes and pays. The role's designed caller is a retry loop, for
which a lost race is the ordinary path, and an error cannot hand back
Current without a bespoke type every caller must errors.As before it can
read the one field it needs.

All three legs share one body (internal/storage/issueops), so this is one
reading plus two wrapper checks rather than three backends agreeing. The
wrapper check earned its keep: the unit-of-work leg rolled an ephemeral swap
back, because the shared body reported "no durable tables changed" and that
leg's commit message is what commits the SQL transaction as well as what
versions it. MetadataCASWrite now carries the two facts separately.

The HTTP half lands with the role per the 2026-08-05 scope decision:
POST /v0/beads/issues/{id}:casMetadata, spec first. A lost race is a 200
carrying swapped:false, so the operation has no conflict code; an omitted
`expected`/`value` member means absent and a member holding null is a value,
which is why the handler reads raw members.

No bd command yet: the primary consumer is an embedder, and a CLI front door
is additive. No .golangci.yml entry, for the reason memoryops gives — the
body is an …InTx function no front door can reach.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Review: read Current from the row, and let a client read a present null

Two shape fixes, both of which would have been expensive to change after
anyone depended on them.

A *json.RawMessage wire member cannot READ a present null: encoding/json
nils the pointer before UnmarshalJSON runs, so a generated client decoding
{"swapped":false,"current":null} gets the same value it gets for an omitted
member — and on this operation those mean opposite things. A retry loop
against a null-valued key would read "absent", swap with expected omitted,
mismatch, and livelock on a stream of 200s. x-go-type-skip-optional-pointer
makes the member a bare json.RawMessage, which is an Unmarshaler and
receives the literal; the wire is unchanged in both directions, which is
exactly why nothing else catches the regression, so the decode is now
pinned. nullable: true beside it, for the validating gateway in front of
somebody's server.

The number rule was defended in three places as protecting int64s past 2^53
from a float64 round-trip. It protects nothing: the metadata column decodes
through float64 itself — measured, 9007199254740993 stores as ...992, 1.0 as
1, -0.0 as 0, 1e300 as three hundred and one digits. The false justification
is gone from all three sites and replaced with what the probe actually
showed, plus the guidance it implies: compose an Expected from a previous
Current, and prefer a string for a coordination token.

That guidance was not true yet, because Current was the REQUEST's bytes
rather than the row's — so "what you get is what a later read sees" was
false, and a loop feeding Current back compared its own spelling against the
substrate's forever. Current is now re-read inside the deciding transaction
on every path. One extra SELECT, one statable invariant.

Three contract cases, each watched to fail: Current is the column's bytes
and a Current-driven loop converges (mutant: answer from the plan);
the history entry names the request's actor, counted per actor because
created_at is second-granularity and an ORDER BY decided this case on a coin
toss when it was first written (mutant: stamp a constant); a wisp swap
writes no durable event row, which two layers had been holding by
coincidence with nothing asserting either.

Also: the serializer-omits-null footgun in the spec (an absent `value` is a
DELETE, and Jackson/Pydantic/omitempty all omit nulls by default),
canonicalizeForContract's own float64 blind spot named in the coverage
paragraph, and the leaf now says corrupt stored metadata is an untyped error
— a 500 — because the request was well formed and the issue exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Close the three capability holes CI found, and one my sweep could not

All three are the same shape: a surface that loses the new role silently,
and none of them is reachable from the package tests I ran locally.

internal/storage/uow's notifying wrapper is the important one. Its
recordingIssueUC inherited CompareAndSetMetadataKey from the embedded use
case, so a swap through a notifying unit of work recorded nothing and the
hook the DoltStorage chain fires for the same write was lost. Its
notifyingProvider did not answer MetadataCAS at all, so a caller's type
assertion simply stopped matching and the wrapper dropped the capability
along with its hooks. Both are now declared, and the recorder reads the
use case's own "a row write landed" rather than inferring it from the
request — which is strictly better than what the hook decorator can do, and
snapshots anyPlane because this role resolves a wisp id itself.

cmd/bd's serveIdentityStore is the second stub store to nil-panic on the
promoted accessor, after serveRolesStore in the same commit. And
serve_proxied_integration_test.go carries a SECOND hardcoded capabilities
golden, whose own comment says two literals that must match are one literal
— it was right, and it was still missed.

My local sweep ran the metadata-cas contract on all three legs but not the
whole internal/storage/uow package, which takes nine minutes; that is where
the parity guards live. Ran it whole this time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: name the step with no number — the surfaces that EMBED

A new accessor arrives on every embedder promoted rather than declared, so
the build stays green and the first symptom is a nil dereference in somebody
else's stub. Four such places caught this role after its own slice was
green: RoleFiresHooks, both halves of uow's notifying wrapper, and two
cmd/bd stub stores. Grep for the embed, not for the interface — and run
internal/storage/uow whole, because the parity guards live there and the
package takes nine minutes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Review Sim <ci@beads.test>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 04:15:34 -07:00
CI Bot
19e4af709a feat(httpapi): stream the events journal over SSE (bd-opisf)
GET /v0/beads/events:watch holds a text/event-stream response open and emits
each committed mutation as it lands, so a consumer learns about a write when it
happens instead of on its next interval. It is a SIBLING of the paged read, not
a mode of it: `follow=true` would have given one operationId two media types,
two lifetimes, two limit rules and two capacity stories, and a client generated
from the document could not have described either.

THE CURSOR IS STILL THE CONTRACT, which is what makes this cheap to publish.
The server keeps no per-consumer state and remembers no subscription; a stream
is the `since` reads a poller would have performed, performed on its behalf
against the same ReadEventsJournalPage through the same read-only cursor. Every
event carries `id: <seq>`, so the checkpoint a client holds while streaming is
the number it would have held while polling, and `Last-Event-ID` — the header a
browser's EventSource resends by itself — takes exactly the value `since` takes.
That header WINS over the parameter, because the client that sends it did not
choose to: an EventSource reconnects to the URL it was built with, so honoring
`since` there would replay every record since the consumer started, on every
reconnect, forever. A nonempty unparseable one is a 400 naming the header rather
than a silent fallback, which would start a stream somewhere the client did not
ask for and never tell it; an EMPTY one means "no id yet" and defers to `since`,
because a client that always sets the header sends it empty on its first
connect and refusing that would break the request the header exists to serve.
Delivery is at-least-once against the client's own checkpoint: exactly once and
in order within a stream, duplicates only across a reconnect. No engine or
storage change — this consumes the poll read path.

EVERY REFUSAL IS DECIDED BEFORE THE FIRST BYTE — validation, the disabled 409,
the flush check, the stream cap, and then the first read, which is what turns a
pruned-past cursor into the paged read's own 410 rather than a 200 that
immediately reports its own failure. Exactly one failure can arrive after the
status is spent: a prune racing an open stream, emitted as a NAMED `truncated`
event carrying the 410's own body in the same encoding on one line (asserted
against Write's output, not merely agreeing with it), behind a raised `retry`
and followed by a close. Naming only the exception is what lets a bare
`onmessage` consumer treat an unrecognized event as "stop, this is not a
record"; one that ignores it does not lose records silently, it stalls loudly on
connect-time 410s, slowly. The other unreportable failure — a stored payload
that will not encode — ends the stream too, and names its seq in the log,
because it will end every stream that reaches that row and nothing else would
say which one it is.

THREE LIMITS HAD TO MOVE, and the middle one is the trap.

  - requestDeadline (60s) does not apply to a streaming row. For every other
    operation it is the backstop against a request holding resources forever;
    here it would do nothing but sever the stream. Each READ carries a 15s
    deadline instead — a liveness budget rather than a database one, since it
    also bounds how long a stream can go without a heartbeat and how long it can
    ignore a shutdown.
  - http.Server.ReadTimeout (30s) had to be cleared per request, BEFORE the
    connect read. There is no WriteTimeout on this server — statusWriter rolls a
    deadline forward before every write, which bounds one stalled write rather
    than a stream, and needs nothing here — but ReadTimeout is a deadline on the
    WHOLE request, and net/http keeps a background read running while a handler
    writes: the read that notices a disconnect. Its expiry cancels the request
    context, so an idle stream would end itself at thirty seconds with no client
    involved, and a database wedged during the connect read would be booked as
    `client_closed`. Clearing it costs no disconnect detection — a client that
    leaves still produces a read error — and a peer that vanishes without a FIN
    is reaped by TCP keepalive, which Go's listener enables by default
    (measured: 15s idle, 15s interval, 9 probes; ~150s).
  - The database slot moves from the request to the read. Sixteen slots and
    forty-eight stream places means one slot per open stream would let a handful
    of idle consumers starve every other operation for hours with /healthz
    green; a stream now takes a slot for the moment of each poll and gives it
    back, and slot pressure delays a pass instead of ending a stream.

Streams are capped at 48, SIXTEEN BELOW the 64-connection limit, and the gap is
load-bearing arithmetic rather than caution: LimitListener returns a connection
slot only when the connection closes, which is after the handler returned and
after this counter came down, so a cap at maxConns could never be observed — the
connect that should earn the 503 would never be accepted, and neither would the
poll it is told to fall back to, nor a mutation, nor a fresh-connection
/healthz. The headroom makes the refusal deliverable; `events_watch_admitted`
carries the live gauge on every connect so accumulation is visible before the
cliff. Each stream jitters its own poll phase by ±10%, because consumers
reconnect together after a restart on one shared `retry` and a fixed interval
would march them into a single read burst per second forever. And a graceful
shutdown signals its streams (RegisterOnShutdown) rather than waiting them out;
the loop consults that signal and the client on EVERY pass, including the
draining ones, so a week-old backlog stays cancellable within one batch.

Tests: handler-level over live delivery (mutations land AFTER the connect),
id=seq, heartbeats, Last-Event-ID precedence including a stale header against a
live `since` and the empty-header polarity, the 400/409/410 connect legs, a
prune under an open stream, the cap and its 503, the cap's headroom against
maxConns, backlog drains cancelled by both a hangup and a shutdown, slot release
on every exit, a stream sharing a single-slot semaphore with a paged read, the
flusher refusal, the read-deadline clear and its ordering against the connect
read, jitter spread, and a clean drain with a stream open — all under -race.
Framing is pinned against a record whose payload carries literal newlines, a CR
and a `data:` line of its own. Two e2e legs through a real `bd serve`: mutate on
one connection while streaming on another under a 10s arrival bound, then
reconnect with Last-Event-ID and require the resume, plus the disabled refusal.
Mutated and each caught: the mid-stream truncated event, Last-Event-ID
precedence, the empty-header polarity, the cap decrement, the cap's headroom,
the drain-cancel fast path, the read-deadline clear and its ordering, the poll
jitter, the shutdown exit, and the deadline exemption.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Agent-Signature: claude-code-claude-opus-5[1m]-unknown-reasoning on behalf of julianknutsen
2026-08-09 07:55:23 +00:00
Steve Yegge
d5277be4e0
Merge pull request #5424 from gastownhall/docs/ado-reconcile-and-otel-spans
docs: document ado.reconcile_interval and inventory every storage span
2026-08-08 15:43:19 -07:00
Julian Knutsen
7fee5e9336
Merge pull request #5431 from gastownhall/httpapi-wave-polish
Wire-surface polish for the v0 write wave: include-leg depth, derive-proof prose, the missing member-list gates, changelog
2026-08-07 19:07:20 -07:00
Julian Knutsen
e29b5eadb4
Write down how to retire a test against a contract, and ship the tool for it (#5425)
Follow-up to the four-slice deduplication pass (#5398, #5399, #5405, #5406).
No behaviour change.

That pass retired ~700 lines net, and its reviews found about a dozen coverage
losses and six tests that could not fail — every one of them green under
go build, go vet, go test and golangci-lint. The reasoning that caught them
lived in review transcripts and a scratch directory. Written down nowhere, the
next pass rediscovers it by shipping the loss first.

The engdoc section states each rule as the check that would have caught the
failure earlier: a mutation verdict is only true of the body you mutated; a
wrapper is a composition the contract structurally cannot see; write a residue
from the decision surface rather than the mutants you happened to run, since
half of what one such residue missed were POSITIVE assertions; ask what the
fixture makes unobservable, because every fixture-defect case had correct
assertions; never let a role-answer assertion replace a raw-row one; cite by
symbol, not line.

scripts/mutation-equivalence.sh is the tool. It ran the entire pass from a
scratch directory, which means it would not have survived it.

A fact-check review then found more wrong with this change than with anything it
describes. The section cited `runTransaction`, which does not exist in the dolt
package — while teaching symbol citation. It claimed no contract case reads
is_blocked; there are 45 references across ten files, several added by the pass
it describes. Three of its counts did not reconcile with the merged record, and
one was curated to make a sentence read true. All corrected, with the counts now
qualified and the pull requests named as the record.

The same review demonstrated three false-REDUNDANT routes in the script — the
one wrong answer that costs coverage — plus a guard that refused the repo only
by accident, and would have hard-reset the caller's own checkout on a slightly
different setup. Build failures and timeouts are now reported UNUSABLE rather
than read as catches; checkout and reset failures are fatal rather than
measuring a different commit; off-file mutations are fatal; A == B is refused;
and the worktree must be a detached scratch worktree of the repo.

A seventh guard came from using it: it measures HEAD, not the working tree, so
uncommitted work is invisible and the run reads INCONCLUSIVE. It now warns.
2026-08-07 18:58:48 -07:00
CI Bot
5bd29359b3 docs(httpapi): stop the orientation prose enumerating a surface it no longer has
doc.go still opened with "the three reads ... and the one write, POST
/v0/beads/issues/{id}:claim" and stated that "the claim OPERATION is the
only mutation this surface has". bd-serve-v0.md still opened its surface
section with "Six operations" over a six-row table, sized the server
under "Claim throughput — the claim is the only mutation in v0", and
scoped the no-hooks and no-auto-commit contracts to the claim alone. The
route table serves twenty-seven operations, eleven of them writes. Both
pages name themselves sources of truth, and bd-serve-v0.md's own header
rule says nothing in it may contradict the spec, doc.go or the reader
role — so this was the rule violating itself.

The counts are not corrected, they are removed. routes.go is the router,
TestSpecRouteParity welds it to the spec by exact set equality in both
directions, and capabilities is derived from the same table: a fourth
copy of the list, spelled in prose, can only ever go stale — as this one
did, silently, across five merged PRs. Both pages now describe the
MECHANISM and point at it. What replaces the table in bd-serve-v0.md is
the surface's shape, which is what a design page is for and which does
not change when an operation is added: what answers from the process,
how reads reach roles, the two spellings a write comes in, and the
posture every write shares.

Statements that were true only of the claim are widened where the code
widened them (the JSON content type is enforced for every body-carrying
operation; hooks fire for no write; the idempotent-write commit skip is
the re-claim, the re-close and the reopen) and narrowed where they were
overstated (the reader property covers three named reads, not "all"
reads — the count, the query, stats, config, memories and the dependency
reads are on sibling roles). The design page's error table gains the
three codes the wave added, with the extension members each carries and
the absence that distinguishes the two dependency_cycle refusals.

Also sweeps the seven stale "one write on this surface" comments in
claim.go, claim_test.go, roles_test.go and update_test.go, and reads.go's
"the three read operations" header over a file that now holds five.

No wire change, no behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 01:34:33 +00:00
CI Bot
a82e688953 docs: document ado.reconcile_interval and inventory every storage span
Both files had their Last reviewed markers refreshed today without the content
gaps being closed, so the freshness signal is currently green on two docs that
are missing real material.

azure-devops.md — `ado.reconcile_interval` is undocumented. It is real,
user-tunable config (internal/ado/reconcile.go, default 10) setting how many
syncs pass between automatic reconciliation scans, and the doc already documents
the `--reconcile` flag that forces one. Adds a Reconciliation section, and notes
that `ado.syncs_since_reconcile` is beads' own counter rather than a setting.

otel-data-model.md — claims a complete schema but lists 7 of 111 storage spans,
so the issueops role surface is invisible to anyone instrumenting from it.
Rather than 100+ prose sections that would drift again, state the rule the code
follows — span name is "storage." + operation name, which is also the
db.operation attribute (InstrumentedStorage.op) — and inventory both families:
72 direct storage.<Method> spans and 39 storage.<Role>.<Method> spans.

Also records why the families do not double-count: the role decorators wrap the
UNINSTRUMENTED store (IssueLifecycle calls Unwrap() before wrapping), so one
logical operation emits one span on either path and the counters stay
comparable.

No behaviour change; documentation catching up to the code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 21:20:41 +00:00
Stephanie Jarmak
c0f18bc03f
Add append-only provenance_events log and bd provenance CLI (#4461)
* Add append-only provenance_events log and bd provenance CLI

A generic primitive for capturing work provenance — the base commit a
worktree forked from, which actor touched a bead when, what landed it —
as an append-only event log keyed to an issue. Inert by default: nothing
writes it unless a producer (git hook, orchestrator, CI) opts in.

- Migration 0052: provenance_events (CHAR(36) UUID pk, FK issues
  ON DELETE CASCADE ON UPDATE CASCADE, indexes on issue_id/ref/kind).
  Sibling to events, per the wisp_events precedent for purpose-specific
  event tables rather than overloading the field-mutation audit log.
  ON UPDATE CASCADE so a bd rename (which updates issues.id in place)
  does not hard-fail the FK, matching the other aux tables (0042).
- Append-only at the event level: only record/log/by-ref operations, no
  UPDATE or DELETE of an individual event. An event's lifecycle is bound
  to its issue — removed only if the issue is deleted (ON DELETE CASCADE),
  identical to the events audit table (0005).
- Opaque actor/ref (bd never interprets them); only kind and ref_kind are
  structurally validated (ZFC). occurred_at (event-time) is separate from
  created_at (ingest-time). A reserved 'ingest-backfill' source is rejected
  so a producer cannot forge a derived event.
- Idempotent record via a content-addressed deterministic id, so a producer
  firing twice is harmless; an event with neither a ref nor occurred_at is
  rejected at the store boundary (no stable id otherwise). occurred_at is
  truncated to whole seconds for both the id basis and the stored value, so
  the id stays reproducible from the stored row (the column is bare DATETIME,
  second precision) — a sub-second --at records idempotently.
- bd provenance record|log|by-ref; logic shared in issueops, both Dolt
  stores delegate (mirrors the events pattern). The record command calls the
  exported ValidateProvenanceEvent for early CLI feedback, with the store as
  the real boundary.
- ADR-0003 records the rationale (why not extend events, why opaque ids,
  event-level append-only vs issue-bound cascade, distributed-merge note).
  CGO tests cover round trip, idempotency, sub-second-at idempotency, opaque
  field round-trip, and the validation guards.

* Fix provenance CLI lint and review nits

Return JSON output errors from provenance commands, silence Cobra usage/error duplication on the new command group, document deterministic provenance IDs accurately, and add the composite issue/occurred_at index promised by the design.

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie

* fix-merge(provenance): renumber to 0063, drop pinned-doc artifacts, stage provenance_events, ADR to engdocs

Maintainer-side rebase tail for PR 4461 after 505 commits of drift; author's
blockers were all fixed on 2026-07-08.

Co-authored-by: sjarmak <sjarmak@users.noreply.github.com>
Agent-Signature: claude-code-sonnet-builder on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix-merge(provenance) round 2: backend contract aliases + latest-migration mock shape

Two more surfaces main grew after the 7/27 approval:
- backend public-contract completeness ratchet (#5321) requires aliases
  for types reachable from the contract: add ProvKind + ProvenanceEvent
- schema lock tests script sqlmock against the LATEST migration's body
  shape; latest is now 0063 (plain CREATE TABLE -> ExecContext), not
  0062's bare CALL (-> DrainCall/QueryContext)

---------

Co-authored-by: sjarmak <sjarmak@users.noreply.github.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Steve Yegge <steve.yegge@gmail.com>
2026-08-07 09:43:20 -07:00
ecuthiell
80b13798dc
docs(lint): align contributor guidance with CI (saa-90w) (#5317)
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-07 05:37:12 -07:00
Julian Knutsen
33e14b5c82
Move the namespace-neutral error vocabulary into its own leaf (#5395)
memoryops needed one sentinel — ErrValidation — and the value the whole repo
matches lived in issueops. So the memory leaf imported the issue package for
one errors.New and pulled internal/types in behind it, claiming memory sits
downstream of issues. It does not: the two are sibling planes over one config
table.

beadserrors now declares the vocabulary that belongs to no plane in
particular — ErrValidation, ErrNotFound, ErrNotInitialized, the ErrUnsupported
type — and issueops re-exports it by alias, exactly as internal/storage and
backend already re-export from issueops. 141 qualified ErrValidation
references across the tree, zero edited: a Go alias preserves identity.
memoryops now depends on beadserrors and context, nothing more.

What belongs down there is what a leaf for a plane nobody has written yet
would need. ErrCloseBlocked and ErrDependencyCycle name issue concepts and
stay in issueops.

beadserrors imports stdlib and nothing else, enforced by a strict depguard
rule rather than asked for in a comment, because it sits beneath every leaf
and whatever it imports is imported by all of them.

Answers Q4 from #5387.
2026-08-07 04:32:06 -07:00
Julian Knutsen
4e8b017fd4
Put the memory plane behind memoryops, conformance-tested at three backends (#5387)
The memory commands get the treatment the sixteen issue commands got in #5378:
a role interface in a leaf package, conformance-tested at all three backends,
with the CLI routes and the HTTP handlers as thin callers over one
implementation.

memoryops is its own namespace rather than an addition to issueops. The two
planes share a config table and nothing else; a memory whose key shadows a
settings name is still a memory, and that separation is now the role's,
pinned by its contract, rather than a filter each caller reapplies.

Closes an unauthenticated disclosure found on the way: memory content was
served in cleartext on GET /v0/beads/config, and reachable through two more
doors — `bd config show` and `bd info --json`, the latter republished into
agent transcripts by MCP get_schema_info. Enumeration is now filtered at the
shared seam (workapi.FilterSettingsEnumeration); `bd config show` is left
alone deliberately, since an operator asks for it by name.

Two wire contracts were settled after the review council, recorded in full on
the PR: the search parameter is spelled `search` rather than `q`, because
issues:query already publishes `q` as a boolean expression and one surface
must not spell two questions one name; and getMemory keeps its 404, because
the status reports the distinction the front door already reports and
`bd recall` exits 1 on a miss where `bd config get` exits 0.
2026-08-07 03:57:04 -07:00
Steve Yegge
f215b312cf
docs: re-review and bump the three 90-day-stale freshness markers (#5394)
* docs: re-review and bump the three 90-day-stale freshness markers

azure-devops.md, json-schema.md, and otel-data-model.md all carried
Last reviewed: 2026-05-08 — exactly 91 days old today, so the doc-flags
freshness gate went red on every branch at once. Each doc was genuinely
re-reviewed against its Freshness source paths on main; drift found and
fixed:

- azure-devops.md: proxied-server mode note (ado sync/status/projects
  hard-error there); --parent and --project flags documented; duplicate
  --issues row dropped; real ado.org error text quoted; ado.pat's
  yaml-only secret storage noted. Source bug found during review filed
  separately: the cmd-level config reader cannot see a yaml-only PAT,
  so the documented Quick Start fails on main (bd-wvf77).

- json-schema.md: bd show/close/update --json return top-level ARRAYS
  (contract-test-pinned); jq recipes fixed accordingly; envelope
  pagination key documented; show's always-present revision field and
  import's unchanged counter added; error envelope (optional code/hint,
  envelope-mode shape, stdout paths, exit 1) corrected; deprecation
  notice's tty-only condition noted.

- otel-data-model.md: hard opt-in via BD_OTEL_ENABLED + standard OTel
  SDK env vars (BD_OTEL_* legacy); console-only trace export; dolt span
  attrs (shared doltSpanAttrs, dolt.remote, dolt.force_push,
  dolt.conflicts, dolt.merge_with_strategy); facade-wave storage role
  spans; seven new Dolt metrics; AI attrs (bd.ai.duration_ms, 4-attempt
  retry); appendix re-pinned to current main.

scripts/check-doc-freshness.sh passes locally.

* docs(otel): complete claim_verify_lost op label enumeration; qualify retry policy as compact-only

Review findings on #5394: op also takes guarded-update
(claim_verify.go:113) and ready-claim (issues.go:439); the retry-policy
sentence describes compact's loop, find_duplicates makes one unretried
call.
2026-08-07 02:35:11 -07:00
CI Bot
59a1df98b5 Merge origin/main into the facade wave 1-4 series
Five files collided. Four were textual; the fifth was a semantic collision
that no textual resolution would have caught, and it is the reason this is a
merge rather than a rebase — resolving it once beats resolving it 25 times.

THE SEMANTIC ONE. main grew a second bd serve arm (serveSourceStore) that
serves a registered backend from the store the root command already opened,
wiring httpapi.Config from two role accessors. This series grew that same
Config from two roles to thirteen and made Listen refuse a PARTIAL set. Both
sides are individually correct and merge cleanly as text; together they bind
nothing — every registered-backend serve fails at startup with "no database
source". Caught by main's own two serve tests, which is what those tests are
for.

serveIssueRoles now returns the whole set as a struct rather than a growing
list of unnamed results, so a role added to Config without being added here
is a compile error at the call site instead of a startup failure. It stays a
struct rather than an httpapi.Config because the gate test in serve_test.go
requires every httpapi.Config literal in cmd/bd to sit in a function that
consulted serveDatabaseSource, and the extraction has not.

Two test doubles needed the new accessors. serveRolesStore returns nil for
the eleven it does not exercise, which is honest: it never reaches Listen,
and reader and claimer are enough to pin the peel DEPTH that test is about,
since serveIssueRoles peels once and calls every accessor on that one value.
serveIdentityStore does reach Listen, so its roles must be non-nil; they
embed their interfaces rather than implementing them, keeping that file's
stated preference for a nil-panic naming the method over a stub that
answered.

THE TEXTUAL FOUR.

.golangci.yml — both sides edited the same forbidigo path lists. Resolved as
set arithmetic against the merge base rather than by picking a side, then
verified programmatically: every addition from either side survives and every
removal from either side survives.

CHANGELOG.md — unioned.

create_proxied_server.go — rerere replayed a prior resolution; verified by
reading that it carries both main's refusal text and this series' CreateOnly
lifecycle flag rather than one at the cost of the other.

serve.go and serve_test.go — took main's substantive rewrites, re-applying
this series' corrections on top: the extended unauthenticated-surface warning
now names bulk delete, and the roles paragraph says "every role accessor"
rather than "both".

serve_test.go's AST gate keys off Reader+Claimer as the roles-source signal.
That is sound for the question it asks — which database source did this
Config name — but the pair is now hand-maintained against a thirteen-role
struct, so it carries a note pointing at bd-lidlu.
2026-08-06 05:40:12 +00:00
CI Bot
4588d867a1 feat(issueops): put bd dep tree behind the TreeWalker role (bd-lx5iu)
`bd dep tree` had two implementations of one walk. The direct route resolved a
partial id, opened a prefix-routed store and called store.GetDependencyTree; the
proxied route took the id verbatim, opened a unit of work and called the use
case. Each then carried its own copy of the `both`-direction merge, the status
prune, the mermaid and human rendering and the empty-tree wording. The copies
agreed, which is the only reason nobody noticed that a `both` walk was two
independent reads with nothing spanning them. Both now build one
issueops.WalkTreeRequest and ask for the role.

THE ROLE, and both its neighbours said in advance why it had to be one.
Relations and EdgeReader each exclude the tree in the same words -- "a recursive
walk has a depth, a cycle policy and a node shape of its own" -- and all three
are on this contract: MaxDepth, the first-visit rule, TreeNode. It is not
CycleDetector either: a cycle has NO anchor and a tree is nothing but its
anchor, and the two disagree about what an edge is -- this walk follows every
type except `relates-to`, the cycle report follows `blocks` and
`conditional-blocks` only -- so one workspace can honestly have no cycles and a
tree that revisits a node.

ONE BODY, THREE LEGS, AND THAT IS ONE VOTE. Unlike every other role in
backend/conformance, all three implementations reach the same function: the two
stores wrap issueops.WalkDependencyTreeInTx in five lines around their own
transaction and the unit of work reaches it through a new domain repository
method. So TestTreeWalkerContract x 3 is one reading of the walk plus an engine
check, not three backends agreeing, and the contract file says so at the top.
What the legs still buy is the WRAPPERS, which is where every measured drift in
this graph family has actually lived -- so the refusals assert SENTINELS rather
than message text and the cap case asserts the typed error's fields. The parts
that decide what the answer MEANS are pure (ValidateWalkTreeRequest,
PruneTreeByStatus, MergeBidirectionalTree) and are pinned without a database.

TWO USER-VISIBLE CHANGES ON THE PROXIED ROUTE, both in the owner's Q11 set, both
in CHANGELOG.md.

  PARTIAL IDS RESOLVE. `bd dep tree a1b2` worked on a direct workspace and
  failed on a team server, because this route passed the argument to storage
  verbatim. It resolves in its own short-lived read unit of work before the role
  is asked for anything, so a lookup that finds nothing reports the lookup
  failure rather than a provider that cannot offer a surface.

  --max-rows IS HONORED. It used to be refused outright ("not supported in
  proxied-server mode") because the unit-of-work path threaded no cap. The cap
  is on the role's request now, so the shared body enforces it for all three
  backends and the flag means the same thing wherever the command runs. It is
  still POST-HOC -- a tree walk has no query filter to thread a cap through --
  and both the leaf doc and --help say so. The shared --max-rows help text
  forked for this (addRoutedMaxRowsFlag): the proxied-server refusal is still
  true for the eight other commands that take the flag.

A DEFECT FIX NOT ON THAT LIST, recorded as AMBIGUITIES.md A-twk-1: a `both` walk
now sees ONE database state on every route, where it used to be two
unsynchronised reads that could show half an edit.

Q7 IS OBEYED. --show-all-paths stays a no-op -- accepted, ignored, and now
DOCUMENTED as ignored -- and TreeNode.Truncated stays always-false for the same
reason. The one-visit rule that makes both dead is now a stated promise
(TreeResult.Nodes) and a conformance case (RendersASharedSubtreeOnce), so
implementing either as a side effect of a later refactor fails a test instead of
shipping.

CONFORMANCE. backend/conformance/tree_walker_contract.go holds fifteen cases
citing the leaf doc by line, run at all three implementations with ZERO parks:
45 subtests green. Seeding a cycle takes raw SQL for the reason the
cycle-detector contract records -- no supported write creates one -- so that one
case uses the fixture's Exec hook.

HTTP. GET /v0/beads/dependencies/tree, spec first and the types generated
against it. TreeNode is x-go-type-pinned to internal/types, so `bd dep tree
--json` and the response body are one compatibility domain, and
DependencyTreePage's Items are pinned in pinning.go -- the trap wave 2 fell into
with DependencyEdges (bd-5o3gt), not repeated here. A root that names nothing is
a 404, which is the difference from GET /v0/beads/dependencies: one anchor
leaves no other answer to preserve. No max_rows on the wire; the cap is a CLI
circuit breaker. `capabilities` gains dependencies.tree, and TreeWalker becomes
a required field of the store-shaped httpapi.Config, which costs the in-tree
tests a line each and no shipped caller anything (bd serve builds a provider).

NOT TOUCHED: the legacy DetectCycles seam bd-5g4f2 is about. This walk reads
GetDependencyTreeInTx and never AppendBlockingGraphInTx, so the two-cycle-answers
problem is exactly where it was.

Verified: build, vet, gofmt clean; golangci-lint 0 issues; make api-check green;
TestTreeWalkerContract 15/15 at dolt, embeddeddolt and uow; internal/httpapi 0
failures including the eight new tree-route cases; the pure tree_walk tests
green; TestEmbeddedMaxRowsNonListPaths green (DepTreeMaxRows still exits 2);
BEADS_TEST_PROXIED_SERVER=1 TestProxiedServerDep2 green including the four new
Tree cases; TestProxiedLookup* green; TestCorpusGolden green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 03:09:18 +00:00
CI Bot
27ffb42e1f feat(init): put bd init's identity seam behind Bootstrapper and InitVerifier (bd-zl3u8)
`bd init` had two implementations of what a workspace's identity IS. The direct
route read issue_prefix out of the config plane at init.go:1358 and _project_id
four hundred lines later, decided adopt-vs-write with two helper predicates, and
wrote both keys by name. The proxied route called domain.BootstrapUseCase, which
held its own list of which keys an identity is made of, wrote them
unconditionally, and folded the Dolt remote into the same call. The two agreed on
the key names and disagreed on the rule, and no test compared them.

ONLY THE IDENTITY SEAM IS A ROLE. Everything else init does — creating .beads/,
writing metadata.json and config.yaml, initializing git, installing hooks,
starting a Dolt server — CREATES the substrate or the workspace around it, and a
role is reached through an accessor ON a store, so it cannot be what makes the
store exist. init.go alone is 3,060 lines; this commit touches the part that
happens inside the database.

TWO ROLES, AGAINST THE VersionReconciler PRECEDENT. That one is a read and a
write on one role because they are two shapes of one question, and every word of
that is true here too. The test that separates them is different: can one caller
be entitled to the READ and not the WRITE? Here yes, and not hypothetically —
`bd init --team-server` adopts a bts-provisioned identity and is forbidden to
write one, and a gateway credential may be read-only. Those paths now hold an
InitVerifier and never obtain a Bootstrapper, which the type system checks;
with one role they would hold the writer and the suppression would stay where it
is today, in three should* predicates on the CLI.

Q8 ANSWERED IN THE ROLE. Bootstrap over an already-identified substrate is
ErrAlreadyIdentified, carried by an *AlreadyIdentifiedError naming what was
found, decided from the same read the write would have followed inside ONE
transaction. Either marker is enough: a substrate several rigs share carries a
prefix one of them chose, and overwriting it renames every id the others are
about to mint. The guard now sits below both front doors instead of in front of
one, so gateway and adopt paths inherit it, and re-identifying is something
somebody asks for rather than something init does twice.

THE ROLE OWNS THE IDENTITY AND NOTHING ELSE. BootstrapProject wrote six values;
BootstrapRequest carries two. The fingerprints, the synced-at marker and the
recorded binary version are refreshed on EVERY init, adopt or not — a fresh clone
of an already-identified database needs its own fingerprints precisely because it
bootstrapped nothing — so a refusable one-time write would mean either a refusal
that stops recording them or a write that is no longer one-time. bd_version has a
second reason: it is VersionReconciler's key. The Dolt remote left too; it is
where the database syncs to, not what it is, and putting them in one call is what
let a remote that could not be created fail a bootstrap that had already
succeeded. domain.BootstrapUseCase is DELETED and domain.ConfigUseCase gains
SetMetadata, the write half of the GetMetadata it already published, so the
unit-of-work body can read, refuse and write in one transaction.

THE DEFECT. Proxied `bd init` against a database another rig had already
identified rewrote both markers with its own derived values. It verifies and
adopts now, printing the direct route's existing line, with the role's refusal as
the backstop. The direct route's two identity reads also became one snapshot, so
the prefix it adopts and the project id it reconciles against can no longer come
from either side of a concurrent write; shouldConsultInitProjectID still gates
whether the id is USED, so which identities init adopts is unchanged.

CONFORMANCE. backend/conformance/bootstrapper_contract.go holds twelve cases —
both roles in one file, because every promise on either is a statement about the
other — run at all three implementations with ZERO PARKS and zero divergences.
They deliberately do not re-test the decision: the validator's and the refusal's
whole tables are pinned without a database in internal/workapi/bootstrap_test.go.
What only a real backend can show is asserted instead: that the identity is there
for the NEXT caller (a new transaction on the unit-of-work backend), that a
refusal and a validation failure leave the markers EXACTLY as they were, that the
verifier reports an unreadable identity as an error rather than as an empty one,
and that a bootstrap costs at most one version-control entry. Two independent
bodies plus an engine check, not three votes: dolt and embeddeddolt share
internal/storage/issueops.BootstrapInTx and VerifyIdentityInTx and differ only in
how they reach a transaction. The bodies live there rather than in an importable
store<role> package because the read and the write it qualifies must see one
snapshot, which storage.DoltStorage cannot express — so there is no constructor
for a depguard entry to deny.

NO HTTP OPERATION, and that is a decision. A server can only serve a database it
is already bound to, and binding it required the identity this role writes, so
the only caller that could reach such an endpoint would be asking a running
server to identify a workspace that is by construction already identified — which
is the refusal. Bootstrapping over an unauthenticated surface is a different risk
class from reading one. The read half already has an endpoint:
ContextResponse.project_id on GET /v0/beads/context is InitVerifier's answer, on
the wire, today. Recorded in both leaf docs and in the checklist.

USER-VISIBLE, for the wave's changelog pass: proxied init adopts instead of
overwriting on a shared database; `bd init` no longer heals a substrate carrying
a prefix and no project id, which is Q8 as written; an explicit BEADS_DIR outside
the repo now gets a project id written from its own metadata.json instead of
being left half-identified. Reasoning and the alternatives are in
/data/tmp/sd-server/AMBIGUITIES.md as A-ini-0..7.

A-ini-0 IS A FALSE PREMISE IN THE BRIEF, not a finding about the code: bd-zl3u8,
bd-jtp1h and bd-yby99 do not exist in the workspace at BEADS_DIR. Verified by two
bd binaries failing to open it and by a direct SQL read of all 5,603 issues.
Everything else the brief asserted about the code checked out, so the spec used
was the brief plus Q8 plus A2.

Verified: build, vet, gofmt clean; golangci-lint 0 issues (its unparam check is
what found resolveInitIssuePrefix's now-dead `write` return, removed here);
TestBootstrapperContract 12/12 at dolt, embeddeddolt and uow; internal/workapi,
internal/storage, internal/telemetry, internal/jira, issueops, the root layering
pins and the whole internal/storage/domain and internal/storage/uow packages
green; internal/httpapi green and `make api-check` clean with no spec or
generated diff; TestInit* and TestResolveInit*/TestApplyInitGateway* and
TestCorpusGolden and TestDoctor* green under BEADS_TEST_EMBEDDED_DOLT=1;
TestProxiedServerExternalCreate, TestProxiedServerPing and TestProxiedServerUpdate
green under BEADS_TEST_PROXIED_SERVER=1, each of which runs
`bd init --proxied-server` in its harness; TestProxiedLookup green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 03:01:33 +00:00
CI Bot
519fb1479b feat(issueops): put bd purge and bd prune behind the Sweeper role (bd-pn231)
`bd purge` selected closed wisps in one transaction and deleted them in
another. A bead closed, unpinned or cited in the window between the two was
judged on one state and acted on in a different one — the TOCTOU the bead
names — and the proxied twin held a second copy of the whole flow: its own
candidate filter, its own glob match, its own closed_at recheck, its own
reference scan and 190 lines of output kept character-identical to the direct
route's by hand.

PURGE AND PRUNE ARE ONE CAPABILITY, which the design left open and the code
answers plainly: both routes already ran ONE function with a `purgeScope`
struct choosing the tier. So issueops.Sweeper has ONE method, reached through
store.Sweeper() and the provider's own accessor, and SweepRequest.Tier selects
the plane — `ephemeral` is purge, `durable` is prune, and the two are disjoint
by construction. What genuinely differed became request fields rather than a
second role: the require-a-filter refusal keys on Tier, and the citation
protection is ASKED FOR (ProtectReferenced) instead of being inferred from a
command name, which is what the shared body did (`scope.cmdName == "prune"`).

Q9 ANSWERED IN THE ROLE. The require-a-filter gate is
workapi.ValidateSweepRequest, below every front door: a durable sweep with
neither a cutoff nor an id pattern is ErrValidation. It is a safety invariant,
not presentation — without it a second front door is one handler away from an
unguarded mass delete, which is exactly what this commit's HTTP operation would
otherwise have been. The CLI keeps its friendly message, and says in place that
it is there for the WORDING while the guard is the role's; the contract case is
what proves the guard survives that branch being deleted.

THE DEFECT, AND ITS PRICE. The whole sweep is now one transaction, so the set
the result describes IS the set it deleted. That gives up the server-backed
store's batched wisp deletion (200 per transaction, added for Dolt's 10s write
timeout after bd-2ehd and ff-tqm): a sweep too large now fails WHOLE. That is
the better failure — the partial one it replaces reported a deleted count of
ZERO while having deleted rows — and the other two backends already worked this
way. It is written down as a promise WITH its cost, and recorded for sign-off.

TWO MORE DEFECTS the single call site removed. `--dry-run` passed force=false
to the delete where the real run passed force=true, so a preview could FAIL
over dependents the sweep would have orphaned — and the CLI answered that
failure with zeros. And `filepath.Match`'s error was discarded on both routes,
so `--pattern '['` printed "No closed beads to prune" and exited 0; it is now
ErrValidation quoting the pattern.

THE CONTRACT IS WHAT WAS DELETED, not what was reported.
backend/conformance/sweeper_contract.go holds ten cases, each citing the leaf
doc, run at all three backends with ZERO PARKS and zero divergences. Every case
that acts asserts row counts through QueryScalar rather than trusting the
result the sweep reported about itself, which on a destructive operation is the
assertion that matters; every case scopes itself with an id pattern, because a
sweep is asked of a whole TIER and an unscoped one would clear the next case's
seeds. They are TWO INDEPENDENT BODIES plus an engine check: dolt and
embeddeddolt share internal/storage/issueops.SweepInTx and differ only in how
they reach a transaction, and the unit-of-work provider reaches the same four
questions through the domain use cases.

THE HTTP OPERATION, spec first (per bd-jtp1h's A2 scope decision):
POST /v0/beads/issues:sweep, capability `issues.sweep` — the first DESTRUCTIVE
operation on a surface that was reads plus one claim. It is a collection-level
custom method with a body rather than DELETE with a query string, because a
dropped query parameter widens what is erased where an unknown body member is
refused by name. The handler decodes six members and calls the role; it does
not implement the safety gate, the tier vocabulary or the glob, all of which
are the role's ErrValidation reaching the wire as a 400 through failSweepErr.
No component carries allOf and none is x-go-type-pinned — there is no canonical
Go struct whose JSON encoding is this body, since the CLI publishes these
numbers under `purged_count`/`pruned_count` — so the projection is tested
instead. `bd serve --help` grew a DESTRUCTIVE OPERATIONS section and the
--allow-non-loopback warning now says peers can bulk-delete closed beads.

`bd gc` keeps the closed-candidate recheck by calling the SAME pure function
the role calls (workapi.FilterSweepCandidates), so the two cannot drift; the
reference matcher and its 10K-body budget moved to internal/workapi with the
scan. cmd/bd/purge.go and purge_proxied_server.go LEFT the forbidigo exception
list, which is how a role commit proves its front doors reached the role.

USER-VISIBLE, for the wave's changelog pass: the version-control label for a
sweep is now `bd: sweep <n> <tier> bead(s)` on both routes, where proxied wrote
`bd: prune <n> bead(s)` and direct wrote `bd: delete <n> issue(s)` — an HTTP
sweep is not `bd prune`, so the label names the operation and the tier from
every door; a malformed `--pattern` now exits non-zero; `--dry-run` no longer
refuses over dependents the real sweep orphans; and `bd serve` gains a
bulk-delete endpoint. Reasoning and the alternatives are in
/data/tmp/sd-server/AMBIGUITIES.md as A-swp-1..9.

Verified: build, vet, gofmt clean; golangci-lint 0 issues; make api-check
(regenerate-and-diff plus the httpapi suite) green; TestSweeperContract 10/10 at
dolt, embeddeddolt and uow; the internal/workapi, internal/storage and
internal/telemetry suites; TestEmbeddedPurge, TestEmbeddedPurgeConcurrent,
TestEmbeddedPrune, TestPrune_protectsBeadCitedByCustomStatusBead and
TestEmbeddedGC under BEADS_TEST_EMBEDDED_DOLT=1; TestProxiedServerPrune under
BEADS_TEST_PROXIED_SERVER=1; TestProxiedLookup*; and the root layering pins.
The protocol corpus has no purge/prune case to account for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 16:40:26 +00:00