* fix(configfile): carry graph_mode and graph_* binding through Save
Graph-preview builds of bd persist a per-workspace format in
metadata.json: "graph_mode": "link" plus graph_scope_url,
graph_authority_id, graph_workspace, graph_schema_version and
graph_ready, beside an ordinary Dolt backend. configfile.Config had no
such fields, so every Save, and Load's config.json -> metadata.json
migration, rewrote the file without them, turning a graph workspace's
metadata into an ordinary Issue workspace's.
Config now carries the marker and the binding as omitempty fields, so
a Save round-trips them unchanged and metadata.json written for an
ordinary workspace is byte-for-byte what it was. GetGraphMode resolves
the marker: absent or "dependency" is the historical default, "link"
is recognized, and any other value is an error rather than a fallback
to the Issue store. Nothing in this build opens a non-default mode;
the CLI refusal follows in the next commit.
Ported from the donnabox fork commit deea8d57df (graph_mode.go,
graph_mode_test.go and the configfile.go fields), without any graph
code. The binding-survives-Save and omitted-from-ordinary-metadata
tests are new.
Co-authored-by: Donna Box <dbox@boxenllc.com>
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): refuse graph_mode workspaces before opening them
A workspace whose metadata.json declares a graph_mode other than
"dependency" holds a graph database behind an ordinary Dolt backend,
and nothing else in its metadata says so. bd admitted it like any
current workspace: `bd list` ran the Issue schema initialization
against the graph database (beads_graph_<id>), `bd init --force` wrote
a .gitignore and a .beads/dolt root into it, `bd dolt start` started a
server for it, and `bd doctor` reported on it as an Issue store.
guardLegacyUpgradeWorkspace, the admission check every store-opening
command, store-less command with workspace effects, init, doctor and
bootstrap already run before touching a workspace, now refuses such a
workspace first, before the legacy classifications that would admit
it or report it as a legacy store to migrate. An unknown graph_mode
value is refused the same way. The refusal is a typed error naming the
workspace and the format, rendered with a hint (as JSON under --json)
and exit code 16, ExitUnsupportedWorkspaceFormat, so a wrapper can
tell "needs a different bd build" from a generic failure.
`bd version`, `bd --version`, `bd help` and completion stay exempt, as
they are for legacy workspaces. validateConfiguredBackend refuses the
same metadata, which covers the store-open paths that skip command
admission (routed writes, cross-repository reads, create --repo).
Tests: guard and store-factory unit tests; a subprocess e2e test that
show/list/create/context/where/dolt start/doctor/bootstrap/init are
refused with exit 16 and leave .beads byte-identical (metadata.json and
legacy config.json, link and unknown markers, embedded and server
layouts), and that store-less commands still run; an embedded test
that an ordinary workspace, with the marker absent or "dependency",
still creates and lists issues and keeps the marker through a
metadata.json rewrite.
The guard hunk and graph_mode_admission_e2e_test.go are ported from
the donnabox fork commit deea8d57df; the exit code, rendering, store
factory backstop and remaining tests are new.
Co-authored-by: Donna Box <dbox@boxenllc.com>
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): refuse graph workspaces reached outside command admission
Command admission refuses a graph_mode workspace a command runs in, but
bd also reaches workspaces before or without admission: a create --repo
target, the path --db or BEADS_DB names, and the stores the factories
open for routed and cross-repository work. On those paths a workspace
marked only by a legacy config.json was changed before anything refused
it: configfile.Load migrates config.json into metadata.json before the
format check sees the loaded metadata, and create --repo initialized
such a target as a new workspace. Where the refusal did come, create
reported it with exit 1 rather than 16.
- checkDiscoveredWorkspaceFormat checks a workspace's metadata as
configfile.LoadForDiscovery reads it, which never rewrites it.
ensureBeadsDirForPath, the dry-run target open and the four store
factories (cgo and nocgo builds) run it before their own load.
- resolveBeadsDirForDBPath and findDatabaseInBeadsDir locate a
workspace with LoadForDiscovery, so resolving --db, BEADS_DB or a
create target no longer migrates it.
- bd create renders a refused target as the workspace refusal, exit 16
with its hint, on the initializing and both dry-run paths. A dry run
without --parent opens no target, so it now checks a local one before
rendering the preview (checkRepoTargetFormat).
- bd bootstrap, when discovery finds no workspace, reads each .beads
from the working directory up to the repository root without
migrating it and refuses one in a format this build cannot open, the
way commands without a workspace already do. A .beads marked only by
config.json is passed over by discovery, so bootstrap reported "no
workspace" there instead of exit 16. The walk now returns its
candidates (undiscoveredWorkspaceCandidates) to both checks: handing
it the check as a func(string) error made gosec's G702 taint analysis
connect BEADS_DIR to every func(string) error that runs git.
- backendNameForErrorFraming reads with LoadForDiscovery, and
loadDoltBackendConfig, bd dolt status and bd dolt show refuse a graph
workspace before their migrating configfile.Load. Admission refuses
first on every path that reaches them today; this keeps a new caller
from migrating a legacy config.json before the refusal.
- The CHANGELOG entry and the configfile comment now claim what holds:
a routed read fails without touching a graph workspace but not with
code 16, and a Save keeps only the known graph_* fields, with the
tags graph-preview builds declare. workspace_gate.go no longer lists
the pre-gate discovery reads as migrating.
- New e2e rows run create --repo from a workspace and from outside one,
create --dry-run with and without --parent, --db, BEADS_DB, init
--proxied-server, bootstrap in a workspace nested in a graph one, and
the hook shim's bd hooks run pre-commit against every fixture; each
refusal must exit 16 and name the format and the hint. bootstrap in a
workspace marked only by config.json now refuses like the other
commands. The store factory test also runs on a config.json graph
workspace, TestWorkspaceMetadataReadersLeaveGraphWorkspaceUnchanged
covers the framing, bd dolt loader and diagnostic reads, and
TestNocgoStoreFactoriesRefuseGraphWorkspace covers the CGO_ENABLED=0
factories; bazel.yml's pure lane runs it by name (PURE_CMD_BD_TESTS,
pinned in scripts/ci_workflow_test.go).
Measured with gocyclo/gocognit: every function this changes stays below
the PR's worst changed function (guardLegacyUpgradeWorkspaceForInit
29/28, resolveBeadsDirForDBPath 26/36); showDoltConfig is 19/31 (was
18/30) and guardUndiscoveredLegacyWorkspace 3/3 (was 9/12).
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Donna Box <dbox@boxenllc.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
The root //:go_srcs filegroup is glob(["**/*.go"]), and a glob follows the
checkout's bazel-bin/bazel-out/bazel-testlogs/bazel-beads symlinks into the
output tree. CI's test step is the first build in a fresh checkout, so it never
sees them, but in any checkout that has built before every test that reads
//:go_srcs (bd_test, bd_embedded_test, bd_embedded_part2_test,
bd_dolt_server_test, bd_proxied_test, bd_managed_local_test) gains the output
base's .go files: their inputs differ on every run, they are never cached, and
loading grows with the output tree.
Measured on main 2563f021ed with the symlinks present, `bazel test
//cmd/bd:bd_test --config=ci --test_filter='^TestResolveCommitHash$'` twice:
both runs executed all 16 shards remotely, and the analysed graph grew from
20,773 to 56,941 configured targets (14 s, then 25 s). With this change the
second run is `(cached)` in 3.6 s. In agent worktrees the glob had reached
433,026 output-tree files, a cached re-run took 36 s to load against ~3 s
without them, and 135 of 202 blocking test minutes of the factory's beads
seats went to these six targets (formula-overhead audit 4; bead ga-ylpbsy.24).
tools/bazel/go_srcs.py gives the root block exclude = ["bazel-*/**"] (the
same pattern ROOT_REPO_FILES_EXCLUDE already uses for //:repo_files); only
the root BUILD.bazel changes. CI's action keys do not change: its checkout has
no such files. scripts/bazel_policy_test.go's
TestBazelRootGoSrcsSkipsConvenienceSymlinks fails without the exclude.
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cmd/bd): repair the embedded-gated tests that rotted while no lane ran them
These tests skip unless BEADS_TEST_EMBEDDED_DOLT=1 but no CI lane selected
them, so they rotted unseen. Running them shows:
- TestEventsTailFollowReportsTruncationOnTheStream raced on a plain
bytes.Buffer that os/exec's copy goroutine writes while the test polls it.
Use the package's syncBuffer.
- TestCorruptMetadataDiagnosticsRunAndDataFailsLoud still asserted that
doctor and init run on a corrupt metadata.json. #4857 deliberately made
every workspace command refuse instead (pinned for doctor by
TestDoctorRejectsCorruptMetadataBeforeSharedServerChecks). Rewrite it to
the shipped contract as TestEmbeddedCorruptMetadataFailsLoudUntilRestored:
version warns, doctor/list/init refuse naming the file without touching
it, and restoring the file brings the data back.
- dolt_remote_origin_guard_embedded_test.go was tagged
`embeddeddolt && cgo`, a tag no lane sets, so it never compiled in CI
(gazelle left it out of bd_test srcs). Retag it `cgo`, as #4153 did for
its dolt_local_only sibling, and add it to srcs. Drop its four bd doctor
git-origin tests: bd doctor runs no checks on an embedded workspace
(GH#3794), so two could never pass and two passed vacuously on empty
output. Rename the surviving tests into the TestEmbedded* prefix.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(embedded): run every BEADS_TEST_EMBEDDED_DOLT-gated cmd/bd test in the embedded lane
The embedded-Dolt cmd lane selected tests with
grep '^func TestEmbedded' cmd/bd/*_embedded_test.go, and only its two
Bazel targets set BEADS_TEST_EMBEDDED_DOLT. A test that skips without the
variable but missed the name prefix or the file suffix skipped in every
lane and reported green: 92 of them, including #7203's --if-revision CLI
coverage (TestIfRevisionMatchAndMismatch, TestReassignFenceCLI, ...).
- embedded-test-shard.sh and gen_embedded_cmd_shard_manifest.py select
TestEmbedded* by name across cmd/bd/*_test.go, as proxied-test-shard.sh
does, leaving authors one rule.
- Rename the gated tests into the TestEmbedded* prefix
(TestDoctorEmbedded<X> -> TestEmbeddedDoctor<X>) and update every
reference.
- Place the new names with gen_embedded_cmd_shard_manifest.py 100
--weights=duration --write (incremental).
- Add TestEmbeddedGatedTestsRunInTheEmbeddedLane: it walks cmd/bd's tests
for any that reach os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" and fails
if a shard of the lane never lists one. The proxied guard's helpers now
take the variable and the script; a fixture test pins the walk.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): pin vet policy to module toolchain (wy-kk1b9a)
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Steve Yegge
* test(bazel): report an unavailable pinned toolchain as a missing host tool
Review fixes for #7668 (adopt-pr iterations 1-2).
The GOTOOLCHAIN pin added a failure path that base did not have. A host
that runs another Go release and cannot fetch go.mod's toolchain
(offline, GOPROXY=off, empty module cache) failed
TestPRRunsGoTestsBazelSkips with a bare "exit status 1": cmd.Output()
keeps the go command's stderr in the ExitError, and t.Fatal printed
only the status.
- Report the failure through skipOrFailWithoutHostTool, naming the
pinned version and the go command's stderr when it printed any.
Plain go test skips; Bazel still fails, and there the registered SDK
is go.mod's toolchain, so nothing is fetched.
- Compare VET_PASSES in a subtest, so the skip covers only the check
that needs the toolchain. Given the parent's t, it skipped the whole
test, and with it the six checks after it that read only the
checkout. A failed subtest still fails the parent, so Bazel stays
fail-closed.
- Pad a two-component version ("go 1.26" with no toolchain line) to
go1.26.0 in goToolchainName, as scripts/ci/gofmt-bin.sh does.
GOTOOLCHAIN=go1.26 is rejected as a language version, not a
toolchain version. A table test pins 1.26, 1.26.7 and 1.26rc1.
- Say in the helper comment that the pin overrides an ambient
GOTOOLCHAIN, local included, and may download into GOMODCACHE.
Validated with go1.26.7-built test binaries under env -i, GOPROXY=off,
with an empty HOME and module cache unless noted:
- go1.26.9 host: the VET_PASSES subtest skips, naming go1.26.7 and
"toolchain not available", and the parent passes. With a planted
scripts/ci/go-test-vet.sh the parent fails, as on base; iteration 1
skipped the whole test there (rc 0). Same on go1.26.5 with
GOTOOLCHAIN=local.
- go1.27.1 host with go1.26.7 cached: passes; base fails on
stdversion. go1.26.7 host: passes.
- Bazel branch (TEST_SRCDIR, BEADS_TEST_GO): SDK go1.26.7 passes. SDK
go1.26.9 fails the subtest and the parent with "executor lacks a
host tool". A non-executable SDK path ends the message with the exec
error, with no dangling ": ".
- go.mod "go 1.26" without a toolchain line: passes on go1.26.0 and
under Bazel with SDK go1.26.0; a go1.26.9 host skips, naming
go1.26.0. Iteration 1's unpadded mutant only skipped under plain go
test (it failed under Bazel); the table test now fails it, and fails
mutants that count dots or drop the regex anchors.
- go test ./scripts on go1.26.7: 306 pass, 0 fail, 1 skip; the only
outcomes not on iteration 1 (305/0/1) are the two new tests passing.
go vet ./scripts and the pinned gofmt -l are clean.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
run-oracle-a.sh's EXIT trap removed its reference worktree only when
`git worktree list --porcelain | grep -qF "$REF_SRC"` matched. Under
pipefail that check reported a registered worktree as absent in two ways:
grep -q exits on its first match while git is still writing a large
listing, so git dies of SIGPIPE (141) and the pipeline fails; and git
lists the resolved path, which $REF_SRC never matches when $TMPDIR has a
trailing slash or runs through a symlink (both macOS defaults). Either
way the ref-src registration leaked as a prunable worktree.
Run `git worktree remove --force` unconditionally. It resolves the path
itself and fails harmlessly (silenced, `|| true`) when nothing is
registered, so cleanup is idempotent and exit codes are unchanged.
TestOracleAScriptRemovesItsReferenceWorktree runs the real script in a
fixture repo with a stand-in go that fails the reference build, covering
a >3 MB listing, a trailing-slash TMPDIR, a symlinked TMPDIR, and a run
that dies before the worktree is added.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
beads_test_env_enter and its Bazel mirror tools/bazel/test_env.sh
already unset the BEADS_DOLT_* connection variables, but not the server
user and password, nor BD_AGENT_MODE, BD_NON_INTERACTIVE and
BEADS_MAIL_DELEGATE. A test run from a shell configured for a shared
Dolt server or for agent use inherited them: with
BEADS_DOLT_SERVER_USER set, TestDoltServerMode and
TestDoltServerModeRoundtrip in internal/configfile fail because the env
value overrides the user the test configured.
Unset all five in both wrappers so the lists stay in step.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Signature: claude-code-claude-fable-5-1-high on behalf of Raymond W. Ko
* fix(scripts): update-nix-vendorhash no longer reports a build failure as success
scripts/update-nix-vendorhash.sh treated the absence of a "got:" hash
line as proof the vendorHash was already correct, so a build failure
unrelated to a hash mismatch (e.g. a Go toolchain too old for a
dependency) was misreported as success and the workflow went green on
a broken build (#7496). Fixed by capturing the intentional-failure
build's real exit status instead of discarding it with `|| true`, and
using that exit status (not a "hash mismatch" string grep) to tell a
genuine no-op apart from an unrelated build break.
The step-5 verification build had the same shape of bug: piping
through `tee` without pipefail meant the if-condition tested tee's
exit status, not the build's, so a failing verification build still
printed "Build successful!". Scoped `set -o pipefail` to just that one
line rather than the whole script, since the earlier hash-extraction
step legitimately relies on `grep -o` returning non-zero on no match.
Fixesgastownhall/beads#7498.
Added an independent `nix build .#default` step to
update-vendor-hash.yml so the workflow no longer trusts the script's
exit status alone, even if a future bug in the script reintroduces a
false-success path.
Fixesgastownhall/beads#7499.
Added TestPinnedDoltCLIAtLeastRecommendedMin, asserting the pinned
Dolt CLI version (scripts/ci/install-dolt.sh) is at least
doltversion.RecommendedMin, so the two policies can't silently drift
apart — RecommendedMin's own doc comment already claims this alignment
but nothing enforced it.
Fixesgastownhall/beads#7500.
Verified by reproducing both update-nix-vendorhash.sh failure modes
with a stub `nix` binary (a non-hash build failure, and a step-5
verification build failure) and confirming the intended successful
hash-update path still works end to end. go build/vet clean,
golangci-lint 0 issues, go test ./scripts/... passes except a
pre-existing unrelated failure (TestControlQueryScriptPreservesReadyProbeFailure,
tracked separately as beads#7293 and untouched by this change).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr
* chore: empty commit to retrigger CI -- retrying after a 3-hour quota recovery window
* chore: empty commit to retrigger CI -- isolated single retry after real-bug fixes landed on sibling PRs
* fix(scripts): add missing BUILD.bazel dep, sharpen vendorhash diagnostics
scripts_test now imports internal/doltversion (for
TestPinnedDoltCLIAtLeastRecommendedMin) but scripts/BUILD.bazel had no
matching dep row, which would fail the target's compile under
rules_go once Bazel actually runs against this branch (the Bazel /
jobs failing so far died in bazel.fork-cert, unrelated to this).
Also, per review: scope set -o pipefail to just the step-5 verify
block (turn it back off after, rather than leaving it on for the rest
of the script with a comment claiming it's scoped when it isn't), and
keep the old "hash mismatch" string check as a sub-case in the
ACTUAL_HASH-empty branch, so a hash mismatch whose got: line fails to
parse (e.g. a future Nix output-format change) is diagnosed
differently from a build that failed for an unrelated reason.
Re-verified all four failure/success paths with a stub nix binary
(non-hash failure, hash-mismatch-with-unparseable-got-line,
step-5-verify failure, and the genuine happy path). go build/vet
clean, golangci-lint 0 issues.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr
* fix(scripts): update-nix-vendorhash preserves operator backups, restores on any handled failure
The script backed up default.nix to a fixed name in the repo root,
default.nix.backup, before editing it -- but that is also the
conventional name an operator gives their own manual backup. Running
the script unconditionally overwrote (then later deleted) any such
file. A failed step-5 verification build also left the candidate hash
in default.nix with no backup at all to restore from, since the
backup had already been deleted right after step 4 applied the real
hash.
Moved the backup into a private per-run scratch directory
(mktemp -d, cleaned on every exit via an EXIT trap) instead of a
fixed repo-root name, so it can never collide with an operator's own
default.nix.backup. Copy with cp -p so a restore brings back the
exact original bytes AND mode, not just content. Kept the backup
around through step 5 instead of deleting it right after step 4, and
added a restore on the step-5 verification-failure path, which
previously left the unverified candidate hash in place.
Also: the "no hash mismatch, already correct" exit-0 path previously
inferred correctness purely from the fake-hash probe build's
unexpected success, without ever building the real, restored file.
It now runs a second build against the restored file and only exits 0
if that succeeds too, surfacing a diagnostic log if it doesn't.
Fixesgastownhall/beads#7529.
Verified with a stub nix binary across five scenarios: a pre-existing
operator default.nix.backup (distinct content + mode 600) survives
untouched in every one; a non-hash build failure restores the
original (content + mode 640); the already-correct path now runs two
builds and only succeeds if both do; the core bug repro (hash
extracted, step-5 verify fails) now restores the original instead of
leaving the unverified candidate hash with no backup; the genuine
happy path still updates the hash correctly. No leftover scratch
directories in any case. go build unaffected (shell-only change).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr
* fix(scripts): update-nix-vendorhash restores the original on an interrupted run
A SIGINT/SIGTERM during the probe build (the long step) reaches this
script and its nix child as one process group; the child dies, this
script exits through the EXIT trap with the placeholder hash still in
default.nix. The plain `rm -rf "$SCRATCH_DIR"` trap deleted the only
backup at that point, leaving no way back except `git checkout
default.nix` -- which also discards any uncommitted operator edits.
Worse than every handled failure case this file already restores
from.
The placeholder is never a valid end state, so the EXIT trap now
checks for it and restores first, before dropping the scratch
directory. Every handled exit path has already restored or verified
default.nix by the time it runs, so this is a no-op there -- it only
fires on an interrupted run.
Found by review on PR #7504 (beads#7529's own follow-up): reproduced
and confirmed with a stub nix that signals its own parent process
group, matching a real Ctrl-C mid-build. Confirmed the bug first
(plain trap: placeholder left in place, backup gone, matching the
report exactly), then confirmed the fix (original hash and mode
restored, operator backup untouched, no leftover scratch dir). Also
re-ran the full existing scenario matrix (non-hash failure, hash
mismatch with step-5 verify failure, genuine happy path) to confirm no
regression.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VN2SgXAHW1eTB1izwWwmyr
* fix(scripts): restore default.nix if vendorhash verification is interrupted
The EXIT trap restored default.nix only while it still held the placeholder
hash. A SIGINT/SIGTERM during the step-5 verification build therefore left
the unverified candidate hash in place and deleted the only backup with the
scratch dir. Track VERIFIED instead: until a build has verified default.nix
as it stands, any exit restores the original. On handled paths this is a
no-op, since they have already restored or verified the file.
Write the build logs under ${TMPDIR:-/tmp} with per-run names. The
verification log had a fixed /tmp name shared by every run and every user.
Add the regression tests the #7498/#7499/#7529 fixes lacked:
- TestUpdateNixVendorHashReportsTheRealBuildResult runs the script with a
stub nix over a non-hash build failure, an unparseable mismatch, a failed
verification, a half-parsed got: hash, a real update, and a probe that
accepts the placeholder (restored file builds, or not). Every failure
exits 1 with default.nix restored byte for byte and mode for mode, the
operator's own default.nix.backup untouched and the named logs kept.
Against main's script it reproduces each false success.
- TestUpdateNixVendorHashInterruptRestoresOriginal signals the script's
process group during the probe build and during verification.
- TestUpdateVendorHashWorkflowBuildsIndependently pins the workflow's
unconditional `nix build .#default` between the script and the
match-report and push steps.
The scenario matrix, operator-backup and file-mode fixture, and workflow
ordering check are adapted from BlueboxC's correction of this PR in their
fork (62e7718e), as described on #7498 and #7529.
Refs #7504, #7496, #7498, #7499, #7500, #7529
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-authored-by: BlueboxC <david@blueboxcomputers.es>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: BlueboxC <david@blueboxcomputers.es>
* test(bazel): pin the item-5 one-wave rule for dolt-race and dolt-int budgets
Closed PR #7542 found that a union with more test actions than its
effective --jobs can regress wall clock by up to +575s, because Bazel
does not schedule the longest action first and a second-wave test adds
its full duration to the critical path instead of overlapping it.
Compute the effective --jobs from .bazelrc (remote-exec's baseline,
overridden by a --jobs line on the union's own config, last one
winning, matching Bazel's BEP cmdLine semantics from #7542) instead of
reading remote-exec's line alone, and explain the second-wave tail in
the budget tests' failure messages.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): clarify why the union's --jobs wins (comment only)
Command-line config outranks rc-file config; the BEP cmdLine ordering
PR #7542 observed is a consequence of that rank, not "last one wins"
by position, so reordering .bazelrc's lines cannot change precedence.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): shard http_served_test and corpus_served_test (embedded lane critical path)
Both served targets ran as one shard each (235-597s) and were the embedded
lane's longest action in most executing runs; every other embedded shard
ends by ~90-155s.
- http_served_test runs through go_test_pinned_shard.sh like
httpclient_served_test: the 19 TestE2E_* cases >= 3s are pinned onto 6
shards (pin_shards.py from three lane runs' test.xml plus one local run,
<= ~88 measured seconds each), the ~28 fast tests round-robin in a 7th.
- corpus_served_test: TestServedCorpus becomes TestServedCorpusPart1..4.
Shapes are dealt round robin in corpus order; each part starts its own
server and seeds its own pair of workspaces. The shapes are independent:
every check is served-vs-embedded parity after the same commands, each
write owns its target ids, and order preservation keeps each part's reads
ahead of its writes. TestCorpusPartsDealEveryShapeOnce checks the deal
(every shape once, in order, no empty part); the manifest pins part k to
shard k, with a 5th round-robin shard for the engine-free tests, and
scripts TestCorpusPartsArePinnedOnePerShard ties corpusParts, the
manifest and shard_count together. BEADS_CORPUS_REPORT now accumulates
every part the process ran.
- pinnedShardTargets and the retired-lane args pins list both targets;
go_test_sources_test gets the two packages' manifests as data.
Embedded goes from 172 to 182 test actions; the dolt-race/dolt-int union
budgets do not include the embedded tag and are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(corpus): seed the served and embedded sides concurrently in each part
Each TestServedCorpusPartN paid the server start and both seeds serially
(about 100-150s of a 190-220s shard in #7537's CI). The embedded side now
runs bd init and the seed on a goroutine while the test goroutine starts
the server and proxy and seeds the served side.
- prepareSide attaches one side (connect or init), writes its assets and
runs the seed in order, returning failures as errors (execBD, runBD
without the testing.T) so it is safe off the test goroutine; a failing
side cancels the other, and a deferred cancel+Wait keeps the goroutine
from outliving the test or its TempDirs.
- The sides share no directory, env or bd process; shapes, their order and
every assertion are unchanged. Each part logs its setup time.
- bazel.yml's embedded lane uploads the served targets' test.xml on every
run (bazel-embedded-served-testxml), so their pinned shards can be sized
from a sharded run of the lane.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): corpus in 3 parts; re-pin http_served_test from a sharded CI run
Measured on this PR's first embedded-lane run (test.xml from the new
bazel-embedded-served-testxml artifact):
- corpus_served_test with concurrent seeding: parts 1-4 ran 135/140/144/114s,
of which setup (part time minus its shapes) was 76/76/76/62s and the 127
shapes 242s together. The setup floor is still ~70s+, so corpusParts goes
from 4 to 3 (round robin; the measured split is 85/81/76s of shapes) and
the manifest and shard_count follow (4 shards: 3 parts + round-robin).
- http_served_test: #7537's pins were sized from single-shard runs, and the
sharded shards ran 73-184s against ~88s estimated. served_pinned_shards.txt
is regenerated with pin_shards.py from that sharded run's per-test
durations: 119/118/120/128/132/123 measured seconds per pinned shard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): pin http_served_test on the max over two sharded CI runs
The second run of the lane on this PR ran the same tests at 0.2-1x their
first-run times (executor load, not the tests), and its re-pinned shards
came out 36-118s against 118-132s estimated. pin_shards.py now takes each
test's maximum over both runs' test.xml, the same rule the other pinned
targets use: 123-135 measured seconds per pinned shard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(corpus): guard part wiring; seed metadata on e2e-t-ifa (review of #7537)
- testServedCorpus fails (before the engine gate, so in every lane) unless
the test's name is TestServedCorpusPart<part>: a TestServedCorpusPart3
calling testServedCorpus(t, 2) would run part 2 twice and silently drop
part 3's shapes. TestCorpusPartNameMustMatchItsPart covers the check and
that servedCorpusParts lists Part1..N in order. Mutation (Part3 -> 2)
fails //backend/http/corpus:corpus_test with "TestServedCorpusPart3 runs
corpus part 2; only TestServedCorpusPart2 may".
- e2e-t-ifa is seeded with metadata seeded=keep. The parts seed separately,
so the two refused (exit 13) conditional updates no longer always follow
gc-update-if-assignment's k=v; their verify show again holds that a
refusal keeps existing metadata. Mutation (served verify output's
seeded=keep rewritten) fails both stale shapes at
$[0].metadata.seeded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): back to 4 corpus parts (served long pole over test-seconds)
maintainer-city-c3's call on the measured trade-off. With concurrent
seeding, 4 parts ran 114-146s each (run A) and 3 parts ran 170-180s (run D).
3 parts save one ~75s setup, about 75 test-seconds per executing run, but
the longest part is the embedded lane's served long pole. That pole is on
the gate's critical path (forks' above all), and 4 parts cut it by ~34s.
corpusParts=4, TestServedCorpusPart4, manifest part 4 on shard 4,
shard_count=5. The part-name guard is unchanged and covers Part4.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): served-timings upload is telemetry; corpus part bounds and ctx
Review of #7538 (maintainer-city-c3):
- "Upload served test timings" gets continue-on-error: true and
timeout-minutes: 2, like the CI analytics upload. An artifact-service 5xx
or stall no longer fails the required embedded lane or holds it until the
job timeout. TestBazelWorkflowSecretsAndFailureSurface allows that
continue-on-error only on this exact step
(bazelServedTimingsContinueOnErrorPath, which also pins its name,
artifact, path and timeout; TestBazelEmbeddedServedTimingsUploadIsTelemetry).
- corpusPartNameErr also refuses a part outside 1..corpusParts, so a stray
TestServedCorpusPart5 fails instead of running no shapes. Two new cases.
- expand takes ctx: prepareSide's {rev:} lookups run under the seeding
context, so a cancel stops them as well.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(bazel): shard http_served_test and corpus_served_test (embedded lane critical path)
Both served targets ran as one shard each (235-597s) and were the embedded
lane's longest action in most executing runs; every other embedded shard
ends by ~90-155s.
- http_served_test runs through go_test_pinned_shard.sh like
httpclient_served_test: the 19 TestE2E_* cases >= 3s are pinned onto 6
shards (pin_shards.py from three lane runs' test.xml plus one local run,
<= ~88 measured seconds each), the ~28 fast tests round-robin in a 7th.
- corpus_served_test: TestServedCorpus becomes TestServedCorpusPart1..4.
Shapes are dealt round robin in corpus order; each part starts its own
server and seeds its own pair of workspaces. The shapes are independent:
every check is served-vs-embedded parity after the same commands, each
write owns its target ids, and order preservation keeps each part's reads
ahead of its writes. TestCorpusPartsDealEveryShapeOnce checks the deal
(every shape once, in order, no empty part); the manifest pins part k to
shard k, with a 5th round-robin shard for the engine-free tests, and
scripts TestCorpusPartsArePinnedOnePerShard ties corpusParts, the
manifest and shard_count together. BEADS_CORPUS_REPORT now accumulates
every part the process ran.
- pinnedShardTargets and the retired-lane args pins list both targets;
go_test_sources_test gets the two packages' manifests as data.
Embedded goes from 172 to 182 test actions; the dolt-race/dolt-int union
budgets do not include the embedded tag and are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(corpus): guard part wiring; seed metadata on e2e-t-ifa (review of #7537)
- testServedCorpus fails (before the engine gate, so in every lane) unless
the test's name is TestServedCorpusPart<part>: a TestServedCorpusPart3
calling testServedCorpus(t, 2) would run part 2 twice and silently drop
part 3's shapes. TestCorpusPartNameMustMatchItsPart covers the check and
that servedCorpusParts lists Part1..N in order. Mutation (Part3 -> 2)
fails //backend/http/corpus:corpus_test with "TestServedCorpusPart3 runs
corpus part 2; only TestServedCorpusPart2 may".
- e2e-t-ifa is seeded with metadata seeded=keep. The parts seed separately,
so the two refused (exit 13) conditional updates no longer always follow
gc-update-if-assignment's k=v; their verify show again holds that a
refusal keeps existing metadata. Mutation (served verify output's
seeded=keep rewritten) fails both stale shapes at
$[0].metadata.seeded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
PR #7535 review nits (bee-ghosttrack, julianknutsen):
- The -test.timeout=19m arg and BEADS_TEST_REQUIRE_EXCLUDE_PERMISSION env
var moved off --config=doltserver-cmd onto cmd/bd:bd_dolt_server_test and
tests/regression:regression_test directly. Nothing pinned either line
once it lived on the target instead of in .bazelrc, so TestBazelDoltIntCmdDoltTimeoutAndEnvPinned
now fails if either is dropped or altered on either target.
- classify_tier_results.py already fails closed on a BEP testResult whose
"status" field is omitted (proto3's NO_STATUS encoding), but no test
exercised that path through read_test_statuses()/classify() -- only
tier_result() with a pre-built status set. Add coverage for the real
shape (an event whose testResult body has no "status" key at all).
Verified against rv7535's mutate.py: E5, E6 and G4 now CAUGHT (previously
SURVIVED on main); the other 38 mutants are unaffected.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): dolt-int gates the server-storage and cmd-dolt tiers in mode remote (item 5 A2)
In mode remote, folds bazel-server-storage (--config=doltserver-integration)
and bazel-cmd-dolt (--config=doltserver-cmd) into one bazel-dolt-int job
(--config=dolt-int), following the dolt-race pattern from #7514. The
members have different gating categories (server-storage plain-gated,
cmd-dolt flag-gated/advisory), so the union carries continue-on-error: true
and a bespoke --label-scoped Shard balance step. All other modes keep the
two lanes running unchanged. Action keys are byte-identical between the
member configs and the union (17 + 46 = 63 TestRunner actions), so cached
results carry over either direction and a revert is a safe rollback.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): dolt-int per-tier results, integration gating, key-parity wording (review)
Coordinator review of #7535 found three must-fixes: integration:"off" was
ignored in remote mode (the union's members already honor it); job-level
continue-on-error made a server-storage-only failure read success on
push/nightly/dispatch, where main stays red; and the PR's key-parity claim
overstated byte-identical actionKeys when only the remote action digest
(not the local actionKey) is unaffected by moving --test_env/--test_arg
onto the targets' own env/args.
Fixes:
- bazel-dolt-int's if now adds `&& inputs.integration != 'off'`, matching
the members (#7491's review); bazelLaneRunModes and
TestBazelDoltIntArmsExactlyOnePerMode cover the "off" case.
- Job-level continue-on-error removed; only the `bazel test` step carries
it, and a new always() "Classify tier results" step
(tools/bazel/classify_tier_results.py, with its own unit tests and a Go
wrapper) computes server_storage_result/cmd_dolt_result from the BEP and
decides job.status per-tier. Confirmed with bazel-gate.sh/ci-gate.sh on
synthetic empty-output inputs that a dead job still fails the gate.
- Corrected the local-actionKey-vs-remote-digest wording in .bazelrc,
bazel.yml's job comment and CI_REQUIRED_CHECK_TOPOLOGY.md.
- Should-fix: test step 23->25 / job 58 minutes with reasoning; --label
coverage added to shard_budget_test.py; stale lane comment fixed in
tests/regression/BUILD.bazel:46; TestBazelDoltIntSoleRunIsInTheUnion's
docstring corrected (members bake sole-run's line in directly, never via
--config=sole-run).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): fail-closed classify, per-tier check_testcases, pinned argv (review)
Final review round for PR #7535 (maintainer-city-c3, bee-ghosttrack,
julianknutsen):
- classify_tier_results.py now fails every tier shut (exit 1) on a
non-PASSED label outside every --tier list, so a newly tagged target
added without updating the step's --tier args cannot silently read as
success (must-fix 1). Its --tier/--check label sets are policy-tied to
the BUILD files' own tag sets by a new
TestBazelDoltIntTierLabelsMatchTheTags.
- A missing or non-integer --rc (a timed-out or cancelled test step)
takes the same fail-everything-shut branch instead of crashing argparse
with no outputs written (bee-ghosttrack's should-fix 2).
- check_testcases.py gained a repeatable --label flag, and bazel.yml's
shared "Every target and shard ran tests" step is split into check-ss
and check-cd, each continue-on-error and folded into its own tier via
classify's new --check NAME=OUTCOME, so a zero-testcase cmd-dolt shard
no longer reddens the job on push/nightly (must-fix 2).
- classify's run: argv is now pinned by exact equality
(bazelDoltIntClassifyRun), and every flag except --tier/--check is
--StoreOnce; a repeated --tier/--check name or an unrecognized flag is
now a hard parser error (must-fix 3).
- "Upload test logs" now also fires on steps.test.outcome == 'failure',
not just failure() (must-fix 4).
- shard_budget.py --label now fails closed (::error, exit 1) on a label
with no executed shard in the BEP at all, instead of silently reporting
"no sharded test target executed two or more shards" (should-fix 3,
carried across both review rounds).
- Updated engdocs/TESTING.md, CI_REQUIRED_CHECK_TOPOLOGY.md, .bazelrc and
scripts/bazel_dolt_int_test.go/ci_workflow_test.go comments for the
nits: stale "Advisory (continue-on-error: true)" wording, the
bazelUnionHasFlagGatedMember comment's stale claim about bazel-dolt-int,
"byte-identical" without specifying remote action digest vs. local
actionKey, and "is unset" vs. ci-gate.sh's actual "was skipped" branch
for an empty union output.
Verified: python3 -I on all four tools/bazel/*_test.py (classify 29,
check_testcases 7, shard_budget 19) green; GOTOOLCHAIN=go1.26.7 go test
./scripts/... green; bazel --nohome_rc test //scripts:scripts_test
--config=ci green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): shard_budget --label fail-closed only on a truly unknown label
The previous commit's fail-closed --label check compared against
read_shard_durations, which only records executed (non-cached) shards, so
a target whose every shard was a cache hit this run had zero entries and
was wrongly treated the same as a --label typo. This was not caught by
the new tests (none exercised an all-cached target) and broke this PR's
own dolt-int CI run: bazel-dolt-int's Shard balance step went red on
//internal/storage/dolt:dolt_server_full_test with "no executed shard for
it", even though that target was correctly, fully cache-hit.
Fix: a new read_tested_labels() reads every label with any testResult
event (cached or executed), and --label's fail-closed check now tests
against that set instead of read_shard_durations. A fully-cached target
still legitimately reports "nothing to judge" through evaluate(); only a
label absent from the BEP altogether is now an error. Added the missing
test row (every shard cached) that would have caught this before push.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(bazel): pin bazel-dolt-int's Upload test logs if-condition
A full 42-mutation re-run for PR #7535's second review round found that
nothing pinned this job's own "Upload test logs" if-condition (must-fix 4):
the other lanes' upload-logs steps are each pinned, but bazel-dolt-int's
(unlike dolt-race, a brand-new job) had no equivalent assertion, so a future
accidental revert to the old failure()-only condition would go uncaught.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): dolt-race gates the dolt-server and proxied tiers in mode remote (item 5 A1b)
In mode remote, bazel-dolt-race (`bazel test //... --config=dolt-race`, the
union of --config=doltserver and --config=doltserver-proxied) runs both tiers
in place of bazel-doltserver and bazel-proxied, and is no longer advisory:
its job.status is both the bazel-doltserver and the bazel-proxied
workflow_call output (union result first, member's otherwise). In the fork
modes, cache and local, the two member jobs run exactly as before: there one
client does both tiers' work without in-flight deduplication, so a union
would cost the sum of the tiers rather than the max. Exactly one arm runs in
any mode, so gate ids, pr.yml, ci-gate.sh and bazel-gate.sh are unchanged.
- bazel.yml: dolt-race if: mode == 'remote' (every event that reaches it);
no continue-on-error; result output and always() recorder; test step
timeout 25 -> 20 (job 55 >= step sum 46 + 5, < 60). Members' if: their old
rule && mode != 'remote'.
- scripts/bazel_dolt_race_test.go (was bazel_dolt_race_shadow_test.go):
union-of-members config, sole-run lines in the union, gating shape, step
and job timeouts, exactly one arm per mode x event with the call outputs
evaluated as GitHub does, and the union's 62 test actions within
--config=remote-exec's --jobs=64.
- Gate simulations evaluate the workflow_call output expressions, and a
gated lane's reporter is the lane or its union lane.
- aquery: all 62 TestRunner actions under --config=dolt-race are
byte-identical (key, argv, env, outputs) to --config=doltserver (with and
without sole-run) and --config=doltserver-proxied on origin/main.
- Docs: CI_REQUIRED_CHECK_TOPOLOGY.md "dolt-race Union Lane", TESTING.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): dolt-race test step 23 min so Bazel logs a 1200 s TIMEOUT first
The proxied tier caps each test at 1200 s; a 20-minute step let GitHub
kill a hung shard before Bazel wrote its TIMEOUT test.log and goroutine
dump. Step 23, job 56 (step sum + 5, under the merge queue's 60).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): say the dolt-race job timeout bounds the job, not the run
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(backend/http/corpus): served command corpus against the reference server profile
Add the S23 suite: every distinct bd command shape gc and its packs and
formulas issue against a work store (124 shapes in testdata/corpus.yaml),
run by the real bd binary against an in-process bd serve and compared
with the same command on an embedded workspace.
The server is bound in non-loopback mode with bearer auth. A reference
proxy masks it to a deployed bd 1.3.0 server's wire surface
(testdata/reference_server.json): exactly that server's capabilities and
handshake members (no wire_revision), a 404 for operations it does not
publish, and a 400 unknown_parameter for query parameters and request
members it does not declare. Every bd run sets BEADS_ACTOR and
CLAUDE_SESSION_ID.
A shape passes when the served run exits like the embedded run, carries
no refusal, and matches its output (json/ids/count/text) and its verify
reads. Known gaps are marked `pending` with the slice that fixes them.
A pending shape still runs, and it fails once it passes, so removing the
mark is how a slice proves itself. Permanent refusals are listed in
testdata/allowlist.yaml with reasons.
A limited listing that cuts through a tie set (same priority, created_at in
one second) lands on different rows on two workspaces seeded seconds apart;
such a shape names its unlimited form in `prefix_of`, and each side's
limited rows must be an ordered prefix of that side's own unlimited listing.
//backend/http/corpus:corpus_test takes //cmd/bd:bd_for_tests (size large,
timeout long), so under bazel the suite runs instead of skipping on a
missing BEADS_TEST_BD_BINARY.
Stacked on S6b/c/d, whose marks this file no longer carries: besides the
S6b/S6c/S6d marks, seven list shapes once marked S6c+S15 (their --limit 0
list walks the keyset cursor without S15) and the S6c+S16 last-step close
(an ordinary epic root never auto-closes) already pass on it.
On S6b/c/d (bazel, BEADS_TEST_EMBEDDED_DOLT=1): 97 pass, 25 pending,
3 allowlisted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(backend/http/corpus): hand the corpus server the LeaseReclaimer role
main gained the LeaseReclaimer role (issues.reclaim) after S23 was written,
and httpapi.Config needs it when built from roles rather than a Provider, so on the restacked
S6bcd TestServedCorpus could not bind its in-process server ("no database
source"). The harness now passes s.LeaseReclaimer() with the rest, as
backend/http's e2e harness does. The reference proxy still masks the
issues.reclaim token, so gc-reclaim stays pending.
On the restacked S6bcd (bazel, BEADS_TEST_EMBEDDED_DOLT=1): 97 pass,
25 pending, 3 allowed, unchanged from before the restack.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(backend/http/corpus): run the corpus in CI, hold allowlisted shapes
Review fixes for #7483 (review attempt 1): both majors, and four of the
seven minors and nits.
No CI lane ran TestServedCorpus. corpus_test sets neither
BEADS_TEST_EMBEDDED_DOLT nor BEADS_HTTP_TEST_REQUIRED, so the suite
skipped everywhere. corpus_served_test, an embedded-tagged sh_test over
go_test_variant.sh like http_served_test, runs it with both set and with
Bazel's prebuilt bd, and pr_risk_bazel_coverage_test pins its args and
env. Every other file in the package builds only under cgo, so a new
cgo-free test fails a BEADS_HTTP_TEST_REQUIRED=1 run that would otherwise
pass having run nothing. README.md and engdocs/TESTING.md say which lane
runs the suite.
The allowlist branch returned before the embedded exit check, so an
allowlisted shape was never held to what it says on the embedded
backend. The embedded run is now checked against the shape's expect_exit
and expect_stderr before the allowlist branch, and an embedded mismatch
fails as a corpus bug whatever the shape's pending mark or allowlist
entry. TestCorpusFileIsWellFormed requires an allowlisted shape to state
expect_exit and to carry nothing the allowlist would skip (compare: exit,
no verify). The three allowlisted shapes now pin their embedded refusals,
and the bd sql reasons say the command needs a sql-server workspace
instead of calling it local-only.
Minors and nits taken:
- two stale-guard shapes (--if-assignee nobody, --if-status open) expect
exit 13 on both sides; each kills a client that drops that guard;
- defer_until and due_at are compared as given instead of scrubbed to
<ts>, which kills a client that shifts --defer;
- gc-reclaim's pending mark names SRV-1 alone: the reference profile
does not advertise issues.reclaim, so the client refuses at its
capability preflight;
- refusalMarkers matches "does not advertise capability" unquoted, which
also catches the JSON-escaped preflight error.
Declined:
- a failure on empty-vs-empty comparisons, which needs an allow_empty
opt-in. Review attempt 2 seeded rows for two of the three ready shapes
that compared two empty lists (below); gc-ready-graph-anchor still
does;
- the context route's query check in the reference proxy: the server
already answers an unknown query parameter there with a 400
unknown_parameter, which the proxy passes through, so the served run
still fails; only the proxy's own refusal note is missing;
- a shared server constructor for the two e2e harnesses, which the review
marks optional.
Review attempt 2: four of its six minors and nits.
- gc-ready-ephemeral-assignee-unlimited and gc-ready-assigned-tier
compared two empty lists, so a client that sent the wrong assignee or
dropped --include-ephemeral passed both. Two seed rows assigned to the
worker, an open task (P2) and an open wisp (P1), give them
[e2e-wisp-wr e2e-wr] and [e2e-wisp-wr]. The priorities differ, so
--limit=1 cuts no tie. A client that rewrites the assignee, or drops
include_ephemeral, now fails both shapes; without the rows it passed
them.
- templatePattern looped forever on a profile path with an unclosed "{",
growing its pattern without bound. It now returns an error, which
loadReferenceProfile reports with the operation and path, and
TestCorpusFileIsWellFormed loads the profile, so corpus_test catches a
bad edit without the embedded engine.
- TestCorpusFileIsWellFormed compiles every expect_stderr and every
allowlist refusal, and runShape compiles expect_stderr once and fails
the shape as a corpus bug instead of panicking on a bad pattern.
- README.md documents the stdin, setup, ordered and ignore_fields
fields.
Not in this commit:
- the PR description (shape counts, allowlist reasons, gc-reclaim's
SRV-1 mark, the Bazel run line) is brought up to date when this commit
is pushed;
- hoisting runBD, minimalBDEnv, buildBD and the skip gate into
internal/testutil, or cross-reference comments in their three copies,
which the review marks optional and which reaches outside this
package.
The full corpus gives 127 shapes: 99 pass, 25 pending, 3 allowed, with
no shape changing status from the attempt 1 fixup, on this tree and
merged with main at 0a163ca285.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(build): use native Darwin linker (wy-6fric6)
Keep pinned LLVM Clang and Linux toolchain inputs; pair Darwin SDKs with the native Apple linker to support macOS 27 TAPI targets.
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Steve Yegge
* fix(build): canonicalize LLVM attributes (wy-6fric6)
Apply bazel mod tidy ordering; linker/compiler settings are unchanged.
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of Steve Yegge
* test(dbproxy/server): disable dolt event flush so send-metrics cannot race TempDir cleanup
Every dolt sql-server this suite stops cleanly forks a detached
`dolt send-metrics` child on exit. It outlives the test, rewrites
<HOME>/.dolt/eventsData (HOME is a t.TempDir()), and races t.TempDir's
RemoveAll: "TempDir RemoveAll cleanup: ... 001/.dolt: directory not empty".
main.yml's Test (macOS) job (plain go test) failed on this in about half
of the pushes since 2026-10-09; Bazel never did because
tools/bazel/test_env.sh exports DOLT_DISABLE_EVENT_FLUSH=1. TestMain now
exports it too, guarded by TestSuiteDisablesDoltEventFlush.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(scripts): skip Linux-CI-only rrc and worker-env-sync script tests off Linux
TestRRCRunLanes runs .github/scripts/rrc-run-lanes.sh, which uses bash 4's
`mapfile` (exit 127 under macOS /bin/bash 3.2). TestWorkerEnvSyncCopyAgainstFixture
runs tools/rbe/worker-env-sync-copy, whose GNU `sed -i -E` becomes
"-i with backup suffix -E" plus a basic regex under BSD sed, so the pin is
never rewritten. Both scripts run only on Linux runners (bazel.yml
rrc-seed/rrc-verify, worker-env-autosync.yml); per their owner they stay
Linux-only and the tests skip elsewhere with the reason.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The host-tool-exec policy scanner (scripts/host_tool_exec_policy_test.go)
had two remaining soundness gaps flagged in gastownhall/beads#7491's
review:
- M19: a discarded PrebuiltBD()/findPrebuiltBDBinary() error (e.g.
`bd, _ := PrebuiltBD()`) kept the "never empty under Bazel" invariant
trusted for `bd`, even though a real lookup failure under Bazel also
returns "" there. prebuiltResultVars now only trusts the result when
the error is bound to a named variable that is itself checked with an
aborting `if err != nil`.
- M21: an abort-style gate (`if bazeltest.IsBazel() { return }`) counted
as gating an exec that ran before it, in source order, within the same
block. The abort-style check now requires the gate to dominate the
exec (ifStmt.End() <= pos); containment-style gating is unaffected
since nesting already guarantees order.
Flips the two tests that pinned these as documented limitations into
mutation fixtures that must fail, adds three new M19 fixtures (discarded
error, bound-but-unchecked error, and the fixed/checked shape), and
updates the package and gated() doc comments. TestNoUngatedHostToolExecs
UnderBazel confirms no real call site in the tree regresses.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci-policy): close review leftovers from #7485/#7486
Follow-ups from maintainer review on the host-tool exec/shell policy
tests and the bazel.yml integration gate:
- host_tool_shell_policy_test.go: recognize indented function headers
and `function name {` forms when finding a function's closing brace,
so scripts/test.sh's indented cleanup_shared_server() no longer
mis-attributes its trailing lines to the wrong function; classify
same-line `command -v gofmt && gofmt` (and `go`) as the guarded
presence-check idiom instead of flagging it.
- host_tool_exec_policy_test.go: make condPolarity reject mixed-polarity
&& / || compounds (one recognized side, one not) instead of guessing
at the recognized side's polarity, which was unsound in both
directions; add pinning tests documenting the exec-before-gate and
function-value-helper cases as known, safety-conservative limitations
rather than bugs. Fixing this exposed a real ungated fallback in
cmd/bd/scripttest_test.go (a `go build` that could still run under
Bazel on a prebuilt-binary basename mismatch); fixed by symlinking the
prebuilt binary under the expected name instead of rebuilding it.
- bazel.yml: bazel-server-storage now honors the `integration` input
like bazel-integration/bazel-cmd-dolt do, so an explicit "off" skips
it too (latent today: no caller passes "off"). Updated
ci_workflow_test.go's policy tests to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci-policy): close should-fix B (go command -v guard evasion) on #7491
shellCommandVGuard previously excused go the same as gofmt, but under an
exit-127 host-tool stub `command -v` finds the stub and the guarded exec
still runs, and on a worker genuinely lacking go the exec was silently
skipped -- the worker-dependent behavior this policy exists to ban. Drop
go from the guard entirely, require the guarded exec to sit directly after
the guard's own `&&` (no intervening `;`, `||`, `|`, or leading `!`), and
require an exact gofmt match so a same-prefixed sibling like `gofmt-foo`/
`go-junit-report` can't borrow the guard. Add every should-fix B shape as
a fixture that must still be flagged, plus the generalized gofmt-side
cases (leading `!`, exact-name, post-`;` exec).
Also: correct host_tool_exec_policy_test.go's doc claim that a
function-value helper call site is "not a soundness hole" -- mixed with
one gated direct call site, the invisible function-value call site's
ungated-ness is silently lost and the combined verdict flips to gated;
pinned as review M22. And recognize a function closed by `} >&2` as
ended, so abort-style safety computed inside it does not leak into the
rest of the file (no scoped script has this shape today, but a bare `}`
equality check would silently miss it the moment one does).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* test: ban ungated host go/gofmt execs under Bazel; pin compact_gc_fixture pure/norace
Follow-up to #7478. Adds two policy tests on //scripts:go_sources_test:
- host_tool_exec_policy_test.go: an AST scan of every *_test.go file and
internal/testutil's non-test .go files for exec.Command/CommandContext/
LookPath("go"/"gofmt") that isn't gated by bazeltest.IsBazel()/TEST_SRCDIR
in its enclosing function, with an explicit, justified allowlist for the
handful of sites gated by caller dataflow instead (bazeltest.PrebuiltBD()
checked before calling a separate helper). "dolt" is out of scope: the
hermetic copy resolves first on Bazel's PATH, unlike go/gofmt.
- host_tool_shell_policy_test.go: the same policy for the shell scripts
under scripts/ that Bazel sh_test/go_test targets actually execute
(fmt-check.sh, the repochecks guards, gofmt-bin.sh, the migration-test
harness), matching the gating pattern #7478 added to fmt-check.sh and
scripts/ci/lib/test-env.sh.
Both are mutation-verified (an added ungated exec.Command("go","version")/
`go build` is detected, then reverted cleanly).
Also sets pure = "on" and race = "off" on cmd/bd/BUILD.bazel's
compact_gc_fixture go_binary, matching :bd_for_tests/:cmd_norace: it is a
testonly, stdlib-only fixture, so neither cgo nor the race detector buys it
anything, under any --config. Verified TestRunCompactDoltTargetsOnlyAuthorizedActiveDatabase
still passes via `bazel --nohome_rc test` under both --config=prcore and
--config=doltserver-cmd.
Known pre-existing issue, out of scope here: gosec's GoVersion() call still
execs a host `go` from third-party code this policy doesn't reach.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(scripts): address review of host-tool exec/shell policy (#7485)
Fixes all 6 maintainer-city-c3 review items against the ban-ungated-go/
gofmt-exec policy, each with a mutation test proving the fix catches
the described regression:
1. Shell scan now gates per-branch/per-call-site (shellLineGating),
not by any TEST_SRCDIR mention anywhere in the whole file.
2. Shell scanner adds gofmt and `go tool` detection, fixes the
heredoc-start regex (no longer swallows the rest of the script on an
arithmetic `<<` shift or a `<<<` here-string), and replaces the
blanket printf/echo/git-grep line skip with a quoted-span/command-
substitution-aware check that still catches a real exec joined onto
a message line with `&&`.
3. shellPolicyScope gains pr-lint.sh, check-versions.sh,
upgrade-smoke-test.sh and the 4 .github/scripts shard fallbacks,
each cited against its real BUILD.bazel/.bzl wiring; corrects two
factually wrong "not executed" doc-comment claims (pr-lint.sh and
upgrade-smoke-test.sh ARE Bazel-executed); and adds
TestShellPolicyScopeCoversBazelShellData, a mechanical backstop that
fails on a Bazel-referenced .sh under scripts/ or .github/scripts/
missing from scope.
4. hostToolExecAllowlist no longer exempts goBuildBDCommand's 9 call
sites by helper name; helperIsGatedByAllCallers checks every real
call site is itself gated.
5. Both allowlists are keyed by file + enclosing function + exact
snippet, not line number.
6. gated()/shellLineGating understand condition polarity, so an
inverted `if !IsBazel() { t.Skip() }` (or its shell equivalent) is
correctly NOT counted as a gate.
Also extends the Go scanner's condPolarity to recognize
findPrebuiltBDBinary()/bazeltest.PrebuiltBD() result variables as a
TEST_SRCDIR-equivalent gate (proven safe by reading their
implementation), and the shell scanner's analogous shellBazelSetVars
for CANDIDATE_BIN/SOURCE_TAG_SQLITE_BIN/BEADS_TEST_*_BINARY, including
through a script's own default-value variable alias
(`VAR="${BAZEL_VAR:-default}"`).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(scripts): give go_sources_test runfiles access to the new shell scope
The shell-policy rewrite (host_tool_shell_policy_test.go) added
shellPolicyScope entries for scripts/ci/pr-lint.sh,
scripts/check-versions.sh, scripts/upgrade-smoke-test.sh, and the four
.github/scripts/*-test-shard.sh files, plus a scope-coverage guard
(TestShellPolicyScopeCoversBazelShellData) that walks every tracked
BUILD.bazel file. None of those were runfiles of
//scripts:go_sources_test, so bazel test passed locally under plain
`go test` but failed in CI's hermetic Bazel sandbox with "no such file
or directory" for the new scope entries and "found only 0 BUILD.bazel
files" for the guard's walk.
Add //:repo_other_files (every tracked non-Go, non-Markdown file,
including every BUILD/.bzl file) to go_sources_test's data, mirroring
the same pattern already used by //scripts:scripts_test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(scripts): rename AST-fixture function so it isn't counted as a real test
tools/bazel/equivalence.py compares `go test`'s discovered top-level
tests against what Bazel actually ran, using a raw-text regex
(`^func\s+(Test\w*)\s*\(\s*\w+\s+\*testing\.T\s*\)`) rather than
compiling the file. The "inverted IsBazel skip" fixture in
host_tool_exec_policy_test.go embeds a column-0
`func TestSomething(t *testing.T) {` inside a backtick string used to
build an AST for findHostToolExecCalls, which the regex scan counted
as a real top-level test that Bazel's go_sources_test then "failed to
run" (CI's Bazel / test leg: EQUIVALENCE: FAIL on "scripts
TestSomething"). Rename the fixture's inner function to
bazelOnlyFixture; the test's behavior is unchanged since the name was
never semantically significant to the scan being exercised.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(scripts): close host-go-policy gaps from PR #7485 re-review
Four fixes from the final review round:
1. test.sh/conformance.sh/release.sh ARE in scope (they run under Bazel
via scripts/** data globs); drop the false "not in scope" claims and
add justified fn-keyed allowlist entries for their 8 flagged execs.
2. Go AST: an exec inside the Bazel-true branch, before its own abort,
is no longer wrongly counted as gated (M17); a gate inside a sibling
closure no longer covers execs outside it.
3. Shell scanner: abort/skip gate state now resets at function-scope end
(a top-level `go build` after build_candidate in
upgrade-smoke-test.sh is correctly flagged), and `bash -c 'go build'`
is detected again.
4. Split host_tool_shell_policy_test.go into its own go_test
(shell_host_tool_policy_test) with narrow, curated data instead of
riding go_sources_test's //:repo_other_files; restored
go_sources_test's data to its pre-PR scope. Dropped
TestShellPolicyScopeCoversBazelShellData: its regex backstop could
not see Starlark labels/.bzl/globs and still missed real in-scope
scripts, so it wasn't earning its cache-rerun cost. The new target
carries no tags, so it still runs in the required "Bazel / test"
lane.
Each of 1-3 ships with a mutation-proof fixture that fails on the prior
code and passes on the fix.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(scripts): rename sibling-closure fixture to avoid false test-equivalence hit
tools/bazel/equivalence.py line-scans *_test.go files for a column-0
`func Test\w*(t *testing.T)` to build its "what go test runs" list; it
reads raw text, not compiled Go, so the fixture's embedded raw-string
Go source (used to feed findHostToolExecCalls a parseable AST) was
mistaken for a real top-level test the package defines, and the
equivalence check failed because no go_test target "ran" it. Rename
the fixture function from TestX to siblingClosureFixture, matching the
existing convention a few lines down (bazelOnlyFixture) that documents
the same pitfall.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci: F5 S2 Variant A nogo ownership for the integration configuration group
No full Bazel lane validates the integration / doltserver-integration /
doltserver-cmd configurations anymore (all three now pass
--norun_validations). Instead bazel-integration gains a narrow "Nogo:
integration library files" step that keeps full nogo (vet + golangci-lint)
on the configuration's one non-test library package,
//internal/testutil/integration, via
`bazel build --config=integration --@rules_go//go/config:race --keep_going
--output_groups=nogo_fix -- //internal/testutil/integration:integration`.
This accepts a scoped coverage loss (f5-spec.md §3/§5 S2, Variant A): the 56
integration-tagged _test.go files (listed in
tools/nogo/unvalidated_integration_tests.txt, derived from source by the new
T4 test) no longer get a vet pass in any lane, since golangci-lint already
skips all _test.go files repo-wide (.golangci.yml's run.tests: false) and no
lane builds those test binaries with validations on. Verified directly with
real `bazel build` runs: a planted vet finding in the library package still
fails the new step; the same finding in an integration-tagged _test.go file
no longer fails anything; a planted finding in a plain race-configuration
library still fails bazel-test; and GoCompilePkg action keys for the
integration configuration are identical with and without
--norun_validations (it is a build-request option, not a configuration
flag, so no farm cache invalidation).
scripts/nogo_lint_policy_test.go: the nogoConfigurations table gains a
"integ" row with an ownerIsNogoOnlyStep flag and a dedicated
checkNogoOnlyStepRow, generalizing T1/T2/T5/T6 to cover an owner that is a
narrow build step rather than a full lane; adds T4
(TestNogoIntegrationStepCoversItsConfiguration), which derives the step's
expected target set from source via go/build's own tag matching instead of
a hardcoded list.
Also fixes four should-fix items from julianknutsen's review of #7482:
1. expandBazelrcConfig now follows `common:` lines and --config= tokens
anywhere on a line, not just a first-position `build:`/`test:` one.
2. TestNogoRaceOwnerBuildsEverything now expands every config the owner job
can reach (including shell-variable-gated ones like sole-run), not just
the statically-named "ci".
3. The package-gate check now compares full key-flag fingerprints instead
of substring-matching --@rules_go//go/config:race / --norun_validations.
4. tools/bazel/nogo_ownership.py now fails loudly (exit 2) when both the
owner's and non-owners' action-key sets are empty, instead of reporting
a vacuous pass.
Each fix has new mutation-test coverage in TestNogoPolicyRejectsMutations
(should-fixes 1-3) or nogo_ownership_test.py (should-fix 4).
engdocs/LINTING.md and engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md updated to
remove the stale "integration still validates in all three lanes" claim and
document the Variant A trade-off.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci): make T4's source scan work under Bazel, not just plain go test
scripts:scripts_test only declares //:repo_other_files as data (every
tracked file but Go and Markdown source, by design: a Go-only change
should not re-run the CI/workflow policy tests). TestNogoIntegrationStep
CoversItsConfiguration's source scan needs every non-test .go file, so
under `bazel test //scripts:scripts_test` its runfiles tree had none and
the scan always came back empty.
Fixes:
- scripts/BUILD.bazel: add //:repo_go_srcs to scripts_test's data, with a
comment explaining why (this is the one test in the target that needs it).
- integrationOnlyLibraryPackages: use bazelPolicyRoot(t) + repoFiles(t, ...)
(the existing Bazel-safe, runfiles-aware file listing other policy tests
in this package already use) instead of sourceRepoRoot(t) +
filepath.WalkDir, which only sees what //scripts's own go_test target
declares.
Verified with real `bazel test --config=integration --@rules_go//go/config:race`
and `--config=ci --@rules_go//go/config:race` runs of //scripts:scripts_test
(both pass); CI caught the original gap (PR #7486's first push failed
Bazel / integration and Bazel / test with exactly this).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): address independent review + bee-ghosttrack findings on F5 S2 Variant A
Independent review of #7486, must-fixes 1-7: pipefail on the nogo-only
step's tee pipeline; exact source-scan equality check for
tools/nogo/unvalidated_integration_tests.txt; expandBazelrcConfig
follows common:-lines and non-first-position --config= tokens in both
the Go policy test and nogo_ownership.py; mirrored
nogo_ownership.py should-fix guarding against a non-owner action-key
query silently matching nothing.
Cost fix: split T4/T6's whole-repo .go source scan into its own
go_test target (//scripts:nogo_integration_scan_test) so a Go-only
change no longer re-runs all of scripts_test; extracted the shared
bazelInvocations and ciWorkflow helpers into helper-only files so
both targets can depend on them without violating the
one-target-per-test-file partition rule.
bee-ghosttrack review: checkNogoOnlyStepRow's own fingerprint now
excludes test:-sourced .bazelrc lines pulled in via --config=X
(buildOrCommonLines), since the nogo-only step is a `bazel build`
invocation that never reads a test: line; added mutation s2i
reproducing the gap this closes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(nogo): pin integration nogo-only step's target, fingerprint its lane, add behavioral test
Addresses the remaining should-fixes from the independent review of #7486:
1. TestNogoIntegrationStepScriptFailsWithoutMaskingBazelsExitCode
(scripts/nogo_integration_step_script_test.go) extracts the real
"Nogo: integration library files" step script from bazel.yml and runs it
under plain `bash -e` (GitHub's actual run: semantics, no pipefail added)
against a stub bazel, closing the gap a text-only pin can't: it catches
rc=0 before `exit "$rc"`, `|| :`/`|| true` on the tee pipeline, or a
missing/negated `set -o pipefail`, regardless of how the mutation is
spelled.
2. The nogo-only step's target set is now pinned by exact go_library
label, not just package: nogo_integration_scan_test.go derives the
label(s) from source + BUILD.bazel and nogo_lint_policy_test.go pins
them literally (its own go_test target has no //:repo_go_srcs). Both
reject a `-//` exclusion pattern outright.
3. checkNonOwnerCommandLineAddsNoKeyFlag is now also applied to the
integ row's owner's own full lane: under Variant A that lane validates
nothing, so it is effectively a non-owner for this check's purpose, and
must not override a key-affecting flag on its command line either.
4. Latent items: TestNoUnconditionalValidationOff now also scans
.github/actions/setup-bazel/write-bazelrc.sh for run_validations (every
line it emits lacks a :config suffix, so any occurrence would turn
validation off everywhere with no owner lane to catch it).
tools/bazel/nogo_ownership.py's docstring is corrected: it had said S2
was simply "out of scope," which is now stale since S2 has its own
validating lane (the nogo-only step, checked statically); the script's
dynamic aquery-based cross-check remains S1-only because S2's owner has
no non-owner test lane building the same configuration to compare
actions against.
Skipped (not cheap / not reachable):
- bazel-server-storage's `if:` not modeling an "integration" input: no such
input exists to model; the gate is RBE-enabled-only by design, and
resolving this requires understanding pr.yml's broader gating semantics,
not a textual fix.
- defaults.run.shell parsing: bazel.yml has no `defaults:` block today
(confirmed by grep), so there is nothing to parse; the gap is latent only
for a future change, not a live bug.
- race-off scan's future-wrong-fix edge case: the review itself
characterizes this as not reachable with today's file set.
New mutation cases s2j/s2k/s2l in TestNogoPolicyRejectsMutations cover
items 2 and 3. All mutation-verifications performed by hand-editing the
real files, confirming FAIL, and reverting via Edit (never git
stash/checkout).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* test(nogo): reject F5 S1 ownership mutations the policy tests missed
Follow-up to #7479's independent review.
- nogoKeyFlags parses whole tokens instead of a prefix regex, so
--@rules_go//go/config:race=false on a non-owner no longer fingerprints
as the owner's race build (1f).
- The owner's run_validations check expands every --config its bazel
commands select (including BAZEL_SOLE_RUN/BAZEL_FRESH) recursively, so
e.g. `test:sole-run --config=doltserver` is caught (1e).
- T3 scans every argument of the owner's `bazel test`, including
backslash-continued lines, for negative target patterns (1g).
- T1 pairs each non-owner job with the --config its own bazel command
selects instead of by list index (2).
- TestNogoPolicyRejectsMutations feeds T1/T3/T5/T6 each mutation 1a-1g
and 2 and requires a rejection.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(bazel): nogo_ownership.py compares nogo actions by action key
Owner and non-owner nogo coverage was compared by target label, which
treats one label compiled in two configurations as covered. Compare
RunNogo and ValidateNogo actions by Bazel action key instead. Non-owner
universes drop manual-tagged tests (bazel test //... never builds them)
and add //cmd/bd:bd_for_tests, the package gates' target. The
fingerprint mirrors nogoKeyFlags' whole-token parsing.
nogo_ownership_test.py covers this with a fake bazel.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): correct nogo lane-sharing and one-required-lane claims
- CI_REQUIRED_CHECK_TOPOLOGY.md: only the nogoConfigurations rows
(race, pure, js/wasm) validate in exactly one required lane; the
integration configuration still validates in its three required lanes
until F5 S2.
- .bazelrc: --norun_validations lanes share compile, link and nogo
actions with `test`, not test PASS results (each lane's test actions
have their own env and args). Comment only.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(bazel): remove host-go dep from compact GC test; drop frozen shard blocks
Replace the ungated `go build` in compact_gc_target_test.go's dolt-stand-in
fixture with a Bazel-built go_binary (cmd/bd:compact_gc_fixture) wired via
rlocationpath env + data, mirroring the existing preflight_go_fixture
pattern. Falls back to `go build` only when not running under Bazel
(bazeltest.IsBazel()). Also gate test_repo_beads_guard_test.go's `go env
GOCACHE`/`GOMODCACHE` probes on !bazeltest.IsBazel(), since they previously
ran unconditionally in every cmd/bd TestMain.
Separately, delete the legacy 20-shard embedded-cmd and 15-shard
proxied-cmd "frozen" manifest blocks in .github/scripts/*-test-shards.txt:
their only consumers were pr-risk.yml fork/push jobs that have since
migrated to bazel.yml. Update gen_embedded_cmd_shard_manifest.py and
gen_proxied_shard_manifest.py to drop the frozen-block rendering/guard
logic and default to the live Bazel-only shard counts (100 and 34),
and refresh stale comments/docs (cmd/bd/BUILD.bazel,
scripts/shard_manifest_test.go, engdocs/TESTING.md,
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md) that still described the deleted
blocks. Verified no other BUILD/.github/scripts references read them.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/38069936930
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
* fix(bazel): gate remaining host-go calls; fix shard-manifest --weights footgun
Address PR #7478 review findings (bee-ghosttrack):
- scripts/ci/lib/test-env.sh: skip the GOCACHE/GOMODCACHE `go env` probes
when TEST_SRCDIR is set, so scripts/test.sh's beads_test_env_enter never
shells out to a bare `go` when reached from a Bazel-run go_test (e.g.
//scripts:shell_scripts_test's TestTestScriptPrebuiltTestBinaryContract),
where a farm masks host go/dolt to exit 127. Nothing on that path needs a
cache directory.
- scripts/ci/fmt-check.sh: describe_gofmt() no longer calls a PATH `go`
under Bazel. It now resolves BEADS_TEST_GO (the hermetic SDK go already
wired into scripts/prlintmake:prlintmake_test's env) via TEST_SRCDIR, and
skips the version-reporting arm entirely when that isn't set (e.g.
scripts/repochecks:fmt_test), rather than ever touching a possibly-masked
PATH `go`.
- Verified with a PATH stub (go/dolt scripts that log and exit 127) via
`bazel --nohome_rc test --config=prcore //scripts/... //cmd/bd:bd_test
--test_env=PATH=<stubdir>:/usr/bin:/bin:/usr/local/bin`: 26/26 targets
pass, zero stub invocations logged.
- gen_proxied_shard_manifest.py / gen_embedded_cmd_shard_manifest.py:
default --weights to duration now that the manifests' only remaining
blocks are duration-packed (the legacy inits-packed blocks were deleted
alongside the PR Risk fork/push jobs that read them). Previously the bare
`--write` default (inits) silently clobbered the live block's duration
header and repacked with the wrong cost model; --check couldn't see it.
Added TestProxiedShardManifestBareWriteIsIdempotent and
TestEmbeddedCmdShardManifestBareWriteIsIdempotent to pin a bare --write
as a no-op against the committed files.
- cmd/bd/compact_gc_target_test.go: resolve the non-Bazel fallback fixture
path from this file's own directory (runtime.Caller) instead of the test
binary's working directory.
- Refresh stale comments in .github/workflows/bazel.yml,
scripts/shard_manifest_test.go, scripts/pr_risk_bazel_coverage_test.go,
and scripts/embedded_shard_count_test.go that still described the deleted
legacy 20/15-shard blocks and their retired fork/push jobs in the present
tense.
- Confirmed cmd/bd/BUILD.bazel's compact_gc_fixture go_binary (implicit
pure/race defaults) still passes TestRunCompactDoltTargetsOnlyAuthorizedActiveDatabase
under `bazel test //cmd/bd:bd_test`.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(bazel): one nogo-validating lane per race configuration (F5 S1)
The race configuration (--@rules_go//go/config:race) is built by five
lanes that duplicate the same RunNogo/ValidateNogo work: bazel-test
(--config=ci), bazel-embedded, bazel-doltserver, bazel-proxied, and
the advisory bazel-dolt-race shadow, plus the package-mcp/package-npm
gates' explicit race build. --norun_validations is a build-request
option, not a configuration flag (proven here by aquery: GoCompilePkg/
GoLink/RunNogo/ValidateNogo action keys are byte-identical with and
without it), so passing it on every lane but bazel-test removes the
duplicate work without touching a single action key or dropping any
lint coverage: bazel-test's --config=ci already validates every file
any of the others compiles.
scripts/nogo_lint_policy_test.go replaces the old blanket
"--norun_validations never appears in .bazelrc" check with
TestNogoConfigurationsHaveOneValidatingLane (T1: one owner per
configuration fingerprint), TestNogoOwnersAreRequired (T2: the owner
is a required CI Gate check whose `if:` covers every non-owner's),
TestNogoRaceOwnerBuildsEverything (T3: the owner builds //... whole,
no exclusions) and TestNoUnconditionalValidationOff (T5: nothing else
may disable validation). tools/bazel/nogo_ownership.py (make
nogo-ownership) is a dynamic, advisory aquery cross-check of the same
property, run on demand rather than from inside a Bazel action.
Verified locally: aquery shows identical action keys for the affected
lanes with/without --norun_validations; a planted printf-format vet
finding in internal/idgen/hash.go failed the owner build and passed
the non-owner build (then reverted, never committed);
scripts/bazel_dolt_race_shadow_test.go and the full //scripts suite
stay green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): close S1 nogo-ownership review gaps (allowlist, owner pin, real aquery)
Addresses maintainer-city-c3's review of #7479:
1. T6 TestRunValidationsOnlyOnAllowlistedLines replaces the coverage gap
left when main's blanket run_validations ban was dropped: only the four
race-group non-owners' "test:<cfg> --norun_validations" lines in
.bazelrc, and the two package-gate bazel-build steps in bazel.yml, are
permitted. A later test:integration/doltserver-cmd --norun_validations
(out of T1's table, S2) now fails here instead of landing silently.
Verified by mutation: adding test:integration --norun_validations fails
the new test; reverting passes.
3. TestNogoRaceOwnerBuildsEverything (T3) now also pins that the race
owner's "bazel test //... --config=ci" step has no step-level if: or
continue-on-error, and that its job has none either. Verified by
mutation: adding if: ${{ always() }} to that step fails; reverting
passes.
2. tools/bazel/nogo_ownership.py's aquery comparison was querying the
identical mnemonic("RunNogo", //...) for the owner and every non-owner,
so `missing` could never be non-empty. It now derives each non-owner's
actual built target set the way --build_tests_only + --test_tag_filters
does: the deps closure of tests(//...) filtered to targets carrying one
of the lane's tags, read exactly from `query --output=xml` (not attr()'s
substring regex match, which would wrongly fold "dolt-server-proxied"
and "dolt-server-cmd" into a "dolt-server" filter). Confirmed against a
real (offline, --nohome_rc) bazel run: reports OK at head, and reports
100+ missing targets when the owner's query is artificially narrowed to
a subpackage -- it can now actually fail.
Also: refreshed a stale .bazelrc comment claiming nogo runs beside every
lane's compile, which S1 made untrue for race-group non-owners.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): mode cache reads the remote repo contents cache through rbe-cache
Mode cache (fork and Dependabot PRs while rbe-fork is closed, rbe=cache
dispatches) now reads Bazel's remote repo contents cache: the trees rrc-seed
writes on main pushes, served by rbe-cache's anonymous GetActionResult and
ByteStream.Read (Caddy :8443 -> nativelink-anon -> AC_OSS, read only).
setup-bazel's write-bazelrc.sh adds the trusted readers' two lines to its
fork-cache rc (passed with --bazelrc, so the startup line applies):
startup --experimental_remote_repo_contents_cache
common --loading_phase_threads=64
The committed .bazelrc stays untouched (design R2): the flag is a startup
option and cannot live in build:fork-cache, and developer and macOS
machines never read.
Gated on the new .github/actions/setup-bazel/cache-rrc-probe.sh (a copy of
gascity's tools/rbe/cache-rrc-probe.sh): one GetCapabilities to rbe-cache
(connect 3s, total 5s). Only an answer (gRPC 0, cache_capabilities present)
writes the lines; anything else writes none and logs a ::notice:: with the
reason, and the script never fails on it. Kill switch: fork_rrc_read=on in
that script; set it to off in a one-line PR to stop mode cache reading.
Measured (scratch origin/main, no credentials, 8 CPUs, cold output base,
`build --nobuild //...`): repository rules 665-672 -> 38-41 (the rest are
bd_releases/dolt_release and local or toolchain repos, not reproducible),
analysis 26.5-26.7s -> 12.1-14.8s, wall about 36s -> 25-32s.
Tests: TestSetupBazelRCWriter runs the writer against a stand-in rbe-cache
(answers, answers with zstd, PERMISSION_DENIED, refused): exactly the
reader's lines after --config=fork-cache and any zstd line only when it
answers, a notice otherwise; remote-exec and rbe-fork never get them nor
probe. TestBazelCacheModeReachesTheRC checks no simulated mode carries them
with the probe refused. TestCacheRRCProbe runs the real probe (errors,
malformed, timeout, refused, untrusted cert, kill switch off without a
request).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): mode cache rrc review fixes: reader opt-in, containment, AC probe
Scope. write-bazelrc.sh enabled the remote repo contents cache for every
mode cache job, bazel-release-cross included, whose configs
rrc-lane-commands.txt does not seed. setup-bazel now takes BAZEL_RRC_READ
("true" or empty, anything else fails): only with it does mode cache probe
rbe-cache and add the reader's lines. The 8 trusted reader jobs pass it,
and nothing else. TestBazelRRCOptInMirrorsTheTrustedReaders pins that set
to bazelRRCReadLanes; TestBazelRRCLaneCommandsCoverEveryReader now checks
every job that enables the flag either way (reader step or opt-in) is a
reading lane whose bazel configs are all seeded. TestSetupBazelRCWriter
covers a job without the opt-in (no probe, no lines) and bad values.
Runner. The 5 readers that run in mode cache (test, pure, integration,
doltserver, cmd-dolt) fall back to ubuntu-24.04 instead of ubuntu-latest:
one runs-on line each; mode remote stays Blacksmith, and the remote-only
readers and every other job are unchanged.
Visibility. write-bazelrc.sh writes one step-summary line in mode cache:
whether the job reads, and why (the probe's reason, with rbe-cache's
GetCapabilities TCP connect time, DNS excluded).
Probe. cache-rrc-probe.sh is re-copied from gascity: it also asks
GetActionResult for a fixed key no action has and a ByteStream Read of
that digest as a blob, and passes only if both answer NOT_FOUND (gRPC 5),
proving Caddy, nativelink-anon, the action cache and the CAS answer; it
reports the TCP connect time minus the DNS lookup. serveCapabilities
answers both; TestCacheRRCProbe adds their failure answers.
Containment and limits. The rrc-verify issue body and
CI_REQUIRED_CHECK_TOPOLOGY.md no longer say to set RBE_REPO_CONTENTS_CACHE
off (mode cache never sees it, and off also stops rrc-verify): commit
fork_rrc_read=off in both probe scripts (beads has no fresh-merge, so an
open fork PR keeps its old switch until it is pushed again); for immediate
server-side containment, rbe-cache-gate close on core; and, because the
mode remote readers read the same oss action cache entries, set the
variable to seed on a mismatch (no lane reads, rrc-seed and
rrc-verify keep running), off only to stop seeding too, and leave it as
it is when the run only could not check. The doc also says a
closed or down cache only falls back at probe time (after loading, files
are fetched lazily and losing rbe-cache fails the command, bazel#30218),
and notes the 512 MiB anonymous CAS limit under gascity's R4.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): bazel-dolt-race opts in to the rrc read like the other trusted readers
#7464 added bazel-dolt-race as a trusted reader after this branch was cut, so
TestBazelRRCOptInMirrorsTheTrustedReaders now covers it: pass BAZEL_RRC_READ
(it runs only in mode remote, so this changes nothing today). It never runs in
mode cache, so it keeps the lanes' ubuntu-latest fallback: the cache-reader
runner rule (ubuntu-24.04) excludes it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#7374 re-vendored gascity's signed-duration fix for exec-log spawns, but
build_tests() still kept the sign of BEP testResult timings. rbe-west
scheduler/worker clock skew makes a remotely executed test report
queueTime -1 ms (-24 ms at worst), and the collector rejected the whole
artifact on tests[].results[].queue_ms: about 19% of beads artifacts,
holding 52% of beads' CI worker time.
Byte copy of gascity 2ddde0d94be15d95086cc2bcb169303ea1243456: clamp
duration_ms and every timingBreakdown child with _clamp_duration_ms.
Updates the two sha256 pins; testdata/ci_analytics is unchanged.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run every BEADS_TEST_PROXIED_SERVER-gated cmd/bd test in the proxied lane
proxied-test-shard.sh selects the proxied lane's tests by name prefix
(TestProxiedServer*/TestServerMode*), and BEADS_TEST_PROXIED_SERVER=1 is set
only by //cmd/bd:bd_proxied_test. Seven cmd/bd tests skip without that
variable but are named outside the prefix, so they ran in no CI lane:
TestProxiedDeleteIfRevisionSingleWinner
TestProxiedIfRevision{CloseMismatchRefuses,CloseReplaysMoleculeAutoClose,
CloseWarnsOnForcedOpenChildren,DeleteMatchAndMismatch,
OutranksReassignFence}
TestHistoryRemoteSupportedDirectAndProxiedParity
Rename them into the TestProxiedServer* prefix (the repo's existing
convention, see if_revision_proxied_integration_test.go) and place them
incrementally in the 34-shard manifest block. The prefix stays the
selection rather than widening to TestProxied*, which would pull ~33
ungated unit tests into the lane as duplicates.
Running them exposed real rot: the history parity test had been broken
since #7436 scoped shared-server projects to a per-database user, because
its `bd dolt remote remove` needs DOLT_REMOTE. It now uses
newSharedProxiedRootProject, like the other remote-removing tests #7436
moved.
Guard: TestProxiedGatedTestsRunInTheProxiedLane (//scripts:
go_test_sources_test) walks cmd/bd's test call graph for any top-level test
that reaches an `os.Getenv("BEADS_TEST_PROXIED_SERVER") != "1"` gate and
fails unless proxied-test-shard.sh lists it in some shard;
TestProxiedGatedTestsDetector pins the walk against a fixture.
Fixes ga-vnycm2.43.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(scripts): indent the gated-test detector's fixture so equivalence.py does not count it
tools/bazel/equivalence.py finds a package's tests with a line-anchored
^func TestX(t *testing.T) scan, so the fixture's TestDirect..TestUngated
read as scripts tests Bazel never ran (EQUIVALENCE: FAIL on Bazel / test).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): add advisory dolt-race shadow lane (item 5 phase 1)
Add --config=dolt-race, the union of --config=doltserver and
--config=doltserver-proxied with one tag filter for both tiers, and an
advisory bazel-dolt-race job that runs it beside bazel-doltserver and
bazel-proxied. The shadow measures one client per group of lanes against
the two lanes it would replace before any cutover (item 5 slice A1a).
The job runs in mode remote on pull_request and merge_group events only.
It has job-level continue-on-error, no job or workflow_call output, and no
gate names it, so it cannot change CI Gate or PR Risk Gate. Its
ci-analytics step is copied verbatim with LANE=dolt-race, and rrc seeds
the new config.
Policy tests pin the config as the exact union of its members' lines (tag
filter aside), the job's run modes and events, its advisory shape, the
analytics lane map and the rrc command list. aquery shows every member
target's TestRunner action (actionKey, argv, env, inputs, outputs) is
unchanged under the union config.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): give the dolt-race shadow timeout headroom; fix gate-wait wording
Review follow-up on the advisory bazel-dolt-race job (maintainer-city-c3,
findings 1, 2 and 5):
- Raise the job's timeout-minutes from 30 to 35. continue-on-error covers
a failed or timed-out step, but not a job GitHub cancels at its own
timeout, which could make the bazel call non-success and turn CI Gate
red. TestBazelDoltRaceShadowIsAdvisory now reads both timeouts from the
workflow and requires job >= test step + 10 (and a step timeout at all).
- Correct the docs and comments: the shadow cannot fail the gate, but
CI Gate still waits for it, because its needs on the bazel call
completes only when every job in it has finished.
- Correct the checks comment: check_testcases covers every target in the
union, including the dolt-server tier its own lane never checked.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): bound every dolt-race shadow step; job timeout covers their sum
Review follow-up (S1): a hang in any bazel-dolt-race step without a
timeout would run into the job's own timeout, and a job GitHub cancels
there is not covered by continue-on-error. Every step now has a
timeout-minutes, and the job's timeout covers their sum:
- checkout 5, rrc read 1, setup-bazel 10, flaky query 5, checks 3 each,
test-log upload 3. The test step (25) and the analytics steps (3, 2)
keep bazel-proxied's values; bazel-proxied sets no timeout on the
others, so these bound them.
- The step timeouts sum to 60 minutes, so the job timeout goes from 35 to
65.
- TestBazelDoltRaceShadowIsAdvisory now requires a timeout on every step
and job >= sum(step timeouts) + 5, read from the workflow, alongside the
existing job >= test step + 10.
Also cite the farm maintainer's confirmation of NativeLink 1.7.1's
in-flight Execute dedupe in the job comment.
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): keep dolt-race shadow timeout under the merge queue's
Tighten the shadow's non-test step budgets so the job timeout (step sum
+ 5) is 55 minutes, and pin that it stays below the 60-minute merge
queue check timeout: CI Gate waits for the shadow, so a hung shadow
must not drop a group from the queue.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(backend/http): add the public HTTP backend door, cmd/bd registration, and bd connect
internal/httpclient (S1-S5) had no production caller for the per-open
backends.OpenOptions.Credential seam and no way for a workspace to actually
select it. This slice (S6) closes both gaps in OSS's own idiom, without
carrying over anything from bd-enterprise's gateway/box-login/storage-profile
machinery:
- backend/http (bdhttp) is the public, out-of-tree extension door: Register
adds "http" to the backend registry and installs its transport, Open/
Handshake dial a server directly with no workspace on disk, and the
registered OpenWith hook threads a per-call credential through
beads.OpenBestAvailableWith for a multi-tenant embedder.
- internal/httpclient/open_with.go is the backends.Backend.OpenWith hook
itself: a fresh dial per open (opts.Credential/HTTPClient/UserAgent),
distinct from the process-wide registered dialer Open/OpenReadOnly use.
- cmd/bd/http_backend.go registers the backend for the OSS binary, so a
workspace whose metadata.json names "http" opens through the ordinary
OpenBestAvailable path; any other unregistered backend name still refuses.
- cmd/bd/connect.go adds the minimal `bd connect <url>` command: a handshake
that must succeed before anything is written, a per-user activation
sidecar (.beads/http_target.json, never git-tracked), a plaintext-to-
non-loopback refusal without --allow-plaintext, and a backend-switch
refusal without --force. No credential is ever accepted on the command
line or written to disk.
- cmd/bd/doctor/gitignore.go requires the new http_target.json /
http_local_metadata.json sidecars be git-ignored, the same way the
proxied-server client info already is.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(storage): keep external-dependency blocking over http unless the server enforces it
bd ready / bd close / claim hard-failed against the new http backend with
"GetAllDependencyRecords not supported by the http backend": cmd/bd's
storage chain wraps every opened store in the externaldeps policy
decorator, which needs either the narrow ExternalDependencyQueryStore
interface or the broad GetAllDependencyRecords fallback to resolve
blocking state, and the http client implemented neither.
The decorator now stays on for a remote store. It is skipped only when the
server's handshake advertises policy.external_dependencies (the server
enforces the policy before answering). Without that capability the client
resolves blocking itself from the listing and chunked dependency reads, so
an issue with an unresolvable external: dependency stays out of bd ready
exactly as it does on a local backend; it is never silently skipped
(DESIGN 3.6). Server-side enforcement is slice S10.
storage.RemoteBackendStore marks a DoltStorage that is a pure network
client of a remote bd serve; the http backend's Store satisfies it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): suppress the CreatedBy stamp on create against a remote backend
bd create failed against the http backend with "Issue.CreatedBy: ... is not
expressible on the v0 wire": the http client's create role deliberately
refuses any POPULATED CreatedBy/CreatedAt (the creation stamp belongs to the
server's own journal entry, not a value the caller supplies), but
cmd/bd/create.go unconditionally stamped getActorWithGit() onto every
created issue regardless of backend. The refusal only fires on a non-zero
field, so suppressing the stamp for a registered remote backend is wire-
compatible by construction, not a workaround.
Add isRemoteBackendWorkspace()/creationActorStamp() to backend_support.go,
reusing the existing resolveCommandBeadsDir(dbPath) +
backendNameForErrorFraming(beadsDir) + backends.IsRemote(name) pattern
already used elsewhere in that file, and use creationActorStamp() at both
create.go call sites (the dry-run preview and the real create).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(backend/http): add an embedder linkage test over public packages only
Prove the one promise S6 makes to an out-of-tree embedder (gc is the
motivating case): importing nothing but github.com/steveyegge/beads,
.../backend, and .../backend/http is enough to register the http backend,
open a workspace through it with a per-call credential
(beads.OpenBestAvailableWith), and read through the role-accessor surface
(IssueReader().Ready) against a stand-in server — with no internal/...
import in the test file, which is itself the thing under test.
Written fresh for OSS beads S6; no bd-enterprise source copied.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs: document the HTTP backend, bd connect, and the public backend/http door
Add a README section with bd connect usage, the credential ladder, and the
plaintext/force refusals, plus a CHANGELOG entry for the new cmd/bd
registration, bd connect command, and the public backend/http package.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): address bee-ghosttrack review on #7288 (connect --clear restore, persisted --allow-plaintext)
Closes the CHANGES_REQUESTED items on the HTTP backend door PR:
- Must-fix: `bd connect --force` over a workspace with its own local
(dolt) identity now has writes succeed instead of tripping "workspace
identity mismatch detected" — validateWorkspaceIdentity (cmd/bd/main.go)
and the sidecar (internal/httpclient/metadata.go) now compare against
the http backend's own pinned project id. New e2e regression test
TestE2E_ForceConnectWritesSucceedDespiteDifferingLocalProjectID
(backend/http/e2e_cli_test.go), confirmed to fail without the fix.
- Should-fix 1: `bd connect --clear` now restores metadata.json's
backend selection to whatever it was before this workspace connected,
instead of leaving it pinned to "http" with no sidecar to dial.
Target.PreviousBackend (internal/httpclient/target.go) is recorded by
Attach (backend/http/attach.go) and round-tripped through the sidecar;
connect.go reads it back on --clear. New tests in
backend/http/attach_test.go and internal/httpclient/target_test.go.
- Should-fix 2: `bd connect --allow-plaintext` is now remembered in the
sidecar (Target.AllowInsecureCredential), scoped to the target it was
granted for, so every later ordinary command's dial honors the grant
without also needing BEADS_HTTP_ALLOW_INSECURE=1 set
(internal/httpclient/dial.go). Fixed BearerProvider.warnInsecure's
wording (internal/httpclient/credential.go), which used to assert a
token was being sent immediately above a refusal saying the opposite.
New test TestDialWithWiresTargetAllowInsecureCredentialThroughToTheGuard.
- Should-fix 3: README no longer claims HTTP mode works "exactly as
with a local database" — it lists what round-trips and points at the
divergence ledger (engdocs/design/http-divergence-ledger.md) for what
differs.
- Verified, no code change needed: Attach already handles an existing
metadata.json project_id coherently (only cfg.Backend is touched); bd
show's isNotFoundErr already matches a http 404 (issueops.ErrNotFound
is storage.ErrNotFound, the same sentinel a local miss uses), so it
reads as "not found" rather than an unsupported-operation error.
- Doc fix: guardInsecureCredential's doc comment and refusal message
were stale against should-fix 2's own change — they named only
DialOptions.AllowInsecureCredential and the env var as ways to get
past the guard, not the newly-persisted target.AllowInsecureCredential
or `bd connect --allow-plaintext` itself.
Gates run clean on this branch: go build/vet, go test over
internal/httpclient, backend/http (including the full e2e suite under
BEADS_TEST_EMBEDDED_DOLT=1) and cmd/bd, make api-check, make
ci-pr-lint, make bazel-sync-check, bazel run //:gazelle -- -mode=diff,
and internal/storage's TestEveryLegWiresEveryRoleContract.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(backend/http): give the e2e server the BatchGetter role #7248 added to the required set
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix: exact http ready totals, connect capability row, doc corrections
Ready totals over http (review major). externaldeps drops externally
blocked rows client-side for a store that cannot express ExcludeIDs
(today httpclient.Store). dropBlockedIssuesWithCounts compacted the
fetched rows in place (`kept := rows[:0]`), and
readyWorkWithCountsAndTotal then re-scanned `rows` to count the blocked
rows it had dropped. By then the compaction had overwritten them, so
whenever a blocked row came back ahead of a kept one the re-scan
undercounted the drops: `bd ready --json`'s total and text-mode `bd
ready`'s count (CountReadyWork) overstated ready work against any server
that does not advertise policy.external_dependencies. Both drop helpers
now copy into a fresh slice, and dropBlockedIssuesWithCounts reports how
many rows it dropped, so nothing re-scans `rows`.
fakeStore gains an excludeIDsUnsupported switch that, like
httpclient.Store, refuses any ExcludeIDs. The new
TestReadyTotalsWhenStoreCannotExpressExcludeIDs puts the blocked issue
first and checks the page and total for an unlimited filter, a Limit
whose bumped window holds the whole set, and one the set outgrows, then
CountReadyWork. TestServedReadyExcludesUnsatisfiedExternalDependencyWithoutServerCapability
now also checks the total and count against a real in-process bd serve.
Against the previous decorator the unit test's unlimited case reports
total and count 3 beside a 2-row page, and the served test 2 beside a
1-row page.
Review minors, all wording:
- `bd connect --clear` pointed at `bd config set backend dolt`, which
writes the database's config table rather than metadata.json and, in
the state that prints the advice (metadata.json says http, sidecar
gone), fails the pre-open identity check. It now says to reconnect, or
to set "backend" in metadata.json.
- CHANGELOG described the superseded skip-at-wrap design; it now
describes the always-wrap decorator with its capability probe and
client-side drop.
- RemoteBackendStore's docs no longer name a consumer; nothing in-tree
checks it.
- creationActorStamp's comment pointed at backend/http/lifecycle.go; the
refusal lives in internal/httpclient/lifecycle.go.
Capability row. TestProxyCapabilityRegistryCoversCommandTree fails at
60b7128204: the command tree walk finds `connect`, which this PR adds,
with no row in cmd/bd/capability_registry.go ("1 command(s) have no
proxied-server capability row: connect"). The test predates the PR
(#6582 is an ancestor of its base) and passes at the base dabf20ef07.
connect opts out of store init (skipStoreAnnotation) and never reaches
the store factory, so it belongs in proxyPermittedPaths beside init and
setup. The row is runtime-neutral: validateProxyRegistryBeforeProvider
returns nil both for a permitted row and for a path with no row, so
this only records the policy decision the coverage test asks for.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(httpclient): regenerate the divergence ledger doc after the S5 rebase
The rebase onto feat/http-client-batch-apply's rebased head (S3/S4 merged to
main) left the ledger doc's summary line at a conflict-resolution placeholder
count. Regenerate it from internal/httpclient/encode.Ledger() so
TestDivergenceLedgerDocMatchesLedger passes against the merged ledger.go.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(backend/http): host-scope S6's test BEADS_HTTP_TOKEN values
Rebasing onto feat/http-client-batch-apply's rebased head pulled in main's
#7247 review-fix round, which made BEADS_HTTP_TOKEN/BEADS_HTTP_TOKEN_COMMAND
host-scoped (host[:port]=value, scopedEnvValue/parseScopedEnv in
internal/httpclient/credential.go) after S6's own tests were written against
the unscoped form. Five S6 tests across backend/http/open_test.go,
backend/http/register_test.go and
internal/httpclient/insecure_credential_guard_test.go set a bare token and
now hit the new ladder's "an unscoped token would be used for whatever
server..." refusal. Scope each to the test server's actual host
(127.0.0.1 for the httptest.Server-backed recording servers,
nonLoopbackTestHost for the insecure-credential-guard integration tests)
so they exercise the same ladder rung as before.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): resync go_srcs/repo_files for the new backend/http and test/embedder packages
make bazel-sync-check failed after the rebase: tools/bazel/go_srcs.py's
managed repo_files filegroups for backend/http and test/embedder (new in
this slice) hadn't been generated, and the root BUILD.bazel / tools/bazel's
release_cross_build target list hadn't picked up the new package either.
Regenerate with `python3 tools/bazel/go_srcs.py`; `bazel run //:gazelle --
-mode=diff` reports no further drift.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): route swarm's create paths through creationActorStamp after main's currentActor change
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix: police http's served roles, refuse close --claim-next, exact totals
External-dependency policy over http (review blocker). cmd/bd wraps
every store, http included, in the externaldeps decorator, and its role
accessors rebuilt each role over legacy methods httpclient.Store refuses
(SearchIssuesWithCounts, GetDependencyTree, ClaimReadyIssue, IsBlocked,
GetDependencyRecordsForIssues, BatchCloserWithPolicy). Over http `bd
list`, `bd dep tree`, `bd ready --claim` and a policed `bd close`
therefore exited 1, even in a workspace with no external dependency.
Over a store that reports storage.RemoteBackendStore, each accessor now
starts from the inner store's served role and layers the policy around
it (remote_roles.go):
- IssueReader serves List and Get from the served reader; Ready and a
ready-flagged List go through the policy reader.
- IssueClaimer refuses an externally blocked claim before the served
one.
- ReadyClaimer uses the served claim-next while no issue, closed ones
included, holds an unsatisfied external ref. Otherwise it claims down
the policy's filtered ready page through the served Claimer, because
claimNext cannot exclude ids.
- BlockingAnnotator and TreeWalker merge external blockers into, or add
external leaves to, the served answer. They read only the named
issues' edges through the served EdgeReader, 100 anchors per call.
- BatchCloser refuses externally blocked open items and sends the rest
to the served closer as one request. --force keeps the base path, and
a next claim is refused whole (below).
RemoteBackendStore now only chooses this composition. Whether the policy
runs is still ServerEnforcesExternalDependencyPolicy's answer (design
3.6), and the storage and httpclient docs say so. remote_roles_test.go
drives each role over a store that refuses the legacy methods.
TestServedRolesKeepTheExternalDependencyPolicy runs every role over a
real http server. TestE2E_BDOverHTTP adds `bd list` (JSON, text and
--deps), `bd dep tree`, `bd ready`, `bd list --ready`, `bd ready
--claim`, a refused `bd close --claim-next`, and a refused then forced
close of an externally blocked issue. At the fixup before the
composition, the role tests then written (bar one pin of the next-claim
refusal, since replaced) and the e2e's list, dep tree, claim and close
steps of that round were red.
Close with a next claim over http (review major). batchCloseIssues has
no member for a next claim, so httpclient refuses any ClaimNext before
the dial (ledger W-CloseBatchRequest.ClaimNext). Over http `bd close
--claim-next` therefore always exited 1 and closed nothing. With an
external ref anywhere in the workspace (a closed holder or --force
included) the decorator's base path failed first, with an opaque
`BatchCloserWithPolicy` unsupported. remoteBatchCloser now refuses a
next claim whole, before it reads or closes anything, and the error
says nothing closed: close without --claim-next, then run `bd ready
--claim`. A request that is invalid on every backend is still
ErrValidation first, and a server that enforces the policy itself gets
the request unchanged.
Composing close-then-claim was not taken. The role closes and claims in
one transaction and answers with an error or with outcomes, never both,
so a claim that failed after the closes committed could not report
them; httpclient refuses the same request for that reason. A CLI-level
close then claim that can report a partial result is the follow-up.
The review also suggested leaving closed holders out of the blocking
state. They stay in, as on every backend: a closed issue is never a
claim candidate, so they change no claim, and leaving them out would
cost a status read per holder because the wire has no batch read.
TestRemoteBatchCloserRefusesANextClaim covers an open holder, a closed
one, none, --force, and a server that enforces the policy; with the
previous remote_roles.go the first four are red.
TestRemoteBatchCloserValidatesANextClaimFirst keeps ErrValidation ahead
of the refusal. Dropping the filter check, or loading the blocking
state before refusing, is each caught. TestE2E_BDOverHTTP runs `bd
close --claim-next` while the holder is open and again after it is
closed; with the previous remote_roles.go both steps are red and print
`operation "BatchCloserWithPolicy" not supported`. Ledger F-close no
longer says --claim-next is served, and no longer names a
renderEscapedHTTPRefusal that does not exist. L14 says the decorator's
claim leg applies while any issue, closed or not, holds an unsatisfied
ref. The README and CHANGELOG mention the refusal, and the decorator's
IsBlocked, IsBlockedBatch and CloseIssueChecked say they refuse over a
store that serves its roles.
Ready totals with external blockers (review majors, both rounds).
CountReadyWork asks for Limit=0, and readyWorkWithCountsAndTotal took
any Limit<=0 to mean "the window held the whole ready set". Over http a
zero limit never reaches the wire: the server answers with its default
page (workapi.DefaultReadyLimit, 100 rows) while the bridge reports the
full total, so past 100 ready issues text-mode `bd ready` kept counting
externally blocked issues beyond the first page. Exactness is now judged
from the rows that came back (total <= len(rows)). A window that holds
only part of the set no longer subtracts every externally blocked id in
the workspace either: that set includes closed issues and ones the
filter excludes, none of them in the total, so the count came out low.
It refetches the filtered ready set, ids only, and subtracts the blocked
rows in it; the old bound is left as the fallback for a refetch that
comes back truncated too.
fakeStore gains the bridge's shape: a defaultPage for a zero Limit,
with GetReadyWorkWithCountsAndTotal still totalling the whole set.
TestReadyTotalsWhenAnUnlimitedFetchComesBackAsOnePage (a blocked issue
past the first page) and
TestReadyTotalsIgnoreExternallyBlockedIssuesOutsideTheReadySet (a
blocked issue outside the ready set) have served counterparts,
TestServedReadyCountExcludesAnExternallyBlockedIssuePastTheDefaultPage
and TestServedReadyTotalIgnoresAnExternallyBlockedIssueOutsideTheFilter
(an assignee filter). The first pair is red on the Limit<=0
condition, the second on the subtract-every-blocked-id branch.
Docs: ledger L8 now says which decorator paths take the chunked
GetAllDependencyRecords scan and which read only named edges, and L14
records the decorator's composed claim leg; http-divergence-ledger.md
is regenerated. The README names the verified commands, and the README
and CHANGELOG say how cross-repo dependencies behave over http.
Minors from the reviews:
- Drop creationActorStamp/isRemoteBackendWorkspace and restore the base
stamps at all 11 sites. The arm guarded a refusal the http create
role does not make: since #7247 (actorStampedCreateMember) it accepts
a CreatedBy naming the actor. Because the arm keyed on the CWD
workspace, a create routed from an http workspace into a local one
(--repo) stored an empty created_by. TestE2E_BDOverHTTP checks
created_by for a direct and a routed create; the routed check is red
with the arm restored. The CHANGELOG sentence describing the arm goes
with it.
- bdhttp.Attach decides a reconnect from the raw metadata.json backend.
GetBackend() reads an unregistered "http" as the dolt default, so in
an embedder that never called Register a reconnect recorded dolt as
the backend --clear restores. The reconnect test now seeds postgres
(TestAttachOnAReconnectKeepsTheEarlierPreviousBackend) and is red
with the old check.
- README, CHANGELOG and `bd connect --help` document the host-scoped
BEADS_HTTP_TOKEN=host[:port]=<token> form (and _COMMAND), and say why
a bare value is refused.
- bdhttp.Attach also gitignores http_local_metadata.json, the per-user
file the store writes on first use, so an embedder activating through
Attach alone gets the coverage `bd connect` already had.
TestAttachGitignoresBothPerUserFiles pins it.
- ServerSnapshot.WireRevision's doc names the handshake's actual check
instead of a comparison `bd connect` never makes.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(backend/http): give the e2e server the LeaseReclaimer role #7412 added to the required set
main's #7412 made LeaseReclaimer part of httpapi's required role set and
updated main's own binders, but backend/http's e2eServeConfig predates it,
so every TestE2E_* that binds the in-process server failed with
"httpapi: no database source". Same shape as e63f7da36a for BatchGetter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(externaldeps): lead the --claim-next refusal with the wire fact; pin the L14 leg and connect --clear
Review i4 minors/nits on the stranded fixup, carried as a separate commit:
- remoteBatchCloser's --claim-next refusal now says the served batch close
cannot carry a next claim (the real cause), rather than blaming
externally blocked work a workspace may not have. The test asserts the
reason, that nothing closed, and the remedy.
- GetDependencyTree's doc says a served store refuses its reads and that
bd dep tree goes through remoteTreeWalker.
- README no longer gives `bd ready --claim` local semantics everywhere; it
names when ledger row L14's two-step claim applies.
- TestE2E_BDOverHTTP claims once more after the external-ref holder is
closed, pinning L14's composed leg (a mutant whose composed leg claims
nothing turns only that step red).
- New TestE2E_ConnectClearRestoresThePreviousBackend: --clear restores
the prior dolt backend with its identity intact, removes the sidecar,
and a second --clear changes nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(backend/http): run the served e2e under bazel
//backend/http:http_test declared no bd binary, so under bazel the e2e
files (bazeltest.PrebuiltBD) failed on BEADS_TEST_BD_BINARY with
BEADS_HTTP_TEST_REQUIRED=1 and only "passed" by skipping otherwise. It now
takes //cmd/bd:bd_for_tests (visibility added), size large / timeout long:
the served cases run 4-5 minutes on the farm under BEADS_TEST_EMBEDDED_DOLT=1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(backend/http): run the e2e tier in the embedded lane as http_served_test
The previous commit gives //backend/http:http_test the bd binary, but no
lane sets BEADS_TEST_EMBEDDED_DOLT for it, so its TestE2E_* cases still
skipped in CI. http_served_test runs the same test binary through
//tools/bazel:go_test_variant.sh with BEADS_TEST_EMBEDDED_DOLT=1 and
BEADS_HTTP_TEST_REQUIRED=1 (a skip fails it). It is tagged embedded, so
the --config=embedded lane runs it and prcore/ci keep their current
shape, the same split as //internal/httpclient:httpclient_served_test.
Under plain --config=embedded all 8 TestE2E_* cases run, none skipped.
TestBazelRetiredLanesCannotBeNarrowed pins every embedded-lane target's
args and env; the new target is added to that pin list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(externaldeps): cap the ready rows kept, not the bumped http window
An http store cannot carry ExcludeIDs on listReadyWork, so readyExclusion
fetches the unfiltered ready set, widened by len(blocked) under a limit,
and drops the externally blocked rows client-side. The http ready bridge
enforced MaxRows on the rows it fetched rather than the rows kept. So
`bd ready --limit N --max-rows N` exited 2 whenever more than N issues
were ready and any workspace issue held an unsatisfied external blocker
(a closed one counts), even with no blocked row in the window, and a
blocked row in the page counted against the cap although the caller
never sees it. Locally neither can happen: issueops enforces the cap on
the filtered page it delivers, after the Limit trim
(finishReadyWorkWithCounts).
The client-side path now takes MaxRows off the inner query and enforces
the caller's cap on the rows it keeps, after the trim, in all three
listings. The inner query keeps the cap whenever nothing is dropped
client-side. MaxRows never rode the wire, so the fetch is the size it
was; refetchedReadyTotal already clears the cap on its own over-fetch.
TestServedReadyMaxRowsCountsTheRowsKept runs the decorator over a served
http store holding three free issues and one externally blocked one: a
cap equal to the limit, a cap under a wider limit, an unlimited listing,
and a kept page wider than the cap, which must still return
ErrTooManyRows with the caller's cap and source. Each case runs through
GetReadyWork, GetReadyWorkWithCounts and GetReadyWorkWithCountsAndTotal.
Leaving the cap on the inner query fails the first three cases in all
three methods, dropping the re-applied cap fails the fourth, and
enforcing it before the Limit trim fails the first.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
rbe-west's pool workers advertise gastownhall/gascity main's worker-env
pin; when gascity re-pins (a Go/dolt/OS bump, e.g. #7387) beads' copy
falls out of step and every remote-execution run stops in bazel.yml's
preflight until a human notices and copies the fix by hand (#7419 took
~100 minutes). This adds a scheduled workflow (every 15 minutes, plus
workflow_dispatch) that runs tools/rbe/worker-env-sync against gascity
main, and on a real mismatch (exit 1) copies the manifest and pin with
the new tools/rbe/worker-env-sync-copy, opens/updates a PR on a fixed
branch with status/needs-review-auto, and arms auto-merge.
GitHub never runs checks on a PR a plain GITHUB_TOKEN opens, so this
needs a GitHub App token -- but gastownhall-autofix (contents:write on
this repo) cannot open or label a PR. Two Apps split the two write
roles instead: gastownhall-autofix mints push-token (contents:write
only) and does the git push of bot/worker-env-sync via plain git;
gascity-autofix (pull-requests:write only) mints pr-token and does
everything PR-shaped -- create/update, the label, and arming
auto-merge -- via gh. peter-evans/create-pull-request takes one token
for the whole flow, so it can't honor this split; plain git and gh
replace it. Every ${{ }} is resolved into a step's env: first, never
written inside a run: script.
Both Apps' id/private-key secrets live in the same autofix environment,
gated by its deployment-branch policy to main; schedule and
workflow_dispatch are never fork events, so neither key is ever
readable by a fork.
Needs admin setup: the gascity-autofix App (id 5259233) installed on
gastownhall/beads (and gascity) with pull-requests:write and metadata
read, and its GASCITY_AUTOFIX_APP_ID / GASCITY_AUTOFIX_APP_PRIVATE_KEY
secrets added to the autofix environment alongside the existing
AUTOFIX_APP_ID / AUTOFIX_APP_PRIVATE_KEY.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* docs(proxied): name the Close3/Close4 legacy pins and fix the Close3 doc comment (#7173 follow-up)
Two non-blocking findings from the #7173 adoption review (bd-vrdrew,
bd-dars9p), both in bee's code, neither addressed in that PR:
- The doc comment on TestProxiedServerClose3 still called it "the second
half of TestProxiedServerClose"; after the second split it is the middle
of the three parents (Close, Close3, Close4). Say so, and name Close4
as its own second half.
- In the frozen 15-shard block of .github/scripts/proxied-cmd-test-shards.txt
the lines `15 13 TestProxiedServerClose3` and `15 2 TestProxiedServerClose4`
sat inside the shard-1 and shard-7 groups (#7173's last commit moved the
pins from 1 and 7 to 13 and 2 without moving the lines), and the header's
"Deliberate exception (Oct 2026)" paragraph did not name them. The two
lines now sit alphabetically inside their own shard groups, the header
names them with the reason and the measured times, and the generator's
hand-pin paragraph (scripts/ci/gen_proxied_shard_manifest.py) names them
too so the two texts stay in step. The lines were inert either way:
proxied-test-shard.sh matches each line by field.
No test moves shard: the 15-shard block's line set is unchanged apart from
order (sorted diff empty) and the 30-shard block is byte-identical.
Verified: gofmt -l (clean); go vet ./cmd/bd/ (exit 0);
gen_proxied_shard_manifest.py 30 --weights=duration --check (exit 0,
heaviest 435.1s est, unchanged); 15 --check reports the same 30 hashed
names missing as main (the frozen block is expected to fail it);
proxied-test-shard.sh list-only: Close3 on 13/15, Close4 on 2/15, neither
on 1/15 or 7/15; go test ./scripts/ -run 'Shard|Manifest|Coverage' ok 38.6s.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(proxied): count CloseB in the Close3 comment; say why shard 5 got no pin
Maintainer fixup from the adoption review of #7382 (bd-g6pyb0), on top
of bee's commit, which stays as written. Comment and manifest-header
text only: no test body, shard assignment or routing changes.
- TestProxiedServerClose3's doc comment called it "the middle of the
three parents TestProxiedServerClose was split into (Close, Close3,
Close4)". That held at the PR's original base, but #7418, on main
since, moved five of Close's eleven cases into a new
TestProxiedServerCloseB, so the 23 bd-init subtests of the pre-#7173
TestProxiedServerClose now sit in four parents: Close 6, CloseB 5,
Close3 6, Close4 6 (their union is exactly the pre-#7173 set, no
duplicates). The comment now says Close3 was split off Close, names
all four parents, and keeps "Its own second half is
TestProxiedServerClose4."
- The frozen 15-shard block's header said #7173 pinned Close3 and
Close4 onto "the two lightest shards of that PR's own run (6m51s and
7m57s)". On that run (37206446441, at 765248af5) shard 5 was lighter
still, at 5m50s; #7173 passed over it on purpose, because it was
light only because those 12 subtests had just left it and putting
Close3 back would recreate most of the set that timed out. The header
now says "other than shard 5 itself" and gives that reason.
This commit also supersedes the parts of the parent commit's message
that the rebase onto c21880105f outdated:
- "after the second split it is the middle of the three parents (Close,
Close3, Close4)": four parents now, as above.
- "the 30-shard block is byte-identical": the Bazel-only block is 34
shards now (#7418 made it 44, #7436 34), and that block is
byte-identical to the base's.
- "gen_proxied_shard_manifest.py 30 --weights=duration --check (exit 0,
heaviest 435.1s est, unchanged)": with no 30-shard block left, that
command now exits 1 ("has no block for total_shards=30"). Its current
form, 34 --weights=duration --check, exits 0 with the heaviest shard
at 196.1 s est., the same before and after this commit.
- "15 --check reports the same 30 hashed names missing as main": it now
reports 53, the same set as against the base's manifest.
Verified: git diff --check clean; gofmt -l clean (the Go change is
comment-only); with '#' lines stripped as proxied-test-shard.sh strips
them, the manifest is identical before and after;
gen_proxied_shard_manifest.py 15 --check and 34 --weights=duration
--check print the same output before and after (exit 1 with 53
missing; exit 0, heaviest 196.1 s); BEADS_TEST_SHARD_LIST_ONLY=1
proxied-test-shard.sh lists for 1, 2, 7 and 13 of 15 are unchanged
(Close3 on 13, Close4 on 2, neither on 1 or 7); go test ./scripts/ -run
'Shard|Manifest|Coverage' ok (21 top-level tests, none skipped).
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <julianknutsen@users.noreply.github.com>
Port gascity's remote repo contents cache wiring (gascity #7467, #7478) to
bazel.yml, gated on the repository variable RBE_REPO_CONTENTS_CACHE
(unset/off: nothing; seed: rrc-seed on push to main; canary: + bazel-test
reads; on: + every test lane reads). Mode remote only.
- rbe job: an rrc step turns the variable into the rrc output.
- Readers: the eight test lanes append startup
--experimental_remote_repo_contents_cache and common
--loading_phase_threads=64 to .bazelrc.local before Set up Bazel.
- rrc-seed (push to main, id-token: write): a 30-minute
rbe-rrc-writer-beads certificate from rbe-west's mint
(.github/scripts/rrc-writer-credential.sh), then every lane's command
(.github/scripts/rrc-lane-commands.txt) with --nobuild, --remote_cache
only and --remote_upload_local_results (.github/scripts/rrc-run-lanes.sh).
- rrc-verify (schedule/dispatch, issues: write): cold fetch of every lane
and tools/bazel/rrc_verify.py against the cache; a mismatch opens an
rrc-verify issue.
- pr.yml, nightly.yml and bazel-farm.yml grant id-token: write and
issues: write on their bazel.yml call (startup_failure otherwise);
TestBazelWorkflowCallersGrantEveryRequestedPermission guards it.
rrc-writer-credential.sh, rrc_verify.py and rrc_verify_test.py are byte
copies of gascity's, pinned by sha256.
Refs ga-vnycm2.29.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
//internal/storage/embeddeddolt:embeddeddolt_conformance_core_test ran
conformance.RunAll minus Audit as one unsharded action, 108-150s on
rbe-west, the embedded lane's longest after #7443. Every RunAll group builds
its own store from the factory, so the groups share no state and can run in
separate processes; about half the time was two groups,
ReadyCountsPageChunking (~33s) and Portable (~20s).
The new embeddeddolt_conformance_core_slow_test runs exactly those two
groups (-test.run=^TestConformance$/^(ReadyCountsPageChunking|Portable)$),
and the core target skips them along with Audit. The core target skips
only named groups, so any group added to or renamed in RunAll still runs
there: between them the two targets run every core group exactly once
(rbe-west: 57 + 2 groups, no overlap, the same 59 as before). No test
changed. The frozen-partition check now derives both targets' selectors
from the retired job's core selector plus conformanceCoreSlowGroups; the
retired-lane arg pins, bazel.yml's --whole list and the BUILD rules carry
the new target.
rbe-west, `bazel test //... --config=embedded --nocache_test_results`:
conformance core 148s -> 55s + 57s.
Bead: ga-vnycm2.31
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
After #7440 //internal/httpclient:httpclient_served_test set the embedded
lane (149-167s slowest shard in recent CI runs, 63-167s across PRs since
#7416). Not a regression: its top-level tests are all serial and rules_go's
round-robin put the long ones wherever its index split dropped them, so
shard times swung with load and with which tests a PR added (whole-lane
rbe-west run on main: 40-119s over 16 shards).
The target now runs through tools/bazel/go_test_pinned_shard.sh, as
bd_dolt_server_test and regression_test do: every top-level test measured
at >= 3s (max over three whole-lane runs) is pinned onto 17 shards
(served_pinned_shards.txt, tools/bazel/pin_shards.py), and the binary's own
round-robin deals the other ~620 over the remaining 9. Every test still runs
in exactly one shard (TestPinnedShardWrapperSplit), the manifest may name
only real tests, and the lane's check_testcases.py fails an empty shard.
No test changed; the heaviest single test (~52s, no subtests) stays whole.
The pinned-wrapper policy listed only dolt-server-cmd targets, since the
wrapper selects and skips; pinnedShardTargets now carries each reviewed
target's lane tag, and httpclient_served_test is reviewed in as the
embedded lane's (its args stay pinned in
TestBazelRetiredLanesCannotBeNarrowed).
rbe-west, `bazel test //... --config=embedded --nocache_test_results`:
served shards max 119s -> 77s, median 73s -> 54s; lane critical path
138-153s -> 116s.
Bead: ga-vnycm2.30
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
//internal/storage/embeddeddolt:embeddeddolt_embedded_test became the
embedded lane's floor once the cmd tier was split (CI on #7438: 20 shards,
220s max, 162s median). Its tests are serial, so a shard's wall time is the
sum of its tests; measured on rbe-west with the whole lane running (as CI
runs it), its 20 shards took 110-161s.
- TestCreateBatchFastPathsMatchPerRow_Embedded (111s) runs
createbatchequiv's scenarios as two top-level tests (small+depadd,
apply+waitsfor) through a new createbatchequiv.RunNamed; the same
scenarios and checks. TestCreateBatchEquivalencePartsCoverEveryScenario
fails unless the parts name every createbatchequiv.Run scenario exactly
once.
- TestWithPeerAuth_WarnsWhenStoredPeerSuppressesAmbientPassword (81s, eight
cases each opening its own store) runs its table as two top-level tests,
cases 1-4 and 5-8; cases and assertions unchanged.
- Durations re-measured on rbe-west under the whole lane's load; the
Bazel-only block is now 40 shards (LPT heaviest = the heaviest single
test, TestLifecycleCloseReopenContract ~71s). One target: 40 is under
Bazel's 50-shard cap, so no offset split.
rbe-west, `bazel test //... --config=embedded --nocache_test_results`:
storage shards max 161s -> 89s, median ~118s -> 63s.
Bead: ga-vnycm2.28
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
//cmd/bd:bd_embedded_test was the embedded lane's floor: at Bazel's 50-shard
cap per rule, each shard's wall time was its slowest top-level test, and
~40 tests ran 65-130s on rbe-west.
- 34 slow TestEmbedded* functions are split into 89 smaller top-level
functions. Subtests and table cases are moved verbatim: no assertion,
flag or data changes, the same t.Run set per file, the same skip guard and
t.Parallel. Subtests that read state earlier ones left stay together, in
order; each new function repeats the shared setup it needs.
- The manifest's cmd/bd block is now 100 shards, run by two targets:
bd_embedded_test (shards 1-50) and the new bd_embedded_part2_test
(51-100), through go_test_manifest_shard.sh --shard-offset/--shard-total.
check_shard_coverage.py takes SHARDS@OFFSET/TOTAL and fails unless the
targets' ranges tile the block exactly once, so every test runs in exactly
one shard of one target; the policy tests read both rules' ranges from
cmd/bd/BUILD.bazel.
- Durations re-measured on rbe-west with the split tests and repacked. The
frozen legacy 20-shard block drops the renamed names (no workflow reads
it; new names hash-distribute as before).
Not split: the *Concurrent stress tests (their bd subprocesses queue on the
workspace's single writer lock: TestEmbeddedUpdateConcurrent is ~100
serialized bd commands), TestEmbeddedPurgeWispsPlaneRetention (one stateful
sequence), and the MaxRowsList* tests (setup is 21 sequential creates every
subtest reads).
rbe-west, both targets, --nocache_test_results: slowest shard 138-163s ->
109s (TestEmbeddedUpdateConcurrent alone; every other shard <=86s), median
80-84s -> 40s, shard-seconds 4.1-4.4k -> 4.2k, critical path 213-306s ->
113s.
Bead: ga-vnycm2.17
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
//cmd/bd:bd_embedded_test sits at Bazel's 50-shard cap and was the embedded
lane's floor (shards 62-250s). Measured on rbe-west, every shard's wall time
was its single slowest top-level test, and most of a test's time was `bd
init` (~20s under -race: one Dolt commit per schema migration) plus a fixed
1s per bd subprocess exit (the race runtime's atexit_sleep_ms).
- bdInit seeds .beads/embeddeddolt/<db> from a per-process schema template
(an empty database with every migration applied, made once per shard by a
non-race build of internal/storage/embeddeddolt/cmd) before running the
real `bd init`, which then skips the migration chain and does everything
else itself (identity, prefix, config, hooks, agent files, git commit).
Only for argument lists it understands (--prefix/--database plus
--quiet/--skip-hooks/--skip-agents/--non-interactive), and it checks the
result is the seeded embedded database. BEADS_TEST_FRESH_INIT=1 turns it
off. The TestEmbeddedInit* tests run their own init and are unaffected.
- GORACE=atexit_sleep_ms=0 on the target: the bd subprocesses no longer
sleep 1s before exiting. A race they hit while running still fails the
command as before.
- TestEmbeddedInitSelectedExcludeRouting (serial, t.Setenv, 7 full inits,
~113s) is split into ...Stealth and ...Fork top-level functions; same
cases, same assertions.
- The generator packs with a model of the shard process (pack_concurrent):
tests without a top-level t.Parallel run one after another, the rest
share -test.parallel=4. The durations are re-measured on rbe-west.
rbe-west, `bazel test //cmd/bd:bd_embedded_test --config=embedded
--nocache_test_results`: test-seconds 20.2k -> ~10.3k, shard-seconds
8.0k -> ~4.2k, slowest shard 243s -> 138s / 163s (two runs, before the
repack).
Bead: ga-vnycm2.17
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): scope the proxied lane's shared-server projects to their own database (44 -> 28 shards)
//cmd/bd:bd_proxied_test spends nearly all its time in `bd init
--proxied-server` against the shard's shared dolt sql-server. Measured
where that goes (dolt query log): of ~2,031 statements per init, the ~220
INFORMATION_SCHEMA queries (migration guards and pre-repair probes) are
~80% of server time at a handful of databases, the 81 DOLT_COMMITs only
~0.5s. Dolt evaluates INFORMATION_SCHEMA by walking every database the
session can see, so as root each init cost the server ~1.2 CPU-seconds
more per database already on it (locally 6.7 CPU-s with 1 database, 21
with 11): a shard's k-th init paid for the k-1 projects before it.
- proxied_shared_harness_test.go: newSharedProxiedProject now provisions
the database server-side (root creates it plus a passwordless user of
the same name with ALL on it and nothing else) and inits with
--proxied-server-external-user, so each project's sessions see one
database. newSharedProxiedRootProject keeps the old root/bd-creates-it
shape for the four tests that need a server-wide privilege (DOLT_REMOTE
remove, CREATE DATABASE, one compact statement); a run with the server
log in test outputs showed no other denied statement.
- Dropping finished databases instead was tried and rejected: a DROP
DATABASE fails other sessions' in-flight INFORMATION_SCHEMA queries
("no root value found in session"), ~35 init failures per run.
- No assertion changed. The bd-creates-the-database path stays covered
by the root variant, the managed-local and non-shared init tests.
- proxied_test_durations.json re-measured (two rbe-west runs): 19,618 ->
6,345 cost-seconds. 44 -> 28 shards, LPT-packed to ~227 per shard.
rbe-west, --nocache_test_results (BEP attempt durations): before 44
shards, sum 6,063s, max 186s; scoped users at 44 shards, sum 2,337s, max
68s; at 20 shards, sum 1,950s, max 136s.
Bead: ga-vnycm2.16
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): proxied lane 28 -> 34 shards for a <=100s slowest shard
PR CI at 28 shards (~227 packed cost each): 2,285 shard-seconds, median
78s, but max 113s; equally packed shards spread up to 1.45x the median.
34 shards (~187 each) targets a slowest shard under 100s.
Bead: ga-vnycm2.16
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): fetch pinned Ubuntu debs in parallel from a gascity mirror first
cc_sysroot_noble_amd64 downloaded its 7 pinned .debs one at a time from
snapshot.ubuntu.com, with only archive.ubuntu.com behind it. On
2026-10-09 the snapshot returned 503, and Bazel's retries against it
cost ~95 s per cold fetch before falling back (and archive.ubuntu.com
drops superseded versions, which would fail the fetch outright).
Same change as gastownhall/gascity (ga-vnycm2.11): deb_sysroot takes URL
templates ({path}, {file}, {sha256}) and starts every package download at
once (rctx.download block = False). NOBLE_DEB_URLS tries gascity's
content-addressed release mirror (toolchain-noble-debs-20261001, which
holds these exact pins; beads already takes its LLVM slice from a gascity
release), the snapshot, Launchpad's librarian, then archive.ubuntu.com.
The sha256 pins are unchanged and decide what is accepted; the unpacked
sysroot is byte-identical. The hermetic-cc guard now checks every
deb_sysroot: placeholders, the snapshot kept, two or more hosts, no
moving mirror first, no ICU.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37966817176
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
//cmd/bd:bd_proxied_test's 30-shard block was packed from a local capture
of each test's <testsuite time>, which for a t.Parallel test spans the time
it sat paused behind its shard's other tests. Cheap tests that waited
looked as heavy as the shard, so the "balanced" block ran 38-286s per shard
in CI (356s on rbe-west).
- scripts/ci/proxied_test_durations.json: re-measured on rbe-west (two
--nocache_test_results runs), each test's cost the sum of its leaf
tests' <testcase time>, averaged over both runs.
- Measured where a test's time goes: nearly all of it is
`bd init --proxied-server` against the shard's dolt sql-server (~2,000
statements, 81 DOLT_COMMITs, ~6-12 server CPU-seconds per init), one per
subtest. Top-level tests with ~20 such subtests ran up to ~350s alone,
above any shard budget, so the fifteen heaviest are split into two or
three top-level functions (TestProxiedServerUpdate/-UpdateB/-UpdateC,
...). Each subtest is moved verbatim; the split functions repeat the
original's preamble.
- 30 -> 44 shards, LPT-packed to <=430 cost-seconds per shard.
rbe-west, --nocache_test_results: before 30 shards, max 356s, 5,671
shard-seconds; after 44 shards, max 159s (median 138s), 5,883
shard-seconds.
Bead: ga-vnycm2.3
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Client-side loading and analysis is CPU-bound and gates every action in a
remote lane. Analysis-only measurements on Blacksmith (ga-vnycm2.8, seeded
runner cache, gazelle tools from the repository cache): bazel-test 25.6 s on
4 vCPU vs 16.3 s on 8; the other lanes 17-38 s on 2 vCPU vs 12-22 s on 4.
rbe-prewarm runs no Bazel client and stays on 2 vCPU (owner decision,
ga-vnycm2.12).
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): cut embedded/test lane long shards (served template, bulk seed, rebalance)
Embedded lane (bazel-embedded):
- RunBatchGetterAcceptsExactlyTheCap seeds its 1000 issues through a new
optional BatchGetterFixture.CreateIssues hook (one write) instead of 1000
single-row commits. Wired for the embedded, served and dolt-server legs;
the UOW leg keeps the per-row fallback. Same rows, same GetMany
assertions. TestBatchGetterContract 199s -> ~7s and
TestServedBatchGetterContract 204s -> ~7s on rbe-west.
- The served-surface harness (newServedEnv) copies one per-process,
freshly initialized reference workspace instead of initializing an
embedded Dolt schema per case, the same pristine-template pattern
internal/storage/embeddeddolt's tests already use (bd-7q32v).
//internal/httpclient:httpclient_served_test: 3,713 -> ~1,200
shard-seconds; 24 -> 16 shards; heaviest shard 390s -> ~108s.
- //internal/storage/embeddeddolt:embeddeddolt_embedded_test: re-measured
every test on rbe-west (scripts/ci/embedded_storage_test_durations.json)
and repacked 15 -> 20 shards (packed heaviest ~110s; was 352s in CI).
Test lane (bazel-test):
- //internal/httpclient:httpclient_test: unsharded (137-228s) -> 2 shards
(76s / 97s measured).
Bead: ga-vnycm2.3
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(dolt): seed the BatchGetter cap case without prefix validation
The dolt-server leg's shared test store is configured with prefix "test",
so the batch CreateIssues path's prefix validation refused the fixture's
bge-* ids (the per-row CreateIssue the other seeds use does not validate).
Bead: ga-vnycm2.3
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): shard //internal/storage/embeddeddolt:embeddeddolt_test 9 ways
At 8 round-robin shards one shard drew the package's three heaviest tests
(153s on rbe-west in this PR's test lane); 9 spreads them.
Bead: ga-vnycm2.3
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): serve gazelle's go_repository tools from the repository cache
Every bazel.yml lane starts from a fresh output base, and before any go_deps
repository can be fetched gazelle compiles fetch_repo and gazelle with
`go build` in @bazel_gazelle_go_repository_tools: 18-24 s of every 2 vCPU
lane's loading phase (the top ci-analytics repo_fetch entry in the
embedded, proxied, cmd-dolt and test lanes).
tools/bazel/go_repository_tools.bzl replaces that repository (override_repo
on gazelle's non_module_deps extension). The build is reproducible, so the
three binaries are pinned by sha256 per gazelle release, Go SDK and host.
The rule asks the repository cache for the pinned digests first (Bazel
re-hashes every hit, so a restored runner cache can only yield the pinned
bytes); on a miss it builds exactly as gazelle does, inside env -i, and
stores a matching build in the repository cache. An unpinned or stale
combination still builds from source and prints the digests to pin.
scripts/bazel_go_repository_tools_test.go keeps the pins in step with
MODULE.bazel's gazelle and go_sdk versions. The runner cache key already
includes MODULE.bazel, so the first push to main after this saves them.
Analysis-only (--nobuild) per lane on Blacksmith 2 vCPU, main vs this change
with a seeded runner cache (ga-vnycm2.8): test 54.0 -> 38.3 s, embedded
53.6 -> 31.1 s, integration 49.6 -> 35.2 s, doltserver 48.5 -> 33.0 s,
proxied 46.6 -> 31.6 s, server-storage 33.5 -> 17.3 s, cmd-dolt 50.3 -> 33.0 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): pin gazelle's go_repository tools for Go 1.26.9 too
Ready for the next go_sdk bump (gascity#7387 moves to 1.26.9 for the
2026-10-08 advisories): the same binaries keep coming from the repository
cache instead of failing the pin test.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): pin the slowest cmd/bd Dolt-server tests to their own shards
The cmd-dolt lane's wall time when its tests re-run is the bd_test race
compile plus its slowest shard. rules_go shards round-robin by test index,
so whichever shard drew TestSharedServerConcurrent and the TestInit*
family ran 117-170s while the median shard ran ~55s.
tools/bazel/go_test_pinned_shard.sh runs a go_test binary with the tests a
committed manifest pins (measured >= 5s, bin-packed longest-first by
tools/bazel/pin_shards.py) in shards of their own, and every other test in
the binary's own round-robin shards with the pinned set skipped, so each
test still runs exactly once and new tests need no manifest edit.
bd_dolt_server_test goes from 16 to 32 shards (16 pinned + 16 round-robin),
regression_test from 8 to 16 (10 + 6; its go_test becomes the manual
regression_suite binary behind an sh_test of the old name).
Pinning exposed a pre-existing order dependence: TestCLI_Import failed
whenever it was the binary's first test, because runBDInProcess reused the
workspace discovery an earlier in-process execution cached and `bd init` in
a fresh directory resolved the previous one's .beads. The in-process
runners now reset beads/git discovery caches per execution.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): pin cmd/bd Dolt-server shards from two measured runs; regression in 14 shards
The bd manifest now takes each test's slowest of the 16- and 32-shard runs
(pin_shards.py over both), and regression_test drops to 10 pinned + 4
round-robin shards: its unpinned tests total ~128s, so 6 round-robin
shards were mostly fixed cost.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(scripts): place the pinned-shard tests by what they read (#7414 split)
The manifest-name scan reads every cmd/bd and tests/regression _test.go
file, so it moves to go_test_sources_test (with tests/regression's
manifest); the wrapper, generator and wiring checks stay in scripts_test.
Shared helpers live in a helpers-only file both targets compile, and
repoFiles takes #7414's minimum-file count.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): split //scripts:scripts_test by what it reads; partition //:repo_files
//scripts:scripts_test declared //:repo_files (the whole checkout), so every
PR re-ran it, and it took ~181s on rbe-west. Most of that (60-79s) was
TestBazelGatedLanesNeverRetryFlakyTests line-regexping the --run_under
wrapper's pinned dolt binary (tools/bazel/hermetic_bin/dolt, in every
test's runfiles); it now skips binary files, as git grep -I does.
tools/bazel/go_srcs.py's per-package repo_files block now partitions each
package's files into repo_go_srcs, repo_go_test_srcs, repo_doc_files and
repo_other_files, aggregated at the root; //:repo_files is their union
(same 4679 files). The //scripts tests are split into go_test targets by
what they read (traced per test with strace over a runfiles-shaped tree):
scripts_test CI/Bazel policy: //:repo_other_files (+1 Go file)
shell_scripts_test script fixtures: //scripts files, a few root files
go_test_sources_test _test.go scans: //:repo_go_test_srcs + shard files
go_sources_test check-testing-short.sh: every .go file
tracked_files_test every tracked file: //:repo_files
Shared helpers move to helpers-only files; TestScriptsTestTargetsPartition-
TheTests fails on a test file in no target or in two. //test/docsync and
the //scripts/repochecks slices declare partitions or single files instead
of //:repo_files where they can.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): skip only the non-checkout hermetic_bin dir in the flaky-retry walk
Skipping every binary file also stopped scanning tracked binaries (images,
.binpb fixtures). Skip just tools/bazel/hermetic_bin, the --run_under
wrapper's runfiles directory, which is not in the checkout, so the walk's
coverage of tracked files is unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(httpclient): lift the HTTP client store and roles from bd-enterprise (MIT contribution)
Contributed by gascity from bd-enterprise (github.com/gascity/bd-enterprise,
internal/enterprise/httpstore @ 49d1df2f6) to OSS beads under the MIT
license, with the owner's explicit approval. Every lifted file carries a
provenance header naming its source path.
Lifted into internal/httpclient: the Store (DoltStorage over the wire), the
dial/register glue, the read roles (reader, querier, counters, edges,
relations, graph, tree, cycles, stats, config, memories, journal), the write
roles (lifecycle, claimer, releaser, deleter, dependency editor, metadata
CAS, commenter, batch closer/creator/applier, ready claimer, sweeper), and
their tests. The store transport seam (WireClient/WireDialer) and the
store-dependent tests deferred from the S2 lift are restored here.
Not lifted: the hosted gateway (httpgw/gwauth), box-login, storage profiles
and gasworks. dial.go only mentions the gateway in comments.
This commit is the mechanical lift and does NOT build yet. It references
enterprise-only APIs absent from OSS: issueops.ReadyLister,
journalops.Watcher, storage.ServerEnforcedPolicy, the
NonCommitGraphBackend/RemoteWorkspaceBackend markers, DoltStorage
differences, and BatchCloseRequest.claim_next. The following commits
reconcile it with OSS (S3/S4/S5).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(httpclient): reconcile the bd-enterprise lift's compile-time API surface against OSS
The mechanical lift (02a9dad17) carried several bd-enterprise-only shapes
that do not exist in OSS beads:
- ReadyLister, journalops.Watcher/StreamWire, and ServerEnforcedPolicy have
no OSS wire counterpart; drop the role, its served/journal-watch tests,
and the policy enforcement path entirely rather than leave dead code that
cannot compile against OSS's apigen types.
- claim_hook_gate_test.go exercised a hook surface OSS does not publish.
- recordingWire/fakeWire no longer embed a StreamWire double now that no
read role can reach a watch stream; testTarget's two conflicting
package-scope declarations (store_test.go and helpers_test.go) are split
apart as pinnedProjectTarget vs. the generic helper.
- unsupported_gen.go is regenerated from scratch against OSS's actual
storage.DoltStorage via a new, from-scratch reflection-based generator
(internal/storage/unsupportedgen) rather than hand-copied from
bd-enterprise's generated output; completeness_test.go's
legitimatelyUnsupported map grows the handful of OSS-only or
previously-missed methods (AddRemoteWithRef, RenameLabel, etc.) this
regeneration surfaces.
- write_ledger_test.go and the divergence ledger doc gain the
force_notes_overwrite and claim pending rows OSS's apigen.UpdateRequest
and SweepRequest actually carry, and drop the Remaining/LiveDependent
assertions SweepResult/SweepSkips have no wire member to ever populate.
go build ./... and go vet ./... are clean after this change. No behavior
beyond the removed surfaces changes here; read-role and write-role
reconciliation land in follow-up commits.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): stop sending listIssues a reverse/sort vocabulary OSS does not publish
The bd-enterprise lift's sortedPage pushdown assumed GET /v0/beads/issues
(listIssues) shared the same sort vocabulary and a reverse parameter with
GET /v0/beads/issues:query (handleQueryIssues) -- a different operation.
OSS's openapi.v0.yaml publishes listIssues a CLOSED two-value `sort` enum
(created, priority; deliberately smaller than the nine values :query
takes) and no `reverse` parameter at all. Sending either would earn an
unknown-parameter or invalid_value refusal from the real server, not a
silently-wrong answer -- which is why this only surfaces against a served
backend, not a Go-side unit test of the request builder.
Add listSortPushdownEligible to gate the pushdown fast path to the two
keyset orders listIssues can actually page, unconditionally refusing any
reversed request (listIssues has no parameter that names a reversed
direction, so a reversed request of any order walks and sorts
client-side). Drop the unconditional `reverse` send from sortedPage.
served_list_test.go's three affected cases encoded the wrong,
enterprise-only wire shape (expecting reverse=false on the fallback path,
expecting a reversed order to push down with sort+reverse, and expecting a
Go-side/SQL-side mix in listIssues' sort enum); corrected to match OSS's
actual, provable wire contract rather than left red. list_sort_test.go's
publishedParamEnum helper now reads the OpenAPI document via the
go:embed'd spec.OpenAPIV0() accessor instead of a relative os.ReadFile, so
it no longer depends on the test's working directory (this also fixes the
same test failing under Bazel's sandbox, which does not expose arbitrary
relative-path files).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): reconcile write-role tests against OSS's BatchClose/Update shapes
OSS's apigen.BatchCloseRequest and BatchCloseResponse publish no
claim_next/claimed_next member at all (ledger row
W-CloseBatchRequest.ClaimNext) -- bd-enterprise's claim-folded-into-close
shape has no OSS wire counterpart. Replace the three tests that exercised
decoding a served claim (TestBatchCloseRefusesAServedClaimCarryingNoIssue,
TestBatchCloseRefusesAServedClaimTheRequestNeverEarned,
TestBatchCloseKeepsTheClaimItEarned) with
TestBatchCloseRefusesClaimNextUnconditionally, which pins the actual OSS
behavior: a CloseBatchRequest naming ClaimNext refuses before any dial, on
both the unserved and served legs, regardless of what a server would
otherwise answer. Drop the now-dead "claim-next" case and helper claim
variable from TestBatchCloseServesTheSingleItemShapeAndRefusesTheRest.
Also:
- Add the ForceNotesOverwrite case to
TestUpdateRefusesEveryMemberTheWireExcludes's hand-maintained member
map -- a ledgered-pending UpdateRequest member (see
W-UpdateRequest.ForceNotesOverwrite) the lift's test left uncovered.
- Remove write_roles_test.go's duplicate `ptr[T]` generic helper, which
redeclared the one already in helpers_test.go and never compiled.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(conformance): pin post-write RowVersion propagation across Update/Close/Reopen
types.Issue.RowVersion is json:"-", so an implementation that forgot to
populate it on the way back from storage would still compile and still
pass every existing content assertion in lifecycle_close_reopen_contract.go
-- none of them read RowVersion. Add
RunLifecycleResultsCarryThePostWriteRowVersion, which chains each write's
result token into the next write's ExpectedVersion (Update -> Close ->
Reopen) with no intervening read, so a dropped or stale token fails closed
on the next write's optimistic-concurrency guard rather than passing
silently.
Register it in role_bundle_cases.go's roleContractCases dispatch table at
its declaration-order position -- TestRoleContractCasesCoverEveryContractCase
was failing because the function existed with no dispatcher entry.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* chore(bazel): sync BUILD files via gazelle for the S3 reconciliation
bazel run //:gazelle after the compile/read-role/write-role reconciliation
commits:
- internal/httpclient/BUILD.bazel: regenerate srcs/deps for the files
added, removed and newly imported above (drops the deleted
policy/journal-watch/readylister files, adds the spec dep for
list_sort_test.go's embedded OpenAPI read). Manually add
`data = glob(["*.go"])` to the go_test rule, matching the idiom already
used by //internal/httpclient/encode: completeness_test.go and
divergence_citation_gate_test.go read their own package's .go sources at
runtime (a regex scan of unsupported_gen.go and an AST walk of the
package directory), and Bazel's go_test sandbox only copies in files a
rule declares as `data` -- `srcs` are compiled away, not left readable.
A `# keep` directive protects the attribute across future gazelle runs
(confirmed gazelle does not manage `data` on this rule).
- internal/storage/BUILD.bazel,
internal/storage/unsupportedgen/BUILD.bazel: pick up the new
from-scratch unsupportedgen generator package.
bazel test //internal/httpclient/... //backend/conformance/... now passes
all 4 targets (previously 4 of the package's tests failed only under
Bazel's sandbox: TestInterfaceCompleteness,
TestTheCountGroupVocabularyMatchesTheSharedOne,
TestFlaglessListSortIsTheSameOrderNamed,
TestDivergenceParkCitationsResolveToLiveLedgerRows).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(conformance): close two pre-existing leg-wiring gaps found while registering http
- RunLifecycleResultsCarryThePostWriteRowVersion (added in 536561124) was
never wired into dolt/embeddeddolt/uow's own lifecycle_close_reopen test
tables, only into the http leg's. TestEveryLegWiresEveryRoleContract
catches this once a fourth leg forces the test to actually run again;
wire the missing t.Run into all three.
- The three StatsReporter "suppressed rows" cases (GateIssues/TemplateIssues
breakdown) were never added to served_graph_aux_test.go even though the
wire already carries both fields end-to-end (types.Statistics is shared
verbatim by apigen.StatsResponse and the client's decode). Wire them; all
three pass over HTTP with no server or client change needed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(conformance): register the http leg in the role-contract lock
Registers "http" (wiringRoot internal/httpclient) in contractLegRegistry.
The served-surface harness (served_harness_test.go + 16 served_*_test.go
files) already wired 495 of 576 role-contract entrypoints from the
bd-enterprise lift; this change:
- Waives the 3 RunCycleDetectorIncludeTracks* cases: issueops.
DetectCyclesRequest.IncludeTracks has no plumbing on either side of the
wire today (cycles.go always calls DetectCycles with a zero-value
request; the client's DetectCycles ignores its request parameter). No
slice through S13 adds one, so the waiver says a new slice is needed
rather than pointing at an existing one.
- Sets adoptionCeiling: 75 for the remaining, not-yet-wired write-role and
internal-plumbing entrypoints (lifecycle writes, DependencyEditor,
MetadataCAS, Lease, Sweeper, VersionReconciler, Bootstrapper/
InitVerifier, cross-record invariants) that S4 and later write slices
pick up and ratchet down.
- Adds //internal/httpclient:go_srcs to storage_test's data: the leg's
wiring root lives outside internal/storage, and the Bazel sandbox has no
other reason to materialize those sources for a test that reads them at
runtime.
TestEveryLegWiresEveryRoleContract, TestEveryRoleMethodHasAContractCase,
and RunUnsupportedContract (already wired pre-lift in
internal/httpclient/completeness_test.go) all pass for the http leg.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): close review's HIGH 2-5 gaps and reconcile their test doubles
HIGH 2 (BatchCloser), HIGH 3 (created_by stamping, absent-parent 404/400,
force_notes_overwrite carriage) and HIGH 4 (CapabilityError now unwraps to a
typed *storage.ErrUnsupported with Capability) were implemented in prior
commits on this branch; this change finishes HIGH 5 and the verification the
review asked for:
- role_reader.go's Reader.Get and bridge.go's raw GetIssue both now stitch
Revision back onto Issue.RowVersion via parseRevision, so a guarded write
built off either read carries the row's real token instead of a decoded
zero. Both are pinned by round-trip served tests that guard a write with
the read's token and require a stale-token replay to refuse.
- Fixed create_test.go's stale assertion that created_by must stay empty
(contradicted the HIGH 3 stamping fix it predated).
- Fixed recordingWire's canned IssueDetails{} response (roles_test.go): it
omitted Revision, which is a required field on every getIssue response
per the OpenAPI spec, and the HIGH 5 stitch now reads it unconditionally.
- Synced internal/httpclient/wire/BUILD.bazel via gazelle: handshake_test.go's
new internal/storage import had no corresponding dep.
- gofmt'd write_roles_test.go's stubWire struct tags.
Verified: served HTTP conformance tier (BEADS_TEST_EMBEDDED_DOLT=1
BEADS_HTTP_TEST_REQUIRED=1 go test ./internal/httpclient/, -timeout=40m)
600 PASS / 0 FAIL / 72 SKIP (all ledgered known-divergence waivers). One
TLS-handshake-timing test failed in that run and passed 3/3 in isolation
(unrelated file, pre-existing flake under machine load, not reproducible).
Also green: default go test (minus the pre-existing, unrelated
TestEngdocsAndRootMarkdownLinks doc-link drift and the environmental
internal/storage/uow dolt-server-spawn hang), bazel build/test for the
touched packages, make api-check, and gofmt/vet.
Live-mutated and reverted 3 guards to confirm their tests are load-bearing:
role_reader.go's RowVersion stitch, bridge.go's RowVersion stitch, and
lifecycle.go's force_notes_overwrite wiring.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(ci): add served HTTP conformance as Tier 3 of the conformance gate
The served tier (internal/httpclient's served_*_test.go corpus) never ran
in CI, so HIGH 2-5's regressions (and any future ones) would merge
undetected. Add it as Tier 3 of scripts/conformance.sh, the existing
required entrypoint conformance.yml already runs on every PR/push/merge_group
(not a new workflow, not wired through ci-gate.sh's aggregation -- this
repo's convention for this gate is a single standalone job):
CGO_ENABLED=1 BEADS_TEST_EMBEDDED_DOLT=1 BEADS_HTTP_TEST_REQUIRED=1 \
go test -timeout=40m ./internal/httpclient/
Left unsharded: Tier 1 and Tier 2 are themselves single unsharded
invocations, and Tier 3 is one Go package, not a multi-package corpus the
repo's existing package-sharding scripts apply to. No Bazel-lane addition:
bazel-coverage's embedded/dolt_server outputs only ever substitute for
main.yml/pr-risk.yml jobs, never for conformance.yml's own job, so there is
no existing Bazel-lane relationship for this gate to extend.
Bumped the job's timeout-minutes 45 -> 100 (30m + 10m + 40m budgeted
sub-timeouts plus checkout/setup-go overhead) and updated the three policy
tests that pin this script/workflow byte-for-byte
(TestConformanceScriptUsesExplicitTimeoutBudgets,
TestConformanceScriptPropagatesGoTestFailures,
TestConformanceWorkflowHasOuterTimeoutBudget), re-included
internal/httpclient/** in both path filters (push and pull_request, kept
identical) since served_*_test.go files are the suite itself and would
otherwise be excluded by the shared !**_test.go negation, and documented the
new filter entry in ci_f7c_advisory_test.go's own-entries map.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): root-cause and fix the Happy-Eyeballs TLS flake in store_dependent_test.go
TestTransportForSucceedsWithEnvCAFile (and 3 Handshake-based siblings that
assert on a successful dial through a configured CA: SidecarCAFile,
CAFileEnvAgreeingWithSidecarSucceeds, DialWithNilTransportHTTPClientGetsCAInjectedOnACopy)
flaked once with "tls: first record does not look like a TLS handshake".
Root cause: our code, same class S2 already fixed for TransportFor's direct
callers (forceIPv4Loopback in ca_test.go/ca_wrong_ca_test.go). Every server
these tests dial is addressed by hostname "localhost" (ca.startServer
rewrites it there deliberately, so SNI is actually sent), which Go's
default dialer races a tcp6 dial to ::1:<port> against the real tcp4 dial to
127.0.0.1:<port> for (RFC 6555). An unrelated local process already bound to
a colliding ephemeral port on ::1 can occasionally win that race before the
real dial completes, and the client speaks TLS to that unrelated process
instead -- the misleading handshake error, not a real hang or a product bug.
These four tests go through Handshake with DialOptions{} (HTTPClient left
nil deliberately, since that is itself the nil-HTTPClient branch of DialWith
under test), so they never get a *http.Transport back to mutate the way
forceIPv4Loopback's existing callers do. Fixed via warmIPv4OnlyCATransport:
pre-build the target's CA-scoped transport through the same public
TransportFor the product code itself calls (so it lands in the exact
(path, content)-keyed cache transportForFile's cache hit returns), force
tcp4-only on that cached pointer, and let Handshake's own DialWith run
unchanged -- same branch under test, same cached transport, no sleeps, no
retries. Verified with -race -count=50 on all four: 200/200 pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): refuse DetectCycles(IncludeTracks=true) instead of silently ignoring it
roles_graph.go's httpCycleDetector.DetectCycles took its request as `_`,
forwarding nothing: a caller asking for the widened (tracks-following) walk
silently got the narrower default walk back with no error. listDependencyCycles
has no include_tracks wire parameter at all, so there is no way to honor the
field — only to refuse it loudly or drop it silently. Switch to the typed,
ledger-cited refusal (encode.RefusedError via refuse(), citing new row
"L-cycles-tracks") that CountEdges' own wire bound already uses, add the
ledger row and its Op, regenerate the auto-generated ledger doc, and pin the
behavior with a unit test (no served-tier case needed: the refusal is
pre-dial by design, same reasoning as the existing edge-count bound pin).
Also fix leg_contract_wiring_test.go's cycleTracksWireGapWaiverReason, which
had gone stale: it described the http leg as silently forwarding nothing for
this field, which was true before this change and is not after. The waiver
itself (the three RunCycleDetectorIncludeTracks* contract cases stay unwired
for the http leg) is unaffected — they were never wired, and still are not —
only the reason text needed updating to describe a typed refusal instead of a
silent ignore.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(storage): name the http leg's permanent contract gaps instead of budgeting them
The http leg's role-contract adoption ceiling (internal/storage/contract_leg_registry_test.go)
sat at a flat 75, with a prose "adopting" note that bundled every skip into
one vague bucket — including MetadataCAS, which is in fact fully wired (0
missing cases), a false claim left over from an earlier wiring pass.
Twenty-six of the 75 skips are not S4-and-later work at all: they are
PERMANENT gaps that no future slice closes, because the contract itself has
no publicops role for an HTTP client to route through:
- Importer (4 cases): uow is Importer's one accessor, same precedent the
dolt and embeddeddolt legs already carry (importerOneAccessorWaiverReason).
- RetentionFixture/EpochFixture (15 cases) and CrossRecordInvariantFixture
(7 cases): both fixtures carry raw storage hooks (Resolve, Erase, Mint,
CurrentEpoch, GuardedWrite, EnforceCrossRecordInvariant, ...) with no
publicops field at all. dolt, embeddeddolt and uow wire these because
they ARE the storage engine this address/epoch/retention and
cross-record transaction bookkeeping belongs to; the http leg is a
remote client with none of its own to report. There is no wire surface
for this to ever route through short of a new role being designed from
scratch — a new slice, not a missing test line.
These 26 are now named, reasoned waivers in unwiredContractEntrypoints
(two new reason constants: retentionEpochNoRoleWaiverReason and
crossRecordInvariantNoRoleWaiverReason), and the ceiling drops from 75 to 49
to match. The "adopting" note is rewritten to correctly name every remaining
category under budget (lifecycle writes, DependencyEditor, DualWrite,
Sweeper, VersionReconciler, Bootstrapper/InitVerifier, ExpectedRevision/
Refusal) and to drop the false MetadataCAS claim.
Also documents, at conformanceEntrypointsWiredBy, why a leg wiring a
conformance.RunX reference with every hook left nil would read as "adopted"
by this lock's static source grep while actually running as a skip —
matching the loophole internal/httpclient's divergence_citation_gate_test.go
already closes for skipKnownDivergence call sites. No leg currently does
this (verified by inspection); the note sets the review expectation for
future wiring rather than papering over the gap with no record of the policy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(licensing): scrub internal program/org naming and a leaked private ID from OSS-visible text
MEDIUM 8 of the S3 ship review: remove licensing-hygiene leaks found by a
read-only repo-wide survey, while leaving every legitimate "Contributed by
gascity from bd-enterprise ... to OSS beads under the MIT license." provenance
header and every genuine public gascity/gastownhall self-reference (the real
OSS org, beads.gascity.com docs domain, RBE infra) untouched:
- Delete release-gates/ga-7psli2-3-1-baseline-cmd-bd-fix-gate.md: the only
file in release-gates/ keyed by a private `ga-` tracker ID instead of the
directory's established public `be-` convention (30 other files), and the
only one leaking internal filesystem paths (/home/jaword/...,
gc-management/.gc/artifacts) and internal routing terms (mayor/mpr,
"deployer session"). Not referenced from anywhere else in the tree.
- .golangci.yml: drop two private `ga-2ltro.12`/`ga-26w10` history citations
used as nolint-comment justifications; reworded to the same explanatory
content without naming a private tracker ID.
- Reword 19 "gc native-program" occurrences (14 files, comments and one
design doc) to drop the internal program name while keeping the "S3
reconciliation (2026-10)" dating they were attached to.
- cmd/bd/if_revision.go, if_revision_test.go, update.go,
update_proxied_server.go: drop literal citations of gascity's private
bdstore_conditional.go classifier (including a test function name,
TestIfRevisionEnvelopeGascityContract -> ...DownstreamContract) while
keeping the substantive explanation of why this envelope's shape is
load-bearing for a downstream consumer.
- internal/httpclient/lifecycle.go: drop a citation of gascity's private
internal/beads/bdstore.go.
- internal/httpclient/dial.go: drop the internal/enterprise/httpgw package
path, keeping "a gateway dialer outside this repo".
- internal/httpclient/register.go: reword away from naming "the enterprise
distribution" while keeping the build-tag-opt-in behavior it documents.
Verified: go build ./..., go vet ./internal/httpclient/... ./cmd/bd/...,
go test ./internal/httpclient/... and ./cmd/bd/... -run TestIfRevision all
pass; TestDivergenceLedgerDocMatchesLedger still passes (ledger.go and its
doc were edited in lockstep). gofmt -l reports no diffs on any touched file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient/wire): correct the CapExternalDependencies doc's implied false claim
MEDIUM 9 of the S3 ship review. The old comment on CapExternalDependencies
said this client "has no external-dependency policy layer to gate" and that
"OSS publishes no storage.ExternalDependencyQueryStore-backed client policy
... to compose" immediately after naming bd-enterprise's removed policy.go --
easily misread as "OSS has no external-dependency decorator at all." It does:
internal/storage/externaldeps's decorator is unconditional (no build tag) and
is wired into every local storage chain by cmd/bd/storage_chain.go.
Reworded to say precisely what is and is not true: the decorator exists and
runs for local backends; what this http client lacks is a client-side
composition of that decorator around a remote store, which is moot anyway
because a server advertising this capability already applies the policy
itself before answering. (Rolled in the same "gc native-program" wording
removal from the MEDIUM 8 commit, since both edits touch this comment block.)
A second claim from the same review item -- that OSS is said somewhere not to
serve events:watch -- could not be located despite an exhaustive repo-wide
search (this file, engdocs/, docs/reference/events-journal.md,
SERVE_RUNBOOK.md, the divergence ledger, CHANGELOG.md, and a dedicated
read-only research pass). internal/httpapi/events_watch.go,
routes.go's events.watch route entry, and internal/httpclient/wire/stream.go
all confirm OSS serves it unconditionally (gated only per-workspace by the
events journal setting, not by build/distribution). No corresponding false
claim exists in this worktree to fix; noting this rather than editing
something that already reads correctly.
Verified: go build ./internal/httpclient/..., go vet, go test
./internal/httpclient/wire/... pass. gofmt -l clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(unsupportedgen): add a freshness check for the generated shell
LOW item from the S3 ship review: internal/storage/unsupportedgen had no
tests of its own, so the generator could drift from the checked-in
internal/httpclient/unsupported_gen.go (a storage.DoltStorage interface
change, or a hand-written method added/removed on Store) with nothing to
catch it short of a human noticing the diff. TestInterfaceCompleteness
(completeness_test.go) already pins which methods belong in the shell by
name, but not whether the checked-in bytes match what the generator would
produce today.
TestGeneratedShellIsFresh runs the same scan-then-generate pipeline main()
runs for the registered DoltStorage/httpclient/Store consumer and diffs the
result against the checked-in file -- mirrors the existing
TestDivergenceLedgerDocMatchesLedger pattern in
internal/httpclient/encode/ledger_render_test.go. Added
TestScanHandWrittenMethodsExcludesOwnOutput alongside it to pin the
correctness property the freshness check depends on (scanning must not see
the generator's own prior output as hand-written).
BUILD.bazel: added the go_test rule by hand (gazelle cannot infer the
cross-package `data` dependency a relative-path os.ReadFile needs under the
Bazel sandbox) with data = ["//internal/httpclient:go_srcs"], following the
existing cross-package data pattern in
internal/httpclient/encode/BUILD.bazel. Placed go_test before the
go_srcs.py-managed filegroup block, which `make bazel-sync-check` requires.
Verified: `bazel run //:gazelle` leaves the rule untouched, `bazel test
//internal/storage/unsupportedgen:unsupportedgen_test` passes, `make
bazel-sync-check` is clean, go build/vet/test ./... all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient/encode): scrub a private tracker-ID cluster from the shipped divergence-ledger doc
Follow-up to the MEDIUM 8 licensing scrub: internal/httpclient/encode's
ledger_render.go header and ledger.go's comments/pinnedElsewhere constant/two
Why strings cited "ga-b8ddd.12/.19/.23/.30" by name. These feed directly into
engdocs/design/http-divergence-ledger.md -- a shipped, generated OSS design
doc, the most visible surface this private ID cluster reached (Fork A's
MEDIUM 8 survey found ~40 occurrences of "ga-b8ddd" across the httpclient
package; this commit addresses the subset inside package encode that is
checked-in prose/doc text, not test-fixture or assertion values). Reworded to
keep the same substantive explanation (what closed, what's still deferred,
which dual run resolved L17) without naming the private ID.
The remaining ~30 occurrences of "ga-b8ddd" outside package encode
(offrole.go, metadata.go, store.go, roles_test.go, wire/*_test.go,
served_*_test.go) are left for a follow-up pass: scrubbing them individually
needs the same per-file verification this commit did, and is a larger,
separate blast radius than was practical in this pass.
Regenerated engdocs/design/http-divergence-ledger.md via
`go test ./internal/httpclient/encode -run TestDivergenceLedgerDocMatchesLedger
-update-ledger-doc`; diff confirmed only the ga-b8ddd text changed. go build
./..., go vet ./internal/httpclient/..., and go test ./internal/httpclient/...
all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(httpclient): drop a private host reference from a test comment
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* revert(licensing): keep the scrub to the contributed client code
The previous scrub also edited upstream files this PR doesn't otherwise
touch (.golangci.yml, cmd/bd --if-revision files from #7203) and deleted an
upstream release-gate record. Restore them; the provenance scrub applies only
to the code contributed from bd-enterprise.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(httpclient): wire S8 count-scope fields on the client
Encode ParentID/NoParent/ExcludeTypes/ExcludeStatus onto countIssues
(table.go + encode.go), add issues.count.scope to the client capability
table (wire/handshake.go, wire/vocabulary_test.go), and refuse locally
before dialing when the server hasn't advertised the token and any scope
field is set (counter.go's refuseUnservedScope), returning a typed
*storage.ErrUnsupported{Capability:"issues.count.scope"} rather than
dropping fields or taking a guaranteed-400 round trip.
Adds encoding-parity coverage (counts_test.go), a masked-token skew test
proving the pre-dial refusal and zero dials (counts_test.go), roundtrip
and skew-table fixes for the newly-encoded fields (encode/roundtrip_test.go,
encode/skew.go), and wires S8's nine shared Counter conformance cases into
the served http leg (served_counter_test.go), including AddDependency so
the wisp/compose cases run instead of skipping. List stays nil on this
fixture since listTable() has pre-existing, unrelated permanent refusals
for ListRequest.NoParent/ExcludeTypes, so the three *MatchesListCardinality
cases skip here by design.
internal/storage's TestEveryLegWiresEveryRoleContract/http ceiling check
remains red (53 skipped vs ceiling 49) after this change, but strictly
improved: before this change the inherited baseline was 62 skipped (13
over ceiling), all nine of S8's new entrypoints unwired. This change wires
all nine, closing exactly that gap; the remaining 4-over are named
RunBootstrapper*/RunDependencyEditor* cases unrelated to count-scope,
pre-existing debt from other already-merged slices whose ceiling was
never bumped for them. Left the ceiling constant alone rather than
raising it to paper over debt this change doesn't own.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(storage): close the http leg's role-contract ceiling regression
TestEveryLegWiresEveryRoleContract/http went red after the rebase (62
skipped, 13 past its ceiling of 49) before any work in this session; the
prior commit closed 9 of those (S8's Counter cases). The remaining 4-over
were RunBootstrapper* (9 entries) and RunDependencyEditor* (2 entries),
both newly-merged contracts this leg had never classified.
RunBootstrapper*: internal/httpclient/accessors.go already names
Bootstrapper PERMANENTLY UNSERVABLE over http (bd init resolves its
substrate identity entirely client-side, before any server exists to
dial) — add a new named waiver, bootstrapperPermanentlyUnservableWaiverReason,
covering all nine entrypoints. VersionReconciler and InitVerifier are
named permanently unservable by the same accessors.go comment but are
left under the ceiling for their own reviewed change, out of scope here.
RunDependencyEditor*: the OSS server already serves add/remove and the
http leg already wires nearly all of DependencyEditor's contract. The two
missing cases (SameTypeReAddWithChangedMetadataMintsOneVersion,
SameTypeReAddWithIdenticalMetadataIsANoOp) only needed
SetVersionedHistoryEnabled wired onto the fixture, the same way
SetJournalEnabled already is — wired it for real; both pass served.
With Bootstrapper waived and DependencyEditor fully wired, the leg's
unwaived-missing count drops from 53 to 42, so adoptionCeiling moves
from 49 to 42 to keep the ratchet exact, with the adopting prose updated
to match.
TestEveryLegWiresEveryRoleContract is green. Served tier (real, CGO +
embedded dolt): 611 PASS, 0 FAIL, 72 SKIP (up from 609). bazel build/test
and make api-check pass for the touched targets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37307836753
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
* fix(conformance): pass -tags gms_pure_go to Tier 3's served httpclient run
scripts/conformance.sh:61 ran a bare `go test -timeout=40m ./internal/httpclient/`
for Tier 3 (served HTTP client/role conformance), which `make ci-pr-policy`'s
build-tag policy check refuses ("bare go test without -tags=gms_pure_go").
Pass the same -tags Tier 1 already does, and update
TestConformanceScriptUsesExplicitTimeoutBudgets' pinned argv to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): give //internal/httpclient:httpclient_test room to run
The package takes ~350s under --config=ci (BEADS_TEST_SKIP=dolt, no
BEADS_TEST_EMBEDDED_DOLT) — mostly wall-clock wait on the CA tests'
dialer/keep-alive timeouts, not CPU — but the go_test had no size/timeout
override, so it inherited the medium default's 300s and timed out
intermittently under Bazel (and took "Go test equivalence" down with it).
Set size = "large", timeout = "long", matching
//internal/storage/embeddeddolt's go_test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): fix data race in caContextHandler test helper
TestTransportForRaisesMaxIdleConnsPerHostForBursts drives caContextHandler
with 8 concurrent requests; handler() wrote requestSeen/host/sni from every
server goroutine with no synchronization, and the other nine callers (all
single-request) never exercised the race. go test -race reproduced it as
3x DATA RACE across repeated runs.
Guard the fields with a mutex and expose them only through accessor methods
(sawRequest/seenHost/seenSNI), updating every call site in
store_dependent_test.go accordingly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(unsupportedgen): canonicalize reflect-visible stdlib aliases
TestGeneratedShellIsFresh fails on Go >= 1.27: encoding/json.RawMessage
became an alias of encoding/json/jsontext.Value, and reflect only ever sees
an alias's underlying type, so formatType rendered
storage.DoltStorage.MergeMetadata's signature with the new, non-portable
name even though the signature itself did not change. A contributor on
1.27 following the freshness test's own advice ("go generate
./internal/httpclient") would commit a shell that does not compile on the
1.26.7 SDK this repo pins.
Add reflectAliasCanonical, a table mapping a reflect-visible alias target
back to the name a human wrote in source, consulted by namedType. Add an
entry here for any future stdlib alias this generator's output needs to
stay stable across.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): claimOnlyUpdate propagates the claim's RowVersion
bridge.go's GetIssue already stitches details.Revision onto
hydrated.RowVersion when claimOnlyUpdate rehydrates labels/CreatedBy onto
the claim response, but the claim response itself (issueops.Claimer's
bare-row contract) carries no revision at all, and the copy-back only
touched Labels and CreatedBy. So Lifecycle.Update({Actor, IssueID, Claim:
true}) — `bd update X --claim` — handed back Issue.RowVersion == 0 over
http where the local leg returns the real token, and feeding that into the
next write's ExpectedVersion got a spurious mismatch instead of none.
Copy RowVersion back too. Extend
RunLifecycleResultsCarryThePostWriteRowVersion (shared lifecycle/close/reopen
conformance contract) with a claim-shaped first link in the chain: claim,
then guard the next write with the claim result's own token, proving the
claim-only path populates the real version rather than a nonzero constant.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): stitch RowVersion on SearchIssues' exact-id fast path
resolve.go's getIssuesByExactID decodes the same IssueDetails shape
bridge.go's GetIssue does, but never parsed details.Revision the way
GetIssue's own stitch does, so SearchIssues(ctx, "<exact id>") answered
RowVersion 0 — a one-line gap: Issue.RowVersion is json:"-", so the decode
left it at zero, and only the explicit parseRevision stitch recovers it from
the response's own Revision string member.
Add TestSearchIssuesExactIDPopulatesRowVersionForAGuardedWrite, pinning the
same guarded-write round trip TestBridgeGetIssuePopulatesRowVersionForAGuardedWrite
already pins for the plain GetIssue path.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): route every write role's refusals through inexpressible
Review finding 7: write-side refusals were not uniformly *ErrUnsupported.
Read roles already wrap encoder refusals through Store.inexpressible, which
reaches both encode.RefusedError and encode.CapabilityError; the write side's
refuse() returned a bare *encode.RefusedError instead, so a caller doing
errors.As(err, &unsupported) caught the count-scope refusal but missed
Sweeper.Sweep(Limit), DetectCycles(IncludeTracks), ReadyClaimer.ClaimNext,
BatchApplier.ApplyBatch, BatchCloser.CloseBatch, Deleter.Delete,
DependencyEditor.AddDependencies, and BatchCreator.CreateBatch.
Give InexpressibleError a detail field that preserves the original error's
composite text (an "items[%d]..." or "Issue.%s:" prefix a deeper encoder
helper already adds) instead of collapsing the message to the bare ledger
row text, and wrap every remaining write role's exported method in a single
named-return defer calling store.inexpressible, matching the pattern
lifecycle.go's Create/Update/Reopen already use.
Add TestWriteRoleRefusalsMatchTheReadShape, pinning three write-side
refusals (Sweeper.Sweep Limit, CycleDetector.DetectCycles IncludeTracks,
ReadyClaimer.ClaimNext) now classify via errors.As(*storage.ErrUnsupported)
the same way a read-side refusal does.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): stop spelling out Status: StatusOpen on a markdown batch create
Review nit 8: batchcreator.go's refuseUnwirableIssue refuses a populated
Status (apigen.BatchCreateItem has no wire slot for it), and
buildMarkdownBatchRequest always set Status: types.StatusOpen explicitly, so
`bd create --file` refused on every call against an http-backed store.
PreparePublicCreateRequest already defaults an empty Status to StatusOpen on
every backend, including the http client store's own server, so the explicit
assignment here was never a different outcome - only a populated member the
wire can't carry. Leave it unset; the created issue is identical on every
backend, and the http leg's role-level accounting for why Status and
CreatedBy are excluded from its allowlists is documented on
batchCreateCarriedIssueMembers (internal/httpclient/batchcreator.go).
CreatedBy is left as-is: that member genuinely has no wire slot AND the
server does not yet stamp it from Actor (a gap already tracked in this PR's
Follow-ups), so its refusal is correct, not a bug to route around.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(httpapi): correct create.go's CreatedBy comment to caller-asserted
Review nit 8: the comment claimed CreatedBy is stamped from the
"authenticated actor," but actor is parsed straight from the request body
(s.bodyActor) and never authenticated - exactly as this file's own doc
comment states a few lines above. Reword to "caller-asserted actor" and
cross-reference that doc comment so the two stay consistent. No behavior
change: the stamp itself and its test coverage were already correct.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): raise MaxIdleConnsPerHost for a supplied client's nil Transport
Review nit 8: DialWith's doc comment promises the raised MaxIdleConnsPerHost
ceiling on every dial, "whether or not a CA is configured," but the
opts.HTTPClient.Transport == nil arm only injected a transport when the
target's CA was configured (resolved.configured()). A caller-supplied
*http.Client with a nil Transport and no CA fell through untouched, so it
defaulted to http.DefaultTransport's own ceiling of 2 at request time instead
- silently exempting this one shape from the doc's promise and forcing a
fresh TLS handshake on most of a concurrent burst even once the pool is warm.
Route this arm through the same dialTransport helper the opts.HTTPClient ==
nil arm above already uses (the CA-scoped transport when a CA is configured,
the shared baseline otherwise), injected onto a copy so the caller's own
*http.Client fields (Timeout, CheckRedirect, Jar) stay untouched.
Add TestDialWithSuppliedClientNilTransportNoCARaisesMaxIdleConnsPerHost: an
end-to-end burst through wire.Client.Do, traced with httptrace, asserting the
connection pool settles into reuse at a burst size the default ceiling of 2
cannot sustain.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(httpclient): stop citing nonexistent design docs from new source files
Review nit 8: several comments added by this lift cited
engdocs/design/http-client-backend.md, "D8 row N", and beads-design/slices.json
- none of which exist in this repository (the design lived in the
bd-enterprise-internal source this code was lifted from).
Point each citation at what actually ships in-repo instead:
engdocs/design/http-divergence-ledger.md (the generated, human-readable
rendering of the divergence ledger in encode/ledger.go) for the D7/D8/D9
discipline citations in encode.go, ledger.go, bijection_test.go,
write_ledger_test.go, wire/credential.go and store.go's package doc; and a
direct description of the wire-gap it records, rather than a slices.json
scope reference, in leg_contract_wiring_test.go's cycle-tracks waiver
rationale. Regenerate http-divergence-ledger.md from ledger_render.go (which
had the same stale citation) via
`go test ./internal/httpclient/encode -run TestDivergenceLedgerDocMatchesLedger -update-ledger-doc`
so the shipped doc and its generator agree.
No behavior change; comment and generated-doc text only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(httpclient): implement issueops.BatchGetter for the HTTP client leg
The rebase onto origin/main (item 9 of PR #7247's review) brought in
upstream #7248's new issueops.BatchGetter role on storage.DoltStorage,
breaking the HTTP client leg's interface satisfaction. This wires it up
as a read role (no actor, no version guard, writes nothing — dialed as
a POST only because MaxGetManyIDs ids cannot ride a GET query string),
mirroring Counter/Querier rather than the WriteWire-wrapped roles.
- wire/paths.go: add PathIssuesBatchGet ("/v0/beads/issues:batchGet").
- batchgetter.go: httpBatchGetter.GetMany dials the server and stitches
each resolved issue's RowVersion from apigen.BatchGetIssue.Revision
(the role's only wire spelling of the row token, since RowVersion is
json:"-"). checkGetManyIDs pre-validates the cap and blank-id rules,
in the same order internal/storage/issueops.ValidateGetManyRequest
applies them, because batch_get.go's failBatchGetErr collapses every
validation failure into one generic, code-less InvalidArgument the
client could never decode back into *issueops.TooManyIDsError.
- accessors.go: add the BatchGetter() accessor (26th wire-backed role).
- unsupported_gen.go: regenerated, dropping the now-implemented stub.
- completeness_test.go: move BatchGetter into wireBackedAccessors and
bump the asserted accessor-matrix total from 28 to 29.
- served_test.go / served_harness_test.go: both package-local "by hand"
httpapi.Config builders (serveRoles, serveConfig) now wire
cfg.BatchGetter, fixing ~56 served-tier tests that broke once
BatchGetter joined httpapi.Listen's required role set.
- served_batch_getter_test.go: wire the HTTP leg into all 14
backend/conformance.RunBatchGetterXxx cases, following
served_graph_counter_test.go's pattern. This surfaced a real bug:
GetMany built Missing via append([]string(nil), body.Missing...),
which stays nil when the server reports no misses, violating the
role's "Missing is never nil on success" contract — fixed to build
via make([]string, len(body.Missing)) + copy.
Verified: go build/vet clean; internal/httpclient suite green
(including the embedded-dolt served tier); TestEveryLegWiresEveryRoleContract
holds the http leg's adoption ceiling at exactly 42 with no change needed;
TestInterfaceCompleteness and TestAccessorMatrixMatchesDesign pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37450731721
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
* test(httpclient): lock contextServer.seen against concurrent handler goroutines
TestDialWithSuppliedClientNilTransportNoCARaisesMaxIdleConnsPerHost drives
the stub with bursts of 8 concurrent requests; the handler appended to
seen without a lock. Readers only look at seen after their requests return.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): sync go_srcs managed blocks for unsupportedgen after the main merge
The PR adds internal/storage/unsupportedgen on a base that predates #7320,
which gave every Bazel package a managed repo_files filegroup. On the merge,
`python3 tools/bazel/go_srcs.py --check` (TestBazelGoSrcsBlocksCurrent) found
3 stale blocks: the package's own repo_files block, its entry in the root
BUILD.bazel repo_files list, and its entry in tools/bazel/BUILD.bazel.
`make bazel-sync` regenerated exactly those; gazelle changed nothing else.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(httpclient): refuse DepAddItem.HasSpawner and ThreadID on applyBatch after the main merge
Main's #7209 gave issueops.DepAddItem two members, HasSpawner and
ThreadID, and a conformance case, RunBatchApplyStampsSpawnerIDOnlyWhenNamed.
ApplyDepAddItem on the v0 wire publishes source, target, type and
metadata only, so on the merge the http leg dropped both silently and
two of the PR's gates went red:
- TestEveryWireExcludedWriteMemberCarriesALedgerRow: the reflective
census found two wire-excluded DepAddItem members with no ledger row.
- TestEveryLegWiresEveryRoleContract/http: the new case left the leg
at 43 unwired cases against its exact ceiling of 42.
applyDepAddItemBody now refuses each member where the role would store
something the wire cannot say, under two new KindRefuse rows,
W-DepAddItem.HasSpawner and W-DepAddItem.ThreadID. The spawner flag
refuses only on a waits-for edge, because the role ignores it on every
other type and stores the same row either way. The ledger doc is
regenerated (264 rows).
The served tier wires the new case as a park on W-DepAddItem.HasSpawner,
owned by the family's existing bead, which brings the http leg back to
its ceiling. With the skip removed the case fails on the client's
refusal; with the refusal also removed it fails on the drop the row
names: spawner_id "" where the role stamps the resolved target.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(httpclient): address #7247 relaunch review iteration 1
Review iteration 1 of the relaunched adoption raised six majors. This
commit fixes all six. Five are the majors the first adoption's
iteration 1 raised. Its fixup, 696803f250 (tag
adopt-pr/bd-f5illd/stranded-fixup-696803f250), fixed them but never
reached the branch. That fix is ported here onto the contributor's
current head, with its reasoning unchanged; the sixth is new.
Creates refused what the CLI sends. Every `bd create` carries
CreatedBy, a workspace whose config.yaml names a prefix also carries
IDPrefix, and the client refused both.
- The server stamps created_by from the actor on all three create
shapes: batch_create.go and batch_apply.go now match create.go.
- The client sends a CreatedBy that names the actor and refuses any
other value, which the server's stamp would silently replace.
createIssue and the apply create item share
refuseUnwirableIssueMembers.
- IDPrefix refuses only where the role would act on it: an explicit,
unforced id.
- `bd create --file` stays refused over http. Dropping its Status was
safe but not enough: the batch create still refuses its Provenance
and Owner. The markdown.go and batchcreator.go comments that called
it servable now say so.
keysetFilter ignored AfterPriority. It now compares priority first,
mirroring sqlbuild's KeysetPriorityCreatedAtIDPredicate, and a new
served case resumes a priority keyset position. AfterID or AfterPriority
without AfterCreatedAt no longer reads as a zero-instant position that
discarded every row (the AfterID minor).
Claim with a patch was refused on a false ledger claim. `claim`,
`force_assignee_transfer` and `force_close_policy` now ride updateBody
like `force_notes_overwrite`, so a claim with a patch or a version guard
is one updateIssue call. A claim ALONE sent to a server that predates
the member (#6890) falls back to claimIssue. The three ledger rows are
retired, and the served cases parked on them now run.
OpenReadOnly returns a writable store. It stays writable, as a
documented exception: backends.Backend.OpenReadOnly also serves the
root pre-run's classified reads, and `bd ready --claim` writes through
that store. The exception is named in the backends.Backend contract,
the store_factory invariant and the store doc, and
TestReadOnlyOpenServesTheClaimBdReadyMakes pins it. An OpenNonMutating
backend hook is the follow-up.
Security surface.
- http_target.json is in the doctor gitignore template and its required
patterns.
- BEADS_HTTP_TOKEN and BEADS_HTTP_TOKEN_COMMAND are host-scoped with
BEADS_HTTP_CA_FILE's "host[:port]=value" syntax. A value scoped to
another server falls through to the next rung, and a command scoped
elsewhere never runs.
- LoadTarget and SaveTarget refuse a url that carries userinfo, and
Target.String drops userinfo.
The served Tier 3 had no CI home. The new target
//internal/httpclient:httpclient_served_test is an `embedded`-tagged
sh_test. It reruns httpclient_test's binary through
tools/bazel/go_test_variant.sh with the script's
BEADS_TEST_EMBEDDED_DOLT=1 and BEADS_HTTP_TEST_REQUIRED=1, in 24 shards
on the embedded lane's race build. bazel.yml's bazel-embedded job
already runs `bazel test //... --config=embedded`, so no workflow
changes.
- -test.timeout=19m, not the script's 40m: Go's own timeout then fires,
and dumps every goroutine, before the action's 1200s limit.
- TestBazelRetiredLanesCannotBeNarrowed pins its args and env. The new
TestConformanceServedTierRunsUnderBazel ties the script's Tier 3
switches to the rule.
- The new untagged TestServedTierIsLinkedWhenRequired fails a required
run built without cgo, which drops every served file (the sentinel
minor).
- A first 12-shard run found two test defects; both are fixed.
TestServedBatchApplyBoundsTheItemCount's 1000-item apply outlasts the
client's 60s per-request timeout under -race, so a
race_detector_{on,off}_test.go pair scales its "at bound" half to 150
there, as embeddeddolt's TestBatchApplyContract does. And
TestTheCompositionCatchesAWrongParameter's dropped-filter case leaned
on rows earlier tests had seeded: run alone, or first in its shard,
it failed. It now seeds its own row outside the scope.
- conformance.sh, its script test and
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md name the target.
Deviations from the review's suggested fixes:
- A bare BEADS_HTTP_TOKEN(_COMMAND) fails closed rather than falling
back, and the error does not echo the value: a bare token split at an
"=" would put token text into the pattern half. Dropping the bare
syntax is safe today: the backend has no Register call site and no
docs/ page names the variables.
- Target.String drops userinfo instead of using url.URL.Redacted, which
masks only the password, so a token used as the username would print.
- The pre-#6890 claimIssue fallback leaves RowVersion unset instead of
borrowing the follow-up read's token. That read is a second snapshot,
so its token could postdate a write this claim never saw and arm a
guard that should miss; a guard built from the unset token fails
closed. Against a current server the claim rides updateIssue, whose
`revision` is stitched as for any update.
Also fixed: the nit about the completeness failure message, which now
says to regenerate the shell, and the offrole.go tombstone comment that
named a skip list which no longer exists. Left for follow-up (all
latent, since httpclient.Register has no production caller): vocabOnce
latching a failed fetch, snapshot holding s.mu across the handshake
dial, unchunked graph anchors, private tracker IDs in park owners, and
the bounded journal limit above 10000.
Validation:
- git diff --check; gofmt -l; go vet over internal/httpclient/... and
scripts
- go test -tags gms_pure_go over internal/httpapi/...,
internal/httpclient/..., internal/storage/backends/... and
backend/conformance/...; the doctor gitignore tests; the cmd/bd
events-journal construction guard; ./scripts
- bazel test --config=embedded
//internal/httpclient:httpclient_served_test on the lane's remote
executors: 24 shards, 702 top-level tests, 632 pass, 70 skip (69
ledgered KNOWN DIVERGENCE, plus
TestCAFileHygieneRefusesGroupWritableFile, which finds no
supplementary group on the executor), 0 fail, 0 data races; slowest
shard 283s. tools/bazel/check_testcases.py --bep passes.
- each of the 702 tests run alone (go test -run '^Name$', without
-race): 0 fail
- the whole tier under go test without -race, with the script's
switches and -timeout=40m, before the two test fixes: 633 pass, 69
skip, 0 fail
- removing BEADS_HTTP_TEST_REQUIRED from the rule fails both policy
tests
- bazel test --config=ci over //internal/httpclient/...,
//internal/httpapi/..., //scripts/..., //test/docsync/...,
//internal/storage/backends/..., //backend/conformance/... and
//cmd/bd/doctor/...: 32 tests pass
- make ci-pr-lint: nogo native plus windows/darwin, passes
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
* fix(storage): parent-child cascade propagates only exogenous blockedness (wy-3eb07b, #6506)
A parent-child edge used to cascade the parent's is_blocked bit to its
children WHATEVER set it. So the "close gate on the epic" idiom — P carries
blocks edges onto its own children C1 and C2 so P cannot close before them —
darkened the very children P was waiting for. Neither child could reach
bd ready, so neither could be worked, so neither could close: a permanent
lock, and the one direction cmd/bd/dep.go does not refuse.
The contract (#6506): a parent-child edge propagates only EXOGENOUS
blockedness. P counts as blocked FOR CHILD C iff P has a blocking reason —
an open blocks/conditional-blocks target, or a live waits-for gate — whose
target is NOT one of P's own parent-child children, or P's own parent is
(recursively) exogenously blocked. P's OWN is_blocked is unchanged: it really
does depend on open children. Only the propagation narrows.
DESIGN: a second-pass predicate on the parent row, not WITH RECURSIVE.
Both parent-child legs of shouldBeBlockedIDsUnionScopedSQL keep their
p.is_blocked = 1 conjunct and gain parentCascadesBlockedSQL, which says "p's
blockedness is not solely explained by p's own children":
EXISTS (blocking reason of p whose target is not p's child)
OR NOT EXISTS (blocking reason of p whose target IS p's child)
The second disjunct carries the contract's recursive clause through the
EXISTING fixpoint instead of through SQL recursion: a parent that is blocked
with no child-targeted reason of its own is dark either exogenously or
because ITS parent is, and either way its children inherit — one level per
pass, exactly as the cascade always converged.
A recursive CTE was the other candidate and is rejected on cost, not on
engine support. Its seed set is the direct-exogenous set, which includes the
waits-for gate, and the seed cannot be confined by the batch scope (%[3]s)
because a batch's ancestors are by definition outside the batch. That would
put an UNSCOPED whole-table gate evaluation into every batched mark/unmark —
the statement shape #6288 removed, now on every write instead of only on the
full repair. The predicate here is correlated to a parent that is already
known blocked, so it probes idx_dependencies_issue for a handful of rows.
The scope splice, the DISTINCT waits-for barrier and the doctor-count
lockstep are all untouched.
One gap this form does not express: a parent that is BOTH blocked by its own
open child AND darkened by an exogenously blocked ancestor is not
distinguishable from one blocked only by its children, so its children stay
bright. That is an under-propagation in a corner the contract would darken;
it needs a second stored bit (or the CTE above) to see, and it cannot
recreate the lock this commit removes.
Also: AffectedByDepChange{,ForWisp}InTx now seeds the parent's other children
on a parent-child edge change (appendSiblingsUnderParentInTx). Writing that
edge reclassifies the parent's blocking reason on the new child from
exogenous to child-derived, which flips the answer for the SIBLINGS — the
close-gate epic is built one child at a time, and the second edge is what
un-darkens the first. Without it the incremental write path and the full
repair disagree, which the new tests catch as a nonzero doctor count.
Tests (internal/storage/dolt/blocked_consistency_test.go), all four seeded
through the real write path and each verified RED against the pre-fix SQL
except the control:
- ParentBlockedOnlyByItsOwnChildren: the repro, plus a GetReadyWork
assertion that the children are workable again.
- ExogenousParentStillCascades: the control — the fix narrows the cascade,
it does not remove it. Green before and after.
- ThreeLevelChainBothWays: GP exogenous darkens P and C; GP blocked only by
its child P darkens neither. Repaired from an inverted plane too, since
the chain is the case that needs more than one pass.
- WaitsForGateOverOwnChildren: a gate whose spawner is the parent's own
child does not darken; one on an outside spawner does.
TestRecomputeAllIsBlocked_CascadesThroughParentChild is unchanged and green,
including its single-pass lower-bound count — which is why the legs keep the
p.is_blocked = 1 conjunct rather than deriving the cascade from the graph
alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(dep): warn when a parent blocks on its own child (wy-3eb07b, #6506)
`bd dep add P C` where C is already P's parent-child child is the "close gate
on the epic" idiom: P carries a blocks edge onto its own child so P cannot
close before it. dep.go refuses the OTHER direction (a child blocking on its
dotted-ID parent, "would create a deadlock") and says nothing about this one,
which is the direction that used to deadlock for real — the cascade darkened
the child, so it could never reach bd ready, be worked, or close.
Since the cascade fix the edge is harmless, so this is an advisory, not a
refusal: it names what the edge now means (the parent stays blocked; the
child and its siblings are NOT hidden from bd ready) so an operator does not
have to rediscover the contract from a query.
It fires only for blocks/conditional-blocks, only after the edge has actually
landed and committed, and stays silent on any read error — an advisory must
not turn a successful write into an error, and must not precede a refusal for
an edge that was never written (the hierarchy and cycle guards reject this
pair whenever both edges already exist, in either order, which is why the
shape usually arrives one edge at a time or through import).
Unlike the D1 implicit-blocks warning it is NOT gated on an interactive
stderr: that one fires on the documented-default majority path, where
training callers to ignore stderr is the risk; this one fires on a rare
deliberate shape, and an agent wiring a close gate is exactly the caller who
should read what it does.
Wired into both direct paths (`bd dep add` and `bd dep --blocks`); the
proxied-server paths return earlier and are untouched.
Tests beside the existing dep-add warning tests: the predicate table
(including the two controls — right id with a blocks edge is not a child, and
the parent's other children do not match), the message content, and the type
gate proving it returns before touching the store.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(doctor): report parent-blocks-on-own-child edges (wy-3eb07b, #6506)
A new "Parent Close Gates" check lists every blocking edge from a parent onto
one of its own parent-child children — the close-gate idiom, where P blocks
on C1 and C2 so P cannot close before them.
INFORMATIONAL, with no --fix and no effect on the overall result. Before the
cascade fix these edges were a trap: the parent's blocked bit propagated to
its children whatever set it, so the gate hid the very children it was
waiting for and nothing under it could become ready. The cascade now
propagates only exogenous blockedness, so the edges are legal and usually
deliberate. What an operator wants from doctor is therefore the inventory —
which beads in this store use the idiom, and where a pre-6506 store's
mysteriously invisible children came from — not a repair, and certainly not a
warning that would train people to remove a working gate.
It is the mirror of the existing "Child-Parent Dependencies" check, which
finds a CHILD blocking on its PARENT (still a deadlock, still fixable). The
two differ in how they match: that one reads dotted-ID ancestry, this one
matches real parent-child edges, because the close gate is wired between
unrelated ids.
Tests beside the child-parent ones: the empty inventory (with an ordinary
hierarchy present, which must not be mistaken for a gate), and the reported
case with two controls in the same fixture — an exogenous blocker on the
parent and an ordinary blocker on the child, neither of which may appear.
Gate note: these live in package cmd/bd/doctor, which the brief's cmd/bd
selector does not reach; `go test ./cmd/bd/doctor/` was run in full.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* revert(dep): drop the unreachable parent-blocks-own-child warning (wy-3eb07b, #6506)
This reverts 34b7f5b42. The warning it added could never print.
Its precondition is "a blocks edge from P onto C succeeded AND C is already
P's parent-child child", and the store refuses exactly that edge.
CheckBlockingHierarchyInTx (internal/storage/issueops/dependencies.go) rejects
a blocker that is the source's own descendant, and it sits OUTSIDE the
SkipCycleCheck gate, so no caller opts out of it; batch_apply.go:533 applies it
again on the DependencyEditor path `bd dep add` actually uses. Written in the
other order, the parent-child edge that closes the pair is refused as a cycle.
So the reverted code paid a GetDependentsWithMetadata read on EVERY
blocks/conditional-blocks `bd dep add` to emit something no operator could
ever see, and its 102-line test pinned a predicate instead of a behaviour.
What an operator DOES hit is the refusal — and the cascade fix left its stated
reason false. It read "blocked status cascades to descendants, so <blocker>
would inherit the block and never close". Since #6506 a parent-child edge
propagates only the parent's exogenous blockedness, so the descendant does not
inherit and nothing deadlocks. The refusal still stands, because a blocks edge
is not how the hierarchy says "I close after my children" — so the reason now
names the form that is: a waits-for gate over the children, `bd gate`.
The same false sentence was the OpenAPI description of
Problem.blocker_is_ancestor, so it is corrected there too, with types.gen.go
regenerated by `go generate ./internal/httpapi/apigen/` to keep `make
api-check` green.
The doctor "Parent Close Gates" check from 4c5f2f77b is KEPT. A warning on a
write path that refuses the write is noise; an inventory of the edges a store
already carries — from import, from merge, from the pre-refusal past — is what
an operator can act on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(storage): the parent-child exogeneity test is over the whole SUBTREE (wy-3eb07b, #6506)
B1 of the adversarial review: 648ab0af6 tested "is this blocking reason's
target one of my own children" as a DIRECT parent-child child, but #6506's
contract is about blockedness originating outside the parent's own SUBTREE.
At depth 1 the fix worked; at depth 2 it left the permanent lock alive.
P --blocks--> G, pc C -> P, pc C2 -> P, pc G -> C
G is not P's child, so P read exogenous, so P darkened C, so C darkened G —
and G is the only row whose close can free P. Exactly the shape-1 mutual lock
the issue says no contract intends, one level down. Pinned as
TestParentChildCascade_ParentBlocksItsOwnGrandchild, verified RED at
4c5f2f77b: p/c/c2/g all blocked, all three children missing from ready work,
and the full repair changing nothing.
The test is now a real ancestor walk (subtreeWalkDepth levels of indexed
LEFT JOINs up from the reason's target), so a reason anywhere under the
blocker counts as its own.
Also B2 — the cost regression, which is the same edit, because the shape that
makes the test correct and the shape that makes it affordable are one
decision. The union's parent-child legs no longer carry a per-row correlated
EXISTS. They subtract an uncorrelated set instead:
AND p.id NOT IN (parents explained by their own subtree)
and the UNBATCHED statements (full repair, doctor count) do not even spell
that set as a subquery: it is read ONCE per fixpoint pass and bound as ids.
A repair pass is four statements and each would otherwise derive the same set
twice; reading it once is the whole difference. The batched mark/unmark keep
the spliced subquery, where it is confined to the batch's own parents by the
existing scope splice and costs a few rows.
MEASURED, gastownhall/beads#6506 bar: full repair and doctor count <= 1.5x
origin/main, write path <= 1.2x. Committed benchmark
(internal/storage/dolt/blocked_scale_bench_test.go, -short-skipped), 1000
issues / 2870 deps / 333 blocked rows, embedded dolt 2.2.0, run INTERLEAVED
main/head to cancel machine drift, four pairs, medians:
origin/main 71c4cd08b this branch ratio
full repair 48 71 43 46 -> 47 ms 66 56 54 55 -> 55 ms 1.18x
doctor count 55 62 44 53 -> 54 ms 67 61 57 57 -> 57 ms 1.06x
write path, per CloseIssue 37 30 21 22 -> 26 ms 30 23 16 24 -> 23 ms 0.91x
All three inside the bar; the worst single PAIR is 1.38x on the repair and
1.30x on the count. The two sides disagree on blocked rows by construction
(604 on main, 333 here) — that IS the fix.
NOT inside a bar, and disclosed rather than hidden: the benchmark's fourth
timing, a 200-id RecomputeIsBlockedForIDsInTx (the bulk/import batched path,
which no metric in the review covered), is 53-63 ms on main against 400-556 ms
here — 6.9x at the medians. The cause is structural and measured: a batched
pass runs many small statements, each deriving the set twice, and the set
costs a few ms however it is written. The per-pass read that fixes the
unbatched statements makes the SMALL-batch case worse, not better (one
whole-store read costs more than the subqueries a two-id batch pays), so the
fix is a per-batch-scoped read threaded through expandBatchTemplate's
placeholder plumbing — a separate change, and the reason this timing is
committed rather than left in a scratch file.
Three shapes were built and measured before this one, all at the same scale:
- the review's own: per-row correlated EXISTS with two more inside,
3.3-3.7x full repair, 5.2x doctor count;
- the set as a DISTINCT derived table LEFT JOINed on p.id, which is what B2
recommended: ~30x. dolt plans a join operand as the inner side of a nested
loop and rebuilds it per outer row — the correlated shape in a derived
table's clothes. This is the one finding that contradicts the review, and
it is why the set is subtracted with NOT IN instead;
- WITH RECURSIVE for the ancestor walk: blew the driver read timeout on the
full repair. A recursive CTE pays a query's overhead per ITERATION; the
seeded walk cost 45 ms per evaluation against 6 ms for four levels of
joins, which is why the walk has a horizon (subtreeWalkDepth = 4, epic ->
sub-epic -> leg -> task). Past it a reason reads as exogenous, which
propagates MORE and hides nothing the old code showed.
Seeding: AffectedByDepChange{,ForWisp}InTx now walks UP from a parent-child
edge's target, not just to it. Hanging G under C changes nothing about C and
everything about any ANCESTOR of C that blocks on G, so each such ancestor's
sibling set is reseeded — otherwise the incremental write path and the full
repair disagree, which the new test catches as a nonzero doctor count. The
walk is filtered to ancestors that carry a blocking reason at all: without
that filter a 70-deep chain reseeds every ancestor's subtree on every edge
written into it, which timed out TestGetReadyWork_ParentFilterReturnsDeepDescendants.
CHANGELOG now names the limitation the review ruled acceptable but undocumented
(a sub-epic blocked both by its own children and by an exogenously blocked
ancestor keeps its children visible), marked tracked upstream; bee files the
follow-up issue at PR time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(doctor): rune-safe Detail truncation, Category on the N/A returns (wy-3eb07b, #6506)
The "Parent Close Gates" check joins its inventory with U+2192, three bytes
wide, and then cut the joined string at byte 200. Two of every three offsets
land mid-rune, so `bd doctor --json` could carry invalid UTF-8 in Detail — a
display nuisance in the terminal, but a value a JSON consumer has to decode.
truncateDetail walks back to a rune boundary first; the test drives every
entry count from 1 to 120 so the cut lands at all three offsets within the
arrow and asserts utf8.ValidString on each.
The same byte cut appears twice more in this file (:101, :561) on ASCII-only
details. Left alone here: they are not this bead's, and changing them would
put a behaviour change in two checks a reviewer of #6506 has no reason to be
reading.
The check's two "N/A" early returns also omitted Category, unlike its three
real returns, which drops them out of the categorised report. Both now carry
CategoryMetadata.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* perf(storage): read the batched exogeneity set once per batch, not per statement (wy-3eb07b, #6506)
D1, the PR blocker from the round-1 review. The batched mark/unmark templates
spliced parentsExplainedBySubtreeSQL — the "which blocked parents are explained
only by their own subtree" query — into both of their parent-child legs,
batch-scoped. The scope made it look cheap. It was not: dolt pays for a
`p.id NOT IN (subquery)` once per leg per STATEMENT, and a batched pass is a
mark and an unmark, so one 200-id RecomputeIsBlockedForIDsInTx derived the set
four times and measured 6.9x origin/main (53-63 ms -> 400-556 ms). That is the
bulk/import path and the cost shape #6288 removed.
The set is now READ once per batch and BOUND into both statements as ids —
the same trade the unbatched full repair already makes per pass — which is why
markBlockedTemplate/unmarkBlockedTemplate are built per batch now: the number
of bound ids is a property of the batch. In front of the read sits a cheap
indexed guard, batchParentKindsInTx: a parent KIND with no parent-child row in
the batch has an empty candidate set, hence an empty exogeneity set, so its
read is pure overhead, and the read is a ten-join walk under two CTEs that
dolt spends ~8 ms planning whether it returns rows or not. Most batches name
no wisp parent at all; a batch of roots names neither. Without the guard the
2-id case measured 1.9x, with it 0.94x.
expandBatchTemplate now WALKS the template instead of counting "%s": the batch
scope and the bound exogeneity ids alternate inside the union (leg 3's scope,
leg 3's NOT IN, leg 4's scope, leg 4's NOT IN), so only text order says where
each group goes. It returns an error on a ?/bound mismatch rather than handing
the engine an opaque argument-count failure. The %s budget is back to 6 from
8, re-pinned, with two new pins beside it: the ? group is exactly the bound
exogeneity ids (none for an empty set), and the mismatch is refused.
KEPT, on round-1's measurements: the NOT IN subtraction (the derived-table
JOIN measured ~30x in dolt) and the 4-level LEFT JOIN ancestor walk (WITH
RECURSIVE blew the driver read timeout).
MEASURED. n=1000 issues / 2870 deps / 333-604 blocked, embedded dolt 2.2.0,
TestBlockedStateScaleTiming, SIX interleaved origin/main(71c4cd08b)/head pairs,
medians, worst single pair in brackets. The bench now times three batch sizes
instead of one, because the per-statement and per-id costs pull opposite ways:
metric main head ratio bar
full repair 49 ms 57 ms 1.16x [1.32x] <= 1.5x MET
doctor count (consistent) 48 ms 57 ms 1.18x [1.32x] <= 1.5x MET
write path, per CloseIssue 25 ms 26 ms 1.03x [2.59x] <= 1.2x MET
batched recompute, 200 id 54 ms 69 ms 1.28x [1.57x] <= 1.5x MET
batched recompute, 20 id 17 ms 22 ms 1.29x [1.50x] (measured)
batched recompute, 2 id 22 ms 21 ms 0.94x [1.03x] (measured)
All four stated bars are met on medians. The worst-pair column is at or inside
this box's own noise: over the same six runs origin/main's OWN spread is 2.07x
on the full repair, 2.33x on the write path, 1.92x on the 2-id batch and 1.19x
on the 200-id batch, so a single pair's ratio is not evidence at this
resolution and the medians are what the bars are read off.
DISCLOSED, not a stated bar and not fixed here: the bench FIXTURE BUILD —
2870 AddDependency calls plus 240 CloseIssue through the store — is 75 s ->
102 s, 1.36x [1.61x]. That is the dependency-ADD write path, and the cost is
round 1's incremental seeding (AffectedByDepChangeInTx now walks up to every
reason-carrying ancestor and reseeds its whole sibling set), which is what
buys the incremental/full-repair agreement pinned by the lockstep test in the
next commit. Per-edge it is sub-millisecond; it shows up here because the
fixture writes the hierarchy of a 1000-issue store edge by edge.
GATE: go build ./... ; go vet ./internal/storage/issueops/ ; go test
./internal/storage/issueops/ (ok, includes the rewritten template and sqlmock
pins) ; BEADS_TEST_EMBEDDED_DOLT=1 go test ./internal/storage/dolt/ -run
'ParentChildCascade|RecomputeAllIsBlocked' (ok) ; gofmt -l clean. The full
enumerated gate is on the bead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(storage): pin both sides of the subtree-walk horizon, and the incremental/full-repair lockstep (wy-3eb07b, #6506)
D2 and D3 from the round-1 supervisor ruling.
D2 — THE HORIZON IS NOW PINNED, not commented. "Inside my own subtree" is
decided by a fixed 4-level ancestor walk (subtreeWalkDepth), because a
recursive walk cost 7.5x per evaluation and blew the driver read timeout. Two
tests that differ by ONE EDGE straddle it:
_ReasonAtTheWalkHorizon P blocks a descendant exactly 4 pc levels down.
The walk reaches P, so the reason is
subtree-derived and the chain comes bright.
RED on origin/main (4 rows dark that the fix
brightens) — so it is not vacuous.
_ReasonPastTheWalkHorizon one level further. The reason reads as EXOGENOUS,
so the whole chain stays dark, which is
origin/main's own answer — and that test PASSES
UNCHANGED ON origin/main (verified in a temp
worktree at 71c4cd08b, since removed). It also
asserts no row of the shape appears in
GetReadyWork: no new dark rows AND no new visible
rows past the horizon, checkable rather than
argued. Raising the constant makes it fail and
name the row that changed.
Both also repair from a fully inverted plane, which is the leg that needs the
fixpoint rather than the write path's seeding. The CHANGELOG now names the
4-level horizon beside the existing limitation sentence, and the constant's own
doc names the two pins.
CAN THE HORIZON BE RAISED? Measured, and the answer is "not on this evidence".
Same bench, same box, full repair / doctor count / 20-id batch / 200-id batch:
depth 4 57 ms / 58 ms / 21 ms / 69 ms (shipping)
depth 6 126 ms / 118 ms / 52 ms / 101 ms 2.2x, TWICE reproduced
depth 8 59 ms / 74 ms / 24 ms / 68 ms ~= depth 4, TWICE reproduced
The curve is NON-MONOTONIC: 6 falls off a cliff and 8 comes back. That is a
dolt planner artifact, not a cost model, and raising a semantics-bearing
constant on the strength of a number nobody can explain is the wrong trade —
depth 8 also makes every past-horizon shape newly bright, which is a behavior
change and not this round's business. Left at 4, with the numbers recorded for
whoever picks the knob up.
D3 — THE LOCKSTEP. _IncrementalMatchesFullRepairAcrossEdits walks one graph
through the mutations that reclassify a blocking reason WITHOUT touching the
row that carries it, and after every one asserts both the exact flags and
assertConvergedAndStable: detection counts 0 AND a full repair changes 0 rows.
An under-seeded incremental path (the AffectedByDepChange{,ForWisp}InTx
ancestor walk round 1 added) fails here as a nonzero repair over a graph the
write path just finished maintaining. The mutations: re-parent the reason's
target INTO the blocker's subtree (which is how the shape arrives — the store
refuses a blocks edge straight onto an existing own descendant, and that
refusal's reason text is what 3a5203979 corrected), close the descendant,
reopen it, re-parent it back OUT, remove the blocks edge. The
into/out-of-subtree steps are the ones that flip rows the edge does not name
(the parent's other children), which is exactly what
appendSiblingsUnderAncestorsInTx exists to seed. RED on origin/main for the
semantic values; origin/main's own incremental/full-repair agreement holds
there, so what this pins is the agreement, not the numbers.
GATE: BEADS_TEST_EMBEDDED_DOLT=1 go test ./internal/storage/dolt/ -run
'ParentChildCascade' -count=1 — 8/8 ok; gofmt -l clean; go vet clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(doctor): the close-gate inventory walks the same four parent-child levels as the cascade (wy-3eb07b, #6506)
The "Parent Close Gates" check listed only DIRECT parent->child gates, so the
fix's own headline shape (a parent that blocks on its grandchild) never
appeared in the inventory. The check now joins a descendant-pairs set built
to the same four parent-child levels the cascade uses for "inside my own
subtree" (issueops subtreeWalkDepth), as plain joins over the dependency
union rather than a correlated subquery, which dolt declined to plan once
rows were present. Messages say "own-descendant"; the tests add a grandchild
gate (2 listed, the exogenous edge still not) and assert Category.
CHANGELOG cites the nested-gap follow-up, gastownhall/beads#6601.
Tests run: go build ./... 0; go vet ./cmd/bd/doctor/ 0; gofmt -l clean;
env -u BEADS_ACTOR go test ./cmd/bd/doctor/ -run 'ParentBlocksOwnChild|TruncateDetail' -count=1 PASS (3/3).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* ci: allowlist TestBlockedStateScaleTiming's -short skip (wy-3eb07b, #6506)
scripts/check-testing-short.sh reserves testing.Short() for runtime, stress
and large-fixture skips and keys them by file::func. The scale-timing test
added in this PR builds a 1000-issue / ~2000-edge dolt fixture and times the
blocked-state paths over it: a large-fixture skip, and PR Core runs with
-short, so it needs the entry. One line, the same shape as the
TestUOW_ConcurrentMergeOps_NoLostUpdate row from #4732.
Local: ./scripts/ci/pr-policy.sh rc 0 (check testing.Short boundaries
succeeded; the other nine legs unchanged).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* wy-3eb07b: build(bazel): register blocked_scale_bench_test.go in internal/storage/dolt
The rebase carried the bench test but not its BUILD.bazel entry, so the
Bazel test lane's gazelle sync and Go test equivalence steps failed while
every Go lane was green. This is CI's own bazel-sync patch for 043f9c47b9,
applied unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(blocked): resolve explained parents through their ancestry; walk ancestors on hinted per-table members
Maintainer fixup for #6602 (adopt-pr review iteration 1, quality scorecard
760/1000, request_changes; amended at iteration 2, 906/1000, approve). It
fixes the review's two major findings, the doctor check's message and its
swallowed query failure, the stale core-builder doc, and the stale
test-comment names. At iteration 2 it also takes both nits and corrects a
CHANGELOG sentence this fixup had made false.
- Mixed-arm leak (major). The explained-parent read decides from a
parent's own reason rows, and the fixpoint carries the recursive clause
only for a parent with no reason row of its own. With Q blocked on an
outside X, P a child of Q, and P blocking on its own child C, P read as
explained and C came bright under an epic still gated by X, on the
write path and in full repair alike. pruneExplainedByAncestryInTx now
shrinks every read to the greatest set whose members' blocked parents
are all members. A full-repair read is complete, so a blocked parent it
did not return is unexplained. A batch-scoped read climbs instead,
reading the blocked parents above the batch by id (explainedAmongInTx).
The suggested one-level NOT EXISTS is not enough: put a second close
gate between them (R blocked on X, pc Q -> R, Q blocks on P, pc P -> Q,
P blocks on C) and P's blocked parent Q is in the set, so R's gate
never reaches C.
- ancestorChainInTx (major) walked the derived UNION of both dependency
tables that #7224 removed from isAncestorInTx, on every parent-child
write and merge replay. It now walks reachabilityMembers, the hinted
per-table recursive members split out of reachabilityQuery, with the
reason filter kept in the outer SELECT. TestGraphWalkPlansHonorJoinHints
(embedded) gains an ancestor-chain subtest that pins the lookup plan;
the old walk fails it.
- New dolt tests: TestParentChildCascade_OwnGateUnderExogenousParent
(write path, full repair, and X closed and reopened),
_NestedGatesBothWays (the two-gate chain both ways, then a close of the
children's own blocker, which re-evaluates them through a batch whose
read covers nothing above their parents), and
_WispGateUnderExogenousParent (a wisp gate under an issue epic, an
issue gate under a wisp epic, then an explained wisp gate the walk must
read by id above such a batch). Five mutants of the prune were each
killed: no prune (the PR head), a one-level check, an over-prune, a
scoped read treated as complete, and a scoped read that trusts every
parent above it. The eight existing cascade tests pass under all five.
Iteration 2 killed two more, each by its own section of the wisp test
alone: dropping the lookup of a wisp parent from blockedParentEdgesInTx,
and dropping explainedAmongInTx's wisp leg, which had survived all
eleven cascade tests.
- The doctor check no longer says the children "stay in bd ready", and
a failed or timed-out inventory query is now a warning carrying the
error, as in validation.go's other checks, instead of an OK that reads
as "no gates". TestCheckParentBlocksOwnChildDB_QueryFailureWarns pins
it with a closed handle, and it fails with the old status. A warning
does not touch doctor's overall result, and it can be suppressed.
- The core builder's doc no longer describes the spliced design, and
three test comments name builders that exist.
- explainedAmongInTx says why it does not tolerate a missing table, as
the file's wisp reads do: the prune runs only on a non-empty read,
which came from the same core query over all four tables.
- The CHANGELOG entry no longer calls #6601's shape (a sub-epic gated on
its own children under an exogenously blocked epic) a known limitation
that keeps the children visible. The prune darkens them, as
TestParentChildCascade_OwnGateUnderExogenousParent pins.
Left as-is, all non-gating: the refusal hint's "(bd gate)", an API string
pinned by the golden corpus; a parent-child add/remove timing in the
scale bench, whose 1000 issues cannot show the 50k shape (the plan pin
guards it); and hints for the explained-parent subtree walk and the
doctor query. The measured complexity major is runDiagnostics (50/66),
the same at base; the PR adds two lines to it.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(storage): re-pin the classic batch-apply statement count for the parent-child cascade
The B0 statement-count pins predate this PR's parent-child cascade. The
first full run of the storage tiers on the adopted head measured the
40 (classic) shape at 835 statements on both backends, against the
pinned 786 +/- 2.
All 49 extra statements are reads, seven for each of the shape's seven
distinct parent-child dep adds (its eighth repeats a pair and is a
no-op):
- appendSiblingsUnderAncestorsInTx reseeds the parent's other children:
one ancestorChainInTx walk and four child probes, +5.
- The batched mark/unmark reads its exogeneity set first
(batchExogeneity): the parent-kind probe and the issue-parent read, +2.
Measured by logging every statement of the classic apply on the CI base
233d151b37 (786) and on b2b93515bf (835), then diffing the two logs by
statement shape. Those seven shapes are the whole difference, each
exactly +7, and no write changed. The 356 and 712 shapes are unchanged
on both backends.
This is the deliberate re-measure and re-pin the pins' doc comment asks
for. The per-change breakdown is in pinnedEmbeddedStatementCounts.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
* feat(schema): never migrate an existing database without explicit consent (bd-qhn8y)
Beads 1.2 release remediation. 1.2.0/1.2.1 auto-applied migrations 54-65 at
store open on ANY command, including reads; older binaries then refuse the
migrated database. MigrateUp now refuses pending main-sequence migrations on
an existing database before any write (dolt_ignore seeding included) unless
the operator consented: bd migrate / bd migrate schema (preview flags
withhold), bd migrate --force, or BD_ALLOW_MIGRATE=1
(BD_ALLOW_REMOTE_MIGRATE honored as an alias). Fresh init and at-latest
databases are unaffected; the #4259 remote gate keeps all its protections.
Also: freeze shipped migrations (hash-pin test, main 0001-0065 + ignored
0001-0024), retract v1.2.0/v1.2.1 in go.mod, agent-executable un-migrate
runbook (docs/recovery/unmigrate-schema-v65.md), and the upgrade smoke +
historical migration harnesses now model the consent contract and assert
the refusal carries guidance.
No version bump here by design: cutting the release stays with the release
owner.
Rebased onto main 800d43eb8d: the go.mod retract hunk is dropped -
main already retracts v1.2.0/v1.2.1 (and v1.1.1) since #5782. The
uow consent grant moves into the TestMain main added in #6441 (a
second TestMain no longer compiles), and the new files are listed
in the BUILD.bazel targets main now carries (gazelle).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(schema): cloned-workspace bootstrap carries creation-consent (bd-qhn8y)
The consent gate treated a workspace `bd init --remote` / `bd bootstrap`
just cloned into existence like a pre-existing database, refusing the
smart gate's provably-safe first-mover migration and stranding init in
the generic open-error path (bypassing the bd-4mpy7 finalize+guidance
ergonomics). Cloning IS creating: those paths now record consent
explicitly, and the #4259 remote-migrate gate still applies to them
unchanged (consent is not its designated-migrator override). Fixes CI
TestEmbeddedInit/remote_behind_schema_gate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(migration): consent first in main's wisp lanes too (bd-qhn8y)
The rebase onto 800d43eb8d brought in #6049's wisp-plane lanes, which
postdate this PR's harness changes. Each one makes an ordinary candidate
command the first touch on an old-schema database, and the consent gate
refuses it:
- historical-dolt-upgrade-test.sh run_wisp_plane_upgrade opens with
`bd list` before any migrate: "candidate could not open the seeded
wisp workspace" for v1.0.1, v1.1.0, v1.1.2 and v1.2.2 (the
Migration Test Harness dolt-runtime shard).
- run_dirty_tracked_wisp_scenario (v1.1.2): the consent refusal stops
the open before MigrateUp reaches the #5816 dirty tables, so the lane
fails "without naming them".
- upgrade-smoke-test.sh "Wisp plane survives upgrade" upgrades with a
bare cand_init, so list/ready/blocked/update all hit the refusal
(4 failures each for v1.3.0 and v1.3.1, both in the PR smoke set).
Use the PR's own pattern: the explicit `bd migrate schema` is the first
candidate touch (cand_upgrade in the smoke script). Base 800d43eb8d
passes these lanes unchanged; with this fixup the full 14-version
historical corpus and all five PR smoke versions pass.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(schema): close consent-gate review findings (bd-qhn8y)
Review fixes for the migration-consent gate (adopt-pr review, attempts
1 and 2).
Majors:
- #4566 deadlock rebuilt one check earlier: on a database that is both
behind and dirty, the consent refusal fired before the dirty-table
guard, whose remedy (`bd dolt commit`) opens without consent. The
working-set-reconcile opens now tolerate MigrateConsentError: embedded
(toleratesConsentRefusal, WSR intent only) and server mode
(LenientOpen's warnLenientOpenRefusal) print WorkingSetWarning and
commit at the current schema; strict, read-only and RemoteSync opens
still refuse. Covered by an embedded test (a refused open writes
nothing: dolt_status, dolt_ignore and schema_migrations unchanged) and
a server-mode integration subtest.
- Bare `bd migrate` consented: isMigrateConsentCommand now matches only
the schema verb, so `bd migrate` (metadata reconcile, including
--update-repo-id, --inspect and --dry-run) no longer consents to a
one-way migration; `bd migrate schema` and --force do.
TestLocalMigrateConsentScope pins the scope through the root pre-run.
The dirty-table warnings name `bd migrate schema` accordingly.
- Runnable JSON hint: the JSON refusal follows the #4259 AgentDirective
convention, a non-runnable hint plus a migrate_consent object
(versions, pending, severity, human_decision_required, options) whose
commands are retargeted with --global for the shared global database.
The text refusal prints the same --global hint under `bd --global`.
- Freeze floor: the hash-pinned freeze now covers what v1.3.x shipped,
main 0001-0066 and ignored 0001-0026; the CHANGELOG puts new
migrations in 0070+ / 0028+.
- Fresh-bootstrap heal: MigrateUpWithLock with heal authority takes the
consent-free migrate path, so a retried bd init whose first pass left
a non-zero cursor converges; other lock callers keep the gate.
Minors: the uow team-server guards are repinned on `bd migrate schema`;
the un-migrate runbook is linked from the recovery index and
cross-linked with the 1.2.1 repair-forward doc; the CHANGELOG BREAKING
entry matches the shipped contract; harness comment grammar.
Docs: the user docs and README named bare `bd migrate` as the step that
applies schema migrations; they now say `bd migrate schema`, which also
exists in the pinned docs release (v1.2.2) and in v1.3.x. The
`bd migrate schema` help no longer claims migrations run automatically
on store open. The Go remedy that sends a remote-gate stop to bare
`bd migrate` for its migrate-or-adopt options is unchanged (pinned by
TestNoticeSharedMigrateRefusalPerDecision).
Upgrade smoke (attempt 2): the consent-gate assertion is graded only when
the old binary left a database. Releases before v1.0 have no
`init --non-interactive`, so prev_init creates nothing there and the
candidate's `bd list` fails with "no beads database found", which has no
consent guidance. Main's Bazel smoke lane (#7331) runs the newest 30
releases, 20 of them pre-1.0, under the required ci-gate; all 20 went red
on the merge. The 10 v1.x releases still take the refusal branch.
Shard manifest (pre-approval CI): the new embedded test
TestEmbeddedMigrateConsent_BehindAndDirtyWorkingSetReconcileRecovers is
listed in the Bazel 15-shard block of embedded-storage-test-shards.txt
(incremental --write; shard 15). Main's ci lane runs
TestStorageEmbeddedShardManifestGeneratorNotStale in //scripts:scripts_test,
which fails on any unlisted top-level embeddeddolt test.
Not addressed (optional): env-only consent docs for embedders, the
unparseable BD_ALLOW_REMOTE_MIGRATE alias hint, and the harness
`|| true` on `cand migrate schema`.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <julianknutsen@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
Since #7348 the pre-push hook ran scripts/pre-push-suite.sh only for stdin
lines whose local ref matched refs/heads/*. For `git push origin HEAD` and
`HEAD:refs/heads/x` git passes the literal local ref HEAD, so the suite never
ran for those pushes: a Go change pushed that way landed with an empty bazel
log, while the same change pushed as `git push origin br1` was refused.
The hook now skips a line when the remote ref is refs/tags/* or the local sha
is the zero sha (a deletion); everything else is a branch push, HEAD included.
TestPrePushHookSuiteKeysOnRemoteRef feeds the real hook HEAD-sourced ref lines
from a scratch repository with a stub suite that records whether it ran: both
HEAD forms run it on a Go change; a docs-only push, a tag pushed by HEAD or by
name, and a deletion do not. On main's hook the two HEAD cases fail.
TestPrePushHookRunsSuiteOnlyForGoChanges indexed "pre-push-suite.sh", which
the hook's header comment on line 3 also contains, so the ordering assertion
held wherever the call block sat. It now indexes the call line; moving the
suite block after the managed beads section fails it.
Fixes#7351
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
TestBazelRetiredLanesCannotBeNarrowed skipped every ModeSymlink entry in
its two BUILD/.bzl walks. Under `bazel test` on a local executor (the
processwrapper-sandbox an unauthorized fork's PR gets for
//scripts:scripts_test, RBE_INSTANCE being empty) the runfiles tree is a
symlink forest, so the walk read no BUILD file and the pin failed with
`got map[]` on every such run (#7350). Both walks now go through
walkBazelFiles, which decides "file" with the package's isFileOrFileLink:
a regular file or a symlink to one; a symlinked directory is never
followed, a dangling link is skipped. TestBazelFileWalkFollowsFileSymlinks
mirrors the checkout's Bazel files as a symlink forest (plus a directory
link and a dangling link) and requires the same file set as the checkout.
Fixes#7350.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>