Commit graph

7 commits

Author SHA1 Message Date
Julian Knutsen
dfbb28fad0 ci(bazel): run the scripts and source-walking tests remotely; retire their go test legs
//scripts:scripts_test and //scripts/check-versions:check-versions_test
drop their host-tools/no-remote-exec/no-remote-cache tags and run on the
rbe-west workers like every other target (the worker image ships bash, git,
python3, jq, yq, curl, openssl, unzip and coreutils; a missing tool fails the
test under Bazel rather than skipping it). Go comes in hermetically as the
registered SDK; check-versions.sh's `go build` is stood in for by the
Bazel-built checker.

The tests that walked the source checkout now get it as data:
tools/bazel/go_srcs.py keeps a managed `repo_files` filegroup in every
package and aggregates them into //:repo_files (exactly the tracked files
Bazel can see; make bazel-sync-check fails on a package without one).
scripts_test and docsync_test take it instead of hand-listed filegroups,
and every TEST_SRCDIR/IsBazel skip and go-test-only block in them is gone.
The release formula under .bazelignore'd .beads/ comes in through a
@beads_formulas local repository; npm-package/ and .claude/ are visible to
Bazel again (only their node_modules/worktrees stay ignored), so the npm
manifest release check and the AGENTS.md skill links run under Bazel too.
TestZZStdioNotLeaked falls back to TestMain's pre-run streams instead of
skipping in a shard without its baseline.

tools/bazel/equivalence_allowlist.txt shrinks from 20 skip entries to the
two cmd/bd tests of plain go test's own bd build fallback, which Bazel never
takes; pr-preflight-platforms now runs them on Linux, macOS and Windows. The
scripts-go-checks scripts-test and allowlisted legs, their scripts
(scripts-go-test.sh, allowlisted-go-tests.sh, run_allowlisted_go_tests.py)
and main.yml's --compile-only warm for them are removed. The vet leg stays
(ga-96smfk.18).

Refs: ga-96smfk.16

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 21:32:02 +00:00
Julian Knutsen
ae1a493e01
test(bazel): fully green //..., tag taxonomy + docker lane, sharding, test sandbox ceiling (#6871)
* fix(doctor): detect test binaries with testing.Testing(), not the file name

getBdBinary refuses to return a test binary as "bd" so fix tests cannot
fork-bomb by re-running themselves. The guard matched only the `go test`
naming (<pkg>.test). Bazel names test binaries <target>_test, so under
`bazel test` every fix that shells out to bd re-ran the whole fix_test suite,
recursively (~60 levels), until the 300s timeout.

testing.Testing() is true for any binary linked as a test by `go test` or
rules_go (both set testing.testBinary=1), whatever its name. The name check
stays as a fallback. bd already links package testing, so the product binary
gains no dependency. Behavior under `go test` is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): tag taxonomy, docker lane, and green //scripts targets (slice B5)

`bazel test //... --config=ci` is now fully green, locally and with
--config=remote-exec.

- scripts_test declares what it reads and runs: this package's own tree
  (:scripts_files), .github/** (//:workflow_files, widened from the
  workflows), and .buildflags/.test-skip for scripts/test.sh. The complexity
  diff test builds a one-commit git fixture instead of git-archiving the
  real checkout's HEAD. Under Bazel it skips the whole-repo testing.Short()
  scan (it would pass vacuously over the runfiles) and the two release
  formula tests (.beads is in .bazelignore). Tagged host-tools +
  no-remote-exec: it runs the host's git/python3/coreutils and some subtests
  skip without git.
- prlintmake_test resolves fmt-check.sh through bazeltest.CallerDir and
  declares it; it needs only bash (gofmt is shimmed), so it stays untagged
  and runs remotely.
- //cmd/bd/doctor/fix:fix_docker_test reruns the fix_test binary (no second
  compile, tools/bazel/go_test_variant.sh) with BEADS_FIX_REQUIRE_DOLT=1, like
  pr.yml's container-backed doctor/fix job. Tagged requires-docker +
  no-remote-exec; `--config=docker` selects it (race, no -short, no
  BEADS_TEST_SKIP=dolt). fix_test itself stays untagged: under prcore it
  skips the Dolt-backed tests exactly as pr-core's `go test` does.
- scripts/bazel_policy_test.go: every BUILD `tags` value must be in the
  taxonomy, be named in the comment above its rule, and host-bound tags must
  come with no-remote-exec; test:prcore must exclude requires-docker,
  embedded and manual.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bd): reset git's cwd cache in TestContextRoutesNameOneWorkspaceTheSameWay

The test chdirs into a fresh git repo but reset only beads' caches. git
caches the cwd's repository context process-wide, so when an earlier test in
the same process had resolved it from a non-repo directory (for example
TestCollectMetadataEntriesNoBeadsDir), GetMainRepoRoot kept answering "not a
git repository". Test order hid it under `go test ./cmd/bd`; Bazel sharding
reorders tests and exposed it. Reproduces with plain go test:

  go test -run '^(TestCollectMetadataEntriesNoBeadsDir|TestContextRoutesNameOneWorkspaceTheSameWay)$' ./cmd/bd

The cache is reset on entry only. Resetting it again in cleanup changes what
later tests inherit, and TestCreateRemoteRepoSkipsLocalDatabaseGuard (which
never resets it) then fails under the unsharded go test order.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): shard bd_test and embeddeddolt_test; keep test re-execs shard-free (slice B6)

- //cmd/bd:bd_test shard_count = 16, //internal/storage/embeddeddolt:
  embeddeddolt_test shard_count = 8. Remote -race under --config=ci: bd_test
  max shard 61-82s (unsharded 252-264s), embeddeddolt max shard 84-122s
  (unsharded 207-377s). The embeddeddolt max is a single test
  (TestWithPeerAuth, ~93s), so more shards would not lower it. Every shard
  runs tests (bd 161-172 top-level tests each, embeddeddolt 15-20).
  protocol (4s), legacysqlite (4s) and dbproxy/server (22s) stay unsharded.
- Tests that re-exec their own binary now pass bazeltest.ShardFreeEnv
  (a no-op under go test), so a helper child does not inherit
  TEST_SHARD_INDEX/TEST_TOTAL_SHARDS/TEST_SHARD_STATUS_FILE and filter away
  the one test it was asked to run: dolt suite_leak_linux, dbproxy
  force_stop_linux, workspacegate (3 helpers), metrics spawn_reap, bd
  worktree_remove, conformance role_bundle, linear synclock (windows).
  scripts/test_script_test.go (child env built from scratch) and
  repro-dolt-prod-timeouts (copies the binary, never runs it) need nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): keep sharded bd_test off ~/.beads and keep its stdio leak guard

Two things sharding //cmd/bd:bd_test exposed:

- config.Initialize walks up from the test's cwd for .beads/config.yaml and,
  under BEADS_TEST_IGNORE_REPO_CONFIG, stops at the module root (go.mod).
  The runfiles tree had no go.mod, so the walk left it; with the output base
  under $HOME it loaded the developer's ~/.beads/config.yaml (here
  dolt.shared-server: true) and two embedded-mode autocommit tests failed
  locally, though not on remote workers. bd_test now declares //:go.mod, so
  the walk stops at the runfiles root as it stops at the checkout under go
  test. Unsharded runs hid this: an earlier test re-initialized config first.
- TestZZStdioNotLeaked skipped in every shard, because the baseline test is
  always in shard 1 and the guard in another. Without a baseline it now
  checks the descriptors instead (a leaked capture leaves a pipe where fd 1
  or 2 was). Under go test the baseline is always captured, so nothing
  changes there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore(bazel): sync BUILD files with test files added on main

Gazelle picks up cmd/bd and internal/notion tests that landed on main after
the last sync (make bazel-sync).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(discovery): add BEADS_CEILING_DIRECTORIES; bound bazel tests' walks

bd finds its workspace by walking up from the working directory for .beads
and .beads/config.yaml. Under `bazel test` the working directory is in the
runfiles tree under the output base, usually below the developer's HOME, so
the walks left the sandbox: tests read the developer's ~/.beads/config.yaml
(results depended on it) and wrote into ~/.beads (interactions.jsonl from the
audit log, .local_version from version tracking).

BEADS_CEILING_DIRECTORIES takes git's GIT_CEILING_DIRECTORIES form (a PATH-
style list of absolute directories): a search starting inside a ceiling never
examines the ceiling or anything above it; the start itself is always
examined. Unset, nothing changes. Honored by every upward .beads/config
search: beads.AncestorDirWalk (all FindBeadsDir walks), config.Initialize
(the go.mod and config.yaml walks, and the worktree-fallback git probe, whose
routing scrub drops GIT_CEILING_DIRECTORIES), findProjectBeadsDir, the legacy
upgrade guard, the --db beads-dir walk, store reopen and findBeadsRepoRoot.
The migration-freeze marker walk is left alone on purpose: it is a write
safety gate, and BD_MIGRATION_FREEZE_FILE is already its explicit override.

tools/bazel/test_env.sh sets BEADS_CEILING_DIRECTORIES and
GIT_CEILING_DIRECTORIES to the runfiles root, TEST_TMPDIR and its own temp
root. It does not fail on an ancestor .beads: a live ~/.beads above the
output base is normal on a developer machine. bd_test no longer needs
//:go.mod as a walk stop.

Canary check: with an invalid-YAML .beads above a scratch --output_base,
compact, issueops, validation and bd_test pass under --config=ci and nothing
is written there; the previous wrapper fails on it and writes
interactions.jsonl into it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bd): check for leaked stdio after m.Run in every process

TestZZStdioNotLeaked only sees the tests that ran before it in the same
process. Under Bazel's 16-way sharding that is ~1/16 of bd_test: the baseline
test is in shard 1, the guard in another shard, and leaks in the other shards
were never checked.

TestMain now compares os.Stdout/os.Stderr after m.Run with the streams from
before it (runTestsAndSweep, so every shard and every go test run) and fails
the process on a leak. The one allowed change is the framework's own
os.Stderr = os.Stdout under -json. TestZZStdioNotLeaked stays for its go test
report and skips when it has no baseline; the descriptor fallback added for
sharding is gone.

Verified: a test that leaks os.Stdout fails `go test -run` for it alone, with
and without -json; a clean -json run passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): keep host-tools results out of the remote cache; tighten tag policy

- //scripts:scripts_test (host-tools, no-remote-exec) ran locally but its
  result was still uploaded to the remote cache and served to other clients,
  though the action key does not cover the host's git/python3/bash. It is now
  also tagged no-remote-cache (neither read nor uploaded). Checked: after a
  --config=remote-exec run and `bazel clean`, a rerun without the disk cache
  executes the test again instead of taking a remote cache hit.
- Under Bazel a missing git or bash now fails those tests instead of
  skipping (the host-tools tag promises them); under go test they still skip.
- Tag policy: host-tools requires no-remote-exec and no-remote-cache;
  `tags = [...] + X` is rejected as non-literal; every --test_tag_filters set
  by test:prcore or a config that expands it (test:ci, transitively) must
  exclude requires-docker/embedded/manual, not only the first prcore line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(bazel): run go_test variants from their package directory

rules_go gives a go_test GO_TEST_RUN_FROM_BAZEL=1 (and its `env`) through
RunEnvironmentInfo; an sh_test that reruns the binary gets neither, so the
binary skipped its chdir into the package directory and ran from the
runfiles root. go_test_variant.sh now exports GO_TEST_RUN_FROM_BAZEL=1, and
its header says the variant target must repeat the go_test's `env`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): docker-lane variants for domain/db, uow and protocol

--config=docker only ran doctor/fix. pr.yml's container jobs also run
internal/storage/domain/... + uow/... (race, no -short, Dolt image pulled)
and cmd/bd/protocol (BEADS_PROTOCOL_REQUIRE_DOLT=1). Under --config=ci those
container suites skip (BEADS_TEST_SKIP=dolt), so without variants their
coverage would vanish if Bazel replaced those jobs.

- testutil: BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 makes RequireDoltContainer
  and StartIsolatedDoltContainer(Handle) fail instead of skip when the Dolt
  container is unavailable (the generic form of BEADS_FIX_REQUIRE_DOLT).
- db_docker_test, uow_docker_test (4 shards) and protocol_docker_test
  (8 shards, BEADS_PROTOCOL_REQUIRE_DOLT=1) rerun the go_test binaries via
  go_test_variant.sh, repeating their `env`; tagged requires-docker +
  no-remote-exec. The other domain/tracker packages have no container tests
  and already run under --config=ci.

`bazel test //... --config=docker --test_arg=-test.v`: 4/4 PASS; db 908
PASS / 0 SKIP, uow 985 PASS / 32 SKIP (conformance capabilities the backend
does not implement, as under go test), protocol 151 PASS / 2 SKIP (tracked
known gaps). With --test_env=BEADS_TEST_SKIP=dolt, db_docker_test fails.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: shard-free env for the embeddeddolt concurrency helper; 2m bd init timeout

- concurrency_test re-execs its test binary with -test.run=^TestHelperProcess$
  inside the now-sharded embeddeddolt_test; pass bazeltest.ShardFreeEnv so a
  child cannot inherit TEST_SHARD_* and filter its one test away (it is
  gated by BEADS_TEST_EMBEDDED_DOLT=1, so dark today).
- initBeadsWorkspace / runBDStdout (TestClose_UpdatesLastTouched and the
  gemini prime hook tests) allowed 30s per bd subprocess. A -race bd on a
  loaded host exceeded that for a single `bd init` (killed at 30.18s in 1 of
  3 sharded remote runs; the whole test otherwise passes in ~21s), so allow
  2 minutes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bd): give TestCreateRemoteRepoSkipsLocalDatabaseGuard its own git state

The test chdirs into a fresh non-repo temp dir but used whatever git/beads
cwd caches earlier tests left. When an earlier test had reset git's cache and
a later one refilled it from the package directory, workspace discovery in
the test returned the checkout's .beads (from a git worktree: the MAIN
checkout's live .beads via the worktree fallback) and `bd create` tried to
open it, so it failed with "could not verify workspace identity" instead of
the expected downstream error. It passed only on stale state (a cache
pointing at a deleted temp repo).

It now resets the git and beads caches after its chdir and again in cleanup,
so it neither depends on nor leaks cwd state. With that fixed,
TestContextRoutesNameOneWorkspaceTheSameWay also resets git's cache in
cleanup. Reproduced with the context-test cleanup reset in place, running
every cmd/bd test declared up to this one (the failing go test order); now
passes there, alone, with the two context tests, and under -shuffle seeds
1-5 (other pre-existing order dependencies show up under shuffle; this test
is not among them).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): re-sync BUILD files with main

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): pin BD_MIGRATION_FREEZE_FILE in test_env

The freeze-marker ancestor walk is intentionally not ceiling-bounded; an
explicit BD_MIGRATION_FREEZE_FILE replaces the walk, so a MIGRATION-FREEZE
above the output base can no longer change test results.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(scripts): give the complexity diff fixture a Go package

complexity.sh expands "${analyzer_paths[@]}" under set -u; with an empty
fixture tree that is an unbound-variable error on bash 3.2 (macOS runners).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 20:19:38 -07:00
Julian Knutsen
469462d56a
test(bazel): run every Bazel test against a hermetic, pinned dolt CLI (#6853)
* test(bazel): run every test against a hermetic, pinned dolt CLI (slice B4)

Tests that shell out to dolt ran whatever /usr/local/bin/dolt the executor
had, a version outside the action key, and skipped (a vacuous, cacheable
PASS) on an executor without one.

- tools/bazel/dolt.bzl: dolt_release repository rule fetching the upstream
  release pinned to scripts/ci/install-dolt.sh (2.2.0), sha256 for
  linux/darwin amd64/arm64; MODULE.bazel declares one repo per platform and
  //tools/bazel:dolt selects the one for the execution platform.
- //tools/bazel:test_env (the --run_under wrapper) carries the pinned dolt
  as data. Bazel merges the run_under runfiles into every test's runfiles, so
  every test action declares dolt as an input; test_env.sh puts it first on
  PATH, exports BEADS_TEST_DOLT_BINARY, and fails the test if it is missing.
  DOLT_ROOT_PATH identity seeding stays in test_env.sh.
- testutil.requireDoltBinaryPresent fails instead of skipping under Bazel.
- TestPinnedDoltCLIMatchesContainerImage also checks tools/bazel/dolt.bzl.
  scripts_test declares the pin files and resolves the repo root through
  bazeltest, so the three dolt pin tests now pass under bazel test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(bazel): declare the Go source dolt_test's scope guards scan

TestEveryRawTxJournalScopeIsScopedOrExempt, TestEveryRawTxVersionScopeIsScopedOrExempt,
TestTxMintingWrappersScopeVersionedHistory and TestHelperSubprocessSentinelsAreComplete
parse internal/storage/dolt and internal/storage/issueops at test time; under
bazel test that source was not in the runfiles, so all four failed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(bazel): tie hermetic dolt download to its version; skip on unpinned platforms

- Key DOLT_SHA256 by version and pass canonical_id=url, so bumping
  DOLT_VERSION without new digests fails instead of silently reusing the
  previous release from the sha256-keyed repository cache.
- //tools/bazel:dolt_version_test runs the hermetic dolt and checks it
  reports DOLT_VERSION.
- Unpinned platforms (e.g. windows) resolve to an incompatible target, so
  bazel build //... skips the wrapper there instead of failing analysis.
- test_env.sh also unsets BEADS_DOLT_BIN so it cannot bypass the pin.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 14:29:49 -07:00
ecuthiell
a2cc86ee3a
test(scripts): stabilize Git Bash shim paths (saa-cgh.55) (#5465)
Resolve shim and state paths under the same minimal Bash environment used by
the test commands so Git for Windows cannot remap /tmp between setup and
execution.

Reset the effective command PATH inside profile-free Bash and prove docker,
sleep, and go resolve to the generated shims before either target script runs.
This keeps ambient host tools and real retry sleeps out of the harnesses while
preserving their exact argv, call-count, and exit-status assertions.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-09 01:05:13 -07:00
matt wilkie
7865493f76
test: port Windows hardening fixes (#4600)
* test: port Windows hardening fixes

Port the live Windows/test hardening fixes from contributor PRs #3797, #3801, #3802, #3806, and #3812 onto current upstream/main.

#3801 is applied to the moved dbproxy server test path. The release-script test harness also normalizes Bash-visible paths so the fake bd is selected reliably from Windows-hosted test runs.

Co-authored-by: sms <114885497+seanmartinsmith@users.noreply.github.com>

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie

* fix(test): keep release-script fixtures out of source tree, preserve PATH

releaseTestTempDir previously created .tmp-release-tests under the
source repo root, requiring a writable checkout to even start the
release-script tests. Use t.TempDir() instead so the tests run from
read-only checkouts and packaging build directories.

runReleaseDryRunWithEnv hard-coded PATH to "<fakebin>:/usr/bin:/bin",
dropping the caller's PATH entirely. On systems where bash or core
utilities live elsewhere (e.g. Nix/Guix), the inner `bash
scripts/release.sh` invocation then failed. Prepend the fake bd
directory to the caller's actual PATH (converted per-entry via
shellPath for Bash) instead of replacing it.

Addresses Codex cross-vendor review finding (2026-07-07) on PR #4600.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie
2026-07-07 12:59:50 -07:00
matt wilkie
bf516a7658
docs(release): harden Homebrew formula procedure (#4571)
Add a release workflow step for Homebrew core formula handling so stable releases track the canonical bump PR, avoid duplicate manual PRs, preserve BrewTestBot bottle handling, and verify the merged formula points at gastownhall/beads. Also fix the local Homebrew upgrade command to use the core formula name, beads.

Agent-Signature: codex-gpt-5.5-high on behalf of matt wilkie
2026-07-04 10:46:49 -07:00
Julian Knutsen
7a1f7e4cde
fix(release): harden shipped release formula adoption (#3668) 2026-05-06 12:14:54 -07:00