* test(bazel): isolate the bazel test environment like pr-core's go test Every `bazel test` process now runs under //tools/bazel:test_env (wired with `test --run_under`), a wrapper that mirrors scripts/ci/lib/test-env.sh: a private HOME/USERPROFILE, XDG_CONFIG_HOME, DOLT_ROOT_PATH and empty GIT_CONFIG_GLOBAL, GIT_CONFIG_NOSYSTEM, BEADS_TEST_IGNORE_REPO_CONFIG, and the same scrub of workspace/Dolt-server variables. Everything lives in one fresh mktemp directory under /tmp that is removed when the test exits, so TMPDIR (t.TempDir) stays short enough for unix sockets and is never shared between concurrent runs. This replaces `--test_env=HOME` (the client's HOME string entered the action key and was shipped to remote workers, where it does not exist: the embeddeddolt suite panicked on "stat /home/ubuntu") and the shared `--test_tmpdir=/tmp/bbt`. No host path or client env enters the action key. bd_test (~10 min under -race -short) and embeddeddolt_test (~6 min) get size = "large" / timeout = "long" instead of Bazel's 300 s default; pr-core gives go test -timeout=30m. rules_shell (already in the module graph) becomes a direct dep for sh_binary. MODULE.bazel.lock records the pybind11 extension that building these test targets evaluates, so `make bazel-sync` is stable. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(bazel): inject the bd binary into subprocess tests (slice B3) Tests that exec bd used to `go build` it themselves, which cannot work under `bazel test` (no Go toolchain or module tree in the sandbox or on a remote worker): 135 cmd/bd subtests failed with "go build failed", and protocol, doctor and uow failed the same way. - //cmd/bd:bd_for_tests: a go_binary named bd, race = "off" (the non-race gms_pure_go binary these tests build today, even under --config=prcore). - bd_test, protocol_test, doctor_test and uow_test declare it as data and set BEADS_TEST_BD_BINARY to its $(rlocationpath). - bazeltest.PrebuiltBD is the one resolver: under Bazel it resolves the rlocationpath through runfiles (via the new bazeltest.RunfileEnv) and requires it; under plain go test it keeps today's semantics (absolute form of BEADS_TEST_BD_BINARY, or "" so the caller builds bd). - cmd/bd's findPrebuiltBDBinary delegates to it, and the sites that read the variable by hand (init_embedded, shared_server_integration, cli_fast, migrate front door) now go through findPrebuiltBDBinary; uow calls it directly. - Raw `go build` sites that never honored the variable (protocol, doctor dolt_e2e, test/conformance e2e, internal/beads integration TestMain) use it only under Bazel, so go test is unchanged; protocol and doctor fail instead of skipping when the injection is miswired. tests/regression is left alone (it builds candidate and baseline binaries on purpose). - The read-only canary still forces a source build under go test; under Bazel the injected binary is already built from this tree. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(bazel): declare what cmd/bd and doctor tests read under bazel With bd injected, the remaining //cmd/bd:bd_test and //cmd/bd/doctor failures under `bazel test` were inputs the tests read from the checkout without declaring them (they were hidden behind the go-build failures before): - source-scanning guards (capability registry, events-journal construction, serve database source) parse cmd/bd, cmd/bd/doctor, cmd/bd/doctor/fix and internal/storage/embeddeddolt/cmd: those packages get go_srcs filegroups (tools/bazel/go_srcs.py) declared as data; serve_test resolves its package dir with bazeltest.CallerDir. - .githooks (managed-section parity) and examples/formulas/primitives are declared through //:bd_test_files. - the gofmt preflight tests exec gofmt: under Bazel they use the Go SDK's, declared as data and named by BEADS_TEST_GOFMT (bazeltest.RunfileEnv). - TestGOMODCACHENotUnderTestHome guards in-test `go build`, which does not happen under Bazel, so it skips there. - doctor's TestCheckDeletionsManifest_LegacyFile only passed because the process cwd happened to be inside a git checkout; its fixture is now a git repo and the check runs in it. go test behavior is unchanged except that the deletions-manifest test no longer depends on the caller's cwd. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(bazel): give test_env a dolt identity; never fail on cleanup beads_test_env_enter sets a dolt user.name/user.email in the fresh DOLT_ROOT_PATH; test_env.sh created it empty, so tests that shell out to dolt (remotecache, storage/dolt) failed with 'Author identity unknown'. Write config_global.json directly (no dolt exec per test). Also keep rm -rf in the EXIT trap from overriding the test's exit status when a leftover child is still writing to the tmp root. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * build(bazel): re-sync BUILD files with main Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
171 lines
5.9 KiB
Python
Executable file
171 lines
5.9 KiB
Python
Executable file
#!/usr/bin/env python3
|
|
"""Maintain `go_srcs` filegroups for tests that scan Go source under Bazel.
|
|
|
|
Some guards parse Go source at test time (journal completeness, role census,
|
|
import boundaries, contract-leg registry). Under `bazel test`, and above all
|
|
under remote execution, a test sees only its declared runfiles, so the source
|
|
it scans must be declared as `data`. Bazel globs stop at package boundaries,
|
|
so a scan over a directory tree needs one filegroup per package plus an
|
|
aggregate. Gazelle does not generate either; this script does, restricted to
|
|
the packages listed below, so only tests that need source pay for it.
|
|
|
|
For every package in PACKAGES, and every package under a root in TREES, the
|
|
BUILD.bazel file gets a managed block holding
|
|
|
|
filegroup(name = "go_srcs", srcs = glob(["**/*.go"]))
|
|
|
|
(the package's own Go files plus those in non-package subdirectories). Each
|
|
TREES root also gets `tree_go_srcs`, aggregating every `go_srcs` under it.
|
|
Blocks in packages that are no longer listed are removed.
|
|
|
|
Run from the repository root after gazelle (see `make bazel-sync`). Output is
|
|
deterministic and gazelle-stable, so a clean sync leaves git clean.
|
|
|
|
With --check nothing is written: stale blocks are printed as a diff and the
|
|
exit status is 1 (see `make bazel-sync-check`).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import difflib
|
|
import os
|
|
import re
|
|
import sys
|
|
|
|
# Packages whose own Go source a test scans.
|
|
PACKAGES = (
|
|
"backend", # //backend:backend_test (public alias census)
|
|
"backend/conformance", # //backend/conformance, //internal/storage
|
|
"beadserrors", # role facade alias targets
|
|
"cmd/bd", # //cmd/bd:bd_test (capability registry, journal, serve scans)
|
|
"cmd/bd/doctor", # //cmd/bd:bd_test (events-journal construction scan)
|
|
"cmd/bd/doctor/fix", # //cmd/bd:bd_test (events-journal construction scan)
|
|
"internal/types", # role facade alias targets
|
|
"issueops", # //backend/conformance (role facade census)
|
|
"journalops", # //backend/conformance (role facade census)
|
|
"memoryops", # //backend/conformance (role facade census)
|
|
)
|
|
|
|
# Roots whose whole Go tree a test scans.
|
|
TREES = (
|
|
"internal/storage", # //internal/storage walks every *_test.go below it
|
|
)
|
|
|
|
BEGIN = "# --- begin go_srcs (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---"
|
|
END = "# --- end go_srcs ---"
|
|
BLOCK_RE = re.compile(r"\n*" + re.escape(BEGIN) + r".*?" + re.escape(END) + r"\n*", re.DOTALL)
|
|
SKIP_DIRS = {"testdata", "node_modules"}
|
|
|
|
|
|
def packages_under(root: str) -> list[str]:
|
|
found = []
|
|
for dirpath, dirnames, files in os.walk(root):
|
|
dirnames[:] = sorted(d for d in dirnames if d not in SKIP_DIRS and not d.startswith("."))
|
|
if "BUILD.bazel" in files:
|
|
found.append(os.path.relpath(dirpath).replace(os.sep, "/"))
|
|
return sorted(found)
|
|
|
|
|
|
def block(pkg: str, tree_members: list[str] | None) -> str:
|
|
lines = [
|
|
BEGIN,
|
|
"",
|
|
"filegroup(",
|
|
' name = "go_srcs",',
|
|
' srcs = glob(',
|
|
' ["**/*.go"],',
|
|
" allow_empty = True,",
|
|
" ),",
|
|
' visibility = ["//:__subpackages__"],',
|
|
")",
|
|
]
|
|
if tree_members is not None:
|
|
lines += [
|
|
"",
|
|
"filegroup(",
|
|
' name = "tree_go_srcs",',
|
|
" srcs = [",
|
|
' ":go_srcs",',
|
|
]
|
|
lines += [f' "//{m}:go_srcs",' for m in tree_members if m != pkg]
|
|
lines += [
|
|
" ],",
|
|
' visibility = ["//:__subpackages__"],',
|
|
")",
|
|
]
|
|
lines += ["", END]
|
|
return "\n".join(lines) + "\n"
|
|
|
|
|
|
def rewrite(path: str, new_block: str | None, check: bool) -> bool:
|
|
"""Bring path's managed block up to date; return True if it was stale.
|
|
|
|
In check mode the file is left alone and the needed change is printed.
|
|
"""
|
|
with open(path) as f:
|
|
src = f.read()
|
|
stripped = BLOCK_RE.sub("\n", src).rstrip("\n") + "\n"
|
|
out = stripped if new_block is None else stripped + "\n" + new_block
|
|
if out == src:
|
|
return False
|
|
if check:
|
|
sys.stdout.writelines(
|
|
difflib.unified_diff(
|
|
src.splitlines(keepends=True),
|
|
out.splitlines(keepends=True),
|
|
fromfile=f"a/{path}",
|
|
tofile=f"b/{path}",
|
|
)
|
|
)
|
|
else:
|
|
with open(path, "w") as f:
|
|
f.write(out)
|
|
return True
|
|
|
|
|
|
def main(argv: list[str]) -> int:
|
|
check = False
|
|
for arg in argv:
|
|
if arg == "--check":
|
|
check = True
|
|
else:
|
|
print(f"usage: go_srcs.py [--check] (unknown argument {arg!r})", file=sys.stderr)
|
|
return 2
|
|
if not (os.path.exists("MODULE.bazel") and os.path.exists("BUILD.bazel")):
|
|
print("run from the repository root", file=sys.stderr)
|
|
return 1
|
|
wanted: dict[str, list[str] | None] = {}
|
|
for pkg in PACKAGES:
|
|
if not os.path.exists(os.path.join(pkg, "BUILD.bazel")):
|
|
print(f"go_srcs.py: {pkg} has no BUILD.bazel", file=sys.stderr)
|
|
return 1
|
|
wanted[pkg] = None
|
|
for root in TREES:
|
|
members = packages_under(root)
|
|
if root not in members:
|
|
print(f"go_srcs.py: tree root {root} has no BUILD.bazel", file=sys.stderr)
|
|
return 1
|
|
for pkg in members:
|
|
wanted.setdefault(pkg, None)
|
|
wanted[root] = members
|
|
stale = []
|
|
for pkg in packages_under("."):
|
|
path = os.path.join(pkg, "BUILD.bazel")
|
|
if pkg in wanted:
|
|
changed = rewrite(path, block(pkg, wanted[pkg]), check)
|
|
else:
|
|
with open(path) as f:
|
|
changed = BEGIN in f.read() and rewrite(path, None, check)
|
|
if changed:
|
|
stale.append(path)
|
|
if check and stale:
|
|
print(
|
|
f"go_srcs.py: {len(stale)} stale go_srcs block(s): {', '.join(stale)}; run `make bazel-sync`",
|
|
file=sys.stderr,
|
|
)
|
|
return 1
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main(sys.argv[1:]))
|