* test(testutil): local dolt sql-server backend behind the Dolt test server API Add a second backend to internal/testutil behind the unchanged container API (EnsureDoltContainerForTestMain, RequireDoltContainer, StartIsolatedDoltContainer[Handle], NewContainerProvider, RestartSharedDoltContainer, DoltContainerCrashed, IsolatedDoltContainer Stop/Start/CurrentPort/Exec): the test process spawns `dolt sql-server` from the pinned dolt CLI instead of a dolthub/dolt-sql-server container. - BEADS_TEST_DOLT_SERVER=local|container selects it; unset keeps plain `go test` on the container (no behavior change) and makes `bazel test` use the local backend. Other values fail instead of guessing. - The CLI (BEADS_TEST_DOLT_BINARY, else PATH) must be the image's version. - Provisioned like testcontainers' dolt module: root@% (DOLT_ROOT_HOST=%), database beads_test, user test/test. Clean env, TZ=UTC, GOMAXPROCS=4, own dolt global config with metrics and the version check disabled. - Loopback port picked here (dolt rejects -P 0), never 3307, retried when another process wins the port; readiness requires the server's own "Server ready" line plus a bounded ping, so a foreign listener on a contested port cannot pass for it. - State lives under the TMPDIR captured at package init, outside every PinSuiteTempRoot root, so suite leak sweeps never see it. The server stays in the test's process group (bazel's group kill reaps it) with Pdeathsig on Linux. - BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 now also makes every TestMain that only warned about an unavailable server exit 1 (DoltUnavailableForTestMain), plus the shared proxied harness. - dolt_benchmark_purge_test finds .dolt_dropped_databases relative to Exec's working directory (the data dir in both backends). - TestDoltServerFingerprint checks each available backend against one checked-in fingerprint (users, grants, databases, server variables); local-backend tests cover the bind-race retry, 3307, crash detection, Stop/Start, Exec cwd, state outside the suite root, and teardown. - Bazel: the requires-docker variants pin BEADS_TEST_DOLT_SERVER=container so the docker lane stays the container A/B control; the test wrapper's dolt global config disables metrics and the version check. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(testutil): select the local Dolt test server only when asked for Under Bazel an unset BEADS_TEST_DOLT_SERVER used to mean the local backend, so every non-ci `bazel test` started servers for Dolt suites no lane runs with one (//internal/storage/dolt:dolt_test went from a 16 s skip to a timeout). Unset now means container everywhere: in a Bazel action without docker that is the pre-existing warn+skip, and only a target's env or --test_env=BEADS_TEST_DOLT_SERVER=local starts local servers. Plain `go test` is unchanged. The docker variants keep their container pin: a rule env beats --test_env, so a local-backend invocation cannot flip the docker lane. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(testutil): harden the local Dolt test server's host footprint - DOLT_DISABLE_EVENT_FLUSH=1 for every dolt the backend runs (and in the Bazel test wrapper): the metrics/versioncheck keys stop network calls but not the detached `dolt send-metrics` child each command forks on exit, which re-created the version probe's removed HOME (one leaked bdt-doltver-* per test process) and raced terminate's RemoveAll. - Every bdt-dolt-*/bdt-doltver-* root records its owner PID, and the first use of the backend in a process removes this user's roots whose owner is dead (the SweepDeadSuiteRoots rule; a copy, since doltserver's tests import testutil). A panicking, timed-out or killed test binary, or a package whose TestMain never terminates the singleton, no longer leaks a 40-100 MB state dir per run. - secure_file_priv is the state root's sfp/ directory, set through a per-launch --config file (dolt ignores other flags with --config): a local server runs as the test user on the host with a passwordless root@%, so LOAD_FILE / INTO OUTFILE must not reach the user's files. The fingerprint treats that path as the container's "". - Fingerprint: character_set_server, collation_server, max_allowed_packet, transaction_isolation, lower_case_table_names, lower_case_file_system (compared on Linux only: APFS reports ON), wait_timeout, innodb_lock_wait_timeout. - The local backend's own tests skip off Linux (no Pdeathsig, so a killed test binary would orphan servers on the macOS runners) unless BEADS_TEST_DOLT_SERVER=local asks for them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci: compare the container and local Dolt test servers in the domain+uow job TestDoltServerFingerprint's container half never ran in CI: only the test-domain-uow job has both the pulled image and the pinned dolt CLI, and it never ran ./internal/testutil. Run the fingerprint there (with BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 so the container half cannot skip) so an image bump that drifts the container side fails in the PR. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
103 lines
5.3 KiB
Bash
Executable file
103 lines
5.3 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Hermetic environment for every `bazel test` action (wired in .bazelrc with
|
|
# `test --run_under=//tools/bazel:test_env`). It mirrors what
|
|
# scripts/ci/lib/test-env.sh does for `go test` in pr-core: a private HOME,
|
|
# XDG config, Dolt root and empty global gitconfig, so no test reads or writes
|
|
# the developer's or the worker's real configuration.
|
|
#
|
|
# Everything lives in one fresh mktemp directory under /tmp, unique per test
|
|
# process: HOME, config and TMPDIR must be short (t.TempDir() paths hold unix
|
|
# sockets, whose sun_path limit is 104 bytes, and Bazel's TEST_TMPDIR is deep
|
|
# under the output base) and must not be shared between concurrent runs of the
|
|
# same target. The directory is removed when the test exits. Only the fixed
|
|
# wrapper text enters the action key; no host path or client env does.
|
|
set -euo pipefail
|
|
|
|
root="$(mktemp -d /tmp/bbt.XXXXXX)"
|
|
# Cleanup must never change the test's exit status: a child the test left
|
|
# running (a detached bd or a Dolt server shutting down) can still be writing
|
|
# when rm runs, and Bazel only reaps it after this wrapper exits.
|
|
trap 'chmod -R u+w "$root" 2>/dev/null || true; rm -rf "$root" 2>/dev/null || true' EXIT
|
|
|
|
mkdir -p "$root/home" "$root/xdg-config" "$root/dolt-root" "$root/tmp"
|
|
: >"$root/gitconfig"
|
|
# Dolt identity, as beads_test_env_enter sets with `dolt config --global`:
|
|
# tests that shell out to dolt commit need an author. Written directly so the
|
|
# wrapper does not depend on a dolt binary. The two *.disabled keys stop every
|
|
# dolt command from checking for a newer release and from sending usage
|
|
# events over the network: calls a hermetic test must not depend on (they
|
|
# change no behavior, only drop a "newer version available" stderr line).
|
|
# They do not stop dolt from forking a detached `dolt send-metrics` child on
|
|
# exit, which outlives the command and re-creates $DOLT_ROOT_PATH/.dolt under
|
|
# a test that already removed it; DOLT_DISABLE_EVENT_FLUSH does.
|
|
mkdir -p "$root/dolt-root/.dolt"
|
|
printf '%s\n' '{"user.email":"test@beads.local","user.name":"beads-test","metrics.disabled":"true","versioncheck.disabled":"true"}' \
|
|
>"$root/dolt-root/.dolt/config_global.json"
|
|
export DOLT_DISABLE_EVENT_FLUSH=1
|
|
|
|
export HOME="$root/home"
|
|
export USERPROFILE="$root/home"
|
|
export XDG_CONFIG_HOME="$root/xdg-config"
|
|
export DOLT_ROOT_PATH="$root/dolt-root"
|
|
export GIT_CONFIG_NOSYSTEM=1
|
|
export GIT_CONFIG_GLOBAL="$root/gitconfig"
|
|
export TMPDIR="$root/tmp"
|
|
export BEADS_TEST_IGNORE_REPO_CONFIG=1
|
|
|
|
# Discovery ceilings. bd walks up from its working directory for .beads and
|
|
# .beads/config.yaml, and git walks up for a repository. A test's working
|
|
# directory is in the runfiles tree under the output base, which is usually
|
|
# below the developer's HOME, so an unbounded walk would read and write their
|
|
# real ~/.beads (and a host config would decide test results). Every walk stops
|
|
# below the runfiles root, TEST_TMPDIR and this wrapper's root; a test's own
|
|
# directories under them stay discoverable. Relying on the ceiling rather than
|
|
# failing on an ancestor .beads is deliberate: a live ~/.beads above the output
|
|
# base is normal on a developer machine.
|
|
ceilings=""
|
|
for d in "${TEST_SRCDIR:-}" "${TEST_TMPDIR:-}" "$root"; do
|
|
if [[ -n "$d" && -d "$d" ]]; then
|
|
ceilings="${ceilings:+$ceilings:}$(cd "$d" && pwd -P)"
|
|
fi
|
|
done
|
|
export BEADS_CEILING_DIRECTORIES="$ceilings"
|
|
export GIT_CEILING_DIRECTORIES="$ceilings"
|
|
# The migration-freeze marker walk is deliberately not bounded by the ceiling;
|
|
# point it at a path that never exists so a MIGRATION-FREEZE file above the
|
|
# output base cannot make write commands refuse inside tests.
|
|
export BD_MIGRATION_FREEZE_FILE="$root/no-freeze-marker"
|
|
|
|
# Same scrub as beads_test_env_enter; `--test_env=NAME` on a command line must
|
|
# not be able to point a test at a live workspace or Dolt server.
|
|
unset BEADS_DIR BEADS_DB BD_DB BD_JSON BD_NO_DB BD_NO_DAEMON BD_ACTOR \
|
|
BEADS_ACTOR GT_ROOT BEADS_DOLT_SHARED_SERVER BEADS_DOLT_SERVER_MODE \
|
|
BEADS_DOLT_AUTO_START BEADS_DOLT_SERVER_HOST BEADS_DOLT_SERVER_PORT \
|
|
BEADS_DOLT_PORT BEADS_DOLT_SERVER_DATABASE BEADS_DOLT_SERVER_SOCKET \
|
|
BEADS_DOLT_PASSWORD BEADS_TEST_REPO_ROOT BEADS_DOLT_BIN
|
|
|
|
# Hermetic host tools: the pinned Dolt CLI (tools/bazel/dolt.bzl) goes first on
|
|
# PATH, so no test runs whatever dolt the executor happens to have, or skips
|
|
# because it has none. The directory comes from this wrapper's own runfiles,
|
|
# which Bazel merges into every test's runfiles, so it is always declared;
|
|
# missing it means broken wiring, and every test fails rather than skipping.
|
|
hermetic_bin=""
|
|
runfiles="${RUNFILES_DIR:-${TEST_SRCDIR:-}}"
|
|
rel="${TEST_WORKSPACE:-_main}/tools/bazel/hermetic_bin"
|
|
if [[ -n "$runfiles" && -x "$runfiles/$rel/dolt" ]]; then
|
|
hermetic_bin="$runfiles/$rel"
|
|
elif [[ -n "${RUNFILES_MANIFEST_FILE:-}" && -f "$RUNFILES_MANIFEST_FILE" ]]; then
|
|
dolt_path="$(awk -v k="$rel/dolt" '$1 == k { print $2; exit }' "$RUNFILES_MANIFEST_FILE")"
|
|
if [[ -n "$dolt_path" && -x "$dolt_path" ]]; then
|
|
hermetic_bin="${dolt_path%/*}"
|
|
fi
|
|
fi
|
|
if [[ -z "$hermetic_bin" ]]; then
|
|
printf 'test_env: hermetic dolt not found at %s in the runfiles of this test (//tools/bazel:hermetic_bin)\n' "$rel/dolt" >&2
|
|
exit 1
|
|
fi
|
|
export PATH="$hermetic_bin:${PATH:-/bin:/usr/bin}"
|
|
export BEADS_TEST_DOLT_BINARY="$hermetic_bin/dolt"
|
|
|
|
# Not exec: the trap must run to remove $root. The exit status is the test's.
|
|
status=0
|
|
"$@" || status=$?
|
|
exit "$status"
|