* test(testutil): local dolt sql-server backend behind the Dolt test server API Add a second backend to internal/testutil behind the unchanged container API (EnsureDoltContainerForTestMain, RequireDoltContainer, StartIsolatedDoltContainer[Handle], NewContainerProvider, RestartSharedDoltContainer, DoltContainerCrashed, IsolatedDoltContainer Stop/Start/CurrentPort/Exec): the test process spawns `dolt sql-server` from the pinned dolt CLI instead of a dolthub/dolt-sql-server container. - BEADS_TEST_DOLT_SERVER=local|container selects it; unset keeps plain `go test` on the container (no behavior change) and makes `bazel test` use the local backend. Other values fail instead of guessing. - The CLI (BEADS_TEST_DOLT_BINARY, else PATH) must be the image's version. - Provisioned like testcontainers' dolt module: root@% (DOLT_ROOT_HOST=%), database beads_test, user test/test. Clean env, TZ=UTC, GOMAXPROCS=4, own dolt global config with metrics and the version check disabled. - Loopback port picked here (dolt rejects -P 0), never 3307, retried when another process wins the port; readiness requires the server's own "Server ready" line plus a bounded ping, so a foreign listener on a contested port cannot pass for it. - State lives under the TMPDIR captured at package init, outside every PinSuiteTempRoot root, so suite leak sweeps never see it. The server stays in the test's process group (bazel's group kill reaps it) with Pdeathsig on Linux. - BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 now also makes every TestMain that only warned about an unavailable server exit 1 (DoltUnavailableForTestMain), plus the shared proxied harness. - dolt_benchmark_purge_test finds .dolt_dropped_databases relative to Exec's working directory (the data dir in both backends). - TestDoltServerFingerprint checks each available backend against one checked-in fingerprint (users, grants, databases, server variables); local-backend tests cover the bind-race retry, 3307, crash detection, Stop/Start, Exec cwd, state outside the suite root, and teardown. - Bazel: the requires-docker variants pin BEADS_TEST_DOLT_SERVER=container so the docker lane stays the container A/B control; the test wrapper's dolt global config disables metrics and the version check. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(testutil): select the local Dolt test server only when asked for Under Bazel an unset BEADS_TEST_DOLT_SERVER used to mean the local backend, so every non-ci `bazel test` started servers for Dolt suites no lane runs with one (//internal/storage/dolt:dolt_test went from a 16 s skip to a timeout). Unset now means container everywhere: in a Bazel action without docker that is the pre-existing warn+skip, and only a target's env or --test_env=BEADS_TEST_DOLT_SERVER=local starts local servers. Plain `go test` is unchanged. The docker variants keep their container pin: a rule env beats --test_env, so a local-backend invocation cannot flip the docker lane. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(testutil): harden the local Dolt test server's host footprint - DOLT_DISABLE_EVENT_FLUSH=1 for every dolt the backend runs (and in the Bazel test wrapper): the metrics/versioncheck keys stop network calls but not the detached `dolt send-metrics` child each command forks on exit, which re-created the version probe's removed HOME (one leaked bdt-doltver-* per test process) and raced terminate's RemoveAll. - Every bdt-dolt-*/bdt-doltver-* root records its owner PID, and the first use of the backend in a process removes this user's roots whose owner is dead (the SweepDeadSuiteRoots rule; a copy, since doltserver's tests import testutil). A panicking, timed-out or killed test binary, or a package whose TestMain never terminates the singleton, no longer leaks a 40-100 MB state dir per run. - secure_file_priv is the state root's sfp/ directory, set through a per-launch --config file (dolt ignores other flags with --config): a local server runs as the test user on the host with a passwordless root@%, so LOAD_FILE / INTO OUTFILE must not reach the user's files. The fingerprint treats that path as the container's "". - Fingerprint: character_set_server, collation_server, max_allowed_packet, transaction_isolation, lower_case_table_names, lower_case_file_system (compared on Linux only: APFS reports ON), wait_timeout, innodb_lock_wait_timeout. - The local backend's own tests skip off Linux (no Pdeathsig, so a killed test binary would orphan servers on the macOS runners) unless BEADS_TEST_DOLT_SERVER=local asks for them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci: compare the container and local Dolt test servers in the domain+uow job TestDoltServerFingerprint's container half never ran in CI: only the test-domain-uow job has both the pulled image and the pinned dolt CLI, and it never ran ./internal/testutil. Run the fingerprint there (with BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 so the container half cannot skip) so an image bump that drifts the container side fails in the PR. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
141 lines
5.1 KiB
Go
141 lines
5.1 KiB
Go
//go:build cgo
|
|
|
|
package main
|
|
|
|
import (
|
|
"context"
|
|
"database/sql"
|
|
"fmt"
|
|
"os"
|
|
|
|
"github.com/steveyegge/beads/internal/storage/dolt"
|
|
"github.com/steveyegge/beads/internal/storage/schema"
|
|
"github.com/steveyegge/beads/internal/testutil"
|
|
)
|
|
|
|
func init() {
|
|
beforeTestsHook = startTestDoltServer
|
|
}
|
|
|
|
// testSharedDB is the name of the shared database for branch-per-test isolation.
|
|
var testSharedDB string
|
|
|
|
// testSharedConn is a raw *sql.DB for branch operations in the shared database.
|
|
var testSharedConn *sql.DB
|
|
|
|
// startTestDoltServer starts a dedicated Dolt SQL server in a container
|
|
// on a dynamic port using the shared testutil helper. This prevents tests
|
|
// from creating testdb_* databases on the production Dolt server.
|
|
// Returns a cleanup function that stops the server and removes the container.
|
|
func startTestDoltServer() func() {
|
|
if os.Getenv("BEADS_TEST_PROXIED_LOCAL") == "1" {
|
|
return func() {}
|
|
}
|
|
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") == "1" {
|
|
return func() {}
|
|
}
|
|
if os.Getenv("BEADS_TEST_PROXIED_SERVER") == "1" {
|
|
return func() { testutil.TerminateDoltContainer() }
|
|
}
|
|
if err := testutil.EnsureDoltContainerForTestMain(); err != nil {
|
|
if testutil.DoltUnavailableForTestMain(err) {
|
|
// Nothing has been started yet; there is nothing to clean up.
|
|
os.Exit(1)
|
|
}
|
|
return func() {}
|
|
}
|
|
|
|
testDoltServerPort = testutil.DoltContainerPortInt()
|
|
|
|
// Set up shared database for branch-per-test isolation (bd-xmf).
|
|
// Instead of CREATE/DROP DATABASE per test, tests branch from this
|
|
// shared DB, eliminating ~1-2s of overhead per test.
|
|
testSharedDB = "cmdbd_pkg_shared"
|
|
db, err := testutil.SetupSharedTestDB(testDoltServerPort, testSharedDB)
|
|
if err != nil {
|
|
fmt.Fprintf(os.Stderr, "WARNING: shared DB setup failed: %v (falling back to per-test DBs)\n", err)
|
|
testSharedDB = ""
|
|
} else {
|
|
testSharedConn = db
|
|
if err := initCmdBDSharedSchema(testDoltServerPort); err != nil {
|
|
fmt.Fprintf(os.Stderr, "WARNING: shared schema init failed: %v (falling back to per-test DBs)\n", err)
|
|
testSharedDB = ""
|
|
db.Close()
|
|
testSharedConn = nil
|
|
}
|
|
}
|
|
|
|
return func() {
|
|
if testSharedConn != nil {
|
|
testSharedConn.Close()
|
|
testSharedConn = nil
|
|
}
|
|
testSharedDB = ""
|
|
testDoltServerPort = 0
|
|
os.Unsetenv("BEADS_DOLT_PORT")
|
|
testutil.TerminateDoltContainer()
|
|
}
|
|
}
|
|
|
|
// initCmdBDSharedSchema initializes the schema and config on the shared database
|
|
// and commits to main so branches get a clean snapshot.
|
|
func initCmdBDSharedSchema(port int) error {
|
|
ctx, cancel := context.WithTimeout(context.Background(), testStoreOpenTimeout)
|
|
defer cancel()
|
|
cfg := &dolt.Config{
|
|
Path: "/tmp/cmdbd-shared-init",
|
|
ServerHost: "127.0.0.1",
|
|
ServerPort: port,
|
|
Database: testSharedDB,
|
|
// CreateIfMissing must be true even though SetupSharedTestDB already
|
|
// ran `CREATE DATABASE IF NOT EXISTS` moments ago: on a container
|
|
// that only just reported ready, a fresh connection's SHOW DATABASES
|
|
// catalog probe can still lag that DDL (GH-1851-class propagation
|
|
// race), producing a false "database not found" here. Without this,
|
|
// dolt.New takes the immediate-fail branch with no retry at all;
|
|
// with it, dolt.New treats the resulting "database exists" (1007) on
|
|
// its own CREATE DATABASE as proof of existence and falls through to
|
|
// its already-bounded post-create catalog-registration backoff.
|
|
CreateIfMissing: true,
|
|
}
|
|
|
|
// This database lives only inside a container startTestDoltServer just
|
|
// created for this test binary's run and is torn down with it at process
|
|
// exit, so the #4259 remote-migrate gate's "don't independently fork a
|
|
// shared remote's schema" concern does not apply — there is no persistent
|
|
// remote here to fork away from. Without this override, the gate blocks
|
|
// whenever the container image's baked-in schema version lags the current
|
|
// migration set, forcing every store-opening test in the binary onto the
|
|
// slow per-test-DB fallback path (openExistingTestDB and friends in
|
|
// test_helpers_test.go), which in turn becomes unstable — confirmed via
|
|
// be-fgd round-2 triage — once two or more full-migration replays land
|
|
// back-to-back against the same long-lived container.
|
|
schema.SetForceAllowRemoteMigrate(true)
|
|
defer schema.SetForceAllowRemoteMigrate(false)
|
|
store, err := dolt.New(ctx, cfg)
|
|
if err != nil {
|
|
return fmt.Errorf("New: %w", err)
|
|
}
|
|
defer store.Close()
|
|
|
|
if err := store.SetConfig(ctx, "issue_prefix", "test"); err != nil {
|
|
return fmt.Errorf("SetConfig(issue_prefix): %w", err)
|
|
}
|
|
if err := store.SetConfig(ctx, "types.custom", "molecule,gate,convoy,merge-request,slot,agent,role,rig,event,message"); err != nil {
|
|
return fmt.Errorf("SetConfig(types.custom): %w", err)
|
|
}
|
|
|
|
// Commit schema to main so branches get a clean snapshot
|
|
db := store.DB()
|
|
if _, err := db.ExecContext(ctx, "CALL DOLT_ADD('-A')"); err != nil {
|
|
return fmt.Errorf("DOLT_ADD: %w", err)
|
|
}
|
|
if _, err := db.ExecContext(ctx, "CALL DOLT_COMMIT('--allow-empty', '-m', 'test: init shared schema')"); err != nil {
|
|
return fmt.Errorf("DOLT_COMMIT: %w", err)
|
|
}
|
|
if err := testutil.MaterializeLocalTableSchemasForBranchTests(ctx, db); err != nil {
|
|
return fmt.Errorf("materialize local table schemas: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|