beads/cmd/bd/test_helpers_test.go
Jim Wordelman 893e42cdaa
perf(export): incremental auto-export via dolt_diff (#5806)
* perf(export): incremental auto-export via dolt_diff

Every bd command that mutates state triggers maybeAutoExport, which
re-serializes every non-infra, non-template issue plus its relational
data to issues.jsonl. On a mature repo with 46k beads (99.5% closed),
that full sweep takes ~110s per trigger and blocks the foreground bd
call, making commands like `bd list` and downstream callers appear to
hang for minutes.

This patches the export in place instead of rewriting from scratch:

- New DiffStore capability interface (internal/storage/diff_store.go)
  returning upserted/removed issue IDs between two Dolt commit hashes.
- DoltStore.ChangedIssueIDs implementation using dolt_diff() unioned
  across issues/labels/dependencies/comments tables. Issue-table removals
  flow to Removed; everything else flows to Upserted. Commit refs are
  validated and inlined since dolt_diff() does not accept prepared
  statement bind params.
- tryIncrementalExport loads the existing issues.jsonl into an
  insertion-ordered map, fetches fresh JSON only for upserted IDs via
  GetIssuesByIDs + the existing bulk loaders, drops removed IDs, and
  atomically renames the result over the old file.
- Falls back to full exportToFile on: no prior commit, missing file,
  store does not implement DiffStore, dolt_diff errors, or the changed
  set exceeds incrementalExportThreshold (5000).
- Full-export paths that run longer than 3s now print an actionable
  stderr warning with the three mitigation levers (bd purge --force,
  export.interval, export.auto).
- saveExportAutoState now writes to a tmp file and renames. The
  previous os.WriteFile leaves a brief O_TRUNC window where a
  concurrent reader sees an empty file, silently parses it as a zero
  exportAutoState, and treats the next run as "no prior commit" - which
  then forces a full export on a repo that would otherwise be
  incremental-eligible.

Safety: upserted IDs that GetIssuesByIDs does not return are NOT dropped
from the export. Treating a fetch miss as a delete is destructive: any
transient fetch hiccup would wipe otherwise-valid records. Real
deletions arrive via changed.Removed from the issues-table diff; only
explicit template/infra filter rejections take items out of the export.

Measured on a 46k-bead repo: 1m48s (full) -> 228ms (incremental with
~5 changed beads). Roughly 470x on the common steady-state path.

Tests (cmd/bd/export_auto_test.go, 8 cases):
- orderedIssueLines insertion-order + in-place replace semantics.
- loadExistingIssueLines parses issues, skips memories, handles
  malformed lines and missing files.
- ChangedIssueIDs detects row modifications, label-only changes, and
  true issue-row deletions with the correct Upserted/Removed split.
- tryIncrementalExport patches changed issues in place, drops deletions,
  drops issues flipped to template between runs, and falls back on
  missing file and threshold breach.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(test): stop dolt test-store hang on fallback per-test-DB path (be-fgd)

Root cause: context.Background() in cmd/bd/test_helpers_test.go's
per-test-DB fallback path had no deadline, so a slow (not stuck) full
v0->v65 migration replay could block indefinitely once the shared-schema
fast path fell back. Goroutine dump confirmed the block site: a per-test
dolt.New() sitting in mysqlConn.readWithTimeout/net.Read.

Two-part fix:
- Make the shared-DB fast path succeed reliably instead of always
  falling back: initCmdBDSharedSchema now sets
  schema.SetForceAllowRemoteMigrate(true) around its dolt.New() so the
  v52->v65 migration applies against the ephemeral per-run test
  container (the #4259 remote-migrate gate exists to protect persistent
  remotes, which doesn't apply here).
- Bound the rare/backstop fallback path itself: testStoreOpenTimeout is
  calibrated from a real isolated measurement (219.62s for one full
  migration replay, forced via a temporary diagnostic hook and reverted)
  rather than guessed, giving ~37% margin over that observed duration.

Refs be-hka, be-fgd.

* test(export): red — bulk-seed pool read timeout plumbing (refs be-uoat)

TestNewTestStoreWithReadTimeout_AppliesConfiguredTimeout and the
timeout-aware setupIncrementalExportTestWithReadTimeout wrapper
reference newTestStoreWithPrefixAndReadTimeout, which does not exist
yet (cmd/bd fails to vet/build: "undefined:
newTestStoreWithPrefixAndReadTimeout"). This expresses the be-uoat
round-2 gate's acceptance criterion: the bulk-seed helper used by
TestTryIncrementalExport_ThresholdExceededFallsBack needs a
caller-configurable Config.PoolReadTimeout so it stops flaking under
host contention on the shared-branch MaxOpenConns=1 connection,
without changing the pool timeout for any other test.

* fix(test): green — thread configurable PoolReadTimeout into shared-branch test store helpers (refs be-uoat)

Add newTestStoreSharedBranchWithReadTimeout and
newTestStoreWithPrefixAndReadTimeout, siblings of the existing
newTestStoreSharedBranch / newTestStoreWithPrefix that additionally
thread a caller-supplied dolt.Config.PoolReadTimeout through both the
shared-branch fast path and the per-test-DB fallback.

cmd/bd now vets and builds clean; TestTryIncrementalExport_ThresholdExceededFallsBack
and TestNewTestStoreWithReadTimeout_AppliesConfiguredTimeout both run
(PASS/SKIP per Dolt test-server availability, no compile error).

Existing callers are untouched — newTestStore/newTestStoreWithPrefix
still default to defaultPoolReadTimeout (10s) via internal/storage/dolt.
This only affects the one bulk-seed helper that opts in.

* fix(test): green — assert on returned error instead of subtest propagation in PoolReadTimeout plumbing test (refs be-8wnq)

TestNewTestStoreWithReadTimeout_AppliesConfiguredTimeout's 'unreasonably
short timeout fails fast' case wrapped its expected-to-fail store creation
in t.Run and checked the returned bool. Go's testing package unconditionally
propagates a failed subtest to the parent test and process exit code, so
that pattern could never report an overall PASS even when the PoolReadTimeout
plumbing was working correctly.

Add tryNewTestStoreWithReadTimeout, the dolt.New portion of
newTestStoreWithPrefixAndReadTimeout with the error returned directly
instead of calling t.Fatal. The test now asserts on that error value with
no subtest wrapper for the expected-to-fail branch.

* docs(release-gate): PASS — incremental auto-export via dolt_diff, round 4 (be-y1jo)

Deploy gate re-evaluation for be-hka's incremental auto-export feature,
round 2 of deploy (be-y1jo, superseding be-uoat's criterion-3 FAIL).

Round 3 (be-8wnq) root-caused be-uoat's flake to the pool's 10s default
read timeout colliding with a 5001-row bulk seed under host contention,
and fixed it by threading a configurable PoolReadTimeout (5m for this
one bulk-seed path) through the shared test-store helpers -- soaked 3/3
PASS. Round 4 (be-unlq, commit fc5caadcf) fixed a separate, deterministic
test-logic defect (Go subtest failure unconditionally propagating to the
parent) that round 3 found in the new plumbing-proof test.

Independently re-verified rather than trusted: all 8 diff-owned tests
green against a real Dolt container, including 3 fresh runs of the
previously-flaky test (43.9-45.5s each, zero failures -- 6/6 real-
container passes post-fix combined with round 3's soak). Build/vet/
gofmt clean. Policy lane's one failure independently confirmed
pre-existing and diff-unrelated (.githooks/commit-msg, byte-identical
to origin/main). Ancestry-scope check's single stray-commit flag
independently confirmed a benign artifact of the extraction workflow
(the flagged commit predates be-hka; its 4 files are a strict subset
of the approved 7-file scope).

refs be-y1jo, be-hka, be-uoat, be-8wnq, be-unlq

* test(feat): red — Fix PR #5806 review: incremental auto-export never fires (state hash is not a commit) + memory/owner/format contract breaks (refs be-shbed)

* fix: green — Fix PR #5806 review: incremental auto-export never fires (state hash is not a commit) + memory/owner/format contract breaks (refs be-shbed)

* fix: green — missingJSONLIssueIDsInStore no longer blocks incremental export on proven deletions (refs be-nti0e)

Round 2 fix for bee-ghosttrack's CHANGES_REQUESTED on PR #5806.
TestMaybeAutoExport_SecondRunTakesIncrementalPath_ServerMode regressed
because missingJSONLIssueIDsInStore treated any JSONL-only issue id as
corruption -- including ids a legitimate `bd delete` removed between
exports -- permanently wedging tryIncrementalExport's fast path in
server mode.

missingJSONLIssueIDsInStore now takes the last successful export's diff
anchor (state.LastDiffAnchor) and, for whatever candidates remain after
the existing ephemeral/template/infra-type filtering, asks
ChangedIssueIDs (dolt_diff over issues/labels/dependencies/comments)
whether each was actually removed since that anchor. Proven deletions
are excluded from the missing set; anything not provably deleted still
falls back to the original conservative refusal. No anchor (cold start)
falls through unchanged, same as before this check existed.

Also fixes cmd/bd/export_auto_missing_store_test.go's call site for the
new 3-arg signature, and renames export_auto_test.go's idSet helper to
idSetFromIDs to resolve a rebase collision with PR #5822's own idSet in
stale_embedded_test.go.

refs be-nti0e

* test(export): assert the incremental path ran, not just that a diff was reached (refs be-re1zl)

PR #5806 round-5 review, finding 1. Deleting e2e-b makes it an orphan-guard
candidate, so the guard's own ChangedIssueIDs call satisfies a bare
`changedIssueIDsCalls < 1` check even when tryIncrementalExport falls back to
a full export -- the test passed while proving nothing about the path it is
named for.

Use both oracles the reviewer verified against a real server: an exact call
count of 2 (orphan-guard deletion probe + incremental diff), and
LastDirtyIDs == {e2e-a}, which maybeAutoExport explicitly nils on the
full-export fallback.

* fix(export): require the diff anchor to be on HEAD's history before trusting a removal (refs be-re1zl)

PR #5806 round-5 review, finding 2: "removed since anchor" is not the same
claim as "deleted by bd delete". dolt_diff(anchor, WORKING) reports a row as
removed whenever it is absent at WORKING, which is equally true when the
history moved out from under the anchor -- a CALL DOLT_RESET('--hard', ...),
a branch checkout, or any other data-dir rewind on a shared server. The orphan
guard honoured that as proof of deletion, so auto-export silently dropped a
live record from issues.jsonl: exactly the #4988 corruption class the guard
exists to prevent.

Gate the deletion proof on the anchor still being reachable from HEAD
(CommitExists queries dolt_log, i.e. HEAD's forward history). Fails closed --
a store that cannot answer, or an error while asking, keeps refusing, which is
the conservative pre-existing behaviour.

TestMaybeAutoExport_HistoryRewindDoesNotProveDeletion reproduces the reviewer's
repro (export, create rw-x, export, DOLT_RESET --hard back to the baseline,
export) and fails without this change.

Also records the residual the reviewer noted: an unresolvable anchor (after a
history squash or GC) plus a pending delete is a permanent wedge, tracked by
 #5896.

* docs(changelog): scope incremental export and the delete guard to server mode (refs be-re1zl)

PR #5806 round-5 review, finding 3 and note (b).

- Add the orphan-guard change as its own user-visible entry, including how
  narrow the deletion proof deliberately is.
- State that both the incremental path and the guard are server/DiffStore-mode
  only: EmbeddedDoltStore implements neither DiffStore nor StateHasher, so in
  embedded mode every cycle is still a full export and bd delete still wedges
  auto-export there -- #5896 stays open for it.
- Add the anchor-never-advances caveat: with auto-commit off and nothing
  committing, the diff range grows until the 5000-id threshold sends every
  export back to a full rewrite.
- Stop claiming the memory-record handling "matches full-export behavior
  exactly". It deliberately does not: the full path refuses to overwrite a
  JSONL containing memory lines, the incremental path preserves them verbatim
  and proceeds. Preserving is the better behaviour; say so instead.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-08-22 06:49:19 -07:00

443 lines
15 KiB
Go

//go:build cgo
// Cgo-only test helpers for cmd/bd. Helpers in this file pull in
// internal/storage/dolt, database/sql, and the embedded Dolt server, which
// require cgo to link. Pure-Go-compatible helpers (captureStdout,
// stdioMutex, runCommandInDir, etc.) live in test_helpers_pure_test.go and
// are intentionally untagged so non-cgo tests in this package compile under
// CGO_ENABLED=0 with the gms_pure_go build tag.
package main
import (
"context"
"database/sql"
"fmt"
"os"
"path/filepath"
"testing"
"time"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/storage/dolt"
"github.com/steveyegge/beads/internal/storage/doltutil"
"github.com/steveyegge/beads/internal/testutil"
)
// testDoltServerPort is the port of the shared test Dolt server (0 = not running).
var testDoltServerPort int
// testStoreOpenTimeout bounds dolt.New/dolt.NewFromConfig calls in these
// helpers. The testcontainers wait strategy only confirms the TCP listener
// is up, not that Dolt's SQL engine can serve queries yet (see
// waitForDoltReady in internal/testutil/testdoltserver.go for the same class
// of race on shared-container startup). A query issued with an unbounded
// context in that narrow window can block indefinitely instead of erroring,
// because nothing ever cancels it to unstick the read — confirmed via
// goroutine dump during be-fgd round-2 triage: a per-test dolt.New() sat in
// mysqlConn.readWithTimeout / net.Read for 2+ minutes after the container
// had already logged ready.
//
// The fallback branch (per-test DB, exercised only when shared-schema init
// in test_dolt_server_cgo_test.go fails) runs a full v0->v65 migration
// replay from scratch, which is genuinely slow, not hung: a single isolated
// measurement (BEADS_TEST_FORCE_FALLBACK_DIAG diagnostic, reverted after
// use) clocked one such replay at 219.62s on this hardware. 60s (an earlier
// calibration attempt, matching contributor_routing_e2e_test.go's precedent)
// measurably undershot this and produced a clean "context deadline exceeded"
// at exactly 60.01s instead of the real result. 300s gives that observed
// value ~37% margin for run-to-run variance while still failing fast
// relative to an actually-stuck connection. In normal operation the shared
// fast path (newTestStoreSharedBranch) avoids this branch entirely and
// resolves in under a second; this bound only matters for the rare/backstop
// fallback case.
const testStoreOpenTimeout = 300 * time.Second
// writeTestMetadata writes metadata.json in the .beads directory (parent of dbPath)
// so that NewFromConfig can find the correct database name and server settings when
// routing reopens a store by path.
func writeTestMetadata(t *testing.T, dbPath string, database string) {
t.Helper()
beadsDir := filepath.Dir(dbPath)
if err := os.MkdirAll(beadsDir, 0755); err != nil {
t.Fatalf("Failed to create beads dir: %v", err)
}
cfg := &configfile.Config{
Database: "dolt",
Backend: configfile.BackendDolt,
DoltMode: configfile.DoltModeServer,
DoltDatabase: database,
DoltServerHost: "127.0.0.1",
DoltServerPort: testDoltServerPort,
}
if err := cfg.Save(beadsDir); err != nil {
t.Fatalf("Failed to write test metadata.json: %v", err)
}
}
// newTestStore creates a dolt store with issue_prefix configured (bd-166).
// Uses shared database with branch-per-test isolation (bd-xmf) to avoid
// the overhead of CREATE/DROP DATABASE per test.
// Falls back to per-test databases if the shared DB is not available.
func newTestStore(t *testing.T, dbPath string) *dolt.DoltStore {
t.Helper()
return newTestStoreWithPrefix(t, dbPath, "test")
}
// newTestStoreIsolatedDB creates a dolt store with its own dedicated database.
// Use this instead of newTestStoreWithPrefix when the test needs a truly separate
// database (e.g., routing tests that create multiple stores with different paths
// and expect routing to reopen them by path via metadata.json).
func newTestStoreIsolatedDB(t *testing.T, dbPath string, prefix string) *dolt.DoltStore {
t.Helper()
ensureTestMode(t)
if testDoltServerPort == 0 {
t.Skip("Dolt test server not available, skipping")
}
if testutil.DoltContainerCrashed() {
t.Skipf("Dolt test server crashed: %v", testutil.DoltContainerCrashError())
}
ctx, cancel := context.WithTimeout(context.Background(), testStoreOpenTimeout)
defer cancel()
cfg := &dolt.Config{
Path: dbPath,
ServerHost: "127.0.0.1",
ServerPort: testDoltServerPort,
Database: uniqueTestDBName(t),
CreateIfMissing: true,
}
writeTestMetadata(t, dbPath, cfg.Database)
doltNewMutex.Lock()
s, err := dolt.New(ctx, cfg)
doltNewMutex.Unlock()
if err != nil {
t.Fatalf("Failed to create dolt store: %v", err)
}
if err := s.SetConfig(ctx, "issue_prefix", prefix); err != nil {
s.Close()
t.Fatalf("Failed to set issue_prefix: %v", err)
}
if err := s.SetConfig(ctx, "types.custom", "molecule,gate,convoy,merge-request,slot,agent,role,rig,event,message"); err != nil {
s.Close()
t.Fatalf("Failed to set types.custom: %v", err)
}
t.Cleanup(func() {
s.Close()
if cfg.Database != "" {
dropTestDatabase(cfg.Database, testDoltServerPort)
}
})
return s
}
// newTestStoreWithPrefix creates a dolt store with custom issue_prefix configured.
// Uses shared database with branch-per-test isolation (bd-xmf) when available,
// falling back to per-test databases otherwise.
func newTestStoreWithPrefix(t *testing.T, dbPath string, prefix string) *dolt.DoltStore {
t.Helper()
ensureTestMode(t)
if testDoltServerPort == 0 {
t.Skip("Dolt test server not available, skipping")
}
if testutil.DoltContainerCrashed() {
t.Skipf("Dolt test server crashed: %v", testutil.DoltContainerCrashError())
}
// Fast path: use shared DB with branch-per-test isolation (bd-xmf)
if testSharedDB != "" {
return newTestStoreSharedBranch(t, dbPath, prefix)
}
// Fallback: per-test database (original slow path)
ctx, cancel := context.WithTimeout(context.Background(), testStoreOpenTimeout)
defer cancel()
cfg := &dolt.Config{
Path: dbPath,
ServerHost: "127.0.0.1",
ServerPort: testDoltServerPort,
Database: uniqueTestDBName(t),
CreateIfMissing: true,
}
writeTestMetadata(t, dbPath, cfg.Database)
doltNewMutex.Lock()
s, err := dolt.New(ctx, cfg)
doltNewMutex.Unlock()
if err != nil {
t.Fatalf("Failed to create dolt store: %v", err)
}
if err := s.SetConfig(ctx, "issue_prefix", prefix); err != nil {
s.Close()
t.Fatalf("Failed to set issue_prefix: %v", err)
}
if err := s.SetConfig(ctx, "types.custom", "molecule,gate,convoy,merge-request,slot,agent,role,rig,event,message"); err != nil {
s.Close()
t.Fatalf("Failed to set types.custom: %v", err)
}
t.Cleanup(func() {
s.Close()
if cfg.Database != "" {
dropTestDatabase(cfg.Database, testDoltServerPort)
}
})
return s
}
// newTestStoreSharedBranch creates a store using the shared database with
// branch-per-test isolation. Each test gets its own Dolt branch, avoiding
// the expensive CREATE DATABASE + schema init + DROP DATABASE + PURGE cycle.
func newTestStoreSharedBranch(t *testing.T, dbPath string, prefix string) *dolt.DoltStore {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), testStoreOpenTimeout)
defer cancel()
// Write metadata.json pointing to the shared database
writeTestMetadata(t, dbPath, testSharedDB)
// Open store against the shared database with MaxOpenConns=1
// (required for DOLT_CHECKOUT session affinity)
doltNewMutex.Lock()
s, err := dolt.New(ctx, &dolt.Config{
Path: dbPath,
ServerHost: "127.0.0.1",
ServerPort: testDoltServerPort,
Database: testSharedDB,
MaxOpenConns: 1,
})
doltNewMutex.Unlock()
if err != nil {
t.Fatalf("Failed to create dolt store (shared): %v", err)
}
// Create isolated branch for this test
_, branchCleanup := testutil.StartTestBranch(t, s.DB(), testSharedDB)
// Set prefix for this test (overrides the shared schema's default)
if err := s.SetConfig(ctx, "issue_prefix", prefix); err != nil {
branchCleanup()
s.Close()
t.Fatalf("Failed to set issue_prefix: %v", err)
}
t.Cleanup(func() {
branchCleanup()
s.Close()
})
return s
}
// newTestStoreSharedBranchWithReadTimeout is newTestStoreSharedBranch with a
// caller-specified Config.PoolReadTimeout. Existing shared-branch callers
// stay on the 10s default (defaultPoolReadTimeout in internal/storage/dolt);
// this is for the rare test whose own write volume — not server health —
// needs more slack, e.g. a bulk seed of thousands of rows over the single
// held connection that MaxOpenConns=1 forces on this path (be-uoat round 2).
func newTestStoreSharedBranchWithReadTimeout(t *testing.T, dbPath string, prefix string, readTimeout time.Duration) *dolt.DoltStore {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), testStoreOpenTimeout)
defer cancel()
// Write metadata.json pointing to the shared database
writeTestMetadata(t, dbPath, testSharedDB)
// Open store against the shared database with MaxOpenConns=1
// (required for DOLT_CHECKOUT session affinity)
doltNewMutex.Lock()
s, err := dolt.New(ctx, &dolt.Config{
Path: dbPath,
ServerHost: "127.0.0.1",
ServerPort: testDoltServerPort,
Database: testSharedDB,
MaxOpenConns: 1,
PoolReadTimeout: readTimeout,
})
doltNewMutex.Unlock()
if err != nil {
t.Fatalf("Failed to create dolt store (shared): %v", err)
}
// Create isolated branch for this test
_, branchCleanup := testutil.StartTestBranch(t, s.DB(), testSharedDB)
// Set prefix for this test (overrides the shared schema's default)
if err := s.SetConfig(ctx, "issue_prefix", prefix); err != nil {
branchCleanup()
s.Close()
t.Fatalf("Failed to set issue_prefix: %v", err)
}
t.Cleanup(func() {
branchCleanup()
s.Close()
})
return s
}
// newTestStoreWithPrefixAndReadTimeout is newTestStoreWithPrefix with a
// caller-specified Config.PoolReadTimeout, threaded through whichever branch
// (shared-DB fast path or per-test-DB fallback) actually runs. See
// newTestStoreSharedBranchWithReadTimeout for why this knob exists.
func newTestStoreWithPrefixAndReadTimeout(t *testing.T, dbPath string, prefix string, readTimeout time.Duration) *dolt.DoltStore {
t.Helper()
ensureTestMode(t)
if testDoltServerPort == 0 {
t.Skip("Dolt test server not available, skipping")
}
if testutil.DoltContainerCrashed() {
t.Skipf("Dolt test server crashed: %v", testutil.DoltContainerCrashError())
}
// Fast path: use shared DB with branch-per-test isolation (bd-xmf)
if testSharedDB != "" {
return newTestStoreSharedBranchWithReadTimeout(t, dbPath, prefix, readTimeout)
}
// Fallback: per-test database (original slow path)
ctx, cancel := context.WithTimeout(context.Background(), testStoreOpenTimeout)
defer cancel()
cfg := &dolt.Config{
Path: dbPath,
ServerHost: "127.0.0.1",
ServerPort: testDoltServerPort,
Database: uniqueTestDBName(t),
CreateIfMissing: true,
PoolReadTimeout: readTimeout,
}
writeTestMetadata(t, dbPath, cfg.Database)
doltNewMutex.Lock()
s, err := dolt.New(ctx, cfg)
doltNewMutex.Unlock()
if err != nil {
t.Fatalf("Failed to create dolt store: %v", err)
}
if err := s.SetConfig(ctx, "issue_prefix", prefix); err != nil {
s.Close()
t.Fatalf("Failed to set issue_prefix: %v", err)
}
if err := s.SetConfig(ctx, "types.custom", "molecule,gate,convoy,merge-request,slot,agent,role,rig,event,message"); err != nil {
s.Close()
t.Fatalf("Failed to set types.custom: %v", err)
}
t.Cleanup(func() {
s.Close()
if cfg.Database != "" {
dropTestDatabase(cfg.Database, testDoltServerPort)
}
})
return s
}
// tryNewTestStoreWithReadTimeout is the dolt.New portion of
// newTestStoreWithPrefixAndReadTimeout (same branching, same Config shape),
// returning the error directly instead of calling t.Fatal on it. Use this
// when the caller expects the open itself to fail — e.g. proving a short
// PoolReadTimeout is actually honored — since a t.Run subtest's failure
// always propagates FAIL to the parent test and the process exit code in
// Go's testing package regardless of what the parent does with t.Run's bool
// return; there is no way to observe an "expected" subtest failure without
// also failing the overall run.
func tryNewTestStoreWithReadTimeout(t *testing.T, dbPath string, readTimeout time.Duration) (*dolt.DoltStore, error) {
t.Helper()
ensureTestMode(t)
if testDoltServerPort == 0 {
t.Skip("Dolt test server not available, skipping")
}
if testutil.DoltContainerCrashed() {
t.Skipf("Dolt test server crashed: %v", testutil.DoltContainerCrashError())
}
ctx, cancel := context.WithTimeout(context.Background(), testStoreOpenTimeout)
defer cancel()
// Fast path: use shared DB with branch-per-test isolation (bd-xmf)
if testSharedDB != "" {
writeTestMetadata(t, dbPath, testSharedDB)
doltNewMutex.Lock()
s, err := dolt.New(ctx, &dolt.Config{
Path: dbPath,
ServerHost: "127.0.0.1",
ServerPort: testDoltServerPort,
Database: testSharedDB,
MaxOpenConns: 1,
PoolReadTimeout: readTimeout,
})
doltNewMutex.Unlock()
return s, err
}
// Fallback: per-test database (original slow path)
cfg := &dolt.Config{
Path: dbPath,
ServerHost: "127.0.0.1",
ServerPort: testDoltServerPort,
Database: uniqueTestDBName(t),
CreateIfMissing: true,
PoolReadTimeout: readTimeout,
}
writeTestMetadata(t, dbPath, cfg.Database)
doltNewMutex.Lock()
s, err := dolt.New(ctx, cfg)
doltNewMutex.Unlock()
return s, err
}
// dropTestDatabase drops a test database from the shared server (best-effort cleanup).
func dropTestDatabase(dbName string, port int) {
dsn := doltutil.ServerDSN{Host: "127.0.0.1", Port: port, User: "root"}.String()
db, err := sql.Open("mysql", dsn)
if err != nil {
return
}
defer db.Close()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
//nolint:gosec // G201: dbName is generated by uniqueTestDBName (testdb_ + random hex)
_, _ = db.ExecContext(ctx, fmt.Sprintf("DROP DATABASE IF EXISTS `%s`", dbName))
// Purge dropped databases from Dolt's trash directory to reclaim disk space
_, _ = db.ExecContext(ctx, "CALL dolt_purge_dropped_databases()")
}
// openExistingTestDB reopens an existing Dolt store for verification in tests.
// It tries NewFromConfig first (reads metadata.json for correct database name),
// then falls back to direct open for BEADS_DB or other non-standard paths.
func openExistingTestDB(t *testing.T, dbPath string) (*dolt.DoltStore, error) {
t.Helper()
// Serialize dolt.New() to avoid race in Dolt's InitStatusVariables (bd-cqjoi)
doltNewMutex.Lock()
defer doltNewMutex.Unlock()
ctx, cancel := context.WithTimeout(context.Background(), testStoreOpenTimeout)
defer cancel()
// Try NewFromConfig which reads metadata.json for correct database name
beadsDir := filepath.Dir(dbPath)
if store, err := dolt.NewFromConfig(ctx, beadsDir); err == nil {
return store, nil
}
// Fallback: open directly with test server config
cfg := &dolt.Config{Path: dbPath}
if testDoltServerPort != 0 {
cfg.ServerHost = "127.0.0.1"
cfg.ServerPort = testDoltServerPort
}
return dolt.New(ctx, cfg)
}