beads/internal/testutil
Jim Wordelman a1ac9f4dcb
fix(testutil): wait for database visibility in SetupSharedTestDB (#5788)
* test(storage): be-s9d setup-shared-db visibility tests for dolt.New / initSharedSchema

Adds five tests pinning the behavioral contract that
testutil.SetupSharedTestDB must result in a database that is fully
visible to fresh MySQL sessions on the same Dolt server. The bug
under be-s9d (originally observed in the BEADS_TEST_EXTERNAL_DOLT_PORT
TestMain branch added in cfd38cda) is that CREATE DATABASE in
SetupSharedTestDB is left in the working set without an explicit
DOLT_COMMIT, so fresh connections that open their own session do not
see the new database. The reviewer's diagnosis (be-nx7 reconciliation
notes) calls out adding CALL DOLT_COMMIT after CREATE DATABASE, or
retry-with-USE in the external-port branch, as the fix surface.

State today (validator's failing tests, written before the fix):

  PASS  TestSetupSharedTestDB_FreshRawConnSeesDatabase
        SHOW DATABASES on a fresh raw mysql conn does list the new db
        (regression-guard for the existing-good behavior).
  FAIL  TestSetupSharedTestDB_NewStoreOpens
        dolt.New() fails with "database not found" against a freshly
        SetupSharedTestDB-created db, even though the prior test shows
        SHOW DATABASES does see it. This is the literal failure mode
        — the difference is dolt.New's initDB connection shape (empty
        ServerUser, internal-built DSN) hits a stricter visibility
        boundary than my User=root probe. Captures the bug.
  PASS  TestSetupSharedTestDB_FreshConnAfterSecondSetupCall
        Idempotent CREATE-IF-NOT-EXISTS path stays visible across two
        setup calls. Regression-guard for any DOLT_COMMIT side effect
        the fix introduces.
  PASS  TestSetupSharedTestDB_RefusesProductionPort
        FIREWALL guard against port 3307 (DefaultSQLPort) intact.
        Defensive contract — must not regress through the be-s9d fix.
  FAIL  TestInitSharedSchema_AfterFreshDatabase
        After initSharedSchema completes (all migrations apply,
        DOLT_ADD + DOLT_COMMIT runs), a fresh connection's SHOW
        TABLES returns zero rows. Schema commits on the
        initSharedSchema session do not surface to other sessions —
        the deeper invariant the fix must restore.

Both failing tests pass on the testcontainer path that masks the bug
in CreateIfMissing=true mode, which is why TestMain external-port is
the only place the suite-wide runtime hits it; my tests pin the
contract at the dolt.New / SHOW TABLES level so the fix is verifiable
without a second host-resident dolt sql-server. The benchstat-geomean
write-bench measurement that be-s9d also requests is a downstream
measurement task, not a test, and is not in this commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(testutil): wait for database visibility in SetupSharedTestDB (be-s9d)

The BEADS_TEST_EXTERNAL_DOLT_PORT branch added in cfd38cda was supposed to
let CI/dev runs target a host-resident dolt sql-server, but the polecat
tests added in 295b4552 surfaced a startup race: SetupSharedTestDB returns
right after CREATE DATABASE, and a sibling pool opened by dolt.New() inside
initSharedSchema can outrace the Dolt server's catalog refresh and fail
with "Error 1049 (HY000): database not found: <name>" — Dolt's own wording
from go-mysql-server, distinct from MySQL's "Unknown database" handled in
isRetryableError since GH-1851.

Three load-bearing changes:

1. testutil.SetupSharedTestDB now polls USE <dbName> on the same handle
   with bounded exponential backoff (50ms → 1s, 10s deadline) before
   returning. The catalog race is what 295b4552's
   TestSetupSharedTestDB_NewStoreOpens / TestInitSharedSchema_AfterFreshDatabase
   were pinning; both flip from RED → GREEN with this poll.

2. dolt.isRetryableError now also retries "database not found", same
   reason as the existing "unknown database" branch (GH-1851). Defensive
   backstop in case any code path opens a fresh pool against the new
   database without first going through SetupSharedTestDB's poll.

3. testmain_test.go now sets BOTH BEADS_DOLT_SERVER_PORT and
   BEADS_DOLT_PORT (and unsets both at end). applyConfigDefaults reads
   BEADS_DOLT_SERVER_PORT first and only falls through to the legacy
   BEADS_DOLT_PORT, so a pre-existing BEADS_DOLT_SERVER_PORT in the
   surrounding agent shell (gc-beads-bd exports the rig's gc.endpoint
   port) silently won over the testmain-set BEADS_DOLT_PORT and routed
   the bench to the wrong server. Both branches (external-port +
   testcontainer) pin both vars.

Tests exercising these paths (added in 295b4552):
  PASS  TestSetupSharedTestDB_FreshRawConnSeesDatabase
  PASS  TestSetupSharedTestDB_NewStoreOpens         (was FAIL at baseline)
  PASS  TestSetupSharedTestDB_FreshConnAfterSecondSetupCall
  PASS  TestSetupSharedTestDB_RefusesProductionPort
  PASS  TestInitSharedSchema_AfterFreshDatabase     (was FAIL at baseline)

Run with TESTCONTAINERS_RYUK_DISABLED=true BEADS_TEST_EXTERNAL_DOLT_PORT=33400.

Unblocks the 10K write-bench measurement called for in be-eei §8 guardrail 5
(CreateIssue/UpdateIssue regression vs pre-D4v2). Bench results to follow
in a separate be-s9d note.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: release gate PASS for shared-test-DB-visibility-waits theme (be-pow0)

* fix(testutil): narrow to test-only per review — drop retry-classifier change, reword visibility-wait comments

bee-ghosttrack's review (2026-09-03) on #5788 flagged that the
isRetryableError "database not found" case was a real production
behaviour change riding along in a PR described as test infrastructure:
it turns a genuinely-missing-database failure from immediate into a full
~30s backoff (serverRetryMaxElapsed) across six withRetryClassified call
sites. Dropping it entirely — SetupSharedTestDB's own visibility poll in
testdoltbranch.go already resolves the catalog race the classifier case
was meant to backstop, so it was never load-bearing for this fix.

Also rewords waitForDatabaseVisible's doc comment to describe what USE on
a pooled connection actually proves — that one connection observed the
catalog refresh, not a pool-wide guarantee, and that USE mutates that
connection's session default database as a side effect — and notes as a
comment (not a behaviour change) that the backoff cap can let delay reach
~1.6s before capping, which is harmless inside the 10s bound.

testmain_test.go's teardown also unset BEADS_DOLT_SERVER_PORT, the other
half of the same port-pin hunk the rebase already dropped in favour of
main's ambient-port FATAL guard (#5837); that half merged cleanly instead
of conflicting, so it survived silently until now. Removed — main never
sets that var, so unsetting it was dead code.

TestSetupSharedTestDB_NewStoreOpens and the rest of the visibility suite
still pass without the classifier change (podman testcontainer run: 5
PASS / 0 FAIL / 0 SKIP).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore: remove leaked internal release-gate doc from PR branch

  release-gates/be-pow0-shared-test-db-visibility-gate.md

Tracked our internal gc-management/beads-rig release-gate checklist and
should never have been committed to this upstream branch -- it's process
residue from our deploy tooling, not part of the change itself.

Per maintainer bee-ghosttrack's request during review of PR #6571. The
root cause (our deploy tooling auto-committing this file) is already fixed
as of gm-qatxl1; this branch predates that fix, so this commit is cleanup
only -- no other content on this branch is touched.

Ref: be-lk7o3

* fix(testutil): poll SHOW DATABASES exact-match for visibility, not USE (be-2wf6u)

waitForDatabaseVisible was polling USE <dbName>, which Dolt can satisfy by
auto-attaching a database to the session before the server's catalog (what
SHOW DATABASES and dolt.New() actually check) has registered it — a no-op
wait in exactly the racy just-created case it exists to guard against
(bee-ghosttrack PR #5788 R2 item 1).

Switch to the same SHOW DATABASES exact-match iteration
databaseExistsOnServer (internal/storage/dolt/store.go) already uses, via a
small local helper (databaseVisibleNow) rather than a cross-package import,
to avoid coupling internal/testutil to internal/storage/dolt for one
10-line check. Preserves the existing backoff/deadline/ctx-cancellation
shape and the "not visible after" bounded error.

Update the call-site and function-doc comments to stop claiming the wait
proves visibility "to a fresh connection" — success only guarantees the
polling connection's own view is current, not a guarantee for every
connection in db's pool. R1 already corrected the function doc to this
wording; the call site had drifted from it (R2 item 2... no, R1 nit,
flagged again in R2 follow-up).

TestWaitForDatabaseVisible_ReturnsBoundedErrorWhenNeverCreated pins the
bounded-failure contract as a falsifiability/regression guard, not an
old-vs-new discriminator: a name that is never created is a permanent
absence the old USE-based poll's retry branch already handled correctly by
coincidence. R2 itself found the real race too flaky to assert on directly
("a mutation probe... did not discriminate either, because the
fresh-connection test is flaky against its own 45s deadline"), so this test
is exactly what R2 prescribed instead. Its doc comment is corrected to say
so accurately.

Verified against a real Dolt container (TESTCONTAINERS_RYUK_DISABLED=true —
without it, Ryuk's reaper handshake times out on this host and produces a
container-setup failure that looks like a test failure but isn't):
internal/testutil full package green (25.9s), internal/storage/dolt's 5/5
visibility suite green (17.1s).

refs be-2wf6u

* chore(bazel): sync gazelle srcs/deps for PR #5788 test files

Maintainer fixup during adoption. `make bazel-sync` output only:
- internal/storage/dolt: enumerate the PR's new untagged setup_shared_db_test.go
  in go_test srcs (without it Bazel silently runs 0 of the PR's 5 new tests).
- internal/testutil: gazelle-canonical deps for the two non-stdlib imports the
  PR adds to the already-enumerated testdoltbranch_test.go.

Base parity at 4e4ebdb44 is 0 untagged gaps / 100 of 100 packages, so the
omission is PR-introduced. Idempotent on re-run.

* fix(testutil): review fixes for PR #5788 — firewall pin + doc accuracy

Maintainer fixup applying the iteration-1 review synthesis (4 of its 5
findings; all minor/nit, no blockers or majors).

- setup_shared_db_test.go: narrow TestSetupSharedTestDB_RefusesProductionPort
  to the one token only the firewall emits ("production") and add a "dial tcp"
  negative control. The old containsAny(err, "refused", "production", "3307")
  was vacuous: the test carries no skipIfNoServer guard, so it also runs where
  nothing listens on 3307, and there a deleted firewall falls through to
  CREATE DATABASE and fails with "dial tcp 127.0.0.1:3307: connect: connection
  refused" — a string matching both "refused" and "3307". Measured: the old
  assertion is true for the firewall message AND that dial error; the new one
  is true only for the firewall message. Same single-specific-needle pattern
  as the sibling TestSetupSharedTestDB_RefusesAmbientPortMismatch.
- setup_shared_db_test.go: mark the be-nx7 DOLT_COMMIT diagnosis in the file
  header as superseded by the catalog-refresh-race diagnosis — the wait
  succeeds before any DOLT_COMMIT runs (initSharedSchema commits only after
  SetupSharedTestDB returns), so a redundant post-CREATE DOLT_COMMIT is not
  the completion of this fix.
- testdoltbranch.go: state the real bound. The deadline is checked before the
  sleep, so the final capped delay lands past it — probes at
  0/50/150/350/750/1550/3150/4750/6350/7950/9550/11150ms, i.e. ~11.2s
  worst-case return over 12 probes, not the "~10s" the doc claimed. The cap
  comment no longer asserts the total stays inside 10s.
- store.go: comment-only back-reference marking databaseExistsOnServer and
  internal/testutil.databaseVisibleNow as a coupled mirror, so a future edit
  to the consumer's predicate cannot silently diverge the fixture's probe.
  The synthesis prescribed this arm over hoisting into doltutil precisely to
  keep the PR's upstream-requested test-only narrowing (eea022b6e) intact; it
  adds no production behavior.

Synthesis finding 3 (deleting the visibility wait is invisible to the
deterministic suite) carries "Required fix: None" — disclosed in-code and not
fixable without an injectable probe; left as recorded residual.

Validation: gofmt clean; go build + go vet on both packages; focused tests
green with the ambient BEADS_DOLT_* env scrubbed — internal/storage/dolt
5 PASS / 0 FAIL / 0 SKIP, internal/testutil 2 PASS / 0 FAIL / 0 SKIP.
make bazel-sync is a no-op (the added import is stdlib).

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
2026-09-29 17:22:06 -07:00
..
bazeltest test(bazel): hermetic test env + inject bd binary into subprocess tests (#6843) 2026-09-27 11:38:03 -07:00
fixtures build(bazel): integration-tag lane (--config=integration) matching main.yml (#6916) 2026-09-28 14:11:34 -07:00
integration ci(bazel): server-Dolt storage and proxied tiers on remote execution (#6940) 2026-09-28 23:01:40 -07:00
testdata testutil: migrate test Dolt server from binary-spawning to testcontainers 2026-02-28 21:12:41 -08:00
BUILD.bazel fix(testutil): wait for database visibility in SetupSharedTestDB (#5788) 2026-09-29 17:22:06 -07:00
container_provider.go test(testutil): opt-in local dolt sql-server backend for Dolt-backed tests (#6891) 2026-09-28 04:06:49 -07:00
container_provider_windows.go /{cmd,internal}: some cleanup 2026-04-08 14:45:18 -07:00
dockercontext.go fix(test): pin DOCKER_HOST from the active docker context before TestMain swaps HOME (bd-84kos) (#5391) 2026-08-07 00:32:40 -07:00
dockercontext_test.go fix(test): pin DOCKER_HOST from the active docker context before TestMain swaps HOME (bd-84kos) (#5391) 2026-08-07 00:32:40 -07:00
git_test_hooks.go Test isolation follow-up: consolidate and guard hooksPath leakage (#2234) (#2237) 2026-03-01 14:41:08 -08:00
localdoltserver.go test(testutil): opt-in local dolt sql-server backend for Dolt-backed tests (#6891) 2026-09-28 04:06:49 -07:00
localdoltserver_linux.go test(testutil): opt-in local dolt sql-server backend for Dolt-backed tests (#6891) 2026-09-28 04:06:49 -07:00
localdoltserver_other.go test(testutil): opt-in local dolt sql-server backend for Dolt-backed tests (#6891) 2026-09-28 04:06:49 -07:00
localdoltserver_test.go test(testutil): opt-in local dolt sql-server backend for Dolt-backed tests (#6891) 2026-09-28 04:06:49 -07:00
suite_temp.go fix(doltserver): recognize GOTMPDIR as a temp-dir sweep root (Go 1.26+) (#6733) 2026-09-26 04:30:59 -07:00
suite_temp_test.go fix(doctor): reap leaked test dolt sql-servers from CGO_ENABLED=0 suites (#5917) 2026-09-02 18:11:33 -07:00
testdoltbranch.go fix(testutil): wait for database visibility in SetupSharedTestDB (#5788) 2026-09-29 17:22:06 -07:00
testdoltbranch_test.go fix(testutil): wait for database visibility in SetupSharedTestDB (#5788) 2026-09-29 17:22:06 -07:00
testdoltcommon.go test(testutil): opt-in local dolt sql-server backend for Dolt-backed tests (#6891) 2026-09-28 04:06:49 -07:00
testdoltserver.go test(testutil): opt-in local dolt sql-server backend for Dolt-backed tests (#6891) 2026-09-28 04:06:49 -07:00
testdoltserver_env_test.go test(testutil): opt-in local dolt sql-server backend for Dolt-backed tests (#6891) 2026-09-28 04:06:49 -07:00
testdoltserver_unreachable_test.go fix(test): recover when a Dolt test server dies mid-suite (#6756) 2026-09-26 14:27:18 -07:00
testdoltserver_windows.go fix(test): recover when a Dolt test server dies mid-suite (#6756) 2026-09-26 14:27:18 -07:00
tmpfs.go style: run gofmt across entire codebase (bd-nmk12) 2026-02-07 19:35:20 -08:00
tmpfs_test.go style: run gofmt across entire codebase (bd-nmk12) 2026-02-07 19:35:20 -08:00