8 KiB
Testing Guide
Overview
The beads project uses Go tests plus repository wrapper scripts. Prefer the
wrapper scripts for local validation because they apply the repository's normal
local build flags, skip policy, and timeout policy. The current GitHub Actions
PR contract still runs direct go test commands; the CI cleanup plan will move
that contract behind dedicated scripts/ci/* wrappers.
Test Performance
- Go compilation dominates full-suite runtime.
- Target package/test runs are usually the fastest way to validate focused changes.
- Docker-backed Dolt integration tests auto-detect prerequisites and skip when unavailable.
Running Tests
Quick Start
# Run all tests (auto-skips known broken tests)
make test
# Or directly:
./scripts/test.sh
# Run opt-in ICU regex path tests (maintainer-only, not normal validation)
make test-icu-path
# Run specific package
./scripts/test.sh ./cmd/bd/...
# Run specific test pattern
./scripts/test.sh -run TestCreate ./cmd/bd/...
# Verbose output
./scripts/test.sh -v
Environment Variables
# Set custom timeout (default: 3m)
TEST_TIMEOUT=5m ./scripts/test.sh
# Enable verbose output
TEST_VERBOSE=1 ./scripts/test.sh
# Run specific pattern
TEST_RUN=TestCreate ./scripts/test.sh
Docker (Dolt Integration Tests)
Dolt integration tests require Docker with the exact Dolt image cached locally. Tests auto-detect the environment and skip gracefully — no manual configuration needed.
Readiness states
State,Condition,Behavior
doltSkipped,BEADS_TEST_SKIP contains "dolt",Silent skip (no warning)
doltNoDocker,Docker daemon not reachable,WARN + skip
doltNoImage,No Dolt image at all,WARN + skip with pull instruction
doltWrongVersion,Image repo cached but wrong tag,WARN + skip with pull instruction
doltReady,Exact image cached and Docker running,Run tests
States are checked once per test binary and cached. Order of evaluation:
BEADS_TEST_SKIP → Docker availability → exact image → any image version.
Skipping Dolt tests explicitly
Set BEADS_TEST_SKIP to opt out without Docker overhead (~1s docker info):
# Skip Dolt tests silently
BEADS_TEST_SKIP=dolt ./scripts/test.sh
# Skip multiple services (comma-separated)
BEADS_TEST_SKIP=dolt,slow ./scripts/test.sh
Short Mode and Test Boundaries
testing.Short() is reserved for true runtime, stress, and large-fixture skips.
It must not be used as an implicit integration, e2e, API, Docker, or external
dependency boundary.
Use these mechanisms instead:
//go:build integrationor//go:build e2efor named suites.- Environment readiness checks such as
BEADS_TEST_SKIP=dolt,BEADS_TEST_EMBEDDED_DOLT=1, or required API-key checks. - Named wrappers such as
make ci-pr-core, the main integration shards, and the package gate wrappers.
Run make check-testing-short to verify that new testing.Short() usage stays
within the approved runtime/stress/large-fixture allowlist. The PR policy wrapper
runs the same check.
Enabling Dolt tests
# Pull the exact Dolt image to enable integration tests
docker pull dolthub/dolt-sql-server:2.2.0
# Point tests at an existing Dolt server (skips container startup)
BEADS_DOLT_PORT=3308 ./scripts/test.sh
BEADS_DOLT_PORT — when set, tests reuse the server at that port instead of
starting a container. Port 3307 is hardcoded as production and always rejected.
Advanced Usage
# Skip additional tests beyond .test-skip
./scripts/test.sh -skip SomeSlowTest
# Run with custom timeout
./scripts/test.sh -timeout 5m
# Combine flags
./scripts/test.sh -v -run TestCreate ./internal/beads/...
Known Broken Tests
Tests in .test-skip are automatically skipped by scripts/test.sh.
At the time of this review, .test-skip contains only comments and no active
test-name patterns. Treat any new skip as a temporary exception: file the
upstream issue first, record it in .test-skip, and remove the skip when the
test is fixed.
For Claude Code / AI Agents
When running tests during development:
Best Practices
-
Use the test script: Always use
./scripts/test.shinstead ofgo testdirectly- Automatically skips known broken tests
- Uses appropriate timeouts
- Matches local default validation; use future
scripts/ci/*wrappers when reproducing exact CI jobs - Only if intentionally exercising the ICU regex path, use
./scripts/test-icu-path.sh(or deprecatedmake test-full-cgo)
-
Target specific tests when possible:
# Instead of running everything: ./scripts/test.sh # Run just what you changed: ./scripts/test.sh -run TestSpecificFeature ./cmd/bd/... -
Compilation is the bottleneck:
- The 180-second compilation time dominates
- Individual tests are fast
- Use
-runto avoid recompiling unnecessarily
-
Check for new failures:
# If you see a new failure, check if it's known: cat .test-skip
Adding Tests to Skip List
If you discover a broken test:
- File a GitHub issue documenting the problem
- Add to
.test-skip:# Issue #NNN: Brief description TestNameToSkip - Tests in
.test-skipsupport regex patterns
Test Organization
Slowest Tests (>0.05s)
The top slow tests in cmd/bd:
TestDoctorWithBeadsDir(1.68s) - Only significantly slow testTestFlushManagerDebouncing(0.21s)TestDebouncer_*tests (0.06-0.12s each) - Intentional sleeps for concurrency testingTestMultiWorkspaceDeletionSync(0.12s)
Most tests are <0.01s and very fast.
Package Structure
cmd/bd/ - Main CLI tests (82 test files, most of the suite)
internal/beads/ - Core beads library tests
internal/storage/ - Storage backend tests (Dolt embedded/server, conformance)
internal/rpc/ - RPC protocol tests
internal/*/ - Various internal package tests
Continuous Integration
The current CI workflow does not yet call the scripts/ci/* wrappers for every
job. Until workflow migration is complete, use the command documented in the
failing workflow when reproducing an existing status check exactly.
Use scripts/test.sh for local default validation and targeted development
runs.
Use the CI wrappers for the accepted target PR contracts:
make ci-pr-core
make ci-pr-policy
make ci-pr-lint
Use the package gate wrappers when touching package or docs surfaces:
make ci-package-mcp
make ci-package-npm
Coverage Signal Policy
PR confidence is based on behavior checks, not raw coverage percentage.
- Treat Codecov percentages as informational trend data.
- Prefer focused tests for risky paths (storage, sync/git, migrations, state transitions, and corruption/integrity handling) over broad line-coverage churn.
- Add or extend at least one targeted regression test when fixing risky logic.
- Do not block a change solely on overall coverage movement when behavioral checks are strong.
For the current CI/test-surface inventory and cleanup roadmap, see CI_TEST_SURFACE_AUDIT.md. That audit documents where local commands and GitHub Actions currently diverge before the CI cleanup work starts changing workflow behavior.
For accepted CI tier decisions and the implementation order, see CI_CLEANUP_PLAN.md.
Debugging Test Failures
Get detailed output
./scripts/test.sh -v ./path/to/package/...
Run a single test
./scripts/test.sh -run '^TestExactName$' ./cmd/bd/...
Check which tests are being skipped
./scripts/test.sh 2>&1 | head -5
Output shows:
Running: go test -timeout 3m -skip TestFoo|TestBar ./...
Skipping: TestFoo|TestBar
Contributing
When adding new tests:
- Keep tests fast (<0.1s if possible)
- Use
t.Parallel()for independent tests - Clean up resources in
t.Cleanup()ordefer - Avoid sleeps unless testing concurrency
When tests break:
- Fix them if possible
- If unfixable right now, file an issue and add to
.test-skip - Document the issue in
.test-skipwith issue number