beads/docs/TESTING.md
Pablo LION 76aec3cc55
refactor(testutil): expand Dolt readiness to 5-state enum (#2304)
* refactor(testutil): consolidate Dolt readiness into enum

Expand from 3-state to 5-state doltReadiness enum:
- doltNoDocker (renamed from doltNoDaemon for clarity)
- doltNoImage (no image at all)
- doltWrongVersion (repo cached but wrong tag)
- doltSkipped (explicit BEADS_TEST_SKIP=dolt opt-out)
- doltReady

Add isDoltRepoImageCached() to distinguish "no image" from "wrong
version". Add hasTestSkip() for comma-separated BEADS_TEST_SKIP
parsing. Silent skip for explicit opt-out avoids ~1s docker info
overhead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: clarify scope of ddfc5e94

ddfc5e94 title says "consolidate" but actually expands the design:

Code changes (internal/testutil/testdoltserver.go):
- Expand doltReadiness enum from 3 to 5 states
- Rename doltNoDaemon → doltNoDocker (clarity)
- Add doltWrongVersion state (image repo cached, wrong tag)
- Add doltSkipped state (explicit opt-out)
- Introduce BEADS_TEST_SKIP env var (comma-separated service list)
  replacing the owner's implicit Docker-only detection with an
  explicit opt-out mechanism that skips the ~1s docker info cost
- Add isDoltRepoImageCached() helper (docker images <repo> -q)
- Add hasTestSkip(service) helper (parses BEADS_TEST_SKIP)
- Extract doltDockerRepo package var to deduplicate strings.Cut
- Silent skip for doltSkipped (no WARN in StartTestDoltServer)

Documentation (docs/TESTING.md):
- Add 5-state readiness table
- Document BEADS_TEST_SKIP usage and evaluation order

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 03:55:54 -08:00

230 lines
5.6 KiB
Markdown

# Testing Guide
## Overview
The beads project has a comprehensive test suite with **~41,000 lines of code** across **205 files** in `cmd/bd` alone.
## Test Performance
- **Total test time:** ~3 minutes (excluding broken tests)
- **Package count:** 20+ packages with tests
- **Compilation overhead:** ~180 seconds (most of the total time)
- **Individual test time:** Only ~3.8 seconds combined for all 313 tests in cmd/bd
## Running Tests
### Quick Start
```bash
# Run all tests (auto-skips known broken tests)
make test
# Or directly:
./scripts/test.sh
# Run full CGO-enabled suite (no skip list)
make test-full-cgo
# Run specific package
./scripts/test.sh ./cmd/bd/...
# Run specific test pattern
./scripts/test.sh -run TestCreate ./cmd/bd/...
# Verbose output
./scripts/test.sh -v
```
### Environment Variables
```bash
# Set custom timeout (default: 3m)
TEST_TIMEOUT=5m ./scripts/test.sh
# Enable verbose output
TEST_VERBOSE=1 ./scripts/test.sh
# Run specific pattern
TEST_RUN=TestCreate ./scripts/test.sh
```
### Docker (Dolt Integration Tests)
Dolt integration tests require Docker with the exact Dolt image cached locally.
Tests auto-detect the environment and skip gracefully — no manual configuration
needed.
#### Readiness states
```csv
State,Condition,Behavior
doltSkipped,BEADS_TEST_SKIP contains "dolt",Silent skip (no warning)
doltNoDocker,Docker daemon not reachable,WARN + skip
doltNoImage,No Dolt image at all,WARN + skip with pull instruction
doltWrongVersion,Image repo cached but wrong tag,WARN + skip with pull instruction
doltReady,Exact image cached and Docker running,Run tests
```
States are checked once per test binary and cached. Order of evaluation:
`BEADS_TEST_SKIP` → Docker availability → exact image → any image version.
#### Skipping Dolt tests explicitly
Set `BEADS_TEST_SKIP` to opt out without Docker overhead (~1s `docker info`):
```bash
# Skip Dolt tests silently
BEADS_TEST_SKIP=dolt ./scripts/test.sh
# Skip multiple services (comma-separated)
BEADS_TEST_SKIP=dolt,slow ./scripts/test.sh
```
#### Enabling Dolt tests
```bash
# Pull the exact Dolt image to enable integration tests
docker pull dolthub/dolt-sql-server:1.43.0
# Point tests at an existing Dolt server (skips container startup)
BEADS_DOLT_PORT=3308 ./scripts/test.sh
```
`BEADS_DOLT_PORT` — when set, tests reuse the server at that port instead of
starting a container. Port 3307 is hardcoded as production and always rejected.
### Advanced Usage
```bash
# Skip additional tests beyond .test-skip
./scripts/test.sh -skip SomeSlowTest
# Run with custom timeout
./scripts/test.sh -timeout 5m
# Combine flags
./scripts/test.sh -v -run TestCreate ./internal/beads/...
```
## Known Broken Tests
Tests in `.test-skip` are automatically skipped. Current broken tests:
1. **TestFallbackToDirectModeEnablesFlush** (GH #355)
- Location: `cmd/bd/direct_mode_test.go:14`
- Issue: Database deadlock, hangs for 5 minutes
- Impact: Makes test suite extremely slow
## For Claude Code / AI Agents
When running tests during development:
### Best Practices
1. **Use the test script:** Always use `./scripts/test.sh` instead of `go test` directly
- Automatically skips known broken tests
- Uses appropriate timeouts
- Consistent with CI/CD
- For full CGO validation, use `./scripts/test-cgo.sh` (or `make test-full-cgo`)
2. **Target specific tests when possible:**
```bash
# Instead of running everything:
./scripts/test.sh
# Run just what you changed:
./scripts/test.sh -run TestSpecificFeature ./cmd/bd/...
```
3. **Compilation is the bottleneck:**
- The 180-second compilation time dominates
- Individual tests are fast
- Use `-run` to avoid recompiling unnecessarily
4. **Check for new failures:**
```bash
# If you see a new failure, check if it's known:
cat .test-skip
```
### Adding Tests to Skip List
If you discover a broken test:
1. File a GitHub issue documenting the problem
2. Add to `.test-skip`:
```bash
# Issue #NNN: Brief description
TestNameToSkip
```
3. Tests in `.test-skip` support regex patterns
## Test Organization
### Slowest Tests (>0.05s)
The top slow tests in cmd/bd:
- `TestDoctorWithBeadsDir` (1.68s) - Only significantly slow test
- `TestFlushManagerDebouncing` (0.21s)
- `TestDebouncer_*` tests (0.06-0.12s each) - Intentional sleeps for concurrency testing
- `TestMultiWorkspaceDeletionSync` (0.12s)
Most tests are <0.01s and very fast.
### Package Structure
```
cmd/bd/ - Main CLI tests (82 test files, most of the suite)
internal/beads/ - Core beads library tests
internal/storage/ - Storage backend tests (SQLite, memory)
internal/rpc/ - RPC protocol tests
internal/*/ - Various internal package tests
```
## Continuous Integration
The test script is designed to work seamlessly with CI/CD:
```yaml
# Example GitHub Actions
- name: Run tests
run: make test
```
## Debugging Test Failures
### Get detailed output
```bash
./scripts/test.sh -v ./path/to/package/...
```
### Run a single test
```bash
./scripts/test.sh -run '^TestExactName$' ./cmd/bd/...
```
### Check which tests are being skipped
```bash
./scripts/test.sh 2>&1 | head -5
```
Output shows:
```
Running: go test -timeout 3m -skip TestFoo|TestBar ./...
Skipping: TestFoo|TestBar
```
## Contributing
When adding new tests:
1. Keep tests fast (<0.1s if possible)
2. Use `t.Parallel()` for independent tests
3. Clean up resources in `t.Cleanup()` or `defer`
4. Avoid sleeps unless testing concurrency
When tests break:
1. Fix them if possible
2. If unfixable right now, file an issue and add to `.test-skip`
3. Document the issue in `.test-skip` with issue number