- mint.sh + make docs-dev: local preview of the docs/ Mintlify site (Node 22 handling modeled on the Gas City wrapper). - test/docsync: Go guard keeping docs/ and docs.json in exact correspondence — every nav entry has a file, every page is in the nav (tiny stub allowlist for paths released bd binaries print), published pages use root-relative extensionless links that resolve, and engdocs/ plus curated root markdown keep exact GitHub-style paths. Wired into make check-docs. - .github/workflows/docs-mintlify.yml: docsync + baseline-aware mint broken-links check on PRs touching docs/**. - Retire scripts/sync-website-docs.sh and its mirror-drift gate: the six mirrored files now live once in the Mintlify site; the committed Docusaurus mirrors stay frozen so the parallel-run site keeps building. - Repoint inbound links to moved docs across README, guides, examples, npm-package, plugin resources, and Go comments; add docs/SYNC_CONCEPTS.md pointer stub (bd's agent templates print that path). Runtime strings in cmd/bd (prime.go, store_factory_nocgo.go, init_git_hooks.go) still print old paths — covered by stubs, flagged in the PR.
6.3 KiB
Performance Testing Guide
This guide covers beads' performance testing framework, including running benchmarks, profiling, and diagnosing performance issues.
Overview
The beads performance testing framework provides:
- Benchmarks: Measure operation speed on 10K-20K issue databases
- CPU Profiling: Automatic profiling during benchmarks with flamegraph support
- User Diagnostics:
bd doctor --perffor end-user performance analysis - Database Caching: One-time generation of test databases, reused across runs
Performance issues typically only manifest at scale (10K+ issues), so benchmarks focus on large databases.
Running Benchmarks
Full Benchmark Suite
make bench
This runs all benchmarks with:
- 1 second per benchmark (
-benchtime=1s) - 10K and 20K issue databases
- Automatic CPU profiling
- 30 minute timeout
Output includes:
ns/op- Nanoseconds per operationallocs/op- Memory allocations per operation- Profile file path
Quick Benchmarks
For faster iteration during development:
make bench-quick
Uses shorter benchmark time (100ms) for quicker feedback.
Running Specific Benchmarks
# Run only GetReadyWork benchmarks
go test -bench=BenchmarkGetReadyWork -benchtime=1s -tags=bench -run=^$ ./internal/storage/dolt/
# Run only Large (10K) benchmarks
go test -bench=Large -benchtime=1s -tags=bench -run=^$ ./internal/storage/dolt/
Understanding Benchmark Output
BenchmarkGetReadyWork_Large-8 1234 812345 ns/op 12345 B/op 123 allocs/op
| Field | Meaning |
|---|---|
-8 |
Number of CPU cores used |
1234 |
Number of iterations run |
812345 ns/op |
~0.8ms per operation |
12345 B/op |
Bytes allocated per operation |
123 allocs/op |
Heap allocations per operation |
Performance Targets:
GetReadyWork: < 50ms on 20K databaseSearchIssues: < 100ms on 20K databaseCreateIssue: < 10ms
Profiling and Analysis
CPU Profiling
Benchmarks automatically generate CPU profiles:
# Run benchmarks (generates profile)
make bench
# View profile in browser (flamegraph)
cd internal/storage/sqlite
go tool pprof -http=:8080 bench-cpu-*.prof
This opens an interactive web UI at http://localhost:8080 with:
- Flamegraph: Visual call stack (wider = more time)
- Graph: Call graph with time percentages
- Top: Functions by CPU time
Reading Flamegraphs
- Width = Time: Wider bars consume more CPU time
- Stacks grow upward: Callees above callers
- Look for wide bars: These are optimization targets
- Click to zoom: Focus on specific call stacks
Common hotspots in database operations:
- SQL query execution
- JSON encoding/decoding
- Memory allocations
Memory Profiling
For memory-focused analysis:
# Generate memory profile
go test -bench=BenchmarkGetReadyWork_Large -benchtime=1s -tags=bench -run=^$ \
-memprofile=mem.prof ./internal/storage/dolt/
# View memory profile
go tool pprof -http=:8080 mem.prof
Look for:
- Large allocation sizes
- Frequent small allocations
- Retained memory that should be freed
User Diagnostics
Using bd doctor --perf
End users can run performance diagnostics:
bd doctor --perf
This:
- Measures time for common operations
- Generates a CPU profile
- Reports any performance issues
- Provides the profile file path for sharing
Sharing Profiles with Bug Reports
When reporting performance issues:
- Run
bd doctor --perf - Note the profile file path from output
- Attach the
.proffile to the bug report - Include the diagnostic output
Understanding the Report
Performance Diagnostics
=======================
Database size: 15,234 issues
GetReadyWork: 45ms [OK]
SearchIssues: 78ms [OK]
CreateIssue: 8ms [OK]
CPU profile saved: beads-perf-2024-01-15-143022.prof
Status indicators:
[OK]- Within acceptable range[SLOW]- Slower than expected, may need investigation[CRITICAL]- Significantly degraded, likely a bug
Comparing Performance
Using benchstat
Install benchstat:
go install golang.org/x/perf/cmd/benchstat@latest
Compare before/after:
# Save baseline
go test -bench=. -count=5 -tags=bench -run=^$ ./internal/storage/dolt/ > old.txt
# Make changes, then run again
go test -bench=. -count=5 -tags=bench -run=^$ ./internal/storage/dolt/ > new.txt
# Compare
benchstat old.txt new.txt
Output shows:
- Performance change percentage
- Statistical significance (p-value)
- Confidence that change is real
Example:
name old time/op new time/op delta
GetReadyWork_Large-8 812µs ± 2% 654µs ± 1% -19.46% (p=0.000 n=5+5)
Detecting Regressions
A change is significant if:
- Delta is > 5%
- p-value is < 0.05
- Consistent across multiple runs (
-count=5or more)
Tips for Optimization
When to Profile vs Benchmark
| Use Case | Tool |
|---|---|
| "Is this fast enough?" | Benchmark |
| "Why is this slow?" | Profile |
| "Did my change help?" | benchstat |
| "User reports slowness" | bd doctor --perf |
Common Optimization Patterns
- Reduce allocations: Reuse buffers, avoid
appendin loops - Batch database operations: One query instead of N
- Use indexes: Ensure queries hit database indexes
- Avoid N+1 queries: Fetch related data in single query
- Cache computed values: Store results that don't change
Optimization Workflow
- Measure first: Get baseline numbers
- Profile: Identify the actual hotspot
- Optimize: Make targeted changes
- Verify: Confirm improvement with benchstat
- Test: Ensure correctness wasn't sacrificed
Database-Specific Tips
- Check
EXPLAIN QUERY PLANfor slow queries - Ensure indexes exist for filtered columns
- Consider
PRAGMA optimizeafter large imports - Watch for table scans on large tables
Cleaning Up
Remove benchmark artifacts:
make clean
This removes:
- CPU profile files (
bench-cpu-*.prof) - User diagnostic profiles (
beads-perf-*.prof) - Cached benchmark databases are in
/tmp/beads-bench-cache/
To clear cached databases:
rm -rf /tmp/beads-bench-cache/
See Also
- TESTING.md - General testing guide
- ARCHITECTURE.md - System architecture
- INTERNALS.md - Implementation details