From 0a7eef04f8dcf73802f27f2eb2dc80cdcd1ae50b Mon Sep 17 00:00:00 2001 From: beads/refinery Date: Mon, 12 Jan 2026 18:16:51 -0800 Subject: [PATCH] bd daemon sync: 2026-01-12 18:16:51 --- .beads/issues.jsonl | 1 + 1 file changed, 1 insertion(+) diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 2d3274204..5524b5220 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -135,6 +135,7 @@ {"id":"bd-1p8jx","title":"Digest: mol-refinery-patrol","description":"Patrol: Queue empty, no branches processed, inbox clear","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-10T23:04:23.826002-08:00","updated_at":"2026-01-10T23:04:23.826002-08:00","closed_at":"2026-01-10T23:04:23.82595-08:00","close_reason":"Squashed from 11 wisps"} {"id":"bd-1pj6","title":"Proposal: Custom status states via config","description":"Proposal to add 'custom status states' via `bd config`.\nUsers could define an optional issue status enum (e.g., awaiting_review, review_in_progress) in the config.\nThis would enable multi-step pipelines to process issues where each step correlates to a specific status.\n\nExamples:\n- awaiting_verification\n- awaiting_docs\n- awaiting_testing\n","status":"closed","priority":3,"issue_type":"feature","created_at":"2025-11-20T18:55:48.670499-05:00","updated_at":"2025-12-09T18:38:37.668809171-05:00","closed_at":"2025-11-28T23:18:45.862553-08:00"} {"id":"bd-1pr6","title":"Time-dependent worktree detection tests may be flaky","status":"hooked","priority":3,"issue_type":"task","created_at":"2025-12-26T01:21:22.065353-08:00","updated_at":"2025-12-30T15:44:43.360441-08:00"} +{"id":"bd-1qyxt","title":"Session ended: gt-beads-crew-wolf","status":"closed","priority":2,"issue_type":"event","owner":"steve.yegge@gmail.com","created_at":"2026-01-12T18:16:51.038541-08:00","created_by":"beads/crew/wolf","updated_at":"2026-01-12T18:16:51.087133-08:00","closed_at":"2026-01-12T18:16:51.087133-08:00","close_reason":"auto-closed session cost wisp","ephemeral":true} {"id":"bd-1rh","title":"cmd/bd test suite is absurdly slow - 279 tests taking 8+ minutes","description":"# Problem\n\nThe cmd/bd test suite is painfully slow:\n- **279 tests** in cmd/bd alone\n- Full suite takes **8+ minutes** to run\n- Even with the 16 slowest integration tests now tagged with `integration` build tag, the remaining tests still take forever\n\nThis makes the development loop unusable. We can't wait 8+ minutes every time we want to run tests.\n\n# Root Cause Analysis\n\n## 1. Sheer Volume\n279 tests is too many for a single package. Even at 0.1s per test, that's 28 seconds minimum just for cmd/bd.\n\n## 2. Each Test Creates Full Database + Temp Directories\nEvery test does heavy setup:\n- Creates temp directory (`t.TempDir()` or `os.MkdirTemp`)\n- Initializes SQLite database\n- Sets up git repo in many cases\n- Creates full storage layer\n\nExample from the tests:\n```go\nfunc setupCLITestDB(t *testing.T) string {\n tmpDir := createTempDirWithCleanup(t)\n runBDInProcess(t, tmpDir, \"init\", \"--prefix\", \"test\", \"--quiet\")\n return tmpDir\n}\n```\n\nThis happens 279 times!\n\n## 3. Tests Are Not Properly Categorized\nWe have three types of tests mixed together:\n- **Unit tests** - should be fast, test single functions\n- **Integration tests** - test full workflows, need DB/git\n- **End-to-end tests** - test entire CLI commands\n\nThey're all lumped together in cmd/bd, all running every time.\n\n# What We've Already Fixed\n\nAdded `integration` build tags to 16 obviously-slow test files:\n- import_profile_test.go (performance benchmarking tests)\n- export_mtime_test.go (tests with time.Sleep calls)\n- cli_fast_test.go (full CLI integration tests)\n- delete_test.go, import_uncommitted_test.go, sync_local_only_test.go (git integration)\n- And 10 more in internal/ packages\n\nThese are now excluded from the default `go test ./...` run.\n\n# Proposed Solutions\n\n## Option 1: Shared Test Fixtures (Quick Win)\nCreate a shared test database that multiple tests can use:\n```go\nvar testDB *sqlite.SQLiteStorage\nvar testDBOnce sync.Once\n\nfunc getSharedTestDB(t *testing.T) storage.Storage {\n testDBOnce.Do(func() {\n // Create one DB for all tests\n })\n return testDB\n}\n```\n\n**Pros**: Easy to implement, immediate speedup\n**Cons**: Tests become less isolated, harder to debug failures\n\n## Option 2: Table-Driven Tests (Medium Win)\nCollapse similar tests into table-driven tests:\n```go\nfunc TestCreate(t *testing.T) {\n tests := []struct{\n name string\n args []string\n want string\n }{\n {\"basic issue\", []string{\"create\", \"Test\"}, \"created\"},\n {\"with description\", []string{\"create\", \"Test\", \"-d\", \"desc\"}, \"created\"},\n // ... 50 more cases\n }\n \n db := setupOnce(t) // Setup once, not 50 times\n for _, tt := range tests {\n t.Run(tt.name, func(t *testing.T) {\n // test using shared db\n })\n }\n}\n```\n\n**Pros**: Dramatically reduces setup overhead, tests run in parallel\n**Cons**: Requires refactoring, tests share more state\n\n## Option 3: Split cmd/bd Tests Into Packages (Big Win)\nMove tests into focused packages:\n- `cmd/bd/internal/clitests` - CLI integration tests (mark with integration tag)\n- `cmd/bd/internal/unittests` - Fast unit tests\n- Keep only essential tests in cmd/bd\n\n**Pros**: Clean separation, easy to run just fast tests\n**Cons**: Requires significant refactoring\n\n## Option 4: Parallel Execution (Quick Win)\nAdd `t.Parallel()` to independent tests:\n```go\nfunc TestSomething(t *testing.T) {\n t.Parallel() // Run this test concurrently with others\n // ...\n}\n```\n\n**Pros**: Easy to add, can cut time in half on multi-core machines\n**Cons**: Doesn't reduce actual test work, just parallelizes it\n\n## Option 5: In-Memory Databases (Medium Win)\nUse `:memory:` SQLite databases instead of file-based:\n```go\nstore, err := sqlite.New(ctx, \":memory:\")\n```\n\n**Pros**: Faster than disk I/O, easier cleanup\n**Cons**: Some tests need actual file-based DBs (export/import tests)\n\n# Recommended Approach\n\n**Short-term (this week)**:\n1. Add `t.Parallel()` to all independent tests in cmd/bd\n2. Use `:memory:` databases where possible\n3. Create table-driven tests for similar test cases\n\n**Medium-term (next sprint)**:\n4. Split cmd/bd tests into focused packages\n5. Mark more integration tests appropriately\n\n**Long-term (backlog)**:\n6. Consider shared test fixtures with proper isolation\n\n# Current Status\n\nWe've tagged 16 files with `integration` build tag, but the remaining 279 tests in cmd/bd still take 8+ minutes. This issue tracks fixing the cmd/bd test performance specifically.\n\n# Target\n\nGet `go test ./...` (without `-short` or `-tags=integration`) down to **under 30 seconds**.\n\n\n# THE REAL ROOT CAUSE (Updated Analysis)\n\nAfter examining the actual test code, the problem is clear:\n\n## Every Test Creates Its Own Database From Scratch\n\nLook at `create_test.go`:\n```go\nfunc TestCreate_BasicIssue(t *testing.T) {\n tmpDir := t.TempDir() // ← Creates temp dir\n testDB := filepath.Join(tmpDir, \".beads\", \"beads.db\")\n s := newTestStore(t, testDB) // ← Opens NEW SQLite connection\n // ← Runs migrations\n // ← Sets config\n // ... actual test (3 lines)\n}\n\nfunc TestCreate_WithDescription(t *testing.T) {\n tmpDir := t.TempDir() // ← Creates ANOTHER temp dir\n testDB := filepath.Join(tmpDir, \".beads\", \"beads.db\")\n s := newTestStore(t, testDB) // ← Opens ANOTHER SQLite connection\n // ... actual test (3 lines)\n}\n```\n\n**This happens 279 times!**\n\n## These Tests Don't Need Isolation!\n\nMost tests are just checking:\n- \"Can I create an issue with a title?\"\n- \"Can I create an issue with a description?\"\n- \"Can I add labels?\"\n\nThey don't conflict with each other. They could all share ONE database!\n\n## The Fix: Test Suites with Shared Setup\n\nInstead of:\n```go\nfunc TestCreate_BasicIssue(t *testing.T) {\n s := newTestStore(t, t.TempDir()+\"/db\") // ← Expensive!\n // test\n}\n\nfunc TestCreate_WithDesc(t *testing.T) {\n s := newTestStore(t, t.TempDir()+\"/db\") // ← Expensive!\n // test\n}\n```\n\nDo this:\n```go\nfunc TestCreate(t *testing.T) {\n // ONE setup for all subtests\n s := newTestStore(t, t.TempDir()+\"/db\")\n \n t.Run(\"basic_issue\", func(t *testing.T) {\n t.Parallel() // Can run concurrently - tests don't conflict\n // test using shared `s`\n })\n \n t.Run(\"with_description\", func(t *testing.T) {\n t.Parallel()\n // test using shared `s`\n })\n \n // ... 50 more subtests, all using same DB\n}\n```\n\n**Result**: 50 tests → 1 database setup instead of 50!\n\n## Why This Works\n\nSQLite is fine with concurrent reads and isolated transactions. These tests:\n- ✅ Create different issues (no ID conflicts)\n- ✅ Just read back what they created\n- ✅ Don't depend on database state from other tests\n\nThey SHOULD share a database!\n\n## Real Numbers\n\nCurrent:\n- 279 tests × (create dir + init SQLite + migrations) = **8 minutes**\n\nAfter fix:\n- 10 test suites × (create dir + init SQLite + migrations) = **30 seconds**\n- 279 subtests running in parallel using those 10 DBs = **5 seconds**\n\n**Total: ~35 seconds instead of 8 minutes!**\n\n## Implementation Plan\n\n1. **Group related tests** into suites (Create, List, Update, Delete, etc.)\n2. **One setup per suite** instead of per test\n3. **Use t.Run() for subtests** with t.Parallel()\n4. **Keep tests that actually need isolation** separate (export/import tests, git operations)\n\nThis is way better than shuffling tests into folders!","notes":"## Progress Update (2025-11-21)\n\n✅ **Completed**:\n- Audited all 280 tests, created TEST_SUITE_AUDIT.md ([deleted:bd-c49])\n- Refactored create_test.go to shared DB pattern ([deleted:bd-y6d])\n- Proven the pattern works: 11 tests now run in 0.04s with 1 DB instead of 11\n\n❌ **Current Reality**:\n- Overall test suite: Still 8+ minutes (no meaningful change)\n- Only 1 of 76 test files refactored\n- Saved ~10 DB initializations out of 280\n\n## Acceptance Criteria (REALISTIC)\n\nThis task is NOT complete until:\n- [ ] All P1 files refactored (create ✅, dep, stale, comments, list, ready)\n- [ ] Test suite runs in \u003c 2 minutes\n- [ ] Measured and verified actual speedup\n\n## Next Steps\n\n1. Refactor remaining 5 P1 files: dep_test.go, stale_test.go, comments_test.go, list_test.go, ready_test.go\n2. Measure actual time improvement after each file\n3. Continue with P2 files if needed to hit \u003c2min target","status":"closed","priority":1,"issue_type":"bug","created_at":"2025-11-21T11:37:47.886207-05:00","updated_at":"2026-01-08T20:20:48.332191-08:00","closed_at":"2026-01-08T20:20:48.332191-08:00","close_reason":"Test suite now runs in ~17 seconds (was 8+ minutes). Issue is stale - optimizations were made.","dependencies":[{"issue_id":"bd-1rh","depends_on_id":"bd-c49","type":"blocks","created_at":"2025-11-21T11:49:26.347518-05:00","created_by":"daemon"}]} {"id":"bd-1ri4","title":"Test prefix bd-","status":"tombstone","priority":2,"issue_type":"task","created_at":"2025-12-27T14:24:41.240483-08:00","created_by":"stevey","updated_at":"2025-12-27T14:24:50.419881-08:00","deleted_at":"2025-12-27T14:24:50.419881-08:00","deleted_by":"daemon","delete_reason":"delete","original_type":"task"} {"id":"bd-1slh","title":"Investigate charmbracelet-based TUI for beads","description":"Now that we've merged the create-form command (PR #603) which uses charmbracelet/huh, investigate whether beads should have a more comprehensive TUI.\n\nConsiderations:\n- Should this be in core or a separate binary (bd-tui)?\n- What functionality would benefit from a TUI? (list view, issue details, search, bulk operations)\n- Plugin/extension architecture vs build tags vs separate binary\n- Dependency cost vs user experience tradeoff\n- Target audience: humans who want interactive workflows vs CLI/scripting users\n\nRelated: PR #603 added charmbracelet/huh dependency for create-form command.","notes":"Foundation is in place (lipgloss, huh), but not a priority right now","status":"open","priority":3,"issue_type":"feature","created_at":"2025-12-17T14:20:51.503563-08:00","updated_at":"2026-01-02T00:17:14.232446-08:00"}