Commit graph

484 commits

Author SHA1 Message Date
Test User
24981afcf6 fix(backends): address red-team review findings before merge
An adversarial fable red-team of the multi-backend PR surfaced 19 findings; the
verified, non-regressing subset is fixed here (two were confirmed intentional and
left as-is with clarified comments).

Dolt-parity (high):
- sqlkit tx AddLabel/RemoveLabel/CloseIssue recorded no audit event, unlike both
  Dolt references, so bd label add/remove and bd batch / mol squash close lost
  their EventLabel*/EventClosed rows on Postgres/MySQL/SQLite. Delegate to the
  same issueops with-event helpers the embedded oracle uses.
- Heartbeat left updated_at stale on Postgres/SQLite (no ON UPDATE trigger;
  MySQL/Dolt auto-bump), so bd stale diverged. Stamp updated_at on the heartbeat
  UPDATE (claim already stamps it, so no double-assign).

Correctness / hardening:
- postgres/mysql/sqlite stampSchemaVersion: idempotent insert (ON CONFLICT /
  INSERT IGNORE / INSERT OR IGNORE) + re-read, so concurrent first provisions do
  not fail one opener with a duplicate-key error.
- sqlkit tx SetConfig now syncs custom_statuses/custom_types like the oracle.
- pgdialect INSTR->POSITION leaves the call loud when both args carry a
  placeholder, preventing a silent parameter swap.
- creds command runner uses cmd.exe on Windows (sh is POSIX-only).
- Dolt gateway credential command is gated on the Dolt backend, not just
  shared-server mode (was failing closed for SQL workspaces).
- Scrub pg/mysql URL passwords from the bd.args telemetry span.

Conformance rigor:
- E2E normalize() maps zero/epoch timestamps to <ZERO-TS> so a lost timestamp no
  longer reads as byte-identical to a real one.
- E2E logs its candidate set and honors BEADS_CONFORMANCE_REQUIRE so a
  sqlite-only run cannot masquerade as full coverage.

Docs/hygiene:
- Scrub dev-machine paths (/data/tmp/pg-recon, proposal_2.md, "proof-wedge") from
  the Postgres schema DDL comment; clarify the SQLite timestamp comment.

Deliberately not changed (would regress): sqlkit UpdateIssue uses the without-event
variant to MATCH dolt-server (only embedded differs); sub-second timestamp
precision is an intentional per-backend waiver (whole-second round-trip is the
portable contract) — forcing round-down would discard PG/SQLite fidelity.

Verified: embedded + postgres + mysql + sqlite + server-dolt TestConformance and
the pg/mysql/sqlite E2E differential all pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 18:58:00 +00:00
Eddie the Engineer
c79bb32bca fix(backends): maintainer fixup — CI repair + adoption review fixes
Maintainer fixup for the choose-your-own-storage-backend PR, amended across the
pre-review CI-repair and the review-loop apply-fixes iterations so contributor
commits stay intact under one maintainer commit.

CI repair (pre-review):
- default.nix: bump vendorHash for the added Postgres/MySQL driver deps.
- conformance.yml: provision the Postgres service DB with a C-locale cluster so
  text ORDER BY sorts by code point, matching Dolt's utf8mb4_0900_bin oracle.

Review fixes — blockers:
- uow/doltserver_tx: on a non-serialization DOLT_COMMIT failure, ROLLBACK on the
  pinned session before releasing it, and poison (pool-discard via driver.ErrBadConn)
  the connection when the rollback also fails, so a later borrower can't inherit an
  open transaction and implicitly commit orphaned writes. Satisfies the existing
  doltserver_tx_test rollback/poison cases.
- sqlitedialect: translate GREATEST/LEAST to SQLite scalar max/min so the
  ReconcileChildCounters upsert (GREATEST(last_child, ?)) no longer aborts creation
  or import of hierarchical dotted issue IDs (e.g. x-1.2) on the SQLite backend.

Review fixes — majors:
- conformance: add a portable dotted hierarchical child-ID scenario (x-1, x-1.1,
  x-1.2, x-1.2.1) exercising the ReconcileChildCounters GREATEST path on every
  backend — the coverage hole that let the SQLite blocker ship green. Verified
  against the embedded-Dolt oracle and the real SQLite driver.
- postgres: warn at provision/open when the database collation is not code-point
  ordered (C/POSIX/C.*), since text ORDER BY otherwise diverges from Dolt silently.
- sqlite: pin the pool to one connection and add busy_timeout + WAL to the DSN so
  overlapping operations wait instead of surfacing a raw "database is locked"
  (matches the Dolt backend's single-connection posture). Adds a concurrency guard.
- cmd/bd: gate explicit `bd dolt commit` / `bd vc commit` with a typed
  storage.ErrUnsupported on non-Dolt backends instead of the no-op Commit printing
  "Committed."; internal opportunistic auto-commit is untouched.

Review fixes — attempt 2 (stale init backend-flag test contract):
- cmd/bd/init_test, init_embedded_test: the init backend-flag tests still asserted
  the pre-#3151 one-backend contract (--backend=sqlite must fail "SQLite backend
  has been removed"; --backend=postgres is an "unknown backend"), so
  `go test ./cmd/bd -run '^TestInitBackendFlag$'` failed in a clean environment now
  that sqlite/postgres/mysql are supported pluggable backends. Rewrite
  TestInitBackendFlag and the cgo TestEmbeddedInit sibling (rejected_backends →
  sql_backend_flags) for the new contract: --backend=sqlite initializes and
  persists backend=sqlite; --backend=postgres is recognized and fails only on
  missing connection config (not "unknown backend"); a genuinely unknown backend
  is still rejected. Pin BEADS_DIR and scrub BEADS_*/BD_* so the subtests are
  hermetic against an ambient .beads or Dolt server.

Cleanups (non-gating):
- cmd/bd/init: extract checkExistingSQLBackend so the added SQL-backend branches
  don't inflate checkExistingBeadsDataAt's complexity, and load config once.
- configfile: correct the stale "Deprecated: always dolt" Backend field comment.

Review fixes -- attempt 3 (SQLite datetime parity blocker + conformance/rewrite
hardening of the storage-parity gate):
- sqlite/dsn: add _time_format=datetime. Without it modernc binds time.Time via
  t.String() ("2026-07-09 12:34:56.123 +0000 UTC"), a shape SQLite's date functions
  cannot parse, so the counts mega-query's DATE_FORMAT(created_at,...) (strftime on
  SQLite) returns NULL and deps_json exposes a zero created_at while Dolt emits the
  real timestamp. datetime stores whole-second "2026-07-09 12:34:56", matching
  Dolt/MySQL datetime(0). Verified end-to-end against the real modernc driver.
- conformance: add ReadyWorkDepCreatedAtParity -- imports a dependency with a known
  created_at (the bd import / CreateIssuesWithFullOptions bound-time.Time path) and
  asserts the counts-rendered deps_json created_at equals it, closing the blind spot
  that let the SQLite blocker ship green (the counts tests only checked DependencyCount).
  Proven to fail on the pre-fix DSN and pass on both the fixed SQLite backend and the
  embedded-Dolt oracle.
- conformance: add ClaimReadyIssueConcurrentExclusivity -- races 24 workers over 8
  ready issues and asserts each is claimed exactly once, exercising the SQL backends'
  multi-writer claim property that was previously only asserted, never raced. Gated to
  the multi-connection dialects (postgres/mysql) via DialectName; SQLite (single-conn
  pool) and embedded-Dolt skip.
- conformance.sh: run the embedded-Dolt reference conformance with
  BEADS_TEST_EMBEDDED_DOLT=1 and fail loudly if it skips. It self-skips without the env
  var, so the parity gate was reporting success while never running the reference oracle.
- mysqldialect: drive the is_blocked-recompute error-1093 rewrite test from the real
  issueops templates (new issueops.SelfReferentialRecomputeTemplates) instead of a
  hand-copied fixture, so template drift fails a non-gated test rather than only the
  env-gated MySQL conformance suite. All four mark/unmark x issue/wisp templates
  verified rewritten into the materialized derived-table form.

Note: the review re-flagged TestInitBackendFlag as failing, but it passes in a clean run
(plain and gms_pure_go, matching the scorecard's own rerun); the only cmd/bd failure in
this sandbox is an unrelated pre-existing embedded-Dolt/CGO test not in this PR's diff.

Review fixes -- attempt 4 (conformance-gate self-brick + all-wisps inline-dependency
drop blockers; Postgres error-log password-leak major):
- conformance.sh: anchor the embedded-Dolt reference skip/pass guards to top-level
  result lines (^--- SKIP/PASS: TestConformance). The attempt-3 loud-fail guard used
  an unanchored grep, so the indented, legitimate subtest skip of
  ClaimReadyIssueConcurrentExclusivity (embedded-Dolt is single-writer) matched it and
  false-failed the entire storage-parity gate on every run even when the top-level
  suite passed. Whole-suite-skip protection is preserved.
- sqlkit/portable + embeddeddolt/create_issue: the all-wisps arm looped the
  single-issue issueops.CreateIssueInTx, which runs no batch dependency-persist pass,
  so an all-wisps batch carrying inline dependencies (e.g. importing no-history beads
  with inter-bead edges -- export DOES include no-history beads, GH#2619) silently
  dropped every edge (not even reported via OnSkippedDependency) and, one transaction
  per issue, forfeited batch atomicity. Both now mark ephemeral, then run the whole
  batch in ONE transaction through issueops.CreateIssuesInTx, matching the server-mode
  DoltStore all-wisps arm. The embedded-Dolt reference oracle had the same latent drop,
  so the oracle and the new SQL backends were corrected together to keep conformance
  parity (the finding compared sqlkit against the server-mode DoltStore, which already
  persisted these edges).
- conformance: add CreateAllWispsInlineDependencies -- an all-wisp batch with one valid
  inter-wisp edge and one unresolvable edge; asserts the valid edge persists and the
  unresolvable one is reported via OnSkippedDependency. Fails on the pre-fix per-issue
  loop, passes on the batch path; verified on SQLite and the embedded-Dolt oracle, with
  a full embedded-Dolt conformance re-run clean.
- pgdialect: scrub query-param/sslpassword secrets from pgx ParseConfig errors in Open
  and OpenRaw (new ScrubDSNError). pgx's ParseConfigError redacts only URL userinfo, so
  a postgres://u@h/db?password=SECRET DSN (the shape bd init connects with) leaked the
  cleartext secret through the wrapped parse/TLS-config error into logs. Connect-time
  errors were verified not to carry the password. Adds redact_error_test coverage for
  the query-param, sslpassword, userinfo, and libpq-keyword shapes plus the real
  Open/OpenRaw parse-failure path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 17:44:37 +00:00
Test User
1fc38ba778 feat: choose your own storage backend (Postgres / MySQL / SQLite) — rebased on main
Rebase of PR #4601 (feat/choose-your-own-backend) onto current origin/main,
squashed to a single commit because the original 73-commit history built and then
removed the uowStore derisk spike incrementally, which does not replay cleanly
over the 125 commits main has since advanced (notably #4679, the ready-work
counts fix, which merged to main and is now inherited rather than re-applied).
The full pre-rebase history remains on the original branch tip for reference.

Contents (multi-backend feature + the six review follow-ups):
- bd init --backend=postgres|mysql|sqlite provisioning behind the Dolt-parity
  seam; the three SQL backends share *sqlkit.Store over pluggable dialects and
  carry a generated typed-unsupported shell for the non-core surface.
- Passwords in init DSNs are redacted before persisting to metadata.json (driver
  parser + re-parse verification), with a negative-test matrix proving no secret
  survives (pgdialect + mysql).
- The uowStore derisk spike and its report artifacts are removed; the shell
  generator now lives in internal/storage/unsupportedgen (neutral name).
- Compile-time guard `var _ storage.DoltStorage = (*Store)(nil)` on each backend
  plus a regen-idempotence test keep the generated skip lists honest.
- sqlkit exposes CountReadyWork so every SQL backend satisfies
  storage.ReadyWorkCounter and inherits the shared page-pushed ready-counts path.
- Commercial/product names scrubbed from the feature's own code for vendor
  neutrality.

Rebase reconciliations: adopted main's metadata-load hardening in the store
factories alongside the backend dispatch; combined the config interface
(GetMetadata + ReconcileVersion) and the conformance registrations (ReadyCounts +
Claim/lease); took main's versions of the shared-infra concurrency (uow
doltserver_tx) and delete-cascade (domain/issue_delete) evolutions; implemented
the interface's new UnclaimIssue on sqlkit (delegating to issueops) and added it
to the skip lists.

Verified: go build (cgo + CGO_ENABLED=0) and go vet clean; unit tests
(sqlbuild/issueops/pgdialect/mysql/unsupportedgen) and the SQLite conformance
suite green (RunAll, incl. ReadyCounts equivalence and Claim/lease on the real
sqlkit backend).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 17:44:37 +00:00
beads/crew/emma
a3bf3ee381 refactor: remove SQLite backend (Phase 6)
Delete internal/storage/sqlite/ entirely (131 files, ~40K LOC). Update
factory to route all backends through Dolt registry. Rewrite SQLite
migration reader with raw database/sql. Remove to-SQLite escape hatch.
Update all production and test code to use Dolt or memory stores.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-15 10:57:05 -08:00
beads/crew/emma
8e10fd09d3 refactor: remove JSONL sync layer (Phase 5)
Remove autoflush, autoimport, dirty tracking, export hashes, and JSONL
sync machinery. Dolt handles persistence natively, eliminating the need
for SQLite-to-JSONL synchronization. -11,458 lines removed across 69 files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-15 10:37:22 -08:00
beads/crew/emma
7eed8b8a1e refactor: remove tombstone/soft-delete system (Phase 4)
Remove the tombstone soft-delete mechanism across the codebase. Dolt handles
delete propagation natively, making tombstones obsolete.

Key changes:
- Remove StatusTombstone constant and all status checks
- Remove DeletedAt/DeletedBy/DeleteReason/OriginalType fields from Issue struct
- Remove resurrection mechanism (JSONL parent recovery), replace with existence checks
- Simplify schema CHECK constraint (2 clauses instead of 3)
- Remove compact_tombstone.go, resurrection.go, migrate_tombstones.go
- Remove tombstone-specific tests (resurrection, expired tombstones, migration)
- Update doctor/fix validation to skip tombstone-specific checks
- Preserve migration files for backward compatibility with existing databases
- 104 files changed, 6117 deletions, 422 insertions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-14 21:22:24 -08:00
Steve Yegge
3c246d3910
Merge pull request #1740 from sjsyrek/fix/ready-deferred-parent-propagation
fix(ready): exclude children of deferred parents from ready work
2026-02-14 10:59:31 -08:00
Steven Syrek
015fee66d6 fix(sqlite): add missing columns to schema probe expectedSchema
The schema_probe.go expectedSchema map was missing 16 columns that exist
in the issues table (from schema.go CREATE TABLE and migrations):
spec_id, created_by, owner, closed_by_session, wisp_type, is_template,
crystallizes, mol_type, work_type, quality_score, source_system, metadata,
event_kind, actor, target, payload.

This caused the schema probe to not detect databases missing these columns
after a failed or incomplete migration, which meant the migration retry
logic (in store.go) would not trigger. Users upgrading from older versions
would then hit "no such column: spec_id" errors when queries referencing
these columns ran.

By including all columns in expectedSchema, the probe correctly identifies
incompatible schemas and triggers the migration retry path, fixing the
initialization failures reported in #1535 and #1516.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-14 18:44:13 +01:00
Steven Syrek
b5ca8372c3 fix(ready): exclude children of deferred parents from ready work
When a parent issue has defer_until set to a future date, its children
should also be excluded from bd ready output. Previously only the parent
itself was filtered, leaving orphaned children appearing as actionable.

Adds NOT EXISTS clauses to both the GetReadyWork query and the
ready_issues views (SQLite and Dolt) to check for deferred parents via
parent-child dependencies.

Fixes #1190

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-14 18:16:12 +01:00
David Ahmann
42a26b13a7 ci: fix baseline fmt/lint/windows+nix regressions
Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-12 09:17:14 -08:00
beads/crew/emma
e8edba13e4 revert: restore SQLite backend and JSONL sync layer (premature dolt-test-woho transition)
The Dolt transition epic (dolt-test-woho) was executed on Feb 10, 11 days
before Phase 2 planned date of ~Feb 21, and before the upstream blocker
(dolt-test-djfr: Dolt-in-Git lands upstream) was resolved.

This reverts the premature removal of SQLite backend (42K+ lines) and JSONL
sync layer (9K+ lines) while preserving all non-transition changes merged
since the transition (PRs #1655, #1660, #1670, #1675, #1680, #1684, #1689).

Restored:
- internal/storage/sqlite/ (131 files, full SQLite storage backend)
- internal/importer/ (7 files, JSONL import engine)
- internal/storage/dolt/dirty.go (dirty tracking for Dolt)
- cmd/bd/autoflush.go, autoimport.go, export.go (JSONL sync layer)
- Storage interface dirty tracking methods
- Factory defaults to SQLite (Dolt available via registry)
- Test helpers create SQLite stores instead of Dolt

Removed transition-only files:
- cmd/bd/config_local.go, import_engine.go, jsonl_bootstrap.go, jsonl_path.go

Applied PR #1680 retry logic to restored internal/importer/importer.go.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-11 15:19:16 -08:00
collins
237f47ba7e feat: delete SQLite storage backend (Phase 2 of Dolt transition)
Remove the entire internal/storage/sqlite/ package (~43K lines) and update
all 80+ files that imported it to use either the Dolt backend directly
(test files) or the storage factory (production files).

Key changes:
- Delete internal/storage/sqlite/ directory (150+ files)
- Update factory to make Dolt the sole backend, treating "sqlite" as alias for "dolt"
- Replace sqlite.New() calls with dolt.New() in tests, factory.New() in production
- Replace *sqlite.SQLiteStorage with storage.Storage interface throughout
- Replace :memory: SQLite URIs with t.TempDir() for Dolt compatibility
- Inline sqlite.GenerateIssueID() as SHA-256 content hashing where needed
- Fix import cycle in routing/routes.go (routing→factory→dolt→routing)
- Preserve backward-compatible beads.NewSQLiteStorage() wrapper in public API

228 files changed, 605 insertions, 43,821 deletions.
go vet ./... passes clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 12:23:15 -08:00
lydia
8403653796 fix: remove remaining dirty tracking refs after rebase (woho.4)
Post-rebase cleanup: upstream moved DeleteIssue to delete.go and added
sync_helpers.go/sync_status.go with dirty tracking calls. Remove those
references to complete the woho.4 dirty tracking strip.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 12:21:14 -08:00
lydia
4be0f382b7 refactor: strip dirty tracking from Storage interface (woho.4)
Phase 2 of Dolt transition: remove JSONL dirty tracking methods from
the Storage interface and all implementations.

Removed from Storage interface:
- GetDirtyIssues, GetDirtyIssueHash, ClearDirtyIssuesByID
- GetExportHash, SetExportHash, ClearAllExportHashes
- GetJSONLFileHash, SetJSONLFileHash

Removed from CompactableStorage interface:
- MarkIssueDirty

Deleted implementation files:
- internal/storage/dolt/dirty.go
- internal/storage/sqlite/dirty.go
- internal/storage/sqlite/dirty_test.go
- internal/storage/sqlite/hash.go

Cleaned up all callers in cmd/bd/, internal/compact/, internal/importer/,
internal/rpc/, and internal/storage/ packages. Removed or stubbed dead
code paths (incremental export thresholds, JSONL integrity validation).

Note: Internal SQLite markDirty() helpers that INSERT INTO dirty_issues
remain as harmless side effects - follow-up cleanup tracked separately.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 12:21:14 -08:00
jasper
83d3e4fe20 refactor: split queries.go by domain into parsing, search, and delete files (bd-rgyd)
Split the 2263-line queries.go into domain-focused files:
- parsing.go: shared timestamp/JSON parsing utilities (66 lines)
- search.go: SearchIssues and filterByLabelRegex (351 lines)
- delete.go: DeleteIssue, DeleteIssues, and all delete helpers (396 lines)
- queries.go: remaining CRUD and lifecycle operations (1481 lines)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Executed-By: beads/polecats/jasper
Rig: beads
Role: polecats
2026-02-09 11:55:01 -08:00
beads/crew/darcy
9bcc618cd5 fix(migrate-issues): replace raw SQL with Storage interface for Dolt compatibility
migrate-issues used UnderlyingDB() type-assertion to SQLite and raw SQL
queries, which would fail on Dolt backend. Now uses the Storage interface:

- Add SourceRepo filter to IssueFilter (types, SQLite, Dolt SearchIssues)
- Add source_repo to allowed update fields (SQLite + Dolt)
- Replace raw SQL queries with SearchIssues, GetDependencyRecords,
  GetDependents, GetAllDependencyRecords, GetDependencyRecordsForIssues
- Replace raw transaction with RunInTransaction + UpdateIssue
- Remove database/sql and sqlite package imports

Closes: dolt-test-ztzb

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 16:04:12 -08:00
zenchantlive
8de2744551 fix: make --parent filtering include dotted-ID children (GH#1614)
bd list --parent and dotted-ID naming convention now agree on what
constitutes a parent-child relationship. Updated ParentID filtering
in all storage backends to also include issues whose IDs match the
dotted naming convention (strings.HasPrefix(issue.ID, parentID+".")).

From PR #1617 by zenchantlive.

Co-Authored-By: zenchantlive <zenchantlive@users.noreply.github.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-08 15:23:12 -08:00
Arvid Petermann
027a41c9f5 fix: address review comments on parent-child validation
- Fix misleading comments in dependencies.go and transaction.go that
  referenced isChildOf check as a storage-layer safeguard — isChildOf
  only exists in CLI/RPC layers, not in the storage layer
- Add transaction-scoped test (TestTransactionAddDependency_ParentChildCustomTypes)
  covering custom type hierarchies (theme → epic → shot) to ensure both
  direct and transaction code paths stay in sync

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 04:10:28 +00:00
Arvid Petermann
1ec5b51917 fix: remove type-based parent-child validation that rejected custom types
The parent-child validation in AddDependency (both SQLiteStorage and
sqliteTxStorage) enforced that Epic types could not depend on non-Epic
types. This incorrectly rejected valid hierarchies involving custom
types like "theme" → "epic" and "epic" → "shot".

The type-based check was redundant — cycle detection and the isChildOf
check in dep.go already prevent the actual problematic cases (circular
dependencies and duplicate hierarchical links).

Fixes parent-child creation failures with custom issue types configured
via types.custom (e.g., theme, shot).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 04:01:31 +00:00
beads/crew/lizzy
2bb827facb style: run gofmt across entire codebase (bd-nmk12)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 19:35:20 -08:00
beads/crew/lizzy
e755986720 test: add SQLite error scenario coverage for transactions and WAL (dolt-test-ptp)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 19:06:52 -08:00
beads/crew/lizzy
ae108e89c2 hygiene: renumber spec_id migration from 041 to 043
Two migration files shared the 041 prefix (metadata_column and
spec_id_column). Since spec_id_column is registered after
wisp_type_column (042) in the migrations list, renumber it to 043
to match its actual execution order.

Closes dolt-test-v2c

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 18:10:25 -08:00
beads/crew/lizzy
1bcfe0a79d chore: unexport ~29 dead symbols in internal/storage/sqlite
Unexport package-internal symbols with no external callers:
- errors: IsConflict, IsCycle, IsNotFound
- util: IsUniqueConstraintError (removed duplicate), IsForeignKeyConstraintError, IsBusyError
- external_deps: CheckExternalDep, CheckExternalDeps, GetUnsatisfiedExternalDeps, ExternalDepStatus
- collision: CollisionResult, CollisionDetail, RenameDetail
- ids: GetAdaptiveIDLength, IsHierarchicalID, ValidateIssueIDPrefix, GenerateBatchIssueIDs, EnsureIDs
- freshness: FreshnessChecker, NewFreshnessChecker
- migration_invariants: MigrationInvariant, CleanOrphanedRefs
- migrations: MigrationInfo
- schema_probe: SchemaProbeResult, ErrSchemaIncompatible
- ready: ExcludeIDPatternsConfigKey, DefaultExcludeIDPatterns
- config: CustomStatusConfigKey, CustomTypeConfigKey

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 09:29:56 -08:00
beads/crew/lizzy
de567cda79 chore: unexport 14 dead symbols across 8 packages
Unexport package-internal symbols that have no external callers:
- validation: ValidatePrefix, ValidatePrefixWithAllowed, ForUpdate, ForClose, ForDelete, ForReopen
- debug: PrintNormal, PrintlnNormal
- linear: IssuesQuery
- timeparsing: IsCompactDuration, ParseNaturalLanguage
- utils: FindMoleculesJSONLInDir, ParseIssueID
- testutil/fixtures: XLargeFromJSONL
- storage/sqlite: Snapshot type

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 09:24:22 -08:00
beads/crew/emma
f5e0de4daa fix(ci): resolve lint failures and test guard false positive (#1550)
- errcheck: handle fmt.Scanln return value (doctor_validate.go conflict
  resolved - main already has bufio.NewReader fix)
- gosec G204: nolint directive for validated exec.Command in config.go
- unparam: make maxRetries const in beginImmediateWithRetry
- test guard: exclude interactions.jsonl from watched files

Co-authored-by: turian <turian@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-06 22:06:41 -08:00
beads/crew/lydia
a6cbb7cdb2 feat: centralize AI model config into ai.model with DefaultAIModel()
Single source of truth for the default AI model, replacing hardcoded
strings in compact/haiku.go and find_duplicates.go. Configurable via
`bd config set ai.model` or BD_AI_MODEL env var.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 17:45:04 -08:00
Joseph Turian
f763b15cef fix(ci): resolve lint failures and test guard false positive on main
- errcheck: handle fmt.Scanln return value in validate.go
- gosec G204: add nolint directive for validated exec.Command in config.go
- unparam: make maxRetries a const inside beginImmediateWithRetry (always 5)
- test guard: exclude interactions.jsonl (legitimately created by init)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 19:01:35 -05:00
beads/crew/darcy
eb1049baf3 fix(sqlite): make Close() idempotent to prevent WAL retry deadlock (bd-4ri)
SQLiteStorage.Close() was not idempotent -- double-close triggered the WAL
checkpoint retry loop (~3.1s backoff per call). With 81 double-closes from
t.Cleanup overlapping manual Close() in tests, this added ~250s to the test
suite. Use CompareAndSwap on the closed flag so only the first call proceeds.

Also fix cleanup ordering in TestFallbackToDirectModeEnablesFlush to shut
down FlushManager before closing the store, preventing the deadlock where
the FlushManager background goroutine holds a DB connection while Close()
blocks on the sql.DB mutex.

Results: test suite 231s -> 103s, WAL checkpoint warnings 81 -> 0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:47:43 -08:00
beads/crew/elinor
bf3af462f6 fix: centralize error handling patterns in storage layer (bd-bwk2)
Completes the error handling consolidation started when errors.go was created:
- Wrap 10 bare error returns with wrapDBError/wrapDBErrorf for operation context
- Add missing rows.Err() checks in labels.go and epics.go
- Standardize sql.ErrNoRows checks to use errors.Is() across 14 files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:37:27 -08:00
beads/crew/lizzy
6be71339bb fix: use SQLITE_BUSY retry logic for all BEGIN IMMEDIATE calls (bd-ola6)
Three call sites were using raw BEGIN IMMEDIATE without retry, relying
solely on busy_timeout. Replace with beginImmediateWithRetry (exponential
backoff: 10ms..160ms, 5 retries) which handles SQLITE_BUSY_SNAPSHOT and
other cases where busy_timeout alone is insufficient.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:32:21 -08:00
Matt Armstrong
89e4ec510d Fix doctor --fix migration for legacy spec_id 2026-02-06 09:05:49 -08:00
beads/crew/emma
789ffe7221 fix: resolve unused ctx lint warning in filterByLabelRegex
Co-authored-by: paf0186 <paf0186@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-05 09:55:16 -08:00
beads/crew/emma
8d10ad7fdd test: add comprehensive tests for label-pattern and label-regex filtering
Adds 7 test cases covering:
- LabelPattern glob filtering in SearchIssues (5 sub-tests)
- LabelRegex filtering in SearchIssues (5 sub-tests)
- Invalid regex pattern error handling
- LabelPattern in GetReadyWork
- LabelRegex in GetReadyWork
- Invalid regex in GetReadyWork
- Combined pattern + regex filtering

Co-authored-by: paf0186 <paf0186@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-05 09:55:14 -08:00
Patrick Farrell
46b6b7a45e feat: add --label-pattern and --label-regex flags to bd list
Add support for filtering issues by label patterns:
- --label-pattern "tech-*" uses SQLite's GLOB for glob-style matching
- --label-regex "tech-(debt|legacy)" uses Go's regexp for full regex

Changes:
- types/types.go: Add LabelPattern and LabelRegex to IssueFilter and WorkFilter
- cmd/bd/list.go: Add CLI flags and forward to filter/RPC
- internal/rpc/protocol.go: Add fields to ListArgs
- internal/rpc/server_issues_epics.go: Forward to IssueFilter
- internal/storage/sqlite/queries.go: GLOB filtering + app-level regex
- internal/storage/sqlite/transaction.go: Same for transaction mode
- internal/storage/sqlite/ready.go: Same for GetReadyWork

Glob uses SQLite's native GLOB function for performance. Regex is done
at the application level after query results return, since SQLite lacks
built-in regex support without extensions.

Closes bd-710

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-05 09:55:14 -08:00
Steve Yegge
7b47287076
Merge pull request #1280 from rsnodgrass/pr/daemon-lock-transaction
fix: add transaction safety and daemon lock improvements
2026-02-05 09:38:03 -08:00
Ryan Snodgrass
9e397bca93 fix: add WAL checkpoint robustness with retry logic
Add WAL checkpoint retry logic with exponential backoff to prevent data
loss when concurrent readers hold locks during database close.

Security fixes from SECURITY_AUDIT.md:
- Issue #8: WAL checkpoint failure silently losing data

Changes:
- Add 5-retry exponential backoff for WAL checkpoint in Close()
- Warn user if checkpoint fails after retries (potential data loss)
- Add flush_manager_race_test.go for concurrency testing
- Add wal_test.go for checkpoint with concurrent readers

Co-Authored-By: SageOx <ox@sageox.ai>
2026-02-05 09:20:07 -08:00
Joseph Turian
dc038c9272 feat: allow UpdateIssue metadata updates via json.RawMessage/[]byte (#1417)
Add NormalizeMetadataValue helper that accepts string, []byte, or
json.RawMessage for metadata updates. This makes the programmatic API
more convenient for callers that already have their JSON as []byte or
json.RawMessage.

- Add internal/storage/metadata.go with NormalizeMetadataValue function
- Update SQLite and Dolt UpdateIssue to normalize metadata values
- Add comprehensive tests for all input types and error cases

Closes #1417

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 17:55:23 -08:00
paf0186
c96e62e6b5
fix: add atomic compare-and-swap semantics to bd claim (#1474)
* fix: add atomic compare-and-swap semantics to bd claim

The claim operation had a race condition where concurrent callers could
both pass the pre-check and claim the same issue. This fix:

- Adds ClaimIssue method to Storage interface with atomic CAS semantics
- Uses conditional UPDATE (WHERE assignee = '') to prevent race conditions
- Returns ErrAlreadyClaimed error when issue is already assigned
- Implements the fix in SQLite, Memory, and Dolt storage backends
- Updates RPC handler and CLI to use the new atomic method
- Adds comprehensive tests including concurrent claim scenarios

Closes the race window where multiple agents could both claim an issue.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add ClaimIssue to mockStorage in tests

The mockStorage struct used for interface compliance testing was missing
the new ClaimIssue method, causing build failures in CI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(nix): enable proxyVendor to fix GOTOOLCHAIN in module fetch phase

The Nix build was failing because the go-modules fetching phase uses
GOTOOLCHAIN=local by default, ignoring the env.GOTOOLCHAIN=auto setting.

Adding proxyVendor=true changes how modules are fetched in a way that
should allow GOTOOLCHAIN=auto to take effect, enabling Go to download
the required 1.25.6+ toolchain automatically.

Fixes #1476

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix(nix): enable proxyVendor to fix GOTOOLCHAIN in module fetch phase"

This reverts commit a5b758cf1ac8bbadff774f5566144c3f0b94d561.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 15:07:40 -08:00
jane
8457d9ea7d fix: add missing spec_id to SQL queries using scanIssues
The spec_id column was added to the issues schema but several SQL
SELECT statements were not updated to include it. This caused a
scan mismatch error: expected 43 destination arguments in Scan,
not 44.

Fixed queries:
- GetReadyWork (ready.go)
- GetNewlyUnblockedByClose (ready.go)
- GetIssuesByLabel (labels.go)
- TestMigrateContentHashColumn test schema (migrations_test.go)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 15:52:50 -07:00
anupamchugh
a1c447eb66 feat: add spec_id support
Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-02-03 13:58:36 -07:00
dennis
ebead133d8 feat(bd): add wisp_type field support for TTL-based compaction (gt-7mqd.2)
Add wisp_type as a first-class field to classify ephemeral wisps for
TTL-based compaction per the Wisp Compaction Policy (gt-9br):

- Add --wisp-type flag to bd create with validation
- Add --wisp-type filter to bd list for querying by wisp type
- Display wisp_type in bd show output for ephemeral beads
- Update IssueFilter and WorkFilter to support wisp_type filtering
- Add wisp_type to RPC protocol for daemon mode support
- Update all storage layers (sqlite, dolt) with wisp_type queries

Supported wisp types: heartbeat, ping, patrol, gc_report, recovery,
error, escalation - each with different TTLs for compaction.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Executed-By: gastown/crew/dennis
Rig: gastown
Role: crew
2026-02-02 22:50:22 -07:00
dennis
e5c8c1d772 feat(schema): add wisp_type column for TTL-based compaction (gt-7mqd.1)
Adds wisp_type field to Issue struct and database schemas for classifying
ephemeral wisps by category. This enables TTL-based compaction per gt-9br
design (WISP-COMPACTION-POLICY.md).

Categories:
- heartbeat, ping: 6h TTL (high-churn, low forensic value)
- patrol, gc_report: 24h TTL (operational state)
- recovery, error, escalation: 7d TTL (significant events)

Changes:
- Add WispType type with constants in types/types.go
- Add wisp_type VARCHAR(32) column to Dolt schema
- Add SQLite migration 042_wisp_type_column.go
- Update insert/scan functions in both storage backends
- Add wisp_type to allowed update fields

Part of: gt-7mqd (Wisp Compaction Policy implementation)
Designed by: gt-9br, approved by mayor

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Executed-By: gastown/crew/dennis
Rig: gastown
Role: crew
2026-02-02 22:40:40 -07:00
paf0186
4b80094dd5
fix: bd ready now excludes in_progress issues to show only claimable work
* fix: bd ready now excludes in_progress issues

Previously `bd ready` showed both open and in_progress issues, which was
inconsistent with `bd list --ready` (which only shows open issues).

This caused confusion for agents trying to find claimable work - issues
already being worked on would appear in the "ready" list.

Changes:
- `bd ready` now passes Status="open" to GetReadyWork, matching
  `bd list --ready` behavior
- Updated command description to reflect new behavior
- Added test for Status="open" filter behavior
- Clarified GetReadyWork storage layer docstring

The storage layer still supports both behaviors via the filter.Status
field for backwards compatibility with internal callers.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: also pass Status filter through RPC path

The initial fix only addressed the direct-mode path. This commit adds
the Status field to ReadyArgs and passes it through the daemon RPC path
so that `bd ready` works correctly in daemon mode as well.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 21:23:04 -08:00
beads/crew/emma
774cc5bbd0 feat: add comment_count to issue JSON views (list, ready, search)
Add GetCommentCounts() batch query to all storage backends (SQLite, Dolt,
Memory) and wire it into bd list, bd ready, bd search JSON output and RPC
server responses. Uses single GROUP BY query to avoid N+1.

Based on PR #1354 by bjornslib - cherry-picked the Go backend changes,
fixed bugs, and skipped problematic JSONL/lockfile/MCP changes.

Co-Authored-By: bjornslib <bjornslib@users.noreply.github.com>

Executed-By: beads/crew/emma
Rig: beads
Role: crew
2026-01-30 22:51:03 -08:00
Joseph Turian
92fd124f5c
feat: add optional per-issue JSON metadata field (SQLite + Dolt) (#1407)
* feat: add optional JSON metadata field to issues

Add an optional `metadata` field to Issue for storing arbitrary JSON data.
This enables extension points for tool annotations, file lists, and
external system bridging without expanding the core schema.

Changes:
- Add `Metadata json.RawMessage` field to types.Issue
- Add metadata column to SQLite and Dolt schemas
- Add migration 041_metadata_column for existing databases
- Wire through insert/update/get/search in both backends
- Validate metadata is well-formed JSON on create/update
- Include metadata in content hash computation

The field uses json.RawMessage to preserve exact JSON and supports
omitempty for JSONL export compactness.

Closes #1406

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add metadata to SQLite SearchIssues and scanIssues

Include metadata column in:
- SearchIssues SELECT statement (queries.go)
- scanIssues function in dependencies.go
- transaction.go GetIssue and SearchIssues SELECTs
- scanIssueRow helper function

Without this, bd list/ready and other search operations would
return issues with empty Metadata fields even when stored.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add metadata column to remaining SELECT statements

- Add metadata to GetIssuesByLabel in labels.go
- Add metadata to GetReadyWork and GetNewlyUnblockedByClose in ready.go
- Fix TestMigrateContentHashColumn test to include metadata column

This fixes test failures from the initial metadata implementation
where some SELECT statements were missing the new metadata column.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: enable metadata updates and add round-trip tests

- Add "metadata" to SQLite allowedUpdateFields (matches Dolt)
- Add TestIssueMetadataRoundTrip test verifying:
  - Create issue with metadata -> GetIssue returns it
  - SearchIssues returns metadata
  - UpdateIssue can modify metadata
  - Issues without metadata have nil Metadata field

This ensures metadata can be updated via bd update and proves
the feature works across key code paths (GH#1406).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(dolt): handle empty metadata for JSON column type

Dolt's JSON column requires valid JSON, so empty strings cause
"Invalid JSON text" errors. Added jsonMetadata() helper that
returns "{}" when metadata is nil or empty.

This fixes TestBootstrapFromJSONL and related Dolt tests.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: use structural JSON comparison in metadata tests

Replace string comparison with jsonEqual() helper that unmarshals
and uses reflect.DeepEqual. This prevents test brittleness if JSON
gets normalized (whitespace, key order) in the future.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 22:41:28 -08:00
Pablo LION
e880159238
fix(state): allow event type in CreateIssue validation (#1398)
* test(state): add tests for event type validation (GH#1356)

set-state creates event beads internally but event is not a built-in
type, causing validation failures. Add tests that reproduce the bug
in both types and sqlite packages.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(state): allow event type in IsValidWithCustom validation

set-state internally creates event beads for audit trail, but event
was removed from built-in types in 4f0f5744 without updating the
validation path. Add TypeEvent constant and accept it in
IsValidWithCustom and IsBuiltIn, covering all CreateIssue call sites
and multi-repo hydration trust.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(types): simplify IsValidWithCustom and improve doc comments

Replace redundant IsValid + TypeEvent checks with single IsBuiltIn
call in IsValidWithCustom. Clarify IsBuiltIn doc to distinguish core
work types from system-internal types. Remove domain jargon from test
comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(types): add missing test coverage for event type validation

Add negative assertions for IsBuiltIn with custom types (molecule,
gate must return false). Add Normalize guard to prevent event being
aliased to a core type. Add dolt backend test for CreateIssue with
event type without types.custom config.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs(types): clarify TypeEvent and IsBuiltIn doc comments

Use precise language in TypeEvent doc (list specific validation
methods instead of "always accepted by validation"). Note event's
Gas Town origin to prevent confusion. Use "i.e." instead of "like"
in IsBuiltIn doc to signal TypeEvent is currently the only internal
type.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 22:18:44 -08:00
Steve Yegge
b2e1891521
Merge pull request #1305 from myleshenderson/events-jsonl-export
Reviewed by beads/crew/jane. Fixed missing rows.Err() check in SQLite GetAllEventsSince.
2026-01-29 19:53:46 -08:00
beads/crew/jane
b1f0774c31 fix: check rows.Err() in SQLite GetAllEventsSince
The Dolt implementation correctly returns rows.Err() but the SQLite
implementation returned nil, which could silently swallow row iteration
errors and return partial results.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 19:53:18 -08:00
opal
cb1e355511 refactor(rpc): fix leaky abstraction - remove direct sqlite imports (bd-oslm)
The RPC layer was directly importing storage/sqlite and doing type
assertions to *sqlite.SQLiteStorage, which broke abstraction and
prevented Dolt backend from using certain features.

Changes:
- Add CompactableStorage and BatchDeleter interfaces to storage package
- Move CompactionCandidate and DeleteIssuesResult types to internal/types
- Update RPC handlers to use interface type assertions instead of concrete types
- Update compact package to accept interface instead of concrete type
- Remove unnecessary type assertions where methods exist on Storage interface

This enables Dolt (and future backends) to implement optional features
like compaction and batch delete through proper interface extension.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Executed-By: beads/opal
Rig: beads
Role: polecat
2026-01-26 20:28:27 -08:00
jasper
aa663c40d7 fix(storage): move BatchCreateOptions to storage package for backend-agnostic batch creation
Fixes the leaky abstraction where molecules and importer packages directly used
sqlite.BatchCreateOptions and type-asserted to *sqlite.SQLiteStorage.

Changes:
- Add storage.BatchCreateOptions and storage.OrphanHandling as backend-agnostic types
- Add CreateIssuesWithFullOptions to Storage interface
- Implement for SQLite, Dolt, and Memory backends
- Update molecules.go to use Storage interface method instead of sqlite type assertion
- Update importer.go to use storage types instead of sqlite-specific interface
- Update cook.go to use storage interface method
- Add deprecated aliases in sqlite package for backward compatibility

This enables batch create optimizations for all backends including Dolt.

Closes: bd-nrcp

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Executed-By: beads/jasper
Rig: beads
Role: polecat
2026-01-26 20:21:53 -08:00