* fix: backward-compatible JSONL import for all pre-v1.0 formats
Three breaking changes accumulated across versions that prevent
importing old JSONL data:
1. Comment.ID changed from int64 to string in v1.0 — old JSONL
with numeric comment IDs crashes the importer. Add custom
UnmarshalJSON that accepts both types.
2. "wisp" field renamed to "ephemeral" in v0.38 — JSONL from
v0.35–v0.37 silently loses the ephemeral flag. Map the old
field name during import.
3. BondRef.proto_id renamed to source_id — old JSONL silently
loses bond source references. Accept proto_id as alias.
Also improve auto-import error messaging: when upgrade recovery
fails, tell the user their data is safe and suggest bd init
--from-jsonl instead of silently swallowing the error.
* fix: use American English spelling for misspell linter
* test: add unit tests for backward-compatible UnmarshalJSON and legacy import
- TestCommentUnmarshalJSON: string IDs, numeric IDs (pre-v1.0),
zero, large numbers, missing ID, invalid JSON
- TestBondRefUnmarshalJSON: source_id, proto_id (legacy),
both fields (precedence), neither, invalid JSON
- TestImportFromLocalJSONL_LegacyFormats: numeric comment IDs
survive import, wisp→ephemeral mapping (true/false/absent)
bd export now includes persistent memories (from bd remember) as JSONL
lines with {"_type":"memory","key":"...","value":"..."}. bd import
detects these records and restores them via SetConfig, making
bd export | bd import a lossless round-trip for both issues and memories.
New flags:
- bd export --no-memories: exclude memories from export
Backward compatible: JSONL without _type fields imports as before.
Co-Authored-By: Patrick Farrell <pfarrell@whamcloud.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Old polecat agent entries in issues.jsonl have status 'tombstone' which
is not a valid status for import. This causes bd init --from-jsonl to
fail with 'validation failed: invalid status: tombstone'. Skip these
deleted entries during JSONL import since they represent removed beads
that should not be re-imported.
Fixes gb-9sx
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Re-implement the --from-jsonl flag for bd init that was removed during the
JSONL storage layer removal refactor. The flag reads .beads/issues.jsonl and
imports issues into the Dolt server database, enabling fresh clone hydration
in server mode.
Closes GH#2023
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Executed-By: beads/polecats/jasper
Rig: beads
Role: polecats
The Dolt transition epic (dolt-test-woho) was executed on Feb 10, 11 days
before Phase 2 planned date of ~Feb 21, and before the upstream blocker
(dolt-test-djfr: Dolt-in-Git lands upstream) was resolved.
This reverts the premature removal of SQLite backend (42K+ lines) and JSONL
sync layer (9K+ lines) while preserving all non-transition changes merged
since the transition (PRs #1655, #1660, #1670, #1675, #1680, #1684, #1689).
Restored:
- internal/storage/sqlite/ (131 files, full SQLite storage backend)
- internal/importer/ (7 files, JSONL import engine)
- internal/storage/dolt/dirty.go (dirty tracking for Dolt)
- cmd/bd/autoflush.go, autoimport.go, export.go (JSONL sync layer)
- Storage interface dirty tracking methods
- Factory defaults to SQLite (Dolt available via registry)
- Test helpers create SQLite stores instead of Dolt
Removed transition-only files:
- cmd/bd/config_local.go, import_engine.go, jsonl_bootstrap.go, jsonl_path.go
Applied PR #1680 retry logic to restored internal/importer/importer.go.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Executed-By: beads/crew/emma
Rig: beads
Role: crew
Delete the JSONL sync layer files (autoflush.go, autoimport.go, export.go,
import.go, jsonl_deprecated.go) and the internal/importer/ package as part
of the Dolt transition Phase 2.
Preserved actively-used functions in new focused files:
- config_local.go: YAML config parsing (isNoDbModeConfigured, etc.)
- jsonl_path.go: JSONL path resolution and utilities
- jsonl_bootstrap.go: Git-based auto-import on empty database
- import_engine.go: Core import engine inlined from internal/importer/
Removed all FlushManager references, markDirtyAndScheduleFlush calls,
autoImportIfNewer calls, and deleted tests for removed functionality.
38 files changed, -9,753 lines.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add support for processing deletion markers during JSONL import.
When importing, detect compact deletion markers like:
{"id": "gt-123", "_deleted": true, "_deleted_at": "..."}
And call store.DeleteIssue() for those IDs, enabling proper
bidirectional sync with deletion support.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The --protect-left-snapshot mechanism was protecting ALL local issues
by ID alone, ignoring timestamps. This caused newer remote changes to
be incorrectly skipped during cross-worktree sync.
Changes:
- Add BuildIDToTimestampMap() to SnapshotManager for timestamp-aware
snapshot reading
- Change ProtectLocalExportIDs from map[string]bool to map[string]time.Time
- Add shouldProtectFromUpdate() helper that compares timestamps
- Only protect if local snapshot is newer than incoming; allow update
if incoming is newer
This fixes data loss scenarios where:
1. Main worktree closes issue at 11:31
2. Test worktree syncs and incorrectly skips the update
3. Test worktree then pushes stale open state, overwriting mains changes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix sync bug where newly created issues were incorrectly tombstoned during bd sync.
The root cause was git-history-backfill finding issues in local commits on the sync branch, then tombstoning them when they weren't in the merged JSONL. The fix protects issues from the left snapshot (local export) from git-history-backfill.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
When importing JSONL that contains issues in the deletions manifest,
import now:
- Filters out deleted issues before import
- Prints per-issue warning with deletion details (date, actor)
- Shows count of skipped issues in summary
- Suggests --ignore-deletions flag to force import
The new --ignore-deletions flag allows importing issues that are in the
deletions manifest, useful for recovering accidentally deleted issues.
Fixes bd-4zy
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes bd-0b2: The git history backfill mechanism was causing data loss
during JSONL filename migrations (beads.jsonl → issues.jsonl). When issues
existed in the old filename's git history, the backfill incorrectly treated
them as "deleted" and purged them from the database.
Changes:
- Add NoGitHistory field to importer.Options and ImportOptions structs
- Modify purgeDeletedIssues() to skip git history check when flag is set
- Add --no-git-history flag to bd import command
- Add --no-git-history flag to bd sync command
- Update purge_test.go to pass Options argument
Usage:
bd import -i .beads/issues.jsonl --no-git-history
bd sync --no-git-history
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When importing JSONL after merges that include deletions, FK constraint
violations can occur if an issue references a deleted issue. Previously,
import would fail completely. Now it continues and reports skipped dependencies.
Changes:
- Add SkippedDependencies field to Result/ImportResult structs
- Update importDependencies() to detect FK violations using IsForeignKeyConstraintError()
- Log warnings for each skipped dependency with issue IDs and type
- Continue importing remaining dependencies instead of failing
- Display summary of all skipped dependencies at end of import
Example output:
Warning: Skipping dependency due to missing reference: bd-b → bd-a (blocks)
⚠️ Warning: Skipped 2 dependencies due to missing references:
- bd-b → bd-a (blocks)
- bd-c → bd-a (parent-child)
This can happen after merges that delete issues referenced by other issues.
The import continued successfully - you may want to review the skipped dependencies.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes GH-234 by providing automatic resolution for duplicate external_ref
values instead of forcing manual JSONL editing.
Changes:
- Add ClearDuplicateExternalRefs option to importer.Options
- Modify validateNoDuplicateExternalRefs to clear duplicates when enabled
- Keep first occurrence, clear rest when flag is set
- Enhanced error message to suggest the flag
- Add comprehensive tests for the new behavior
Usage: bd import -i issues.jsonl --clear-duplicate-external-refs
Amp-Thread-ID: https://ampcode.com/threads/T-932dcf45-76f2-4994-9b5c-a6eb20a86036
Co-authored-by: Amp <amp@ampcode.com>
When --orphan-handling flag not specified, import was passing empty string
instead of reading config or defaulting to 'allow'. This broke all imports
with hierarchical IDs after git pull.
Fix: Read import.missing_parents config, default to 'allow' if unset.
Priority: flag > config > default
Amp-Thread-ID: https://ampcode.com/threads/T-d089540c-c172-440f-88c9-ff06bde6504d
Co-authored-by: Amp <amp@ampcode.com>
Hash-based IDs make collision resolution unnecessary. The flag was
already non-functional (handleCollisions returns error on collision
regardless of flag value).
Removed:
- --resolve-collisions flag from bd import
- ResolveCollisions field from ImportOptions and importer.Options
- All references in daemon, auto-import, and tests
- Updated error messages to reflect hash IDs don't collide
All import tests pass.
Amp-Thread-ID: https://ampcode.com/threads/T-47dfa0cc-bb71-4467-ac86-f0966a7c5d58
Co-authored-by: Amp <amp@ampcode.com>
- Removed renameImportedIssuePrefixes and supporting functions
- Removed unused imports (fmt, sort, strings, utils)
- Saved ~130 LOC of dead code
- All tests pass
- Removed cmd/bd/import_phases.go (377 LOC of unreachable code)
- Moved helper functions extractPrefix and getPrefixList to import_shared.go
- All tests pass
- Import functionality verified
- Updated manageClosedAt to handle both string and types.Status type assertions
- Added equalTime function for comparing timestamps in import change detection
- Added tests for open→closed and closed→open transitions
- Added comment clarifying closed_at is managed automatically
The bug occurred when UpdateIssue received types.Status instead of string,
causing manageClosedAt to skip setting closed_at when status changed to closed.
Amp-Thread-ID: https://ampcode.com/threads/T-ee774f6d-3b90-4311-976d-60c8dd8fe677
Co-authored-by: Amp <amp@ampcode.com>
- Created internal/importer package with all import logic
- Moved import phases from cmd/bd to internal/importer
- Implemented real importFunc in daemon's checkAndAutoImportIfStale()
- Added single-flight concurrency guard to prevent parallel imports
- Added fast mtime check to avoid unnecessary file reads (99% of requests <0.1ms)
- Fixed import options: RenameOnImport=true instead of SkipPrefixValidation
- Added export trigger after ID remapping to prevent collision loops
- Fixed memory storage interface: added GetDirtyIssueHash, GetExportHash, SetExportHash
- Updated GetDependencyTree signature for reverse parameter
Performance:
- Mtime check: ~0.01ms per request
- Import when needed: ~10-100ms (rare, only after git pull)
- Throughput maintained: 4300+ issues/sec
- No duplicate work with single-flight guard
Fixes critical data corruption bug where daemon served stale data after
git pull, causing fresh JSONL changes to be overwritten.
Amp-Thread-ID: https://ampcode.com/threads/T-71224a2d-b2d7-4173-b21e-449b64f9dd71
Co-authored-by: Amp <amp@ampcode.com>
Enhances rename-prefix command with --repair flag to consolidate databases
with multiple prefixes. Creates shared issue ID utilities to eliminate code
duplication across import and rename operations.
Key changes:
- Add --repair flag to detect and consolidate multiple issue prefixes
- Create internal/utils/issue_id.go with ExtractIssuePrefix() and ExtractIssueNumber()
- Update all duplicate prefix extraction code to use shared utilities
- Add comprehensive tests for repair functionality
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
- Added CheckpointWAL method to SQLite storage
- Import now checkpoints WAL after completion
- Updates main .db file modification time for staleness detection
- PRAGMA wal_checkpoint(FULL) flushes WAL to main database
- Extract field comparison logic into fieldComparator struct
- Reduce complexity from 39 to 11
- Improve code organization and testability
- All tests passing
Amp-Thread-ID: https://ampcode.com/threads/T-4d8cdee6-84e8-4348-ad30-9e59fb3e30cf
Co-authored-by: Amp <amp@ampcode.com>
When importing JSONL that matches the database exactly, import was
reporting '0 created, 0 updated' which was confusing. Now it properly
tracks and reports unchanged issues.
Changes:
- Added Unchanged field to ImportResult
- Track unchanged issues separately from skipped/updated
- Display unchanged count in import summary
- Updated dry-run output to show unchanged count
- Added test to verify correct reporting behavior
Example output: 'Import complete: 0 created, 0 updated, 88 unchanged'
Amp-Thread-ID: https://ampcode.com/threads/T-5dd4843e-9ce3-4fe0-9658-d2227b0a2e4e
Co-authored-by: Amp <amp@ampcode.com>
Based on code review feedback:
- Remove unsafe type assertions that could panic
- Add robust type conversion helpers (strFrom, intFrom, equalStatus, equalIssueType)
- Support multiple numeric types for priority (int, int32, int64, float64)
- Treat empty string and nil as equal for optional fields
- Add default case to detect unknown fields (conservative)
- Add comprehensive unit tests for all edge cases
- Test coverage: enums as strings, numeric conversions, nil/empty handling
Amp-Thread-ID: https://ampcode.com/threads/T-225f8c56-0710-46e9-9db2-dbf90cf91911
Co-authored-by: Amp <amp@ampcode.com>
- Add issueDataChanged() to detect when issue content actually changes
- Only call UpdateIssue when data differs from existing issue
- Unchanged issues are skipped to avoid updating timestamps
- Add tests for idempotent import behavior
- Fixes perpetually dirty JSONL after every bd command
Amp-Thread-ID: https://ampcode.com/threads/T-225f8c56-0710-46e9-9db2-dbf90cf91911
Co-authored-by: Amp <amp@ampcode.com>
This commit fixes all 7 failing autoimport tests by correcting a bug
in import_shared.go where the function returned early when no issue
prefix was configured, preventing any imports from happening.
Changes:
- Fix early return bug in importIssuesCore (import_shared.go:85-121)
- Now continues with import even when prefix is not configured
- Only validates prefixes when a prefix is actually set
- Delete test issues bd-50 through bd-63 (14 cleanup issues)
All tests now pass:
✅ TestAutoImportMultipleCollisionsRemapped
✅ TestAutoImportAllCollisionsRemapped
✅ TestAutoImportExactMatchesOnly
✅ TestAutoImportNewIssuesOnly
✅ TestAutoImportIfNewer
✅ TestAutoImportNoCollision
✅ TestAutoImportClosedAtInvariant
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Boundary-aware ID replacement (prevents wy-1 corrupting wy-10)
- Sort IDs by length descending before replacement
- Validate configured prefix is not empty
- Validate ID format before renaming (numeric suffix required)
- All critical issues from oracle review addressed
- Detects prefix mismatches during import
- Shows warning with mismatched prefixes and counts
- Added --rename-on-import flag to automatically fix prefixes
- Auto-import is lenient about prefixes (doesn't enforce)
- All tests passing
- Implements bd stale command to show issues with execution_state where executor is dead/stopped
- Adds --release flag to automatically release orphaned issues
- Adds --threshold flag to customize heartbeat staleness threshold (default: 300s/5min)
- Handles missing executor instances (LEFT JOIN) for cases where executor was deleted
- Adds QueryContext and BeginTx helper methods to SQLiteStorage for advanced queries
- Fixes ExternalRef comparison bug in import_shared.go (pointer vs string)
- Removes unused imports in import.go
Resolves vc-124