beads/docs
Peter Chanthamynavong 1561374c04
feat(sync): pull-first sync with 3-way merge (#918)
* feat(sync): implement pull-first synchronization strategy

- Add --pull-first flag and logic to sync command
- Introduce 3-way merge stub for issue synchronization
- Add concurrent edit tests for the pull-first flow

Ensures local changes are reconciled with remote updates before pushing to prevent data loss.

* feat(sync): implement 3-way merge and state tracking

- Implement 3-way merge algorithm for issue synchronization
- Add base state storage to track changes between syncs
- Add comprehensive tests for merge logic and persistence

Ensures data consistency and prevents data loss during concurrent
issue updates.

* feat(sync): implement field-level conflict merging

- Implement field-level merge logic for issue conflicts
- Add unit tests for field-level merge strategies

Reduces manual intervention by automatically resolving overlapping updates at the field level.

* refactor(sync): simplify sync flow by removing ZFC checks

The previous sync implementation relied on Zero-False-Convergence (ZFC)
staleness checks which are redundant following the transition to
structural 3-way merging. This legacy logic added complexity and
maintenance overhead without providing additional safety.

This commit introduces a streamlined sync pipeline:
- Remove ZFC staleness validation from primary sync flow
- Update safety documentation to reflect current merge strategy
- Eliminate deprecated unit tests associated with ZFC logic

These changes reduce codebase complexity while maintaining data
integrity through the robust structural 3-way merge implementation.

* feat(sync): default to pull-first sync workflow

- Set pull-first as the primary synchronization workflow
- Refactor core sync logic for better maintainability
- Update concurrent edit tests to validate 3-way merge logic

Reduces merge conflicts by ensuring local state is current before pushing changes.

* refactor(sync): clean up lint issues in merge code

- Remove unused error return from MergeIssues (never returned error)
- Use _ prefix for unused _base parameter in mergeFieldLevel
- Update callers to not expect error from MergeIssues
- Keep nolint:gosec for trusted internal file path

* test(sync): add mode compatibility and upgrade safety tests

Add tests addressing Steve's PR #918 review concerns:

- TestSyncBranchModeWithPullFirst: Verifies sync-branch config
  storage and git branch creation work with pull-first
- TestExternalBeadsDirWithPullFirst: Verifies external BEADS_DIR
  detection and pullFromExternalBeadsRepo
- TestUpgradeFromOldSync: Validates upgrade safety when
  sync_base.jsonl doesn't exist (first sync after upgrade)
- TestMergeIssuesWithBaseState: Comprehensive 3-way merge cases
- TestLabelUnionMerge: Verifies labels use union (no data loss)

Key upgrade behavior validated:
- base=nil (no sync_base.jsonl) safely handles all cases
- Local-only issues kept (StrategyLocal)
- Remote-only issues kept (StrategyRemote)
- Overlapping issues merged (LWW scalars, union labels)

* fix(sync): report line numbers for malformed JSON

Problem:
- JSON decoding errors when loading sync base state lacked line numbers
- Difficult to identify location of syntax errors in large state files

Solution:
- Include line number reporting in JSON decoder errors during state loading
- Add regression tests for malformed sync base file scenarios

Impact:
- Users receive actionable feedback for corrupted state files
- Faster troubleshooting of manual configuration errors

* fix(sync): warn on large clock skew during sync

Problem:
- Unsynchronized clocks between systems could lead to silent merge errors
- No mechanism existed to alert users of significant timestamp drift

Solution:
- Implement clock skew detection during sync merge
- Log a warning when large timestamp differences are found
- Add comprehensive unit tests for skew reporting

Impact:
- Users are alerted to potential synchronization risks
- Easier debugging of time-related merge issues

* fix(sync): defer state update until remote push succeeds

Problem:
- Base state updated before confirming remote push completion
- Failed pushes resulted in inconsistent local state tracking

Solution:
- Defer base state update until after the remote push succeeds

Impact:
- Ensures local state accurately reflects remote repository status
- Prevents state desynchronization during network or push failures

* fix(sync): prevent concurrent sync operations

Problem:
- Multiple sync processes could run simultaneously
- Overlapping operations risk data corruption and race conditions

Solution:
- Implement file-based locking using gofrs/flock
- Add integration tests to verify locking behavior

Impact:
- Guarantees execution of a single sync process at a time
- Eliminates potential for data inconsistency during sync

* docs: document sync architecture and merge model

- Detail the 3-way merge model logic
- Describe the core synchronization architecture principles

* fix(lint): explicitly ignore lock.Unlock return value

errcheck linter flagged bare defer lock.Unlock() calls. Wrap in
anonymous function with explicit _ assignment to acknowledge
intentional ignore of unlock errors during cleanup.

* fix(lint): add sync_merge.go to G304 exclusions

The loadBaseState and saveBaseState functions use file paths derived
from trusted internal sources (beadsDir parameter from config). Add
to existing G304 exclusion list for safe JSONL file operations.

* feat(sync): integrate sync-branch into pull-first flow

When sync.branch is configured, doPullFirstSync now:
- Calls PullFromSyncBranch before merge
- Calls CommitToSyncBranch after export

This ensures sync-branch mode uses the correct branch for
pull/push operations.

* test(sync): add E2E tests for sync-branch and external BEADS_DIR

Adds comprehensive end-to-end tests:
- TestSyncBranchE2E: verifies pull→merge→commit flow with remote changes
- TestExternalBeadsDirE2E: verifies sync with separate beads repository
- TestExternalBeadsDirDetection: edge cases for repo detection
- TestCommitToExternalBeadsRepo: commit handling

* refactor(sync): remove unused rollbackJSONLFromGit

Function was defined but never called. Pull-first flow saves base
state after successful push, making this safety net unnecessary.

* test(sync): add export-only mode E2E test

Add TestExportOnlySync to cover --no-pull flag which was the only
untested sync mode. This completes full mode coverage:

- Normal (pull-first): sync_test.go, sync_merge_test.go
- Sync-branch: sync_modes_test.go:TestSyncBranchE2E (PR#918)
- External BEADS_DIR: sync_external_test.go (PR#918)
- From-main: sync_branch_priority_test.go
- Local-only: sync_local_only_test.go
- Export-only: sync_modes_test.go:TestExportOnlySync (this commit)

Refs: #911

* docs(sync): add sync modes reference section

Document all 6 sync modes with triggers, flows, and use cases.
Include mode selection decision tree and test coverage matrix.

Co-authored-by: Claude <noreply@anthropic.com>

* test(sync): upgrade sync-branch E2E tests to bare repo

- Replace mocked repository with real bare repo setup
- Implement multi-machine simulation in sync tests
- Refactor test logic to handle distributed states

Coverage: sync-branch end-to-end scenarios

* test(sync): add daemon sync-branch E2E tests

- Implement E2E tests for daemon sync-branch flow
- Add test cases for force-overwrite scenarios

Coverage: daemon sync-branch workflow in cmd/bd

* docs(sync): document sync-branch paths and E2E architecture

- Describe sync-branch CLI and Daemon execution flow
- Document the end-to-end test architecture

* build(nix): update vendorHash for gofrs/flock dependency

New dependency added for file-based sync locking changes the
Go module checksum.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-07 21:27:20 -08:00
..
dev-notes docs: fix broken relative links across documentation 2025-12-16 00:46:08 -08:00
ADAPTIVE_IDS.md docs: fix broken relative links across documentation 2025-12-16 00:46:08 -08:00
ADVANCED.md docs: add Database Redirects section to ADVANCED.md (bd-8x43) 2025-12-27 22:04:25 -08:00
AIDER_INTEGRATION.md Add Aider integration for beads issue tracking 2025-11-23 15:13:54 -08:00
ANTIVIRUS.md feat: add Windows code signing infrastructure (bd-14v0) 2025-12-24 00:07:16 -08:00
ARCHITECTURE.md Add close_reason to JSONL format documentation and tests (bd-lxzx) 2025-12-30 17:05:48 -08:00
ATTRIBUTION.md Reorganize project structure: move Go files to internal/beads, docs to docs/ 2025-11-05 21:04:00 -08:00
CLAUDE.md docs: fix broken relative links across documentation 2025-12-16 00:46:08 -08:00
CLAUDE_INTEGRATION.md docs: document bd setup command 2025-12-16 01:08:42 -08:00
CLI_REFERENCE.md feat: add bd state and bd set-state helper commands (bd-7l67) 2025-12-30 15:57:08 -08:00
COLLISION_MATH.md docs: fix broken relative links across documentation 2025-12-16 00:46:08 -08:00
COMMUNITY_TOOLS.md docs: add lazybeads to community tools (#951) 2026-01-07 20:44:34 -08:00
CONFIG.md feat(config): add validation.on-create and validation.on-sync config options (bd-t7jq) 2026-01-01 19:31:12 -08:00
CONTRIBUTOR_NAMESPACE_ISOLATION.md Design contributor namespace isolation for beads pollution prevention 2025-12-30 17:06:36 -08:00
daemon-summary.md feat(daemon): unify auto-sync config for simpler agent workflows (#904) 2026-01-06 12:52:19 -08:00
DAEMON.md fix(docs): standardize sync branch name and daemon syntax (GH#376) 2025-12-25 16:22:09 -08:00
DELETIONS.md docs: update documentation for CLI consolidation (bd-bxqv) 2025-12-27 16:21:08 -08:00
ERROR_HANDLING.md docs: Complete error handling audit 2025-11-24 00:35:04 -08:00
EXCLUSIVE_LOCK.md Reorganize project structure: move Go files to internal/beads, docs to docs/ 2025-11-05 21:04:00 -08:00
EXTENDING.md docs: update documentation for CLI consolidation (bd-bxqv) 2025-12-27 16:21:08 -08:00
FAQ.md docs: fix README and PLUGIN.md documentation issues (#763) 2025-12-27 17:20:00 -08:00
GIT_INTEGRATION.md fix(docs): standardize sync branch name and daemon syntax (GH#376) 2025-12-25 16:22:09 -08:00
graph-links.md refactor: remove Gas Town references from codebase 2025-12-30 14:13:32 -08:00
INSTALLING.md Add bun installation method to INSTALLING.md (#930) 2026-01-06 19:14:38 -08:00
INTERNALS.md bd sync: 2025-11-25 11:35:26 2025-11-25 11:35:26 -08:00
LABELS.md feat: add bd state and bd set-state helper commands (bd-7l67) 2025-12-30 15:57:08 -08:00
LINTING.md Document -short flag, update linting baseline, clarify bd edit MCP exclusion 2025-11-06 19:42:01 -08:00
MOLECULES.md refactor: remove Gas Town references from codebase 2025-12-30 14:13:32 -08:00
MULTI_REPO_AGENTS.md docs: update beads.jsonl references to issues.jsonl 2025-12-16 01:08:46 -08:00
MULTI_REPO_HYDRATION.md Extract SQLite migrations into separate files (bd-fb95094c.7) 2025-11-06 20:06:45 -08:00
MULTI_REPO_MIGRATION.md Merge GH#409 2025-12-16 01:19:07 -08:00
PERFORMANCE_TESTING.md docs: add performance testing guide (bd-zj8e) 2025-11-28 23:26:24 -08:00
PLUGIN.md Update link to install script (#867) 2026-01-03 12:53:58 -08:00
pr-752-chaos-testing-review.md refactor: remove unused bd pin/unpin/hook commands (bd-x0zl) 2025-12-27 16:02:15 -08:00
PROTECTED_BRANCHES.md fix(docs): standardize sync branch name and daemon syntax (GH#376) 2025-12-25 16:22:09 -08:00
QUICKSTART.md feat: add bd admin parent command for cleanup/compact/reset (bd-3u8m) 2025-12-27 16:21:08 -08:00
README_TESTING.md DOCTOR IMPROVEMENTS: visual improvements/grouping + add comprehensive tests + fix gosec warnings (#656) 2025-12-20 03:10:06 -08:00
RELEASING.md docs: fix broken relative links across documentation 2025-12-16 00:46:08 -08:00
ROUTING.md Add routing integration tests and documentation 2025-11-04 23:12:42 -08:00
SETUP.md feat(setup): refactor to recipe-based architecture (bd-i3ed) 2026-01-04 21:57:09 -08:00
SYNC.md feat(sync): pull-first sync with 3-way merge (#918) 2026-01-07 21:27:20 -08:00
TESTING.md chore: code health review - test fix and error comments (bd-9g1z, bd-ork0) 2025-12-22 21:30:57 -08:00
TESTING_PHILOSOPHY.md DOCTOR IMPROVEMENTS: visual improvements/grouping + add comprehensive tests + fix gosec warnings (#656) 2025-12-20 03:10:06 -08:00
TROUBLESHOOTING.md docs: fix README and PLUGIN.md documentation issues (#763) 2025-12-27 17:20:00 -08:00
UI_PHILOSOPHY.md docs: consolidate duplicate UI_PHILOSOPHY files (GH#745) 2025-12-27 18:01:59 -08:00
UNINSTALLING.md chore: remove obsolete BD_GUIDE.md references 2025-12-20 20:50:33 -08:00
WORKTREES.md refactor: remove unused bd pin/unpin/hook commands (bd-x0zl) 2025-12-27 16:02:15 -08:00