diff --git a/docs/DOLT-BACKEND.md b/docs/DOLT-BACKEND.md index 3e9911e4e..a6b2430fb 100644 --- a/docs/DOLT-BACKEND.md +++ b/docs/DOLT-BACKEND.md @@ -1,6 +1,6 @@ # Dolt Backend Documentation -The canonical Dolt backend guide is [DOLT.md](DOLT.md). +The canonical Dolt backend guide is [architecture/dolt.md](architecture/dolt.md). The previous long-form duplicate of this file has been staged at [staged-for-removal/DOLT-BACKEND.md](../engdocs/staged-for-removal/DOLT-BACKEND.md). diff --git a/docs/DOLT.md b/docs/DOLT.md index 48608ee32..472d56198 100644 --- a/docs/DOLT.md +++ b/docs/DOLT.md @@ -1,737 +1,7 @@ # Dolt Backend for Beads -Beads uses Dolt as its storage backend. Dolt provides a version-controlled SQL database with cell-level merge, native branching, and two deployment modes. - -## Why Dolt? - -- **Native version control** — cell-level diffs and merges, not line-based -- **Multi-writer support** — server mode enables concurrent agents -- **Built-in history** — every write creates a Dolt commit -- **Native branching** — Dolt branches independent of git branches -- **Single-binary option** — embedded mode for solo users (no server needed) - -## Getting Started - -### Install Dolt (Server Mode Only) - -Embedded mode includes everything in the `bd` binary; no separate Dolt install -is needed. Install the standalone `dolt` CLI only when you want to run server -mode or work directly with the database via `dolt sql`. - -```bash -# macOS -brew install dolt - -# Linux -curl -L https://github.com/dolthub/dolt/releases/latest/download/install.sh | bash - -# Verify installation -dolt version -``` - -### New Project - -```bash -# Embedded mode (single writer, no server — default for standalone) -bd init - -# Server mode (multi-writer, e.g. orchestrator) -gt dolt start # Start the Dolt server -bd init --server # Initialize with server mode -``` - -### Migrate from SQLite (Legacy) - -If upgrading from an older version that used SQLite: - -> **Note:** The `bd migrate --to-dolt` command was removed in v0.58.0. -> For pre-0.50 installations with JSONL data, use the migration script: -> -> ```bash -> scripts/migrate-jsonl-to-dolt.sh -> ``` -> -> See [Troubleshooting](TROUBLESHOOTING.md#circuit-breaker-server-appears-down-failing-fast) if you encounter connection errors after migration. - -Migration creates backups automatically. Your original SQLite database is preserved as `beads.backup-pre-dolt-*.db`. - -## Modes of Operation - -### Embedded Mode (Solo / Standalone) - -In-process Dolt engine — no separate server needed. This is the default for -standalone Beads users. The `bd` binary includes everything; just `bd init` and go. - -- Single-writer (one process at a time) -- Data lives in `.beads/embeddeddolt/` alongside your code -- Push to GitHub with `bd dolt push` — code and issues in one repo -- Zero ops: no server, no ports, no PID files - -### Server Mode (Multi-Writer / Orchestrator) - -Connects to a running `dolt sql-server` for multi-client access. - -```bash -# Start the server (orchestrator) -gt dolt start - -# Or manually -cd ~/.dolt-data/beads && dolt sql-server --port 3307 -``` - -```bash -# Initialize in server mode -bd init --server - -# Or switch via environment variable -export BEADS_DOLT_SERVER_MODE=1 -``` - -```yaml -# .beads/config.yaml (server mode settings) -dolt: - mode: server - host: 127.0.0.1 - port: 3307 - user: root -``` - -Switch to server mode when you need: -- Multiple agents writing simultaneously -- Orchestrator multi-rig setups -- Federation with remote peers - -## Migrating Between Backends - -You can migrate data between embedded mode and server mode using `bd backup`. -Both directions preserve full Dolt commit history. - -`bd export` is not a substitute for this flow. JSONL exports contain issue -records from the issues table for migration and interoperability; they do not -capture Dolt branches, full commit history, working-set state, or non-issue -tables. Use `bd backup` or a manual Dolt backup when you need a restorable -database backup. - -### Server → Embedded - -1. **Create a backup from the server-mode project:** - - ```bash - # In the server-mode project directory - bd backup init /path/to/backup-dir - bd backup sync - ``` - -2. **Create a new embedded-mode project and restore:** - - ```bash - mkdir new-project && cd new-project - bd init # creates an embedded-mode project by default - bd backup restore --force /path/to/backup-dir - ``` - - `--force` overwrites the freshly-initialized database with the backup - contents. The restore automatically: - - Updates `metadata.json` to match the restored project identity - - Registers the backup directory for future `bd backup sync` - - Backfills the embedded migration tracker (`schema_migrations`) - -3. **Verify:** - - ```bash - bd list - bd backup status - ``` - -### Embedded → Server - -1. **Create a backup from the embedded-mode project:** - - ```bash - # In the embedded-mode project directory - bd backup init /path/to/backup-dir - bd backup sync - ``` - -2. **Create a new server-mode project and restore:** - - ```bash - mkdir new-project && cd new-project - bd init --server # creates a server-mode project - bd backup restore --force /path/to/backup-dir - ``` - -3. **Verify:** - - ```bash - bd list - bd backup status - ``` - -### Backup Commands Reference - -| Command | Description | -|---------|-------------| -| `bd backup init ` | Register a backup destination (filesystem or DoltHub URL) | -| `bd backup sync` | Push database to the configured backup destination | -| `bd backup restore [path]` | Restore from a backup directory (`--force` to overwrite) | -| `bd backup remove` | Unregister the backup destination | -| `bd backup status` | Show backup configuration and last sync time | - -### Notes - -- Data locations differ between modes: `.beads/embeddeddolt/` (embedded) vs `.beads/dolt/` (server) -- The backup directory is a full Dolt backup, not an `issues.jsonl` export — it can be on a local drive, NAS, or DoltHub -- You can also migrate via Dolt remotes (`bd dolt push` / `bd dolt pull`) if both projects share a remote - -The sections below are the canonical backend migration reference. - -## Federation (Peer-to-Peer Sync) - -Federation lets independent Dolt-backed workspaces ("towns") sync issues -directly with each other via `bd federation add-peer`/`sync`/`status`, -without a central hub. Credentials are AES-256 encrypted and stored locally. - -See [FEDERATION.md](FEDERATION.md) for the full setup guide, including -peer configuration, sovereignty tiers, sync/status/topology details, and -troubleshooting. - -## Dolt Remotes - -Use `bd dolt remote add` to configure remotes. This ensures the running Dolt SQL -server sees the remote immediately. Remotes added directly with the `dolt` CLI -are written to filesystem config and may not be visible to the server until -restart. - -```bash -# DoltHub (public or private) -bd dolt remote add origin https://doltremoteapi.dolthub.com/org/beads - -# S3 -bd dolt remote add origin aws://[bucket]/path/to/repo - -# GCS -bd dolt remote add origin gs://[bucket]/path/to/repo - -# Git SSH (GitHub, GitLab, etc.) -bd dolt remote add origin git+ssh://git@github.com/org/repo.git - -# Local file system -bd dolt remote add origin file:///path/to/remote -``` - -### Push/Pull - -```bash -bd dolt push -bd dolt pull -``` - -`bd dolt remote add` registers the remote through the Dolt store API. SQL -remotes are the source of truth for `bd dolt remote list`, `bd dolt push`, and -`bd dolt pull`. - -For git-protocol remotes, credentialed external-server remotes, and cloud -remotes whose credentials are only present in the current shell, `bd dolt push` -and `bd dolt pull` automatically materialize a matching local CLI remote before -using the `dolt` CLI transport. The CLI remote is a local transport mirror, not -a separate configuration source. - -If you are upgrading from an older beads version and previously added remotes -with raw `dolt remote add`, re-register them with `bd dolt remote add -` so they are visible through SQL. `bd doctor` reports legacy CLI-only or -mismatched CLI remotes under `Dolt Remote Migration`. - -> **Sharing a Git repo**: Dolt stores data under `refs/dolt/data`, separate -> from standard Git refs (`refs/heads/`, `refs/tags/`). You can safely point a -> `git+ssh://` remote at the same repository as your project source code. See -> [Dolt Git Remotes](https://docs.dolthub.com/concepts/dolt/git/remotes). - -### List/Remove Remotes - -```bash -bd dolt remote list # Shows SQL-configured remotes -bd dolt remote remove origin # Removes the remote -``` - -## Contributor Onboarding (Clone Bootstrap) - -When someone clones a repository that uses Dolt backend: - -1. Run `bd bootstrap` in the clone -2. If the git remote has `refs/dolt/data` (pushed via `bd dolt push`), - `bd bootstrap` auto-detects it and clones the database from the remote -3. Work continues normally — all existing issues are available - -**No manual steps required** beyond `bd bootstrap`. The auto-detect: -- Probes `origin` for `refs/dolt/data` -- Clones the Dolt database from the remote (instead of creating a fresh one) -- Configures the Dolt remote for future `bd dolt push`/`pull` - -If `sync.remote` is set in `.beads/config.yaml`, that takes precedence -over auto-detection. Any Dolt-compatible remote URL is supported (DoltHub, -S3, GCS, file, or git). On brand-new projects, `bd init` auto-detects -`git origin` and persists it as `sync.remote`, so the first `bd dolt push` -publishes Dolt history to `refs/dolt/data` on the same git remote. - -### Verifying Bootstrap Worked - -```bash -bd list # Should show issues -bd vc log # Should show initial commit -``` - -## Troubleshooting - -### Server Not Running - -**Symptom:** Connection refused errors when using server mode. - -``` -failed to create database: dial tcp 127.0.0.1:3307: connect: connection refused -``` - -**Fix:** -```bash -gt dolt start # Orchestrator command -# Or -gt dolt status # Check if running -``` - -### Bootstrap Not Running - -**Symptom:** `bd list` shows nothing on fresh clone. - -**Check:** -```bash -ls .beads/dolt/ # Should NOT exist (pre-bootstrap) -BD_DEBUG=1 bd list # See bootstrap output -``` - -**Force bootstrap:** -```bash -rm -rf .beads/dolt # Remove broken state -bd list # Re-triggers bootstrap -``` - -### Database Corruption - -**Symptom:** Queries fail, inconsistent data. - -**Diagnosis:** -```bash -bd doctor # Basic checks -bd doctor --deep # Full validation -bd doctor --server # Server mode checks (if applicable) -``` - -**Recovery options:** - -1. **Repair what's fixable:** - ```bash - bd doctor --fix - ``` - -2. **Rebuild from remote:** - ```bash - rm -rf .beads/dolt - bd list # Re-triggers bootstrap - ``` - -### Already Committed `.beads/dolt/` to Git - -If you accidentally committed a Dolt data directory: - -1. Update gitignore: `bd doctor --fix` -2. Remove it from git tracking: `git rm --cached -r .beads/dolt/` (or `.beads/embeddeddolt/`) -3. Commit the removal: `git commit -m "fix: remove accidentally committed dolt data"` -4. To purge from history, use [BFG Repo-Cleaner](https://rtyley.github.io/bfg-repo-cleaner/) or `git filter-repo` - -### Lock Contention (Embedded Mode) - -**Symptom:** "database is locked" errors. - -Embedded mode is single-writer (enforced via file lock). If you need concurrent -access, switch to server mode. See [Migrating Between Backends](#migrating-between-backends). - -## Configuration Reference - -```yaml -# .beads/config.yaml - -# Dolt settings -dolt: - # Auto-commit Dolt history after writes (default: on for embedded, off for server) - auto-commit: on # on | off - - # Storage mode (default: embedded) - mode: embedded # embedded | server - # Server mode settings (only used when mode: server) - host: 127.0.0.1 - port: 3307 - user: root - # Password: env var or credentials file (see below) - - # Shared server mode (GH#2377): all projects share a single Dolt server - # at ~/.beads/shared-server/. Each project uses its own database (prefix-based). - # Eliminates port conflicts and reduces resource usage on multi-project machines. - shared-server: false # true | false -``` - -### Environment Variables - -| Variable | Purpose | -|----------|---------| -| `BEADS_DOLT_PASSWORD` | Server mode password (highest priority) | -| `BEADS_CREDENTIALS_FILE` | Path to credentials file (overrides default location) | -| `BEADS_DOLT_SERVER_MODE` | Enable server mode (set to "1") | -| `BEADS_DOLT_SERVER_HOST` | Server host (default: 127.0.0.1) | -| `BEADS_DOLT_SERVER_PORT` | Server port (default: 3307, or 3308 in shared mode) | -| `BEADS_DOLT_SERVER_TLS` | Enable TLS (set to "1" or "true") | -| `BEADS_DOLT_SERVER_USER` | MySQL connection user | -| `BEADS_DOLT_SHARED_SERVER` | Enable shared server mode (set to "1" or "true") | -| `DOLT_REMOTE_USER` | Clone/push/pull auth user | -| `DOLT_REMOTE_PASSWORD` | Clone/push/pull auth password | -| `BD_DOLT_AUTO_COMMIT` | Override auto-commit setting | - -### Credentials File - -For multi-server setups, you can store passwords in an INI-style credentials file -instead of juggling environment variables per project. Passwords are looked up by -`[host:port]` section, so each project automatically gets the right password based -on its configured server. - -**Password resolution order:** -1. `BEADS_DOLT_PASSWORD` env var (highest priority, existing behavior) -2. Credentials file lookup by `[host:port]` (using the resolved runtime port) -3. Empty string (no password) - -**Port resolution note:** The `[host:port]` used for credential lookup matches the -resolved runtime port (from the port file, env var, or config — in that priority -order), not necessarily the port stored in `metadata.json`. This matters when using -IAP tunnels: if your tunnel maps remote:3307 to localhost:3308, store your password -under `[127.0.0.1:3308]` and the credentials file will match the actual connection. - -**Default location:** `~/.config/beads/credentials` (Linux/macOS), `%APPDATA%\beads\credentials` (Windows) - -**Override location:** Set `BEADS_CREDENTIALS_FILE` env var. - -**File format:** - -```ini -# ~/.config/beads/credentials -[127.0.0.1:3307] -password=localDevPassword - -[beads.company.com:3307] -password=teamServerPassword - -[10.0.1.50:3308] -password=officePassword -``` - -**Permissions:** On Linux/macOS, a warning is printed to stderr if the file is -readable by group or others (mirrors ssh behavior). Set permissions with: - -```bash -chmod 600 ~/.config/beads/credentials -``` - -## Dolt Version Control - -Dolt maintains its own version history, separate from Git: - -```bash -# View Dolt commit history -bd vc log - -# Show diff between Dolt commits -bd vc diff HEAD~1 HEAD - -# Create manual checkpoint -bd vc commit -m "Checkpoint before refactor" -``` - -### Auto-Commit Behavior - -In **embedded mode** (standalone default), each `bd` write command creates a Dolt commit: - -```bash -bd create "New issue" # Creates issue + Dolt commit -``` - -In **server mode** (orchestrator), auto-commit defaults to OFF because the server -manages its own transaction lifecycle. Firing `DOLT_COMMIT` after every write -under concurrent load causes 'database is read only' errors. - -Override for batch operations (embedded) or explicit commits (server): - -```bash -bd --dolt-auto-commit off create "Issue 1" -bd --dolt-auto-commit off create "Issue 2" -bd vc commit -m "Batch: created issues" -``` - -## Server Management (Orchestrator) - -The orchestrator provides integrated Dolt server management: - -```bash -gt dolt start # Start server (background) -gt dolt stop # Stop server -gt dolt status # Show server status -gt dolt logs # View server logs -gt dolt sql # Open SQL shell -``` - -Server runs on port 3307 (avoids MySQL conflict on 3306). - -### Standalone-to-managed-city handoff - -When an existing standalone project is later added to a managed city or -orchestrator, avoid letting two Dolt servers become sources of truth for the -same beads database name. A common split-brain symptom is that `.beads/dolt-server.port` -points at the old standalone server while the shell environment points `bd` at -the managed server with `BEADS_DOLT_PORT` or `BEADS_DOLT_SERVER_PORT`. - -Check before migrating: - -```bash -bd doctor -bd dolt status -``` - -`bd doctor` warns when the runtime managed port differs from the local port -file. The warning is intentionally diagnostic only; do not delete the local port -file until the standalone store has been exported and imported into the managed -server. - -Safe manual handoff: - -```bash -# From the standalone project, without managed-city port overrides: -unset BEADS_DOLT_PORT BEADS_DOLT_SERVER_PORT -bd backup -bd export > /tmp/beads-standalone.jsonl -bd dolt stop - -# Then enter the managed-city environment and import into its Dolt server: -bd import /tmp/beads-standalone.jsonl -bd doctor -``` - -After `bd doctor` shows one healthy store and the imported issue count is -correct, archive the old local Dolt data directory instead of deleting it -immediately. Keep the backup until the managed city has been pushed or otherwise -snapshotted. - -### Shared Server Mode - -On machines with multiple beads projects, each project normally starts its own Dolt server. -Shared server mode runs a single Dolt server at `~/.beads/shared-server/` that serves all projects: - -```bash -# Enable for this project -bd dolt set shared-server true - -# Or enable machine-wide via environment variable -export BEADS_DOLT_SHARED_SERVER=1 - -# Or enable during init -bd init --prefix myproject --shared-server -``` - -**Benefits:** -- No port conflicts between projects (single server on port 3308, avoids orchestrator on 3307) -- Reduced resource usage (one process instead of many) -- Automatic database isolation (each project uses its own database name) - -**How it works:** -- Server state files (PID, port, lock, log) live in `~/.beads/shared-server/` -- Dolt data directory: `~/.beads/shared-server/dolt/` -- Each project's database is stored as a subdirectory (e.g., `~/.beads/shared-server/dolt/myproject/`) -- The file lock mechanism ensures safe concurrent access from multiple projects -- Default port is 3308 (not 3307) to avoid conflict with the orchestrator. Override with `BEADS_DOLT_SERVER_PORT` or `dolt.port` in config.yaml - -**Important:** Each project on a shared server **must have a unique prefix** (database name). -Two projects with the same prefix share the same database — if this happens accidentally, -the project identity check will detect the mismatch and refuse to connect, preventing -silent data corruption. Always use distinct prefixes when running `bd init --shared-server`. - -```bash -# Check shared server status from any project -bd dolt status - -# Show full configuration including shared mode -bd dolt show -``` - -### Data Location (Orchestrator) - -``` -/.dolt-data/ -├── hq/ # Town beads (hq-*) -├── my-project/ # Project rig (mp-*) -├── beads/ # Beads rig (bd-*) -└── other-project/ # Other rig (op-*) -``` - -### Central Dolt Server (macOS LaunchAgent) - -If you do not use the orchestrator but still want a single persistent Dolt -server for multiple projects on macOS, run a custom `LaunchAgent` instead of -spawning per-project embedded instances. - -#### Why Not `brew services start dolt`? - -After installing Dolt with `brew install dolt`, the natural next step is -`brew services start dolt`. However, the Homebrew formula runs -`dolt sql-server` without the `--config` flag, and Dolt does not auto-discover -`config.yaml` from its working directory. The config file must be passed -explicitly with `--config `. - -#### Setup with a Custom LaunchAgent - -Install Dolt and initialize its data directory: - -```bash -brew install dolt -cd /opt/homebrew/var/dolt && dolt init -``` - -Configure Dolt for port 3307: - -```yaml -# /opt/homebrew/var/dolt/config.yaml -log_level: info - -listener: - host: 127.0.0.1 - port: 3307 - max_connections: 100 - -behavior: - autocommit: true -``` - -Create the LaunchAgent plist: - -```bash -cat > ~/Library/LaunchAgents/com.local.dolt-server.plist << 'EOF' - - - - - Label - com.local.dolt-server - ProgramArguments - - /opt/homebrew/bin/dolt - sql-server - --config - /opt/homebrew/var/dolt/config.yaml - - WorkingDirectory - /opt/homebrew/var/dolt - RunAtLoad - - KeepAlive - - StandardOutPath - /opt/homebrew/var/log/dolt.log - StandardErrorPath - /opt/homebrew/var/log/dolt-error.log - - -EOF -``` - -Load and verify the service: - -```bash -launchctl load ~/Library/LaunchAgents/com.local.dolt-server.plist -mysql -h 127.0.0.1 -P 3307 -u root -e "SELECT 1" -``` - -Point beads at the central server: - -```bash -export BEADS_DOLT_SERVER_MODE=1 -export BEADS_DOLT_SERVER_PORT=3307 -``` - -Manage the service: - -```bash -# Stop -launchctl unload ~/Library/LaunchAgents/com.local.dolt-server.plist - -# Restart -launchctl unload ~/Library/LaunchAgents/com.local.dolt-server.plist -launchctl load ~/Library/LaunchAgents/com.local.dolt-server.plist - -# Check logs -tail -f /opt/homebrew/var/log/dolt.log -``` - -## Advanced Dolt Usage - -The `dolt` CLI lets you operate directly on the database for power-user -workflows. The data directory depends on your mode: `.beads/embeddeddolt/` -(embedded) or `.beads/dolt/` (server). - -### Branching - -```bash -cd .beads/dolt # or .beads/embeddeddolt for embedded mode -dolt branch feature-x -dolt checkout feature-x -``` - -### Time Travel - -```bash -dolt log -dolt checkout -dolt sql -q "SELECT * FROM issues" -``` - -### Diff and Blame - -```bash -dolt diff main feature-x -dolt blame issues -``` - -## Migration Cleanup - -After successful migration from SQLite, you may have backup files: - -``` -.beads/beads.backup-pre-dolt-20260122-213600.db -.beads/sqlite.backup-pre-dolt-20260123-192812.db -``` - -These are safe to delete once you've verified Dolt is working: - -```bash -# Verify Dolt works -bd list -bd doctor - -# Then clean up (after appropriate waiting period) -rm .beads/*.backup-*.db -``` - -**Recommendation:** Keep backups for at least a week before deleting. - -## See Also - -- [SYNC_CONCEPTS.md](SYNC_CONCEPTS.md) - The conceptual model behind cross-machine sync (Dolt source of truth, wire format, anti-patterns) -- [SYNC_SETUP.md](SYNC_SETUP.md) - Setting up sync across multiple computers -- [FEDERATION.md](FEDERATION.md) - Peer-to-peer federation setup -- [CONFIG.md](CONFIG.md) - Full configuration reference -- [DEPENDENCIES.md](DEPENDENCIES.md) - Dependencies and gates -- [GIT_INTEGRATION.md](GIT_INTEGRATION.md) - Git worktrees and protected branches -- [TROUBLESHOOTING.md](TROUBLESHOOTING.md) - General troubleshooting +This guide moved to [architecture/dolt.md](architecture/dolt.md) as part of the +Mintlify docs site under `docs/`. This pointer stays because released `bd` +binaries and older links reference `docs/DOLT.md` (including the +`#standalone-to-managed-city-handoff` runbook, now at +[architecture/dolt.md](architecture/dolt.md#standalone-to-managed-city-handoff)). diff --git a/docs/JSON_SCHEMA.md b/docs/JSON_SCHEMA.md index 3854b8a2c..dab0adde8 100644 --- a/docs/JSON_SCHEMA.md +++ b/docs/JSON_SCHEMA.md @@ -1,205 +1,6 @@ # JSON Output Schema Contract -Last reviewed: 2026-05-08 - -Freshness source: `cmd/bd/output.go`, `cmd/bd/errors.go`, and -`cmd/bd/protocol/json_contract_test.go`. - -All `bd` commands that support `--json` output can wrap their response in -a uniform envelope by setting `BD_JSON_ENVELOPE=1`. This will become the -default format in v2.0. - -## Migration Guide - -### Opt in to the envelope format - -```bash -export BD_JSON_ENVELOPE=1 -``` - -### Envelope format (BD_JSON_ENVELOPE=1, default in v2.0) - -Every `--json` command wraps output as: - -```json -{"schema_version": 1, "data": } -``` - -The original payload is untouched inside `.data` — no type corruption, -no field injection. Works identically for objects, arrays, and maps. - -### Updating consumers - -```bash -# Before (legacy): -bd list --json | jq '.[0].id' -bd show beads-abc --json | jq '.title' - -# After (envelope): -bd list --json | jq '.data[0].id' -bd show beads-abc --json | jq '.data.title' - -# Version check: -bd show beads-abc --json | jq '.schema_version' -``` - -### Timeline - -- **Current release**: Legacy format is default. Set `BD_JSON_ENVELOPE=1` to opt in. - A deprecation notice is printed to stderr when `--json` is used without the env var. -- **v2.0**: Envelope becomes the default. `BD_JSON_ENVELOPE=0` available as - temporary escape hatch for one release cycle. - -## Schema Version - -Current version: **1** - -The `schema_version` field is an integer that increments when: -- Fields are added, renamed, or removed -- Output structure changes (e.g., nesting depth) -- Field types change (e.g., string to integer) - -Additive changes (new optional fields) do NOT bump the version. - -## Output Formats - -### Envelope mode (BD_JSON_ENVELOPE=1) - -All commands emit a uniform envelope: - -```json -{ - "schema_version": 1, - "data": { - "id": "beads-abc", - "title": "Example issue", - "status": "open" - } -} -``` - -Arrays are wrapped the same way: - -```json -{ - "schema_version": 1, - "data": [ - {"id": "beads-abc", "title": "First"}, - {"id": "beads-def", "title": "Second"} - ] -} -``` - -### Legacy mode (default, until v2.0) - -### Object commands (show, create, close, update, etc.) - -Commands that return a single issue or result emit a JSON object with -`schema_version` as a top-level field alongside the data: - -```json -{ - "schema_version": 1, - "id": "beads-abc", - "title": "Example issue", - "status": "open", - "priority": 1, - "issue_type": "task", - "created_at": "2026-04-20T12:00:00Z" -} -``` - -### List commands (list, ready, search, stale, etc.) - -Commands that return multiple items emit a raw JSON array: - -```json -[ - {"id": "beads-abc", "title": "First", ...}, - {"id": "beads-def", "title": "Second", ...} -] -``` - -### Error output (stderr) - -Errors with `--json` active emit JSON to stderr: - -```json -{ - "schema_version": 1, - "error": "issue not found: beads-xyz", - "code": "not_found" -} -``` - -## Field Contracts by Command - -### bd list --json - -Required fields per item: -- `id` (string): Issue ID (e.g., "beads-abc") -- `title` (string): Issue title -- `status` (string): open, in_progress, closed, deferred -- `priority` (number): 0-4 -- `issue_type` (string): bug, feature, task, epic, chore -- `created_at` (string): RFC3339 timestamp - -Optional fields: -- `description`, `owner`, `updated_at`, `closed_at` -- `labels` (string[]): Attached labels -- `dependencies` (object[]): Dependency records -- `dependency_count`, `dependent_count`, `comment_count` (number) -- `parent` (string|null): Parent issue ID - -### bd ready --json - -Same schema as `bd list --json`. Items are filtered to unblocked issues only. -Each item includes `dependency_count`, `dependent_count`, `comment_count`, -and optional `parent` fields. - -### bd blocked --json - -Returns issues that are blocked by unresolved dependencies. -Each item includes all standard issue fields plus: -- `blocked_by_count` (number): Number of blocking dependencies -- `blocked_by` (string[]): IDs of blocking issues - -### bd show --json - -Returns a single object (not wrapped in `items`). Same required fields as list -items, plus: -- `description` (string) -- `acceptance_criteria` (string) -- `dependencies` (object[]): Full dependency records -- `comments` (object[]): Comment thread - -### `import --json` - -Returns a summary object when `--json` is active: -- `source` (string): File path or "stdin" -- `created` (number): Issues created -- `skipped` (number): Issues skipped (dedup) -- `dedup_skipped` (number): Issues skipped by `--dedup` title match -- `memories` (number): Memory records imported -- `ids` (string[]): IDs of created issues -- `dry_run` (boolean): Whether `--dry-run` was active - -### bd export --json - -Outputs JSONL (one JSON object per line), not wrapped in an envelope. -Each line is a self-contained issue or memory record. `schema_version` -is included per line. - -## Consumer Guidelines - -1. **Check `schema_version`** on object output. If the version is - higher than expected, log a warning but attempt to parse anyway - (additive changes are backward-compatible). - -2. **For list commands**, parse the output as a JSON array directly. - -3. **Ignore unknown fields**. New fields may be added without bumping - the schema version. - -4. **Use `--json` flag**, not `--format json`. The `--json` flag is - the stable contract; `--format` is for human-readable variants. +This reference moved to [reference/json-schema.md](reference/json-schema.md) as +part of the Mintlify docs site under `docs/`. This pointer stays because +released `bd` binaries reference `docs/JSON_SCHEMA.md` in schema-migration +messages. diff --git a/docs/PLUGIN.md b/docs/PLUGIN.md index ed6d04554..937648a7c 100644 --- a/docs/PLUGIN.md +++ b/docs/PLUGIN.md @@ -1,390 +1,6 @@ # Beads Claude Code Plugin -AI-supervised issue tracker for coding workflows. Manage tasks, discover work, and maintain context with simple slash commands and MCP tools. - -## What is Beads? - -Beads (`bd`) is an issue tracker designed specifically for AI-supervised coding workflows. It helps AI agents and developers: -- Track work with a simple CLI -- Discover and link related tasks during development -- Maintain context across coding sessions -- Sync issues via Dolt remotes for distributed workflows - -## Installation - -### Prerequisites - -1. Install beads CLI: -```bash -curl -sSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash -``` - -2. Install Python and uv (for MCP server): -```bash -# Install uv (if not already installed) -curl -LsSf https://astral.sh/uv/install.sh | sh -``` - -### Install Plugin - -There are two ways to install the beads plugin: - -#### Option 1: From GitHub (Recommended) - -```bash -# In Claude Code -/plugin marketplace add gastownhall/beads -/plugin install beads -``` - -#### Option 2: Local Development - -```bash -# Clone the repository (shell command) -git clone https://github.com/gastownhall/beads -cd beads -``` - -Then in Claude Code: - -``` -# Add local marketplace (Claude Code command) -/plugin marketplace add ./beads - -# Install plugin -/plugin install beads -``` - -**Note:** If you want to install the plugin from a different repo, first `cd` to that repo's directory in your terminal, then use `./beads` (or the relative path to the beads directory) in Claude Code. - -### Restart Claude Code - -After installation, restart Claude Code to activate the MCP server. - -## Quick Start - -```bash -# Initialize beads in your project -/beads:init - -# Create your first issue -/beads:create "Set up project structure" feature 1 - -# See what's ready to work on -/beads:ready - -# Show full workflow guide -/beads:workflow -``` - -## Available Commands - -### Version Management - -- **`/beads:version`** - Check bd CLI, plugin, and MCP server versions - -### Core Workflow Commands - -- **`/beads:ready`** - Find tasks with no blockers, ready to work on -- **`/beads:create [title] [type] [priority]`** - Create a new issue interactively -- **`/beads:show [issue-id]`** - Show detailed information about an issue -- **`/beads:update [issue-id] [status]`** - Update issue status or other fields -- **`/beads:close [issue-id] [reason]`** - Close a completed issue - -### Project Management - -- **`/beads:init`** - Initialize beads in the current project -- **`/beads:workflow`** - Show the AI-supervised issue workflow guide -- **`/beads:stats`** - Show project statistics and progress - -### Agents - -- **`@task-agent`** - Autonomous agent that finds and completes ready tasks - -## MCP Tools Available - -The plugin includes a full-featured MCP server with these tools: - -- **`init`** - Initialize bd in current directory -- **`create`** - Create new issue (bug, feature, task, epic, chore) -- **`list`** - List issues with filters (status, priority, type, assignee) -- **`ready`** - Find tasks with no blockers ready to work on -- **`show`** - Show detailed issue info including dependencies -- **`update`** - Update issue (status, priority, design, notes, etc) -- **`close`** - Close completed issue -- **`dep`** - Add dependency (blocks, related, parent-child, discovered-from) -- **`blocked`** - Get blocked issues -- **`stats`** - Get project statistics - -### MCP Resources - -- **`beads://quickstart`** - Interactive quickstart guide - -## Workflow - -The beads workflow is designed for AI agents but works great for humans too: - -1. **Find ready work**: `/beads:ready` -2. **Claim your task**: `/beads:update in_progress` -3. **Work on it**: Implement, test, document -4. **Discover new work**: Create issues for bugs/TODOs found during work -5. **Complete**: `/beads:close "Done: "` -6. **Repeat**: Check for newly unblocked tasks - -## Issue Types - -- **`bug`** - Something broken that needs fixing -- **`feature`** - New functionality -- **`task`** - Work item (tests, docs, refactoring) -- **`epic`** - Large feature composed of multiple issues -- **`chore`** - Maintenance work (dependencies, tooling) - -## Priority Levels - -- **`0`** - Critical (security, data loss, broken builds) -- **`1`** - High (major features, important bugs) -- **`2`** - Medium (nice-to-have features, minor bugs) -- **`3`** - Low (polish, optimization) -- **`4`** - Backlog (future ideas) - -## Dependency Types - -- **`blocks`** - Hard dependency (issue X blocks issue Y from starting) -- **`related`** - Soft relationship (issues are connected) -- **`parent-child`** - Epic/subtask relationship -- **`discovered-from`** - Track issues discovered during work - -Only `blocks` dependencies affect the ready work queue. - -## Configuration - -### Auto-Approval Configuration - -By default, Claude Code asks for confirmation every time the beads MCP server wants to run a command. This is a security feature, but it can disrupt workflow during active development. - -**Available Options:** - -#### 1. Auto-Approve All Beads Tools (Recommended for Trusted Projects) - -Add to your Claude Code `settings.json`: - -```json -{ - "enabledMcpjsonServers": ["beads"] -} -``` - -This auto-approves all beads commands without prompting. - -#### 2. Auto-Approve Project MCP Servers - -Add to your Claude Code `settings.json`: - -```json -{ - "enableAllProjectMcpServers": true -} -``` - -This auto-approves all MCP servers defined in your project's `.mcp.json` file. Useful when working across multiple projects with different MCP requirements. - -#### 3. Manual Approval (Default) - -No configuration needed. Claude Code will prompt for approval on each MCP tool invocation. - -**Security Trade-offs:** - -- **Manual approval (default)**: Maximum safety, but interrupts workflow frequently -- **Server-level auto-approval**: Convenient for trusted projects, but allows any beads operation without confirmation -- **Project-level auto-approval**: Good balance for multi-project workflows with project-specific trust levels - -**Limitation:** Claude Code doesn't currently support per-tool approval granularity. You cannot auto-approve only read operations (like `bd ready`, `bd show`) while requiring confirmation for mutations (like `bd create`, `bd update`). It's all-or-nothing at the server level. - -**Recommended Configuration:** - -For active development on trusted projects where you're frequently using beads: - -```json -{ - "enabledMcpjsonServers": ["beads"] -} -``` - -For more information, see the [Claude Code settings documentation](https://docs.claude.com/en/docs/claude-code/settings). - -### Environment Variables - -The MCP server supports these environment variables: - -- **`BEADS_PATH`** - Path to bd executable (default: `bd` in PATH) -- **`BEADS_DB`** - Path to beads database file (default: auto-discover from cwd) -- **`BEADS_ACTOR`** - Actor name for audit trail (default: `$USER`) -- **`BEADS_NO_AUTO_FLUSH`** - Disable automatic sync (default: `false`) -- **`BEADS_NO_AUTO_IMPORT`** - Disable automatic import (default: `false`) - -To customize, edit your Claude Code MCP settings or the plugin configuration. - -## Examples - -### Basic Task Management - -```bash -# Create a high-priority bug -/beads:create "Fix authentication" bug 1 - -# See ready work -/beads:ready - -# Start working on bd-10 -/beads:update bd-10 in_progress - -# Complete the task -/beads:close bd-10 "Fixed auth token validation" -``` - -### Discovering Work During Development - -```bash -# Working on bd-10, found a related bug -/beads:create "Add rate limiting to API" feature 2 - -# Link it to current work (via MCP tool) -# Use `dep` tool: issue="bd-11", depends_on="bd-10", type="discovered-from" - -# Close original task -/beads:close bd-10 "Done, discovered bd-11 for rate limiting" -``` - -### Using the Task Agent - -```bash -# Let the agent find and complete ready work -@task-agent - -# The agent will: -# 1. Find ready work with `ready` tool -# 2. Claim a task by updating status -# 3. Execute the work -# 4. Create issues for discoveries -# 5. Close when complete -# 6. Repeat -``` - -## Auto-Sync with Dolt - -Beads automatically commits changes to Dolt history after every write operation. This enables seamless collaboration: - -```bash -# Make changes -bd create "Add feature" -p 1 - -# Changes are automatically committed to Dolt history -# Sync with remotes when ready: -bd dolt push - -# Pull changes from collaborators: -bd dolt pull -bd ready # Shows issues ready to work on (with fresh data) -``` - -## Updating - -The beads plugin has three components that may need updating: - -### 1. Plugin Updates - -Check for plugin updates: -```bash -/plugin update beads -``` - -Claude Code will pull the latest version from GitHub. After updating, **restart Claude Code** to apply MCP server changes. - -### 2. bd CLI Updates - -The plugin requires the `bd` CLI to be installed. Update it separately: - -```bash -# Quick update -curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash - -# Or with Go (server-mode only) -CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest - -# Or with Go (embedded-capable) -CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest -``` - -### 3. Version Compatibility - -The MCP server **automatically checks** bd CLI version on startup and will fail with a clear error if your version is too old. - -Check version compatibility manually: -```bash -/beads:version -``` - -This will show: -- bd CLI version -- Plugin version -- MCP server status -- Compatibility warnings if versions mismatch - -**Recommended update workflow:** -1. Check versions: `/beads:version` -2. Update bd CLI if needed (see above) -3. Update plugin: `/plugin update beads` -4. Restart Claude Code -5. Verify: `/beads:version` - -### Version Numbering - -Beads follows semantic versioning. The plugin version tracks the bd CLI version: -- Plugin 0.9.2 requires bd CLI >= 0.9.0 (checked automatically at startup) -- Major version bumps may introduce breaking changes -- Check CHANGELOG.md for release notes - -## Troubleshooting - -### Plugin not appearing - -1. Check installation: `/plugin list` -2. Restart Claude Code -3. Verify `bd` is in PATH: `which bd` -4. Check uv is installed: `which uv` - -### MCP server not connecting - -1. Check MCP server list: `/mcp` -2. Look for "beads" server with plugin indicator -3. Restart Claude Code to reload MCP servers -4. Check logs for errors - -### Commands not working - -1. Make sure you're in a project with beads initialized: `/beads:init` -2. Check if database exists: `ls -la .beads/` -3. Try direct MCP tool access instead of slash commands -4. Check the beads CLI works: `bd --help` - -### MCP tool errors - -1. Verify `bd` executable location: `BEADS_PATH` env var -2. Check `bd` works in terminal: `bd stats` -3. Review MCP server logs in Claude Code -4. Try reinitializing: `/beads:init` - -## Learn More - -- **GitHub**: https://github.com/gastownhall/beads -- **Documentation**: See README.md in the repository -- **Examples**: Check `examples/` directory for integration patterns -- **MCP Server**: See `integrations/beads-mcp/` for server details - -## Contributing - -Found a bug or have a feature idea? Create an issue in the beads repository! - -## License - -MIT License - see LICENSE file in the repository. +This guide moved to [integrations/claude-code-plugin.md](integrations/claude-code-plugin.md) +as part of the Mintlify docs site under `docs/`. This pointer stays because +released `bd` binaries reference `docs/PLUGIN.md` (for example in `bd doctor` +fix suggestions). diff --git a/docs/RECOVERY.md b/docs/RECOVERY.md index 145b16a35..0b887717b 100644 --- a/docs/RECOVERY.md +++ b/docs/RECOVERY.md @@ -1,251 +1,11 @@ # Recovery Playbooks -Last reviewed: 2026-06-09 - -Freshness source: `cmd/bd/init.go`, `cmd/bd/init_safety.go`, -`cmd/bd/init_safety_test.go`, and `cmd/bd/dolt.go`. - -This document lives next to the ADRs and matches the structure of `bd`'s -error messages: each named refusal in `bd init` and `bd dolt push`/`pull` -points here to a labeled anchor with step-by-step recovery instructions. - -See also: `bd help init-safety`, and -[ADR 0002 — `bd init` safety invariants](../engdocs/adr/0002-init-safety-invariants.md). - -## Table of contents - -- [init-force-refused — `bd init --force`/`--reinit-local` refused because origin has Dolt history](#init-force-refused) -- [init-token-missing — `--discard-remote` refused because `--destroy-token` is missing or wrong](#init-token-missing) -- [init-local-exists — `bd init` refused because local data already exists](#init-local-exists) -- [pk-fork-refused — `bd dolt pull`/`push` refused because a table has different primary keys in its common ancestor](#pk-fork-refused) - ---- - -## init-force-refused - -**Exit code:** `10` (`ExitRemoteDivergenceRefused`) - -**Symptom** - -``` -bd init refuses: remote 'origin' already has Dolt history (refs/dolt/data). - Why: this init mode would create or reuse local history instead of - adopting the remote. ... -``` - -**Why this happens** - -`bd init --force` (or `--reinit-local`) tells `bd` to bypass the local -data-safety guard. `bd init --from-jsonl` selects a local JSONL export as -the source. But the remote already has project history. Proceeding would -create an orphan local Dolt branch with no common ancestor on origin. The -next `bd dolt push` would either fail (no common ancestor) or — worse, if -force-pushed — destroy the team's data. - -**Recovery paths** - -Pick the one that matches your intent. - -### 1. You want to adopt the remote's history (most common) - -``` -bd bootstrap -``` - -This clones the remote's Dolt database into a fresh local `.beads/`. -Your local state is ignored; the team's history becomes yours. - -### 2. You want to diagnose what went wrong before deciding - -``` -bd doctor -bd dolt status -``` - -`bd doctor` walks the local + remote state and names concrete problems. -`bd dolt status` shows the Dolt-level view. Neither modifies anything. - -### 3. You intentionally want to overwrite the remote's history (destructive) - -This is a cross-boundary operation that affects every collaborator. You -need to pair the local-source init (`--reinit-local` or `--from-jsonl`) -with `--discard-remote`. In interactive mode `bd` will prompt for -confirmation; in non-interactive mode you must supply a `--destroy-token`. -See `bd help init-safety` for the token format. - -After `bd init --reinit-local --discard-remote`, your next -`bd dolt push` must be a history-replacing push. Coordinate with your -team before doing this. - ---- - -## init-token-missing - -**Exit code:** `12` (`ExitDestroyTokenMissing`) - -**Symptom** - -``` -bd init refuses: --discard-remote requires an explicit destroy-token in non-interactive mode. -``` - -**Why this happens** - -You're running non-interactively (CI, agent, piped input) and passed -`--discard-remote`. Destructive cross-boundary operations cannot be -authorized silently. - -**Recovery paths** - -### 1. Run interactively - -Re-run in a TTY. `bd init --reinit-local --discard-remote` will prompt -you to type the destroy-token at confirmation time. - -### 2. Supply the token explicitly (CI/automation) - -The token format is `DESTROY-`. For a project whose issue -prefix is `bd`: - -``` -bd init --reinit-local --discard-remote --destroy-token=DESTROY-bd -``` - -Automation should template the token from project state, not from error -output. See [ADR 0002 — Invariant 4](../engdocs/adr/0002-init-safety-invariants.md) -for why the token is never echoed in `bd`'s error messages. - ---- - -## init-local-exists - -**Exit code:** `11` (`ExitLocalExistsRefused`) - -**Symptom** - -``` -Refusing to destroy N issues in non-interactive mode. - See 'bd help init-safety' for the required --destroy-token format. -``` - -Or, in interactive mode, you declined the typed `destroy N issues` -confirmation. - -**Why this happens** - -Local `.beads/` has existing issues. `bd init --reinit-local` would -permanently destroy them. - -**Recovery paths** - -### 1. Export first, then proceed - -``` -bd export > issue-export.jsonl -bd init --reinit-local -``` - -`issue-export.jsonl` lets you re-import individual issues if needed. It is not -a full database backup; use `bd backup` when the Dolt database is healthy -enough to create a restorable backup before reinitializing. - -### 2. Investigate why you hit this - -If you did NOT expect `bd init` to be the right command here, run -`bd doctor` first — you may be looking at a server config issue that a -re-init won't fix. - ---- - -## pk-fork-refused - -**Symptom** - -``` -$ bd dolt pull -Error: ... cannot merge because table dependencies has different primary keys in its common ancestor -``` - -(or the variant without `in its common ancestor`). `bd` follows the error -with a short version of the recovery recipe below. - -**Why this happens** - -The two histories being merged disagree about a table's *primary key set* — -not about row contents. Dolt can cell-merge rows, but it refuses outright to -merge a table whose primary key was reshaped differently on each side (or -whose common ancestor had a different primary key than both sides). The -refusal happens before any row conflicts materialize, so `bd dolt pull`'s -conflict auto-resolver never gets a chance to run. **Retrying never helps**: -the histories are permanently un-mergeable. - -The usual cause is upgrading `bd` independently on two clones while un-synced -changes existed on both sides, across a release whose schema migrations -reshape a primary key. Concretely: the -[#4259](https://github.com/gastownhall/beads/issues/4259) incident — clones -straddling the `0041`/`0043`/`0050` reshapes of `dependencies` (v1.0.4 → -v1.0.6) hit exactly this on the first post-upgrade pull if both sides had -unpushed dependency edits. - -The remote-migrate prevention gate (v1.0.6+) exists to stop this from being -created: it refuses to auto-migrate a remote-backed database and tells you to -designate a single migrator. This playbook is for when the fork already -exists. - -**Recovery: bootstrap from one canonical clone** - -The forked histories cannot be merged, so one side must be chosen as -canonical and every other clone re-cloned from it. Issue *data* survives via -JSONL export/import; only the un-mergeable Dolt *history* is discarded on the -non-canonical clones. - -### 1. Pick the canonical clone - -Usually the most complete / most recently active clone. To compare, run on -each clone (read-only): - -``` -bd stats -bd dolt status -``` - -### 2. On the canonical clone: upgrade, migrate, force-push - -``` -bd version # confirm the new bd binary -bd doctor # sanity-check before publishing -bd dolt push --force # make the remote authoritative -``` - -(`bd`'s migration gate may block here; run `bd migrate --force` on the -canonical clone — that is exactly the designated-migrator case the gate is -asking about. `BD_ALLOW_REMOTE_MIGRATE=1` is the env-var equivalent for -scripted use.) - -### 3. On EVERY other clone: save local-only work, re-clone, re-apply - -``` -bd export --all -o /tmp/beads-local.jsonl # safety net for un-synced work -rm -rf .beads/dolt # discard the un-mergeable history -bd bootstrap # re-clone from the remote -bd import /tmp/beads-local.jsonl # re-apply local-only work -``` - -`bd import` has upsert semantics: issues that only existed on this clone are -re-created, newer local edits are applied, and rows older than what the -remote already has are skipped. Spot-check with `bd stats` afterwards. - -### Prevention (upgrades across PK-reshaping migrations) - -- **Sync before upgrading**: `bd dolt push` + `bd dolt pull` on every clone - while all clones still run the *old* version, then stop editing. Once the new - binary is installed, `bd dolt push`/`bd dolt pull` are gated too, so this must - happen first. -- **One designated migrator**: upgrade one machine, let it migrate, then - `bd dolt push`. -- **Every other clone adopts, does not pull**: after the migrator pushes, each - other clone upgrades the binary and runs `bd bootstrap` to adopt the migrated - database. `bd dolt pull` is *refused* while the clone still has pending - migrations, so do not rely on it; the "sync before" step above is what - preserves these clones' work, because `bd bootstrap` replaces the local - database. +The pattern-coded recovery playbooks moved to +[recovery/init-safety.md](recovery/init-safety.md) as part of the Mintlify docs +site under `docs/`. This pointer stays because released `bd` binaries link to +`docs/RECOVERY.md` runbooks by anchor: + +- `#init-force-refused` → [recovery/init-safety.md](recovery/init-safety.md#init-force-refused) +- `#init-token-missing` → [recovery/init-safety.md](recovery/init-safety.md#init-token-missing) +- `#init-local-exists` → [recovery/init-safety.md](recovery/init-safety.md#init-local-exists) +- `#pk-fork-refused` → [recovery/init-safety.md](recovery/init-safety.md#pk-fork-refused) diff --git a/docs/STORAGE-BACKENDS.md b/docs/STORAGE-BACKENDS.md index d12c82eb0..dae57a768 100644 --- a/docs/STORAGE-BACKENDS.md +++ b/docs/STORAGE-BACKENDS.md @@ -1,277 +1,6 @@ # Storage Backends -Beads uses Dolt as its default storage backend, and Dolt remains the reference implementation. You can also point `bd` at Postgres, MySQL, or SQLite — every backend implements the full issue-tracking core, and the only thing you give up off Dolt is version control. - -One seam, one shared core: the issue-tracking semantics are written once (in `issueops`) and shared by *every* backend — Dolt runs them natively, the SQL family runs them through one thin per-engine dialect. Each backend is a store adapter over that shared core. Version control lives only in the Dolt store; on the SQL backends it returns one clean, typed error. - -```mermaid -flowchart TD - CLI["bd CLI"] --> SEAM["Storage interface
one seam — every backend satisfies all of it"] - - SEAM --> CORE["CORE — issueops
CRUD · search · deps · ready/claim · ordering · id-mint
written once, shared by EVERY backend"] - SEAM --> CAPS["CAPABILITIES — version control
history · branch · remotes · sync"] - - CORE --> DS["Dolt store
runs the core natively"] - CORE --> KIT["sqlkit.Store
SQL-family adapter · Dialect / Readiness / Claim"] - - DS --> DOLT[("Dolt — embedded / server
the conformance ORACLE")] - KIT --> PGD["pgdialect"] --> PG[("Postgres")] - KIT --> MYD["mysqldialect"] --> MY[("MySQL")] - KIT --> LTD["sqlitedialect"] --> LT[("SQLite")] - - CAPS -->|"implemented in the Dolt store"| DS - CAPS -->|"generated typed-unsupported shell"| SHELL["Postgres / MySQL / SQLite
operation X not supported"] - - DOLT -. "conformance-tested: SQL family matches Dolt byte-for-byte" .-> KIT -``` - -
-Plain-text version of the diagram - -``` - bd CLI - | - Storage interface - | - +-------------------------+--------------------------+ - | | - CORE (every backend) CAPABILITIES (Dolt-only) - issueops -- CRUD, search, deps, version control: - ready/claim, ordering, id-mint, history, branch, remotes, sync - cycle checks: written ONCE | - | +-> Dolt store: implemented - +--------------------+ +-> Postgres/MySQL/SQLite: - | | generated typed-"unsupported" - Dolt store sqlkit.Store (operation "X" not supported) - runs the core SQL-family adapter - natively (Dialect/Readiness/Claim) - | | - [Dolt engine] +------+------+---------+ - embedded/server pgdialect mysqldialect sqlitedialect - = conformance | | | - ORACLE Postgres MySQL SQLite - - issueops is shared by EVERY backend (Dolt included); each backend is a store - adapter over it. Dolt runs the core's SQL natively and adds version control; - the SQL family runs it through one thin dialect per engine, and is - conformance-tested to match the Dolt oracle byte-for-byte. -``` - -
- -## Why Choose a Backend? - -- **Dolt stays the default** — `bd init` with no flags behaves exactly as before, and Dolt is the only backend with history, branching, and sync -- **Postgres / MySQL** — put your issues in the database your team already runs, backs up, and monitors; many writers, one server -- **SQLite** — a single file inside `.beads/`, zero servers, zero credentials, pure-Go driver -- **Same `bd` everywhere** — create, list, ready, deps, claims, labels, comments, search all behave identically on every backend; conformance tests enforce it against the Dolt reference -- **Clean degradation** — Dolt-only commands don't half-work on SQL backends; they return one clear, typed error - -## Pick Your Backend - -| Backend | Best for | History | Server needed | Credentials | -|---|---|---|---|---| -| `dolt` (default) | Solo or team work where you want issue history, branching, `bd dolt push`/`pull` | Yes | No (embedded) / optional | Only in server mode | -| `sqlite` | Solo work, throwaway workspaces, CI sandboxes, air-gapped machines | No | No | None | -| `postgres` | Teams with an existing Postgres; many workspaces share one database via per-workspace schemas | No | Yes | Ladder (below) | -| `mysql` | Teams with an existing MySQL; each workspace gets its own database | No | Yes | Ladder (below) | - -Rules of thumb: - -- If you want `bd history`, `bd dolt push`, time-travel, or federation — use **Dolt**. It is the only backend that tracks history. -- If you want the smallest possible footprint and don't need history — use **SQLite**. -- If your organization already operates Postgres or MySQL and you want issues living next to everything else your DBAs manage — use **Postgres** or **MySQL**. Multi-writer concurrency comes from the database server itself. -- If in doubt, use the default. `bd init` without `--backend` is Dolt, unchanged. - -## Getting Started - -The backend is chosen once, at `bd init` time, and recorded in `.beads/metadata.json`. Every later `bd` command reads it from there — no per-command flags. - -### Dolt (default) - -```bash -bd init --prefix myproj -``` - -Nothing changes. See [DOLT.md](DOLT.md) for embedded vs. server mode. - -### SQLite - -```bash -bd init --backend=sqlite --prefix myproj - -# Optional: choose the database file (relative to .beads/; default beads.db) -bd init --backend=sqlite --sqlite-path=issues.db --prefix myproj -``` - -That's it — no server, no credentials. Foreign keys and immediate-transaction locking are configured automatically. - -### Postgres - -Each Beads workspace lives in its own Postgres **schema**, so many workspaces can share one database. - -```bash -bd init --backend=postgres \ - --pg-url='postgres://bd:onlyforinit@db.example.com:5432/beads?sslmode=require' \ - --pg-schema=myproj \ - --prefix myproj -``` - -- `--pg-url` — a standard `postgres://` URL. If omitted, `bd init` falls back to the `BEADS_POSTGRES_URL` environment variable. -- `--pg-schema` — the schema this workspace owns; created and pinned via `search_path`. - -> **Note:** `bd init` connects with the URL exactly as given, so embed the password -> in `--pg-url` for init. It is stripped before anything is written to disk — -> `.beads/metadata.json` stores only the password-free DSN — and every later -> command resolves the password through the credential ladder below. - -```bash -# For all subsequent commands, supply the password via any ladder rung, e.g.: -export BEADS_PG_PASSWORD_COMMAND='vault kv get -field=password secret/beads/pg' - -bd create "first issue" -p 1 -bd ready -``` - -### MySQL - -Each Beads workspace gets its own MySQL **database**. - -```bash -bd init --backend=mysql \ - --mysql-url='bd:onlyforinit@tcp(db.example.com:3306)/' \ - --mysql-database=myproj_beads \ - --prefix myproj -``` - -- `--mysql-url` — go-sql-driver DSN grammar: `user:password@tcp(host:port)/` (note the trailing slash, no scheme). Falls back to `BEADS_MYSQL_URL` if omitted. -- `--mysql-database` — the database this workspace owns. - -Same rule as Postgres: embed the password in `--mysql-url` for init only; it is redacted before persisting, and later commands use the ladder: - -```bash -export BEADS_MYSQL_PASSWORD_COMMAND='vault kv get -field=password secret/beads/mysql' -# or a static password: -export BEADS_MYSQL_PASSWORD=... -``` - -> **Note:** the MySQL DSN grammar cannot carry a password without a username. -> `bd` refuses such DSNs loudly rather than silently connecting passwordless. - -## Credentials & Override Layers - -SQLite needs no credentials. For Postgres and MySQL, the password is resolved fresh at open time by a **ladder** of sources, highest priority first. The first *configured* rung wins, and a configured rung that fails aborts the connection — it never silently falls through to a lower rung. - -Precedence, per backend: - -1. **Password embedded in the DSN** — a password inside `BEADS_POSTGRES_URL` / `BEADS_MYSQL_URL` wins outright; no ladder runs. -2. **Credential command** — `BEADS_PG_PASSWORD_COMMAND` / `BEADS_MYSQL_PASSWORD_COMMAND` -3. **Static env var** — `BEADS_PG_PASSWORD` / `BEADS_MYSQL_PASSWORD` -4. **Credentials file** — an entry matching the DSN's `host:port` -5. **Driver-native fallback** — Postgres only: `PGPASSWORD`, `~/.pgpass`, `PGPASSFILE` still work when nothing above is configured. MySQL has no driver-native fallback; with nothing configured it attempts an empty password. - -> **Note the naming asymmetry:** the URL variables use full names -> (`BEADS_POSTGRES_URL`, `BEADS_MYSQL_URL`) while the password variables -> abbreviate Postgres (`BEADS_PG_PASSWORD`, `BEADS_PG_PASSWORD_COMMAND`). -> There is no `BEADS_PG_URL`. - -### The credential command (rotating secrets) - -This is the same idiom as kubectl's ExecCredential, the AWS CLI's `credential_process`, and git credential helpers: `bd` runs your command with `sh -c` (30-second timeout) and reads the secret from stdout. Output can be a bare token, a kubectl-style JSON envelope (`{"token": "...", "expirationTimestamp": "..."}`), or an OAuth-style one (`{"access_token": "...", "expires_in": 900}`). Results are cached in-process and refreshed shortly before expiry, so repeated opens don't re-spawn your helper. - -```bash -# Vault -export BEADS_PG_PASSWORD_COMMAND="vault kv get -field=password secret/beads/pg" - -# AWS RDS IAM auth (15-minute tokens; caching and expiry are handled for you) -export BEADS_PG_PASSWORD_COMMAND="aws rds generate-db-auth-token \ - --hostname db.example.com --port 5432 --username bd" -``` - -The command variables are read from the environment only — never from workspace metadata — because a persisted command would be arbitrary code executed on every `bd` invocation. - -### The credentials file (static, per-host) - -The lowest configured rung is an INI-style file keyed by `host:port`, at `~/.config/beads/credentials` (Linux/macOS) or `%APPDATA%\beads\credentials` (Windows); override the path with `BEADS_CREDENTIALS_FILE`. - -```ini -# ~/.config/beads/credentials (chmod 600) -[127.0.0.1:5432] -password=localDevPassword - -[db.example.com:3306] -password=teamServerPassword -``` - -If the file is readable by group or others, `bd` prints an ssh-style warning. The file rung is deliberately forgiving: a missing file or unmatched section just means "not configured" and the ladder continues — fail-closed behavior is reserved for rungs you explicitly configured, like a command. - -### Dolt server credentials - -The Dolt backend in server mode has its own rungs: `BEADS_DOLT_PASSWORD`, then the credentials file, with `BEADS_DOLT_SERVER_USER` overriding the username. There is also `BEADS_DOLT_CREDENTIAL_COMMAND`, an *identity* credential: the minted token is presented as the connection username to an authenticating gateway server, which verifies it and routes to the database. This is for gateway deployments only — the direct Postgres/MySQL backends reject `BEADS_PG_CREDENTIAL_COMMAND` / `BEADS_MYSQL_CREDENTIAL_COMMAND` with an error telling you to use the `_PASSWORD_COMMAND` form instead. - -### Secrets never reach disk - -`bd init` redacts the password from your URL before writing `.beads/metadata.json`, using the database driver's own parser, and then re-parses to verify nothing survived — refusing to init rather than persisting a secret it can't prove is gone. Check for yourself: - -```bash -cat .beads/metadata.json -# {"backend":"postgres","postgres_dsn":"postgres://bd@db.example.com:5432/beads?sslmode=require", ...} -``` - -## What You Keep and What You Give Up - -Every backend implements the complete issue-tracking core — the same shared implementation, not a re-derivation per backend: - -- Issue CRUD, bulk creates, close/reopen/delete -- Search, counts, filtering, sorting -- Dependencies, cycle detection, dependency trees -- Ready/blocked work queries, claims and leases -- Labels, comments, the full event/audit trail -- Config, metadata slots, statistics, transactions, streaming iterators - -What is Dolt-only — by design, since these *are* version control: - -- **History & time travel** — `bd history`, as-of queries, diffs -- **Branching & merging** — branch, checkout, merge, conflict resolution -- **Remotes & sync** — `bd dolt push` / `bd dolt pull`, fetch, sync -- **Federation** — peer management -- **Compaction** — snapshot-based memory compaction - -On Postgres, MySQL, and SQLite these operations don't limp along or approximate — each returns one clean, typed error: - -``` -$ bd history myproj-9q9 -Error: failed to get history: operation "History" not supported by the sqlite backend -``` - -`bd init` sets the expectation up front: - -``` -✓ bd initialized with the SQLite backend - History: not tracked (SQLite backend has no version control; use Dolt for history) -``` - -A few practical consequences: - -- Writes on SQL backends are durable the moment the command returns — each write commits its own SQL transaction. (`bd`'s internal commit hooks become harmless no-ops, not errors.) -- The Dolt-only maintenance tail (auto-commit, auto-export, auto-backup, auto-push) is skipped automatically on non-Dolt backends. -- "Sync" on Postgres/MySQL is your database server: everyone connects to the same database, so there is nothing to push or pull. Back up issues with your normal database tooling, or `bd export` for JSONL interoperability. - -## How It Stays Correct - -The SQL backends are held to the embedded-Dolt reference by a conformance harness, run locally and in CI (with real Postgres and MySQL servers) on every pull request: - -- **Tier 1 (in-process):** the same behavior corpus the Dolt reference passes — CRUD, search, dependencies, ready/blocked, claims and leases, plus a ~94-case audit corpus of edge behaviors — runs against every backend. Two extra gates make capability gaps impossible to hide: a completeness test proves the set of unsupported operations exactly matches an audited allowlist, and a contract test calls every one of them to prove it returns the typed error rather than panicking or lying. -- **Tier 2 (end-to-end):** a real `bd` binary runs `bd init --backend=...` and a CLI scenario corpus in an isolated workspace per backend, and the normalized output must be byte-identical to the Dolt reference. Known-divergence entries are visible in the registry and can only shrink — currently there are none. - -```bash -./scripts/conformance.sh -# Postgres/MySQL profiles self-skip unless you point them at a server: -BEADS_PG_TEST_URL="postgres://user:pass@127.0.0.1:5432/beads_test" \ -BEADS_MYSQL_TEST_URL="user:pass@tcp(127.0.0.1:3306)/" \ -./scripts/conformance.sh -``` - -Adding a new backend is one declarative profile entry in `test/conformance/profiles.go` plus a store-factory arm; both tiers pick it up automatically. - -See [DOLT.md](DOLT.md) for the default backend, [CONFIG.md](CONFIG.md) for configuration, and [TROUBLESHOOTING.md](TROUBLESHOOTING.md) if a connection misbehaves. +This guide moved to [architecture/storage-backends.md](architecture/storage-backends.md) +as part of the Mintlify docs site under `docs/`. This pointer stays because +released `bd` binaries reference `docs/STORAGE-BACKENDS.md` (for example in +`bd init --backend` help). diff --git a/docs/architecture/dolt.md b/docs/architecture/dolt.md new file mode 100644 index 000000000..d087e071f --- /dev/null +++ b/docs/architecture/dolt.md @@ -0,0 +1,739 @@ +--- +title: Dolt Backend for Beads +--- + +Beads uses Dolt as its storage backend. Dolt provides a version-controlled SQL database with cell-level merge, native branching, and two deployment modes. + +## Why Dolt? + +- **Native version control** — cell-level diffs and merges, not line-based +- **Multi-writer support** — server mode enables concurrent agents +- **Built-in history** — every write creates a Dolt commit +- **Native branching** — Dolt branches independent of git branches +- **Single-binary option** — embedded mode for solo users (no server needed) + +## Getting Started + +### Install Dolt (Server Mode Only) + +Embedded mode includes everything in the `bd` binary; no separate Dolt install +is needed. Install the standalone `dolt` CLI only when you want to run server +mode or work directly with the database via `dolt sql`. + +```bash +# macOS +brew install dolt + +# Linux +curl -L https://github.com/dolthub/dolt/releases/latest/download/install.sh | bash + +# Verify installation +dolt version +``` + +### New Project + +```bash +# Embedded mode (single writer, no server — default for standalone) +bd init + +# Server mode (multi-writer, e.g. orchestrator) +gt dolt start # Start the Dolt server +bd init --server # Initialize with server mode +``` + +### Migrate from SQLite (Legacy) + +If upgrading from an older version that used SQLite: + +> **Note:** The `bd migrate --to-dolt` command was removed in v0.58.0. +> For pre-0.50 installations with JSONL data, use the migration script: +> +> ```bash +> scripts/migrate-jsonl-to-dolt.sh +> ``` +> +> See [Troubleshooting](/reference/troubleshooting#circuit-breaker-server-appears-down-failing-fast) if you encounter connection errors after migration. + +Migration creates backups automatically. Your original SQLite database is preserved as `beads.backup-pre-dolt-*.db`. + +## Modes of Operation + +### Embedded Mode (Solo / Standalone) + +In-process Dolt engine — no separate server needed. This is the default for +standalone Beads users. The `bd` binary includes everything; just `bd init` and go. + +- Single-writer (one process at a time) +- Data lives in `.beads/embeddeddolt/` alongside your code +- Push to GitHub with `bd dolt push` — code and issues in one repo +- Zero ops: no server, no ports, no PID files + +### Server Mode (Multi-Writer / Orchestrator) + +Connects to a running `dolt sql-server` for multi-client access. + +```bash +# Start the server (orchestrator) +gt dolt start + +# Or manually +cd ~/.dolt-data/beads && dolt sql-server --port 3307 +``` + +```bash +# Initialize in server mode +bd init --server + +# Or switch via environment variable +export BEADS_DOLT_SERVER_MODE=1 +``` + +```yaml +# .beads/config.yaml (server mode settings) +dolt: + mode: server + host: 127.0.0.1 + port: 3307 + user: root +``` + +Switch to server mode when you need: +- Multiple agents writing simultaneously +- Orchestrator multi-rig setups +- Federation with remote peers + +## Migrating Between Backends + +You can migrate data between embedded mode and server mode using `bd backup`. +Both directions preserve full Dolt commit history. + +`bd export` is not a substitute for this flow. JSONL exports contain issue +records from the issues table for migration and interoperability; they do not +capture Dolt branches, full commit history, working-set state, or non-issue +tables. Use `bd backup` or a manual Dolt backup when you need a restorable +database backup. + +### Server → Embedded + +1. **Create a backup from the server-mode project:** + + ```bash + # In the server-mode project directory + bd backup init /path/to/backup-dir + bd backup sync + ``` + +2. **Create a new embedded-mode project and restore:** + + ```bash + mkdir new-project && cd new-project + bd init # creates an embedded-mode project by default + bd backup restore --force /path/to/backup-dir + ``` + + `--force` overwrites the freshly-initialized database with the backup + contents. The restore automatically: + - Updates `metadata.json` to match the restored project identity + - Registers the backup directory for future `bd backup sync` + - Backfills the embedded migration tracker (`schema_migrations`) + +3. **Verify:** + + ```bash + bd list + bd backup status + ``` + +### Embedded → Server + +1. **Create a backup from the embedded-mode project:** + + ```bash + # In the embedded-mode project directory + bd backup init /path/to/backup-dir + bd backup sync + ``` + +2. **Create a new server-mode project and restore:** + + ```bash + mkdir new-project && cd new-project + bd init --server # creates a server-mode project + bd backup restore --force /path/to/backup-dir + ``` + +3. **Verify:** + + ```bash + bd list + bd backup status + ``` + +### Backup Commands Reference + +| Command | Description | +|---------|-------------| +| `bd backup init ` | Register a backup destination (filesystem or DoltHub URL) | +| `bd backup sync` | Push database to the configured backup destination | +| `bd backup restore [path]` | Restore from a backup directory (`--force` to overwrite) | +| `bd backup remove` | Unregister the backup destination | +| `bd backup status` | Show backup configuration and last sync time | + +### Notes + +- Data locations differ between modes: `.beads/embeddeddolt/` (embedded) vs `.beads/dolt/` (server) +- The backup directory is a full Dolt backup, not an `issues.jsonl` export — it can be on a local drive, NAS, or DoltHub +- You can also migrate via Dolt remotes (`bd dolt push` / `bd dolt pull`) if both projects share a remote + +The sections below are the canonical backend migration reference. + +## Federation (Peer-to-Peer Sync) + +Federation lets independent Dolt-backed workspaces ("towns") sync issues +directly with each other via `bd federation add-peer`/`sync`/`status`, +without a central hub. Credentials are AES-256 encrypted and stored locally. + +See [FEDERATION.md](/multi-agent/federation) for the full setup guide, including +peer configuration, sovereignty tiers, sync/status/topology details, and +troubleshooting. + +## Dolt Remotes + +Use `bd dolt remote add` to configure remotes. This ensures the running Dolt SQL +server sees the remote immediately. Remotes added directly with the `dolt` CLI +are written to filesystem config and may not be visible to the server until +restart. + +```bash +# DoltHub (public or private) +bd dolt remote add origin https://doltremoteapi.dolthub.com/org/beads + +# S3 +bd dolt remote add origin aws://[bucket]/path/to/repo + +# GCS +bd dolt remote add origin gs://[bucket]/path/to/repo + +# Git SSH (GitHub, GitLab, etc.) +bd dolt remote add origin git+ssh://git@github.com/org/repo.git + +# Local file system +bd dolt remote add origin file:///path/to/remote +``` + +### Push/Pull + +```bash +bd dolt push +bd dolt pull +``` + +`bd dolt remote add` registers the remote through the Dolt store API. SQL +remotes are the source of truth for `bd dolt remote list`, `bd dolt push`, and +`bd dolt pull`. + +For git-protocol remotes, credentialed external-server remotes, and cloud +remotes whose credentials are only present in the current shell, `bd dolt push` +and `bd dolt pull` automatically materialize a matching local CLI remote before +using the `dolt` CLI transport. The CLI remote is a local transport mirror, not +a separate configuration source. + +If you are upgrading from an older beads version and previously added remotes +with raw `dolt remote add`, re-register them with `bd dolt remote add +` so they are visible through SQL. `bd doctor` reports legacy CLI-only or +mismatched CLI remotes under `Dolt Remote Migration`. + +> **Sharing a Git repo**: Dolt stores data under `refs/dolt/data`, separate +> from standard Git refs (`refs/heads/`, `refs/tags/`). You can safely point a +> `git+ssh://` remote at the same repository as your project source code. See +> [Dolt Git Remotes](https://docs.dolthub.com/concepts/dolt/git/remotes). + +### List/Remove Remotes + +```bash +bd dolt remote list # Shows SQL-configured remotes +bd dolt remote remove origin # Removes the remote +``` + +## Contributor Onboarding (Clone Bootstrap) + +When someone clones a repository that uses Dolt backend: + +1. Run `bd bootstrap` in the clone +2. If the git remote has `refs/dolt/data` (pushed via `bd dolt push`), + `bd bootstrap` auto-detects it and clones the database from the remote +3. Work continues normally — all existing issues are available + +**No manual steps required** beyond `bd bootstrap`. The auto-detect: +- Probes `origin` for `refs/dolt/data` +- Clones the Dolt database from the remote (instead of creating a fresh one) +- Configures the Dolt remote for future `bd dolt push`/`pull` + +If `sync.remote` is set in `.beads/config.yaml`, that takes precedence +over auto-detection. Any Dolt-compatible remote URL is supported (DoltHub, +S3, GCS, file, or git). On brand-new projects, `bd init` auto-detects +`git origin` and persists it as `sync.remote`, so the first `bd dolt push` +publishes Dolt history to `refs/dolt/data` on the same git remote. + +### Verifying Bootstrap Worked + +```bash +bd list # Should show issues +bd vc log # Should show initial commit +``` + +## Troubleshooting + +### Server Not Running + +**Symptom:** Connection refused errors when using server mode. + +``` +failed to create database: dial tcp 127.0.0.1:3307: connect: connection refused +``` + +**Fix:** +```bash +gt dolt start # Orchestrator command +# Or +gt dolt status # Check if running +``` + +### Bootstrap Not Running + +**Symptom:** `bd list` shows nothing on fresh clone. + +**Check:** +```bash +ls .beads/dolt/ # Should NOT exist (pre-bootstrap) +BD_DEBUG=1 bd list # See bootstrap output +``` + +**Force bootstrap:** +```bash +rm -rf .beads/dolt # Remove broken state +bd list # Re-triggers bootstrap +``` + +### Database Corruption + +**Symptom:** Queries fail, inconsistent data. + +**Diagnosis:** +```bash +bd doctor # Basic checks +bd doctor --deep # Full validation +bd doctor --server # Server mode checks (if applicable) +``` + +**Recovery options:** + +1. **Repair what's fixable:** + ```bash + bd doctor --fix + ``` + +2. **Rebuild from remote:** + ```bash + rm -rf .beads/dolt + bd list # Re-triggers bootstrap + ``` + +### Already Committed `.beads/dolt/` to Git + +If you accidentally committed a Dolt data directory: + +1. Update gitignore: `bd doctor --fix` +2. Remove it from git tracking: `git rm --cached -r .beads/dolt/` (or `.beads/embeddeddolt/`) +3. Commit the removal: `git commit -m "fix: remove accidentally committed dolt data"` +4. To purge from history, use [BFG Repo-Cleaner](https://rtyley.github.io/bfg-repo-cleaner/) or `git filter-repo` + +### Lock Contention (Embedded Mode) + +**Symptom:** "database is locked" errors. + +Embedded mode is single-writer (enforced via file lock). If you need concurrent +access, switch to server mode. See [Migrating Between Backends](#migrating-between-backends). + +## Configuration Reference + +```yaml +# .beads/config.yaml + +# Dolt settings +dolt: + # Auto-commit Dolt history after writes (default: on for embedded, off for server) + auto-commit: on # on | off + + # Storage mode (default: embedded) + mode: embedded # embedded | server + # Server mode settings (only used when mode: server) + host: 127.0.0.1 + port: 3307 + user: root + # Password: env var or credentials file (see below) + + # Shared server mode (GH#2377): all projects share a single Dolt server + # at ~/.beads/shared-server/. Each project uses its own database (prefix-based). + # Eliminates port conflicts and reduces resource usage on multi-project machines. + shared-server: false # true | false +``` + +### Environment Variables + +| Variable | Purpose | +|----------|---------| +| `BEADS_DOLT_PASSWORD` | Server mode password (highest priority) | +| `BEADS_CREDENTIALS_FILE` | Path to credentials file (overrides default location) | +| `BEADS_DOLT_SERVER_MODE` | Enable server mode (set to "1") | +| `BEADS_DOLT_SERVER_HOST` | Server host (default: 127.0.0.1) | +| `BEADS_DOLT_SERVER_PORT` | Server port (default: 3307, or 3308 in shared mode) | +| `BEADS_DOLT_SERVER_TLS` | Enable TLS (set to "1" or "true") | +| `BEADS_DOLT_SERVER_USER` | MySQL connection user | +| `BEADS_DOLT_SHARED_SERVER` | Enable shared server mode (set to "1" or "true") | +| `DOLT_REMOTE_USER` | Clone/push/pull auth user | +| `DOLT_REMOTE_PASSWORD` | Clone/push/pull auth password | +| `BD_DOLT_AUTO_COMMIT` | Override auto-commit setting | + +### Credentials File + +For multi-server setups, you can store passwords in an INI-style credentials file +instead of juggling environment variables per project. Passwords are looked up by +`[host:port]` section, so each project automatically gets the right password based +on its configured server. + +**Password resolution order:** +1. `BEADS_DOLT_PASSWORD` env var (highest priority, existing behavior) +2. Credentials file lookup by `[host:port]` (using the resolved runtime port) +3. Empty string (no password) + +**Port resolution note:** The `[host:port]` used for credential lookup matches the +resolved runtime port (from the port file, env var, or config — in that priority +order), not necessarily the port stored in `metadata.json`. This matters when using +IAP tunnels: if your tunnel maps remote:3307 to localhost:3308, store your password +under `[127.0.0.1:3308]` and the credentials file will match the actual connection. + +**Default location:** `~/.config/beads/credentials` (Linux/macOS), `%APPDATA%\beads\credentials` (Windows) + +**Override location:** Set `BEADS_CREDENTIALS_FILE` env var. + +**File format:** + +```ini +# ~/.config/beads/credentials +[127.0.0.1:3307] +password=localDevPassword + +[beads.company.com:3307] +password=teamServerPassword + +[10.0.1.50:3308] +password=officePassword +``` + +**Permissions:** On Linux/macOS, a warning is printed to stderr if the file is +readable by group or others (mirrors ssh behavior). Set permissions with: + +```bash +chmod 600 ~/.config/beads/credentials +``` + +## Dolt Version Control + +Dolt maintains its own version history, separate from Git: + +```bash +# View Dolt commit history +bd vc log + +# Show diff between Dolt commits +bd vc diff HEAD~1 HEAD + +# Create manual checkpoint +bd vc commit -m "Checkpoint before refactor" +``` + +### Auto-Commit Behavior + +In **embedded mode** (standalone default), each `bd` write command creates a Dolt commit: + +```bash +bd create "New issue" # Creates issue + Dolt commit +``` + +In **server mode** (orchestrator), auto-commit defaults to OFF because the server +manages its own transaction lifecycle. Firing `DOLT_COMMIT` after every write +under concurrent load causes 'database is read only' errors. + +Override for batch operations (embedded) or explicit commits (server): + +```bash +bd --dolt-auto-commit off create "Issue 1" +bd --dolt-auto-commit off create "Issue 2" +bd vc commit -m "Batch: created issues" +``` + +## Server Management (Orchestrator) + +The orchestrator provides integrated Dolt server management: + +```bash +gt dolt start # Start server (background) +gt dolt stop # Stop server +gt dolt status # Show server status +gt dolt logs # View server logs +gt dolt sql # Open SQL shell +``` + +Server runs on port 3307 (avoids MySQL conflict on 3306). + +### Standalone-to-managed-city handoff + +When an existing standalone project is later added to a managed city or +orchestrator, avoid letting two Dolt servers become sources of truth for the +same beads database name. A common split-brain symptom is that `.beads/dolt-server.port` +points at the old standalone server while the shell environment points `bd` at +the managed server with `BEADS_DOLT_PORT` or `BEADS_DOLT_SERVER_PORT`. + +Check before migrating: + +```bash +bd doctor +bd dolt status +``` + +`bd doctor` warns when the runtime managed port differs from the local port +file. The warning is intentionally diagnostic only; do not delete the local port +file until the standalone store has been exported and imported into the managed +server. + +Safe manual handoff: + +```bash +# From the standalone project, without managed-city port overrides: +unset BEADS_DOLT_PORT BEADS_DOLT_SERVER_PORT +bd backup +bd export > /tmp/beads-standalone.jsonl +bd dolt stop + +# Then enter the managed-city environment and import into its Dolt server: +bd import /tmp/beads-standalone.jsonl +bd doctor +``` + +After `bd doctor` shows one healthy store and the imported issue count is +correct, archive the old local Dolt data directory instead of deleting it +immediately. Keep the backup until the managed city has been pushed or otherwise +snapshotted. + +### Shared Server Mode + +On machines with multiple beads projects, each project normally starts its own Dolt server. +Shared server mode runs a single Dolt server at `~/.beads/shared-server/` that serves all projects: + +```bash +# Enable for this project +bd dolt set shared-server true + +# Or enable machine-wide via environment variable +export BEADS_DOLT_SHARED_SERVER=1 + +# Or enable during init +bd init --prefix myproject --shared-server +``` + +**Benefits:** +- No port conflicts between projects (single server on port 3308, avoids orchestrator on 3307) +- Reduced resource usage (one process instead of many) +- Automatic database isolation (each project uses its own database name) + +**How it works:** +- Server state files (PID, port, lock, log) live in `~/.beads/shared-server/` +- Dolt data directory: `~/.beads/shared-server/dolt/` +- Each project's database is stored as a subdirectory (e.g., `~/.beads/shared-server/dolt/myproject/`) +- The file lock mechanism ensures safe concurrent access from multiple projects +- Default port is 3308 (not 3307) to avoid conflict with the orchestrator. Override with `BEADS_DOLT_SERVER_PORT` or `dolt.port` in config.yaml + +**Important:** Each project on a shared server **must have a unique prefix** (database name). +Two projects with the same prefix share the same database — if this happens accidentally, +the project identity check will detect the mismatch and refuse to connect, preventing +silent data corruption. Always use distinct prefixes when running `bd init --shared-server`. + +```bash +# Check shared server status from any project +bd dolt status + +# Show full configuration including shared mode +bd dolt show +``` + +### Data Location (Orchestrator) + +``` +/.dolt-data/ +├── hq/ # Town beads (hq-*) +├── my-project/ # Project rig (mp-*) +├── beads/ # Beads rig (bd-*) +└── other-project/ # Other rig (op-*) +``` + +### Central Dolt Server (macOS LaunchAgent) + +If you do not use the orchestrator but still want a single persistent Dolt +server for multiple projects on macOS, run a custom `LaunchAgent` instead of +spawning per-project embedded instances. + +#### Why Not `brew services start dolt`? + +After installing Dolt with `brew install dolt`, the natural next step is +`brew services start dolt`. However, the Homebrew formula runs +`dolt sql-server` without the `--config` flag, and Dolt does not auto-discover +`config.yaml` from its working directory. The config file must be passed +explicitly with `--config `. + +#### Setup with a Custom LaunchAgent + +Install Dolt and initialize its data directory: + +```bash +brew install dolt +cd /opt/homebrew/var/dolt && dolt init +``` + +Configure Dolt for port 3307: + +```yaml +# /opt/homebrew/var/dolt/config.yaml +log_level: info + +listener: + host: 127.0.0.1 + port: 3307 + max_connections: 100 + +behavior: + autocommit: true +``` + +Create the LaunchAgent plist: + +```bash +cat > ~/Library/LaunchAgents/com.local.dolt-server.plist << 'EOF' + + + + + Label + com.local.dolt-server + ProgramArguments + + /opt/homebrew/bin/dolt + sql-server + --config + /opt/homebrew/var/dolt/config.yaml + + WorkingDirectory + /opt/homebrew/var/dolt + RunAtLoad + + KeepAlive + + StandardOutPath + /opt/homebrew/var/log/dolt.log + StandardErrorPath + /opt/homebrew/var/log/dolt-error.log + + +EOF +``` + +Load and verify the service: + +```bash +launchctl load ~/Library/LaunchAgents/com.local.dolt-server.plist +mysql -h 127.0.0.1 -P 3307 -u root -e "SELECT 1" +``` + +Point beads at the central server: + +```bash +export BEADS_DOLT_SERVER_MODE=1 +export BEADS_DOLT_SERVER_PORT=3307 +``` + +Manage the service: + +```bash +# Stop +launchctl unload ~/Library/LaunchAgents/com.local.dolt-server.plist + +# Restart +launchctl unload ~/Library/LaunchAgents/com.local.dolt-server.plist +launchctl load ~/Library/LaunchAgents/com.local.dolt-server.plist + +# Check logs +tail -f /opt/homebrew/var/log/dolt.log +``` + +## Advanced Dolt Usage + +The `dolt` CLI lets you operate directly on the database for power-user +workflows. The data directory depends on your mode: `.beads/embeddeddolt/` +(embedded) or `.beads/dolt/` (server). + +### Branching + +```bash +cd .beads/dolt # or .beads/embeddeddolt for embedded mode +dolt branch feature-x +dolt checkout feature-x +``` + +### Time Travel + +```bash +dolt log +dolt checkout +dolt sql -q "SELECT * FROM issues" +``` + +### Diff and Blame + +```bash +dolt diff main feature-x +dolt blame issues +``` + +## Migration Cleanup + +After successful migration from SQLite, you may have backup files: + +``` +.beads/beads.backup-pre-dolt-20260122-213600.db +.beads/sqlite.backup-pre-dolt-20260123-192812.db +``` + +These are safe to delete once you've verified Dolt is working: + +```bash +# Verify Dolt works +bd list +bd doctor + +# Then clean up (after appropriate waiting period) +rm .beads/*.backup-*.db +``` + +**Recommendation:** Keep backups for at least a week before deleting. + +## See Also + +- [SYNC_CONCEPTS.md](/core-concepts/sync-concepts) - The conceptual model behind cross-machine sync (Dolt source of truth, wire format, anti-patterns) +- [SYNC_SETUP.md](/getting-started/sync-setup) - Setting up sync across multiple computers +- [FEDERATION.md](/multi-agent/federation) - Peer-to-peer federation setup +- [CONFIG.md](/reference/configuration) - Full configuration reference +- [DEPENDENCIES.md](/core-concepts/dependencies) - Dependencies and gates +- [GIT_INTEGRATION.md](/reference/git-integration) - Git worktrees and protected branches +- [TROUBLESHOOTING.md](/reference/troubleshooting) - General troubleshooting diff --git a/docs/architecture/index.md b/docs/architecture/index.md new file mode 100644 index 000000000..2de268b3d --- /dev/null +++ b/docs/architecture/index.md @@ -0,0 +1,213 @@ +--- +title: Architecture Overview +description: "Understanding Beads' three-layer data model" +--- + +This document explains how Beads' architecture works with Dolt as its storage backend. + +## Architecture + +Beads uses **Dolt** as its sole storage backend -- a version-controlled SQL database that provides git-like semantics (branch, merge, diff, push, pull) natively at the database level. + +By default, Dolt runs in **embedded mode** (in-process, no separate server). For multi-writer +setups (multiple agents, orchestrator), switch to **server mode** which connects to a +running `dolt sql-server`. See the [Dolt Server Mode](#dolt-server-mode) section below for details. + +```mermaid +flowchart TD + subgraph DOLT["🗄️ Dolt Database"] + D[(".beads/dolt/
Version-Controlled SQL")] + end + + subgraph REMOTE["🌐 Dolt Remotes"] + R[("DoltHub / S3 / GCS
Sync & Backup")] + end + + D <-->|"bd dolt push/pull"| R + + U((User)) -->|"bd create
bd update"| D + D -->|"bd list
bd show"| U + + style DOLT fill:#2d5a27,stroke:#4a9c3e,color:#fff + style REMOTE fill:#1a4a6e,stroke:#3a8ac4,color:#fff +``` + + +**Source of Truth** +**Dolt** is the source of truth. Every write auto-commits to Dolt history, providing full version control, branching, and merge capabilities at the database level. + +Recovery is straightforward: pull from a Dolt remote with `bd dolt pull`, or restore from a Dolt-native backup with `bd backup restore`. + + +### Why Dolt? + +- **Version-controlled SQL**: Full SQL queries with native version control +- **Cell-level merge**: Concurrent changes merge automatically at the field level +- **Multi-writer**: Server mode supports concurrent agents +- **Native branching**: Dolt branches independent of git branches +- **Works offline**: All queries run against local database +- **Portable**: `bd export` produces JSONL for migration and interoperability + +## Data Flow + +### Write Path +```text +User runs bd create + → Dolt database updated + → Auto-committed to Dolt history +``` + +### Read Path +```text +User runs bd list + → Dolt SQL query + → Results returned immediately +``` + +### Sync Path +```text +User runs bd dolt push + → Commits pushed to Dolt remote + +User runs bd dolt pull + → Remote commits fetched and merged +``` + +### Multi-Machine Sync Considerations + +When working across multiple machines or clones: + +1. **Always sync before switching machines** + ```bash + bd dolt push # Push changes before leaving + ``` + +2. **Pull before creating new issues** + ```bash + bd dolt pull # Pull changes first on new machine + bd create "New issue" + ``` + +3. **Avoid parallel edits** - If two machines create issues simultaneously without syncing, Dolt's cell-level merge handles most conflicts automatically + +See [Sync Failures Recovery](/recovery/sync-failures) for data loss prevention in multi-machine workflows (Pattern A5/C3). + +## Dolt Server Mode + +The Dolt server handles background synchronization and database operations: + +- Manages the Dolt database backend +- Handles auto-commit for change tracking +- Provides concurrent access for multiple agents +- Logs available at `.beads/dolt/sql-server.log` + + +Start the Dolt server with `bd dolt start`. Check health with `bd doctor`. + + +### Embedded Mode (No Server) + +For CI/CD pipelines, containers, and single-use scenarios, no server is needed. Beads operates in embedded mode automatically when no Dolt server is running: + +```bash +bd create "CI-generated issue" +bd dolt push +``` + +**When embedded mode is appropriate:** +- CI/CD pipelines (Jenkins, GitHub Actions) +- Docker containers +- Ephemeral environments +- Scripts that should not leave background processes + +### Multi-Clone Scenarios + + +**Race Conditions in Multi-Clone Workflows** +When multiple git clones of the same repository run sync operations simultaneously, race conditions can occur during push/pull operations. This is particularly common in: +- Multi-agent AI workflows (multiple Claude/GPT instances) +- Developer workstations with multiple checkouts +- Worktree-based development workflows + +**Prevention:** +1. Stop the Dolt server (`bd dolt stop`) before switching between clones +2. Dolt handles worktrees natively in server mode +3. Use embedded mode for automated workflows + + +See [Sync Failures Recovery](/recovery/sync-failures) for sync race condition troubleshooting (Pattern B2). + +## Recovery Model + +Dolt's version control makes recovery straightforward: + +1. **Lost database?** → Pull from Dolt remote: `bd dolt pull` +2. **Have a backup?** → Restore it: `bd backup restore [path] --force` +3. **Merge conflicts?** → Dolt handles cell-level merge natively + +### Universal Recovery Sequence + +The following sequence resolves the majority of reported issues. For detailed procedures, see [Recovery Runbooks](/recovery/index). + +```bash +bd dolt stop # Stop Dolt server (prevents race conditions) +git worktree prune # Clean orphaned worktrees +bd dolt pull # Pull from Dolt remote +bd dolt start # Restart server +``` + + +**Use `bd doctor --fix` With Care** +Always back up and preview before running `bd doctor --fix`: + +1. **Back up first:** `cp -r .beads .beads.backup` +2. **Preview changes:** `bd doctor --dry-run` — shows what would be fixed without making changes +3. **Review diagnostics:** `bd doctor` (no flags) — diagnostic only, no changes made +4. **Then fix:** `bd doctor --fix` — or `bd doctor --fix -i` to confirm each fix individually + +**Why caution?** The `--fix` flag may remove dependencies it flags as circular, including valid parent-child relationships. Use `--fix-child-parent` only if you're certain the flagged deps are invalid. + +**Other diagnostic tools:** +- `bd blocked` — check which issues are blocked and why +- `bd show ` — inspect a specific issue's state + + +See [Recovery](/recovery/index) for specific procedures and [Database Corruption Recovery](/recovery/database-corruption) for Dolt recovery steps. + +## Design Decisions + +### Why Dolt? + +Dolt is a version-controlled SQL database that provides git-like semantics natively. Unlike plain SQLite (binary merge conflicts) or JSONL (slow queries), Dolt gives you both fast SQL queries and proper merge semantics. + +### Why not a cloud server? + +Beads is designed for offline-first, local-first development. The Dolt server runs locally -- no cloud dependency, no downtime, no vendor lock-in, and full functionality on airplanes or in restricted networks. + +### Trade-offs + +| Benefit | Trade-off | +|---------|-----------| +| Works offline | No real-time collaboration | +| Version-controlled database | Requires Dolt server | +| Cell-level merge | Requires initial setup | +| Local-first speed | Manual sync to remotes | +| SQL queries | Dolt binary dependency | + +### When NOT to use Beads + +Beads is not suitable for: + +- **Large teams (10+)** — Git-based sync doesn't scale well for high-frequency concurrent edits +- **Non-developers** — Requires Git and command-line familiarity +- **Real-time collaboration** — No live updates; requires explicit sync +- **Cross-repository tracking** — Issues are scoped to a single repository +- **Rich media attachments** — Designed for text-based issue tracking + +For these use cases, consider GitHub Issues, Linear, or Jira. + +## Related Documentation + +- [Recovery Runbooks](/recovery/index) — Step-by-step procedures for common issues +- [CLI Reference](/cli-reference/index) — Complete command documentation +- [Getting Started](/index) — Installation and first steps diff --git a/docs/architecture/storage-backends.md b/docs/architecture/storage-backends.md new file mode 100644 index 000000000..d0b430f9c --- /dev/null +++ b/docs/architecture/storage-backends.md @@ -0,0 +1,278 @@ +--- +title: Storage Backends +--- + +Beads uses Dolt as its default storage backend, and Dolt remains the reference implementation. You can also point `bd` at Postgres, MySQL, or SQLite — every backend implements the full issue-tracking core, and the only thing you give up off Dolt is version control. + +One seam, one shared core: the issue-tracking semantics are written once (in `issueops`) and shared by *every* backend — Dolt runs them natively, the SQL family runs them through one thin per-engine dialect. Each backend is a store adapter over that shared core. Version control lives only in the Dolt store; on the SQL backends it returns one clean, typed error. + +```mermaid +flowchart TD + CLI["bd CLI"] --> SEAM["Storage interface
one seam — every backend satisfies all of it"] + + SEAM --> CORE["CORE — issueops
CRUD · search · deps · ready/claim · ordering · id-mint
written once, shared by EVERY backend"] + SEAM --> CAPS["CAPABILITIES — version control
history · branch · remotes · sync"] + + CORE --> DS["Dolt store
runs the core natively"] + CORE --> KIT["sqlkit.Store
SQL-family adapter · Dialect / Readiness / Claim"] + + DS --> DOLT[("Dolt — embedded / server
the conformance ORACLE")] + KIT --> PGD["pgdialect"] --> PG[("Postgres")] + KIT --> MYD["mysqldialect"] --> MY[("MySQL")] + KIT --> LTD["sqlitedialect"] --> LT[("SQLite")] + + CAPS -->|"implemented in the Dolt store"| DS + CAPS -->|"generated typed-unsupported shell"| SHELL["Postgres / MySQL / SQLite
operation X not supported"] + + DOLT -. "conformance-tested: SQL family matches Dolt byte-for-byte" .-> KIT +``` + + + +``` + bd CLI + | + Storage interface + | + +-------------------------+--------------------------+ + | | + CORE (every backend) CAPABILITIES (Dolt-only) + issueops -- CRUD, search, deps, version control: + ready/claim, ordering, id-mint, history, branch, remotes, sync + cycle checks: written ONCE | + | +-> Dolt store: implemented + +--------------------+ +-> Postgres/MySQL/SQLite: + | | generated typed-"unsupported" + Dolt store sqlkit.Store (operation "X" not supported) + runs the core SQL-family adapter + natively (Dialect/Readiness/Claim) + | | + [Dolt engine] +------+------+---------+ + embedded/server pgdialect mysqldialect sqlitedialect + = conformance | | | + ORACLE Postgres MySQL SQLite + + issueops is shared by EVERY backend (Dolt included); each backend is a store + adapter over it. Dolt runs the core's SQL natively and adds version control; + the SQL family runs it through one thin dialect per engine, and is + conformance-tested to match the Dolt oracle byte-for-byte. +``` + + + +## Why Choose a Backend? + +- **Dolt stays the default** — `bd init` with no flags behaves exactly as before, and Dolt is the only backend with history, branching, and sync +- **Postgres / MySQL** — put your issues in the database your team already runs, backs up, and monitors; many writers, one server +- **SQLite** — a single file inside `.beads/`, zero servers, zero credentials, pure-Go driver +- **Same `bd` everywhere** — create, list, ready, deps, claims, labels, comments, search all behave identically on every backend; conformance tests enforce it against the Dolt reference +- **Clean degradation** — Dolt-only commands don't half-work on SQL backends; they return one clear, typed error + +## Pick Your Backend + +| Backend | Best for | History | Server needed | Credentials | +|---|---|---|---|---| +| `dolt` (default) | Solo or team work where you want issue history, branching, `bd dolt push`/`pull` | Yes | No (embedded) / optional | Only in server mode | +| `sqlite` | Solo work, throwaway workspaces, CI sandboxes, air-gapped machines | No | No | None | +| `postgres` | Teams with an existing Postgres; many workspaces share one database via per-workspace schemas | No | Yes | Ladder (below) | +| `mysql` | Teams with an existing MySQL; each workspace gets its own database | No | Yes | Ladder (below) | + +Rules of thumb: + +- If you want `bd history`, `bd dolt push`, time-travel, or federation — use **Dolt**. It is the only backend that tracks history. +- If you want the smallest possible footprint and don't need history — use **SQLite**. +- If your organization already operates Postgres or MySQL and you want issues living next to everything else your DBAs manage — use **Postgres** or **MySQL**. Multi-writer concurrency comes from the database server itself. +- If in doubt, use the default. `bd init` without `--backend` is Dolt, unchanged. + +## Getting Started + +The backend is chosen once, at `bd init` time, and recorded in `.beads/metadata.json`. Every later `bd` command reads it from there — no per-command flags. + +### Dolt (default) + +```bash +bd init --prefix myproj +``` + +Nothing changes. See [DOLT.md](/architecture/dolt) for embedded vs. server mode. + +### SQLite + +```bash +bd init --backend=sqlite --prefix myproj + +# Optional: choose the database file (relative to .beads/; default beads.db) +bd init --backend=sqlite --sqlite-path=issues.db --prefix myproj +``` + +That's it — no server, no credentials. Foreign keys and immediate-transaction locking are configured automatically. + +### Postgres + +Each Beads workspace lives in its own Postgres **schema**, so many workspaces can share one database. + +```bash +bd init --backend=postgres \ + --pg-url='postgres://bd:onlyforinit@db.example.com:5432/beads?sslmode=require' \ + --pg-schema=myproj \ + --prefix myproj +``` + +- `--pg-url` — a standard `postgres://` URL. If omitted, `bd init` falls back to the `BEADS_POSTGRES_URL` environment variable. +- `--pg-schema` — the schema this workspace owns; created and pinned via `search_path`. + +> **Note:** `bd init` connects with the URL exactly as given, so embed the password +> in `--pg-url` for init. It is stripped before anything is written to disk — +> `.beads/metadata.json` stores only the password-free DSN — and every later +> command resolves the password through the credential ladder below. + +```bash +# For all subsequent commands, supply the password via any ladder rung, e.g.: +export BEADS_PG_PASSWORD_COMMAND='vault kv get -field=password secret/beads/pg' + +bd create "first issue" -p 1 +bd ready +``` + +### MySQL + +Each Beads workspace gets its own MySQL **database**. + +```bash +bd init --backend=mysql \ + --mysql-url='bd:onlyforinit@tcp(db.example.com:3306)/' \ + --mysql-database=myproj_beads \ + --prefix myproj +``` + +- `--mysql-url` — go-sql-driver DSN grammar: `user:password@tcp(host:port)/` (note the trailing slash, no scheme). Falls back to `BEADS_MYSQL_URL` if omitted. +- `--mysql-database` — the database this workspace owns. + +Same rule as Postgres: embed the password in `--mysql-url` for init only; it is redacted before persisting, and later commands use the ladder: + +```bash +export BEADS_MYSQL_PASSWORD_COMMAND='vault kv get -field=password secret/beads/mysql' +# or a static password: +export BEADS_MYSQL_PASSWORD=... +``` + +> **Note:** the MySQL DSN grammar cannot carry a password without a username. +> `bd` refuses such DSNs loudly rather than silently connecting passwordless. + +## Credentials & Override Layers + +SQLite needs no credentials. For Postgres and MySQL, the password is resolved fresh at open time by a **ladder** of sources, highest priority first. The first *configured* rung wins, and a configured rung that fails aborts the connection — it never silently falls through to a lower rung. + +Precedence, per backend: + +1. **Password embedded in the DSN** — a password inside `BEADS_POSTGRES_URL` / `BEADS_MYSQL_URL` wins outright; no ladder runs. +2. **Credential command** — `BEADS_PG_PASSWORD_COMMAND` / `BEADS_MYSQL_PASSWORD_COMMAND` +3. **Static env var** — `BEADS_PG_PASSWORD` / `BEADS_MYSQL_PASSWORD` +4. **Credentials file** — an entry matching the DSN's `host:port` +5. **Driver-native fallback** — Postgres only: `PGPASSWORD`, `~/.pgpass`, `PGPASSFILE` still work when nothing above is configured. MySQL has no driver-native fallback; with nothing configured it attempts an empty password. + +> **Note the naming asymmetry:** the URL variables use full names +> (`BEADS_POSTGRES_URL`, `BEADS_MYSQL_URL`) while the password variables +> abbreviate Postgres (`BEADS_PG_PASSWORD`, `BEADS_PG_PASSWORD_COMMAND`). +> There is no `BEADS_PG_URL`. + +### The credential command (rotating secrets) + +This is the same idiom as kubectl's ExecCredential, the AWS CLI's `credential_process`, and git credential helpers: `bd` runs your command with `sh -c` (30-second timeout) and reads the secret from stdout. Output can be a bare token, a kubectl-style JSON envelope (`{"token": "...", "expirationTimestamp": "..."}`), or an OAuth-style one (`{"access_token": "...", "expires_in": 900}`). Results are cached in-process and refreshed shortly before expiry, so repeated opens don't re-spawn your helper. + +```bash +# Vault +export BEADS_PG_PASSWORD_COMMAND="vault kv get -field=password secret/beads/pg" + +# AWS RDS IAM auth (15-minute tokens; caching and expiry are handled for you) +export BEADS_PG_PASSWORD_COMMAND="aws rds generate-db-auth-token \ + --hostname db.example.com --port 5432 --username bd" +``` + +The command variables are read from the environment only — never from workspace metadata — because a persisted command would be arbitrary code executed on every `bd` invocation. + +### The credentials file (static, per-host) + +The lowest configured rung is an INI-style file keyed by `host:port`, at `~/.config/beads/credentials` (Linux/macOS) or `%APPDATA%\beads\credentials` (Windows); override the path with `BEADS_CREDENTIALS_FILE`. + +```ini +# ~/.config/beads/credentials (chmod 600) +[127.0.0.1:5432] +password=localDevPassword + +[db.example.com:3306] +password=teamServerPassword +``` + +If the file is readable by group or others, `bd` prints an ssh-style warning. The file rung is deliberately forgiving: a missing file or unmatched section just means "not configured" and the ladder continues — fail-closed behavior is reserved for rungs you explicitly configured, like a command. + +### Dolt server credentials + +The Dolt backend in server mode has its own rungs: `BEADS_DOLT_PASSWORD`, then the credentials file, with `BEADS_DOLT_SERVER_USER` overriding the username. There is also `BEADS_DOLT_CREDENTIAL_COMMAND`, an *identity* credential: the minted token is presented as the connection username to an authenticating gateway server, which verifies it and routes to the database. This is for gateway deployments only — the direct Postgres/MySQL backends reject `BEADS_PG_CREDENTIAL_COMMAND` / `BEADS_MYSQL_CREDENTIAL_COMMAND` with an error telling you to use the `_PASSWORD_COMMAND` form instead. + +### Secrets never reach disk + +`bd init` redacts the password from your URL before writing `.beads/metadata.json`, using the database driver's own parser, and then re-parses to verify nothing survived — refusing to init rather than persisting a secret it can't prove is gone. Check for yourself: + +```bash +cat .beads/metadata.json +# {"backend":"postgres","postgres_dsn":"postgres://bd@db.example.com:5432/beads?sslmode=require", ...} +``` + +## What You Keep and What You Give Up + +Every backend implements the complete issue-tracking core — the same shared implementation, not a re-derivation per backend: + +- Issue CRUD, bulk creates, close/reopen/delete +- Search, counts, filtering, sorting +- Dependencies, cycle detection, dependency trees +- Ready/blocked work queries, claims and leases +- Labels, comments, the full event/audit trail +- Config, metadata slots, statistics, transactions, streaming iterators + +What is Dolt-only — by design, since these *are* version control: + +- **History & time travel** — `bd history`, as-of queries, diffs +- **Branching & merging** — branch, checkout, merge, conflict resolution +- **Remotes & sync** — `bd dolt push` / `bd dolt pull`, fetch, sync +- **Federation** — peer management +- **Compaction** — snapshot-based memory compaction + +On Postgres, MySQL, and SQLite these operations don't limp along or approximate — each returns one clean, typed error: + +``` +$ bd history myproj-9q9 +Error: failed to get history: operation "History" not supported by the sqlite backend +``` + +`bd init` sets the expectation up front: + +``` +✓ bd initialized with the SQLite backend + History: not tracked (SQLite backend has no version control; use Dolt for history) +``` + +A few practical consequences: + +- Writes on SQL backends are durable the moment the command returns — each write commits its own SQL transaction. (`bd`'s internal commit hooks become harmless no-ops, not errors.) +- The Dolt-only maintenance tail (auto-commit, auto-export, auto-backup, auto-push) is skipped automatically on non-Dolt backends. +- "Sync" on Postgres/MySQL is your database server: everyone connects to the same database, so there is nothing to push or pull. Back up issues with your normal database tooling, or `bd export` for JSONL interoperability. + +## How It Stays Correct + +The SQL backends are held to the embedded-Dolt reference by a conformance harness, run locally and in CI (with real Postgres and MySQL servers) on every pull request: + +- **Tier 1 (in-process):** the same behavior corpus the Dolt reference passes — CRUD, search, dependencies, ready/blocked, claims and leases, plus a ~94-case audit corpus of edge behaviors — runs against every backend. Two extra gates make capability gaps impossible to hide: a completeness test proves the set of unsupported operations exactly matches an audited allowlist, and a contract test calls every one of them to prove it returns the typed error rather than panicking or lying. +- **Tier 2 (end-to-end):** a real `bd` binary runs `bd init --backend=...` and a CLI scenario corpus in an isolated workspace per backend, and the normalized output must be byte-identical to the Dolt reference. Known-divergence entries are visible in the registry and can only shrink — currently there are none. + +```bash +./scripts/conformance.sh +# Postgres/MySQL profiles self-skip unless you point them at a server: +BEADS_PG_TEST_URL="postgres://user:pass@127.0.0.1:5432/beads_test" \ +BEADS_MYSQL_TEST_URL="user:pass@tcp(127.0.0.1:3306)/" \ +./scripts/conformance.sh +``` + +Adding a new backend is one declarative profile entry in `test/conformance/profiles.go` plus a store-factory arm; both tiers pick it up automatically. + +See [DOLT.md](/architecture/dolt) for the default backend, [CONFIG.md](/reference/configuration) for configuration, and [TROUBLESHOOTING.md](/reference/troubleshooting) if a connection misbehaves. diff --git a/docs/COMMUNITY_TOOLS.md b/docs/community-tools.md similarity index 99% rename from docs/COMMUNITY_TOOLS.md rename to docs/community-tools.md index 64e732835..b8d2dc0cb 100644 --- a/docs/COMMUNITY_TOOLS.md +++ b/docs/community-tools.md @@ -1,4 +1,6 @@ -# Beads Community Tools +--- +title: Community Tools +--- A curated list of community-built UIs, extensions, and integrations for Beads. Ranked by activity and maturity. diff --git a/docs/ADAPTIVE_IDS.md b/docs/core-concepts/adaptive-ids.md similarity index 94% rename from docs/ADAPTIVE_IDS.md rename to docs/core-concepts/adaptive-ids.md index 5e968c7cb..ca7dd8368 100644 --- a/docs/ADAPTIVE_IDS.md +++ b/docs/core-concepts/adaptive-ids.md @@ -1,4 +1,6 @@ -# Adaptive ID Length +--- +title: Adaptive ID Length +--- **Feature:** bd-ea2a13 **Status:** Implemented (v0.21+) @@ -212,10 +214,10 @@ bd config set issue_id_mode hash - **Hash IDs** (this document): Collision-free across parallel branches and agents; IDs are less predictable but always unique. - **Counter IDs**: Human-friendly and sequential; require care in multi-branch workflows where counters can diverge. -See [CONFIG.md](CONFIG.md) for full documentation on `issue_id_mode=counter`, including migration +See [CONFIG.md](/reference/configuration) for full documentation on `issue_id_mode=counter`, including migration guidance and per-prefix counter isolation. ## Related -- [Migration Guide](../README.md#migration) - Converting from sequential to hash IDs -- [Configuration](CONFIG.md) - All configuration options +- [Migration Guide](https://github.com/gastownhall/beads/blob/main/README.md#migration) - Converting from sequential to hash IDs +- [Configuration](/reference/configuration) - All configuration options diff --git a/docs/DEPENDENCIES.md b/docs/core-concepts/dependencies.md similarity index 95% rename from docs/DEPENDENCIES.md rename to docs/core-concepts/dependencies.md index f3f728c54..874681292 100644 --- a/docs/DEPENDENCIES.md +++ b/docs/core-concepts/dependencies.md @@ -1,4 +1,6 @@ -# Dependencies and Gates +--- +title: Dependencies and Gates +--- Beads includes a full dependency system for ordering work and a gate system for bridging external conditions (PR merges, CI runs, timers) into the @@ -319,8 +321,8 @@ bd ready # Only "Design" is ready ## See Also -- [QUICKSTART.md](QUICKSTART.md) — First steps with dependencies -- [MOLECULES.md](MOLECULES.md) — Molecule workflows using gates and dependencies -- [MULTI_REPO_AGENTS.md](MULTI_REPO_AGENTS.md) — Cross-repo dependency patterns -- [DOLT.md](DOLT.md) — Dolt backend configuration -- [CLI_REFERENCE.md](CLI_REFERENCE.md) — Full command reference +- [QUICKSTART.md](/getting-started/quickstart) — First steps with dependencies +- [MOLECULES.md](/workflows/molecules) — Molecule workflows using gates and dependencies +- [MULTI_REPO_AGENTS.md](/multi-agent/coordination) — Cross-repo dependency patterns +- [DOLT.md](/architecture/dolt) — Dolt backend configuration +- [CLI_REFERENCE.md](/cli-reference/index) — Full command reference diff --git a/docs/graph-links.md b/docs/core-concepts/graph-links.md similarity index 95% rename from docs/graph-links.md rename to docs/core-concepts/graph-links.md index 96c39e305..b1f98f347 100644 --- a/docs/graph-links.md +++ b/docs/core-concepts/graph-links.md @@ -1,4 +1,6 @@ -# Graph Links in Beads +--- +title: Graph Links in Beads +--- Beads supports several types of links between issues to create a knowledge graph. These links enable rich querying and traversal beyond simple blocking dependencies. @@ -274,6 +276,6 @@ Build conversation chains (via orchestrator mail): ## See Also -- [Messaging](../engdocs/messaging.md) - Mail commands and threading -- [Dependencies](QUICKSTART.md#dependencies) - Blocking dependencies -- [CLI Reference](CLI_REFERENCE.md) - All commands +- [Messaging](https://github.com/gastownhall/beads/blob/main/engdocs/messaging.md) - Mail commands and threading +- [Dependencies](/getting-started/quickstart#dependencies) - Blocking dependencies +- [CLI Reference](/cli-reference/index) - All commands diff --git a/docs/core-concepts/hash-ids.md b/docs/core-concepts/hash-ids.md new file mode 100644 index 000000000..60de0b84e --- /dev/null +++ b/docs/core-concepts/hash-ids.md @@ -0,0 +1,125 @@ +--- +title: Hash-based IDs +--- + +Understanding beads' collision-resistant ID system. + +## The Problem + +Traditional sequential IDs (`#1`, `#2`, `#3`) break when: +- Multiple agents create issues simultaneously +- Different branches have independent numbering +- Forks diverge and later merge + +## The Solution + +Beads uses hash-based IDs: + +``` +bd-a1b2c3 # Short hash +bd-f14c # Even shorter +bd-a3f8e9.1 # Hierarchical (child of bd-a3f8e9) +``` + +**Properties:** +- Globally unique (content-based hash) +- No coordination needed between creators +- Merge-friendly across branches +- Predictable length (configurable) + +## How Hashes Work + +IDs are generated from: +- Issue title +- Creation timestamp +- Random salt + +```bash +# Create issue - ID assigned automatically +bd create "Fix authentication bug" +# Returns: bd-7x2f + +# The ID is deterministic for same content+timestamp +``` + +## Hierarchical IDs + +For epics and subtasks: + +```bash +# Parent epic +bd create "Auth System" -t epic +# Returns: bd-a3f8e9 + +# Children auto-increment +bd create "Design UI" --parent bd-a3f8e9 # bd-a3f8e9.1 +bd create "Backend" --parent bd-a3f8e9 # bd-a3f8e9.2 +bd create "Tests" --parent bd-a3f8e9 # bd-a3f8e9.3 +``` + +Benefits: +- Clear parent-child relationship +- No namespace collision (parent hash is unique) +- Up to 3 levels of nesting + +## ID Configuration + +Configure ID prefix and length: + +```bash +# Set prefix (default: bd) +bd config set id.prefix myproject + +# Set hash length (default: 4) +bd config set id.hash_length 6 + +# New issues use new format +bd create "Test" +# Returns: myproject-a1b2c3 +``` + +## Collision Handling + +While rare, collisions are handled automatically: + +1. On import, if hash collision detected +2. Beads appends disambiguator +3. Both issues preserved + +```bash +# Check for collisions +bd info --schema --json | jq '.collision_count' +``` + +## Working with IDs + +```bash +# Partial ID matching +bd show a1b2 # Finds bd-a1b2... +bd show auth # Fuzzy match by title + +# Full ID required for ambiguous cases +bd show bd-a1b2c3d4 + +# List with full IDs +bd list --full-ids +``` + +## Migration from Sequential IDs + +If migrating from a system with sequential IDs: + +```bash +# Bootstrap from a JSONL export (preserves original IDs in metadata) +bd init --from-jsonl old-issues.jsonl + +# View original ID +bd show bd-new --json | jq '.original_id' +``` + +## Best Practices + +1. **Use short references** - `bd-a1b2` is usually unique enough +2. **Use `--json` for scripts** - Parse full ID programmatically +3. **Reference by hash in commits** - `Fixed bd-a1b2` in commit messages +4. **Let hierarchies form naturally** - Create epics, add children as needed diff --git a/docs/core-concepts/index.md b/docs/core-concepts/index.md new file mode 100644 index 000000000..94084ed4c --- /dev/null +++ b/docs/core-concepts/index.md @@ -0,0 +1,72 @@ +--- +title: Core Concepts +--- + +Understanding the fundamental concepts behind beads. + +## Design Philosophy + +Beads was built with these principles: + +1. **Dolt as source of truth** - Issues stored in a version-controlled SQL database (Dolt), enabling collaboration via Dolt-native replication +2. **AI-native workflows** - Hash-based IDs, JSON output, dependency-aware execution +3. **Local-first operation** - Dolt database for fast queries, background sync +4. **Declarative workflows** - Formulas define repeatable patterns + +## Key Components + +### Issues + +Work items with: +- **ID** - Hash-based (e.g., `bd-a1b2`) or hierarchical (e.g., `bd-a1b2.1`) +- **Type** - `bug`, `feature`, `task`, `epic`, `chore` +- **Priority** - 0 (critical) to 4 (backlog) +- **Status** - `open`, `in_progress`, `closed` +- **Labels** - Flexible tagging +- **Dependencies** - Blocking relationships + +### Dependencies + +Four types of relationships: + +| Type | Description | Affects Ready Queue | +|------|-------------|---------------------| +| `blocks` | Hard dependency (X blocks Y) | Yes | +| `parent-child` | Epic/subtask relationship | No | +| `discovered-from` | Track issues found during work | No | +| `related` | Soft relationship | No | + +### Dolt Server Mode + +Dolt provides the database backend for beads: +- Start with `bd dolt start` +- Handles auto-commit and sync +- Logs available at `.beads/dolt/sql-server.log` +- Check health with `bd doctor` + +### Dolt Sync + +The synchronization mechanism: + +``` +Dolt DB (.beads/dolt/) + ↕ dolt commit +Local Dolt history + ↕ dolt push/pull +Remote Dolt repository +``` + +### Formulas + +Declarative workflow templates: +- Define steps with dependencies +- Variable substitution +- Gates for async coordination +- Aspect-oriented transformations + +## Navigation + +- [Issues & Dependencies](/core-concepts/issues) +- [Dolt Server Mode](#dolt-server-mode) +- [Dolt Sync](#dolt-sync) +- [Hash-based IDs](/core-concepts/hash-ids) diff --git a/docs/core-concepts/issues.md b/docs/core-concepts/issues.md new file mode 100644 index 000000000..98aa8dfea --- /dev/null +++ b/docs/core-concepts/issues.md @@ -0,0 +1,179 @@ +--- +title: "Issues & Dependencies" +--- + +Understanding the issue model in beads. + +## Issue Structure + +Every issue has: + +```bash +bd show bd-42 --json +``` + +```json +{ + "id": "bd-42", + "title": "Implement authentication", + "description": "Add JWT-based auth", + "type": "feature", + "status": "open", + "priority": 1, + "labels": ["backend", "security"], + "created_at": "2024-01-15T10:30:00Z", + "updated_at": "2024-01-15T10:30:00Z" +} +``` + +## Issue Types + +| Type | Use Case | +|------|----------| +| `bug` | Something broken that needs fixing | +| `feature` | New functionality | +| `task` | Work item (tests, docs, refactoring) | +| `epic` | Large feature with subtasks | +| `chore` | Maintenance (dependencies, tooling) | + +## Priorities + +| Priority | Level | Examples | +|----------|-------|----------| +| 0 | Critical | Security, data loss, broken builds | +| 1 | High | Major features, important bugs | +| 2 | Medium | Nice-to-have features, minor bugs | +| 3 | Low | Polish, optimization | +| 4 | Backlog | Future ideas | + +## Creating Issues + +```bash +# Basic issue +bd create "Fix login bug" -t bug -p 1 + +# With description +bd create "Add password reset" \ + --description="Users need to reset forgotten passwords via email" \ + -t feature -p 2 + +# With labels +bd create "Update dependencies" -t chore -l "maintenance,security" + +# JSON output for agents +bd create "Task" -t task --json +``` + +## Dependencies + +### Blocking Dependencies + +The `blocks` relationship affects the ready queue: + +```bash +# Add dependency: bd-2 depends on bd-1 +bd dep add bd-2 bd-1 + +# View dependencies +bd dep tree bd-2 + +# See blocked issues +bd blocked + +# See ready work (not blocked) +bd ready +``` + +### Structural Relationships + +These don't affect the ready queue: + +```bash +# Parent-child (epic subtasks) +bd create "Epic" -t epic +bd create "Subtask" --parent bd-42 + +# Discovered-from (found during work) +bd create "Found bug" --deps discovered-from:bd-42 + +# Related (soft link) +bd relate bd-1 bd-2 +``` + +### Dependency Types + +| Type | Description | Ready Queue Impact | +|------|-------------|-------------------| +| `blocks` | Hard dependency | Yes - blocked items not ready | +| `parent-child` | Epic/subtask hierarchy | No | +| `discovered-from` | Tracks origin of discovery | No | +| `related` | Soft relationship | No | + +## Hierarchical Issues + +For large features, use hierarchical IDs: + +```bash +# Create epic +bd create "Auth System" -t epic -p 1 +# Returns: bd-a3f8e9 + +# Child tasks auto-number +bd create "Design login UI" --parent bd-a3f8e9 # bd-a3f8e9.1 +bd create "Backend validation" --parent bd-a3f8e9 # bd-a3f8e9.2 + +# View hierarchy +bd dep tree bd-a3f8e9 +``` + +## Updating Issues + +```bash +# Change status +bd update bd-42 --claim + +# Change priority +bd update bd-42 --priority 0 + +# Add labels +bd update bd-42 --add-label urgent + +# Multiple changes +bd update bd-42 --claim --priority 1 --add-label "in-review" +``` + +## Closing Issues + +```bash +# Simple close +bd close bd-42 + +# With reason +bd close bd-42 --reason "Implemented in PR #123" + +# JSON output +bd close bd-42 --json +``` + +## Searching and Filtering + +```bash +# By status +bd list --status open +bd list --status in_progress + +# By priority +bd list --priority 1 +bd list --priority 0,1 # Multiple + +# By type +bd list --type bug +bd list --type feature,task + +# By label +bd list --label-any urgent,critical +bd list --label-all backend,security + +# Combined filters +bd list --status open --priority 1 --type bug --json +``` diff --git a/docs/LABELS.md b/docs/core-concepts/labels.md similarity index 98% rename from docs/LABELS.md rename to docs/core-concepts/labels.md index bf12d4934..42253e706 100644 --- a/docs/LABELS.md +++ b/docs/core-concepts/labels.md @@ -1,4 +1,6 @@ -# Labels in Beads +--- +title: Labels +--- Labels provide flexible, multi-dimensional categorization for issues beyond the structured fields (status, priority, type). Use labels for cross-cutting concerns, technical metadata, and contextual tagging without schema changes. @@ -745,7 +747,7 @@ The `set-state` command atomically: 2. Removes the old dimension label if present 3. Adds the new dimension:value label (cache) -See [CLI_REFERENCE.md](CLI_REFERENCE.md#state-labels-as-cache) for full command reference. +See [CLI_REFERENCE.md](/cli-reference/set-state) for full command reference. ## Troubleshooting @@ -784,6 +786,6 @@ bd doctor ## See Also -- [README.md](../README.md) - Main documentation -- [AGENTS.md](../AGENTS.md) - AI agent integration guide -- [ADVANCED.md](ADVANCED.md) - Advanced features and configuration +- [README.md](https://github.com/gastownhall/beads/blob/main/README.md) - Main documentation +- [AGENTS.md](https://github.com/gastownhall/beads/blob/main/AGENTS.md) - AI agent integration guide +- [ADVANCED.md](/reference/advanced) - Advanced features and configuration diff --git a/docs/METADATA.md b/docs/core-concepts/metadata.md similarity index 92% rename from docs/METADATA.md rename to docs/core-concepts/metadata.md index f51573f12..8d2968d2e 100644 --- a/docs/METADATA.md +++ b/docs/core-concepts/metadata.md @@ -1,11 +1,13 @@ -# Issue Metadata +--- +title: Issue Metadata +--- The `metadata` field on issues accepts arbitrary JSON. Any valid JSON value is stored as-is. Metadata is the preferred extension point for data that is specific to an integration, orchestrator, team workflow, or experimental automation. Before adding first-class fields, commands, or schema changes, check the -[Project Charter](../engdocs/PROJECT_CHARTER.md#schema-boundary). +[Project Charter](https://github.com/gastownhall/beads/blob/main/engdocs/PROJECT_CHARTER.md#schema-boundary). ## Example: Agent Execution Metadata @@ -83,5 +85,5 @@ Avoid these prefixes in user-defined keys to prevent conflicts with future Beads ## Related -- [Project Charter](../engdocs/PROJECT_CHARTER.md) - Product scope and schema boundary +- [Project Charter](https://github.com/gastownhall/beads/blob/main/engdocs/PROJECT_CHARTER.md) - Product scope and schema boundary - [#1416](https://github.com/gastownhall/beads/issues/1416) - Optional schema enforcement (future) diff --git a/docs/SYNC_CONCEPTS.md b/docs/core-concepts/sync-concepts.md similarity index 98% rename from docs/SYNC_CONCEPTS.md rename to docs/core-concepts/sync-concepts.md index 75c25cf4e..476fdfb68 100644 --- a/docs/SYNC_CONCEPTS.md +++ b/docs/core-concepts/sync-concepts.md @@ -1,4 +1,6 @@ -# Sync Concepts +--- +title: Sync Concepts +--- Beads issue data lives in Dolt. The local Dolt database is the source of truth for `bd list`, `bd show`, `bd ready`, and every write command. diff --git a/docs/getting-started/ide-setup.md b/docs/getting-started/ide-setup.md new file mode 100644 index 000000000..70f63c486 --- /dev/null +++ b/docs/getting-started/ide-setup.md @@ -0,0 +1,209 @@ +--- +title: IDE Setup +--- + +Configure your IDE for optimal beads integration. + +## Claude Code + +The recommended approach for Claude Code: + +```bash +# Setup Claude Code integration +bd setup claude +``` + +This installs: +- **SessionStart hook** - Runs `bd prime` when Claude Code starts +- **SessionStart compact refresh** - Runs `bd prime` after context compaction + +**How it works:** +1. SessionStart hook runs `bd prime` automatically +2. `bd prime` injects ~1-2k tokens of workflow context +3. You use `bd` CLI commands directly +4. Git hooks refresh exports and legacy fallbacks; Dolt remotes handle sync + +**Verify installation:** +```bash +bd setup claude --check +``` + +### Manual Setup + +If you prefer manual configuration, add to your Claude Code hooks: + +```json +{ + "hooks": { + "SessionStart": ["bd prime"], + "SessionStart": ["bd prime"] + } +} +``` + +## Cursor IDE + +```bash +# Setup Cursor integration +bd setup cursor +``` + +This creates `.cursor/rules/beads.mdc` with beads-aware rules. + +**Verify:** +```bash +bd setup cursor --check +``` + +## Aider + +```bash +# Setup Aider integration +bd setup aider +``` + +This creates/updates `.aider.conf.yml` with beads context. + +**Verify:** +```bash +bd setup aider --check +``` + +## GitHub Copilot + +For VS Code with GitHub Copilot, use the MCP server: + +```bash +# Install MCP server +uv tool install beads-mcp +``` + +Create `.vscode/mcp.json` in your project: + +```json +{ + "servers": { + "beads": { + "command": "beads-mcp" + } + } +} +``` + +**For all projects:** Add to VS Code user-level MCP config: + +| Platform | Path | +|----------|------| +| macOS | `~/Library/Application Support/Code/User/mcp.json` | +| Linux | `~/.config/Code/User/mcp.json` | +| Windows | `%APPDATA%\Code\User\mcp.json` | + +```json +{ + "servers": { + "beads": { + "command": "beads-mcp", + "args": [] + } + } +} +``` + +Initialize beads and reload VS Code: + +```bash +bd init --quiet +``` + +See [GitHub Copilot Integration](/integrations/github-copilot) for detailed setup. + +## Context Injection with `bd prime` + +All integrations use `bd prime` to inject context: + +```bash +bd prime +``` + +This outputs a compact (~1-2k tokens) workflow reference including: +- Available commands +- Current project status +- Workflow patterns +- Best practices +- Persistent memories from `bd remember` + +`bd prime` prints memories near the top and starts with a truncation warning. If your host stores the full hook output in a file and only shows a preview, have the agent read the full file before continuing. + +For memory-only hooks: + +```bash +bd prime --memories-only +``` + +**Why context efficiency matters:** +- Compute cost scales with tokens +- Latency increases with context size +- Models attend better to smaller, focused contexts + +## MCP Server (Alternative) + +For MCP-only environments (Claude Desktop, no shell access): + +```bash +# Install MCP server +pip install beads-mcp +``` + +Add to Claude Desktop config: +```json +{ + "mcpServers": { + "beads": { + "command": "beads-mcp" + } + } +} +``` + +**Trade-offs:** +- Works in MCP-only environments +- Higher context overhead (10-50k tokens for tool schemas) +- Additional latency from MCP protocol + +See [MCP Server](/integrations/mcp-server) for detailed configuration. + +## Git Hooks + +Ensure git hooks are installed for export refresh and legacy fallback behavior: + +```bash +bd hooks install +``` + +This installs: +- **pre-commit** - Validates changes before commit +- **post-merge** - Imports changes after pull +- **pre-push** - Ensures sync before push + +**Check hook status:** +```bash +bd info # Shows warnings if hooks are outdated +``` + +## Verifying Your Setup + +Run a complete health check: + +```bash +# Check version +bd version + +# Check project health +bd doctor + +# Check hooks +bd hooks status + +# Check editor integration +bd setup claude --check # or cursor, aider +``` diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md new file mode 100644 index 000000000..ce3887543 --- /dev/null +++ b/docs/getting-started/installation.md @@ -0,0 +1,403 @@ +--- +title: Installation +--- + +Complete installation guide for all platforms. + +## Components Overview + +Beads has several components - here's what they are and when you need them: + +| Component | What It Is | When You Need It | +|-----------|------------|------------------| +| **bd CLI** | Core command-line tool | Always - this is the foundation | +| **Claude Code Plugin** | Slash commands + enhanced UX | Optional - if you want `/beads:ready`, `/beads:create` commands | +| **MCP Server (beads-mcp)** | Model Context Protocol interface | Only for MCP-only environments (Claude Desktop, Amp) | + +**How they relate:** +- The **bd CLI** is the core - install it first via Homebrew, npm, or script +- The **Plugin** enhances Claude Code with slash commands but *requires* the CLI installed +- The **MCP server** is an *alternative* to the CLI for environments without shell access + +**Important:** Beads is installed system-wide, not cloned into your project. The `.beads/` directory in your project only contains the issue database. + +**Typical setups:** + +| Environment | What to Install | +|-------------|-----------------| +| Claude Code, Cursor, Windsurf | bd CLI (+ optional Plugin for Claude Code) | +| GitHub Copilot (VS Code) | bd CLI + MCP server | +| Claude Desktop (no shell) | MCP server only | +| Terminal / scripts | bd CLI only | +| CI/CD pipelines | bd CLI only | + +**Are they mutually exclusive?** No - you can have CLI + Plugin + MCP all installed. They don't conflict. But most users only need the CLI. + +## Quick Install (Recommended) + +### Homebrew (macOS/Linux) + +```bash +brew install beads +``` + +Homebrew core's `beads` formula is the supported Homebrew package. If you +previously installed the old tap formula as `bd`, see +[Migrating from the old Homebrew tap](/getting-started/upgrading#homebrew) to +switch to the core formula. + +**Why Homebrew?** +- Simple one-command install +- Automatic updates via `brew upgrade` +- No need to install Go +- Handles PATH setup automatically + +### Mise-en-place (macOS/Linux/Windows) + +You can install beads using [mise](https://mise.jdx.dev) from the latest GitHub release: + +```bash +mise install github:gastownhall/beads +mise use -g github:gastownhall/beads +``` + +The `-g` enables beads globally. To enable project-specific versions, omit it. + +**Why Mise?** +- Same as Homebrew: simple, updates via `mise up`, works without Go, handles PATH +- Supports all platforms +- Always the latest release +- May optionally use a different release version for specific projects + +Mise's Go backend follows the same caveats as `go install`; prefer the release backend above. + +### Quick Install Script (All Platforms) + +```bash +curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash +``` + +The installer will: +- Detect your platform (macOS/Linux/FreeBSD, amd64/arm64) +- Verify downloaded release archives against release `checksums.txt` +- Fall back to the supported `go install` modes if Go is available +- Fall back to building from source if needed +- Guide you through PATH setup if necessary + +On macOS, the script preserves the downloaded binary signature by default. If you explicitly want ad-hoc local re-signing, opt in: + +```bash +BEADS_INSTALL_RESIGN_MACOS=1 curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash +``` + +### Comparison of Installation Methods + +| Method | Best For | Updates | Prerequisites | Notes | +|--------|----------|---------|---------------|-------| +| **Homebrew** | macOS/Linux users | `brew upgrade beads` | Homebrew | Recommended. Handles everything automatically | +| **Mise** | All platforms | `mise up` | mise | Installs the latest GitHub release | +| **npm** | JS/Node.js projects | `npm update -g @beads/bd` | Node.js | Convenient if npm is your ecosystem | +| **bun** | JS/Bun.js projects | `bun install -g --trust @beads/bd` | Bun.js | Convenient if bun is your ecosystem | +| **Install script** | Quick setup, CI/CD | Re-run script | curl, bash | Good for automation and one-liners | +| **go install (nocgo)** | Go developers, simplest install | Re-run command | Go 1.24+ | **Server-mode only** (no embedded Dolt) | +| **go install (cgo)** | Go developers wanting embedded mode | Re-run command | Go 1.24+, C compiler | Full embedded-Dolt support | +| **From source** | Contributors only | `git pull && go build` | Go, git | Full control, can modify code | +| **AUR (Arch)** | Arch Linux users | `yay -Syu` | yay/paru | Community-maintained | + +**TL;DR:** Use Homebrew if available. Use npm if you're in a Node.js environment. Use the script for quick one-off installs or CI. + +## Go Install and Build Dependencies + +Use Homebrew, npm, or the install script if you do not specifically need `go install`. + +`go install` has two supported modes that give different capabilities: + +- **Server-mode only (nocgo, simplest):** `CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest`. Works on any machine with a Go toolchain, no C compiler needed. Produces a server-mode-only binary — you must run an external `dolt sql-server` and use `bd init --server`. See [Dolt](/architecture/dolt) for server-mode setup. +- **Embedded-capable (cgo):** `CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest`. Requires a C compiler (gcc/clang on Unix, MinGW on Windows). Produces a binary with the default embedded-Dolt backend — `bd init` Just Works. + +ICU headers are not required. The embedded-capable command uses `gms_pure_go` so go-mysql-server uses Go's stdlib regexp instead of ICU. + +Use the `github.com/steveyegge/beads` path for `go install`. The repository now lives under `gastownhall/beads`, but released Go modules still declare `github.com/steveyegge/beads` for compatibility. + +If you don't have a preference, `brew install beads` or the install script give you the embedded-capable build with no fuss. + +### Build Dependencies (Contributors Only) + + +These dependencies are only needed if you build from source. If you installed via Homebrew, npm, or the install script, skip this section entirely. + + +Building from source requires a C compiler (for CGO / embedded Dolt). ICU is +not required — all builds use the `gms_pure_go` tag which selects Go's +stdlib `regexp` instead of ICU regex. See +[ICU-POLICY.md](https://github.com/gastownhall/beads/blob/main/engdocs/ICU-POLICY.md) +for details. + +macOS (Homebrew): +```bash +brew install zstd +``` + +Linux (Debian/Ubuntu): +```bash +sudo apt-get install -y libzstd-dev +``` + +Linux (Fedora/RHEL): +```bash +sudo dnf install -y libzstd-devel +``` + +For maintainers only: if you intentionally need to run +[scripts/test-icu-path.sh](https://github.com/gastownhall/beads/blob/main/scripts/test-icu-path.sh) +(which exercises the leftover ICU code path), install ICU headers: +`brew install icu4c` (macOS) or `sudo apt-get install -y libicu-dev` (Linux). +This is not needed for normal development. + +## Platform-Specific Installation + +### macOS + +**Via Homebrew** (recommended): +```bash +brew install beads +``` + +**Via go install** (server-mode only): +```bash +CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest +``` + +**Via go install** (embedded-capable): +```bash +CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest +``` + +**From source**: +```bash +git clone https://github.com/gastownhall/beads +cd beads +make build +sudo mv bd /usr/local/bin/ +``` + +### Linux + +**Via Homebrew** (works on Linux too): +```bash +brew install beads +``` + +**Arch Linux** (AUR): +```bash +# Install from AUR +yay -S beads-git +# or +paru -S beads-git +``` + +**Via go install** (server-mode only): +```bash +CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest +``` + +**Via go install** (embedded-capable): +```bash +CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest +``` + +### FreeBSD + +**Via quick install script**: +```bash +curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash +``` + +**Via go install** (server-mode only): +```bash +CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest +``` + +### Windows 11 + +Beads ships with native Windows support—no MSYS or MinGW required. + +**Prerequisites:** +- [Go 1.24+](https://go.dev/dl/) installed (add `%USERPROFILE%\go\bin` to your `PATH`) +- Git for Windows + +**Via PowerShell script**: +```pwsh +irm https://raw.githubusercontent.com/gastownhall/beads/main/install.ps1 | iex +``` + +The script installs a prebuilt Windows release if available. Go is only required for `go install` or building from source. + +**Via go install** (server-mode only): +```pwsh +$env:CGO_ENABLED="0"; go install github.com/steveyegge/beads/cmd/bd@latest +``` + +**Via go install** (embedded-capable): +```pwsh +$env:CGO_ENABLED="1"; $env:GOFLAGS="-tags=gms_pure_go"; go install github.com/steveyegge/beads/cmd/bd@latest +``` + +## IDE and Editor Integrations + +### CLI + Hooks (Recommended) + +The recommended approach for Claude Code, Cursor, Windsurf, and other editors with shell access: + +```bash +# 1. Install bd CLI (see Quick Install above) +brew install beads + +# 2. Initialize in your project +cd your-project +bd init --quiet + +# 3. Setup editor integration (choose one) +bd setup claude # Claude Code - installs SessionStart hooks +bd setup cursor # Cursor IDE - creates .cursor/rules/beads.mdc +bd setup aider # Aider - creates .aider.conf.yml +bd setup codex # Codex CLI - installs Beads skill, AGENTS.md guidance, and native hooks +bd setup factory # Factory.ai Droid - creates/updates AGENTS.md +bd setup mux # Mux - creates/updates AGENTS.md +``` + +**How it works:** +- `bd init` creates or updates `AGENTS.md` and installs project Claude/Codex integrations by default unless you use `--skip-agents` or `--stealth` +- Editor hooks/rules inject `bd prime` automatically on session start +- Codex 0.129.0+ uses native `/hooks` for startup and compaction-aware context refresh +- `bd prime` provides ~1-2k tokens of workflow context +- You use `bd` CLI commands directly +- Git hooks (installed by `bd init`) refresh exports and legacy fallbacks; `bd dolt push/pull` syncs the database +- `bd onboard` prints the small manual snippet for unsupported agents or custom instruction files + +**Why this is recommended:** +- **Context efficient** - ~1-2k tokens vs 10-50k for MCP tool schemas +- **Lower latency** - Direct CLI calls, no MCP protocol overhead +- **Universal** - Works with any editor that has shell access + +### MCP Server (Alternative) + +Use MCP only when CLI is unavailable (Claude Desktop, Sourcegraph Amp without shell): + +```bash +# Using uv (recommended) +uv tool install beads-mcp + +# Or using pip +pip install beads-mcp +``` + +**Configuration for Claude Desktop** (macOS): + +Add to `~/Library/Application Support/Claude/claude_desktop_config.json`: + +```json +{ + "mcpServers": { + "beads": { + "command": "beads-mcp" + } + } +} +``` + +## Verifying Installation + +After installing, verify bd is working: + +```bash +bd version +bd help +``` + +## Troubleshooting + +### `bd: command not found` + +bd is not in your PATH: + +```bash +# Check if installed +go list -f {{.Target}} github.com/steveyegge/beads/cmd/bd + +# Add Go bin to PATH (add to ~/.bashrc or ~/.zshrc) +export PATH="$PATH:$(go env GOPATH)/bin" + +# Or reinstall with the recommended installer +curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash +``` + +### `zsh: killed bd` or crashes on macOS + +This is typically caused by CGO/SQLite compatibility issues: + +```bash +# Install an embedded-capable build +CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest +``` + +## Updating bd + +Upgrade checklist: + +1. With your current `bd`, sync remote-backed databases before installing the + new binary: + `bd dolt push` + `bd dolt pull` +2. Back up before migration: + `bd export --all -o .beads/backup/pre-migrate-$(date +%Y%m%d).jsonl` +3. Upgrade using the command for your install method below. +4. After upgrading: + `bd info --whats-new` + `bd hooks install` + `bd version` +5. If crossing a schema migration on a remote-backed database, only the + designated migrator runs: + `bd migrate --force` + `bd dolt push` + +Other clones should install the new binary and run `bd bootstrap`, not +independently migrate. For the full procedure, see [Upgrading](/getting-started/upgrading). + +### Quick install script (macOS/Linux/FreeBSD) + +```bash +curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash +``` + +### PowerShell installer (Windows) + +```pwsh +irm https://raw.githubusercontent.com/gastownhall/beads/main/install.ps1 | iex +``` + +### Homebrew + +```bash +brew upgrade beads +``` + +### go install + +```bash +# Server-mode only +CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest + +# Embedded-capable +CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest +``` + +For post-upgrade steps (hooks, migrations), see [Upgrading](/getting-started/upgrading). + +## Next Steps + +After installation: + +1. **Initialize a project**: `cd your-project && bd init` +2. **Learn the basics**: See [Quick Start](/getting-started/quickstart) +3. **Configure your agent**: See [IDE Setup](/getting-started/ide-setup), or run `bd setup --list` diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md new file mode 100644 index 000000000..9131dedd2 --- /dev/null +++ b/docs/getting-started/quickstart.md @@ -0,0 +1,367 @@ +--- +title: Quick Start +--- + +Get up and running with Beads in a few minutes. + +## Why Beads? + +Flat issue trackers (GitHub Issues, Jira, etc.) show you a list of open items. You pick one. But if that item depends on something else that isn't done yet, you've wasted time. Multiply this across a team of AI agents and humans, and you get thrashing. + +Beads tracks **dependencies between issues** and computes a **ready queue** — only items with no active blockers appear. Here's the difference: + +**Flat tracker (GitHub Issues):** +``` +Open issues: Set up database, Create API, Add authentication +→ An agent picks "Add authentication" and gets stuck immediately +``` + +**Beads:** +```bash +$ bd ready +1. [P1] [task] bd-1: Set up database + +$ bd ready --explain --json | jq '.blocked[0]' +{ + "id": "bd-3", + "title": "Add authentication", + "blocked_by": [{"id": "bd-2", "title": "Create API", "status": "open"}] +} +``` + +The agent picks the right task every time. No wasted cycles. + +## Installation + +Install `bd` using [the full installation guide](/getting-started/installation) (Homebrew, install script, npm, or `go install`). + +**Developing in a clone of this repository:** use `make install` so the binary gets correct build metadata and a consistent install path. Avoid ad-hoc `go build` / `go install` without the Makefile unless you know what you are doing — see the repository `README` and `AGENTS.md`. + +```bash +bd --help +``` + +## Initialize + +First time in a repository: + +```bash +# Basic setup (prompts for contributor mode) +bd init + +# For AI agents (non-interactive) +bd init --quiet + +# OSS contributor (fork workflow with separate planning repo) +bd init --contributor + +# Team member (branch workflow for collaboration) +bd init --team + +# Protected main branch (GitHub/GitLab) +# Note: Dolt stores data under refs/dolt/data, separate from +# Git refs, so no --branch flag is needed. +``` + +The wizard will: +- Create `.beads/` directory and embedded Dolt database +- **Prompt for your role** (maintainer or contributor) unless a flag is provided +- Import existing issues from git (if any) +- Prompt to install git hooks (recommended) +- Prompt to configure git merge driver (recommended) + +Notes: +- Dolt is the default (and only) storage backend. Data is stored in `.beads/embeddeddolt/`. +- By default, Dolt runs in **embedded mode** (in-process, no server needed). +- For multi-writer setups, use `bd init --server` to connect to a `dolt sql-server` instead. +- To import issues from an older installation, run `bd init --from-jsonl`. + +### Role configuration + +During `bd init`, you'll be asked: "Contributing to someone else's repo? [y/N]" + +- Answer **Y** if you're contributing to a fork (runs contributor wizard) +- Answer **N** if you're the maintainer or have push access + +This sets `git config beads.role` which determines how beads routes issues: + +| Role | Use case | Issue storage | +|------|----------|---------------| +| `maintainer` | Repo owner, team with push access | In-repo `.beads/` | +| `contributor` | Fork contributor, OSS contributor | Separate planning repo | + +You can also configure manually: + +```bash +# Set as contributor +git config beads.role contributor + +# Set as maintainer +git config beads.role maintainer + +# Check current role +git config --get beads.role +``` + +**Note:** If `beads.role` is not configured, beads falls back to URL-based detection (deprecated). Run `bd doctor` to check configuration status. + +## Your first issues + +```bash +# Create a few issues +bd create "Set up database" -p 1 -t task +bd create "Create API" -p 2 -t feature +bd create "Add authentication" -p 2 -t feature + +# List them +bd list +``` + +**Note:** Issue IDs are hash-based (e.g., `bd-a1b2`, `bd-f14c`) to prevent collisions when multiple agents/branches work concurrently. + +## Hierarchical issues (epics) + +For large features, use hierarchical IDs to organize work: + +```bash +# Create epic (generates parent hash ID) +bd create "Auth System" -t epic -p 1 +# Returns: bd-a3f8e9 + +# Create child tasks (use --parent to attach to the epic) +bd create "Design login UI" -p 1 --parent bd-a3f8e9 # bd-a3f8e9.1 +bd create "Backend validation" -p 1 --parent bd-a3f8e9 # bd-a3f8e9.2 +bd create "Integration tests" -p 1 --parent bd-a3f8e9 # bd-a3f8e9.3 + +# View hierarchy +bd dep tree bd-a3f8e9 +``` + +Output: +``` +Dependency tree for bd-a3f8e9: + +> bd-a3f8e9: Auth System [epic] [P1] (open) + > bd-a3f8e9.1: Design login UI [P1] (open) + > bd-a3f8e9.2: Backend validation [P1] (open) + > bd-a3f8e9.3: Integration tests [P1] (open) +``` + +## Add dependencies + +```bash +# API depends on database +bd dep add bd-2 bd-1 + +# Auth depends on API +bd dep add bd-3 bd-2 + +# View the tree +bd dep tree bd-3 +``` + +Output: +``` +Dependency tree for bd-3: + +> bd-3: Add authentication [P2] (open) + > bd-2: Create API [P2] (open) + > bd-1: Set up database [P1] (open) +``` + +**Dependency visibility:** `bd list` shows blocking dependencies inline: +``` +○ bd-a1b2 [P1] [task] - Set up database +○ bd-f14c [P2] [feature] - Create API (blocked by: bd-a1b2) +○ bd-g25d [P2] [feature] - Add authentication (blocked by: bd-f14c) +``` + +## Find ready work + +```bash +bd ready +``` + +Output: +``` +Ready work (1 issues with no blockers): + +1. [P1] bd-1: Set up database +``` + +Only bd-1 is ready because bd-2 and bd-3 are blocked. + +**Understanding why:** Use `--explain` to see the full graph reasoning: + +```bash +bd ready --explain +``` + +Output: +``` +Ready Work Explanation + +● Ready (1 issues): + + bd-1 [P1] Set up database + Reason: no blocking dependencies + Unblocks: 1 issue(s) + +● Blocked (2 issues): + + bd-2 [P2] Create API + ← blocked by bd-1: Set up database [open] + + bd-3 [P2] Add authentication + ← blocked by bd-2: Create API [open] + +─ Summary: 1 ready, 2 blocked +``` + +**Note:** `bd ready` is not the same as `bd list --status open`. The `list` command shows all open issues regardless of blockers. The `ready` command computes the dependency graph and only shows truly unblocked work. + +## Work the queue + +```bash +# Start working on bd-1 +bd update bd-1 --claim + +# Complete it +bd close bd-1 --reason "Database setup complete" + +# Check ready work again +bd ready +``` + +Now bd-2 is ready. + +## Track progress + +```bash +# See blocked issues +bd blocked + +# View statistics +bd stats +``` + +## Team sync + +Share issues with your team using Dolt remotes. Dolt stores data under `refs/dolt/data` on the same Git remote, separate from standard Git refs. In repos with `origin`, `bd init` configures that Dolt remote automatically. + +```bash +# Verify the remote, or add one if the repo had no origin during init +bd dolt remote list +bd dolt remote add origin git+ssh://git@github.com/org/repo.git # if needed + +# Push your issues +bd dolt push + +# Pull teammates' changes +bd dolt pull +``` + +When a teammate clones the repo, `bd bootstrap` auto-detects the existing database on `refs/dolt/data`, clones it, and wires `origin` for future `bd dolt push` / `bd dolt pull`. + +See [`bd dolt`](/cli-reference/dolt) for CLI details. For remote configuration and federation, see the repository docs [DOLT.md](https://github.com/gastownhall/beads/blob/main/docs/DOLT.md) and [FEDERATION-SETUP.md](https://github.com/gastownhall/beads/blob/main/FEDERATION-SETUP.md). + +## Optional: Notion sync + +If you keep project issues in Notion, save an integration token first: + +```bash +bd config set notion.token +``` + +Then either create a new Beads database under a parent page or connect to an existing target: + +```bash +bd notion init --parent +# or +bd notion connect --url +``` + +The same auth value can also come from `NOTION_TOKEN`. Directly setting `notion.data_source_id` remains available as an escape hatch for advanced setups. + +Check which auth source is active and whether the target schema is ready: + +```bash +bd notion status +bd notion status --json +``` + +Preview or run sync: + +```bash +bd notion sync --dry-run +bd notion sync +bd notion sync --pull +bd notion sync --push +``` + +## Database location + +By default (embedded mode), data is stored in `.beads/embeddeddolt/` within your repository. +In server mode, data is managed by the external `dolt sql-server`. + +## Migrating databases + +After upgrading bd, use `bd migrate` to check for and migrate old database files: + +```bash +# Inspect migration plan (AI agents) +bd migrate --inspect --json + +# Check schema and config +bd info --schema --json + +# Preview migration changes +bd migrate --dry-run + +# Migrate old databases to beads.db +bd migrate + +# Migrate and clean up old files +bd migrate --cleanup --yes +``` + +**AI agents:** Use `--inspect` to analyze migration safety before running. The system verifies required config keys and data integrity invariants. + +## Database maintenance + +As your project accumulates closed issues, the database grows. Manage size with these commands: + +```bash +# View compaction statistics +bd admin compact --stats + +# Preview compaction candidates (30+ days closed) +bd admin compact --analyze --json + +# Apply agent-generated summary +bd admin compact --apply --id bd-42 --summary summary.txt + +# Immediately delete closed issues (CAUTION: permanent!) +bd admin cleanup --force +``` + +**When to compact:** +- Database file > 10MB with many old closed issues +- After major project milestones when old issues are no longer relevant +- Before archiving a project phase + +**Note:** Compaction is permanent graceful decay. Original content is discarded but viewable via `bd restore ` from git history. + +## Next steps + +- Add labels: `bd create "Task" -l "backend,urgent"` +- Filter ready work: `bd ready --priority 1` +- Explain the graph: `bd ready --explain` +- Check graph integrity: `bd graph check` +- Search issues: `bd list --status open` +- Detect cycles: `bd dep cycles` +- Gates for PR/CI sync: [`bd gate`](/cli-reference/gate) +- More sync scenarios: [`bd dolt`](/cli-reference/dolt) +- Full command list: [CLI Reference](/cli-reference/index) + +See the [repository README](https://github.com/gastownhall/beads/blob/main/README.md) for an overview and links to deeper docs. diff --git a/docs/SYNC_SETUP.md b/docs/getting-started/sync-setup.md similarity index 93% rename from docs/SYNC_SETUP.md rename to docs/getting-started/sync-setup.md index e757194f4..7ea1b55e9 100644 --- a/docs/SYNC_SETUP.md +++ b/docs/getting-started/sync-setup.md @@ -1,4 +1,6 @@ -# Sync Setup Guide +--- +title: Sync Setup Guide +--- Set up beads with Dolt sync so your issues follow you across computers. @@ -8,7 +10,7 @@ You need two tools installed on every machine: | Tool | Minimum Version | Install | |------|-----------------|---------| -| **bd** (beads CLI) | 0.59.0+ | See [INSTALLING.md](INSTALLING.md) | +| **bd** (beads CLI) | 0.59.0+ | See [INSTALLING.md](/getting-started/installation) | | **Dolt** | 2.1.10+ | `brew install dolt` or [dolt install script](https://github.com/dolthub/dolt/releases/latest/download/install.sh) | Verify both are installed: @@ -276,10 +278,10 @@ bd dolt start ## See Also -- [SYNC_CONCEPTS.md](SYNC_CONCEPTS.md) — The conceptual model behind this setup (why Dolt is the source of truth, what JSONL is for) -- [QUICKSTART.md](QUICKSTART.md) — Getting started with beads -- [DOLT.md](DOLT.md) — Dolt backend details, server modes, federation, remote types, and sync modes -- [INSTALLING.md](INSTALLING.md) — Installation for all platforms +- [SYNC_CONCEPTS.md](/core-concepts/sync-concepts) — The conceptual model behind this setup (why Dolt is the source of truth, what JSONL is for) +- [QUICKSTART.md](/getting-started/quickstart) — Getting started with beads +- [DOLT.md](/architecture/dolt) — Dolt backend details, server modes, federation, remote types, and sync modes +- [INSTALLING.md](/getting-started/installation) — Installation for all platforms ## Attribution diff --git a/docs/getting-started/upgrading.md b/docs/getting-started/upgrading.md new file mode 100644 index 000000000..542c8df97 --- /dev/null +++ b/docs/getting-started/upgrading.md @@ -0,0 +1,368 @@ +--- +title: Upgrading +--- + +How to upgrade bd and keep your projects in sync. + +## Checking for Updates + +```bash +# Current version +bd version + +# What's new in recent versions +bd info --whats-new +bd info --whats-new --json # Machine-readable +``` + +## Short Version + +1. With your current `bd`, sync remote-backed databases before installing the + new binary: + `bd dolt push` + `bd dolt pull` +2. Back up before migration: + `bd export --all -o .beads/backup/pre-migrate-$(date +%Y%m%d).jsonl` +3. Upgrade using the command that matches your install method. +4. After upgrading: + `bd info --whats-new` + `bd hooks install` + `bd version` +5. If crossing a schema migration on a remote-backed database, only the + designated migrator runs: + `bd migrate --force` + `bd dolt push` + +Other clones should install the new binary and run `bd bootstrap`, not +independently migrate. The full procedure is below. + +## Upgrading + +Use the command that matches your install method. + +| Install method | Platforms | Command | +|---|---|---| +| Quick install script | macOS, Linux, FreeBSD | `curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh \| bash` | +| PowerShell installer | Windows | `irm https://raw.githubusercontent.com/gastownhall/beads/main/install.ps1 \| iex` | +| Homebrew | macOS, Linux | `brew upgrade beads` | +| go install (server-mode only) | macOS, Linux, FreeBSD, Windows | `CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest` | +| go install (embedded-capable) | macOS, Linux, Windows | `CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest` | +| npm | macOS, Linux, Windows | `npm update -g @beads/bd` | +| bun | macOS, Linux, Windows | `bun install -g --trust @beads/bd` | +| From source (Unix shell) | macOS, Linux, FreeBSD | `git pull && make build` | + +### Quick install script (macOS/Linux/FreeBSD) + +```bash +curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash +``` + +### PowerShell installer (Windows) + +```pwsh +irm https://raw.githubusercontent.com/gastownhall/beads/main/install.ps1 | iex +``` + +### Homebrew + +```bash +brew upgrade beads +``` + + +If you still have the old tap formula installed as `bd`, switch to the +Homebrew core formula: + +```bash +brew uninstall bd +brew untap gastownhall/beads 2>/dev/null || true +brew untap steveyegge/beads 2>/dev/null || true +brew install beads +``` + +### go install + +```bash +# Server-mode only +CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest + +# Embedded-capable +CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest +``` + +### From Source + +```bash +cd beads +git pull +make build +sudo mv bd /usr/local/bin/ +``` + +## After Upgrading + +**Important:** After upgrading, update your hooks: + +```bash +# 1. Check what changed +bd info --whats-new + +# 2. Update git hooks to match new version +bd hooks install + +# 3. Check for any outdated hooks +bd info # Shows warnings if hooks are outdated + +# 4. If using Dolt backend, restart the server +bd dolt stop && bd dolt start +``` + +**Why update hooks?** Git hooks are versioned with bd. Outdated hooks may miss export refresh, legacy fallback, or safety fixes. + +## Database Migrations + +After major upgrades, check for database migrations: + +```bash +# Inspect migration plan (AI agents) +bd migrate --inspect --json + +# Preview migration changes +bd migrate --dry-run + +# Apply migrations +bd migrate + +# Migrate and clean up old files +bd migrate --cleanup --yes +``` + +### Remote-backed databases and multiple clones + +`bd` refuses to silently apply pending schema migrations to a database that has +a Dolt remote configured. Migrating more than one clone of a shared remote +independently forks the schema, after which `bd dolt pull` can no longer merge — +the break is silent and, across a primary-key-reshaping migration, unrecoverable +([#4259](https://github.com/gastownhall/beads/issues/4259)). The supported flow +is: one machine migrates and publishes; every other clone re-clones the migrated +database. + +This applies to **every** upgrade that crosses a pending migration on a +remote-backed database — the same procedure whether you are moving to a +prerelease or to a stable release. + +The gate is **state-aware by default** +([#4516](https://github.com/gastownhall/beads/issues/4516)): before blocking, +`bd` consults the remote's *cached* schema state and + +- **auto-migrates** when the remote is at the same schema version as this + clone — no one has migrated yet, so this clone is a safe first-mover + (concurrent first-movers converge to identical tables). It reminds you to + `bd dolt push` afterwards. +- **stops and directs you to adopt** (`bd bootstrap`) when the remote has + already been migrated by another clone. +- **stops for a human decision** when this clone and the remote applied + different content for the same migration (a genuine fork), or when the + remote's schema state cannot be read from the cached ref. + +Set `BD_SMART_GATE=0` to opt out and make the gate block unconditionally. +The recipes below are the explicit path and work the same in either mode. + +**Important ordering:** once the new binary is installed, a database with +pending migrations is gated on **every** open — `bd dolt push` and `bd dolt +pull` are refused too, not just `bd migrate`. So do all syncing with your +**current** binary, *before* you install the new one. + +**Back up before you migrate.** Schema migrations assume the database matches +the shape the previous migrations left behind; real databases sometimes drift +(interrupted writes, tooling bugs, very old bootstraps). A JSONL export is +cheap, issue-complete, and importable by any bd version: + +```bash +bd export --all -o .beads/backup/pre-migrate-$(date +%Y%m%d).jsonl +``` + +`bd export` captures issues, not Dolt history or config — for a full snapshot +also copy the `.beads` directory (or `dolt backup` in server mode) while no +`bd` command is running. + +**Single clone (including a solo user with a remote):** + +```bash +bd dolt push # 1. CURRENT binary: publish all local work +bd export --all -o .beads/backup/pre-migrate.jsonl # 2. backup (see above) +# 3. install the new binary (see Upgrading above) +bd migrate --force # 4. migrate as the designated migrator +bd dolt push # 5. publish the migrated schema +bd version # 6. confirm the new version is active +``` + +`--force` confirms you are the single designated migrator so this run may +migrate the remote-backed database. For scripted or CI use, +`BD_ALLOW_REMOTE_MIGRATE=1 bd migrate` is the env-var equivalent. + +**Multiple clones sharing one remote:** + +```bash +# 1. With your CURRENT (old) binary, on EVERY clone: publish all work and get in +# sync, then stop editing until the upgrade is done. +bd dolt push +bd dolt pull + +# 2. Designated migrator ONLY: back up, install the new binary, then migrate +# and publish. +bd export --all -o .beads/backup/pre-migrate.jsonl +bd migrate --force +bd dolt push + +# 3. Every OTHER clone: install the new binary, then ADOPT the migrated database. +# (bd dolt pull is refused here — the clone still has pending migrations — so +# re-clone instead. Safe because step 1 already pushed all work.) +bd bootstrap +``` + +`bd bootstrap` replaces the local database, so any work not pushed in step 1 is +lost — that is why step 1 publishes everything first. If a clone was instead +migrated independently and `bd dolt pull` later fails with `cannot merge because +table dependencies has different primary keys in its common ancestor`, the +schema has already forked — follow the recovery playbook: +[RECOVERY.md#pk-fork-refused](https://github.com/gastownhall/beads/blob/main/docs/RECOVERY.md#pk-fork-refused). + + +In **server mode**, `bd doctor` adds a migration-content-skew check that flags a +forked schema against the cached remote ref — a useful post-upgrade +verification. It is not available in embedded mode; there, confirm the upgrade +with `bd version` and a normal read such as `bd ready`. + + +## Cross-era Upgrades + +If you're upgrading from a much older version of bd, your project may use a different storage backend. bd has gone through several storage eras: + +| Era | Versions | Storage | +|---|---|---| +| SQLite | v0.30–v0.50 | `.beads/beads.db` | +| Dolt server | v0.50–v0.58 | `.beads/dolt/` (external server) | +| Embedded Dolt (old) | v0.59–v0.63.2 | `.beads/dolt/` (in-process) | +| Embedded Dolt (current) | v0.63.3+ | `.beads/embeddeddolt/` | + +### From v0.63.3+ (current era) + +Upgrade the binary and run: + +```bash +bd migrate +``` + +If the project was initialized before `bd init` automatically wired git origin +as the Dolt remote, verify the remote after upgrading: + +```bash +bd dolt remote list +``` + +When the list is empty, fix it on the machine whose local database is +authoritative: + +```bash +bd export -o .beads/issues.pre-remote.jsonl # optional issue audit export +bd dolt remote add origin git+ssh://git@github.com/org/repo.git +bd dolt push +``` + +Commit the resulting `.beads/config.yaml` change so other clones can run +`bd bootstrap` or `bd dolt pull`. + +### From v0.59–v0.63.2 (old embedded) + +Direct upgrade works automatically: + +```bash +# Just use the new binary — it handles the conversion +bd list +``` + +### From v0.50–v0.58 (Dolt server era) + +The old binary used an external Dolt SQL server. The new binary uses an embedded engine. + +```bash +# 1. Export your data while the old binary still works +bd list --json -n 0 --all > .beads/issues.jsonl + +# 2. Stop the Dolt server +dolt sql-server --stop # or kill the process + +# 3. Remove stale server metadata and old storage directories +rm -f .beads/metadata.json .beads/config.json +rm -rf .beads/dolt .beads/embeddeddolt + +# 4. Initialize with the new binary +bd init --from-jsonl --quiet + +# 5. Verify +bd list --all +``` + +### From v0.30–v0.50 (SQLite era) + +The old binary stored data in SQLite. The new binary uses Dolt. + +**Recommended: use the migration script** (requires `sqlite3` and `jq`): + +```bash +# Download the script from the beads repo +curl -fsSLO https://raw.githubusercontent.com/gastownhall/beads/main/scripts/migrate-sqlite-to-current.sh +chmod +x migrate-sqlite-to-current.sh + +# Run it in your project directory +./migrate-sqlite-to-current.sh +``` + +The script exports issues, dependencies, and labels from SQLite, handles type normalization, and imports everything into the new Dolt backend. + +**Alternative: manual export with the old binary.** Old binaries are always available on [GitHub Releases](https://github.com/gastownhall/beads/releases). Download the version that matches your project, then: + +```bash +# 1. Export with the old binary +./bd-old list --json -n 0 --all > .beads/issues.jsonl + +# 2. Import with the current binary +bd init --from-jsonl --quiet + +# 3. Verify +bd list --all +``` + +> **Note:** The manual export preserves issue content but not dependencies or labels. Use the migration script for a more complete transfer. + +## Troubleshooting Upgrades + +### Hooks out of date + +```bash +bd hooks install +``` + +### Database schema changed + +```bash +bd migrate --dry-run +bd migrate +``` + +### Recovery after upgrade + +If you need to restore from a backup: + +```bash +bd init +bd backup restore [path] --force +``` + +Or pull from a Dolt remote: + +```bash +bd dolt pull +``` diff --git a/docs/images/favicon.svg b/docs/images/favicon.svg new file mode 100644 index 000000000..e2337f6e4 --- /dev/null +++ b/docs/images/favicon.svg @@ -0,0 +1,4 @@ + + + bd + diff --git a/docs/images/logo.svg b/docs/images/logo.svg new file mode 100644 index 000000000..e2337f6e4 --- /dev/null +++ b/docs/images/logo.svg @@ -0,0 +1,4 @@ + + + bd + diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 000000000..6d4e96895 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,85 @@ +--- +title: Introduction +--- + +**Beads** (`bd`) is a Dolt-powered issue tracker designed for AI-supervised coding workflows. + +## Why Beads? + +Traditional issue trackers (Jira, GitHub Issues) weren't designed for AI agents. Beads was built from the ground up for: + +- **AI-native workflows** - Hash-based IDs prevent collisions when multiple agents work concurrently +- **Dolt-backed storage** - Issues stored in a version-controlled SQL database, enabling collaboration via Dolt-native replication +- **Dependency-aware execution** - `bd ready` shows only unblocked work +- **Formula system** - Declarative templates for repeatable workflows +- **Multi-agent coordination** - Routing, gates, and molecules for complex workflows + +## Quick Start + +```bash +# Install via Homebrew (macOS/Linux) +brew install beads + +# Or quick install (macOS/Linux/FreeBSD) +curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash + +# Initialize in your project +cd your-project +bd init --quiet + +# Create your first issue +bd create "Set up database" -p 1 -t task + +# See ready work +bd ready +``` + +## Core Concepts + +| Concept | Description | +|---------|-------------| +| **Issues** | Work items with priorities, types, labels, and dependencies | +| **Dependencies** | `blocks`, `parent-child`, `discovered-from`, `related` | +| **Dolt Server** | Database server for multi-writer access and performance | +| **Formulas** | Declarative workflow templates (TOML or JSON) | +| **Molecules** | Work graphs with parent-child relationships | +| **Gates** | Async coordination primitives (human, timer, GitHub) | + +## For AI Agents + +Beads is optimized for AI coding agents: + +```bash +# Always use --json for programmatic access +bd list --json +bd show bd-42 --json + +# Track discovered work during implementation +bd create "Found bug in auth" --description="Details..." \ + --deps discovered-from:bd-100 --json + +# Push changes at end of session +bd dolt push +``` + +See the [Claude Code integration](/integrations/claude-code) for detailed agent instructions. + +## Architecture + +``` +Dolt DB (.beads/embeddeddolt/ in embedded mode, + .beads/dolt/ in server mode; gitignored) + ↕ dolt commit +Local Dolt history + ↕ dolt push/pull +Remote Dolt repository (shared across machines) +``` + +The magic is automatic synchronization via Dolt's version-controlled database with built-in replication. + +## Next Steps + +- [Installation](/getting-started/installation) - Get bd installed +- [Quick Start](/getting-started/quickstart) - Create your first issues +- [CLI Reference](/cli-reference/index) - All available commands +- [Workflows](/workflows/index) - Formulas, molecules, and gates diff --git a/docs/integrations/aider.md b/docs/integrations/aider.md new file mode 100644 index 000000000..ff1a44d62 --- /dev/null +++ b/docs/integrations/aider.md @@ -0,0 +1,137 @@ +--- +title: Aider +--- + +How to use beads with Aider. + +[Aider](https://aider.chat/) is a human-in-the-loop AI pair programming tool: unlike autonomous agents such as [Claude Code](/integrations/claude-code), it doesn't run shell commands on its own. The beads integration works with that design — the AI **suggests** `bd` commands, and you confirm each one with aider's `/run` command. + +## Setup + +### Prerequisites + +- beads installed and initialized in your project (see [Installation](/getting-started/installation)); run `bd init` if there's no `.beads/` directory yet +- aider installed: `pip install aider-chat` (or `pipx install aider-chat`) + +### Quick Setup + +```bash +bd setup aider +``` + +This creates: + +- `.aider.conf.yml` — tells aider to load the beads instructions +- `.aider/BEADS.md` — workflow instructions the AI reads +- `.aider/README.md` — quick reference for humans + +### Verify Setup + +```bash +bd setup aider --check +``` + +### Remove the Integration + +```bash +bd setup aider --remove +``` + +This removes `.aider.conf.yml`, `.aider/BEADS.md`, and `.aider/README.md`. + +## Configuration + +The generated `.aider.conf.yml` loads the beads instructions into aider's read-only context: + +```yaml +# Beads Issue Tracking Integration for Aider +# Auto-generated by 'bd setup aider' + +# Load Beads workflow instructions for the AI +read: + - .aider/BEADS.md +``` + +`.aider/BEADS.md` holds the workflow rules the AI follows: track all work in bd (never markdown TODOs), suggest `bd ready` to find work, suggest `bd create` for new issues, suggest `bd dolt push` at end of session — and always *suggest* commands for you to run via `/run`. You can edit it to add project-specific instructions, but rerunning `bd setup aider` regenerates it. + +## Workflow + +### Start Session + +```bash +# Aider will have access to issues via .aider.conf.yml +aider + +# Or manually inject context +bd prime | aider --message-file - +``` + +### During Work + +Use bd commands alongside aider: + +```bash +# In another terminal or after exiting aider +bd create "Found bug during work" --deps discovered-from:bd-42 --json +bd update bd-42 --claim +bd ready +``` + +### End Session + +```bash +bd dolt push +``` + +## Best Practices + +1. **Keep issues visible** - Use `bd prime` to inject issue context +2. **Push regularly** - Run `bd dolt push` after significant changes +3. **Use discovered-from** - Track issues found during work +4. **Document context** - Include descriptions in issues + +## Example Workflow + +```bash +# 1. Check ready work +bd ready + +# 2. Start aider with issue context +aider --message "Working on bd-42: Fix auth bug" + +# 3. Work in aider... + +# 4. Create discovered issues +bd create "Found related bug" --deps discovered-from:bd-42 --json + +# 5. Complete and push +bd close bd-42 --reason "Fixed" +bd dolt push +``` + +## Troubleshooting + +### Config not loading + +```bash +# Check config exists +cat .aider.conf.yml + +# Regenerate +bd setup aider +``` + +### Issues not visible + +```bash +# Use bd prime to inject issue context +bd prime | aider --message-file - + +# Or check database health +bd doctor +``` + +## See Also + +- [Claude Code](/integrations/claude-code) +- [IDE Setup](/getting-started/ide-setup) diff --git a/docs/ADO_CONFIG.md b/docs/integrations/azure-devops.md similarity index 99% rename from docs/ADO_CONFIG.md rename to docs/integrations/azure-devops.md index c4cda9278..15043b827 100644 --- a/docs/ADO_CONFIG.md +++ b/docs/integrations/azure-devops.md @@ -1,4 +1,6 @@ -# Azure DevOps (ADO) Integration Configuration +--- +title: Azure DevOps (ADO) Integration Configuration +--- Last reviewed: 2026-05-08 diff --git a/docs/integrations/claude-code-plugin.md b/docs/integrations/claude-code-plugin.md new file mode 100644 index 000000000..8d9c8dbed --- /dev/null +++ b/docs/integrations/claude-code-plugin.md @@ -0,0 +1,392 @@ +--- +title: Beads Claude Code Plugin +--- + +AI-supervised issue tracker for coding workflows. Manage tasks, discover work, and maintain context with simple slash commands and MCP tools. + +## What is Beads? + +Beads (`bd`) is an issue tracker designed specifically for AI-supervised coding workflows. It helps AI agents and developers: +- Track work with a simple CLI +- Discover and link related tasks during development +- Maintain context across coding sessions +- Sync issues via Dolt remotes for distributed workflows + +## Installation + +### Prerequisites + +1. Install beads CLI: +```bash +curl -sSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash +``` + +2. Install Python and uv (for MCP server): +```bash +# Install uv (if not already installed) +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +### Install Plugin + +There are two ways to install the beads plugin: + +#### Option 1: From GitHub (Recommended) + +```bash +# In Claude Code +/plugin marketplace add gastownhall/beads +/plugin install beads +``` + +#### Option 2: Local Development + +```bash +# Clone the repository (shell command) +git clone https://github.com/gastownhall/beads +cd beads +``` + +Then in Claude Code: + +``` +# Add local marketplace (Claude Code command) +/plugin marketplace add ./beads + +# Install plugin +/plugin install beads +``` + +**Note:** If you want to install the plugin from a different repo, first `cd` to that repo's directory in your terminal, then use `./beads` (or the relative path to the beads directory) in Claude Code. + +### Restart Claude Code + +After installation, restart Claude Code to activate the MCP server. + +## Quick Start + +```bash +# Initialize beads in your project +/beads:init + +# Create your first issue +/beads:create "Set up project structure" feature 1 + +# See what's ready to work on +/beads:ready + +# Show full workflow guide +/beads:workflow +``` + +## Available Commands + +### Version Management + +- **`/beads:version`** - Check bd CLI, plugin, and MCP server versions + +### Core Workflow Commands + +- **`/beads:ready`** - Find tasks with no blockers, ready to work on +- **`/beads:create [title] [type] [priority]`** - Create a new issue interactively +- **`/beads:show [issue-id]`** - Show detailed information about an issue +- **`/beads:update [issue-id] [status]`** - Update issue status or other fields +- **`/beads:close [issue-id] [reason]`** - Close a completed issue + +### Project Management + +- **`/beads:init`** - Initialize beads in the current project +- **`/beads:workflow`** - Show the AI-supervised issue workflow guide +- **`/beads:stats`** - Show project statistics and progress + +### Agents + +- **`@task-agent`** - Autonomous agent that finds and completes ready tasks + +## MCP Tools Available + +The plugin includes a full-featured MCP server with these tools: + +- **`init`** - Initialize bd in current directory +- **`create`** - Create new issue (bug, feature, task, epic, chore) +- **`list`** - List issues with filters (status, priority, type, assignee) +- **`ready`** - Find tasks with no blockers ready to work on +- **`show`** - Show detailed issue info including dependencies +- **`update`** - Update issue (status, priority, design, notes, etc) +- **`close`** - Close completed issue +- **`dep`** - Add dependency (blocks, related, parent-child, discovered-from) +- **`blocked`** - Get blocked issues +- **`stats`** - Get project statistics + +### MCP Resources + +- **`beads://quickstart`** - Interactive quickstart guide + +## Workflow + +The beads workflow is designed for AI agents but works great for humans too: + +1. **Find ready work**: `/beads:ready` +2. **Claim your task**: `/beads:update in_progress` +3. **Work on it**: Implement, test, document +4. **Discover new work**: Create issues for bugs/TODOs found during work +5. **Complete**: `/beads:close "Done: "` +6. **Repeat**: Check for newly unblocked tasks + +## Issue Types + +- **`bug`** - Something broken that needs fixing +- **`feature`** - New functionality +- **`task`** - Work item (tests, docs, refactoring) +- **`epic`** - Large feature composed of multiple issues +- **`chore`** - Maintenance work (dependencies, tooling) + +## Priority Levels + +- **`0`** - Critical (security, data loss, broken builds) +- **`1`** - High (major features, important bugs) +- **`2`** - Medium (nice-to-have features, minor bugs) +- **`3`** - Low (polish, optimization) +- **`4`** - Backlog (future ideas) + +## Dependency Types + +- **`blocks`** - Hard dependency (issue X blocks issue Y from starting) +- **`related`** - Soft relationship (issues are connected) +- **`parent-child`** - Epic/subtask relationship +- **`discovered-from`** - Track issues discovered during work + +Only `blocks` dependencies affect the ready work queue. + +## Configuration + +### Auto-Approval Configuration + +By default, Claude Code asks for confirmation every time the beads MCP server wants to run a command. This is a security feature, but it can disrupt workflow during active development. + +**Available Options:** + +#### 1. Auto-Approve All Beads Tools (Recommended for Trusted Projects) + +Add to your Claude Code `settings.json`: + +```json +{ + "enabledMcpjsonServers": ["beads"] +} +``` + +This auto-approves all beads commands without prompting. + +#### 2. Auto-Approve Project MCP Servers + +Add to your Claude Code `settings.json`: + +```json +{ + "enableAllProjectMcpServers": true +} +``` + +This auto-approves all MCP servers defined in your project's `.mcp.json` file. Useful when working across multiple projects with different MCP requirements. + +#### 3. Manual Approval (Default) + +No configuration needed. Claude Code will prompt for approval on each MCP tool invocation. + +**Security Trade-offs:** + +- **Manual approval (default)**: Maximum safety, but interrupts workflow frequently +- **Server-level auto-approval**: Convenient for trusted projects, but allows any beads operation without confirmation +- **Project-level auto-approval**: Good balance for multi-project workflows with project-specific trust levels + +**Limitation:** Claude Code doesn't currently support per-tool approval granularity. You cannot auto-approve only read operations (like `bd ready`, `bd show`) while requiring confirmation for mutations (like `bd create`, `bd update`). It's all-or-nothing at the server level. + +**Recommended Configuration:** + +For active development on trusted projects where you're frequently using beads: + +```json +{ + "enabledMcpjsonServers": ["beads"] +} +``` + +For more information, see the [Claude Code settings documentation](https://docs.claude.com/en/docs/claude-code/settings). + +### Environment Variables + +The MCP server supports these environment variables: + +- **`BEADS_PATH`** - Path to bd executable (default: `bd` in PATH) +- **`BEADS_DB`** - Path to beads database file (default: auto-discover from cwd) +- **`BEADS_ACTOR`** - Actor name for audit trail (default: `$USER`) +- **`BEADS_NO_AUTO_FLUSH`** - Disable automatic sync (default: `false`) +- **`BEADS_NO_AUTO_IMPORT`** - Disable automatic import (default: `false`) + +To customize, edit your Claude Code MCP settings or the plugin configuration. + +## Examples + +### Basic Task Management + +```bash +# Create a high-priority bug +/beads:create "Fix authentication" bug 1 + +# See ready work +/beads:ready + +# Start working on bd-10 +/beads:update bd-10 in_progress + +# Complete the task +/beads:close bd-10 "Fixed auth token validation" +``` + +### Discovering Work During Development + +```bash +# Working on bd-10, found a related bug +/beads:create "Add rate limiting to API" feature 2 + +# Link it to current work (via MCP tool) +# Use `dep` tool: issue="bd-11", depends_on="bd-10", type="discovered-from" + +# Close original task +/beads:close bd-10 "Done, discovered bd-11 for rate limiting" +``` + +### Using the Task Agent + +```bash +# Let the agent find and complete ready work +@task-agent + +# The agent will: +# 1. Find ready work with `ready` tool +# 2. Claim a task by updating status +# 3. Execute the work +# 4. Create issues for discoveries +# 5. Close when complete +# 6. Repeat +``` + +## Auto-Sync with Dolt + +Beads automatically commits changes to Dolt history after every write operation. This enables seamless collaboration: + +```bash +# Make changes +bd create "Add feature" -p 1 + +# Changes are automatically committed to Dolt history +# Sync with remotes when ready: +bd dolt push + +# Pull changes from collaborators: +bd dolt pull +bd ready # Shows issues ready to work on (with fresh data) +``` + +## Updating + +The beads plugin has three components that may need updating: + +### 1. Plugin Updates + +Check for plugin updates: +```bash +/plugin update beads +``` + +Claude Code will pull the latest version from GitHub. After updating, **restart Claude Code** to apply MCP server changes. + +### 2. bd CLI Updates + +The plugin requires the `bd` CLI to be installed. Update it separately: + +```bash +# Quick update +curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash + +# Or with Go (server-mode only) +CGO_ENABLED=0 go install github.com/steveyegge/beads/cmd/bd@latest + +# Or with Go (embedded-capable) +CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest +``` + +### 3. Version Compatibility + +The MCP server **automatically checks** bd CLI version on startup and will fail with a clear error if your version is too old. + +Check version compatibility manually: +```bash +/beads:version +``` + +This will show: +- bd CLI version +- Plugin version +- MCP server status +- Compatibility warnings if versions mismatch + +**Recommended update workflow:** +1. Check versions: `/beads:version` +2. Update bd CLI if needed (see above) +3. Update plugin: `/plugin update beads` +4. Restart Claude Code +5. Verify: `/beads:version` + +### Version Numbering + +Beads follows semantic versioning. The plugin version tracks the bd CLI version: +- Plugin 0.9.2 requires bd CLI >= 0.9.0 (checked automatically at startup) +- Major version bumps may introduce breaking changes +- Check CHANGELOG.md for release notes + +## Troubleshooting + +### Plugin not appearing + +1. Check installation: `/plugin list` +2. Restart Claude Code +3. Verify `bd` is in PATH: `which bd` +4. Check uv is installed: `which uv` + +### MCP server not connecting + +1. Check MCP server list: `/mcp` +2. Look for "beads" server with plugin indicator +3. Restart Claude Code to reload MCP servers +4. Check logs for errors + +### Commands not working + +1. Make sure you're in a project with beads initialized: `/beads:init` +2. Check if database exists: `ls -la .beads/` +3. Try direct MCP tool access instead of slash commands +4. Check the beads CLI works: `bd --help` + +### MCP tool errors + +1. Verify `bd` executable location: `BEADS_PATH` env var +2. Check `bd` works in terminal: `bd stats` +3. Review MCP server logs in Claude Code +4. Try reinitializing: `/beads:init` + +## Learn More + +- **GitHub**: https://github.com/gastownhall/beads +- **Documentation**: See README.md in the repository +- **Examples**: Check `examples/` directory for integration patterns +- **MCP Server**: See `integrations/beads-mcp/` for server details + +## Contributing + +Found a bug or have a feature idea? Create an issue in the beads repository! + +## License + +MIT License - see LICENSE file in the repository. diff --git a/docs/integrations/claude-code.md b/docs/integrations/claude-code.md new file mode 100644 index 000000000..a8eb29bcd --- /dev/null +++ b/docs/integrations/claude-code.md @@ -0,0 +1,181 @@ +--- +title: Claude Code +--- + +How to use beads with Claude Code. + +## Setup + +### Quick Setup + +```bash +bd setup claude +``` + +This installs: +- **SessionStart hook** - Runs `bd prime` on session start +- **SessionStart compact refresh** - Runs `bd prime` after context compaction + +### Manual Setup + +Add to your Claude Code hooks configuration: + +```json +{ + "hooks": { + "SessionStart": ["bd prime"], + "SessionStart": ["bd prime"] + } +} +``` + +### Verify Setup + +```bash +bd setup claude --check +``` + +## How It Works + +1. **Session starts** → `bd prime` injects ~1-2k tokens of context +2. **You work** → Use `bd` CLI commands directly +3. **Session compacts** → `bd prime` refreshes workflow context before compaction +4. **Session ends** → Changes synced via git + +## Essential Commands for Agents + +### Creating Issues + +```bash +# Always include description for context +bd create "Fix authentication bug" \ + --description="Login fails with special characters in password" \ + -t bug -p 1 --json + +# Link discovered issues +bd create "Found SQL injection" \ + --description="User input not sanitized in query builder" \ + --deps discovered-from:bd-42 --json +``` + +### Working on Issues + +```bash +# Find ready work +bd ready --json + +# Start work +bd update bd-42 --claim --json + +# Complete work +bd close bd-42 --reason "Fixed in commit abc123" --json +``` + +### Querying + +```bash +# List open issues +bd list --status open --json + +# Show issue details +bd show bd-42 --json + +# Check blocked issues +bd blocked --json +``` + +### Syncing + +```bash +# ALWAYS run at session end +bd dolt push +``` + +## Best Practices + +### Always Use `--json` + +```bash +bd list --json # Parse programmatically +bd create "Task" --json # Get issue ID from output +bd show bd-42 --json # Structured data +``` + +### Always Include Descriptions + +```bash +# Good +bd create "Fix auth bug" \ + --description="Login fails when password contains quotes" \ + -t bug -p 1 --json + +# Bad - no context for future work +bd create "Fix auth bug" -t bug -p 1 --json +``` + +### Link Related Work + +```bash +# When you discover issues during work +bd create "Found related bug" \ + --deps discovered-from:bd-current --json +``` + +### Push Before Session End + +```bash +# ALWAYS run before ending +bd dolt push +``` + +## Plugin (Optional) + +For enhanced UX with slash commands: + +```bash +# In Claude Code +/plugin marketplace add gastownhall/beads +/plugin install beads +# Restart Claude Code +``` + +Adds slash commands: +- `/beads:ready` - Show ready work +- `/beads:create` - Create issue +- `/beads:show` - Show issue +- `/beads:update` - Update issue +- `/beads:close` - Close issue + +## Troubleshooting + +### Context not injected + +```bash +# Check hook setup +bd setup claude --check + +# Manually prime +bd prime +``` + +### Changes not syncing + +```bash +# Force push +bd dolt push + +# Check system health +bd doctor +``` + +### Database not found + +```bash +# Initialize beads +bd init --quiet +``` + +## See Also + +- [MCP Server](/integrations/mcp-server) - For MCP-only environments +- [IDE Setup](/getting-started/ide-setup) - Other editors diff --git a/docs/integrations/codex.md b/docs/integrations/codex.md new file mode 100644 index 000000000..bd5ec0965 --- /dev/null +++ b/docs/integrations/codex.md @@ -0,0 +1,32 @@ +--- +title: Codex +--- + +Use Beads with Codex through the `beads` skill, managed `AGENTS.md` guidance, and native Codex hooks. + +```bash +bd setup codex +bd setup codex --check +``` + +Project setup writes: + +- `.agents/skills/beads/` for the Beads skill. +- `AGENTS.md` with a managed Beads section. +- `.codex/config.toml` with `[features].hooks = true`. +- `.codex/hooks.json` with the Beads hook fallback. + +`bd init` runs this project setup by default unless `--skip-agents` or `--stealth` is used. Global setup uses `bd setup codex --global` and writes under `$CODEX_HOME` when set, otherwise `~/.codex`. + +Codex 0.129.0+ supports `/hooks`, compact lifecycle hooks, and hook-provided developer context. Beads uses that lifecycle to inject `bd prime` on session start and recover context after compaction. Use `/hooks` to inspect or toggle the installed handlers. + +## Hook Lifecycle + +- `SessionStart` (`startup|resume|clear`) injects full `bd prime` output. +- `PreCompact` (`manual|auto`) checks `bd prime --memories-only` and warns if Beads context is unavailable. +- `PostCompact` (`manual|auto`) records that the session needs a Beads refresh. +- `UserPromptSubmit` injects full `bd prime` once after compaction, then clears the refresh marker. + +`PreCompact` alone does not inject context because Codex ignores plain stdout from compact hooks. The post-compact marker plus first-prompt refresh is the reliable recovery path. + +The Beads Codex plugin stores hooks at `plugins/beads/.codex-plugin/hooks/hooks.json` and declares them as `"hooks": "./hooks/hooks.json"`. Without the plugin, `bd setup codex` installs the same hook config in `.codex/hooks.json` and enables `[features].hooks = true`. diff --git a/docs/integrations/cody.md b/docs/integrations/cody.md new file mode 100644 index 000000000..33b0f16cb --- /dev/null +++ b/docs/integrations/cody.md @@ -0,0 +1,18 @@ +--- +title: Sourcegraph Cody +--- + +Use Beads with Sourcegraph Cody through a project rules file. + +```bash +bd setup cody +bd setup cody --check +``` + +The setup command creates `.cody/rules/beads.md` with Beads workflow guidance. + +## Remove + +```bash +bd setup cody --remove +``` diff --git a/docs/COPILOT_CLI_INTEGRATION.md b/docs/integrations/copilot-cli.md similarity index 96% rename from docs/COPILOT_CLI_INTEGRATION.md rename to docs/integrations/copilot-cli.md index 2ea6c5abd..1a7e7fb5b 100644 --- a/docs/COPILOT_CLI_INTEGRATION.md +++ b/docs/integrations/copilot-cli.md @@ -1,8 +1,10 @@ -# GitHub Copilot CLI Integration Design +--- +title: GitHub Copilot CLI Integration Design +--- This document explains design decisions for GitHub Copilot CLI integration in beads. -For **VS Code + MCP**, see [COPILOT_INTEGRATION.md](COPILOT_INTEGRATION.md). +For **VS Code + MCP**, see [COPILOT_INTEGRATION.md](/integrations/github-copilot). ## Integration Approach diff --git a/docs/integrations/cursor.md b/docs/integrations/cursor.md new file mode 100644 index 000000000..2fa4764a1 --- /dev/null +++ b/docs/integrations/cursor.md @@ -0,0 +1,149 @@ +--- +title: Cursor +--- + +Use Beads with Cursor through a project rules file **and** agent lifecycle hooks. +The hooks are what keep Beads context alive across compaction — without them, +Cursor only sees the static rules file and the agent must remember to run +`bd prime` itself (which it often forgets after a context compaction). + +```bash +bd setup cursor +bd setup cursor --check +``` + +`bd setup cursor` installs three things: + +- **`.cursor/rules/beads.mdc`** — an always-applied rule with the canonical Beads + workflow guidance (the same content every other editor integration uses), so it + stays in sync as the workflow evolves. Because it is always applied, Cursor + re-includes it every turn, including after a compaction. +- **`.agents/skills/beads/SKILL.md`** — the Beads agent skill, which Cursor loads + natively from `.agents/skills/` (the same skill the Codex integration installs). + It gives the agent an on-demand, progressive-disclosure reference for `bd`. +- **`.cursor/hooks.json`** — agent hooks that run the hidden `bd cursor-hook` + command at key points in the session. Existing (non-Beads) hooks in this file + are preserved. + +The managed hooks are: + +| Event | What it does | +|-------|--------------| +| `sessionStart` | Injects full `bd prime` output so the agent starts every session already knowing the workflow, ready work, and project memories. | +| `preCompact` | Arms a one-shot marker and shows a short reminder that context will be restored. | +| `postToolUse` | After a compaction, re-injects `bd prime` exactly once on the next tool call, then no-ops. | + + +**Requirements** +Hooks run in the Cursor IDE and in recent `cursor-agent` CLI builds (verified on +the 2026.06 line; early-2026 CLI builds only fired shell hooks). After setup, +restart Cursor or start a **new** `cursor-agent` session so the hooks load. + +The hook entries call `bd cursor-hook`, which Cursor resolves on `PATH` — exactly +like the Claude Code and Codex hooks. **But that `PATH` is not your shell's:** +Cursor rebuilds it for hook subprocesses and prepends its own directories, so a +stale `bd` earlier in *that* order silently wins and every hook fails with no +visible error. The classic case is an old Homebrew `bd` (without the +`cursor-hook` command) shadowing a newer build. Make sure the first `bd` Cursor +finds is current — `~/.local/bin` and `~/go/bin` sort ahead of Homebrew in +Cursor's PATH, so installing or symlinking `bd` there is a reliable way to win +resolution. + +In the CLI, the workspace must also be **trusted** — hooks do not run in an +untrusted workspace. + + + +**Auto-installed by `bd init`** +You usually don't need to run `bd setup cursor` by hand. `bd init` installs the +Cursor integration automatically (the same way it sets up Claude Code and +Codex), and stages `.cursor/` so the rules + hooks are committed with your repo. +Run `bd setup cursor` only to (re)install into an existing project or to repair +drift. + + +## Global install (all projects) + +To wire the hooks once for every project instead of per-repo: + +```bash +bd setup cursor --global # writes ~/.cursor/hooks.json +bd setup cursor --check --global +bd setup cursor --remove --global +``` + +Global scope installs the **hooks and the agent skill** (`~/.agents/skills/beads`), +but not a rules file. Cursor has no reliable file-based global *rules* location — +global rules belong in **Cursor Settings → Rules** — so `.cursor/rules/beads.mdc` +is project-scoped. This is fine in practice: the global `sessionStart` hook +already injects the full `bd prime` context into every project, which supersedes +the static rule. + +## Why these specific hooks + +The Claude Code / Codex integrations recover from compaction with a dedicated +post-compaction event plus a prompt-submission hook. Cursor's +[hook API](https://cursor.com/docs/hooks) doesn't offer equivalents, so the +recovery is built from what Cursor *can* do: + +- Cursor has **no post-compaction hook** — only `preCompact`, which is + observational and "cannot block or modify the compaction behavior" (its only + output is `user_message`). So `preCompact` just arms a one-shot marker and + notifies you. +- Only **two** hooks can inject model-visible context (`additional_context`): + `sessionStart` and `postToolUse`. `sessionStart` fires only when a new + conversation is created — not after a compaction, and not when resuming with + `cursor-agent --continue`. `preCompact` and `postToolUse` *do* fire in resumed + sessions, so compaction recovery still works under `--continue`; only the + one-time start-of-session injection is skipped (the resumed conversation + already contains it). +- That makes `postToolUse` the only place to re-inject `bd prime` after a + compaction — which the docs explicitly support ("Useful for ... injecting + context"). + + +**Limitation** +Post-compaction recovery fires on the **next tool call**. If you have a +purely conversational turn right after a compaction (no tool use), the refresh +waits until the next tool runs — no Cursor hook can inject context on a bare +prompt. Agent sessions call tools often, so in practice this is quick. You can +always run `bd prime` manually to force a refresh. + + +## Verifying it works (CLI, no hooks pane) + +The `cursor-agent` CLI has no hooks output pane, so verify the hooks directly: + +```bash +# sessionStart should return {"continue": true, "additional_context": "...bd prime..."} +echo '{"hook_event_name":"sessionStart","workspace_roots":["'"$PWD"'"]}' \ + | bd cursor-hook sessionStart | jq . + +# In a real session, the agent should know your bd workflow without being told, +# and should recover it after a compaction without you running `bd prime`. +``` + +If `additional_context` is empty, the directory is not a Beads workspace yet +(run `bd init`) or `bd` is not on Cursor's `PATH`. Note this command runs *your +shell's* `bd`; Cursor may resolve a different one for hooks (see Requirements +above), so a passing check here doesn't guarantee Cursor runs the same build. + +`bd doctor` also reports three Cursor checks (under integration diagnostics), +paralleling the Claude checks: + +- **Cursor Integration** — flags when Cursor is in use but the hooks aren't installed. +- **Cursor Settings Health** — errors if `.cursor/hooks.json` is malformed JSON + (which would silently disable every hook). +- **Cursor Hook Completeness** — warns if only some of the three managed events + are installed (e.g. after a hand-edit), since recovery needs all three. + +## Remove + +```bash +bd setup cursor --remove +``` + +This removes the rules file and the Beads-managed hook entries while leaving any +of your own hooks in `.cursor/hooks.json` intact. The shared agent skill +(`.agents/skills/beads`) is removed too — unless the Codex integration is still +installed, in which case it is kept because both integrations use it. diff --git a/docs/integrations/factory.md b/docs/integrations/factory.md new file mode 100644 index 000000000..f7301732e --- /dev/null +++ b/docs/integrations/factory.md @@ -0,0 +1,18 @@ +--- +title: Factory.ai Droid +--- + +Use Beads with Factory.ai Droid through managed `AGENTS.md` guidance. + +```bash +bd setup factory +bd setup factory --check +``` + +The setup command creates or updates `AGENTS.md` with a managed Beads section. Factory Droid reads `AGENTS.md` automatically when it starts a session. + +## Remove + +```bash +bd setup factory --remove +``` diff --git a/docs/integrations/gemini.md b/docs/integrations/gemini.md new file mode 100644 index 000000000..712453b03 --- /dev/null +++ b/docs/integrations/gemini.md @@ -0,0 +1,34 @@ +--- +title: Gemini CLI +--- + +Use Beads with Gemini CLI through SessionStart hooks and `GEMINI.md` guidance. + +```bash +bd setup gemini +bd setup gemini --check +``` + +By default, setup installs global hooks in `~/.gemini/settings.json`. For project-local hooks, use: + +```bash +bd setup gemini --project +``` + +The hook runs `bd prime --hook-json` so Gemini receives compact Beads workflow context at session start. The setup also writes Beads guidance to `GEMINI.md`. + +## Stealth Mode + +For CI or other environments where setup should avoid git operations: + +```bash +bd setup gemini --stealth +bd setup gemini --project --stealth +``` + +## Remove + +```bash +bd setup gemini --remove +bd setup gemini --project --remove +``` diff --git a/docs/integrations/github-copilot.md b/docs/integrations/github-copilot.md new file mode 100644 index 000000000..b776c1c28 --- /dev/null +++ b/docs/integrations/github-copilot.md @@ -0,0 +1,141 @@ +--- +title: GitHub Copilot +--- + +How to use beads with GitHub Copilot in VS Code. + +## Setup + +### Quick Setup + +1. Install beads-mcp: + ```bash + uv tool install beads-mcp + ``` + +2. Create `.vscode/mcp.json` in your project: + ```json + { + "servers": { + "beads": { + "command": "beads-mcp" + } + } + } + ``` + + **For all projects:** Add to VS Code user-level MCP config: + + | Platform | Path | + |----------|------| + | macOS | `~/Library/Application Support/Code/User/mcp.json` | + | Linux | `~/.config/Code/User/mcp.json` | + | Windows | `%APPDATA%\Code\User\mcp.json` | + + ```json + { + "servers": { + "beads": { + "command": "beads-mcp", + "args": [] + } + } + } + ``` + +3. Initialize beads: + ```bash + bd init --quiet + ``` + +4. Reload VS Code + +### Verify Setup + +Ask Copilot Chat: "What beads issues are ready to work on?" + +## Using Natural Language + +With MCP configured, interact naturally: + +``` +You: Create a bug for the login timeout +Copilot: Created bd-42: Login timeout bug + +You: What issues are ready? +Copilot: 3 issues ready: bd-42, bd-99, bd-17 + +You: Close bd-42, it's fixed +Copilot: Closed bd-42 +``` + +## MCP Tools + +| Tool | Description | +|------|-------------| +| `beads_ready` | List unblocked issues | +| `beads_create` | Create new issue | +| `beads_show` | Show issue details | +| `beads_update` | Update issue | +| `beads_close` | Close issue | +| `beads_dolt_push` | Push to Dolt remote | +| `beads_dep_add` | Add dependency | +| `beads_dep_tree` | Show dependency tree | + +## Copilot Instructions + +Optionally add `.github/copilot-instructions.md`: + +```markdown +## Issue Tracking + +This project uses **bd (beads)** for issue tracking. +Run `bd prime` for workflow context. + +Quick reference: +- `bd ready` - Find unblocked work +- `bd create "Title" --type task --priority 2` - Create issue +- `bd close ` - Complete work +- `bd dolt push` - Push changes to Dolt remote +``` + +## Troubleshooting + +### Tools not appearing + +1. Check VS Code 1.96+ +2. Verify mcp.json syntax is valid JSON +3. Reload VS Code window +4. Check Output panel for MCP errors + +### "beads-mcp not found" + +```bash +# Check installation +which beads-mcp + +# Reinstall if needed +uv tool install beads-mcp --force +``` + +### No database found + +```bash +bd init --quiet +``` + +## FAQ + +### Do I need to clone beads? + +**No.** Beads is a system-wide CLI tool. Install once, use everywhere. The `.beads/` directory in your project only contains the issue database. + +### What about git hooks? + +Git hooks are optional. They refresh exports and legacy fallback checks, while issue sync uses `bd dolt push` / `bd dolt pull`. + +## See Also + +- [MCP Server](/integrations/mcp-server) - Detailed MCP configuration +- [Installation](/getting-started/installation) - Full install guide +- [Detailed Copilot Guide](https://github.com/gastownhall/beads/blob/main/docs/COPILOT_INTEGRATION.md) - Comprehensive documentation diff --git a/docs/integrations/index.md b/docs/integrations/index.md new file mode 100644 index 000000000..23e2f295f --- /dev/null +++ b/docs/integrations/index.md @@ -0,0 +1,38 @@ +--- +title: Integrations +--- + +Beads integration pages are based on two sources of support in the repository: + +- Built-in `bd setup` recipes from `internal/recipes/recipes.go` +- First-party MCP integrations and editor guides already shipped in the repo + +Run this command to see the built-in setup recipes supported by your installed `bd` binary: + +```bash +bd setup --list +``` + +## Built-in Setup Recipes + +| Recipe | Integration | Primary setup surface | +|--------|-------------|-----------------------| +| `aider` | [Aider](/integrations/aider) | `.aider.conf.yml` and `.aider/` instructions | +| `claude` | [Claude Code](/integrations/claude-code) | Claude hooks and `CLAUDE.md` | +| `codex` | [Codex](/integrations/codex) | Beads skill, `AGENTS.md`, and Codex hooks | +| `cody` | [Sourcegraph Cody](/integrations/cody) | `.cody/rules/beads.md` | +| `cursor` | [Cursor](/integrations/cursor) | `.cursor/rules/beads.mdc` + `.cursor/hooks.json` | +| `factory` | [Factory.ai Droid](/integrations/factory) | `AGENTS.md` | +| `gemini` | [Gemini CLI](/integrations/gemini) | Gemini hooks and `GEMINI.md` | +| `junie` | [Junie](/integrations/junie) | `.junie/guidelines.md` and MCP config | +| `kilocode` | [Kilo Code](/integrations/kilocode) | `.kilocode/rules/beads.md` | +| `mux` | [Mux](/integrations/mux) | `AGENTS.md`, optional `.mux/AGENTS.md`, and Mux hooks | +| `opencode` | [OpenCode](/integrations/opencode) | `AGENTS.md` | +| `windsurf` | [Windsurf](/integrations/windsurf) | `.windsurf/rules/beads.md` | + +## MCP-Based Integrations + +These integrations use the Beads MCP server rather than a dedicated `bd setup` recipe: + +- [MCP Server](/integrations/mcp-server) +- [GitHub Copilot](/integrations/github-copilot) diff --git a/docs/integrations/junie.md b/docs/integrations/junie.md new file mode 100644 index 000000000..d75676282 --- /dev/null +++ b/docs/integrations/junie.md @@ -0,0 +1,215 @@ +--- +title: Junie +--- + +How to use beads with Junie (JetBrains AI Agent). + +## Setup + +### Quick Setup + +```bash +bd setup junie +``` + +This creates: +- **`.junie/guidelines.md`** - Agent instructions for beads workflow +- **`.junie/mcp/mcp.json`** - MCP server configuration + +### Verify Setup + +```bash +bd setup junie --check +``` + +## How It Works + +1. **Session starts** → Junie reads `.junie/guidelines.md` for workflow context +2. **MCP tools available** → Junie can use beads MCP tools directly +3. **You work** → Use `bd` CLI commands or MCP tools +4. **Session ends** → Run `bd dolt push` to push changes to Dolt remote + +## Configuration Files + +### Guidelines (`.junie/guidelines.md`) + +Contains workflow instructions that Junie reads automatically: +- Core workflow rules +- Command reference +- Issue types and priorities +- MCP tool documentation + +### MCP Config (`.junie/mcp/mcp.json`) + +Configures the beads MCP server: + +```json +{ + "mcpServers": { + "beads": { + "command": "bd", + "args": ["mcp"] + } + } +} +``` + +## MCP Tools + +With MCP configured, Junie can use these tools directly: + +| Tool | Description | +| --- | --- | +| `mcp_beads_ready` | Find tasks ready for work | +| `mcp_beads_list` | List issues with filters | +| `mcp_beads_show` | Show issue details | +| `mcp_beads_create` | Create new issues | +| `mcp_beads_update` | Update issue status/priority | +| `mcp_beads_close` | Close completed issues | +| `mcp_beads_dep` | Manage dependencies | +| `mcp_beads_blocked` | Show blocked issues | +| `mcp_beads_stats` | Get issue statistics | + +## CLI Commands + +You can also use the `bd` CLI directly: + +### Creating Issues + +```bash +# Always include description for context +bd create "Fix authentication bug" \ + --description="Login fails with special characters in password" \ + -t bug -p 1 --json + +# Link discovered issues +bd create "Found SQL injection" \ + --description="User input not sanitized in query builder" \ + --deps discovered-from:bd-42 --json +``` + +### Working on Issues + +```bash +# Find ready work +bd ready --json + +# Start work +bd update bd-42 --claim --json + +# Complete work +bd close bd-42 --reason "Fixed in commit abc123" --json +``` + +### Querying + +```bash +# List open issues +bd list --status open --json + +# Show issue details +bd show bd-42 --json + +# Check blocked issues +bd blocked --json +``` + +### Syncing + +```bash +# ALWAYS run at session end +bd dolt push +``` + +## Best Practices + +### Always Use `--json` + +```bash +bd list --json # Parse programmatically +bd create "Task" --json # Get issue ID from output +bd show bd-42 --json # Structured data +``` + +### Always Include Descriptions + +```bash +# Good +bd create "Fix auth bug" \ + --description="Login fails when password contains quotes" \ + -t bug -p 1 --json + +# Bad - no context for future work +bd create "Fix auth bug" -t bug -p 1 --json +``` + +### Link Related Work + +```bash +# When you discover issues during work +bd create "Found related bug" \ + --deps discovered-from:bd-current --json +``` + +### Push Before Session End + +```bash +# ALWAYS run before ending +bd dolt push +``` + +## Troubleshooting + +### Guidelines not loaded + +```bash +# Check setup +bd setup junie --check + +# Reinstall if needed +bd setup junie +``` + +### MCP tools not available + +```bash +# Verify MCP config exists +cat .junie/mcp/mcp.json + +# Test MCP server +bd mcp --help +``` + +### Changes not syncing + +```bash +# Force push +bd dolt push + +# Check system health +bd doctor +``` + +### Database not found + +```bash +# Initialize beads +bd init --quiet +``` + +## Removing Integration + +```bash +bd setup junie --remove +``` + +This removes: +- `.junie/guidelines.md` +- `.junie/mcp/mcp.json` +- Empty `.junie/mcp/` and `.junie/` directories + +## See Also + +- [MCP Server](/integrations/mcp-server) - MCP server details +- [Claude Code](/integrations/claude-code) - Similar hook-based integration +- [IDE Setup](/getting-started/ide-setup) - Other editors diff --git a/docs/integrations/kilocode.md b/docs/integrations/kilocode.md new file mode 100644 index 000000000..813d708e4 --- /dev/null +++ b/docs/integrations/kilocode.md @@ -0,0 +1,18 @@ +--- +title: Kilo Code +--- + +Use Beads with Kilo Code through a project rules file. + +```bash +bd setup kilocode +bd setup kilocode --check +``` + +The setup command creates `.kilocode/rules/beads.md` with Beads workflow guidance. + +## Remove + +```bash +bd setup kilocode --remove +``` diff --git a/docs/integrations/mcp-server.md b/docs/integrations/mcp-server.md new file mode 100644 index 000000000..00a343d19 --- /dev/null +++ b/docs/integrations/mcp-server.md @@ -0,0 +1,182 @@ +--- +title: MCP Server +--- + +Use beads in MCP-only environments. + +## When to Use MCP + +Use MCP server when CLI is unavailable: +- Claude Desktop (no shell access) +- Sourcegraph Amp without shell +- Other MCP-only environments + +**Prefer CLI + hooks** when shell is available - it's more context efficient. + +## Installation + +### Using uv (Recommended) + +```bash +uv tool install beads-mcp +``` + +### Using pip + +```bash +pip install beads-mcp +``` + +## Configuration + +### Claude Desktop (macOS) + +Add to `~/Library/Application Support/Claude/claude_desktop_config.json`: + +```json +{ + "mcpServers": { + "beads": { + "command": "beads-mcp" + } + } +} +``` + +### Claude Desktop (Windows) + +Add to `%APPDATA%\Claude\claude_desktop_config.json`: + +```json +{ + "mcpServers": { + "beads": { + "command": "beads-mcp" + } + } +} +``` + +### Sourcegraph Amp + +Add to MCP settings: + +```json +{ + "beads": { + "command": "beads-mcp", + "args": [] + } +} +``` + +### VS Code / GitHub Copilot + +Create `.vscode/mcp.json` in your project: + +```json +{ + "servers": { + "beads": { + "command": "beads-mcp" + } + } +} +``` + +**For all projects:** Add to VS Code user-level MCP config: + +| Platform | Path | +|----------|------| +| macOS | `~/Library/Application Support/Code/User/mcp.json` | +| Linux | `~/.config/Code/User/mcp.json` | +| Windows | `%APPDATA%\Code\User\mcp.json` | + +```json +{ + "servers": { + "beads": { + "command": "beads-mcp", + "args": [] + } + } +} +``` + +**Note:** Requires VS Code 1.96+ with MCP support enabled. + +See [GitHub Copilot Integration](/integrations/github-copilot) for complete setup guide. + +## Available Tools + +The MCP server exposes these tools: + +| Tool | Description | +|------|-------------| +| `beads_create` | Create new issue | +| `beads_list` | List issues | +| `beads_show` | Show issue details | +| `beads_update` | Update issue | +| `beads_close` | Close issue | +| `beads_ready` | Show ready work | +| `beads_sync` | Sync to git | +| `beads_dep_add` | Add dependency | +| `beads_dep_tree` | Show dependency tree | + +## Usage + +Once configured, use naturally: + +``` +Create an issue for fixing the login bug with priority 1 +``` + +The MCP server translates to appropriate `bd` commands. + +## Trade-offs + +| Aspect | CLI + Hooks | MCP Server | +|--------|-------------|------------| +| Context overhead | ~1-2k tokens | 10-50k tokens | +| Latency | Direct calls | MCP protocol | +| Setup | Hooks config | MCP config | +| Availability | Shell required | MCP environments | + +## Troubleshooting + +### Server won't start + +Check if `beads-mcp` is in PATH: + +```bash +which beads-mcp +``` + +If not found: + +```bash +# Reinstall +pip uninstall beads-mcp +pip install beads-mcp +``` + +### Tools not appearing + +1. Restart Claude Desktop +2. Check MCP config JSON syntax +3. Verify server path + +### Permission errors + +```bash +# Check directory permissions +ls -la .beads/ + +# Initialize if needed +bd init --quiet +``` + +## See Also + +- [Claude Code](/integrations/claude-code) - CLI integration +- [Installation](/getting-started/installation) - Full install guide diff --git a/docs/integrations/mux.md b/docs/integrations/mux.md new file mode 100644 index 000000000..aca4c10ba --- /dev/null +++ b/docs/integrations/mux.md @@ -0,0 +1,38 @@ +--- +title: Mux +--- + +Use Beads with Mux through `AGENTS.md`, optional layered Mux instruction files, and Mux hooks. + +```bash +bd setup mux +bd setup mux --check +``` + +The default setup writes a managed Beads section to root `AGENTS.md`. + +## Workspace and Global Layers + +Mux also supports workspace and global instruction layers: + +```bash +bd setup mux --project +bd setup mux --global +bd setup mux --project --global +``` + +Project setup writes `.mux/AGENTS.md` and installs Mux hook files under `.mux/`: + +- `.mux/init` +- `.mux/tool_post` +- `.mux/tool_env` + +Global setup writes `~/.mux/AGENTS.md`. + +## Remove + +```bash +bd setup mux --remove +bd setup mux --project --remove +bd setup mux --global --remove +``` diff --git a/docs/integrations/opencode.md b/docs/integrations/opencode.md new file mode 100644 index 000000000..3ba072864 --- /dev/null +++ b/docs/integrations/opencode.md @@ -0,0 +1,18 @@ +--- +title: OpenCode +--- + +Use Beads with OpenCode through managed `AGENTS.md` guidance. + +```bash +bd setup opencode +bd setup opencode --check +``` + +The setup command creates or updates `AGENTS.md` with a managed Beads section. Restart OpenCode after setup if it is already running. + +## Remove + +```bash +bd setup opencode --remove +``` diff --git a/docs/integrations/windsurf.md b/docs/integrations/windsurf.md new file mode 100644 index 000000000..85e05c22c --- /dev/null +++ b/docs/integrations/windsurf.md @@ -0,0 +1,18 @@ +--- +title: Windsurf +--- + +Use Beads with Windsurf through a project rules file. + +```bash +bd setup windsurf +bd setup windsurf --check +``` + +The setup command creates `.windsurf/rules/beads.md` with Beads workflow guidance. + +## Remove + +```bash +bd setup windsurf --remove +``` diff --git a/docs/multi-agent/coordination.md b/docs/multi-agent/coordination.md new file mode 100644 index 000000000..16ec4a277 --- /dev/null +++ b/docs/multi-agent/coordination.md @@ -0,0 +1,153 @@ +--- +title: Agent Coordination +--- + +Patterns for coordinating work between multiple AI agents. + +## Work Assignment + +### Pinning Work + +Assign work to a specific agent: + +```bash +# Pin issue to agent +bd pin bd-42 --for agent-1 + +# Pin and start work +bd pin bd-42 --for agent-1 --start + +# Unpin work +bd unpin bd-42 +``` + +### Checking Pinned Work + +```bash +# What's on my hook? +bd hook + +# What's on agent-1's hook? +bd hook --agent agent-1 + +# JSON output +bd hook --json +``` + +## Handoff Patterns + +### Sequential Handoff + +Agent A completes work, hands off to Agent B: + +```bash +# Agent A +bd close bd-42 --reason "Ready for review" +bd pin bd-42 --for agent-b + +# Agent B picks up +bd hook # Sees bd-42 +bd update bd-42 --claim +``` + +### Parallel Work + +Multiple agents work on different issues: + +```bash +# Coordinator +bd pin bd-42 --for agent-a --start +bd pin bd-43 --for agent-b --start +bd pin bd-44 --for agent-c --start + +# Each agent works independently +# Coordinator monitors progress +bd list --status in_progress --json +``` + +### Fan-Out / Fan-In + +Split work, then merge: + +```bash +# Fan-out +bd create "Part A" --parent bd-epic +bd create "Part B" --parent bd-epic +bd create "Part C" --parent bd-epic + +bd pin bd-epic.1 --for agent-a +bd pin bd-epic.2 --for agent-b +bd pin bd-epic.3 --for agent-c + +# Fan-in: wait for all parts +bd dep add bd-merge bd-epic.1 bd-epic.2 bd-epic.3 +``` + +## Agent Discovery + +Find available agents: + +```bash +# List known agents (if using agent registry) +bd agents list + +# Check agent status +bd agents status agent-1 +``` + +## Conflict Prevention + +### File Reservations + +Prevent concurrent edits: + +```bash +# Reserve files before editing +bd reserve auth.go --for agent-1 + +# Check reservations +bd reservations list + +# Release when done +bd reserve --release auth.go +``` + +### Issue Locking + +```bash +# Lock issue for exclusive work +bd lock bd-42 --for agent-1 + +# Unlock when done +bd unlock bd-42 +``` + +## Communication Patterns + +### Via Comments + +```bash +# Agent A leaves note +bd comment add bd-42 "Completed API, needs frontend integration" + +# Agent B reads +bd show bd-42 --full +``` + +### Via Labels + +```bash +# Mark for review +bd update bd-42 --add-label "needs-review" + +# Agent B filters +bd list --label-any needs-review +``` + +## Best Practices + +1. **Clear ownership** - Always pin work to specific agent +2. **Document handoffs** - Use comments to explain context +3. **Use labels for status** - `needs-review`, `blocked`, `ready` +4. **Avoid conflicts** - Use reservations for shared files +5. **Monitor progress** - Regular status checks diff --git a/docs/FEDERATION.md b/docs/multi-agent/federation.md similarity index 98% rename from docs/FEDERATION.md rename to docs/multi-agent/federation.md index 466e852be..fdeae30eb 100644 --- a/docs/FEDERATION.md +++ b/docs/multi-agent/federation.md @@ -1,4 +1,6 @@ -# Federation Setup Guide +--- +title: Federation Setup Guide +--- Federation enables peer-to-peer synchronization of beads databases between multiple workspaces using Dolt remotes. Each workspace maintains its own database @@ -214,7 +216,7 @@ bd doctor --deep ## Reference -- Configuration: See [CONFIG.md](CONFIG.md) for all federation settings +- Configuration: See [CONFIG.md](/reference/configuration) for all federation settings - Source: `cmd/bd/federation.go` - Storage interfaces: `internal/storage/versioned.go` - Dolt implementation: `internal/storage/dolt/store.go` diff --git a/docs/multi-agent/index.md b/docs/multi-agent/index.md new file mode 100644 index 000000000..270fab6db --- /dev/null +++ b/docs/multi-agent/index.md @@ -0,0 +1,68 @@ +--- +title: Multi-Agent +--- + +Beads supports coordination between multiple AI agents and repositories. + +## Overview + +Multi-agent features enable: +- **Routing** - Automatic issue routing to correct repositories +- **Cross-repo dependencies** - Dependencies across repository boundaries +- **Agent coordination** - Work assignment and handoff between agents + +## Key Concepts + +### Routes + +Routes define which repository handles which issues: + +```jsonl +{"pattern": "frontend/*", "target": "frontend-repo"} +{"pattern": "backend/*", "target": "backend-repo"} +{"pattern": "*", "target": "main-repo"} +``` + +### Work Assignment + +Pin work to specific agents: + +```bash +bd pin bd-42 --for agent-1 --start +bd hook --agent agent-1 # Show pinned work +``` + +### Cross-repo Dependencies + +Track dependencies across repositories: + +```bash +bd dep add bd-42 external:other-repo/bd-100 +``` + +## Architecture + +``` +┌─────────────────┐ +│ Main Repo │ +│ (coordinator) │ +└────────┬────────┘ + │ routes + ┌────┴────┐ + │ │ +┌───▼───┐ ┌───▼───┐ +│Frontend│ │Backend│ +│ Repo │ │ Repo │ +└────────┘ └────────┘ +``` + +## Getting Started + +1. **Single repo**: Standard beads workflow +2. **Multi-repo**: Configure routes and cross-repo deps +3. **Multi-agent**: Add work assignment and handoff + +## Navigation + +- [Routing](/multi-agent/routing) - Auto-routing configuration +- [Coordination](/multi-agent/coordination) - Agent coordination patterns diff --git a/docs/MULTI_REPO_MIGRATION.md b/docs/multi-agent/multi-repo-migration.md similarity index 95% rename from docs/MULTI_REPO_MIGRATION.md rename to docs/multi-agent/multi-repo-migration.md index da6fb7704..3588f587e 100644 --- a/docs/MULTI_REPO_MIGRATION.md +++ b/docs/multi-agent/multi-repo-migration.md @@ -1,4 +1,6 @@ -# Multi-Repo Migration Guide +--- +title: Multi-Repo Migration Guide +--- This guide helps you adopt beads' multi-repo workflow for OSS contributions, team collaboration, and multi-phase development. @@ -10,7 +12,7 @@ This guide helps you adopt beads' multi-repo workflow for OSS contributions, tea - [Multi-Phase Development](#multi-phase-development) - Separate repos per phase - [Multiple Personas](#multiple-personas) - Architect vs. implementer separation -**New to beads?** See [QUICKSTART.md](QUICKSTART.md) first. +**New to beads?** See [QUICKSTART.md](/getting-started/quickstart) first. ## What is Multi-Repo Mode? @@ -396,7 +398,7 @@ bd list --json **Problem:** Multiple repos with conflicting changes. -**Solution:** Dolt handles merge conflicts natively with cell-level merge. See [TROUBLESHOOTING.md](TROUBLESHOOTING.md#merge-conflicts) for details. +**Solution:** Dolt handles merge conflicts natively with cell-level merge. See [TROUBLESHOOTING.md](/reference/troubleshooting#merge-conflicts) for details. ### Discovered issues in wrong repository @@ -472,10 +474,10 @@ bd create "Issue" -p 1 ## Next Steps -- **CLI Reference:** See [README.md](../README.md) for command details -- **Configuration Guide:** See [CONFIG.md](CONFIG.md) for all config options -- **Troubleshooting:** See [TROUBLESHOOTING.md](TROUBLESHOOTING.md) -- **Multi-Repo Internals:** See [ROUTING.md#multi-repo-hydration](ROUTING.md#multi-repo-hydration) +- **CLI Reference:** See [README.md](https://github.com/gastownhall/beads/blob/main/README.md) for command details +- **Configuration Guide:** See [CONFIG.md](/reference/configuration) for all config options +- **Troubleshooting:** See [TROUBLESHOOTING.md](/reference/troubleshooting) +- **Multi-Repo Internals:** See [ROUTING.md#multi-repo-hydration](/multi-agent/routing#multi-repo-hydration) ## Related Issues diff --git a/docs/multi-agent/routing.md b/docs/multi-agent/routing.md new file mode 100644 index 000000000..5b18374e3 --- /dev/null +++ b/docs/multi-agent/routing.md @@ -0,0 +1,112 @@ +--- +title: Routing +--- + +Automatic issue routing across repositories. + +## Overview + +Routing enables: +- Issues created in one repo routed to another +- Pattern-based routing rules +- Fallback to default repository + +## Configuration + +Create `.beads/routes.jsonl`: + +```jsonl +{"pattern": "frontend/**", "target": "frontend-repo", "priority": 10} +{"pattern": "backend/**", "target": "backend-repo", "priority": 10} +{"pattern": "docs/**", "target": "docs-repo", "priority": 5} +{"pattern": "*", "target": "main-repo", "priority": 0} +``` + +## Route Fields + +| Field | Description | +|-------|-------------| +| `pattern` | Glob pattern to match | +| `target` | Target repository | +| `priority` | Higher = checked first | + +## Pattern Matching + +Patterns match against: +- Issue title +- Labels +- Explicit path prefix + +**Examples:** +```jsonl +{"pattern": "frontend/*", "target": "frontend"} +{"pattern": "*api*", "target": "backend"} +{"pattern": "label:docs", "target": "docs-repo"} +``` + +## Commands + +```bash +# Show routing table +bd routes list +bd routes list --json + +# Test routing +bd routes test "Fix frontend button" +bd routes test --label frontend + +# Add route +bd routes add "frontend/**" --target frontend-repo --priority 10 + +# Remove route +bd routes remove "frontend/**" +``` + +## Auto-Routing + +When creating issues, beads checks routes: + +```bash +bd create "Fix frontend button alignment" -t bug +# Auto-routed to frontend-repo based on title match +``` + +Override with explicit target: + +```bash +bd create "Fix button" --repo backend-repo +``` + +## Cross-Repo Dependencies + +Track dependencies across repos: + +```bash +# In frontend-repo +bd dep add bd-42 external:backend-repo/bd-100 + +# View cross-repo deps +bd dep tree bd-42 --cross-repo +``` + +## Hydration + +Pull related issues from other repos: + +```bash +# Hydrate issues from related repos +bd hydrate + +# Preview hydration +bd hydrate --dry-run + +# Hydrate specific repo +bd hydrate --from backend-repo +``` + +## Best Practices + +1. **Use specific patterns** - Avoid overly broad matches +2. **Set priorities** - Ensure specific patterns match first +3. **Default fallback** - Always have a `*` pattern with lowest priority +4. **Test routes** - Use `bd routes test` before committing diff --git a/docs/recovery/circular-dependencies.md b/docs/recovery/circular-dependencies.md new file mode 100644 index 000000000..793a81bfc --- /dev/null +++ b/docs/recovery/circular-dependencies.md @@ -0,0 +1,59 @@ +--- +title: Circular Dependencies +description: Detect and break dependency cycles +--- + +This runbook helps you detect and break circular dependency cycles in your issues. + +## Symptoms + +- "circular dependency detected" errors +- `bd blocked` shows unexpected results +- Issues that should be ready appear blocked + +## Diagnosis + +```bash +# Check for blocked issues +bd blocked + +# View dependencies for a specific issue +bd show + +# List all dependencies +bd dep tree +``` + +## Solution + +**Step 1:** Identify the cycle +```bash +bd blocked --verbose +``` + +**Step 2:** Map the dependency chain +```bash +bd show +bd show +# Follow the chain until you return to +``` + +**Step 3:** Determine which dependency to remove +Consider: Which dependency is least critical to the workflow? + +**Step 4:** Remove the problematic dependency +```bash +bd dep remove +``` + +**Step 5:** Verify the cycle is broken +```bash +bd blocked +bd ready +``` + +## Prevention + +- Think "X needs Y" not "X before Y" when adding dependencies +- Use `bd blocked` after adding dependencies to check for cycles +- Keep dependency chains shallow when possible diff --git a/docs/recovery/database-corruption.md b/docs/recovery/database-corruption.md new file mode 100644 index 000000000..9605d4aee --- /dev/null +++ b/docs/recovery/database-corruption.md @@ -0,0 +1,62 @@ +--- +title: Database Corruption +description: Recover from Dolt database corruption +--- + +This runbook helps you recover from database corruption in Beads. + +## Symptoms + +- Error messages during `bd` commands +- "database is locked" errors that persist +- Missing issues that should exist +- Inconsistent database state + +## Diagnosis + +```bash +# Check database integrity +bd doctor + +# Check Dolt server health +bd dolt show +``` + +## Solution + +**Step 1:** Stop the Dolt server +```bash +bd dolt stop +``` + +**Step 2:** Back up current state +```bash +cp -r .beads .beads.backup +``` + +**Step 3:** Preview what doctor would fix +```bash +bd doctor --dry-run +``` + +**Step 4:** Rebuild database +```bash +bd doctor --fix +``` + +**Step 5:** Verify recovery +```bash +bd doctor +bd list +``` + +**Step 6:** Restart the Dolt server +```bash +dolt sql-server +``` + +## Prevention + +- Let the Dolt server handle synchronization +- Use `bd dolt stop` before system shutdown +- Run `bd doctor` periodically to catch issues early diff --git a/docs/recovery/index.md b/docs/recovery/index.md new file mode 100644 index 000000000..3b49f8b8f --- /dev/null +++ b/docs/recovery/index.md @@ -0,0 +1,42 @@ +--- +title: Recovery Overview +description: Diagnose and resolve common Beads issues +--- + +This section provides step-by-step recovery procedures for common Beads issues. Each runbook follows a consistent format: Symptoms, Diagnosis, Solution (5 steps max), and Prevention. + +## Common Issues + +| Issue | Symptoms | Runbook | +|-------|----------|---------| +| Database Corruption | SQLite errors, missing data | [Database Corruption](/recovery/database-corruption) | +| Merge Conflicts | Dolt conflicts during sync | [Merge Conflicts](/recovery/merge-conflicts) | +| Circular Dependencies | Cycle detection errors | [Circular Dependencies](/recovery/circular-dependencies) | +| Sync Failures | `bd dolt push`/`bd dolt pull` errors | [Sync Failures](/recovery/sync-failures) | + +## Quick Diagnostic + +Before diving into specific runbooks, try these quick checks: + +```bash +# Check Beads status +bd status + +# Verify Dolt server is running +bd doctor + +# Check for blocked issues +bd blocked +``` + + +Most issues can be diagnosed with `bd status`. Start there before following specific runbooks. + + +## Getting Help + +If these runbooks don't resolve your issue: + +1. Check the [FAQ](/reference/faq) +2. Search [existing issues](https://github.com/gastownhall/beads/issues) +3. Open a new issue with diagnostic output diff --git a/docs/recovery/init-safety.md b/docs/recovery/init-safety.md new file mode 100644 index 000000000..388842045 --- /dev/null +++ b/docs/recovery/init-safety.md @@ -0,0 +1,253 @@ +--- +title: Recovery Playbooks +--- + +Last reviewed: 2026-06-09 + +Freshness source: `cmd/bd/init.go`, `cmd/bd/init_safety.go`, +`cmd/bd/init_safety_test.go`, and `cmd/bd/dolt.go`. + +This document lives next to the ADRs and matches the structure of `bd`'s +error messages: each named refusal in `bd init` and `bd dolt push`/`pull` +points here to a labeled anchor with step-by-step recovery instructions. + +See also: `bd help init-safety`, and +[ADR 0002 — `bd init` safety invariants](https://github.com/gastownhall/beads/blob/main/engdocs/adr/0002-init-safety-invariants.md). + +## Table of contents + +- [init-force-refused — `bd init --force`/`--reinit-local` refused because origin has Dolt history](#init-force-refused) +- [init-token-missing — `--discard-remote` refused because `--destroy-token` is missing or wrong](#init-token-missing) +- [init-local-exists — `bd init` refused because local data already exists](#init-local-exists) +- [pk-fork-refused — `bd dolt pull`/`push` refused because a table has different primary keys in its common ancestor](#pk-fork-refused) + +--- + +## init-force-refused + +**Exit code:** `10` (`ExitRemoteDivergenceRefused`) + +**Symptom** + +``` +bd init refuses: remote 'origin' already has Dolt history (refs/dolt/data). + Why: this init mode would create or reuse local history instead of + adopting the remote. ... +``` + +**Why this happens** + +`bd init --force` (or `--reinit-local`) tells `bd` to bypass the local +data-safety guard. `bd init --from-jsonl` selects a local JSONL export as +the source. But the remote already has project history. Proceeding would +create an orphan local Dolt branch with no common ancestor on origin. The +next `bd dolt push` would either fail (no common ancestor) or — worse, if +force-pushed — destroy the team's data. + +**Recovery paths** + +Pick the one that matches your intent. + +### 1. You want to adopt the remote's history (most common) + +``` +bd bootstrap +``` + +This clones the remote's Dolt database into a fresh local `.beads/`. +Your local state is ignored; the team's history becomes yours. + +### 2. You want to diagnose what went wrong before deciding + +``` +bd doctor +bd dolt status +``` + +`bd doctor` walks the local + remote state and names concrete problems. +`bd dolt status` shows the Dolt-level view. Neither modifies anything. + +### 3. You intentionally want to overwrite the remote's history (destructive) + +This is a cross-boundary operation that affects every collaborator. You +need to pair the local-source init (`--reinit-local` or `--from-jsonl`) +with `--discard-remote`. In interactive mode `bd` will prompt for +confirmation; in non-interactive mode you must supply a `--destroy-token`. +See `bd help init-safety` for the token format. + +After `bd init --reinit-local --discard-remote`, your next +`bd dolt push` must be a history-replacing push. Coordinate with your +team before doing this. + +--- + +## init-token-missing + +**Exit code:** `12` (`ExitDestroyTokenMissing`) + +**Symptom** + +``` +bd init refuses: --discard-remote requires an explicit destroy-token in non-interactive mode. +``` + +**Why this happens** + +You're running non-interactively (CI, agent, piped input) and passed +`--discard-remote`. Destructive cross-boundary operations cannot be +authorized silently. + +**Recovery paths** + +### 1. Run interactively + +Re-run in a TTY. `bd init --reinit-local --discard-remote` will prompt +you to type the destroy-token at confirmation time. + +### 2. Supply the token explicitly (CI/automation) + +The token format is `DESTROY-`. For a project whose issue +prefix is `bd`: + +``` +bd init --reinit-local --discard-remote --destroy-token=DESTROY-bd +``` + +Automation should template the token from project state, not from error +output. See [ADR 0002 — Invariant 4](https://github.com/gastownhall/beads/blob/main/engdocs/adr/0002-init-safety-invariants.md) +for why the token is never echoed in `bd`'s error messages. + +--- + +## init-local-exists + +**Exit code:** `11` (`ExitLocalExistsRefused`) + +**Symptom** + +``` +Refusing to destroy N issues in non-interactive mode. + See 'bd help init-safety' for the required --destroy-token format. +``` + +Or, in interactive mode, you declined the typed `destroy N issues` +confirmation. + +**Why this happens** + +Local `.beads/` has existing issues. `bd init --reinit-local` would +permanently destroy them. + +**Recovery paths** + +### 1. Export first, then proceed + +``` +bd export > issue-export.jsonl +bd init --reinit-local +``` + +`issue-export.jsonl` lets you re-import individual issues if needed. It is not +a full database backup; use `bd backup` when the Dolt database is healthy +enough to create a restorable backup before reinitializing. + +### 2. Investigate why you hit this + +If you did NOT expect `bd init` to be the right command here, run +`bd doctor` first — you may be looking at a server config issue that a +re-init won't fix. + +--- + +## pk-fork-refused + +**Symptom** + +``` +$ bd dolt pull +Error: ... cannot merge because table dependencies has different primary keys in its common ancestor +``` + +(or the variant without `in its common ancestor`). `bd` follows the error +with a short version of the recovery recipe below. + +**Why this happens** + +The two histories being merged disagree about a table's *primary key set* — +not about row contents. Dolt can cell-merge rows, but it refuses outright to +merge a table whose primary key was reshaped differently on each side (or +whose common ancestor had a different primary key than both sides). The +refusal happens before any row conflicts materialize, so `bd dolt pull`'s +conflict auto-resolver never gets a chance to run. **Retrying never helps**: +the histories are permanently un-mergeable. + +The usual cause is upgrading `bd` independently on two clones while un-synced +changes existed on both sides, across a release whose schema migrations +reshape a primary key. Concretely: the +[#4259](https://github.com/gastownhall/beads/issues/4259) incident — clones +straddling the `0041`/`0043`/`0050` reshapes of `dependencies` (v1.0.4 → +v1.0.6) hit exactly this on the first post-upgrade pull if both sides had +unpushed dependency edits. + +The remote-migrate prevention gate (v1.0.6+) exists to stop this from being +created: it refuses to auto-migrate a remote-backed database and tells you to +designate a single migrator. This playbook is for when the fork already +exists. + +**Recovery: bootstrap from one canonical clone** + +The forked histories cannot be merged, so one side must be chosen as +canonical and every other clone re-cloned from it. Issue *data* survives via +JSONL export/import; only the un-mergeable Dolt *history* is discarded on the +non-canonical clones. + +### 1. Pick the canonical clone + +Usually the most complete / most recently active clone. To compare, run on +each clone (read-only): + +``` +bd stats +bd dolt status +``` + +### 2. On the canonical clone: upgrade, migrate, force-push + +``` +bd version # confirm the new bd binary +bd doctor # sanity-check before publishing +bd dolt push --force # make the remote authoritative +``` + +(`bd`'s migration gate may block here; run `bd migrate --force` on the +canonical clone — that is exactly the designated-migrator case the gate is +asking about. `BD_ALLOW_REMOTE_MIGRATE=1` is the env-var equivalent for +scripted use.) + +### 3. On EVERY other clone: save local-only work, re-clone, re-apply + +``` +bd export --all -o /tmp/beads-local.jsonl # safety net for un-synced work +rm -rf .beads/dolt # discard the un-mergeable history +bd bootstrap # re-clone from the remote +bd import /tmp/beads-local.jsonl # re-apply local-only work +``` + +`bd import` has upsert semantics: issues that only existed on this clone are +re-created, newer local edits are applied, and rows older than what the +remote already has are skipped. Spot-check with `bd stats` afterwards. + +### Prevention (upgrades across PK-reshaping migrations) + +- **Sync before upgrading**: `bd dolt push` + `bd dolt pull` on every clone + while all clones still run the *old* version, then stop editing. Once the new + binary is installed, `bd dolt push`/`bd dolt pull` are gated too, so this must + happen first. +- **One designated migrator**: upgrade one machine, let it migrate, then + `bd dolt push`. +- **Every other clone adopts, does not pull**: after the migrator pushes, each + other clone upgrades the binary and runs `bd bootstrap` to adopt the migrated + database. `bd dolt pull` is *refused* while the clone still has pending + migrations, so do not rely on it; the "sync before" step above is what + preserves these clones' work, because `bd bootstrap` replaces the local + database. diff --git a/docs/recovery/merge-conflicts.md b/docs/recovery/merge-conflicts.md new file mode 100644 index 000000000..b8a743dbe --- /dev/null +++ b/docs/recovery/merge-conflicts.md @@ -0,0 +1,54 @@ +--- +title: Merge Conflicts +description: Resolve Dolt merge conflicts +--- + +This runbook helps you resolve merge conflicts that occur during Dolt sync operations. + +## Symptoms + +- `bd dolt pull` fails with conflict errors +- Different issue states between clones + +## Diagnosis + +```bash +# Check database health +bd doctor + +# Preview what fixes would be applied +bd doctor --dry-run +``` + +## Solution + +**Step 1:** Back up current state +```bash +cp -r .beads .beads.backup +``` + +**Step 2:** Check for conflicts +```bash +bd doctor +``` + +**Step 3:** Fix to reconcile +```bash +bd doctor --fix +``` + +**Step 4:** Verify state +```bash +bd list +bd stats +``` + +**Step 5:** Push resolved state +```bash +bd dolt push +``` + +## Prevention + +- Sync before and after work sessions using `bd dolt pull` / `bd dolt push` +- Avoid concurrent modifications from multiple clones without the Dolt server running diff --git a/docs/recovery/sync-failures.md b/docs/recovery/sync-failures.md new file mode 100644 index 000000000..9d52877e4 --- /dev/null +++ b/docs/recovery/sync-failures.md @@ -0,0 +1,75 @@ +--- +title: Sync Failures +description: Recover from Dolt sync failures +--- + +This runbook helps you recover from Dolt sync failures. + +## Symptoms + +- `bd dolt push` or `bd dolt pull` hangs or times out +- Network-related error messages +- "failed to push" or "failed to pull" errors +- Dolt server not responding + +## Diagnosis + +```bash +# Check Dolt server health +bd doctor +bd dolt show + +# View Dolt server logs +tail -50 .beads/dolt/sql-server.log +``` + +## Solution + +**Step 1:** Stop the Dolt server +```bash +bd dolt stop +``` + +**Step 2:** Check for lock files +```bash +ls -la .beads/*.lock +# Remove stale locks if Dolt server is definitely stopped +rm -f .beads/*.lock +``` + +**Step 3:** Back up and preview fixes +```bash +cp -r .beads .beads.backup +bd doctor --dry-run +``` + +**Step 4:** Apply fixes if needed +```bash +bd doctor --fix +``` + +**Step 5:** Restart the Dolt server +```bash +dolt sql-server +``` + +**Step 6:** Verify sync works +```bash +bd dolt push +bd doctor +``` + +## Common Causes + +| Cause | Solution | +|-------|----------| +| Network timeout | Retry with better connection | +| Stale lock file | Remove lock after stopping Dolt server | +| Corrupted state | Back up, then `bd doctor --fix` | +| Merge conflicts | See [Merge Conflicts](/recovery/merge-conflicts) | + +## Prevention + +- Ensure stable network before sync +- Let sync complete before closing terminal +- Use `bd dolt stop` before system shutdown diff --git a/docs/UNINSTALLING.md b/docs/recovery/uninstalling.md similarity index 98% rename from docs/UNINSTALLING.md rename to docs/recovery/uninstalling.md index 17d1cf9cc..b3185f970 100644 --- a/docs/UNINSTALLING.md +++ b/docs/recovery/uninstalling.md @@ -1,4 +1,6 @@ -# Uninstalling Beads +--- +title: Uninstalling +--- This guide explains how to remove beads from a repository or remove the `bd` binary from a machine. diff --git a/docs/reference/advanced.md b/docs/reference/advanced.md new file mode 100644 index 000000000..e90843e91 --- /dev/null +++ b/docs/reference/advanced.md @@ -0,0 +1,193 @@ +--- +title: Advanced Features +--- + +Advanced beads functionality. + +## Issue Rename + +Rename issues while preserving references: + +```bash +bd rename bd-42 bd-new-id +bd rename bd-42 bd-new-id --dry-run # Preview +``` + +Updates: +- All dependencies pointing to old ID +- All references in other issues +- Comments and descriptions + +## Issue Merge + +Merge duplicate issues: + +```bash +bd merge bd-42 bd-43 --into bd-41 +bd merge bd-42 bd-43 --into bd-41 --dry-run +``` + +What gets merged: +- Dependencies → target +- Text references updated across all issues +- Source issues closed with merge reason + +## Database Compaction + +Reduce database size by compacting old issues: + +```bash +# View compaction statistics +bd admin compact --stats + +# Preview candidates (30+ days closed) +bd admin compact --analyze --json + +# Apply agent-generated summary +bd admin compact --apply --id bd-42 --summary summary.txt + +# Immediate deletion (CAUTION!) +bd admin cleanup --force +``` + +**When to compact:** +- Database > 10MB with old closed issues +- After major milestones +- Before archiving project phase + +## Restore from History + +View deleted or compacted issues from git: + +```bash +bd restore bd-42 --show +bd restore bd-42 --to-file issue.json +``` + +## Database Inspection + +`bd sql` requires Dolt server mode (`bd dolt start`, see Performance Tuning +below); it is not available against the default embedded-mode database. + +```bash +# Schema info +bd info --schema --json + +# Raw database query (server mode only) +bd sql "SELECT * FROM issues LIMIT 5" +``` + +## Extension Data + +For Dolt-backed projects, keep extension state outside the beads database and +connect it to beads through stable CLI surfaces: + +```bash +# Query issues for integration workflows +bd list --json +bd query "status=open AND priority<=2" --json + +# Run direct SQL for inspection (server mode only) +bd sql "SELECT id, title, status FROM issues LIMIT 5" +``` + +Custom tables through direct storage access are a legacy SQLite-only pattern. +See the [bd-example-extension-go example](https://github.com/gastownhall/beads/blob/main/examples/bd-example-extension-go/README.md) +only if you are maintaining a SQLite-backed extension. + +## Audit Data + +Beads records issue lifecycle events in the database for audit and recovery +workflows. There is no standalone `bd events` command; inspect current issue +state through JSON output, or query the audit tables directly when needed: + +```bash +# Current issue state +bd show bd-a1b2 --json + +# Recent stored events for one issue (server mode only) +bd sql "SELECT event_type, actor, created_at FROM events WHERE issue_id = 'bd-a1b2' ORDER BY created_at DESC LIMIT 20" +``` + +Events: +- `issue.created` +- `issue.updated` +- `issue.closed` +- `dependency.added` +- `sync.completed` + +## Batch Operations + +### Create Multiple + +```bash +# Bootstrap a new database from JSONL +bd init --from-jsonl issues.jsonl +``` + +### Update Multiple + +```bash +bd list --status open --priority 4 --json | \ + jq -r '.[].id' | \ + xargs -I {} bd update {} --priority 3 +``` + +### Close Multiple + +```bash +bd list --label "sprint-1" --status open --json | \ + jq -r '.[].id' | \ + xargs -I {} bd close {} --reason "Sprint complete" +``` + +## Integration Access + +Use the CLI as the supported integration boundary: + +```bash +# Machine-readable issue data +bd show bd-a1b2 --json + +# Ready-work queue for automation +bd ready --json + +# Direct SQL inspection against the active Dolt database (server mode only) +bd sql "SELECT id, priority, status FROM issues WHERE status != 'closed'" +``` + +The storage packages under `internal/` are not a public Go API. + +## Performance Tuning + +### Large Databases + +```bash +# Enable WAL mode +bd config set database.wal_mode true + +# Increase cache +bd config set database.cache_size 10000 +``` + +### Many Concurrent Agents + +Beads uses Dolt server mode to handle concurrent access from multiple agents. +The server manages transaction isolation automatically. + +```bash +# Start the Dolt server +bd dolt start + +# Check server health +bd doctor +``` + +### CI/CD Optimization + +In CI/CD environments, beads uses embedded mode by default (no server required): + +```bash +# Just run commands directly — no special flags needed +bd list +``` diff --git a/docs/ANTIVIRUS.md b/docs/reference/antivirus.md similarity index 99% rename from docs/ANTIVIRUS.md rename to docs/reference/antivirus.md index 8018150d3..5b20ab8a3 100644 --- a/docs/ANTIVIRUS.md +++ b/docs/reference/antivirus.md @@ -1,4 +1,6 @@ -# Antivirus False Positives +--- +title: Antivirus False Positives +--- ## Overview diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md new file mode 100644 index 000000000..4b889d4b0 --- /dev/null +++ b/docs/reference/configuration.md @@ -0,0 +1,275 @@ +--- +title: Configuration +--- + +Complete configuration reference for beads. + +beads has two complementary configuration systems: + +1. **Tool-level configuration** (YAML, managed by [Viper](https://github.com/spf13/viper)) — startup flags and tool behavior, stored in `config.yaml` files. +2. **Project-level configuration** (managed by `bd config`) — integration credentials, status maps, and project-specific settings, stored in the Dolt database. Some keys are routed to `config.yaml` instead (see [YAML-only keys](#yaml-only-keys-startup-settings) below). + +For a deeper treatment of every namespace and its semantics, see [docs/CONFIG.md](https://github.com/gastownhall/beads/blob/main/docs/CONFIG.md). + +## Configuration Locations + +`config.yaml` is searched in this order, with later files overriding earlier ones: + +1. `~/.beads/config.yaml` (legacy user-level, lowest priority) +2. `~/.config/bd/config.yaml` (XDG user-level) +3. `/.beads/config.yaml` (project-level, walked up from the current directory) +4. `$BEADS_DIR/config.yaml` (highest priority, when `BEADS_DIR` is set) + +A `config.local.yaml` next to the project `config.yaml` is also merged in last for machine-specific overrides that should not be committed. + +## Precedence + +For Viper-managed (YAML) keys, highest to lowest: + +1. **Command-line flags** (e.g. `--json`, `--db`, `--actor`) +2. **Environment variables** (`BD_*`, plus a small set of legacy `BEADS_*` names — see below) +3. **`config.yaml`** files (in the order listed above) +4. **Built-in defaults** + +Project-level keys written via `bd config set` (Jira, Linear, GitHub, status maps, etc.) live in the Dolt database. They are read at command time and have no env var override. + +## Managing Configuration + +```bash +# Set a value (auto-routes to config.yaml or the database) +bd config set jira.url "https://company.atlassian.net" +bd config set validation.on-create warn # YAML-only key + +# Set many values in one go +bd config set-many jira.url=https://example.atlassian.net jira.project=PROJ + +# Get a value +bd config get jira.url + +# List all database-stored config (with override warnings) +bd config list + +# Show all effective config with provenance (env / config.yaml / default / database) +bd config show +bd config show --source config.yaml +bd config show --json + +# Validate sync-related configuration +bd config validate + +# Remove a value +bd config unset jira.url +``` + +`bd config set` automatically routes the write to the right location: keys in the YAML namespace (see below) are written to the project `config.yaml`; everything else is written to the Dolt database. `beads.role` is stored in git config. + +## YAML-only Keys (Startup Settings) + +These keys must live in `config.yaml`, not the database, because they are read before the database is opened. Writing them with `bd config set` automatically updates `config.yaml`. + +The full namespaces routed to YAML are: + +`routing.*`, `sync.*`, `git.*`, `directory.*`, `repos.*`, `external_projects.*`, `validation.*`, `hierarchy.*`, `ai.*`, `backup.*`, `export.*`, `dolt.*`, `federation.*` + +Plus these individual keys: + +`no-db`, `json`, `db`, `actor`, `identity`, `no-push`, `no-git-ops`, `create.require-description`, `github.token`, `linear.api_key`, `linear.oauth_client_id`, `linear.oauth_client_secret`. + +Secrets in this list are refused on git-tracked `config.yaml` files unless you pass `--force-git-tracked`; export the value as an environment variable instead (e.g. `LINEAR_API_KEY`). + +## Tool-Level Settings (config.yaml) + +| Setting | Flag | Env Var | Default | Description | +|---|---|---|---|---| +| `json` | `--json` | `BD_JSON` | `false` | JSON output for scripting | +| `db` | `--db` | `BD_DB` | (auto-discover) | Database path | +| `actor` | `--actor` | `BEADS_ACTOR` | `git config user.name` | Actor name for audit trail | +| `identity` | `--identity` | `BEADS_IDENTITY` | (git user / hostname) | Sender identity for `bd mail` | +| `no-db` | `--no-db` | `BD_NO_DAEMON` (related) | `false` | Run without opening the database | +| `no-push` | `--no-push` | — | `false` | Skip pushing to Dolt remote | +| `no-git-ops` | — | — | `false` | Disable git ops in `bd prime` close protocol | +| `dolt.auto-commit` | `--dolt-auto-commit` | `BD_DOLT_AUTO_COMMIT` | `on` | Create a Dolt history commit after each successful write | +| `dolt.auto-push` | — | `BD_DOLT_AUTO_PUSH` | `false` | Auto-push to Dolt remote after writes (opt-in) | +| `dolt.shared-server` | `--shared-server` | `BEADS_DOLT_SHARED_SERVER` | `false` | Share one Dolt server at `~/.beads/shared-server/` | +| `dolt.max-conns` | — | `BEADS_DOLT_MAX_CONNS` | `10` | Connection pool size | +| `git.author` | — | — | (none) | Override commit author for beads commits | +| `git.no-gpg-sign` | — | — | `false` | Disable GPG signing for beads commits | +| `create.require-description` | — | `BD_CREATE_REQUIRE_DESCRIPTION` | `false` | Require description on `bd create` | +| `validation.on-create` | — | `BD_VALIDATION_ON_CREATE` | `none` | Template validation: `none`, `warn`, `error` | +| `validation.on-close` | — | `BD_VALIDATION_ON_CLOSE` | `none` | Template validation on close | +| `validation.on-sync` | — | `BD_VALIDATION_ON_SYNC` | `none` | Template validation before sync | +| `validation.metadata.mode` | — | — | `none` | Metadata schema validation | +| `hierarchy.max-depth` | — | — | `3` | Max hierarchical ID nesting depth | +| `backup.enabled` | — | `BD_BACKUP_ENABLED` | `false` | Enable periodic Dolt-native backup | +| `backup.interval` | — | `BD_BACKUP_INTERVAL` | `15m` | Minimum time between auto-backups | +| `backup.git-push` | — | — | `false` | Auto-push backup repo | +| `backup.git-repo` | — | `BD_BACKUP_GIT_REPO` | (none) | Backup git repo URL | +| `export.auto` | — | — | `false` | Refresh `.beads/issues.jsonl` export after every write; not cross-machine sync | +| `export.path` | — | — | `issues.jsonl` | Output filename relative to `.beads/` | +| `import.path` | — | — | `issues.jsonl` | Input filename relative to `.beads/` for implied JSONL imports; use relative paths for portability | +| `export.interval` | — | — | `60s` | Minimum time between auto-exports | +| `export.git-add` | — | — | `false` | Run `git add` on the export file | +| `routing.mode` | — | — | (none) | Multi-repo routing: `auto`, `maintainer`, `contributor`, `explicit` | +| `routing.default` | — | — | `.` | Default routing target | +| `routing.maintainer` | — | — | `.` | Maintainer-routed path | +| `routing.contributor` | — | — | `~/.beads-planning` | Contributor-routed path | +| `list.limit` | `--limit` / `-n` | `BD_LIST_LIMIT` | `50` | Default limit for `bd list` results | +| `directory.labels` | — | — | `{}` | Map directory patterns → labels for monorepos | +| `external_projects` | — | — | `{}` | Map project names → paths for cross-project deps | +| `federation.remote` | — | `BD_FEDERATION_REMOTE` | (none) | Dolt remote URL (`dolthub://`, `gs://`, `s3://`, `az://`, `file://`) | +| `federation.sovereignty` | — | `BD_FEDERATION_SOVEREIGNTY` | (none) | Sovereignty tier: `T1`, `T2`, `T3`, `T4` | +| `federation.allowed-remote-patterns` | — | — | `[]` | Glob patterns restricting allowed remote URLs | +| `federation.exclude_types` | — | — | `[wisp]` | Issue types excluded from federation push | +| `sync.require_confirmation_on_mass_delete` | — | — | `false` | Prompt before pushing >50% issue deletions | +| `output.title-length` | — | — | `255` | Title display in feedback (`0` hides); see routing note below | +| `ai.model` | — | `BD_AI_MODEL` | `claude-haiku-4-5-20251001` | Default AI model | +| `agents.file` | — | — | `AGENTS.md` | Agents instruction filename; see routing note below | + + +**JSONL export is opt-in** + +`export.auto` and `export.git-add` are disabled unless configured explicitly. +`.beads/issues.jsonl` is an optional export for viewers, interchange, and +issue-level migration. It is not the canonical source of truth, not +cross-machine sync, and not a full database backup. + +Workflows that depend on a fresh, git-staged JSONL file should opt in: + +```bash +bd config set export.auto true +bd config set export.git-add true +``` + +Use `bd dolt push` / `bd dolt pull` for sync and `bd backup` for restorable +database backups. + + +Routing note: `output.title-length` and `agents.file` are functionally tool-level settings, but `bd config set` writes them to the Dolt database. They are typically read from `config.yaml` when set there directly. + +`bd config show` is the source of truth for what's currently effective on your machine, including provenance. + +## Project-Level Settings (Database) + +These are written to the Dolt database by `bd config set` and have no env var override. Common namespaces: + +| Namespace | Purpose | +|---|---| +| `jira.*` | Jira integration (URL, project, status_map, type_map) | +| `linear.*` | Linear integration (team_id, state_map, label_type_map, relation_map) | +| `github.*` | GitHub integration (org, repo, label_map) | +| `ado.*` | Azure DevOps integration (org, project, state_map, type_map) | +| `custom.*` | User-defined / custom integrations | +| `status.custom` | Comma-separated list of custom statuses | +| `types.custom` | Comma-separated list of custom issue types | +| `types.infra` | Infra types routed to wisps table | +| `import.orphan_handling` | `allow` (default) \| `resurrect` \| `skip` \| `strict` | +| `compact_*` | Compaction tuning (see [docs/CONFIG.md](https://github.com/gastownhall/beads/blob/main/docs/CONFIG.md)) | +| `issue_id_mode` | `hash` (default) \| `counter` (sequential) | +| `min_hash_length`, `max_hash_length` | Adaptive ID bounds (defaults `4` and `8`) | +| `max_collision_prob` | Hash ID collision tolerance (default `0.25`) | +| `doctor.suppress.*` | Suppress specific `bd doctor` warnings by check slug | + +Issue prefix (`issue_prefix`) is **not** settable via `bd config set` — use `bd init --prefix`, `bd bootstrap`, or `bd rename-prefix`. + +## Environment Variables + +The Viper env prefix is `BD_`. Config keys map to env vars by upper-casing and replacing `.` and `-` with `_` (e.g. `dolt.auto-commit` → `BD_DOLT_AUTO_COMMIT`, `validation.on-create` → `BD_VALIDATION_ON_CREATE`). + +Selected commonly-used variables: + +| Variable | Description | +|---|---| +| `BD_DB`, `BEADS_DB` | Database path (legacy `BEADS_DB` still honored) | +| `BD_JSON` | Force JSON output | +| `BD_DOLT_AUTO_COMMIT` | Override `dolt.auto-commit` (`on`/`off`) | +| `BD_DOLT_AUTO_PUSH` | Override `dolt.auto-push` | +| `BD_BACKUP_ENABLED`, `BD_BACKUP_INTERVAL`, `BD_BACKUP_GIT_REPO` | Override backup settings | +| `BD_AI_MODEL` | Override AI model | +| `BD_FEDERATION_REMOTE`, `BD_FEDERATION_SOVEREIGNTY` | Override federation settings | +| `BD_VALIDATION_ON_CREATE` / `_ON_CLOSE` / `_ON_SYNC` | Override validation modes | +| `BD_NO_PAGER`, `BD_PAGER` | Pager behavior | +| `BD_NON_INTERACTIVE` | Disable prompts | +| `BD_DEBUG` | Enable debug logging | +| `BEADS_DIR` | Force the active beads workspace directory | +| `BEADS_ACTOR` | Actor identity (preferred over `BD_ACTOR`, which is a deprecated alias) | +| `BEADS_IDENTITY` | Sender identity for `bd mail` | +| `BEADS_DOLT_SERVER_MODE`, `BEADS_DOLT_SHARED_SERVER`, `BEADS_DOLT_DATA_DIR`, `BEADS_DOLT_PORT`, ... | Embedded/server Dolt overrides | + +Secrets like API tokens follow integration-specific conventions: `LINEAR_API_KEY`, `GITHUB_TOKEN`, `ANTHROPIC_API_KEY`, `AZURE_DEVOPS_PAT`. These are preferred over storing the value in `config.yaml` for git-tracked projects. + +`bd config show` will display the source of every effective key, making overrides explicit. + +## Example `.beads/config.yaml` + +```yaml +# Default JSON output for scripting +json: true + +# Dolt history & sync +dolt: + auto-commit: on # Create a Dolt commit after each successful write + auto-push: false # Opt-in for single-writer setups + +# Issue creation policies +create: + require-description: true + +validation: + on-create: warn # Warn when creating issues missing required sections + on-close: none + on-sync: none + +# Git commit signing for beads commits (GH#600) +git: + author: "beads-bot " + no-gpg-sign: true + +# Periodic Dolt-native backup to .beads/backup/ +backup: + enabled: true + interval: 15m + +# Optional auto-export of issues.jsonl after writes for viewers/interchange +export: + auto: false + path: issues.jsonl + interval: 60s + git-add: false + +# Optional Dolt federation +federation: + remote: dolthub://myorg/beads + sovereignty: T2 + +# Directory-aware label scoping for monorepos (GH#541) +directory: + labels: + packages/maverick: maverick + packages/agency: agency + +# Cross-project dependency resolution (bd-h807) +external_projects: + beads: ../beads + other-project: /absolute/path/to/other-project + +output: + title-length: 255 +``` + +For machine-specific overrides that should not be committed, drop them in `.beads/config.local.yaml`; it is merged in last. + +## Per-Command Override + +```bash +bd --db /tmp/test.db list # Override database for one command +bd --json --actor "ci-bot" create "Fix things" # Multiple flags +``` + +## Viewing Active Configuration + +```bash +bd config show # Effective config with provenance +bd config show --json # Machine-readable +bd config list # Database-stored config +bd info --json | jq '.config' # Quick snapshot +``` diff --git a/docs/reference/faq.md b/docs/reference/faq.md new file mode 100644 index 000000000..905e27286 --- /dev/null +++ b/docs/reference/faq.md @@ -0,0 +1,154 @@ +--- +title: FAQ +--- + +## General + +### Why beads instead of GitHub Issues or Jira? + +Beads was designed specifically for AI-supervised coding workflows: +- **Hash-based IDs** prevent collisions with concurrent agents +- **Dolt-backed storage** enables branch-based workflows +- **Dependency-aware** ready queue for automated work selection +- **Formula system** for declarative workflow templates + +### What does "beads" stand for? + +Nothing specific - it's a metaphor for linked work items (like beads on a string). + +### Is beads production-ready? + +Yes, beads is used in production for AI-assisted development. The API is stable with semantic versioning. + +## Architecture + +### Why Dolt instead of plain SQLite? + +- **Dolt** provides a version-controlled SQL database with built-in replication +- Git-like branching, diffing, and merging at the database level +- No need for a separate sync format -- Dolt handles it natively + +### Why hash-based IDs instead of sequential? + +Sequential IDs (`#1`, `#2`) break when: +- Multiple agents create issues simultaneously +- Different branches have independent numbering +- Forks diverge and merge + +Hash-based IDs are globally unique without coordination. + +### How does the Dolt server work? + +Beads uses Dolt server mode for concurrent access: +- Transaction isolation for multiple agents +- SQL-based queries for performance +- Automatic retry on conflicts + +In CI/CD or single-agent environments, beads uses embedded mode automatically (no server required). + +## Usage + +### How do I sync issues? + +```bash +# Push changes to Dolt remote: +bd dolt push + +# Pull changes from Dolt remote: +bd dolt pull +``` + +### How do I handle merge conflicts? + +Dolt handles merge conflicts at the database level. If conflicts arise during pull: +```bash +bd doctor --fix +bd dolt push +``` + +### Can multiple agents work on the same repo? + +Yes! That's what beads was designed for: +- Hash IDs prevent collisions +- Pin work to specific agents +- Track who's working on what + +### How do I use beads in CI/CD? + +```bash +# Just run commands directly — beads uses embedded mode in CI +bd list +``` + +## Workflows + +### What are formulas? + +Declarative workflow templates in TOML or JSON. Pour them to create molecules (instances). + +### What are gates? + +Async coordination primitives: +- Human gates wait for approval +- Timer gates wait for duration +- GitHub gates wait for CI/PR events + +### What's the difference between molecules and wisps? + +- **Molecules** persist in `.beads/` and sync with git +- **Wisps** are ephemeral in `.beads-wisp/` and don't sync + +## Integration + +### Should I use CLI or MCP? + +**Use CLI + hooks** when shell is available (Claude Code, Cursor, etc.): +- Lower context overhead (~1-2k vs 10-50k tokens) +- Faster execution +- Universal across editors + +**Use MCP** when CLI unavailable (Claude Desktop). + +### How do I integrate with my editor? + +```bash +bd setup claude # Claude Code +bd setup cursor # Cursor +bd setup aider # Aider +``` + +### Can beads import from GitHub Issues? + +Yes — see the [GitHub integration documentation](https://github.com/gastownhall/beads/blob/main/docs/GITHUB_IMPORT.md) for details. + +## Troubleshooting + +### Why is the Dolt server not starting? + +```bash +# Check server status +bd doctor + +# Check server logs +cat .beads/dolt/sql-server.log + +# Restart the server +bd dolt stop +bd dolt start +``` + +### Why aren't my changes syncing? + +```bash +# Force push to Dolt remote +bd dolt push + +# Check hooks +bd hooks status +``` + +### How do I report a bug? + +1. Check existing issues: https://github.com/gastownhall/beads/issues +2. Include: `bd version`, `bd info --json`, reproduction steps +3. File at: https://github.com/gastownhall/beads/issues/new diff --git a/docs/reference/git-integration.md b/docs/reference/git-integration.md new file mode 100644 index 000000000..3032780c4 --- /dev/null +++ b/docs/reference/git-integration.md @@ -0,0 +1,127 @@ +--- +title: Git Integration +--- + +How beads integrates with git. + +## Overview + +Beads uses git for: +- **Project hosting** - Your code repository also hosts beads configuration +- **Hooks** - Auto-sync on git operations + +Data storage and sync are handled by Dolt (a version-controlled SQL database). + +## File Structure + +``` +.beads/ +├── config.yaml # Project config (git-tracked) +├── metadata.json # Backend metadata (git-tracked) +└── dolt/ # Dolt database and server data (gitignored) +``` + +## Git Hooks + +### Installation + +```bash +bd hooks install +``` + +Installs: +- **pre-commit** - Triggers Dolt commit +- **post-merge** - Triggers Dolt sync after pull +- **pre-push** - Ensures Dolt sync before push + +### Status + +```bash +bd hooks status +``` + +### Uninstall + +```bash +bd hooks uninstall +``` + +## Conflict Resolution + +Dolt handles merge conflicts at the database level using its built-in +merge capabilities. When conflicts arise during sync, Dolt identifies +conflicting rows and allows resolution through SQL. + +```bash +# Check for and fix conflicts +bd doctor --fix +``` + +## Protected Branches + +Dolt stores data under `refs/dolt/data`, separate from Git refs. This means +beads data does not conflict with protected Git branches, and no separate +`beads-sync` branch or protected-branch exception is needed. On new projects +with a Git `origin`, `bd init` configures that origin as the Dolt remote +automatically. + +## Git Worktrees + +Beads works in Git worktrees without extra setup. Linked worktrees discover the +repository's `.beads` workspace and sync issue data through Dolt: + +```bash +# In a linked worktree +bd create "Task" +bd list +bd dolt pull +bd dolt push +``` + +Older beads versions documented a `sync.branch` workflow that created hidden +Git worktrees. That workflow has been removed; current sync uses Dolt remotes. + +## Branch Workflows + +### Feature Branch + +```bash +git checkout -b feature-x +bd create "Feature X" -t feature +# Work... +bd dolt push +git push +``` + +### Fork Workflow + +```bash +# In fork +bd init --contributor +# Work in separate planning repo... +bd dolt push +``` + +### Team Workflow + +```bash +bd init --team +# All team members share the Dolt database +bd dolt pull # Pull latest changes from Dolt remote +bd dolt push # Push your changes to Dolt remote +``` + +### Duplicate Detection + +After merging branches: + +```bash +bd duplicates --auto-merge +``` + +## Best Practices + +1. **Install hooks** - `bd hooks install` +2. **Push regularly** - `bd dolt push` at session end +3. **Pull before work** - `bd dolt pull` to get latest issues +4. **Use normal Git worktrees** - no sync branch is required diff --git a/docs/reference/json-schema.md b/docs/reference/json-schema.md new file mode 100644 index 000000000..f4492be1a --- /dev/null +++ b/docs/reference/json-schema.md @@ -0,0 +1,207 @@ +--- +title: JSON Output Schema Contract +--- + +Last reviewed: 2026-05-08 + +Freshness source: `cmd/bd/output.go`, `cmd/bd/errors.go`, and +`cmd/bd/protocol/json_contract_test.go`. + +All `bd` commands that support `--json` output can wrap their response in +a uniform envelope by setting `BD_JSON_ENVELOPE=1`. This will become the +default format in v2.0. + +## Migration Guide + +### Opt in to the envelope format + +```bash +export BD_JSON_ENVELOPE=1 +``` + +### Envelope format (BD_JSON_ENVELOPE=1, default in v2.0) + +Every `--json` command wraps output as: + +```json +{"schema_version": 1, "data": } +``` + +The original payload is untouched inside `.data` — no type corruption, +no field injection. Works identically for objects, arrays, and maps. + +### Updating consumers + +```bash +# Before (legacy): +bd list --json | jq '.[0].id' +bd show beads-abc --json | jq '.title' + +# After (envelope): +bd list --json | jq '.data[0].id' +bd show beads-abc --json | jq '.data.title' + +# Version check: +bd show beads-abc --json | jq '.schema_version' +``` + +### Timeline + +- **Current release**: Legacy format is default. Set `BD_JSON_ENVELOPE=1` to opt in. + A deprecation notice is printed to stderr when `--json` is used without the env var. +- **v2.0**: Envelope becomes the default. `BD_JSON_ENVELOPE=0` available as + temporary escape hatch for one release cycle. + +## Schema Version + +Current version: **1** + +The `schema_version` field is an integer that increments when: +- Fields are added, renamed, or removed +- Output structure changes (e.g., nesting depth) +- Field types change (e.g., string to integer) + +Additive changes (new optional fields) do NOT bump the version. + +## Output Formats + +### Envelope mode (BD_JSON_ENVELOPE=1) + +All commands emit a uniform envelope: + +```json +{ + "schema_version": 1, + "data": { + "id": "beads-abc", + "title": "Example issue", + "status": "open" + } +} +``` + +Arrays are wrapped the same way: + +```json +{ + "schema_version": 1, + "data": [ + {"id": "beads-abc", "title": "First"}, + {"id": "beads-def", "title": "Second"} + ] +} +``` + +### Legacy mode (default, until v2.0) + +### Object commands (show, create, close, update, etc.) + +Commands that return a single issue or result emit a JSON object with +`schema_version` as a top-level field alongside the data: + +```json +{ + "schema_version": 1, + "id": "beads-abc", + "title": "Example issue", + "status": "open", + "priority": 1, + "issue_type": "task", + "created_at": "2026-04-20T12:00:00Z" +} +``` + +### List commands (list, ready, search, stale, etc.) + +Commands that return multiple items emit a raw JSON array: + +```json +[ + {"id": "beads-abc", "title": "First", ...}, + {"id": "beads-def", "title": "Second", ...} +] +``` + +### Error output (stderr) + +Errors with `--json` active emit JSON to stderr: + +```json +{ + "schema_version": 1, + "error": "issue not found: beads-xyz", + "code": "not_found" +} +``` + +## Field Contracts by Command + +### bd list --json + +Required fields per item: +- `id` (string): Issue ID (e.g., "beads-abc") +- `title` (string): Issue title +- `status` (string): open, in_progress, closed, deferred +- `priority` (number): 0-4 +- `issue_type` (string): bug, feature, task, epic, chore +- `created_at` (string): RFC3339 timestamp + +Optional fields: +- `description`, `owner`, `updated_at`, `closed_at` +- `labels` (string[]): Attached labels +- `dependencies` (object[]): Dependency records +- `dependency_count`, `dependent_count`, `comment_count` (number) +- `parent` (string|null): Parent issue ID + +### bd ready --json + +Same schema as `bd list --json`. Items are filtered to unblocked issues only. +Each item includes `dependency_count`, `dependent_count`, `comment_count`, +and optional `parent` fields. + +### bd blocked --json + +Returns issues that are blocked by unresolved dependencies. +Each item includes all standard issue fields plus: +- `blocked_by_count` (number): Number of blocking dependencies +- `blocked_by` (string[]): IDs of blocking issues + +### bd show --json + +Returns a single object (not wrapped in `items`). Same required fields as list +items, plus: +- `description` (string) +- `acceptance_criteria` (string) +- `dependencies` (object[]): Full dependency records +- `comments` (object[]): Comment thread + +### `import --json` + +Returns a summary object when `--json` is active: +- `source` (string): File path or "stdin" +- `created` (number): Issues created +- `skipped` (number): Issues skipped (dedup) +- `dedup_skipped` (number): Issues skipped by `--dedup` title match +- `memories` (number): Memory records imported +- `ids` (string[]): IDs of created issues +- `dry_run` (boolean): Whether `--dry-run` was active + +### bd export --json + +Outputs JSONL (one JSON object per line), not wrapped in an envelope. +Each line is a self-contained issue or memory record. `schema_version` +is included per line. + +## Consumer Guidelines + +1. **Check `schema_version`** on object output. If the version is + higher than expected, log a warning but attempt to parse anyway + (additive changes are backward-compatible). + +2. **For list commands**, parse the output as a JSON array directly. + +3. **Ignore unknown fields**. New fields may be added without bumping + the schema version. + +4. **Use `--json` flag**, not `--format json`. The `--json` flag is + the stable contract; `--format` is for human-readable variants. diff --git a/docs/OBSERVABILITY.md b/docs/reference/observability.md similarity index 99% rename from docs/OBSERVABILITY.md rename to docs/reference/observability.md index b26bccfc1..535490628 100644 --- a/docs/OBSERVABILITY.md +++ b/docs/reference/observability.md @@ -1,4 +1,6 @@ -# Observability (OpenTelemetry) +--- +title: Observability (OpenTelemetry) +--- Beads exports metrics via OTLP HTTP. Telemetry is **disabled by default** — zero overhead when no variable is set. diff --git a/docs/PROTECTED_BRANCHES.md b/docs/reference/protected-branches.md similarity index 92% rename from docs/PROTECTED_BRANCHES.md rename to docs/reference/protected-branches.md index cfe0ffa21..1086e79e4 100644 --- a/docs/PROTECTED_BRANCHES.md +++ b/docs/reference/protected-branches.md @@ -1,4 +1,6 @@ -# Protected Branches +--- +title: Protected Branches +--- Beads does not need a protected-branch workaround in current releases. @@ -115,6 +117,6 @@ bd hooks install ## See Also -- [WORKTREES.md](WORKTREES.md) - Git worktree behavior -- [GIT_INTEGRATION.md](GIT_INTEGRATION.md) - general Git integration guide -- [RECOVERY.md](RECOVERY.md) - recovery playbooks +- [WORKTREES.md](/reference/worktrees) - Git worktree behavior +- [GIT_INTEGRATION.md](/reference/git-integration) - general Git integration guide +- [RECOVERY.md](/recovery/init-safety) - recovery playbooks diff --git a/docs/reference/troubleshooting.md b/docs/reference/troubleshooting.md new file mode 100644 index 000000000..b26419053 --- /dev/null +++ b/docs/reference/troubleshooting.md @@ -0,0 +1,221 @@ +--- +title: Troubleshooting +--- + +Common issues and solutions. + +## Installation Issues + +### `bd: command not found` + +```bash +# Check if installed +which bd +go list -f {{.Target}} github.com/steveyegge/beads/cmd/bd + +# Add Go bin to PATH +export PATH="$PATH:$(go env GOPATH)/bin" + +# Or reinstall with the recommended installer +curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/install.sh | bash +``` + +### `zsh: killed bd` on macOS + +CGO/SQLite compatibility issue: + +```bash +CGO_ENABLED=1 GOFLAGS=-tags=gms_pure_go go install github.com/steveyegge/beads/cmd/bd@latest +``` + +### Permission denied + +```bash +chmod +x $(which bd) +``` + +## Database Issues + +### Database not found + +```bash +# Initialize beads +bd init --quiet + +# Or specify database +bd --db .beads/beads.db list +``` + +### Database locked + +```bash +# Stop the Dolt server if running +bd dolt stop + +# Try again +bd list +``` + +### Corrupted database + +```bash +# Check and fix database +bd doctor --fix + +# Or pull from Dolt remote +bd dolt pull + +# Or restore from a backup +bd backup restore [path] --force +``` + +## Dolt Server Issues + +### Server not starting + +```bash +# Check server health +bd doctor + +# Check server logs +cat .beads/dolt/sql-server.log + +# Restart the server +bd dolt stop +bd dolt start +``` + +### Version mismatch + +After upgrading bd: + +```bash +bd dolt stop +bd dolt start +``` + +## Sync Issues + +### Changes not syncing + +```bash +# Force push to Dolt remote +bd dolt push + +# Check hooks +bd hooks status +``` + +### Recovery from backup + +```bash +# Restore from a Dolt backup +bd backup restore [path] --force + +# Or pull from Dolt remote +bd dolt pull +``` + +### Merge conflicts + +```bash +# Check for and fix Dolt conflicts +bd doctor --fix + +# Re-push +bd dolt push +``` + +## Git Hook Issues + +### Hooks not running + +```bash +# Check if installed +ls -la .git/hooks/ + +# Reinstall +bd hooks install +``` + +### Hook errors + +```bash +# Check hook script +cat .git/hooks/pre-commit + +# Run manually +.git/hooks/pre-commit +``` + +## Dependency Issues + +### Circular dependencies + +```bash +# Detect cycles +bd dep cycles + +# Remove one dependency +bd dep remove bd-A bd-B +``` + +### Missing dependencies + +```bash +# Check orphan handling +bd config get import.orphan_handling + +# Allow orphans +bd config set import.orphan_handling allow +``` + +## Performance Issues + +### Slow queries + +```bash +# Check database size +ls -lh .beads/beads.db + +# Compact if large +bd admin compact --analyze +``` + +### High memory usage + +```bash +# Reduce cache +bd config set database.cache_size 1000 +``` + +## Getting Help + +### Debug output + +```bash +bd --verbose list +``` + +### Logs + +```bash +cat .beads/dolt/sql-server.log +``` + +### System info + +```bash +bd info --json +``` + +### File an issue + +```bash +# Include this info +bd version +bd info --json +uname -a +``` + +Report at: https://github.com/gastownhall/beads/issues diff --git a/docs/WORKTREES.md b/docs/reference/worktrees.md similarity index 91% rename from docs/WORKTREES.md rename to docs/reference/worktrees.md index 8eb019f71..8d4d59ca2 100644 --- a/docs/WORKTREES.md +++ b/docs/reference/worktrees.md @@ -1,4 +1,6 @@ -# Git Worktrees Guide +--- +title: Git Worktrees Guide +--- Beads works from normal Git worktrees without a separate sync branch. Current beads stores issue data in Dolt under `refs/dolt/data`, so issue sync is @@ -126,6 +128,6 @@ working the same issue concurrently. ## See Also -- [PROTECTED_BRANCHES.md](PROTECTED_BRANCHES.md) - protected branch behavior -- [GIT_INTEGRATION.md](GIT_INTEGRATION.md) - general Git integration guide -- [MULTI_REPO_MIGRATION.md](MULTI_REPO_MIGRATION.md) - multi-workspace patterns +- [PROTECTED_BRANCHES.md](/reference/protected-branches) - protected branch behavior +- [GIT_INTEGRATION.md](/reference/git-integration) - general Git integration guide +- [MULTI_REPO_MIGRATION.md](/multi-agent/multi-repo-migration) - multi-workspace patterns diff --git a/docs/RELATED_PROJECTS.md b/docs/related-projects.md similarity index 90% rename from docs/RELATED_PROJECTS.md rename to docs/related-projects.md index 1cffd7fca..b9b42dcd3 100644 --- a/docs/RELATED_PROJECTS.md +++ b/docs/related-projects.md @@ -1,8 +1,10 @@ -# Related Projects +--- +title: Related Projects +--- Adjacent or complementary tools that solve different problems in the same neighborhood as Beads. These are not Beads integrations (see -[COMMUNITY_TOOLS.md](COMMUNITY_TOOLS.md) for those) — they are +[COMMUNITY_TOOLS.md](/community-tools) for those) — they are independent projects whose users may also find Beads useful, or vice versa. diff --git a/docs/workflows/formulas.md b/docs/workflows/formulas.md new file mode 100644 index 000000000..cb66882ab --- /dev/null +++ b/docs/workflows/formulas.md @@ -0,0 +1,248 @@ +--- +title: Formulas +--- + +Formulas are declarative workflow templates. + +## Formula Format + +Formulas can be written in TOML (preferred) or JSON: + +### TOML Format + +```toml +formula = "feature-workflow" +description = "Standard feature development workflow" +version = 1 +type = "workflow" + +[vars.feature_name] +description = "Name of the feature" +required = true + +[[steps]] +id = "design" +title = "Design {{feature_name}}" +type = "human" +description = "Create design document" + +[[steps]] +id = "implement" +title = "Implement {{feature_name}}" +needs = ["design"] + +[[steps]] +id = "review" +title = "Code review" +needs = ["implement"] +type = "human" + +[[steps]] +id = "merge" +title = "Merge to main" +needs = ["review"] +``` + +### JSON Format + +```json +{ + "formula": "feature-workflow", + "description": "Standard feature development workflow", + "version": 1, + "type": "workflow", + "vars": { + "feature_name": { + "description": "Name of the feature", + "required": true + } + }, + "steps": [ + { + "id": "design", + "title": "Design {{feature_name}}", + "type": "human" + }, + { + "id": "implement", + "title": "Implement {{feature_name}}", + "needs": ["design"] + } + ] +} +``` + +## Formula Types + +| Type | Description | +|------|-------------| +| `workflow` | Standard step sequence | +| `expansion` | Template for expansion operator | +| `aspect` | Cross-cutting concerns | + +## Variables + +Define variables with defaults and constraints: + +```toml +[vars.version] +description = "Release version" +required = true +pattern = "^\\d+\\.\\d+\\.\\d+$" + +[vars.environment] +description = "Target environment" +default = "staging" +enum = ["staging", "production"] +``` + +Use variables in steps: + +```toml +[[steps]] +title = "Deploy {{version}} to {{environment}}" +``` + +## Step Types + +| Type | Description | +|------|-------------| +| `task` | Normal work step (default) | +| `human` | Requires human action | +| `gate` | Async coordination point | + +## Dependencies + +### Sequential + +```toml +[[steps]] +id = "step1" +title = "First step" + +[[steps]] +id = "step2" +title = "Second step" +needs = ["step1"] +``` + +### Parallel then Join + +```toml +[[steps]] +id = "test-unit" +title = "Unit tests" + +[[steps]] +id = "test-integration" +title = "Integration tests" + +[[steps]] +id = "deploy" +title = "Deploy" +needs = ["test-unit", "test-integration"] # Waits for both +``` + +## Gates + +Add gates for async coordination: + +```toml +[[steps]] +id = "approval" +title = "Manager approval" +type = "human" + +[steps.gate] +type = "human" +approvers = ["manager"] + +[[steps]] +id = "deploy" +title = "Deploy to production" +needs = ["approval"] +``` + +## Aspects (Cross-cutting) + +Apply transformations to matching steps: + +```toml +formula = "security-scan" +type = "aspect" + +[[advice]] +target = "*.deploy" # Match all deploy steps + +[advice.before] +id = "security-scan-{step.id}" +title = "Security scan before {step.title}" +``` + +## Formula Locations + +Formulas are searched in order: +1. `.beads/formulas/` (project-level) +2. `~/.beads/formulas/` (user-level) +3. Built-in formulas + +## Using Formulas + +```bash +# List available formulas +bd mol list + +# Pour formula into molecule +bd pour --var key=value + +# Preview what would be created +bd pour --dry-run +``` + +## Creating Custom Formulas + +1. Create file: `.beads/formulas/my-workflow.formula.toml` +2. Define structure (see examples above) +3. Use with: `bd pour my-workflow` + +## Example: Release Formula + +```toml +formula = "release" +description = "Standard release workflow" +version = 1 + +[vars.version] +required = true +pattern = "^\\d+\\.\\d+\\.\\d+$" + +[[steps]] +id = "bump-version" +title = "Bump version to {{version}}" + +[[steps]] +id = "changelog" +title = "Update CHANGELOG" +needs = ["bump-version"] + +[[steps]] +id = "test" +title = "Run full test suite" +needs = ["changelog"] + +[[steps]] +id = "build" +title = "Build release artifacts" +needs = ["test"] + +[[steps]] +id = "tag" +title = "Create git tag v{{version}}" +needs = ["build"] + +[[steps]] +id = "publish" +title = "Publish release" +needs = ["tag"] +type = "human" +``` diff --git a/docs/workflows/gates.md b/docs/workflows/gates.md new file mode 100644 index 000000000..44b033ad5 --- /dev/null +++ b/docs/workflows/gates.md @@ -0,0 +1,212 @@ +--- +title: Gates +--- + +Gates are async coordination primitives for workflow orchestration. + +## What are Gates? + +Gates block step progression until a condition is met: +- Human approval +- Timer expiration +- External event (GitHub PR, CI, etc.) + +## Gate Types + +### Human Gate + +Wait for human approval: + +```toml +[[steps]] +id = "deploy-approval" +title = "Approval for production deploy" +type = "human" + +[steps.gate] +type = "human" +approvers = ["team-lead", "security"] +require_all = false # Any approver can approve +``` + +### Timer Gate + +Wait for a duration: + +```toml +[[steps]] +id = "cooldown" +title = "Wait for cooldown period" + +[steps.gate] +type = "timer" +duration = "24h" +``` + +Durations: `30m`, `2h`, `24h`, `7d` + +### GitHub Gate + +Wait for GitHub events: + +```toml +[[steps]] +id = "wait-for-ci" +title = "Wait for CI to pass" + +[steps.gate] +type = "github" +event = "check_suite" +status = "success" +``` + +```toml +[[steps]] +id = "wait-for-merge" +title = "Wait for PR merge" + +[steps.gate] +type = "github" +event = "pull_request" +action = "closed" +merged = true +``` + +## Gate States + +| State | Description | +|-------|-------------| +| `pending` | Waiting for condition | +| `open` | Condition met, can proceed | +| `closed` | Step completed | + +## Using Gates in Workflows + +### Approval Flow + +```toml +formula = "production-deploy" + +[[steps]] +id = "build" +title = "Build production artifacts" + +[[steps]] +id = "staging" +title = "Deploy to staging" +needs = ["build"] + +[[steps]] +id = "qa-approval" +title = "QA sign-off" +needs = ["staging"] +type = "human" + +[steps.gate] +type = "human" +approvers = ["qa-team"] + +[[steps]] +id = "production" +title = "Deploy to production" +needs = ["qa-approval"] +``` + +### Scheduled Release + +```toml +formula = "scheduled-release" + +[[steps]] +id = "prepare" +title = "Prepare release" + +[[steps]] +id = "wait-window" +title = "Wait for release window" +needs = ["prepare"] + +[steps.gate] +type = "timer" +duration = "2h" + +[[steps]] +id = "deploy" +title = "Deploy release" +needs = ["wait-window"] +``` + +### CI Integration + +```toml +formula = "ci-gated-deploy" + +[[steps]] +id = "create-pr" +title = "Create pull request" + +[[steps]] +id = "wait-ci" +title = "Wait for CI" +needs = ["create-pr"] + +[steps.gate] +type = "github" +event = "check_suite" +status = "success" + +[[steps]] +id = "merge" +title = "Merge PR" +needs = ["wait-ci"] +type = "human" +``` + +## Gate Operations + +### Check Gate Status + +```bash +bd show bd-xyz.3 # Shows gate state +bd show bd-xyz.3 --json | jq '.gate' +``` + +### Manual Gate Override + +For human gates: + +```bash +bd gate approve bd-xyz.3 --approver "team-lead" +``` + +### Skip Gate (Emergency) + +```bash +bd gate skip bd-xyz.3 --reason "Emergency deploy" +``` + +## waits-for Dependency + +The `waits-for` dependency type creates fan-in patterns: + +```toml +[[steps]] +id = "test-a" +title = "Test suite A" + +[[steps]] +id = "test-b" +title = "Test suite B" + +[[steps]] +id = "integration" +title = "Integration tests" +waits_for = ["test-a", "test-b"] # Fan-in: waits for all +``` + +## Best Practices + +1. **Use human gates for critical decisions** - Don't auto-approve production +2. **Add timeout to timer gates** - Prevent indefinite blocking +3. **Document gate requirements** - Make approvers clear +4. **Use CI gates for quality** - Block on test failures diff --git a/docs/workflows/index.md b/docs/workflows/index.md new file mode 100644 index 000000000..4ae75f907 --- /dev/null +++ b/docs/workflows/index.md @@ -0,0 +1,90 @@ +--- +title: Workflows +--- + +Beads provides powerful workflow primitives for complex, multi-step processes. + +## Chemistry Metaphor + +Beads uses a molecular chemistry metaphor: + +| Phase | Storage | Synced | Use Case | +|-------|---------|--------|----------| +| **Proto** (solid) | Built-in | N/A | Reusable templates | +| **Mol** (liquid) | `.beads/` | Yes | Persistent work | +| **Wisp** (vapor) | `.beads-wisp/` | No | Ephemeral operations | + +## Core Concepts + +### Formulas + +Declarative workflow templates in TOML or JSON: + +```toml +formula = "feature-workflow" +version = 1 +type = "workflow" + +[[steps]] +id = "design" +title = "Design the feature" +type = "human" + +[[steps]] +id = "implement" +title = "Implement the feature" +needs = ["design"] +``` + +### Molecules + +Work graphs with parent-child relationships: +- Created by instantiating formulas with `bd pour` +- Steps have dependencies (`needs`) +- Progress tracked via issue status + +### Gates + +Async coordination primitives: +- **Human gates** - Wait for human approval +- **Timer gates** - Wait for duration +- **GitHub gates** - Wait for PR merge, CI, etc. + +### Wisps + +Ephemeral operations that don't sync to git: +- Created with `bd wisp` +- Stored in `.beads-wisp/` (gitignored) +- Auto-expire after completion + +## Workflow Commands + +| Command | Description | +|---------|-------------| +| `bd pour` | Instantiate formula as molecule | +| `bd wisp` | Create ephemeral wisp | +| `bd mol list` | List molecules | +| `bd pin` | Pin work to agent | +| `bd hook` | Show pinned work | + +## Simple Example + +```bash +# Create a release workflow +bd pour release --var version=1.0.0 + +# View the molecule +bd mol show release-1.0.0 + +# Work through steps +bd update release-1.0.0.1 --claim +bd close release-1.0.0.1 +# Next step becomes ready... +``` + +## Navigation + +- [Molecules](/workflows/molecules) - Work graphs and execution +- [Formulas](/workflows/formulas) - Declarative templates +- [Gates](/workflows/gates) - Async coordination +- [Wisps](/workflows/wisps) - Ephemeral operations diff --git a/docs/workflows/molecules.md b/docs/workflows/molecules.md new file mode 100644 index 000000000..b3d138083 --- /dev/null +++ b/docs/workflows/molecules.md @@ -0,0 +1,156 @@ +--- +title: Molecules +--- + +Molecules are work graphs created from formulas. + +## What is a Molecule? + +A molecule is a persistent instance of a formula: +- Contains steps with dependencies +- Tracked in `.beads/` (syncs with git) +- Steps map to issues with parent-child relationships + +## Creating Molecules + +### From Formula + +```bash +# Pour a formula into a molecule +bd pour [--var key=value] +``` + +**Example:** +```bash +bd pour release --var version=1.0.0 +``` + +This creates: +- Parent issue: `bd-xyz` (the molecule root) +- Child issues: `bd-xyz.1`, `bd-xyz.2`, etc. (the steps) + +### Listing Molecules + +```bash +bd mol list +bd mol list --json +``` + +### Viewing a Molecule + +```bash +bd mol show +bd dep tree # Shows full hierarchy +``` + +## Working with Molecules + +### Step Dependencies + +Steps have `needs` dependencies: + +```toml +[[steps]] +id = "implement" +title = "Implement feature" +needs = ["design"] # Must complete design first +``` + +The `bd ready` command respects these: + +```bash +bd ready # Only shows steps with completed dependencies +``` + +### Progressing Through Steps + +```bash +# Start a step +bd update bd-xyz.1 --claim + +# Complete a step +bd close bd-xyz.1 --reason "Done" + +# Check what's ready next +bd ready +``` + +### Viewing Progress + +```bash +# See blocked steps +bd blocked + +# See molecule stats +bd stats +``` + +## Molecule Lifecycle + +``` +Formula (template) + ↓ bd pour +Molecule (instance) + ↓ work steps +Completed Molecule + ↓ optional cleanup +Archived +``` + +## Advanced Features + +### Bond Points + +Formulas can define bond points for composition: + +```toml +[compose] +[[compose.bond_points]] +id = "entry" +step = "design" +position = "before" +``` + +### Hooks + +Step-completion hooks are not currently exposed as runnable formula +actions. The historical `on_complete.run` example was invalid: `run` is +not a formula field, and `on_complete` runtime expansion is tracked +separately until it is wired end to end. + +### Pinning Work + +Assign molecules to agents: + +```bash +# Pin to current agent +bd pin bd-xyz --start + +# Check what's pinned +bd hook +``` + +## Example Workflow + +```bash +# 1. Create molecule from formula +bd pour feature-workflow --var name="dark-mode" + +# 2. View structure +bd dep tree bd-xyz + +# 3. Start first step +bd update bd-xyz.1 --claim + +# 4. Complete and progress +bd close bd-xyz.1 +bd ready # Shows next steps + +# 5. Continue until complete +``` + +## See Also + +- [Formulas](/workflows/formulas) - Creating templates +- [Gates](/workflows/gates) - Async coordination +- [Wisps](/workflows/wisps) - Ephemeral workflows diff --git a/docs/TODO.md b/docs/workflows/todo.md similarity index 99% rename from docs/TODO.md rename to docs/workflows/todo.md index 95d4df532..ad326f802 100644 --- a/docs/TODO.md +++ b/docs/workflows/todo.md @@ -1,4 +1,6 @@ -# TODO Command +--- +title: TODO Command +--- The `bd todo` command provides a lightweight interface for managing TODO items as task-type issues. diff --git a/docs/workflows/wisps.md b/docs/workflows/wisps.md new file mode 100644 index 000000000..39e23168d --- /dev/null +++ b/docs/workflows/wisps.md @@ -0,0 +1,125 @@ +--- +title: Wisps +--- + +Wisps are ephemeral workflows that don't sync to git. + +## What are Wisps? + +Wisps are "vapor phase" molecules: +- Stored in `.beads-wisp/` (gitignored) +- Don't sync with git +- Auto-expire after completion +- Perfect for temporary operations + +## Use Cases + +| Scenario | Why Wisp? | +|----------|-----------| +| Local experiments | No need to pollute git history | +| CI/CD pipelines | Ephemeral by nature | +| Scratch workflows | Quick throwaway work | +| Agent coordination | Local-only coordination | + +## Creating Wisps + +```bash +# Create wisp from formula +bd wisp create [--var key=value] + +# Example +bd wisp create quick-check --var target=auth-module +``` + +## Wisp Commands + +```bash +# List wisps +bd wisp list +bd wisp list --json + +# Show wisp details +bd wisp show + +# Delete wisp +bd wisp delete + +# Delete all completed wisps +bd wisp cleanup +``` + +## Wisp vs Molecule + +| Aspect | Molecule | Wisp | +|--------|----------|------| +| Storage | `.beads/` | `.beads-wisp/` | +| Git sync | Yes | No | +| Persistence | Permanent | Ephemeral | +| Use case | Tracked work | Temporary ops | + +## Phase Control + +Use `bd mol bond` to control phase: + +```bash +# Force liquid (persistent molecule) +bd mol bond --pour + +# Force vapor (ephemeral wisp) +bd mol bond --wisp +``` + +## Example: Quick Check Workflow + +Create a wisp for running checks: + +```toml +# .beads/formulas/quick-check.formula.toml +formula = "quick-check" +description = "Quick local checks" + +[[steps]] +id = "lint" +title = "Run linter" + +[[steps]] +id = "test" +title = "Run tests" +needs = ["lint"] + +[[steps]] +id = "build" +title = "Build project" +needs = ["test"] +``` + +Use as wisp: + +```bash +bd wisp create quick-check +# Work through steps... +bd wisp cleanup # Remove when done +``` + +## Auto-Expiration + +Wisps can auto-expire: + +```toml +[wisp] +expires_after = "24h" # Auto-delete after 24 hours +``` + +Or cleanup manually: + +```bash +bd wisp cleanup --all # Remove all wisps +bd wisp cleanup --completed # Remove only completed +``` + +## Best Practices + +1. **Use wisps for local-only work** - Don't sync to git +2. **Clean up regularly** - `bd wisp cleanup` +3. **Use molecules for tracked work** - Wisps are ephemeral +4. **Consider CI/CD wisps** - Perfect for pipeline steps diff --git a/docs/CONTRIBUTOR_NAMESPACE_ISOLATION.md b/engdocs/CONTRIBUTOR_NAMESPACE_ISOLATION.md similarity index 100% rename from docs/CONTRIBUTOR_NAMESPACE_ISOLATION.md rename to engdocs/CONTRIBUTOR_NAMESPACE_ISOLATION.md diff --git a/docs/ERROR_HANDLING.md b/engdocs/ERROR_HANDLING.md similarity index 100% rename from docs/ERROR_HANDLING.md rename to engdocs/ERROR_HANDLING.md diff --git a/scripts/check-doc-freshness.sh b/scripts/check-doc-freshness.sh index cfed9c3ee..754b1285a 100755 --- a/scripts/check-doc-freshness.sh +++ b/scripts/check-doc-freshness.sh @@ -22,7 +22,7 @@ DOCS=( "docs/ADO_CONFIG.md|cmd/bd/ado*.go;internal/ado/" "docs/JSON_SCHEMA.md|cmd/bd/output.go;cmd/bd/errors.go;cmd/bd/protocol/json_contract_test.go" "docs/RECOVERY.md|cmd/bd/init.go;cmd/bd/init_safety.go;cmd/bd/init_safety_test.go" - "docs/ERROR_HANDLING.md|cmd/bd/*.go;cmd/bd/errors.go" + "engdocs/ERROR_HANDLING.md|cmd/bd/*.go;cmd/bd/errors.go" "engdocs/LINTING.md|.golangci.yml" "engdocs/design/otel/otel-data-model.md|internal/telemetry/;internal/storage/dolt/store.go;internal/compact/haiku.go;cmd/bd/find_duplicates.go;internal/hooks/" )