Merge upstream/main into fix/config-unset, rebuilt on #6574's unset

#6574 (bd-zj95) landed its own fix for this PR's first bug, the nested-key
unset, with a line walker that also refuses flow-style and block-scalar
values. This merge takes main's commentOutYamlKey and drops this branch's
node-level commentOutNestedYamlKey/commentOutFlatYamlKey in its favour.
What this PR still adds on top:

- UnsetYamlConfig/UnsetUserYamlConfig return (bool, error), derived by
  comparing content before and after, so `bd config unset` reports where
  the unset landed instead of asserting a write. A missing project
  config.yaml is (false, nil), not an error.
- The database-backed unset path also clears the config.yaml layer. If the
  file refuses the edit, the error says the database row is already gone.
- A key whose value is the indented block beneath it (mapping or list) is
  refused, in main's refusal style, instead of being commented out and
  orphaning the block. main's unsupportedUnsetShape doc named this as a
  known silent gap; the line matcher now covers it, single-segment keys
  included.
- commentOutYamlKey splits instead of scanning, so an unset keeps the
  file's trailing newline. Without this a no-op unset would compare as a
  change.

The CHANGELOG entry is rewritten to claim only what is still new, and sits
under [Unreleased] next to #6574's entry (#6643).

Agent-Signature: claude-code-opus-5-5-unknown-reasoning on behalf of Harry Miller
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Harry Miller 2026-09-23 09:46:47 -07:00
commit d168df1841
No known key found for this signature in database
467 changed files with 46510 additions and 4844 deletions

View file

@ -134,6 +134,20 @@ needs = ["preflight-git"]
description = """
Ensure we're up to date with origin and branch is properly synced.
**Every `origin/...` below means the branch you are RELEASING FROM, which is
`release/x.y.z` — never `main`.** Resetting a release branch onto main's tip
destroys the reviewed release-prep commit and leaves you tagging an untested
main SHA, which is exactly the failure the release-branch policy exists to
prevent. Pin the name first and use the variable:
```bash
# The branch you are releasing from. For a prerelease this is still the BASE
# version's branch: an RC for 1.3.0 is cut on release/1.3.0.
REL_BRANCH="release/1.3.0"
git rev-parse --abbrev-ref HEAD # must equal $REL_BRANCH
```
```bash
# Fetch latest from origin
git fetch origin
@ -155,21 +169,20 @@ git pull --rebase
Recovering from divergence:
If your branch has diverged from origin (e.g., after a botched commit):
```bash
# Option 1: Reset to origin (loses local commits)
git reset --hard origin/main
# Option 2: Rebase local commits on top of origin
git rebase origin/main
# Option 3: Create a backup branch first
# Option 1: Back up first, ALWAYS
git branch backup-before-release
git reset --hard origin/main
# Option 2: Reset to the release branch's origin (loses local commits)
git reset --hard "origin/$REL_BRANCH"
# Option 3: Rebase local commits on top of it
git rebase "origin/$REL_BRANCH"
```
After pulling, verify sync:
```bash
git status -sb
# Should show: "## main...origin/main" (no ahead/behind)
# Should show: "## release/1.3.0...origin/release/1.3.0" (no ahead/behind)
```
"""
@ -326,158 +339,68 @@ Focus on workflow-impacting changes agents need to know.
"""
[[steps]]
id = "bump-version-go"
title = "Bump version in version.go"
id = "bump-versions"
title = "Bump every version reference"
needs = ["update-info-go"]
description = """
Update the Go version constant.
Run the one script that owns every version reference.
```bash
# macOS
sed -i '' 's/Version = "[^"]*"/Version = "{{version}}"/' cmd/bd/version.go
# Linux
sed -i 's/Version = "[^"]*"/Version = "{{version}}"/' cmd/bd/version.go
./scripts/update-versions.sh {{version}}
```
This is the ONLY sanctioned way to bump. Hand-rolled sed per file is how the
set drifts: this formula used to carry its own copies and pointed at a
`claude-plugin/` path that has not existed since the plugins moved under
`plugins/beads/`, while missing the copilot plugin, the MCP lockfile and the
tracked git-hook markers entirely — so it would have failed outright, or
shipped inconsistent versions, on every step after the stale path.
`scripts/update-versions.sh` rewrites, in one pass:
- `cmd/bd/version.go` — the canonical version everything else is compared to
- `plugins/beads/.claude-plugin/plugin.json`
- `plugins/beads/.codex-plugin/plugin.json`
- `plugins/beads/.copilot-plugin/plugin.json`
- `.claude-plugin/marketplace.json`
- `integrations/beads-mcp/pyproject.toml`
- `integrations/beads-mcp/src/beads_mcp/__init__.py`
- `integrations/beads-mcp/uv.lock` — via `uv lock`
- `npm-package/package.json`
- `.githooks/*` — the `BEGIN`/`END BEADS INTEGRATION v<version>` markers
- `default.nix`
- `cmd/bd/winres/winres.json`, `cmd/bd/winres/manifest.xml`
- `README.md` — the static Alpha badge, if one is present (it is not today;
README uses live npm/PyPI/GitHub badges)
Two of those have burned a release before, so check the script's output rather
than assuming:
- **`uv` must be installed.** Without it the script WARNS and leaves
`uv.lock` stale. The release workflow's MCP gate runs `uv sync --locked`,
which fails only in the tag-triggered run — after the tag exists and can no
longer be rewritten. That burned v1.1.1 and cost v1.1.0 and v1.1.2 their
first runs. If you see `uv not found` or `uv lock failed`, stop and fix it:
```bash
uv lock --directory integrations/beads-mcp
```
- **The `.githooks/*` markers are rewritten wholesale**, not matched against
the current version, because they can already be drifted when the bump runs
(they still said v1.1.0 during the v1.2.0 bump).
Not covered here, by design: `CHANGELOG.md` and `cmd/bd/info.go` (earlier
steps), and `default.nix`'s `vendorHash` (next step).
Verify:
```bash
grep 'Version = ' cmd/bd/version.go
```
"""
[[steps]]
id = "bump-plugin-json"
title = "Bump version in plugin JSON files"
needs = ["bump-version-go"]
description = """
Update Claude plugin manifest versions.
```bash
# macOS/Linux with jq
jq '.version = "{{version}}"' claude-plugin/.claude-plugin/plugin.json > tmp && mv tmp claude-plugin/.claude-plugin/plugin.json
jq '.plugins[0].version = "{{version}}"' .claude-plugin/marketplace.json > tmp && mv tmp .claude-plugin/marketplace.json
```
Verify:
```bash
jq -r '.version' claude-plugin/.claude-plugin/plugin.json
jq -r '.plugins[0].version' .claude-plugin/marketplace.json
```
"""
[[steps]]
id = "bump-mcp-python"
title = "Bump version in MCP Python package"
needs = ["bump-plugin-json"]
description = """
Update the beads-mcp Python package version.
```bash
# macOS
sed -i '' 's/version = "[^"]*"/version = "{{version}}"/' integrations/beads-mcp/pyproject.toml
sed -i '' 's/__version__ = "[^"]*"/__version__ = "{{version}}"/' integrations/beads-mcp/src/beads_mcp/__init__.py
# Linux
sed -i 's/version = "[^"]*"/version = "{{version}}"/' integrations/beads-mcp/pyproject.toml
sed -i 's/__version__ = "[^"]*"/__version__ = "{{version}}"/' integrations/beads-mcp/src/beads_mcp/__init__.py
```
Verify:
```bash
grep 'version = ' integrations/beads-mcp/pyproject.toml | head -1
grep '__version__ = ' integrations/beads-mcp/src/beads_mcp/__init__.py
```
"""
[[steps]]
id = "bump-npm-package"
title = "Bump version in npm package"
needs = ["bump-mcp-python"]
description = """
Update the npm package version.
```bash
# macOS/Linux with jq
jq '.version = "{{version}}"' npm-package/package.json > tmp && mv tmp npm-package/package.json
```
Verify:
```bash
jq -r '.version' npm-package/package.json
```
"""
[[steps]]
id = "bump-winres"
title = "Bump version in Windows resource metadata"
needs = ["bump-npm-package"]
description = """
Update Windows PE resource metadata.
```bash
# macOS
sed -i '' 's/"file_version": "[^"]*"/"file_version": "{{version}}"/' cmd/bd/winres/winres.json
sed -i '' 's/"product_version": "[^"]*"/"product_version": "{{version}}"/' cmd/bd/winres/winres.json
sed -i '' 's/"FileVersion": "[^"]*"/"FileVersion": "{{version}}"/' cmd/bd/winres/winres.json
sed -i '' 's/"ProductVersion": "[^"]*"/"ProductVersion": "{{version}}"/' cmd/bd/winres/winres.json
sed -i '' 's/version="[0-9][^"]*\\.0"/version="{{version}}.0"/' cmd/bd/winres/manifest.xml
# Linux
sed -i 's/"file_version": "[^"]*"/"file_version": "{{version}}"/' cmd/bd/winres/winres.json
sed -i 's/"product_version": "[^"]*"/"product_version": "{{version}}"/' cmd/bd/winres/winres.json
sed -i 's/"FileVersion": "[^"]*"/"FileVersion": "{{version}}"/' cmd/bd/winres/winres.json
sed -i 's/"ProductVersion": "[^"]*"/"ProductVersion": "{{version}}"/' cmd/bd/winres/winres.json
sed -i 's/version="[0-9][^"]*\\.0"/version="{{version}}.0"/' cmd/bd/winres/manifest.xml
```
Verify:
```bash
grep '{{version}}' cmd/bd/winres/winres.json
grep 'version="{{version}}.0"' cmd/bd/winres/manifest.xml
```
"""
[[steps]]
id = "bump-readme"
title = "Check README static version references"
needs = ["bump-winres"]
description = """
README.md currently uses live npm/PyPI/GitHub release badges instead of a static
Alpha version string. Verify no old static version badge needs updating.
```bash
rg 'Alpha \\(v|v[0-9]+\\.[0-9]+\\.[0-9]+' README.md || true
```
If a static release reference is reintroduced later, update it here.
"""
[[steps]]
id = "bump-default-nix"
title = "Bump version in default.nix"
needs = ["bump-readme"]
description = """
Update the Nix package version.
```bash
# macOS
sed -i '' 's/version = "[^"]*";/version = "{{version}}";/' default.nix
# Linux
sed -i 's/version = "[^"]*";/version = "{{version}}";/' default.nix
```
Verify:
```bash
grep 'version = ' default.nix
git diff --stat
```
"""
[[steps]]
id = "update-vendorhash"
title = "Update Nix vendorHash if needed"
needs = ["bump-default-nix"]
needs = ["bump-versions"]
description = """
Update vendorHash if go.mod/go.sum changed since last release.
@ -533,21 +456,45 @@ id = "verify-versions"
title = "Verify version consistency"
needs = ["stamp-changelog"]
description = """
Confirm all versions match {{version}}.
Run the gate, not a hand-listed grep pile — the grep list is what drifts.
```bash
grep 'Version = ' cmd/bd/version.go
jq -r '.version' claude-plugin/.claude-plugin/plugin.json
jq -r '.plugins[0].version' .claude-plugin/marketplace.json
jq -r '.version' npm-package/package.json
grep 'version = ' integrations/beads-mcp/pyproject.toml
grep '__version__ = ' integrations/beads-mcp/src/beads_mcp/__init__.py
grep 'version = ' default.nix
grep '{{version}}' cmd/bd/winres/winres.json
grep 'version="{{version}}.0"' cmd/bd/winres/manifest.xml
./scripts/check-versions.sh
```
All should show {{version}}.
`scripts/check-versions.sh` reads the canonical version out of
`cmd/bd/version.go` and compares the gated files against it: all three plugin
manifests (claude, codex, copilot), the marketplace, the MCP pyproject /
`__init__.py` / `uv.lock` pin (plus `uv lock --check` for freshness when `uv`
is present), the npm package, both `BEADS INTEGRATION` markers in every
`.githooks/*` file, and the Windows PE metadata in `winres.json` and
`manifest.xml`. It exits non-zero on any mismatch and names the file. This is
the same script the release-tag pre-push hook and CI run, so a green run here
is the real signal.
It must print `✓ Version files in sync at: {{version}}` before you continue.
**If it fails, do NOT just re-run the bump — that is a no-op here.**
`update-versions.sh` derives the OLD version from `cmd/bd/version.go`, which by
this point already reads `{{version}}`, so its substitutions rewrite
`{{version}} -> {{version}}` and leave the drifted file exactly as it was while
printing a green "Version constants updated". Only the `.githooks` markers
(rewritten wholesale) and `uv.lock` (regenerated) heal on a re-run. The script
prints the three remedies that do work; the usual one is to edit the named file
to `{{version}}` and re-run the check.
`check-versions.sh` does NOT gate `default.nix` or `README.md`. Confirm
`default.nix` yourself:
```bash
grep 'version = ' default.nix # should be {{version}}
```
(For a prerelease, do not expect the version string to appear uniformly in the
winres files: `file_version`/`product_version` and the manifest's four-part
version carry the BASE version because PE fields must be purely numeric, while
`FileVersion`/`ProductVersion` carry the full `{{version}}`. check-versions.sh
now encodes that split, so let it do the checking.)
"""
[[steps]]
@ -566,37 +513,74 @@ Review the commit to ensure all expected files are included.
"""
[[steps]]
id = "create-tag"
title = "Create release tag"
id = "push-main"
title = "Land the release prep as a PR"
needs = ["commit-release"]
description = """
Create annotated git tag.
Releases are cut from `release/x.y.z`, not from the tip of `main` — see the
"Release Branches" section of RELEASING.md. **Nothing is pushed straight to the
release branch, and nothing is tagged before the PR merges.**
The branch is named for the BASE version: a `1.3.0-rc.1` run still targets
`release/1.3.0`. Derive it rather than substituting `{{version}}` directly,
which would name a branch nothing ever creates and fail with
`src refspec ... does not match any`:
```bash
BASE="$(printf '%s' "{{version}}" | sed 's/-.*//')"
REL_BRANCH="release/$BASE"
echo "$REL_BRANCH"
```
Create the prep branch, push it, and open the PR:
```bash
git checkout -b "release-prep/v{{version}}"
git push -u origin "release-prep/v{{version}}"
gh pr create --base "$REL_BRANCH" --label status/needs-review-auto
```
Wait for it to merge, then take the merged state:
```bash
git checkout "$REL_BRANCH" && git pull
```
After the release ships, every commit that landed only on the release branch
needs a forward-port PR to `main`, the version bump included:
```bash
git log --oneline "main..$REL_BRANCH"
```
"""
[[steps]]
id = "create-tag"
title = "Create release tag on the merged commit"
needs = ["push-main"]
description = """
Create the annotated tag on the MERGED commit, with the release branch checked
out and pulled (previous step).
```bash
git rev-parse --abbrev-ref HEAD # must be the release branch, not the prep branch
git tag -a v{{version}} -m "Release v{{version}}"
```
Tagging before the merge is the failure to avoid, in two shapes. With branch
protection on, the direct push is rejected after the tag already exists
locally. With a squash merge, the tag points at a commit that is not an
ancestor of the release branch — and goreleaser packages `CHANGELOG.md` *from
the tag* into every published archive, so the release would ship the
pre-review notes.
Verify: `git tag -l | tail -5`
"""
[[steps]]
id = "push-main"
title = "Push to main"
needs = ["create-tag"]
description = """
Push the release commit to origin.
```bash
git push origin main
```
If rejected, someone else pushed. Pull, rebase, try again.
"""
[[steps]]
id = "push-tag"
title = "Push release tag"
needs = ["push-main"]
needs = ["create-tag"]
description = """
Push the version tag to trigger CI release.

View file

@ -9,7 +9,7 @@
"name": "beads",
"source": "./plugins/beads",
"description": "AI-supervised issue tracker for coding workflows",
"version": "1.2.2"
"version": "1.3.0"
}
]
}

View file

@ -1,5 +1,5 @@
#!/usr/bin/env sh
# --- BEGIN BEADS INTEGRATION v1.2.2 ---
# --- BEGIN BEADS INTEGRATION v1.3.0 ---
# This section is managed by beads. Do not remove these markers.
if command -v bd >/dev/null 2>&1; then
export BD_GIT_HOOK=1
@ -56,4 +56,4 @@ if command -v bd >/dev/null 2>&1; then
fi
if [ "$_bd_exit" -ne 0 ]; then exit "$_bd_exit"; fi
fi
# --- END BEADS INTEGRATION v1.2.2 ---
# --- END BEADS INTEGRATION v1.3.0 ---

View file

@ -1,5 +1,5 @@
#!/usr/bin/env sh
# --- BEGIN BEADS INTEGRATION v1.2.2 ---
# --- BEGIN BEADS INTEGRATION v1.3.0 ---
# This section is managed by beads. Do not remove these markers.
if command -v bd >/dev/null 2>&1; then
export BD_GIT_HOOK=1
@ -56,4 +56,4 @@ if command -v bd >/dev/null 2>&1; then
fi
if [ "$_bd_exit" -ne 0 ]; then exit "$_bd_exit"; fi
fi
# --- END BEADS INTEGRATION v1.2.2 ---
# --- END BEADS INTEGRATION v1.3.0 ---

View file

@ -40,7 +40,7 @@ if [ -n "$staged_go_files" ]; then
echo "$staged_go_files" | xargs git add
fi
# --- BEGIN BEADS INTEGRATION v1.2.2 ---
# --- BEGIN BEADS INTEGRATION v1.3.0 ---
# This section is managed by beads. Do not remove these markers.
if command -v bd >/dev/null 2>&1; then
export BD_GIT_HOOK=1
@ -97,4 +97,4 @@ if command -v bd >/dev/null 2>&1; then
fi
if [ "$_bd_exit" -ne 0 ]; then exit "$_bd_exit"; fi
fi
# --- END BEADS INTEGRATION v1.2.2 ---
# --- END BEADS INTEGRATION v1.3.0 ---

View file

@ -39,7 +39,7 @@ fi
exec < <(printf '%s\n' "$push_refs")
# --- BEGIN BEADS INTEGRATION v1.2.2 ---
# --- BEGIN BEADS INTEGRATION v1.3.0 ---
# This section is managed by beads. Do not remove these markers.
if command -v bd >/dev/null 2>&1; then
export BD_GIT_HOOK=1
@ -96,4 +96,4 @@ if command -v bd >/dev/null 2>&1; then
fi
if [ "$_bd_exit" -ne 0 ]; then exit "$_bd_exit"; fi
fi
# --- END BEADS INTEGRATION v1.2.2 ---
# --- END BEADS INTEGRATION v1.3.0 ---

View file

@ -1,5 +1,5 @@
#!/usr/bin/env sh
# --- BEGIN BEADS INTEGRATION v1.2.2 ---
# --- BEGIN BEADS INTEGRATION v1.3.0 ---
# This section is managed by beads. Do not remove these markers.
if command -v bd >/dev/null 2>&1; then
export BD_GIT_HOOK=1
@ -56,4 +56,4 @@ if command -v bd >/dev/null 2>&1; then
fi
if [ "$_bd_exit" -ne 0 ]; then exit "$_bd_exit"; fi
fi
# --- END BEADS INTEGRATION v1.2.2 ---
# --- END BEADS INTEGRATION v1.3.0 ---

View file

@ -86,7 +86,10 @@ jobs:
ci_time "install golangci-lint" -- go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.10.1
status=0
ci_time "pr-policy wrapper" -- make ci-pr-policy || status=$?
ci_time "pr-core gotestsum" -- gotestsum --junitfile artifacts/pr-core-junit.xml --format testdox -- -race -short -skip '^TestEmbedded' ./... || status=$?
# -timeout=30m keeps this in step with the lane it measures
# (scripts/ci/pr-core.sh). Without it ./cmd/bd can die on `go test`'s
# 10m default and the measurement reports a panic instead of a duration.
ci_time "pr-core gotestsum" -- gotestsum --junitfile artifacts/pr-core-junit.xml --format testdox -- -race -short -timeout=30m -skip '^TestEmbedded' ./... || status=$?
ci_time "pr-lint wrapper" -- make ci-pr-lint || status=$?
exit "$status"
@ -128,7 +131,10 @@ jobs:
source scripts/ci/lib/timing.sh
source ./.buildflags
ci_time "macos short go test" -- go test -v -race -short -skip '^TestEmbedded' ./...
# -timeout=30m matches the macOS lanes this measures (pr.yml, main.yml).
# This is the job that produced the ./cmd/bd duration distribution, and
# at 10m it clips its own longest samples into timeout panics.
ci_time "macos short go test" -- go test -v -race -short -timeout=30m -skip '^TestEmbedded' ./...
macos-candidates:
name: Measure macOS no-short candidates

99
.github/workflows/close-stale-needs.yml vendored Normal file
View file

@ -0,0 +1,99 @@
name: Close stale needs-info / needs-repro issues
on:
schedule:
- cron: '37 9 * * *'
workflow_dispatch:
permissions: {}
jobs:
close-needs-info:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: Close stale needs-info issues (14 days)
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
script: |
const labels = ['status/needs-info', 'status/needs-repro'];
const days = 14;
const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000);
for (const label of labels) {
const issues = await github.paginate(github.rest.issues.listForRepo, {
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
labels: label,
per_page: 100,
});
for (const issue of issues) {
if (issue.pull_request) continue;
const events = await github.paginate(github.rest.issues.listEvents, {
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
per_page: 100,
});
const labelEvents = events
.filter(e => e.event === 'labeled' && e.label?.name === label);
if (labelEvents.length === 0) continue;
const labeledAt = new Date(Math.max(
...labelEvents.map(e => new Date(e.created_at).getTime())
));
const comments = await github.paginate(github.rest.issues.listComments, {
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
since: labeledAt.toISOString(),
per_page: 100,
});
const visibleRequest = comments.find(c => {
if (c.user?.login === issue.user?.login) return false;
const lower = (c.body || '').toLowerCase();
if (lower.includes('closing')) return false;
if (!/14\s*-?\s*days?/.test(lower)) return false;
if (!lower.includes('reply') && !lower.includes('respond')) return false;
if (label === 'status/needs-repro') return lower.includes('repro') || lower.includes('reproduction');
return lower.includes('info') || lower.includes('information') || lower.includes('details');
});
if (!visibleRequest) {
console.log(`No visible request for #${issue.number} (${label}), skipping.`);
continue;
}
const requestAt = new Date(visibleRequest.created_at);
if (requestAt > cutoff) continue;
const authorReplied = comments.some(c =>
c.user?.login === issue.user?.login &&
new Date(c.created_at) > requestAt
);
if (authorReplied) continue;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
body: `Closing — this was marked \`${label}\` and a response was requested 14 or more days ago with no reply.\n\nIf this is still relevant, please reply with the requested details and we'll reopen.`,
});
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
state: 'closed',
state_reason: 'not_planned',
});
console.log(`Closed #${issue.number} (${label}, requested ${requestAt.toISOString()})`);
}
}

View file

@ -5,7 +5,7 @@ name: Conformance
on:
pull_request:
branches: [ main ]
branches: [ main, 'release/**' ]
merge_group:
concurrency:

View file

@ -6,9 +6,11 @@ on:
tags:
- 'v*'
# On PR to main: test only the last 5 releases for fast feedback
# On PR to main or a release branch: test only the last 5 releases for fast
# feedback. Release-targeted PRs are exactly the ones where cross-version
# upgrade breakage is most expensive to discover at tag time.
pull_request:
branches: [ main ]
branches: [ main, 'release/**' ]
# Allow manual trigger with custom version count
workflow_dispatch:

View file

@ -483,7 +483,7 @@ jobs:
# macOS: full test suite, no coverage (faster)
- os: macos-latest
coverage: false
test-flags: '-v -race -short -timeout=25m'
test-flags: '-v -race -short'
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
@ -557,6 +557,13 @@ jobs:
GOCACHE: ${{ runner.temp }}/go-cache/non-race
run: go build -v -tags gms_pure_go ./cmd/bd
# Both lanes below pass -timeout=30m explicitly. ./cmd/bd has outgrown
# `go test`'s 10m per-package default (see scripts/ci/pr-core.sh and
# #6091), and when it trips the alarm the panic names whichever test was
# in flight, so it reads like a hang rather than the package running out
# of clock. The flag lives on the invocation, not in matrix.test-flags,
# so editing the matrix cannot silently drop it. 30m matches every other
# -race lane here for this same code.
- name: Test (with coverage + JUnit XML)
if: matrix.coverage
env:
@ -566,14 +573,14 @@ jobs:
gotestsum \
--junitfile junit.xml \
--format testdox \
-- -tags gms_pure_go -race -short -timeout=25m -coverprofile=coverage.out -skip '^TestEmbedded' ./...
-- -tags gms_pure_go -race -short -timeout=30m -coverprofile=coverage.out -skip '^TestEmbedded' ./...
- name: Test
if: ${{ !matrix.coverage }}
env:
BEADS_TEST_BD_BINARY: ${{ github.workspace }}/bd
GOCACHE: ${{ runner.temp }}/go-cache/race
run: go test -tags gms_pure_go ${{ matrix.test-flags }} -skip '^TestEmbedded' ./...
run: go test -tags gms_pure_go ${{ matrix.test-flags }} -timeout=30m -skip '^TestEmbedded' ./...
- name: Upload coverage to Codecov
if: matrix.coverage && !cancelled()

View file

@ -56,6 +56,11 @@ jobs:
cache-identity: v1.1.0-b0f3dd607c3fb989ee08d0a6854fba80d0402971eb108f9af6170bc14d491a34
- version: v1.1.2
cache-identity: v1.1.2-a72d71ed374955dc9f83a0f90b54bd7b6a0016709dd1676ae2e368651ed401c2
# The v1.1.2 tree re-tagged by the recovery release, so it adds no new
# storage regime — but it is the tag field upgrades actually start
# from, and it was never in this corpus.
- version: v1.2.2
cache-identity: v1.2.2-8140098a51d3b81d5548d1c5e6db1a2d9930e5d141efe2a4bff7d079c4d321e8
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@ -98,7 +103,10 @@ jobs:
sudo timeout 180 apt-get install -y jq libicu74
- name: Install Dolt test runtime
if: contains(fromJSON('["v0.55.4","v0.56.1","v0.57.0","v0.62.0"]'), matrix.version)
# v0.55.4-v0.62.0 host their fixture on this server; v1.0.1-v1.1.2 use
# the same pinned binary as a read-only oracle over the wisp plane,
# which has no `bd dolt sql` equivalent.
if: contains(fromJSON('["v0.55.4","v0.56.1","v0.57.0","v0.62.0","v1.0.1","v1.1.0","v1.1.2","v1.2.2"]'), matrix.version)
run: |
archive="$RUNNER_TEMP/dolt-linux-amd64.tar.gz"
runtime="$RUNNER_TEMP/dolt-runtime"

View file

@ -2,7 +2,7 @@ name: PR Risk
on:
pull_request:
branches: [ main ]
branches: [ main, 'release/**' ]
merge_group:
concurrency:

View file

@ -2,7 +2,10 @@ name: PR
on:
pull_request:
branches: [ main ]
# release/** is here because release-prep and release-fix PRs are based on
# the release branch, not on main. Without it they run almost no CI and the
# first full signal is the tag build, where a failure burns the tag.
branches: [ main, 'release/**' ]
merge_group:
concurrency:
@ -960,7 +963,14 @@ jobs:
env:
BEADS_TEST_BD_BINARY: ${{ github.workspace }}/bd
GOCACHE: ${{ runner.temp }}/go-cache/race
run: go test -tags gms_pure_go -v -race -short -skip '^TestEmbedded' ./...
# -timeout is explicit for the same reason it is in scripts/ci/pr-core.sh
# (#6091): ./cmd/bd has outgrown `go test`'s 10m per-package default. On
# this runner it lands between 357s and 600s across recent runs — #6091's
# own macOS lane passed at 596.7s, 3.3s under the wall — so the package
# intermittently dies on the alarm and the panic names whichever test was
# in flight, which reads like a hang instead of the clock running out. 30m
# matches every other -race lane here for this same code.
run: go test -tags gms_pure_go -v -race -short -timeout=30m -skip '^TestEmbedded' ./...
# Producer-side guard for the Beads<->consumer CLI contract. Runs the whole
# cmd/bd/protocol package: the golden corpus (regenerated from this branch's

View file

@ -21,17 +21,25 @@ on:
paths:
- 'cmd/bd/proxied_local_*'
- 'cmd/bd/proxied_server*.go'
- 'cmd/bd/dolt*.go'
- 'cmd/bd/*_proxied_server.go'
- 'cmd/bd/*managed_local*'
- 'cmd/bd/proxied_outage_reconnect_integration_test.go'
- 'cmd/bd/db_proxy_child.go'
- 'cmd/bd/init*.go'
- 'cmd/bd/proxied_integration_helpers_test.go'
- 'cmd/bd/store_factory*.go'
- 'cmd/bd/test_dolt_server_cgo_test.go'
- 'cmd/bd/tracker_proxy_uow_conformance_integration_test.go'
- 'cmd/bd/uow_factory.go'
- 'internal/configfile/**'
- 'internal/doltserver/**'
- 'internal/storage/dbproxy/**'
- 'internal/storage/uow/**'
- 'internal/tracker/conformance/**'
- 'internal/tracker/uow_store.go'
- 'internal/tracker/engine.go'
- 'internal/tracker/store.go'
- 'go.mod'
- 'go.sum'
- '.github/workflows/proxied-local-smoke.yml'
@ -39,17 +47,25 @@ on:
paths:
- 'cmd/bd/proxied_local_*'
- 'cmd/bd/proxied_server*.go'
- 'cmd/bd/dolt*.go'
- 'cmd/bd/*_proxied_server.go'
- 'cmd/bd/*managed_local*'
- 'cmd/bd/proxied_outage_reconnect_integration_test.go'
- 'cmd/bd/db_proxy_child.go'
- 'cmd/bd/init*.go'
- 'cmd/bd/proxied_integration_helpers_test.go'
- 'cmd/bd/store_factory*.go'
- 'cmd/bd/test_dolt_server_cgo_test.go'
- 'cmd/bd/tracker_proxy_uow_conformance_integration_test.go'
- 'cmd/bd/uow_factory.go'
- 'internal/configfile/**'
- 'internal/doltserver/**'
- 'internal/storage/dbproxy/**'
- 'internal/storage/uow/**'
- 'internal/tracker/conformance/**'
- 'internal/tracker/uow_store.go'
- 'internal/tracker/engine.go'
- 'internal/tracker/store.go'
- 'go.mod'
- 'go.sum'
- '.github/workflows/proxied-local-smoke.yml'
@ -104,7 +120,12 @@ jobs:
TestManagedLocalProxiedLegacyRecordFailClosed \
TestManagedLocalProxiedOrphanBackendReaped \
TestManagedLocalProxiedNProcessColdStartConvergence \
TestManagedLocalProxiedStopStartInterleave
TestManagedLocalProxiedStopStartInterleave \
TestManagedLocalProxiedDoltStatusReportsLiveTopology \
TestManagedLocalProxiedDoltStartRefusesOverLiveProxy \
TestManagedLocalProxiedDoltStartRefusesWithProxyDown \
TestManagedLocalProxiedDoltLifecycleLeavesOtherTopologiesAlone \
TestManagedLocalProxiedTrackerUOWConformance
do
if ! grep -qx "$test_name" <<<"$listed"; then
echo "managed-local lifecycle test $test_name is missing; got: '$listed'" >&2

View file

@ -5,7 +5,7 @@ on:
push:
branches: [main]
pull_request:
branches: [main]
branches: [main, 'release/**']
# Allow manual trigger
workflow_dispatch:

65
.github/workflows/remove-needs-info.yml vendored Normal file
View file

@ -0,0 +1,65 @@
name: Remove needs-info on author response
on:
issue_comment:
types: [created]
pull_request_target:
types: [synchronize]
permissions: {}
jobs:
# pull_request_target is safe here because this job never checks out or runs
# pull request code; it only removes labels from the issue/PR metadata.
remove-label:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- name: Remove needs-info / needs-repro on author response
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
script: |
const labels = ['status/needs-info', 'status/needs-repro'];
let number, author;
if (context.eventName === 'issue_comment') {
number = context.payload.issue.number;
author = context.payload.issue.user.login;
if (context.payload.comment.user.login !== author) return;
} else {
number = context.payload.pull_request.number;
author = context.payload.pull_request.user.login;
if (context.payload.sender.login !== author) return;
}
const current = await github.rest.issues.listLabelsOnIssue({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: number,
});
const currentNames = current.data.map(l => l.name);
for (const label of labels) {
if (!currentNames.includes(label)) continue;
// Idempotent removal: concurrent triggers can race to remove the
// same label, leaving the loser with a 404 "Label does not
// exist". Treat an already-absent label as success rather than
// failing the job.
try {
await github.rest.issues.removeLabel({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: number,
name: label,
});
console.log(`Removed '${label}' from #${number}`);
} catch (err) {
if (err.status === 404) {
console.log(`'${label}' already absent from #${number} (concurrent removal); nothing to do`);
continue;
}
throw err;
}
}

View file

@ -0,0 +1,61 @@
name: Remove needs-triage when triaged
on:
issues:
types: [labeled]
pull_request_target:
types: [labeled]
permissions: {}
jobs:
# pull_request_target is safe here because this job never checks out or runs
# pull request code; it only removes labels from the issue/PR metadata.
remove-triage-label:
# Olivia owns label, comment, and inbox removal as one recoverable transaction.
if: github.event.sender.login != 'gascityinc-sarah[bot]'
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- name: Remove needs-triage when a non-status label is added
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
script: |
const added = context.payload.label.name;
if (added.startsWith('status/')) return;
const number = context.issue?.number || context.payload.pull_request?.number;
const target = 'status/needs-triage';
const current = await github.rest.issues.listLabelsOnIssue({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: number,
});
if (!current.data.some(l => l.name === target)) return;
// The check above narrows a race but cannot close it: adding
// several non-status labels at once fires one `labeled` event
// (and one job) each, and they run concurrently. Every run sees
// the label present here, then all race to remove it. The winner
// gets 204; the losers get 404 "Label does not exist". Treat an
// already-absent label as success so a benign lost race does not
// paint a red X on the PR's checks.
try {
await github.rest.issues.removeLabel({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: number,
name: target,
});
console.log(`Removed '${target}' from #${number} (triaged with '${added}')`);
} catch (err) {
if (err.status === 404) {
console.log(`'${target}' already absent from #${number} (concurrent removal); nothing to do`);
return;
}
throw err;
}

View file

@ -0,0 +1,82 @@
name: Request repro or info from reporter
on:
issues:
types: [labeled]
permissions: {}
jobs:
post-request:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: Post visible request comment for needs-repro or needs-info
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
script: |
const needsLabels = ['status/needs-info', 'status/needs-repro'];
const label = context.payload.label?.name;
if (!needsLabels.includes(label)) return;
const number = context.payload.issue?.number;
if (!number) {
core.setFailed('Unable to determine issue number');
return;
}
const owner = context.repo.owner;
const repo = context.repo.repo;
const reporter = context.payload.issue?.user?.login;
const events = await github.paginate(github.rest.issues.listEvents, {
owner,
repo,
issue_number: number,
per_page: 100,
});
const labelEvents = events.filter(e => e.event === 'labeled' && e.label?.name === label);
const labeledAt = labelEvents.length
? new Date(Math.max(...labelEvents.map(e => new Date(e.created_at).getTime())))
: null;
const existingComments = await github.paginate(github.rest.issues.listComments, {
owner,
repo,
issue_number: number,
...(labeledAt ? { since: labeledAt.toISOString() } : {}),
per_page: 100,
});
const hasVisibleRequest = existingComments.some(c => {
if (reporter && c.user?.login === reporter) return false;
const lower = (c.body || '').toLowerCase();
if (lower.includes('closing')) return false;
if (!/14\s*-?\s*days?/.test(lower)) return false;
if (!lower.includes('reply') && !lower.includes('respond')) return false;
if (label === 'status/needs-repro') return lower.includes('repro') || lower.includes('reproduction');
return lower.includes('info') || lower.includes('information') || lower.includes('details');
});
if (hasVisibleRequest) {
console.log(`Visible request already exists for #${number}, skipping.`);
return;
}
const requestBodies = {
'status/needs-repro':
'Thanks for the report! To help us investigate, please reply with a minimal reproduction within 14 days.\n\n' +
'If there is no response in that window, this issue may be closed — but you are welcome to respond or reopen it at any time.',
'status/needs-info':
'Thanks for the report! To help us investigate, please reply with the requested information or details within 14 days.\n\n' +
'If there is no response in that window, this issue may be closed — but you are welcome to respond or reopen it at any time.',
};
await github.rest.issues.createComment({
owner,
repo,
issue_number: number,
body: requestBodies[label],
});
console.log(`Posted needs request for #${number} (${label})`);

41
.github/workflows/triage-label.yml vendored Normal file
View file

@ -0,0 +1,41 @@
name: Auto-label new issues and PRs
on:
issues:
types: [opened, reopened]
pull_request_target:
types: [opened, reopened, ready_for_review]
permissions: {}
jobs:
# pull_request_target is safe here because this job never checks out or runs
# pull request code; it only labels the issue/PR from event metadata.
add-triage-label:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- name: Add needs-triage label
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
script: |
const pullRequest = context.payload.pull_request;
if (pullRequest?.draft) {
console.log(`Skipping draft PR #${pullRequest.number}`);
return;
}
const number = context.issue?.number || pullRequest?.number;
if (!number) {
core.setFailed('Unable to determine issue or PR number');
return;
}
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: number,
labels: ['status/needs-triage']
});

View file

@ -7,13 +7,408 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
- **The smart migrate gate no longer auto-migrates a clone whose data is behind
the remote, and `bd dolt pull` now works from that state**
([#6575](https://github.com/gastownhall/beads/issues/6575)). The gate's
first-mover verdict was computed from schema facts alone, so a clone level
with the remote on schema but behind it in unpulled data commits was
classified a safe first-mover and migrated in place — minting local-only
schema commits on a history missing those commits, which can leave every
later `bd dolt pull` refusing to merge
([#6368](https://github.com/gastownhall/beads/issues/6368)). The
equal-version path now reads the branch position its callers already supply
and stops with a pull-first remedy instead.
The stop applies whenever the clone has commits left to pull, whether or not
it also has local commits of its own. Since bd auto-commits every write, a
clone that is both behind and ahead is the ordinary multi-machine state and
reaches the same wedge; it is told its pull will *merge* rather than
fast-forward, and how to resolve conflicts if it reports them. A clone that
is level, or that only has unpushed local commits with nothing to pull,
still auto-migrates exactly as before.
Crucially, the refusal no longer blocks its own remedy. `bd dolt pull` opens
the store too, so on a refused clone it used to hit the very refusal that
prescribed it — and an embedded workspace has no external `dolt` binary to
fall back to, which left `BD_ALLOW_REMOTE_MIGRATE=1` (performing the
migration the stop exists to prevent) as the only way forward. `bd dolt pull`
now opens leniently for this one refusal, the way `bd dolt commit` does for
the dirty-working-set refusal
([#4566](https://github.com/gastownhall/beads/issues/4566)); every other gate
refusal still fails that open. This supersedes the 1.3.0 upgrade note's
statement that the pending-migration gate refuses `bd dolt pull`: for the
data-behind stop it does not, by design. Re-run the command you were blocked
on once the pull completes and the migration proceeds.
Commands that keep working through the stop say the same thing. `bd list`,
`bd ready` and `bd show` succeed against the old schema, and `bd dolt commit`
still commits the working set; on embedded storage all of them used to print
the blunt migrate-or-adopt coordination bullets while doing so — naming
`bd migrate --force && bd dolt push`, which on a data-behind clone applies
the migration this stop exists to prevent and then fails the push
non-fast-forward, and `bd bootstrap`, which no-ops against an existing
workspace. They now print the pull-first guidance, shape-branched the same
way the fatal refusal is, alongside the note that the command in hand
continued at the current schema. Refusals that are *not* the data-behind stop
keep the coordination bullets, which are right for them.
`--json` callers get the same remedy the terminal does: `observed`,
`expected` and `options` describe the pull (a single `pull-first` option)
rather than the migrate-or-adopt decision that does not apply here, plus a
`data_behind_shape` of `fast-forward` or `diverged`. For the fast-forward
shape on a non-shared store — one unconditional option, no local commits,
nothing discarded — `human_decision_required` is now `false`, so an agent
can run the pull instead of stalling for approval of a step the same payload
calls riskless. It stays `true` for the diverged shape (the pull merges and
can need conflict resolution) and on a shared store (the follow-up consent
step needs an operator who can confirm every co-resident client is upgraded).
On a shared Dolt sql-server the guidance still carries #5920's consequence —
migrating promotes the schema for every co-resident client — and names the
consent step the retry needs there: `bd migrate schema --force`. It is the
forced form because this stop always has a remote configured (behind-ness is
read from the remote-tracking ref), and the bare `bd migrate schema` consent
is only read for a shared database with *no* remote; the flag consents to
migrating a remote-backed shared store, and by then the pull has landed the
commits the clone was missing. The payload's `docs` link points at the
data-behind section rather than the migrate-or-adopt recipe, and that
section — plus the upgrade and init-safety ordering rules — now records the
`bd dolt pull` exception instead of stating the pull is always refused.
Both existing escape hatches are unchanged: `bd migrate --force` /
`BD_ALLOW_REMOTE_MIGRATE=1` are still consulted before the smart gate, and
`BD_SMART_GATE=0` still yields the blunt gate unconditionally (including for
`bd dolt pull`, which is what opting out means). Pulling before the first
open after upgrading avoids the state entirely.
### Added
- **`bd backup` works on a proxied-server workspace bd runs the Dolt server
for.** `bd backup init`, `sync`, `remove`, `status` and `restore` are routed
over the proxied provider; before this, a proxied workspace — the default
topology since 1.3.0 — had no backup path at all. `bd backup restore` stops
the proxy and its Dolt child before replacing the database and leaves the
workspace quiescent, so the next command relaunches against the restored
data.
The family stays **refused by design** on a proxied workspace pointed at a
Dolt server bd does not own (an external host, socket, or a beads-team-server
database): `CALL DOLT_BACKUP('add', …)` registers the backup remote on the
*server*, where it is global to every client connected to it, so one
workspace's backup decision would silently become everyone's — the shape
`bd backup`'s own help text already warns about for auto-backup. A `file://`
destination has a second problem on top, since the server resolves that path
on its own filesystem. The refusal keeps the `proxy.backup.unsupported` code
and now carries `"reason": "design"`; a backup story for those deployments
belongs to whoever runs the server. Whether a *remote-scheme* destination
(DoltHub, `aws://`, `gs://`) should eventually be allowed there is recorded as
an open question rather than settled — the server pushes those over the
network from its own environment, which is the same credential question the
next slice has to answer for `dolt push`.
- **`bd count` supports repeatable `--metadata-field key=value` filters**
([#6023](https://github.com/gastownhall/beads/issues/6023)), so callers can
count the same metadata-scoped set `bd list` returns without fetching every
row.
### Fixed
- **`bd dolt start` no longer puts a second sql-server over a proxied
workspace's data directory.** On a proxied-server workspace the default root
IS `.beads/dolt`, and `bd dolt start` knew nothing about proxied mode: with
the proxy quiesced it launched an unsupervised `dolt sql-server` directly
over that root, after which ordinary bd commands failed (`invalid
connection`) because the relaunched proxy found a foreign server on its
port; with the proxy live it instead adopted the proxy's own dolt child into
bd's classic PID/port records, leaving two managers for one process. It is
now a typed refusal with code `proxy.dolt_start.conflict`. The proxy owns
its backend's lifecycle — it starts on demand and `bd dolt stop` shuts it
down. Direct-server and embedded workspaces are unaffected.
- **`bd dolt status` tells the truth on a proxied workspace.** It read the
classic PID file, which proxied mode never writes, and so reported `Dolt
server: not running` while the proxy was serving CRUD — the wrong answer
that sent operators to `bd dolt start` in the first place. It now reads the
proxy's own records and reports the proxy and its dolt backend separately,
without starting either. **JSON output shape change**: on a proxied
workspace `bd dolt status --json` now emits
`{"mode": "proxied-server", "root": ..., "running": <proxy up>, "proxy_pid":
..., "proxy_port": ..., "backend_managed": ..., "backend_running": ...,
"backend_pid": ..., "backend_port": ..., "idle_timeout": ...}` instead of
the always-false `{"running": false, "pid": 0, "port": 0}`. `running`
describes the proxy, the endpoint every bd command connects through;
`backend_managed` is false on external proxied topologies, where the dolt
server is not bd's process to report on.
- **`bd mol ready --gated` no longer refuses under `--proxied-server` when
`BEADS_MAX_ROWS` is set**
([#6293](https://github.com/gastownhall/beads/pull/6293)). The proxied
capability front door keyed its rules on cobra's leaf name, and `bd ready`
and `bd mol ready --gated` share the leaf "ready", so the row cap refusal
written for the former also refused the latter — a proxy-supported command
with no `--max-rows` flag to refuse. Rules are keyed on the full command
path now.
### Changed
- **Proxied-server refusals now say *why* they refuse.** The JSON a refused
command prints gains a `reason` field next to the existing `code`, `error`
and `mutates`: `design` for a refusal that is expected to stay (shared
history, multi-repo routing, destructive admin, strict `--readonly`) and
`unimplemented` for a capability gap with a named owner. Adding `reason` left
every existing code, message and exit status as it was, and `reason` is absent
on refusals that report a runtime state rather than a policy, so existing
consumers are unaffected. The policy behind it moved into one registry
(`cmd/bd/capability_registry.go`) that every command must appear in.
- **A command with no proxied-server route now fails with a typed error**
(`proxy.store.unrouted`) instead of the bare string `proxy server store
should be uow provider`.
- **`bd backup restore --json` prints a result object on every topology**
(`{"restored": true, "source": "<dir>"}`). It previously printed nothing at
all, which left a caller unable to tell a completed restore from a silently
skipped one. This applies to direct/embedded workspaces too, not only proxied
ones.
- **`bd backup init`, `sync`, `remove` and `restore` no longer print a usage
block after a failure.** They now silence cobra's own error and usage
rendering like the rest of the CLI, so a failure is one error line on stderr
instead of the message followed by `Error: exit code 1` and the full usage
text. Exit statuses are unchanged; this applies on every topology.
- **Proxied `--repo` and row-cap refusals are typed again**
([#6293](https://github.com/gastownhall/beads/pull/6293)). `bd list --repo`
under `--proxied-server` answers `--json` with the stable
`{"code": "proxy.repo.unsupported", …, "mutates": false}` on stdout rather
than prose on stderr, and `bd ready --claim --max-rows N` refuses with the
same `proxy.max_rows.unsupported` shape as `bd ready --max-rows N`: `--claim`
does not exempt a row cap the proxied route cannot enforce either way.
- **The proxied `bd admin compact` refusal names the command it means**
([#6293](https://github.com/gastownhall/beads/pull/6293)). The message is now
"only 'bd admin compact --dolt' is supported in proxied-server mode"; it read
"only 'compact --dolt'", which names the root `bd compact` — a different
command, with no `--dolt` flag and its own proxied route. Scripts matching
the old text need updating.
- **`bd preflight` honors the `json` config default**
([#6293](https://github.com/gastownhall/beads/pull/6293)). Its `--json` flag
is bound to the same global every sibling command binds, so `json: true` in
the config file now selects JSON output for `bd preflight` as it already did
elsewhere. Previously only the explicit flag was honored — and, because an
unbound flag reports as unset, `bd preflight --json` did not reach every
JSON-aware renderer.
- **`bd gate check` resolves bead gates whose target lives in a prefix-routed
rig** ([#5859](https://github.com/gastownhall/beads/pull/5859)). After a local
miss, the evaluator follows the target bead ID through `routes.jsonl` and
reads the owning store without writing to it. This covers explicit gate
checks in embedded, server, and proxied-server command paths; the legacy
`<rig>:<bead-id>` await value remains accepted for compatibility.
### Fixed
- **A dotted config key now round-trips: what `bd config set` writes,
`bd config get` and bd's own readers find**
([#6574](https://github.com/gastownhall/beads/pull/6574),
[#6594](https://github.com/gastownhall/beads/issues/6594), bd-zj95). Setting a
key like `sync.remote` or `dolt.host` into a `config.yaml` with no matching
section appended a key whose *name* contained the dot — a top-level
`sync.remote: "..."` — instead of nesting it under `sync:`. Viper finds a key
spelled that way; the direct reader `GetStringFromDir` splits on the dot and
walks nested mappings, so it does not. The value was set and invisible at the
same time depending on which reader asked, and `bd init --remote` in a fresh
workspace was the ordinary way in: the remote was recorded and then not found.
New dotted keys are written nested now. An existing flat spelling is updated
in place for compatibility with other writers of the shared config file —
rewriting that one line and nothing else, the comment on it included — and
bd's direct reader recognizes both forms with the same precedence as Viper.
`bd doctor` warns when both forms coexist. `bd config unset` removes the
nested form — it only ever matched the flat spelling, so unsetting
`sync.remote` silently left the remote live. Keys the write does not own,
including dotted ones somebody else put there, are left exactly as they are.
Four things that used to succeed now report an error instead, because each one
produced a value no reader could see, or claimed to remove a value it had
left live. On `bd config set`: a key under a parent that already holds a value
(`sync: enabled`, then `bd config set sync.remote`), and a key in a file whose
top level is not a mapping. On `bd config unset`: a key inside a flow-style
mapping (`sync: {remote: "..."}`), which bd matches by line and so cannot
edit, and a key whose value is a block scalar (`remote: |`, in either
spelling), where commenting the key out would leave the body behind as a value
of its own. Unsetting a key that is not set remains a successful no-op in
every shape.
- **`bd config unset` no longer reports success while the key stays set, and
now names where the unset actually landed.** A key present in
`config.yaml` but not claimed by `IsYamlOnlyKey` was deleted from the
database only, so it stayed effective; it is now cleared from both, and the
reported location - `Unset <key> (in database, config.yaml)` - is derived
from what the writes actually changed rather than from a pre-check of viper's
merged value, which counted defaults and environment variables as present in
the file. A yaml-only key that was not in the file prints `<key> was not set
in config.yaml` instead of claiming a write. A workspace with no project
`config.yaml` at all is an answer rather than a failure, so a database-backed
unset there no longer deletes the row and then exits non-zero. Unsetting a
key whose value is the indented block beneath it (a mapping or a list), which
used to comment out the key line and orphan the block into a `config.yaml`
that no longer parsed, is now refused alongside the flow-style and
block-scalar shapes above. An unset also keeps the file's trailing newline.
## [1.3.0] - 2026-09-15
The first tested release off `main` since the 1.1 line. [1.2.2] was a recovery
release that re-shipped the v1.1.2 code under a higher version number, so a
v1.2.2 user is running 1.1-era code and meets **everything** below at once:
the changes listed here, plus the [1.2.1] changes that release deliberately
withheld. Read the upgrade notes before installing.
### Upgrade notes
**On an embedded or local store, the first invocation migrates your schema, in
place, from v53 to v66.** A v1.2.2 (or any 1.1.x) database sits at main-series
schema v53; this binary knows v66, so the first command that opens the store
applies 13 main-series migrations, `0054_add_lease_columns` through
`0066_add_events_journal_actor`. Two of those passes rewrite rows rather than
just reshaping tables — the aux-row id rekey and the `events` dolt_ignore flip
described under **Changed** — so on a large store the first invocation is
*noticeably* slower than the ones after it. It is crash-resumable and picks up
where it left off, but do not interrupt it if you can avoid it.
**A shared Dolt sql-server is never auto-migrated** (#5920, #6048): migrating it
promotes the schema for every client at once, so it waits for explicit consent
via `bd migrate schema`. Remote-backed stores keep their existing
designated-migrator gate. Upgrade all of that server's clients *first* — see
[Shared servers](https://beads.gascity.com/getting-started/upgrading#shared-servers).
**The counter restarts partway through, and that is not a loop.** The
clone-local (dolt_ignored) series runs after the main one, through the same
printer and its own numbering, and it moves 0011 → 0026 on this upgrade. So the
run is about **28 migrations**, not 13, and what you see on stderr is 13 lines
counting up to 0066 followed by 15 lines starting again at 0012:
```
Applying migration 0065: widen_wisp_comments_text…
Applying migration 0066: add_events_journal_actor…
Applying migration 0012: create_leases… ← clone-local series, not a restart
```
A counter that jumps backwards is the signature operators kill runs over. Let
it finish.
Progress prints only when stderr is a terminal. Piped and CI runs see nothing
at all, which is deliberate — a silent-looking CI upgrade is not a stuck one.
**Back up first, with the binary you have now.** Take the backup *before* you
install 1.3.0. Under the new binary `bd export` triggers the auto-migration
before it exports, so a snapshot taken afterwards is a post-migration snapshot
and cannot protect you against the migration going wrong. On a remote-backed
store, finish syncing with the old binary too: once 1.3.0 is installed the
pending-migration gate refuses `bd dolt push` and `bd dolt pull` as well, not
just `bd migrate`.
```bash
# with your CURRENT bd, before installing 1.3.0:
bd dolt push # remote-backed stores only
bd export --all -o .beads/backup/pre-1.3.0-$(date +%Y%m%d).jsonl
```
A JSONL export is cheap, issue-complete, and importable by any bd version. If
you want a Dolt-native snapshot that keeps history and config, configure a
destination and sync it — **bare `bd backup` takes no backup**, it is a command
group that prints help and exits 0:
```bash
bd backup init <path-or-dolthub-url> # once, to configure a destination
bd backup sync # take the snapshot
```
**Upgrade every client that shares a store, together.** The forward schema-skew
guard means an older co-resident binary — a second `bd` earlier in `PATH`, a
long-running `bd serve`, another clone's cron job — refuses a database migrated
past what it knows, rather than proceeding blind. That is the guard working, not
a bug, but it makes a mixed-version fleet a broken fleet: one machine running
1.3.0 takes the whole store forward and every 1.2.2 client stops. Run
`which -a bd` after installing, and on a remote-backed store follow the existing
designated-migrator procedure (one clone migrates and pushes; the rest pull or
re-clone). See
[Upgrading](https://beads.gascity.com/getting-started/upgrading) for the
per-install-method recipes and the multi-clone flow.
**If you need to go back**, the rollback is a schema-cursor rollback, not a
downgrade of the data: the procedure is written up in the
[recovery runbook](https://beads.gascity.com/recovery/accidental-1-2-1-release)
(repo copy: `docs/recovery/accidental-1-2-1-release.md`). Its worked example is
the v53↔v65 case from the accidental 1.2.1 release; the steps are the same for
v66, with the version numbers adjusted.
### Breaking changes for v1.2.2 users
**This is a highlights list, not the complete set.** A v1.2.2 user is crossing
two releases at once, and the breaks below are the ones most likely to stop a
script or a service. The full set is the `[1.2.1]` section further down plus the
**Changed** section here — read `[1.2.1]` in full before upgrading, since
[1.2.2] withheld it and v1.2.1 itself was pulled, so nobody on the supported
line has seen it.
After upgrading, `bd upgrade review` prints exactly the entries between the
version you were running and this one. (Prefer it to `bd info --whats-new`,
which dumps the entire release history.)
**Carried from [1.2.1] — never shipped to v1.2.2 users:**
- **`bd update --status <done-status>` now enforces close policy.** Moving an
issue to `closed` (or any configured done-category status) via `bd update`,
`bd batch update`, or the issueops facade refuses when the issue has open
children or a live direct blocker, matching `bd close`. Override with
`bd update --force` (`update <id> status=closed force=true` in `bd batch`;
`UpdateRequest.ForceClosePolicy` for facade consumers). An unforced refusal
rolls back the entire batch.
- **`bd search` includes closed issues by default** (bd-t5yex). The dominant
query is "was this already found/filed/fixed?", where excluding closed issues
produced a false "no". Narrow with `--status open` to get the old behavior.
`bd list` keeps its open-only default.
- **The no-ID "last touched issue" fallback on `bd update` / `bd close` is
interactive-only** (bd-m00pb,
[#4839](https://github.com/gastownhall/beads/pull/4839)). A scripted
`bd update $ID …` with an empty `$ID` now refuses in argument validation
instead of mutating whatever was touched last. The fallback requires a
terminal stdin; `BD_NON_INTERACTIVE=1` and `CI=1/true` also disable it. Set
`BD_LAST_TOUCHED_FALLBACK=1` if a script genuinely relied on it.
- **`bd human list` hides done/frozen and pinned beads by default, and
validates `--status`** (#5332). It previously passed `--status` through
unvalidated and showed closed beads. A `--status` typo is now an error rather
than an empty list. Every bead *type* still shows.
- **`bd dolt push` and `bd sync` no longer adopt a git-origin-derived Dolt
remote without consent** (#5068). Both used to silently derive a remote from
`git remote get-url origin`, persist `sync.remote`, commit that config change
under your git identity, and upload the full issue history — which published
a whole issue database to a public origin on a command the user believed
targeted an already-configured remote. Adoption now prompts (defaulting to
no) and fails closed non-interactively. `--yes`/`-y` consents ahead of time.
- **`bd --readonly serve` is refused instead of binding a server that cannot do
what it advertises.** On a Dolt SQL-server workspace the flag was previously a
silent no-op and the server came up fully writable, so anything scripted as a
"safe" read-only server **does not start after this upgrade** — drop the flag.
Worth checking before you restart a long-running `bd serve` as part of the
fleet upgrade above.
- **`bd config list` and `GET /v0/beads/config` no longer enumerate the `kv.`
plane**, which is where `bd remember` memories live. That closed an
unauthenticated `GET /v0/beads/config` handing out every stored memory. A
script that read memories out of `bd config list` now gets empty output rather
than an error; use `bd kv` / `bd remember` instead.
- **The published `backend` package drops orphan handling** (bd-gwryr).
`backend.OrphanHandling` and its four constants, the `storage` originals they
aliased, `BatchCreateOptions.OrphanHandling` and `issueops.CheckOrphan` are
removed. They never did anything except cost a query, but a Go consumer that
names them no longer compiles.
**New in 1.3.0** (full entries under **Changed**):
- An explicitly configured Dolt server port now outranks the ambient
`BEADS_DOLT_PORT` environment variable.
- Actor matching decodes an exact `--` run to `/` instead of collapsing it to a
generic separator, so `gastown--mayor` matches `gastown/mayor` and stops
matching `gastown__mayor`.
### Added
- **The events journal records WHO performed each mutation.** `bd_events_journal`
gains an `actor` column (migration 0066 plus its ignored-series twin 0025, so
upgraded workspaces and fresh clones converge on the same shape), stamped
@ -56,29 +451,223 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
concept, themes are labels here, and `--label theme:x` is the native way to
say it.
- **New `*.gate.lock` files appear next to and inside `.beads`** ([#5046](https://github.com/gastownhall/beads/pull/5046),
[#5093](https://github.com/gastownhall/beads/pull/5093)). A two-level
cooperative gate now serializes the operations that cannot safely overlap —
ordinary commands take it shared, maintenance like `bd backup restore` takes
it exclusive. It is `flock`-based, so the file is the lock's *name*, not its
state: the gate is created once (mode `0600`) and deliberately never deleted,
because deleting it is how two processes end up holding two different locks
for the same resource. Each gate sits *beside* what it guards, never inside
it: the workspace gate for `<dir>/.beads` is `<dir>/.beads.gate.lock` in the
project root, and the physical-root gate is `.beads/embeddeddolt.gate.lock` in
the default embedded mode (`.beads/dolt.gate.lock` under a server layout;
under the shared server it lives beside `~/.beads/shared-server/dolt` and
never enters the workspace at all; a remote Dolt host has no physical gate).
Both are covered by the `*.gate.lock*` pattern `bd doctor` maintains in the
project and `.beads` gitignores, so an upgraded workspace picks up the ignore
rule on the next `bd doctor --fix`; a workspace with no `.beads` directory is
not gated at all, rather than scattering lock files into whatever directory
`bd` was run from.
- **`sort` on `GET /v0/beads/issues`.** The listing served one order —
`(created_at DESC, id ASC)` — because the cursor is a keyset position in it,
and the spec said so in its own words: "the sort order is welded to the cursor
contract, and a new order needs new surface." The cost of that welding fell on
every client that wanted `bd list`'s ordering, because the only way to get it
was to page the whole result set and re-sort locally. Measured over a
1400-row store at the 200-row page an HTTP client actually uses, that is
**7 requests plus a client-side comparator; `?sort=priority` is 1**
(`TestProxiedServerListSortRetiresTheWalk`, which runs both strategies against
a real `bd serve` and checks both against `bd list --json` row for row). It is
the one cost on this surface that got worse as a project grew.
**Two values, and the set is closed.** `created` is the existing order, now
spellable. `priority` is `(priority ASC, created_at DESC, id ASC)` — `bd
list`'s flagless ordering, which is also what `bd list --sort priority`
produces, so one served order retires the walk for both. The other seven
orders `bd list --sort` takes are not offered: each value here is a cursor
contract needing a key proven total, and `id` is a natural-numeric order no
database expresses, `updated` moves on every write, `closed` is nullable, and
`status`/`title`/`type`/`assignee` are mutable and unindexed.
**Absent `sort` still means `created`, permanently.** It is the compatibility
contract for every client written before the parameter existed; changing it
would alter which rows a truncated page contains, with no error to notice it
by. A server that predates the parameter answers `param: "sort"` with
`reason: "unknown_parameter"`, which is the per-parameter capability probe a
client dispatches on to fall back.
**The cursor now carries its order, and a mismatch is refused.** This is the
half that makes the parameter safe rather than merely useful. The token was
base64 of `{t,i}` with no binding, so a `created`-order cursor replayed under
`sort=priority` would have decoded perfectly and been read as a position in a
different total order — a page that both skips and duplicates rows, served
with a 200, undetectable by the client. Tokens are now `v2` with an explicit
order member and decode refuses any token whose order differs from the
request's, as `invalid_cursor` (documented recovery: restart paging).
Outstanding `v1` tokens stay readable as the `created`-order positions they
are, so **no traversal in flight has to restart**. This does not contradict
"the token carries no filters": filters select the set, the order decides what
the position means.
**`priority` is a mutable key**, which `created_at` is not, and the spec says
so: under `created` only new rows move relative to a walk, while a priority
update moves an existing row too, so it can be seen twice or missed. That is
the already-documented "a cursor pins a position, not a snapshot" caveat
reached by a second route, not a new class of error — unchanged data never
skips or repeats under either order.
### Changed
- **`bd gate check` resolves bead gates whose target lives in a prefix-routed
rig** ([#5859](https://github.com/gastownhall/beads/pull/5859)). After a local
miss, the evaluator follows the target bead ID through `routes.jsonl` and
reads the owning store without writing to it. This covers explicit gate
checks in embedded, server, and proxied-server command paths; the legacy
`<rig>:<bead-id>` await value remains accepted for compatibility.
- Shared Dolt databases (sql-server mode) no longer auto-apply schema
migrations on a version bump. Migrating a shared database promotes the
schema for every connected bd client at once and locks out clients still
on an older binary, so bd now refuses without explicit consent: run
`bd migrate schema` once after upgrading every client, or set
`BD_ALLOW_REMOTE_MIGRATE=1` in scripted use. Reads keep working on the
current schema meanwhile. Embedded (single-user) databases still
auto-migrate silently, unchanged. (#5920)
- **Commands that cannot work under `--proxied-server` now refuse before any
provider opens** ([#6293](https://github.com/gastownhall/beads/pull/6293)).
`bd doctor`, `bd backup`, `bd restore`, `bd diff`, `bd migrate`, `bd branch`,
`bd conflicts`, `bd vc`, `bd federation`, `bd repo`, `bd flatten`, `bd sync`,
`bd dolt push|pull|commit`, `bd dolt remote add|list|reset-data`, and
`bd admin compact` without `--dolt` return a typed refusal carrying a stable
code (`proxy.doctor.unsupported` and friends) instead of failing partway
through, or silently doing nothing. Two changes worth planning for:
- `bd doctor` under `--proxied-server` used to print a note to stderr and
exit **0**; it now exits **1** with `proxy.doctor.unsupported`. A script
that treated bare `bd doctor` as a no-op in this mode needs to stop calling
it, or tolerate the non-zero status.
- Under `--json`, a refusal is strict JSON on **stdout**
(`{"code": …, "error": …, "mutates": false}`) rather than prose on stderr,
so a wrapper parsing stdout gets a machine-readable answer and
`"mutates": false` states that the refusal touched nothing.
- **Write commands now refuse to run while a MIGRATION-FREEZE sentinel sits
at the town root** (dc-6jaq), mirroring the gate the gt CLI already applies
to `gt mail send`/`nudge`/`sling`/`assign`. `bd create`/`update`/`close`/
`remember`/`import` and every other command gated by `CheckReadonly`
(~120 call sites, `bd q` included) now print `⛔ town is frozen for
migration (by <operator>)` and exit 1 instead of writing to a store mid-
migration. The check runs twice: once early in the root command, before
version-bump auto-migration or JSONL auto-import can touch the store, and
again at each write command's own chokepoint. Read commands (`list`,
`show`, `ready`, …) keep working during a freeze — the same early check
also skips version tracking and auto-migration for them, so a frozen store
is never rewritten just because someone ran a read. `--dry-run`/`--inspect`
previews are blocked at the per-command chokepoint instead, same as strict
`--readonly` already blocks them. Clear the freeze with `gt migrate thaw`.
- The smart migration gate's "auto-migrate as safe first-mover" and
auto-fast-forward arms are now embedded-only. On a shared server the gate
always stops for consent, because neither argument can observe the other
clients attached to that server. (#5920, #4516)
- The newly exported Go packages (`backend`, `backend/conformance`, `beadserrors`,
`issueops`, `journalops`, `memoryops`, `schema`, `test/conformance`) are marked
EXPERIMENTAL: they are not yet covered by the project's compatibility promise and
may change in a minor release. Pin an exact beads version and re-run the
conformance suite on every bump.
- **The optimistic-concurrency token is now a JSON string** — responses carry
`revision` as an opaque decimal string and `expected_version` must be sent as
that string. JSON-number tokens exceeded JavaScript's 2^53 integer precision.
This member never shipped in a tagged release.
- CI: migration-test and upgrade-smoke now seed and assert wisp-plane data
(wisps, wisp deps/comments, leases, events, ignored-track cursor) across
v1.0.1/v1.1.x/v1.2.2 → candidate upgrades.
- **`--profile` is now `--cpu-profile`, with no alias**
([#5126](https://github.com/gastownhall/beads/pull/5126), bd-ugz). The
persistent flag that writes a CPU profile is spelled `--cpu-profile`; the old
`--profile` spelling is gone rather than deprecated, so it fails as an unknown
flag instead of silently doing nothing. `--profile` is the conventional name
for a *named configuration*, not for pprof output, and holding it hostage to
a debugging aid nobody passes twice a year was the wrong trade. Scripts and
aliases that profile bd need the new spelling.
- **Migration 0061 rekeys every events, comments, snapshot and
compaction-snapshot row to a content-derived id**
([#5150](https://github.com/gastownhall/beads/pull/5150), bd-ri8bd). Those
four aux tables minted random UUIDv7 ids, which is fine for a single writer
and wrong for replication: unversioned newest-wins replication has no merge
pass, so two clones recording the same fact produced two rows that never
converged. Ids are now UUIDv5 over a SHA-256 digest of the row's own content
(frozen column order, plus the table name and a duplicate-disambiguating
ordinal), so the same fact derives the same id everywhere and convergence is
a property of the key rather than of a reconciliation step. New rows derive
their id at insert; existing rows are converted by a **one-time bulk rekey**
on the first open after upgrade, which is the main reason that invocation is
slower than the ones after it. The pass is crash-resumable through a
`local_metadata` sentinel and records completion clone-locally, so it runs
once per clone and is not repeated on the next command. `wisp_events` and
`wisp_comments` are deliberately excluded — they are clone-local and never
merge. **Any external reference to an event, comment or snapshot id taken
before the upgrade will not resolve afterwards.**
- **The `events` audit table is now clone-local**
([#5162](https://github.com/gastownhall/beads/pull/5162), bd-red8u).
Migration 0062 moves `events` onto the dolt_ignored plane in a self-committing
drop/commit/recreate that preserves every row. Events are single-writer audit
rows with low cross-machine value, and minting a versioned Dolt commit for
each one was the dominant source of commit churn on a busy store. The
consequence is the point: **`events` rows no longer replicate.** They keep
full SQL durability locally and `bd history <id> --events` still reads them,
but a pull no longer brings another machine's events over, and a push no
longer sends yours. Events already committed before the flip stay in the
remote's history until the next squash window. Comments stay versioned.
- **The `interactions.jsonl` audit sidecar is opt-in**
([#4688](https://github.com/gastownhall/beads/pull/4688), fixes
[#4687](https://github.com/gastownhall/beads/issues/4687)). `audit.enabled`
now defaults to **false** and `bd init` no longer creates the file, so
status/assignee/priority field-change logging writes nothing unless you ask
for it. Set `audit.enabled=true` (or `BD_AUDIT_ENABLED=1`) to restore the old
behavior. The database-backed replacement is `bd history <id> --events`, which
needs no sidecar. Anything that read `.beads/interactions.jsonl` on the
assumption it would exist needs the config key set.
- **`bd import` refuses a redirected stdin rather than quietly ignoring it**
([#5171](https://github.com/gastownhall/beads/pull/5171), bd-axluy). Bare
`bd import` imports the default JSONL, and `bd import -` imports stdin; a
`bd import < file.jsonl` looked like the second and behaved like the first,
importing a different file than the one on the command line with no way to
notice. With stdin redirected, no `-`, and no named source, `bd import` now
errors and names both fixes. `/dev/null` is a character device, so a scripted
bare `bd import` under a subprocess still imports the default file as before.
- **Auto-backup defaults to off under a Dolt sql-server** (wy-zrmqr). The
automatic backup used to switch itself on whenever a git remote existed,
regardless of storage mode — which on one shared sql-server meant ~31 clients
independently deciding to back up the same database, pinning its CPU. The
default is now off in server, proxied-server and shared-server modes (and in
nocgo builds, which are always server-backed); embedded mode is unchanged and
still enables it when a git remote is present. An explicitly configured
`backup.enabled` (or `BD_BACKUP_ENABLED`) always wins over the default, and
`bd config get backup.enabled` now prints the effective value with its source
— `default (auto: off in sql-server mode)` — rather than a bare `false` that
could not be told apart from a configured one.
- **`bd hooks install --chain` and `--force` are accepted no-ops**
([#5284](https://github.com/gastownhall/beads/pull/5284), bd-5vdt8). Managed
`BEGIN`/`END BEADS INTEGRATION` marker sections made both flags meaningless:
an install now replaces only the content between its own markers, so content
outside them is *always* preserved (what `--force` used to negate) and an
existing hook *always* keeps running alongside the bd section (what `--chain`
used to request). The flags remain registered so existing invocations do not
break, and `--json` still echoes them, but neither changes what is written.
A hook with no markers is handled by shape: a legacy bd hook is replaced, and
a foreign one gets a one-time `<hook>.backup` sidecar before the section is
injected. Symlinked and git-tracked hook paths are refused outright, before
anything is written.
- **Write commands refuse to run while a migration freeze marker is present**
(dc-6jaq). When a file named `MIGRATION-FREEZE` exists in the workspace
directory, the working directory, or any ancestor of either — or at the
path named by `BD_MIGRATION_FREEZE_FILE`, which is authoritative when set
— `bd create`/`update`/`close`/`remember`/`import` and every other write
command (~120 call sites, `bd q` included), plus the destructive
`bd init --reinit-local` and `bd bootstrap`, print `⛔ workspace is frozen
for migration` naming the marker's path and exit **14**, a distinct code
scripts can branch on, instead of writing to a store mid-migration.
Operators freeze a workspace by creating the file (optionally
`operator<TAB>RFC3339 timestamp<TAB>reason` on one line, echoed back in
the refusal) and thaw it by removing the file. Read commands (`list`,
`show`, `ready`, …) keep working during a freeze, and bd's own maintenance
stands down with them — version tracking, auto-migration, JSONL
auto-import, Dolt auto-commit, auto-backup, auto-export and auto-push are
all skipped, so a frozen store is never rewritten just because someone ran
a read. A running `bd serve` is not gated: stop it before freezing. If bd
cannot tell whether a marker is present, it refuses rather than assuming
the workspace is open.
- **`bd dep add` names the implicit `type=blocks` default, but only to an
interactive operator** (#5854). Creating an edge with no `-t/--type` silently
@ -154,23 +743,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
mode still fails closed there — a repo-local auto-start is a different
database, not a port refresh — but now says so in its own words.
- **`bd config unset` no longer reports success while the key stays set, and
now names where the unset actually landed.** Three separate ways the old
command lied: a key present in `config.yaml` but not claimed by
`IsYamlOnlyKey` was deleted from the database only, so it stayed effective;
`commentOutYamlKey` matched only the flat `a.b: v` spelling, so a key written
in the nested form `SetYamlConfig` produces was never touched; and commenting
a key whose value is the block beneath it orphaned that block, leaving a
`config.yaml` that no longer parsed. The nested form is now edited at the
YAML node level (source line preserved, so comments and layout survive), a
block opener and a leaf that does not own its own line are refused rather
than corrupted, and the reported location - `Unset <key> (in database,
config.yaml)` - is derived from what the write actually changed rather than
from a pre-check of viper's merged value. A key that was never in
`config.yaml`, and a workspace with no project `config.yaml` at all, are
answers rather than failures: neither claims a file write, and neither leaves
the database row deleted behind a non-zero exit.
- **`bd reclaim` summarizes the leases its replica guard declined instead of
naming every one, every run** (wy-sp2l4). A lease granted by another replica
is by construction never reclaimed here, so the audit was not a one-off: it
@ -200,8 +772,134 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
encoding changes equivalence class silently, with no error to notice. Longer
or mixed runs, `__` and `---` included, are unaffected and still collapse.
- **`bd ready --claim` now refuses a row cap under `--proxied-server`.** The
proxied ready role cannot enforce `--max-rows`/`BEADS_MAX_ROWS`, so bd fails
loudly rather than silently dropping the limit; `--claim` no longer exempts
the command. Agent rigs that set the cap globally must unset it for proxied
`bd ready --claim`. Direct mode is unchanged, and a claim there still
succeeds against a ready pool larger than the cap. (#6269)
### Fixed
- **Server-mode issue mutations no longer revert concurrent writers' committed
rows** ([#5740](https://github.com/gastownhall/beads/pull/5740)). The
issue-mutation path ran `DOLT_ADD`/`DOLT_COMMIT` inside its still-open SQL
transaction; `DOLT_ADD` stages the whole table as of the transaction's
BEGIN-time root, and the Dolt commit was minted before the commit-time merge
that reconciles concurrent writers — so a mutation could silently write every
row another session had committed during its window back to that session's
BEGIN-time value, seen in production as claims reverting minutes after they
were made. The Dolt commit now runs after the SQL transaction commits, against
the post-merge working set. Its failure is logged rather than returned: the
data is durably committed by then and rides the next Dolt commit, and treating
an applied mutation as failed is what made retries double-apply and claim
verification disown claims the caller held. Reported and fixed by
@nova-submodules.
- **Removed-backend rejection now detects an existing Dolt database** and gives
the exact `metadata.json` heal (change `"backend"` to `"dolt"`, or delete the
field) instead of a destructive export/reinitialize path. Workspaces that bd
v1.2.x opened as Dolt despite a stale `"backend"` value are one edit from
healthy, and the message now says so. Rejection remains fail-closed and
read-only — no storage database is opened or modified — and still exits 1.
(Detection helper ported from @steveyegge's #4740.)
- `bd migrate` no longer aborts with a duplicate-primary-key error when the same
dependency edge exists in two typed target columns; duplicate edges are now
merged deterministically (issue-ref wins over wisp/external), renaming an issue
no longer leaves its dependency rows on a stale primary key, and databases left
half-rekeyed by earlier binaries are detected and repaired on the next
migration pass (#5268).
- **Legacy tracked `ignored_schema_migrations` self-heals** by untracking it at
open time, unwedging `bd dolt pull` on upgraded databases (#4356).
- **A v1.1.x/v1.2.x ignored-migration cursor is believed** when
`leases.granted_node` is merely not yet migrated, so upgrades apply only the
pending ignored tail instead of replaying `ignored/0007` and silently
restamping `wisps.updated_at` (#5981 class, #5366 follow-up).
- **Auto-export heals after `bd delete` instead of permanently refusing** — the
orphan guard proves deletions against the store's own history (embedded and
server modes), stores wedged by earlier binaries recover on first run, and
`export-state.json` is now versioned (#5896).
- **`bd flatten` and `bd compact` now finish with a full Dolt GC** (all
generations), so re-running them on a previously-GC'd store actually reclaims
the orphaned history and retires orphaned `--as-of` addresses; `bd gc` gains
`--full` and hints when a default pass reclaims little (#5907).
- **`--storage-class` is honored on the proxied-server and `--file` routes** and
rejected as a plan-wide `--graph` flag; it was silently ignored there.
- **Delete and cascade `dep_remove` journal rows are attributed to the
requesting actor** (#5985); `bd events export`/`tail` now say when the journal
is disabled.
- **Truthful errors when a server-scoped credential cannot reach the requested
database** (missing / denied / wrong-session attribution); `--init-if-missing`
is honored in proxied-server mode.
- **The aux row re-key survives dolt#11131-class encoding drift** instead of
failing mid-upgrade with a raw `invalid hash length` error. Drifted tables are
skipped with a warning, recorded clone-locally in `aux_row_rekey_drifted`, and
re-keyed on a later pass, rather than leaving the database unopenable. This
already reached 1.1.x and v1.2.2 users — it is the [1.1.2] fix — and lands on
the main line here, so a store upgrading from v1.2.2 is not newly exposed
(#5064, #4380). Diagnosis and fix by @marcodelpin, carried and reworked by
@maphew.
- **Server mode honors `Config.LenientOpen`**, so the dirty-table guard's
documented recovery is executable. The refusal tells you to run
`bd dolt commit` / `bd vc commit`, but against an external Dolt server those
commands could not open the store to do it — the flag reached server mode and
`newServerMode` never read it, while the field's own doc comment said "Ignored
in server mode". That left the #4566 deadlock intact for exactly the case with
no way out: embedded mode can move the database directory aside and
`bd bootstrap`, and there is no equivalent against a shared server. Fixed by
@Toady00 (#5783, #5781).
- **`bd bootstrap` probes git remotes for Dolt data instead of rejecting them
outright** (#6037, #5743, #5663). `bd init` deliberately persists a
git-origin-derived `sync.remote` into the tracked `.beads/config.yaml` — a git
origin is a valid Dolt remote before `refs/dolt/data` exists, because the first
`bd dolt push` creates that ref on the same remote — but bootstrap classified
those URLs as code-repo URLs and returned before the origin auto-detect that
would have hydrated the clone, `git+https://` and `git+ssh://` included.
Committed `sync.remote` values from earlier releases hydrate again. The failure
was silent because one outcome code covered both "a database already exists,
nothing to do" and "I refused and left you with nothing": bootstrap printed
`✓ Database already exists`, exited 0, and created nothing, after which
`bd init --reinit-local` pointed back at `bd bootstrap` in a closed loop.
Bootstrap now exits non-zero whenever it declines to set up a database.
- **`--set-metadata` again stores numbers/booleans/null as typed JSON scalars**
(v1.2.2 behavior); string-forcing lives on `--metadata-json`. Values written by
in-window dev builds are left as strings; re-apply the flag to retype a key.
- **`bd create --json` (and `POST /v0/beads/issues`) again echo sub-second
RFC3339Nano timestamps**; reads (`show`/`list`) remain second-precision as in
v1.2.2.
- **`bd purge` and `bd prune` select candidates by TIER, so a typed wisp is
ephemeral however it was minted**
([#5995](https://github.com/gastownhall/beads/pull/5995)). The two sweeps
split the world between them — `bd purge` clears the ephemeral tier, `bd
prune` the durable one — but membership was decided by the raw `ephemeral`
column, and a wisp minted before that column was set carried a `wisp_type`
with `ephemeral = 0`. Such a bead belonged to neither sweep: `bd purge`
reported "No closed ephemeral beads to purge" while `bd prune` skipped it as
a wisp. One production database had 858 such rows / 7.3 MB that no sweep
could ever reach. The tiers are now complementary predicates over both
columns — ephemeral is `ephemeral = 1 OR wisp_type` is non-empty, durable is
its exact complement — evaluated across both planes rather than taking the
wisps-plane fast path, because legacy typed wisps live in the `issues` table.
A non-empty `wisp_type` now also implies ephemeral at mint, on every create
path, so no new bead can land in the gap. **A previously unreachable typed
wisp is now a `bd purge` candidate**, so the first purge after upgrade may
clear considerably more than usual. NoHistory beads (wisps plane, no
`wisp_type`) stay durable-tier, unchanged.
- **`bd prime` says when it could NOT read the memory plane**
([#5877](https://github.com/gastownhall/beads/issues/5877)). A broken or
unreachable store made prime omit the memory section entirely, so a session
@ -275,9 +973,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`EmbeddedDoltStore` implements neither `DiffStore` nor `StateHasher`, so in
the default embedded mode the incremental path is inert (`incremental
skipped — store does not implement DiffStore`) and every cycle is still a
full export. The deletion-proof guard below is likewise server-mode only, so
[#5896](https://github.com/gastownhall/beads/issues/5896) stays open for
embedded mode, where `bd delete` still wedges auto-export.
full export. The deletion-proof guard below started out server-mode only for
the same reason; [#5896](https://github.com/gastownhall/beads/issues/5896) is
now closed for embedded mode too — see the store-history heal entry below.
**Caveat: the diff anchor only advances on real commits.** In server mode
dolt auto-commit is off, so with no commits being made the anchor stays put
@ -417,8 +1115,85 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`ParentID`. Programmatic callers that set them got unfiltered results back and
no error — a full list where a scoped one was asked for.
- Proxied-server mode (and `bd serve`) ran schema migrations on every store
open without consulting any migration gate, so an upgraded client silently
migrated a shared database — including from read-only commands. The gate now
runs on that path too: read commands warn and continue on the current
schema, writes refuse, and `bd serve` fails to start until the schema is
reconciled. (#5043)
- `bd migrate schema` now works in proxied-server mode instead of refusing.
The migration runs during the provider open under the verb's consent. (#5043)
- Migration-gate and schema-skew failures on the proxied open path and at
`bd serve` startup now print their full actionable block instead of a
single truncated line. (#5043)
- **`bd doctor` now honors strict `--readonly` and the migration freeze**
(#6028). `bd doctor --fix` / `--clean` refuse to mutate under either gate —
the freeze refusal exits 14 and names the marker, and there is no doctor-side
override — and plain `bd doctor` diagnosis no longer rewrites `.local_version`
or auto-applies schema migrations while a freeze is active or `--readonly` is
set; it reports the pending migration instead. Both gates key on the directory
doctor was pointed at, not just the one it was launched in. Diagnosis itself
keeps working under a freeze.
- **An externally-managed Dolt sql-server is no longer mistaken for bd's own**,
which had silently disarmed the shared-store migrate gate (#6118). When the
port arrived from `BEADS_DOLT_SERVER_PORT`, from `bd init --server-port`, or
from a stale port file left by a server that had since died, bd classified a
genuinely shared server as workspace-owned and migrated its schema in place —
no prompt, no warning, exit 0 — after which every older client on that server
was hard-refused with `schema version mismatch`. Ownership is now *proven*
from bd's own live port and PID record rather than inferred from how the
endpoint was named: bd must have bound that port and the process answering it
must still be alive. Everything else is shared, and stays gated.
- **A `dolt.mode: server` workspace declared in `.beads/config.yaml` is no
longer refused as a legacy workspace** (#6119). The upgrade guard resolved the
connection mode from `metadata.json` alone, so a workspace that names its mode
only in config.yaml fell through to the embedded arm and was refused outright
— including workspaces this release had just created and was already
operating on. The guard now resolves the mode through the same
`IsDoltServerMode` precedence chain the rest of bd uses, and a
`.local_version` witness naming bd 1.0 or later vetoes the legacy verdict in
every mode rather than only in server mode.
- **A server-mode workspace with no `metadata.json` no longer opens a phantom
embedded database** and answers `bd list` with a false-empty result and exit 0
(#6120). Config substitution now gates on `IsDoltServerMode`, and
`BEADS_DOLT_SERVER_MODE` is honored on that path instead of being ignored.
### Security
- **Release binaries now build with Go 1.26.7** instead of 1.26.5, closing seven
standard-library advisories reachable from `bd`: quadratic `net/url` path
resolution, `html/template` JavaScript-regexp context tracking, unbounded
post-handshake `crypto/tls` messages, a missing `ReadHeaderTimeout` on
`net/http`'s unencrypted HTTP/2 check, unbounded recursion in `encoding/xml`
and `encoding/asn1`, and the `x/net/idna` Punycode bug in `net/http`'s
vendored copy. `bd serve` and bd's outbound TLS calls exercise these paths.
The `go` directive stays at 1.26.5, so nothing changes for code importing the
beads module.
- **Dependency bumps clearing all 25 open advisories.** `golang.org/x/crypto`
0.55.0, `golang.org/x/mod` 0.40.0, `klauspost/compress` 1.18.7,
`moby/go-archive` 0.3.3, `kin-openapi` 0.144.0 with `oapi-codegen` 2.7.1, and
a refreshed `integrations/beads-mcp/uv.lock`. None of the fixed code was
reachable from the shipped `bd` binary: the go-archive and x/crypto/ssh paths
are testcontainers-only, the compress bug is in the `s2` codec bd does not
link, and the kin-openapi critical is in `openapi3filter` middleware that
enters the graph only through the `tool` directive for spec codegen. No
behavior change; no dolt bump was required.
### Documentation
- **Known limitation: multi-rig prefix routing (`routes.jsonl`) is not
supported with proxied-server rigs** — cross-rig bead gates stay pending,
`bd close` of a bead gate in proxied-server mode requires `bd gate check`
or `--force`, and routed lookups cannot open proxied-server targets
(#5861).
- **The heartbeat/re-home invariant and the two states it can strand are now
documented where the code lives** (wy-sp2l4): a heartbeat proves the holder
is alive, not that the lease moved, so `granted_node` is backfilled but never
@ -461,53 +1236,6 @@ stopgap (`BD_IGNORE_SCHEMA_SKEW=1`).
`go install github.com/steveyegge/beads/cmd/bd@latest` resolves to this
release instead of the accidental one. (The retract block is also
carried on main so future tags keep the retractions.)
- **`sort` on `GET /v0/beads/issues`.** The listing served one order —
`(created_at DESC, id ASC)` — because the cursor is a keyset position in it,
and the spec said so in its own words: "the sort order is welded to the cursor
contract, and a new order needs new surface." The cost of that welding fell on
every client that wanted `bd list`'s ordering, because the only way to get it
was to page the whole result set and re-sort locally. Measured over a
1400-row store at the 200-row page an HTTP client actually uses, that is
**7 requests plus a client-side comparator; `?sort=priority` is 1**
(`TestProxiedServerListSortRetiresTheWalk`, which runs both strategies against
a real `bd serve` and checks both against `bd list --json` row for row). It is
the one cost on this surface that got worse as a project grew.
**Two values, and the set is closed.** `created` is the existing order, now
spellable. `priority` is `(priority ASC, created_at DESC, id ASC)` — `bd
list`'s flagless ordering, which is also what `bd list --sort priority`
produces, so one served order retires the walk for both. The other seven
orders `bd list --sort` takes are not offered: each value here is a cursor
contract needing a key proven total, and `id` is a natural-numeric order no
database expresses, `updated` moves on every write, `closed` is nullable, and
`status`/`title`/`type`/`assignee` are mutable and unindexed.
**Absent `sort` still means `created`, permanently.** It is the compatibility
contract for every client written before the parameter existed; changing it
would alter which rows a truncated page contains, with no error to notice it
by. A server that predates the parameter answers `param: "sort"` with
`reason: "unknown_parameter"`, which is the per-parameter capability probe a
client dispatches on to fall back.
**The cursor now carries its order, and a mismatch is refused.** This is the
half that makes the parameter safe rather than merely useful. The token was
base64 of `{t,i}` with no binding, so a `created`-order cursor replayed under
`sort=priority` would have decoded perfectly and been read as a position in a
different total order — a page that both skips and duplicates rows, served
with a 200, undetectable by the client. Tokens are now `v2` with an explicit
order member and decode refuses any token whose order differs from the
request's, as `invalid_cursor` (documented recovery: restart paging).
Outstanding `v1` tokens stay readable as the `created`-order positions they
are, so **no traversal in flight has to restart**. This does not contradict
"the token carries no filters": filters select the set, the order decides what
the position means.
**`priority` is a mutable key**, which `created_at` is not, and the spec says
so: under `created` only new rows move relative to a walk, while a priority
update moves an existing row too, so it can be seen twice or missed. That is
the already-documented "a cursor pins a position, not a snapshot" caveat
reached by a second route, not a new class of error — unchanged data never
skips or repeats under either order.
## [1.2.1] - 2026-08-11
@ -2798,27 +3526,6 @@ remote-migrate gate from a blunt block into a state-aware one.
### Fixed
- **Explicit external capability dependencies now block embedded ready work
and appear in dependency trees.** References such as
`external:payments:checkout` were stored in `depends_on_external` but then
dropped by local-issue hydration, so their source still appeared in
`bd ready` and `bd dep tree` omitted the edge. Beads now resolves configured
foreign projects read-only at query time, excludes unsatisfied sources
before ready pagination and atomic claim selection, includes the refs in
blocked output, and renders them as synthetic blocked/closed tree leaves.
- **`--label-any` is no longer silently dropped by `bd ready` and
`bd ready --claim`.** The ready-work WHERE builder emitted clauses for
`--label` and `--exclude-label` but none for `--label-any`, so the OR-set
filter was ignored on the ready/claim path (with or without `--parent`) on
every backend — while `bd list`/`bd search` honored it. On an *atomic claim*
this was dangerous rather than merely wrong: a worker fencing itself to its
own lane (`bd ready --claim --label-any lane-a --parent epic-1`) would
happily claim another lane's issue and believe it was fenced. `--label-any`
now emits an OR-set membership clause that AND-combines with `--label`,
`--exclude-label`, and `--parent`, exactly as the flag help promises; an
exhausted lane now claims nothing instead of falling back to unfenced work.
- **A failed v53 migration no longer traps the database, and the v53 repair
now covers `wisp_dependencies` split-column drift.** rc.2 repaired the
`issues` rig columns, but a database coming from schema v49 could still fail

View file

@ -75,8 +75,16 @@ export CGO_ENABLED := 1
# When go.mod requires a newer Go version than the locally installed one,
# GOTOOLCHAIN=auto downloads the right compiler but coverage instrumentation
# may still use the local toolchain's compile tool, causing version mismatch.
# Force the go.mod version to ensure all tools match.
# Force one exact version so all tools match.
#
# The `toolchain` directive wins over the `go` directive when present: `go` is
# the floor we promise importers, `toolchain` is what we actually build with,
# and reading `go` here would silently downgrade every make target below the
# toolchain the release binaries are built with.
GO_VERSION := $(shell sed -n 's/^toolchain go//p' go.mod)
ifeq ($(GO_VERSION),)
GO_VERSION := $(shell sed -n 's/^go //p' go.mod)
endif
ifneq ($(GO_VERSION),)
export GOTOOLCHAIN := go$(GO_VERSION)
endif

View file

@ -6,13 +6,16 @@ This document describes the complete release process for beads, including GitHub
- [Overview](#overview)
- [Prerequisites](#prerequisites)
- [Release Branches](#release-branches)
- [Release Checklist](#release-checklist)
- [1. Prepare Release](#1-prepare-release)
- [2. GitHub Release](#2-github-release)
- [3. Homebrew Update](#3-homebrew-update)
- [4. PyPI Release (MCP Server)](#4-pypi-release-mcp-server)
- [5. npm Package Release](#5-npm-package-release)
- [6. Verify Release](#6-verify-release)
- [5. Plugin Marketplace Update](#5-plugin-marketplace-update)
- [6. npm Package Release](#6-npm-package-release)
- [7. Verify Release](#7-verify-release)
- [Version Numbering](#version-numbering)
- [Prerelease / Release Candidate (RC) Workflow](#prerelease--release-candidate-rc-workflow)
- [Hotfix Releases](#hotfix-releases)
- [Rollback Procedure](#rollback-procedure)
@ -28,17 +31,21 @@ A beads release involves multiple distribution channels:
### The Easy Way (Recommended)
For routine releases, use the fully automated release script:
For routine releases, run the release molecule:
```bash
./scripts/release.sh 0.22.0
bd mol wisp beads-release --var version=1.3.0
```
This handles version bump, tests, git tag, Homebrew update, and local
installation in one shot. See [scripts/README.md](scripts/README.md#releasesh--the-easy-button)
for details. The rest of this document is the manual / step-by-step process,
useful for understanding what `release.sh` does and for handling edge cases
(hotfixes, rollbacks, manual PyPI/npm publishes).
`.beads/formulas/beads-release.formula.toml` defines the whole flow as
resumable steps with a CI gate: preflight → CHANGELOG/`info.go` →
version bump → commit/tag/push → gate on `release.yml` → verify
GitHub/npm/PyPI/Homebrew → local install. `./scripts/release.sh 1.3.0` is a
thin gateway that prints this command; it does not run a release itself.
The rest of this document is the manual / step-by-step process, useful for
understanding what the molecule does and for handling edge cases (hotfixes,
rollbacks, manual PyPI/npm publishes).
## Prerequisites
@ -79,21 +86,105 @@ python3 --version
twine --version
```
## Release Branches
Releases are cut from a `release/x.y.z` branch, not from whatever `main`
happens to be when the tag is pushed. `main` moves continuously and the
release-critical gates do not all run on every PR, so "tag the tip of main"
means tagging a SHA nobody has fully tested. A burned tag is never reused
(the v1.1.1 and v1.2.0 precedents), so that gamble is expensive.
**Cut the branch from a SHA that is green on both `Main` and
`Nightly Full Tests`.** The `Main` workflow runs on push to `main` and the
`Nightly Full Tests` workflow runs on a 2am UTC schedule, so the newest SHA
with both is usually a few hours behind the tip:
```bash
gh run list --workflow Main --branch main --status success --limit 5 \
--json headSha,conclusion,createdAt
gh run list --workflow "Nightly Full Tests" --status success --limit 3 \
--json headSha,conclusion,createdAt
git fetch origin
git branch release/1.3.0 <the-green-sha>
git push origin release/1.3.0
```
**Everything targeted at the release lands on the release branch as a PR.**
Base the PR on `release/x.y.z`, not `main`, and label it
`status/needs-review-auto` so it goes through the automated review workflow
like any other change. That includes the release-prep PR itself (version
bump, CHANGELOG, `cmd/bd/info.go`) and any fix the release validation turns
up. Nothing is pushed straight to the release branch.
**Tag from the release branch**, following [1. Prepare Release](#1-prepare-release)
onward with the branch checked out in place of `main`.
**Forward-port to `main` after the release ships.** Every commit that landed
only on the release branch — the version bump included — needs a PR back to
`main`, or the next release silently drops it and the version files disagree
with the newest tag. Do this as one PR immediately after the tag is pushed,
while the set of release-only commits is still obvious:
```bash
git log --oneline main..release/1.3.0
```
(See PR #5782 for the shape of a forward-port PR.)
## Release Checklist
Before starting a release:
- [ ] `release/x.y.z` branch cut from a SHA green on `Main` **and**
`Nightly Full Tests` (see [Release Branches](#release-branches))
- [ ] All tests passing (`go test ./...`)
- [ ] npm package tests passing (`cd npm-package && npm run test:all`)
- [ ] **Upgrade smoke tests pass** (`make test-upgrade`) — see [Release Stability Gate](engdocs/RELEASE-STABILITY-GATE.md)
- [ ] **Regression tests pass** (`make test-regression`)
- [ ] **Every release target cross-compiles** — see
[Cross-compile before tagging](#cross-compile-before-tagging). PR CI does
not build them ([#5662](https://github.com/gastownhall/beads/issues/5662)),
and a target that fails at tag time burns the tag.
- [ ] **CHANGELOG.md updated with release notes** (see format below)
- [ ] **CHANGELOG rollup checked** — nothing left under `[Unreleased]` that
belongs in this release, and nothing filed under a *previous* release's
section that shipped after that tag. Verify with
`git merge-base --is-ancestor <commit> <previous-tag>` when in doubt.
- [ ] **`cmd/bd/info.go` `versionChanges` entry added** for this version
- [ ] **Breaking changes documented** with migration steps and recovery instructions
- [ ] **Docs cutover planned** — `docs/cli-docs.pin` bump plus
`./scripts/generate-cli-docs.sh`, and the release line on the docs
homepage (`docs/index.md`). See
[Documentation Site (Mintlify)](#documentation-site-mintlify).
- [ ] `./scripts/check-versions.sh` passes
- [ ] No uncommitted changes
- [ ] On `main` branch and up to date with origin
- [ ] On the `release/x.y.z` branch and up to date with origin
### Cross-compile before tagging
`.goreleaser.yml` builds darwin, linux, windows **and freebsd**, but PR CI
builds none of the cross targets — the v1.2.0 tag burned on a freebsd
compilation failure that no pre-tag gate could have caught
([#5661](https://github.com/gastownhall/beads/pull/5661) fixed the break,
[#5662](https://github.com/gastownhall/beads/issues/5662) tracks the CI gap).
Until that gap is closed, build them by hand before tagging:
```bash
goreleaser build --snapshot --clean
```
A failure here costs a rebase. The same failure after the tag is pushed costs
the tag.
## 1. Prepare Release
> The snippets below say `main` where they mean "the branch you are releasing
> from". Since [Release Branches](#release-branches), that is `release/x.y.z`:
> substitute it for `main` in every `git push` and `git checkout` here, and
> land the changes as a PR against the release branch rather than pushing
> directly.
### Update CHANGELOG.md
**IMPORTANT: Do this FIRST before running bump-version script.**
@ -117,58 +208,86 @@ Add release notes to CHANGELOG.md:
- Changed behavior of B (migration guide)
```
Commit the CHANGELOG changes:
Commit the CHANGELOG changes and open a PR against the release branch:
```bash
git add CHANGELOG.md
git commit -m "docs: Add CHANGELOG entry for v0.22.0"
git push origin main
git checkout -b release-prep/v0.22.0
git push -u origin release-prep/v0.22.0
gh pr create --base release/0.22.0 --label status/needs-review-auto
```
### Update Version and Create Release Tag
Use the version bump script to update all version references and create the release tag:
`./scripts/update-versions.sh` rewrites every version reference in one pass.
It does no git operations — commit, tag and push are yours. (`bump-version.sh`
is retired; it now only prints a pointer here.)
```bash
# Dry run - shows what will change
./scripts/bump-version.sh 0.22.0
# Full release with all local installations
./scripts/bump-version.sh 0.22.0 --commit --tag --push --all
./scripts/update-versions.sh 0.22.0
./scripts/check-versions.sh # must pass before you commit
```
**Available flags:**
| Flag | Description |
|------|-------------|
| `--commit` | Create a git commit with version changes |
| `--tag` | Create annotated git tag (requires --commit) |
| `--push` | Push commit and tag to origin (requires --tag) |
| `--install` | Build and install bd to `~/go/bin` AND `~/.local/bin` |
| `--mcp-local` | Install beads-mcp from local source via uv/pip |
| `--upgrade-mcp` | Upgrade beads-mcp from PyPI (after PyPI publish) |
| `--restart-servers` | Restart all Dolt servers to pick up new version |
| `--all` | Shorthand for `--install --mcp-local --restart-servers` |
This updates:
- `cmd/bd/version.go` - CLI version constant
- `integrations/beads-mcp/pyproject.toml` - MCP server version
- `integrations/beads-mcp/src/beads_mcp/__init__.py` - MCP Python version
- `plugins/beads/.claude-plugin/plugin.json` - Claude plugin version
- `plugins/beads/.codex-plugin/plugin.json` - Codex plugin version
- `.claude-plugin/marketplace.json` - Claude marketplace version
- `npm-package/package.json` - npm package version
- `cmd/bd/templates/hooks/*` - Git hook versions
- `README.md` - Documentation version
- `PLUGIN.md` - Version requirements
- `CHANGELOG.md` - Creates release entry from [Unreleased]
- `cmd/bd/version.go` — CLI version constant. This is the canonical version:
`check-versions.sh` reads it and compares the gated files against it. Note
that "gated" is narrower than "updated" — `default.nix` and `README.md` are
bumped here but checked by nothing, so eyeball them.
- `integrations/beads-mcp/pyproject.toml` — MCP server version
- `integrations/beads-mcp/src/beads_mcp/__init__.py` — MCP Python version
- `integrations/beads-mcp/uv.lock` — the `beads-mcp` pin, via `uv lock`. A
stale lock fails the release workflow's MCP gate *after* the tag exists and
can no longer be rewritten; that burned the v1.1.1 tag. Install `uv` before
bumping, or the script warns and skips it.
- `plugins/beads/.claude-plugin/plugin.json` — Claude plugin version
- `plugins/beads/.codex-plugin/plugin.json` — Codex plugin version
- `plugins/beads/.copilot-plugin/plugin.json` — Copilot plugin version
- `.claude-plugin/marketplace.json` — Claude marketplace version
- `npm-package/package.json` — npm package version
- `.githooks/*` — the `BEGIN`/`END BEADS INTEGRATION v<version>` markers on the
tracked managed hook sections, held byte-equal to the `cmd/bd/hooks.go`
generator by `TestTrackedManagedHookSectionsMatchGenerator`. Skipping them
reddens the branch only after the push (that was the v1.2.0 bump).
- `default.nix` — Nix package version
- `cmd/bd/winres/winres.json`, `cmd/bd/winres/manifest.xml` — Windows PE
resource metadata (the numeric fields take the base version, with any
prerelease suffix stripped)
- `README.md` — the static Alpha version badge, if one is present
The `--commit --tag --push` flags will:
1. Create a git commit with all version changes
2. Create an annotated tag `v0.22.0`
3. Push both commit and tag to origin
It does **not** touch `CHANGELOG.md`, `cmd/bd/info.go`, or `default.nix`'s
`vendorHash`. Write the release notes first (above), and run
`./scripts/update-nix-vendorhash.sh` if `go.mod`/`go.sum` changed since the
last tag.
This triggers GitHub Actions to build release artifacts automatically.
Then commit and land it as a PR — **not** a direct push, and **do not tag
yet**:
```bash
git add -A
git commit -m "chore: bump version to 0.22.0"
git checkout -b release-prep/v0.22.0
git push -u origin release-prep/v0.22.0
gh pr create --base release/0.22.0 --label status/needs-review-auto
```
Tag only after that PR merges, and tag the **merged** commit:
```bash
git checkout release/0.22.0 && git pull
git tag -a v0.22.0 -m "Release v0.22.0"
git push origin v0.22.0
```
Two reasons the order matters, both of which have teeth. With branch
protection on, a direct push to the release branch is rejected — after you
have already minted the tag locally. And with a squash merge, a tag created
before the merge points at a commit that is not an ancestor of
`release/x.y.z`; goreleaser packages `CHANGELOG.md` *from the tag* into every
published archive, so the release would ship the pre-review notes.
Pushing the tag triggers GitHub Actions to build release artifacts
automatically.
The tag workflow re-runs release-critical package gates before publishing:
@ -186,12 +305,14 @@ release.
```bash
# 1. Update CHANGELOG.md and cmd/bd/info.go with release notes (manual step)
# 2. Bump version and install everything locally
./scripts/bump-version.sh 0.22.0 --commit --all
# 2. Bump versions and verify
./scripts/update-versions.sh 0.22.0
./scripts/check-versions.sh
# 3. Test locally, then tag and push
# 3. Land it on the release branch as a PR (label status/needs-review-auto)
# 4. Test locally, then tag the merged commit and push
git tag -a v0.22.0 -m "Release v0.22.0"
git push origin main
git push origin v0.22.0
```
@ -199,18 +320,6 @@ The release workflow is intentionally gated to `refs/tags/v*`. A manual
workflow dispatch from a branch will skip publishing jobs; manual reruns must
select the release tag.
**Alternative (step-by-step):**
```bash
# Just commit
./scripts/bump-version.sh 0.22.0 --commit
# Then manually tag and push
git tag -a v0.22.0 -m "Release v0.22.0"
git push origin main
git push origin v0.22.0
```
## 2. GitHub Release
### Using GoReleaser (Recommended)
@ -383,19 +492,33 @@ top-level CLI commands" passes vacuously for it, e.g. wy-gx5rj for `bd
sync`). Bump it as part of THIS release, not a follow-up:
```bash
# After tagging (see "Update Version and Create Release Tag" above):
# After tagging (see "Update Version and Create Release Tag" above).
# generate-cli-docs.sh builds bd from the pinned tag, so the tag must exist.
echo "v0.22.0" > docs/cli-docs.pin
./scripts/generate-cli-docs.sh
git add docs/cli-docs.pin docs/CLI_REFERENCE.md docs/cli-reference docs/docs.json
git commit -m "docs: bump CLI docs pin to v0.22.0"
git push origin main
```
The pin bump lands on `main`, since that is where Mintlify deploys from — fold
it into the forward-port PR described in [Release Branches](#release-branches)
rather than pushing it to the release branch (or to `main`) directly.
Skipping this step doesn't fail fast — Check 4 stays green (it validates
against the *old* pin) until the next bump, at which point every command
added across the skipped releases shows up at once as a pile of "missing
from docs/CLI_REFERENCE.md" failures with no obvious release to blame.
**The docs homepage names the release line, and it is hand-maintained.**
`docs/index.md` carries a line of the form "These docs are for the X.Y.Z
release of beads" with a link to that tag's release notes. Nothing generates
or validates it, so it goes stale silently — it still said 1.1.0 when the pin
had already moved to v1.2.2. Update it in the same commit as the pin:
```bash
rg 'These docs are for the' docs/index.md
```
## 6. npm Package Release
The npm package wraps the native binary for Node.js environments.
@ -416,7 +539,7 @@ npm org ls beads
# Navigate to npm package
cd npm-package
# Version should already be updated by bump-version.sh
# Version should already be updated by update-versions.sh
cat package.json | grep version
# Run all tests
@ -525,14 +648,32 @@ curl -fsSL https://raw.githubusercontent.com/gastownhall/beads/main/scripts/inst
bd version
```
### CLI Docs Pin
### Docs Cutover
```bash
cat docs/cli-docs.pin # should be the tag just released, not an older one
cat docs/cli-docs.pin # the tag just released, not an older one
rg 'These docs are for the' docs/index.md # the version just released
```
If it's stale, do the bump from the "Documentation Site (Mintlify)" step
above before calling the release done.
If either is stale, do the bump from the
[Documentation Site (Mintlify)](#documentation-site-mintlify) step above
before calling the release done.
### Release Notes Rollup
```bash
# [Unreleased] should be empty, and the newest dated section is this release
rg -n '^## \[' CHANGELOG.md | head -3
```
Also confirm nothing shipped after the *previous* tag is still filed under
that tag's section — the v1.2.2 section carried an entry for a commit that was
never in v1.2.2, which is invisible until someone reads the release notes and
looks for a feature their binary does not have:
```bash
git merge-base --is-ancestor <commit> <previous-tag> && echo "in the tag" || echo "MISFILED"
```
## Prerelease / Release Candidate (RC) Workflow
@ -556,32 +697,42 @@ identifier (e.g. `1.1.0-rc.1`); Python tooling normalizes this to PEP 440 form
### Cut an RC
**An RC is cut from the release branch, like everything else.** This is the
point of an RC: it validates the SHA the stable release will ship. Tagging the
tip of `main` instead validates a *different* SHA by construction — the stable
tag comes off `release/x.y.z` — and re-introduces the "tag a SHA nobody has
fully tested" failure that [Release Branches](#release-branches) exists to
prevent. Note the branch is named for the **base** version: an RC for 1.1.0 is
cut on `release/1.1.0`, not `release/1.1.0-rc.1`.
```bash
# 0. Cut (or check out) the release branch — see Release Branches above.
git checkout release/1.1.0
# 1. Update CHANGELOG.md and cmd/bd/info.go with the RC notes (manual step),
# same as a stable release. Date the CHANGELOG section.
# 2. Bump versions. update-versions.sh accepts a prerelease identifier and,
# 2. Bump versions. update-versions.sh accepts a prerelease identifier, and
# also refreshes uv.lock (PEP 440 normalizes 1.1.0-rc.1 to 1.1.0rc1).
./scripts/update-versions.sh 1.1.0-rc.1
# Windows PE numeric fields (winres file_version/product_version and the
# manifest <assemblyIdentity> version) are set to the base version 1.1.0,
# because PE versions must be purely numeric; gen-winres.sh strips the
# prerelease suffix the same way at build time.
# prerelease suffix the same way at build time, and check-versions.sh
# checks those fields against the base version for exactly this reason.
# 3. Keep the MCP lockfile in sync (PEP 440 normalizes to 1.1.0rc1), or the
# Package Gate (MCP) check goes red:
(cd integrations/beads-mcp && uv lock)
# 4. Validate locally.
# 3. Validate locally.
./scripts/check-versions.sh
# 5. Open a PR for the RC prep and have it reviewed. RC prep should land
# through normal review, not auto-merge.
# 4. Open a PR for the RC prep against release/1.1.0 and have it reviewed.
# RC prep should land through normal review, not auto-merge.
```
After the RC prep is merged to `main`, cut the tag from the merge commit:
After the RC prep merges, cut the tag from the merged commit on the release
branch:
```bash
git checkout main && git pull
git checkout release/1.1.0 && git pull
git tag -a v1.1.0-rc.1 -m "Release candidate v1.1.0-rc.1"
git push origin v1.1.0-rc.1
```
@ -595,36 +746,36 @@ above. Tag creation is restricted to release maintainers; see
- Install the RC from the GitHub prerelease assets and exercise the changes it
is gating before promoting.
- To promote to stable, bump to the base version with no suffix
(`./scripts/update-versions.sh 1.1.0`). The stable release **does** publish
to Homebrew/PyPI/npm, so follow the standard
(`./scripts/update-versions.sh 1.1.0`) **on the same release branch**, so the
stable tag lands on the SHA the RC validated. The stable release **does**
publish to Homebrew/PyPI/npm, so follow the standard
[Prepare Release](#1-prepare-release) steps from there.
## Hotfix Releases
For urgent bug fixes:
A hotfix is a release branch cut from the tag instead of from a green `main`
SHA; everything else — PRs against the branch, the forward-port afterwards —
is the same as [Release Branches](#release-branches).
```bash
# Create hotfix branch from tag
git checkout -b hotfix/v0.22.1 v0.22.0
# Create the release branch from the tag being fixed
git checkout -b release/0.22.1 v0.22.0
git push origin release/0.22.1
# Make fixes
# ... edit files ...
# Bump version to 0.22.1
./scripts/bump-version.sh 0.22.1 --commit
# Land the fix as a PR against release/0.22.1, then bump
./scripts/update-versions.sh 0.22.1
./scripts/check-versions.sh
git commit -am "chore: bump version to 0.22.1"
# Tag and release
git tag -a v0.22.1 -m "Hotfix release v0.22.1"
git push origin hotfix/v0.22.1
git push origin release/0.22.1
git push origin v0.22.1
# Follow normal release process
goreleaser release --clean
# Merge back to main
git checkout main
git merge hotfix/v0.22.1
git push origin main
# Forward-port the fix AND the version bump to main
git log --oneline main..release/0.22.1
```
## Rollback Procedure
@ -715,13 +866,10 @@ After a successful release:
1. **Upgrade local beads-mcp installation** to the new version:
```bash
# Option 1: Use the bump-version.sh script (recommended during version bump)
./scripts/bump-version.sh <version> --upgrade-mcp
# Option 2: Manual upgrade via pip (if installed globally)
# If installed globally
pip install --upgrade beads-mcp
# Option 3: Manual upgrade via uv tool (if installed as a tool)
# If installed as a uv tool
uv tool upgrade beads-mcp
# Verify the new version
@ -731,20 +879,17 @@ After a successful release:
# The MCP server will load the newly installed version
```
**Note:** The `--upgrade-mcp` flag can be combined with other flags:
```bash
# Update versions, commit, install bd binary, and upgrade beads-mcp all at once
./scripts/bump-version.sh 0.24.3 --commit --install --upgrade-mcp
```
2. **Forward-port the release branch to `main`** — see
[Release Branches](#release-branches). Include the `docs/cli-docs.pin` bump
and the `docs/index.md` release line.
2. **Verify the upgraded CLI**:
3. **Verify the upgraded CLI**:
```bash
bd version
bd doctor quick
```
3. **Announce** on relevant channels (Twitter, blog, etc.)
4. **Update documentation** if needed
4. **Announce** on relevant channels (Twitter, blog, etc.)
5. **Close milestone** on GitHub if using milestones
6. **Update project board** if using project management
7. **Monitor** for issues in the first 24-48 hours
@ -793,13 +938,38 @@ goreleaser build --snapshot --clean
Beads follows [Semantic Versioning](https://semver.org/):
- **MAJOR** (x.0.0): Breaking changes
- **MINOR** (0.x.0): New features, backwards compatible
- **PATCH** (0.0.x): Bug fixes, backwards compatible
- **MAJOR** (x.0.0): Removals and incompatible interface changes — a command,
flag, config key, schema table or published Go API that is deleted or
changes shape such that a caller written against the old one cannot be
fixed by changing its inputs.
- **MINOR** (0.x.0): New features, plus **documented behavioral changes that
ship with migration notes**. A command that starts refusing input it used to
accept, changes a default, or changes which rows it returns is a minor bump
when the change is written up in CHANGELOG.md with the migration step and
the opt-out. It is not a major bump: the interface is unchanged and the
caller keeps working after a documented adjustment.
- **PATCH** (0.0.x): Bug fixes, no behavioral change beyond the bug.
The MINOR clause is a description of practice, not a loophole. It carries an
obligation: every behavioral break in a minor release must appear in
CHANGELOG.md under that release, name its override or migration step, and be
summarized in `cmd/bd/info.go` so `bd info --whats-new` surfaces it to agents.
A behavioral break that is *not* documented that way is a bug in the release,
not a minor bump.
**1.3.0 is a worked example.** It ships seven documented behavioral breaks —
`bd update --status <done-status>` enforcing close policy, `bd search`
including closed issues, the interactive-only last-touched fallback, `bd human
list` status handling, `bd dolt push` remote-adoption consent, Dolt port
precedence over `BEADS_DOLT_PORT`, and actor `--` decoding — plus a schema
migration and a flag rename (`--profile` → `--cpu-profile`). Every one has a
CHANGELOG entry with its override, and the release is a MINOR bump.
Examples:
- `0.21.5` → `0.22.0`: New features (minor bump)
- `0.22.0` → `0.22.1`: Bug fix (patch bump)
- `1.2.2` → `1.3.0`: New features plus documented behavioral breaks with
migration notes (minor bump)
- `0.22.1` → `1.0.0`: Stable release (major bump)
**Prereleases:** append a SemVer prerelease identifier for release candidates,

View file

@ -5,6 +5,7 @@ import (
"errors"
"reflect"
"testing"
"time"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/types"
@ -539,6 +540,46 @@ func RunBatchCreatorDoesNotMutateTheCallerRequest(t *testing.T, ctx context.Cont
}
}
// RunBatchCreatorEchoesSubSecondTimestamps is the batch half of
// RunLifecycleCreateEchoesSubSecondTimestamps: every entry of a batch result
// carries the timestamps its write sent, at full precision, not the whole
// seconds the DATETIME(0) columns answer a re-read with.
//
// PER ITEM, because both bodies build the result one snapshot at a time and a
// fix applied to the first item only would still pass a check on result[0].
func RunBatchCreatorEchoesSubSecondTimestamps(t *testing.T, ctx context.Context, fixture BatchCreatorFixture) {
t.Helper()
first := time.Date(2019, 3, 4, 5, 6, 7, 123456789, time.UTC)
second := time.Date(2020, 4, 5, 6, 7, 8, 246813579, time.UTC)
request := publicops.CreateBatchRequest{
Actor: "batch-writer",
Items: []publicops.BatchCreateItem{
batchCreatorItem(&types.Issue{Title: "nano first", Priority: 2, IssueType: types.TypeTask, CreatedAt: first, UpdatedAt: first}),
batchCreatorItem(&types.Issue{Title: "nano second", Priority: 2, IssueType: types.TypeTask, CreatedAt: second, UpdatedAt: second}),
},
}
want := []time.Time{first, second}
result, err := fixture.BatchCreator.CreateBatch(ctx, request)
if err != nil {
t.Fatalf("CreateBatch with sub-second timestamps: %v", err)
}
if len(result.Issues) != len(want) {
t.Fatalf("CreateBatch returned %d issues for %d items", len(result.Issues), len(want))
}
for i, issue := range result.Issues {
if issue == nil {
t.Fatalf("CreateBatch result issue %d is nil", i)
}
if !issue.CreatedAt.Equal(want[i]) {
t.Errorf("result issue %d created_at = %v, want %v exactly — a batch echo rounded to the second cannot order same-second creates",
i, issue.CreatedAt.UTC(), want[i])
}
if !issue.UpdatedAt.Equal(want[i]) {
t.Errorf("result issue %d updated_at = %v, want %v exactly", i, issue.UpdatedAt.UTC(), want[i])
}
}
}
// batchCreatorItem is the item every case builds, so a case names only the
// fields it is about.
func batchCreatorItem(issue *types.Issue) publicops.BatchCreateItem {

View file

@ -277,7 +277,16 @@ func RunImporterWiresTheCrossPlaneEdgeBetweenItsRows(t *testing.T, ctx context.C
if result.Created != 3 {
t.Errorf("Created = %d, want 3", result.Created)
}
// BOTH BELTS PER DIRECTION: the pinned cell fails a misrouted edge (the
// expected cell reads 0), and the summed total fails a DUPLICATE landing in
// any of the six table x column cells, which no per-cell count can see. The
// two together also make "the other plane holds none" exact: total 1 with
// the expected cell at 1 leaves every remaining cell empty.
assertImporterPlaneEdgeCount(t, ctx, fixture, "wisp_dependencies", "depends_on_issue_id", wisp, durable, 1)
assertImporterPlaneEdgeCount(t, ctx, fixture, "dependencies", "depends_on_issue_id", wisp, durable, 0)
assertImporterEdgeCount(t, ctx, fixture, wisp, durable, 1)
assertImporterPlaneEdgeCount(t, ctx, fixture, "dependencies", "depends_on_wisp_id", depender, wisp, 1)
assertImporterPlaneEdgeCount(t, ctx, fixture, "wisp_dependencies", "depends_on_wisp_id", depender, wisp, 0)
assertImporterEdgeCount(t, ctx, fixture, depender, wisp, 1)
assertImporterSkipped(t, result, nil)
}
@ -415,7 +424,9 @@ func assertImporterRowCount(t *testing.T, ctx context.Context, fixture ImporterF
// importerEdgeCount counts the stored edges from source to target across BOTH
// dependency tables and all three target columns. A dropped edge is dropped on
// every plane, so the cases that expect zero want zero everywhere, and a
// per-table count could report one while the row sat in the other.
// per-table count could report one while the row sat in the other. WHICH of the
// six cells a row landed in is a placement detail the cases that care about it
// assert through assertImporterPlaneEdgeCount instead.
func importerEdgeCount(t *testing.T, ctx context.Context, fixture ImporterFixture, source, target string) int {
t.Helper()
var got int
@ -438,6 +449,29 @@ func assertImporterEdgeCount(t *testing.T, ctx context.Context, fixture Importer
}
}
// assertImporterPlaneEdgeCount pins WHERE an edge landed: one table and one
// target column, the way RunImporterWiresTheCrossPlaneEdgeBetweenItsRows
// documents its claim. importerEdgeCount deliberately sums both planes, which
// is right for "dropped everywhere" and wrong for "wired in the right place":
// an edge written to the other plane's table, or under the wrong target
// column, counts as one there too. ONE CALL PINS ONE CELL, so the cross-plane
// case asserts the expected cell holds exactly one row and the mirror cell of
// the other plane holds none, and keeps importerEdgeCount's summed total
// alongside them — a duplicate under one of the four cells neither call names
// is invisible to this helper and is exactly what the sum still catches.
func assertImporterPlaneEdgeCount(t *testing.T, ctx context.Context, fixture ImporterFixture, table, column, source, target string, want int) {
t.Helper()
//nolint:gosec // G201: table and column are hardcoded names from this contract's call sites.
query := "SELECT COUNT(*) FROM " + table + " WHERE issue_id = ? AND " + column + " = ?"
var got int
if err := fixture.QueryScalar(ctx, query, []any{source, target}, &got); err != nil {
t.Fatalf("count %s.%s edges %s -> %s: %v", table, column, source, target, err)
}
if got != want {
t.Errorf("%s.%s rows %s -> %s = %d, want %d", table, column, source, target, got, want)
}
}
// assertImporterScalar asserts one stored column of one durable row.
func assertImporterScalar(t *testing.T, ctx context.Context, fixture ImporterFixture, what, id, want string) {
t.Helper()

View file

@ -565,6 +565,79 @@ func assertLifecycleCreateMembers(t *testing.T, id, label string, issue *types.I
}
}
// RunLifecycleCreateEchoesSubSecondTimestamps pins that a create RESULT carries
// the timestamps the write actually sent, at full precision, rather than the
// whole seconds a read of the row answers with.
//
// created_at/updated_at are DATETIME(0) columns, so any snapshot built by
// re-reading the row it just wrote comes back rounded. That is fine for every
// read verb — show and list have always answered whole seconds — but create is
// different: its echo is the ONLY record a caller has of the instant the row was
// written, and agents order same-second creates by it. A backend that hydrates
// its create result and stops there silently loses that ordering, which is
// exactly the regression this case exists for.
//
// THE PRECISION PIN IS THE EXPLICIT ARM, not the auto-stamped one. A create that
// names no time is stamped from the clock, and whether that stamp happens to
// carry a non-zero fraction is not something a test can require without being
// flaky by a billionth; a caller-supplied time with nanoseconds in it is
// deterministic, and PrepareIssueForInsert preserves a non-zero input. The
// auto-stamp arm is kept as a bounded smoke check that the overlay did not
// replace the stamp with something unrelated.
//
// The stored ROW is deliberately NOT asserted here. What it rounds to is the
// column's business, and a later widening to DATETIME(6) must not have to edit
// this case.
func RunLifecycleCreateEchoesSubSecondTimestamps(t *testing.T, ctx context.Context, fixture LifecycleCreateFixture) {
t.Helper()
id := fixture.IssuePrefix + "-lcc-nanoecho"
createdAt := time.Date(2019, 3, 4, 5, 6, 7, 123456789, time.UTC)
updatedAt := time.Date(2019, 3, 4, 5, 6, 8, 987654321, time.UTC)
created, err := fixture.Lifecycle.Create(ctx, publicops.CreateRequest{
Actor: "writer",
ForceIDPrefix: true,
Issue: &types.Issue{
ID: id, Title: "sub-second echo", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask,
CreatedAt: createdAt, UpdatedAt: updatedAt,
},
})
if err != nil {
t.Fatalf("create %s with sub-second timestamps: %v", id, err)
}
if created.Issue == nil {
t.Fatalf("create %s returned no issue", id)
}
if !created.Issue.CreatedAt.Equal(createdAt) {
t.Errorf("%s create result created_at = %v, want %v exactly — the echo is the caller's only record of when the row was written, and rounding it to the second makes same-second creates unorderable",
id, created.Issue.CreatedAt.UTC(), createdAt)
}
if !created.Issue.UpdatedAt.Equal(updatedAt) {
t.Errorf("%s create result updated_at = %v, want %v exactly", id, created.Issue.UpdatedAt.UTC(), updatedAt)
}
// The auto-stamped arm: no precision requirement, only that the overlay
// still hands back a stamp from this call rather than a stale or zero one.
stamped := fixture.IssuePrefix + "-lcc-nanoauto"
lower := time.Now().UTC().Add(-lifecycleCreateClockSlack)
auto, err := fixture.Lifecycle.Create(ctx, publicops.CreateRequest{
Actor: "writer",
ForceIDPrefix: true,
Issue: lifecycleCreateIssue(stamped),
})
if err != nil {
t.Fatalf("create %s without timestamps: %v", stamped, err)
}
upper := time.Now().UTC().Add(lifecycleCreateClockSlack)
if auto.Issue == nil {
t.Fatalf("create %s returned no issue", stamped)
}
echoed := auto.Issue.CreatedAt.UTC()
if echoed.Before(lower) || echoed.After(upper) {
t.Errorf("%s create result created_at = %v, want a stamp from this call between %v and %v", stamped, echoed, lower, upper)
}
}
// lifecycleCreateClockSlack widens the window the auto-stamp arm allows around a
// creation time the implementation wrote from its own clock. Every backend here
// stamps in Go, in this process, so the two clocks are the same one; the slack

View file

@ -94,6 +94,7 @@ var roleContractCases = []roleContract{
RunBatchCreatorRecordsOneHistoryEntry,
RunBatchCreatorRecordsNoHistoryForAnEphemeralBatch,
RunBatchCreatorDoesNotMutateTheCallerRequest,
RunBatchCreatorEchoesSubSecondTimestamps,
),
roleCases("BlockingAnnotator", "BlockingAnnotator()", oncePerRole,
@ -412,6 +413,7 @@ var roleContractCases = []roleContract{
RunLifecycleCreateRefusesAForeignIDPrefix,
RunLifecycleCreateInheritsParentLabels,
RunLifecycleCreateWritesEveryScalarField,
RunLifecycleCreateEchoesSubSecondTimestamps,
),
roleCases("LifecycleUpdate", "IssueLifecycle()", oncePerRole,

View file

@ -26,6 +26,7 @@ import (
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/dolt"
"github.com/steveyegge/beads/internal/storage/domain"
"github.com/steveyegge/beads/internal/storage/schema"
"github.com/steveyegge/beads/internal/types"
"github.com/steveyegge/beads/internal/workspacegate"
)
@ -36,12 +37,17 @@ import (
// safe to repeat.
type Storage = beads.Storage
func configuredBackendUnavailable(backend string) error {
// configuredBackendUnavailable is the public open path's fail-closed refusal for
// metadata naming a removed or unrecognized backend. beadsDir and cfg let the
// refusal detect an already-present Dolt database and name the exact
// metadata.json edit that heals the workspace, instead of the export-and-
// reinitialize path that would destroy it.
func configuredBackendUnavailable(backend, beadsDir string, cfg *configfile.Config) error {
switch backend {
case configfile.BackendPostgres, configfile.BackendMySQL, configfile.BackendSQLite:
return configfile.RemovedBackendError(backend)
return configfile.RemovedBackendErrorAt(backend, beadsDir, cfg)
default:
return configfile.UnknownBackendError(backend)
return configfile.UnknownBackendErrorAt(backend, beadsDir, cfg)
}
}
@ -270,6 +276,39 @@ type (
StatusEntry = storage.StatusEntry
)
// AllowSharedSchemaMigration authorizes this process to apply pending schema
// migrations to a database that is SHARED with other bd clients — a Dolt
// sql-server, where migrating promotes the schema version for every connected
// client at once and clients still running an older bd will refuse the
// database until they are upgraded (gastownhall/beads#5920).
//
// Without it, an embedder that upgrades its beads dependency across a schema
// bump gets a migration-gate error from every writable Open* call, whose
// guidance names CLI commands (`bd migrate schema`) that mean nothing inside a
// library process. This is the programmatic equivalent of that command.
//
// It is process-local and set-or-clear, which is the point: the alternative —
// os.Setenv("BD_ALLOW_REMOTE_MIGRATE", "1") — is process-GLOBAL and inherited
// by every child process the embedder spawns, including git hooks and dolt
// subprocesses.
//
// The parity with that env var is only in the process-local mechanism, not the
// reach: BD_ALLOW_REMOTE_MIGRATE=1 unlocks BOTH the no-remote and the
// remote-backed shared arms, while this authorizes ONLY the no-remote arm. A
// remote-backed shared store still needs --force / AllowRemoteMigrateEnv,
// because #4259 cross-clone coordination is a stronger, different contract.
//
// Call it before the Open* call that should perform the migration, and clear
// it afterwards. Only grant it once the operator has confirmed that every
// other client of the server is upgraded; it is a coordination decision the
// library cannot make, because other clients' versions are not observable from
// this process.
//
// Embedded (single-writer) databases never need it: they still auto-migrate.
func AllowSharedSchemaMigration(allow bool) {
schema.SetSharedMigrateConsent(allow)
}
// Open opens a Dolt-backed beads database at the given path.
// This always opens in embedded mode. Use OpenFromConfig to respect
// server mode settings from metadata.json.

View file

@ -31,7 +31,7 @@ func OpenBestAvailable(ctx context.Context, beadsDir string) (Storage, error) {
cfg = configfile.DefaultConfig()
}
if !configfile.IsSupportedBackend(cfg.Backend) {
return nil, configuredBackendUnavailable(cfg.Backend)
return nil, configuredBackendUnavailable(cfg.Backend, beadsDir, cfg)
}
// Dispatch to a registered extension backend before any Dolt path, mirroring

View file

@ -25,7 +25,7 @@ func OpenBestAvailable(ctx context.Context, beadsDir string) (Storage, error) {
cfg = configfile.DefaultConfig()
}
if !configfile.IsSupportedBackend(cfg.Backend) {
return nil, configuredBackendUnavailable(cfg.Backend)
return nil, configuredBackendUnavailable(cfg.Backend, beadsDir, cfg)
}
// Dispatch to a registered extension backend before any Dolt path, mirroring

View file

@ -26,6 +26,12 @@
//
// The package imports stdlib and nothing else, and it must stay that way: it
// sits beneath every leaf, so anything it imports is imported by all of them.
//
// # Stability
//
// EXPERIMENTAL. This package is not yet covered by the project's compatibility
// promise and may change in a minor release. Every leaf re-exports from here,
// so a change to this vocabulary reaches all of them at once.
package beadserrors
import (

View file

@ -43,7 +43,7 @@ func TestRegisteredBackendDispatchesReadWriteAndReadOnly(t *testing.T) {
registerContractBackend(t, name)
beadsDir := writeContractBackendConfig(t, name)
if err := validateConfiguredBackend(&configfile.Config{Backend: name}); err != nil {
if err := validateConfiguredBackend(&configfile.Config{Backend: name}, beadsDir); err != nil {
t.Fatalf("validateConfiguredBackend() rejected registered backend: %v", err)
}
if _, err := newDoltStoreFromConfig(t.Context(), beadsDir); !errors.Is(err, errRegistryReadWrite) {
@ -78,7 +78,7 @@ func TestOSSRegistersNoRemovedBackends(t *testing.T) {
if backends.Registered(name) {
t.Errorf("OSS unexpectedly registered removed backend %q", name)
}
if err := validateConfiguredBackend(&configfile.Config{Backend: name}); err == nil {
if err := validateConfiguredBackend(&configfile.Config{Backend: name}, t.TempDir()); err == nil {
t.Errorf("OSS unexpectedly accepted removed backend %q", name)
}
}

View file

@ -7,7 +7,12 @@ import (
"github.com/steveyegge/beads/internal/storage/backends"
)
func validateConfiguredBackend(cfg *configfile.Config) error {
// validateConfiguredBackend fails closed on metadata that selects a removed or
// unrecognized backend. beadsDir is the workspace whose metadata cfg came from;
// the rejection inspects it (read-only) so a workspace that already holds a Dolt
// database is told to fix the stale "backend" value rather than to export and
// reinitialize. Pass "" only where there is genuinely no workspace directory.
func validateConfiguredBackend(cfg *configfile.Config, beadsDir string) error {
if cfg == nil {
return nil
}
@ -16,11 +21,11 @@ func validateConfiguredBackend(cfg *configfile.Config) error {
}
switch cfg.Backend {
case configfile.BackendPostgres, configfile.BackendMySQL, configfile.BackendSQLite:
return configfile.RemovedBackendError(cfg.Backend)
return configfile.RemovedBackendErrorAt(cfg.Backend, beadsDir, cfg)
case "", configfile.BackendDolt:
return nil
default:
return configfile.UnknownBackendError(cfg.Backend)
return configfile.UnknownBackendErrorAt(cfg.Backend, beadsDir, cfg)
}
}
@ -33,8 +38,8 @@ func registeredBackendWorkspaceIsBeadsDir(cfg *configfile.Config) bool {
return backends.WorkspaceIsBeadsDir(cfg.GetBackend())
}
func requireDoltBackend(cfg *configfile.Config) error {
if err := validateConfiguredBackend(cfg); err != nil {
func requireDoltBackend(cfg *configfile.Config, beadsDir string) error {
if err := validateConfiguredBackend(cfg, beadsDir); err != nil {
return err
}
if cfg != nil && cfg.GetBackend() != configfile.BackendDolt {
@ -64,7 +69,7 @@ func loadDoltBackendConfig(beadsDir string) (*configfile.Config, error) {
if cfg == nil {
cfg = configfile.DefaultConfig()
}
if err := requireDoltBackend(cfg); err != nil {
if err := requireDoltBackend(cfg, beadsDir); err != nil {
return nil, err
}
return cfg, nil

View file

@ -56,7 +56,9 @@ func newBackupStatusCommand(sizeDatabase backupSizeFunc) *cobra.Command {
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
if usesProxiedServer() {
return HandleErrorRespectJSON("backup status is not supported in proxied-server mode")
if err := requireLocalProxiedBackup("backup status"); err != nil {
return err
}
}
evt := metrics.NewCommandEvent("backup-status")
defer func() {

View file

@ -46,10 +46,13 @@ DoltHub (recommended for cloud backup):
After adding, run 'bd backup sync' to push your data.`,
Args: cobra.ExactArgs(1),
// As on `backup status`: the failures here are configuration and I/O
// errors, not usage errors, and cobra's default prints the message, then
// "Error: exit code 1", then the whole usage block — three things where one
// was wanted, and on --json it lands after the JSON object.
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
if usesProxiedServer() {
return HandleErrorRespectJSON("backup init is not supported in proxied-server mode")
}
evt := metrics.NewCommandEvent("backup-init")
defer func() {
if c := metrics.Global(); c != nil {
@ -60,6 +63,10 @@ After adding, run 'bd backup sync' to push your data.`,
ctx := rootCtx
rawPath := args[0]
if usesProxiedServer() {
return runBackupInitProxied(ctx, rawPath)
}
if store == nil {
return fmt.Errorf("no store available")
}
@ -124,10 +131,9 @@ backup location configured with 'bd backup init'.
The backup is atomic — if the sync fails, the previous backup state is preserved.
Run 'bd backup init <path>' first to configure a destination.`,
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
if usesProxiedServer() {
return HandleErrorRespectJSON("backup sync is not supported in proxied-server mode")
}
evt := metrics.NewCommandEvent("backup-sync")
defer func() {
if c := metrics.Global(); c != nil {
@ -136,6 +142,9 @@ Run 'bd backup init <path>' first to configure a destination.`,
}()
ctx := rootCtx
if usesProxiedServer() {
return runBackupSyncProxied(ctx)
}
if store == nil {
return fmt.Errorf("no store available")
}
@ -358,7 +367,14 @@ func showDoltBackupStatusJSON() map[string]interface{} {
// doltBackupSize returns the active database size when the current store
// instance can measure its storage locally. Unsupported backends preserve the
// optional status-field contract instead of failing the command.
//
// Proxied mode has no store to ask — the provider owns the connection — so the
// fork lives here, where "which backend measures this database" is already the
// question being answered.
func doltBackupSize(ctx context.Context) (int64, bool, error) {
if usesProxiedServer() {
return doltBackupSizeProxied(ctx)
}
if store == nil {
return 0, false, fmt.Errorf("no storage backend is open")
}
@ -405,11 +421,10 @@ var backupRemoveCmd = &cobra.Command{
This unregisters the backup remote from Dolt and removes the local
backup configuration. The backup data at the destination is not deleted.`,
Aliases: []string{"rm"},
Aliases: []string{"rm"},
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
if usesProxiedServer() {
return HandleErrorRespectJSON("backup remove is not supported in proxied-server mode")
}
evt := metrics.NewCommandEvent("backup-remove")
defer func() {
if c := metrics.Global(); c != nil {
@ -418,6 +433,9 @@ backup configuration. The backup data at the destination is not deleted.`,
}()
ctx := rootCtx
if usesProxiedServer() {
return runBackupRemoveProxied(ctx)
}
if store == nil {
return fmt.Errorf("no store available")
}

View file

@ -0,0 +1,107 @@
//go:build cgo
package main
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/steveyegge/beads/internal/storage/dbproxy/proxy"
"github.com/steveyegge/beads/internal/storage/dbproxy/server"
)
// TestProxiedServerBackupRefusedOnExternalTopology is the other half of slice
// S3: the backup family is honored on managed-local and refused BY DESIGN
// against a Dolt server bd does not own.
//
// The refusal matters more than the capability. `CALL DOLT_BACKUP('add', …)`
// registers the remote on the server, where it is global to every client of
// that server, so one workspace's backup decision becomes everyone's. With a
// file:/// destination there is a second problem on top: the server resolves
// that path on its own filesystem, so honoring it here would either fail at the
// point of use or write the backup to a directory on the server's host that the
// operator asking for it will never look in. The second outcome is worse than
// any refusal.
//
// Named TestProxiedServer* so .github/scripts/proxied-test-shard.sh runs it in
// the external lane, which is the only one with an external Dolt server.
func TestProxiedServerBackupRefusedOnExternalTopology(t *testing.T) {
requireProxiedServerEnv(t)
bd := buildEmbeddedBD(t)
p := newSharedProxiedProject(t, bd, "bkext")
dest := filepath.Join(t.TempDir(), "dolt-backup")
// Refusals are decided before the provider is constructed, so take the
// topology down first: a pidfile appearing during one of these commands
// would mean the refusal came too late to have cost nothing.
if err := proxy.Shutdown(p.proxyRoot); err != nil {
t.Logf("proxy.Shutdown(%s) before the refusal probes: %v", p.proxyRoot, err)
}
for _, tc := range []struct {
name string
args []string
}{
{"status", []string{"backup", "status"}},
{"init", []string{"backup", "init", dest}},
{"sync", []string{"backup", "sync"}},
{"remove", []string{"backup", "remove"}},
{"restore", []string{"backup", "restore", dest}},
} {
t.Run(tc.name, func(t *testing.T) {
args := append([]string{"--json"}, tc.args...)
out, err := bdProxiedRun(t, bd, p.dir, args...)
if err == nil {
t.Fatalf("bd %s succeeded on an external proxied topology:\n%s",
strings.Join(tc.args, " "), out)
}
var refusal struct {
Code string `json:"code"`
Error string `json:"error"`
Mutates bool `json:"mutates"`
Reason string `json:"reason"`
}
start := strings.Index(string(out), "{")
if start < 0 {
t.Fatalf("no typed refusal JSON:\n%s", out)
}
if err := json.Unmarshal(out[start:], &refusal); err != nil {
t.Fatalf("parse refusal JSON: %v\n%s", err, out)
}
if refusal.Code != "proxy.backup.unsupported" {
t.Errorf("code = %q, want proxy.backup.unsupported", refusal.Code)
}
if refusal.Mutates {
t.Errorf("refusal reported mutates=true")
}
// design, not unimplemented: no later slice is going to lift this.
if refusal.Reason != string(ProxyReasonDesign) {
t.Errorf("reason = %q, want design", refusal.Reason)
}
// The reason has to be the one that is true of every destination
// scheme. "resolved on the server's filesystem" is a file://
// property, and `bd backup init` also takes https/aws/gs.
if !strings.Contains(refusal.Error, "the backup remote is registered on the server, where it is global to every client") {
t.Errorf("refusal message does not say why: %q", refusal.Error)
}
// Nothing was started and nothing was written: the refusal is free.
for _, name := range []string{proxy.PIDFileName, server.PIDFileName} {
if _, err := os.Stat(filepath.Join(p.proxyRoot, name)); err == nil {
t.Errorf("a refused backup command started the topology (%s)", name)
}
}
if _, err := os.Stat(filepath.Join(p.beadsDir, "dolt-backup.json")); err == nil {
t.Errorf("a refused backup command wrote the local backup config")
}
if _, err := os.Stat(dest); err == nil {
t.Errorf("a refused backup command created %s", dest)
}
})
}
}

View file

@ -0,0 +1,208 @@
//go:build cgo && unix
package main
import (
"context"
"encoding/json"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"time"
"github.com/steveyegge/beads/internal/storage/dbproxy/proxy"
"github.com/steveyegge/beads/internal/storage/dbproxy/server"
"github.com/steveyegge/beads/internal/storage/versioncontrolops"
)
// TestManagedLocalProxiedBackupRoundTrip is the claim slice S3 makes: a
// managed-local proxied workspace can take a Dolt-native backup and get its
// data back from it.
//
// It is deliberately a ROUND TRIP rather than a pair of exit-code assertions.
// A backup command that reports success without producing a restorable
// artifact is the failure that matters here, and the only way to rule it out is
// to restore the artifact and look at the issues that come back: the issue
// created BEFORE the backup must survive, and the one created AFTER it must be
// gone. Both directions are load-bearing — a restore that changed nothing would
// pass the first check alone.
//
// Named TestManagedLocalProxied* so the proxied-local smoke lane
// (.github/workflows/proxied-local-smoke.yml) runs it; that lane is the only
// one that sets BEADS_TEST_PROXIED_LOCAL.
func TestManagedLocalProxiedBackupRoundTrip(t *testing.T) {
requireManagedLocalProxiedEnv(t)
bd := buildEmbeddedBD(t)
p := bdManagedLocalInit(t, bd, "bkrt", 5*time.Minute)
before := bdProxiedCreate(t, bd, p.dir, "survives the restore", "-p", "1")
// Status first: it is the read-only probe that proves the route exists at
// all, and it must answer with a real measured size rather than the
// "optional field omitted" the external topologies produce.
status := readProxiedBackupStatus(t, bd, p)
if status.Dolt.Configured {
t.Fatalf("fresh workspace reports a configured backup: %+v", status.Dolt)
}
if status.DatabaseSize.Bytes <= 0 {
t.Fatalf("backup status measured no database: %+v", status.DatabaseSize)
}
dest := filepath.Join(t.TempDir(), "dolt-backup")
if out, err := bdProxiedRun(t, bd, p.dir, "backup", "init", dest); err != nil {
t.Fatalf("bd backup init %s: %v\n%s", dest, err, out)
}
if out, err := bdProxiedRun(t, bd, p.dir, "backup", "sync"); err != nil {
t.Fatalf("bd backup sync: %v\n%s", err, out)
}
// A Dolt backup is a chunk store with a manifest; an empty directory here
// would mean the command reported a success it did not perform.
if _, err := os.Stat(filepath.Join(dest, "manifest")); err != nil {
t.Fatalf("backup sync produced no manifest at %s: %v", dest, err)
}
after := bdProxiedCreate(t, bd, p.dir, "must not survive the restore", "-p", "1")
out, err := bdProxiedRun(t, bd, p.dir, "backup", "restore", dest, "--force", "--json")
if err != nil {
t.Fatalf("bd backup restore %s --force: %v\n%s", dest, err, out)
}
var result struct {
Restored bool `json:"restored"`
Source string `json:"source"`
}
if err := json.Unmarshal(out, &result); err != nil {
t.Fatalf("parse restore JSON: %v\n%s", err, out)
}
if !result.Restored || result.Source != dest {
t.Fatalf("unexpected restore result: %+v; want restored=true, source=%s", result, dest)
}
// The restore quiesces the topology and leaves it down, so the next command
// relaunches against the restored data. Assert that rather than assume it:
// a restore that ran with the old server still up would be the undefined
// case this design exists to avoid.
for _, name := range []string{proxy.PIDFileName, server.PIDFileName} {
if _, err := os.Stat(filepath.Join(p.proxyRoot, name)); err == nil {
t.Fatalf("restore left %s in place; the topology should be quiescent afterwards", name)
}
}
issues := bdProxiedListJSON(t, bd, p)
var ids []string
for _, issue := range issues {
ids = append(ids, issue.ID)
}
if !slices.Contains(ids, before.ID) {
t.Fatalf("restore lost the issue the backup contained (%s); got %v", before.ID, ids)
}
if slices.Contains(ids, after.ID) {
t.Fatalf("restore did not replace the database: %s was created after the backup and is still here; got %v",
after.ID, ids)
}
// The restore registers its source as the backup destination, so a plain
// `bd backup sync` works straight afterwards with no second init.
restored := readProxiedBackupStatus(t, bd, p)
if !restored.Dolt.Configured || !strings.HasSuffix(restored.Dolt.BackupURL, dest) {
t.Fatalf("restore did not register %s as the backup destination: %+v", dest, restored.Dolt)
}
if out, err := bdProxiedRun(t, bd, p.dir, "backup", "sync"); err != nil {
t.Fatalf("bd backup sync after restore: %v\n%s", err, out)
}
if out, err := bdProxiedRun(t, bd, p.dir, "backup", "remove"); err != nil {
t.Fatalf("bd backup remove: %v\n%s", err, out)
}
removed := readProxiedBackupStatus(t, bd, p)
if removed.Dolt.Configured {
t.Fatalf("backup remove left the destination configured: %+v", removed.Dolt)
}
// Removing the destination unregisters the remote; it does not delete the
// backup, which is what makes `bd backup remove` safe to run.
if _, err := os.Stat(filepath.Join(dest, "manifest")); err != nil {
t.Fatalf("backup remove deleted the backup data at %s: %v", dest, err)
}
if out, err := bdProxiedRun(t, bd, p.dir, "backup", "sync"); err == nil {
t.Fatalf("bd backup sync succeeded with no destination configured:\n%s", out)
}
}
// A backup with a malformed migration table can replace the database but fail
// the following provider construction at schema initialization. That failure must
// still leave both processes down, even though no provider was returned.
func TestManagedLocalProxiedBackupRestoreReopenFailure(t *testing.T) {
requireManagedLocalProxiedEnv(t)
bd := buildEmbeddedBD(t)
p := bdManagedLocalInit(t, bd, "bkrf", 5*time.Minute)
dest := filepath.Join(t.TempDir(), "malformed-backup")
if out, err := bdProxiedRun(t, bd, p.dir, "backup", "init", dest); err != nil {
t.Fatalf("backup init: %v\n%s", err, out)
}
// Damage the migration table in the backup, then repair the live table so
// the command's initial provider can open normally. Only its post-restore
// constructor will see the malformed table and fail after starting Dolt.
held := openHeldManagedProxiedConn(t, bd, p)
ctx := context.Background()
if _, err := held.Conn.ExecContext(ctx, "ALTER TABLE schema_migrations RENAME COLUMN version TO broken_version"); err != nil {
t.Fatalf("damage migration table: %v", err)
}
if _, err := held.Conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', 'Simulate a malformed migration table in the backup')"); err != nil {
t.Fatalf("commit malformed migration table: %v", err)
}
if err := versioncontrolops.BackupSync(ctx, held.Conn, proxiedBackupTargetName); err != nil {
t.Fatalf("back up malformed schema: %v", err)
}
if _, err := held.Conn.ExecContext(ctx, "ALTER TABLE schema_migrations RENAME COLUMN broken_version TO version"); err != nil {
t.Fatalf("repair live migration table: %v", err)
}
held.Release()
out, err := bdProxiedRun(t, bd, p.dir, "backup", "restore", dest, "--force")
if err == nil {
t.Fatalf("restore unexpectedly reopened a malformed schema:\n%s", out)
}
if !strings.Contains(string(out), "the restored database did not reopen") || !strings.Contains(string(out), "schema") {
t.Fatalf("expected post-restore schema initialization failure, got:\n%s", out)
}
if strings.Contains(string(out), "restored and reconciled") {
t.Fatalf("failed reopen reported successful reconciliation:\n%s", out)
}
for _, name := range []string{proxy.PIDFileName, server.PIDFileName} {
if _, err := os.Stat(filepath.Join(p.proxyRoot, name)); !os.IsNotExist(err) {
t.Errorf("failed reopen should remove %s; stat error: %v", name, err)
}
}
}
type proxiedBackupStatus struct {
Dolt struct {
Configured bool `json:"configured"`
BackupURL string `json:"backup_url"`
BackupName string `json:"backup_name"`
} `json:"dolt"`
DatabaseSize struct {
Bytes int64 `json:"bytes"`
} `json:"database_size"`
}
func readProxiedBackupStatus(t *testing.T, bd string, p proxiedProject) proxiedBackupStatus {
t.Helper()
out, err := bdProxiedRun(t, bd, p.dir, "backup", "status", "--json")
if err != nil {
t.Fatalf("bd backup status --json: %v\n%s", err, out)
}
start := strings.Index(string(out), "{")
if start < 0 {
t.Fatalf("bd backup status --json produced no JSON:\n%s", out)
}
var status proxiedBackupStatus
if err := json.Unmarshal(out[start:], &status); err != nil {
t.Fatalf("parse backup status JSON: %v\n%s", err, out)
}
return status
}

View file

@ -0,0 +1,285 @@
package main
import (
"encoding/json"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"github.com/spf13/cobra"
"github.com/steveyegge/beads/internal/configfile"
)
// TestProxiedRestoreFailureMessage pins the one distinction the operator acts
// on: whether the data came back. A teardown failure happens after the step
// succeeded, so reporting it as "restore failed" is not a wording nit — it
// tells someone whose database IS restored to restore it again.
func TestProxiedRestoreFailureMessage(t *testing.T) {
boom := errors.New("proxy shutdown: connection refused")
for _, tc := range []struct {
name string
afterReconcile bool
err error
wantContains []string
wantMissing []string
}{
{
name: "restore itself failed",
err: errors.New("DOLT_BACKUP: no such backup"),
// No claim that anything was restored.
wantContains: []string{"restore failed", "no such backup"},
wantMissing: []string{"the data is restored"},
},
{
name: "restore succeeded, teardown failed",
err: &proxiedTeardownError{err: boom},
wantContains: []string{"the data is restored", "bd dolt stop --force", "bd backup init"},
wantMissing: []string{"restore failed"},
},
{
name: "reconcile failed for real",
afterReconcile: true,
err: errors.New("open provider: schema is behind"),
wantContains: []string{"the restored database did not reopen", "schema is behind"},
wantMissing: []string{"restore failed"},
},
{
name: "reconcile succeeded, teardown failed",
afterReconcile: true,
err: &proxiedTeardownError{err: boom},
wantContains: []string{"the data is restored and reconciled", "bd dolt stop --force"},
// Nothing left to re-register: the reconcile already ran.
wantMissing: []string{"restore failed", "bd backup init"},
},
} {
t.Run(tc.name, func(t *testing.T) {
got := proxiedRestoreFailureMessage("/tmp/backup", tc.afterReconcile, tc.err)
for _, want := range tc.wantContains {
if !strings.Contains(got, want) {
t.Errorf("message %q does not contain %q", got, want)
}
}
for _, unwanted := range tc.wantMissing {
if strings.Contains(got, unwanted) {
t.Errorf("message %q must not contain %q", got, unwanted)
}
}
})
}
}
// backupFamilyPaths is every registry path the backup family owns. Keeping the
// list in one place is what makes "the whole family moves together" an
// assertion rather than a convention.
var backupFamilyPaths = []string{
"backup", "backup init", "backup sync", "backup remove", "backup status", "backup restore",
}
// TestProxiedBackupHonoredOnlyOnManagedLocal is the locality policy.
//
// A Dolt backup destination is resolved by the SERVER: CALL DOLT_BACKUP('add',
// name, 'file:///…') names a path on the machine running dolt, not on the
// machine running bd. On managed-local that is the same machine and the same
// user, because bd spawned the child itself. On every other proxied shape it is
// somebody else's filesystem, so honoring the command would either fail at the
// point of use or — much worse — succeed and write the backup somewhere the
// operator cannot see. That is why these stay refused with Reason=design.
func TestProxiedBackupHonoredOnlyOnManagedLocal(t *testing.T) {
for _, path := range backupFamilyPaths {
row, ok := LookupCapabilityRow(path, "")
if !ok {
t.Fatalf("%q has no registry row", path)
}
if got := row.ruleFor(ProxyTopologyManagedLocal); got.Outcome != ProxyOutcomeHonored {
t.Errorf("%q on managed-local: outcome %q, want honored", path, got.Outcome)
}
for _, topology := range []ProxyTopology{
ProxyTopologyExternalTCP, ProxyTopologyExternalUnix,
ProxyTopologyTeamServer, ProxyTopologyUnknown,
} {
got := row.ruleFor(topology)
if got.Outcome != ProxyOutcomeRefused {
t.Errorf("%q on %s: outcome %q, want refused", path, topology, got.Outcome)
}
if got.Reason != ProxyReasonDesign {
t.Errorf("%q on %s: reason %q, want design", path, topology, got.Reason)
}
if got.Code != "proxy.backup.unsupported" {
t.Errorf("%q on %s: code %q, want the frozen proxy.backup.unsupported", path, topology, got.Code)
}
if !strings.Contains(got.Message, "is not supported in proxied-server mode") {
t.Errorf("%q on %s: message %q dropped the frozen prefix", path, topology, got.Message)
}
}
}
}
// TestProxiedBackupGateRefusesByTopology exercises the gate itself rather than
// the table, so a row that is right while the validator ignores the topology
// still fails.
func TestProxiedBackupGateRefusesByTopology(t *testing.T) {
newBackupCmd := func(sub string) *cobra.Command {
root := &cobra.Command{Use: "bd"}
parent := &cobra.Command{Use: "backup"}
child := &cobra.Command{Use: sub}
parent.AddCommand(child)
root.AddCommand(parent)
return child
}
for _, sub := range []string{"init", "sync", "remove", "status", "restore"} {
t.Run(sub, func(t *testing.T) {
if err := validateProxyRegistryBeforeProvider(newBackupCmd(sub), ProxyTopologyManagedLocal); err != nil {
t.Fatalf("managed-local: gate refused %q: %v", sub, err)
}
err := validateProxyRegistryBeforeProvider(newBackupCmd(sub), ProxyTopologyExternalTCP)
if err == nil {
t.Fatalf("external-tcp: gate permitted %q", sub)
}
})
}
}
// TestProxiedBackupRefusalJSONCarriesDesignReason pins what a consumer sees.
// The three frozen fields are unchanged and reason now says "design", which is
// the difference that matters: this refusal is not waiting on a later slice.
func TestProxiedBackupRefusalJSONCarriesDesignReason(t *testing.T) {
t.Setenv("BD_JSON_ENVELOPE", "")
oldJSON := jsonOutput
jsonOutput = true
t.Cleanup(func() { jsonOutput = oldJSON })
root := &cobra.Command{Use: "bd"}
parent := &cobra.Command{Use: "backup"}
child := &cobra.Command{Use: "sync"}
parent.AddCommand(child)
root.AddCommand(parent)
out := captureStdout(t, func() error {
_ = validateProxyRegistryBeforeProvider(child, ProxyTopologyExternalTCP)
return nil
})
for _, want := range []string{
`"code": "proxy.backup.unsupported"`,
`"mutates": false`,
`"reason": "design"`,
} {
if !strings.Contains(out, want) {
t.Fatalf("refusal JSON missing %s:\n%s", want, out)
}
}
}
// TestResolveProxiedTopology covers the resolution the gate depends on. The
// team-server case is the one worth reading twice: ownership beats transport,
// because a bts-managed database is somebody else's store whether bd reaches it
// over a socket, a port, or a child it spawned itself.
func TestResolveProxiedTopology(t *testing.T) {
for _, tc := range []struct {
name string
cfg configfile.Config
sidecar *configfile.ProxiedServerClientInfo
rawCfg string // when set, written verbatim instead of cfg
rawCar string // when set, written verbatim instead of sidecar
want ProxyTopology
}{
{
name: "managed local has no external block",
cfg: configfile.Config{DoltMode: configfile.DoltModeProxiedServer},
sidecar: &configfile.ProxiedServerClientInfo{},
want: ProxyTopologyManagedLocal,
},
{
name: "no sidecar at all is still managed local",
cfg: configfile.Config{DoltMode: configfile.DoltModeProxiedServer},
sidecar: nil,
want: ProxyTopologyManagedLocal,
},
{
name: "external host and port",
cfg: configfile.Config{DoltMode: configfile.DoltModeProxiedServer},
sidecar: &configfile.ProxiedServerClientInfo{
External: &configfile.ExternalDoltConfig{Host: "db.example.com", Port: 3306},
},
want: ProxyTopologyExternalTCP,
},
{
name: "external unix socket",
cfg: configfile.Config{DoltMode: configfile.DoltModeProxiedServer},
sidecar: &configfile.ProxiedServerClientInfo{
External: &configfile.ExternalDoltConfig{Socket: "/var/run/dolt.sock"},
},
want: ProxyTopologyExternalUnix,
},
{
name: "team server beats the transport",
cfg: configfile.Config{DoltMode: configfile.DoltModeProxiedServer, DoltTeamServer: true},
sidecar: &configfile.ProxiedServerClientInfo{
External: &configfile.ExternalDoltConfig{Host: "bts.example.com", Port: 3306},
},
want: ProxyTopologyTeamServer,
},
{
name: "team server on a local child is still team server",
cfg: configfile.Config{DoltMode: configfile.DoltModeProxiedServer, DoltTeamServer: true},
sidecar: &configfile.ProxiedServerClientInfo{},
want: ProxyTopologyTeamServer,
},
{
name: "unreadable sidecar fails closed",
cfg: configfile.Config{DoltMode: configfile.DoltModeProxiedServer},
rawCar: "{ not json",
want: ProxyTopologyUnknown,
},
{
name: "unreadable workspace config fails closed",
rawCfg: "{ not json",
want: ProxyTopologyUnknown,
},
} {
t.Run(tc.name, func(t *testing.T) {
beadsDir := filepath.Join(t.TempDir(), ".beads")
if err := os.MkdirAll(beadsDir, 0o750); err != nil {
t.Fatal(err)
}
switch {
case tc.rawCfg != "":
writeTestFile(t, configfile.ConfigPath(beadsDir), tc.rawCfg)
default:
cfg := tc.cfg
cfg.Backend = configfile.BackendDolt
data, err := json.Marshal(cfg)
if err != nil {
t.Fatal(err)
}
writeTestFile(t, configfile.ConfigPath(beadsDir), string(data))
}
switch {
case tc.rawCar != "":
writeTestFile(t, configfile.ProxiedServerClientInfoPath(beadsDir), tc.rawCar)
case tc.sidecar != nil:
data, err := json.Marshal(tc.sidecar)
if err != nil {
t.Fatal(err)
}
writeTestFile(t, configfile.ProxiedServerClientInfoPath(beadsDir), string(data))
}
if got := resolveProxiedTopology(beadsDir); got != tc.want {
t.Fatalf("resolveProxiedTopology = %q, want %q", got, tc.want)
}
})
}
}
func writeTestFile(t *testing.T, path, body string) {
t.Helper()
if err := os.WriteFile(path, []byte(body), 0o600); err != nil {
t.Fatal(err)
}
}

View file

@ -0,0 +1,477 @@
package main
import (
"context"
"database/sql"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/dbproxy/proxy"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/storage/uow"
"github.com/steveyegge/beads/internal/storage/versioncontrolops"
)
// Dolt-native backup over the proxied-server provider.
//
// The mechanism needed no proxy work at all: CALL DOLT_BACKUP(...) is ordinary
// SQL, RunNonTx already hands out a raw pinned connection (the seam `compact
// --dolt` and `gc` run on), the UOW DSN sets a connect timeout but no read
// deadline, and an active connection keeps the idle reaper from arming — so a
// backup that takes minutes is safe from both the driver and the 30s reaper.
//
// What the family DID need is the locality policy, which lives in the registry
// (see capability_registry.go's backup rows) because it is what the front door
// enforces. The guard below repeats it at the route because these functions are
// reachable from RunE, and a route that trusts the gate is one refactor away
// from writing a backup onto a host it does not own.
//
// None of these routes touch commandDidWrite or commandDidExplicitDoltCommit.
// Those two drive the post-run Dolt auto-commit, which is an embedded/direct
// concern — PersistentPostRunE's proxied branch never reads them — and the
// other proxied routes leave them alone for the same reason.
// proxiedBackupTargetName is the Dolt backup remote bd registers. Same name the
// direct path uses, so a workspace that changes topology still finds its
// destination.
const proxiedBackupTargetName = defaultDoltBackupName
// requireLocalProxiedBackup re-checks the locality policy at the route.
// Defense in depth, not a second policy: the rule and its wording come from the
// same registry row the pre-provider gate consults, so the two cannot drift.
func requireLocalProxiedBackup(path string) error {
row, ok := LookupCapabilityRow(path, "")
if !ok {
// Unreachable: TestProxyCapabilityRegistryCoversCommandTree fails the
// build for a backup path with no row. Refuse rather than guess.
return HandleErrorRespectJSON("%s has no proxied-server capability row", path)
}
rule := row.ruleFor(resolveProxiedTopology(beads.FindBeadsDir()))
if rule.Outcome == ProxyOutcomeHonored {
return nil
}
return HandleProxyCapabilityError(proxyCapabilityErrorFor(rule))
}
// proxiedActiveDatabase asks the server which database the provider opened,
// rather than re-deriving it from the workspace config. The provider already
// resolved that question (metadata.json, then --database/--db), and a second
// ladder here would be a second answer waiting to disagree with it.
func proxiedActiveDatabase(ctx context.Context) (string, error) {
var database string
err := runProxiedNonTx(ctx, func(ctx context.Context, conn *sql.Conn) error {
var name sql.NullString
if err := conn.QueryRowContext(ctx, "SELECT DATABASE()").Scan(&name); err != nil {
return fmt.Errorf("resolve active database: %w", err)
}
if !name.Valid || name.String == "" {
return fmt.Errorf("the proxied connection has no database selected")
}
database = name.String
return nil
})
return database, err
}
// runBackupInitProxied registers the backup destination, mirroring the direct
// path's conflict handling: Dolt refuses a second remote at an address another
// one already holds, and a re-init to a new path has to replace the old entry
// rather than fail.
func runBackupInitProxied(ctx context.Context, rawPath string) error {
if err := requireLocalProxiedBackup("backup init"); err != nil {
return err
}
backupURL := resolveDoltBackupURL(rawPath)
err := runProxiedNonTx(ctx, func(ctx context.Context, conn *sql.Conn) error {
addErr := versioncontrolops.BackupAdd(ctx, conn, proxiedBackupTargetName, backupURL)
if addErr == nil {
return nil
}
switch {
case strings.Contains(addErr.Error(), "already exists"):
// Same name, different URL: replace it.
_ = versioncontrolops.BackupRemove(ctx, conn, proxiedBackupTargetName)
case versioncontrolops.ExtractAddressConflictName(addErr) != "":
// A different name (auto-export registers "backup_export") already
// points at this URL; take the address over.
_ = versioncontrolops.BackupRemove(ctx, conn, versioncontrolops.ExtractAddressConflictName(addErr))
default:
return addErr
}
return versioncontrolops.BackupAdd(ctx, conn, proxiedBackupTargetName, backupURL)
})
if err != nil {
return HandleErrorRespectJSON("failed to add backup destination: %v", err)
}
if err := saveDoltBackupConfig(backupURL); err != nil {
fmt.Fprintf(os.Stderr, "Warning: backup registered but failed to save config: %v\n", err)
}
if jsonOutput {
return outputJSON(map[string]interface{}{
"backup_url": backupURL,
"backup_name": proxiedBackupTargetName,
"initialized": true,
})
}
fmt.Printf("Backup destination configured: %s\n", backupURL)
fmt.Println("Run 'bd backup sync' to push your data.")
return nil
}
// runBackupSyncProxied pushes the database to the configured destination.
func runBackupSyncProxied(ctx context.Context) error {
if err := requireLocalProxiedBackup("backup sync"); err != nil {
return err
}
start := time.Now()
err := runProxiedNonTx(ctx, func(ctx context.Context, conn *sql.Conn) error {
if err := commitWorkingSetBeforeBackup(ctx, conn); err != nil {
// Same posture as the direct path: a failed pre-backup commit
// means the backup may miss the newest writes, which is worth
// saying out loud and is not worth refusing to back up over.
fmt.Fprintf(os.Stderr, "Warning: failed to commit pending changes: %v\n", err)
}
return versioncontrolops.BackupSync(ctx, conn, proxiedBackupTargetName)
})
if err != nil {
if strings.Contains(err.Error(), "no backup") || strings.Contains(err.Error(), "not found") {
return HandleErrorRespectJSON("no backup destination configured. Run 'bd backup init <path>' first")
}
return HandleErrorRespectJSON("backup sync failed: %v", err)
}
elapsed := time.Since(start)
if err := updateDoltBackupState(elapsed); err != nil {
fmt.Fprintf(os.Stderr, "Warning: backup synced but failed to update state: %v\n", err)
}
if jsonOutput {
return outputJSON(map[string]interface{}{
"synced": true,
"duration": elapsed.String(),
})
}
fmt.Printf("Backup synced in %s\n", elapsed.Round(time.Millisecond))
return nil
}
// commitWorkingSetBeforeBackup is the proxied twin of the direct path's
// "bd: pre-backup commit". DOLT_BACKUP syncs COMMITTED history, so anything
// still in the working set would silently be missing from the backup.
//
// It is normally a no-op: every proxied write transaction ends in
// DOLT_COMMIT('-Am') (uow doltServerTx.Commit), so the working set is already
// clean. The pending check is what keeps this from minting an empty commit on
// every sync — Dolt rejects those server-side and logs each one.
func commitWorkingSetBeforeBackup(ctx context.Context, conn *sql.Conn) error {
pending, err := issueops.HasPendingChanges(ctx, conn)
if err != nil {
return fmt.Errorf("check pending changes: %w", err)
}
if !pending {
return nil
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_COMMIT('-Am', ?)", "bd: pre-backup commit"); err != nil {
if issueops.IsNothingToCommitError(err) {
return nil
}
return err
}
return nil
}
// runBackupRemoveProxied unregisters the destination and drops the local
// sidecars. The backup data itself is left alone, as in direct mode.
func runBackupRemoveProxied(ctx context.Context) error {
if err := requireLocalProxiedBackup("backup remove"); err != nil {
return err
}
err := runProxiedNonTx(ctx, func(ctx context.Context, conn *sql.Conn) error {
if err := versioncontrolops.BackupRemove(ctx, conn, proxiedBackupTargetName); err != nil {
return err
}
// auto-export may have registered a second remote at the same URL.
_ = versioncontrolops.BackupRemove(ctx, conn, "backup_export")
return nil
})
if err != nil {
if strings.Contains(err.Error(), "not found") || strings.Contains(err.Error(), "no backup") {
return HandleErrorRespectJSON("no backup destination configured")
}
return HandleErrorRespectJSON("failed to remove backup: %v", err)
}
if path, err := doltBackupConfigPath(); err == nil {
_ = os.Remove(path)
}
if path, err := doltBackupStatePath(); err == nil {
_ = os.Remove(path)
}
if jsonOutput {
return outputJSON(map[string]interface{}{"removed": true})
}
fmt.Println("Backup destination removed.")
return nil
}
// doltBackupSizeProxied measures the managed-local database on disk, which is
// what `bd backup status` reports on every other topology that can answer.
//
// It reads the directory rather than asking Dolt because that is what the
// direct stores do (DoltStore.ActiveDatabaseSize measures
// localActiveDatabaseDir), so the number means the same thing in both modes.
// The database NAME still comes from the server, so the directory measured is
// the one this connection is actually using.
func doltBackupSizeProxied(ctx context.Context) (int64, bool, error) {
beadsDir := beads.FindBeadsDir()
if beadsDir == "" {
return 0, false, nil
}
root, err := resolveProxiedServerRootPath(beadsDir)
if err != nil {
return 0, false, nil
}
database, err := proxiedActiveDatabase(ctx)
if err != nil {
return 0, false, err
}
dir := filepath.Join(root, database)
if _, err := os.Stat(dir); err != nil {
// No local directory means nothing bd can measure; the status contract
// treats size as optional rather than failing the command.
return 0, false, nil
}
size, err := storage.MeasureDirectorySize(ctx, dir)
if err != nil {
return 0, false, fmt.Errorf("measure active database directory %q: %w", dir, err)
}
return size, true, nil
}
// runBackupRestoreProxied restores the database from a Dolt backup, quiescing
// the topology first.
//
// DOLT_BACKUP('restore') REPLACES the database under the server. Doing that
// while clients hold connections to it is undefined, so the restore runs with
// bd as the only client and nothing else pointed at the store:
//
// 1. `bd backup restore` already holds the workspace and physical-root gates
// EXCLUSIVELY (commandNeedsExclusiveGate), so no other gated bd command —
// including `bd serve` — is using this workspace.
// 2. This function closes the provider the root pre-run opened and shuts the
// proxy and its dolt child down, so no connection survives into the
// replace.
// 3. It reopens a provider of its own, which relaunches the topology with
// exactly one client, and runs the restore on that.
// 4. It shuts the topology down again and reopens once more to finish: the
// reopen proves the restored database is serviceable, and it is where the
// project identity and the backup remote are reconciled. A connection
// pinned across the replace would be reading a database that no longer
// exists underneath it.
//
// The final shutdown leaves the workspace quiescent, so the next bd command
// relaunches lazily against the restored data.
func runBackupRestoreProxied(ctx context.Context, dir string, force bool) error {
if err := requireLocalProxiedBackup("backup restore"); err != nil {
return err
}
beadsDir := beads.FindBeadsDir()
if beadsDir == "" {
return HandleErrorRespectJSON("%s", activeWorkspaceNotFoundError())
}
root, err := resolveProxiedServerRootPath(beadsDir)
if err != nil {
return HandleErrorRespectJSON("%v", err)
}
backupURL, err := versioncontrolops.DirToFileURL(dir)
if err != nil {
return HandleErrorRespectJSON("%v", err)
}
// Ask the live provider which database to replace before taking it away.
database, err := proxiedActiveDatabase(ctx)
if err != nil {
return HandleErrorRespectJSON("%v", err)
}
if err := quiesceProxiedTopology(ctx, root); err != nil {
return HandleErrorRespectJSON("%v", err)
}
if err := withQuiescedProxiedProvider(ctx, beadsDir, database, root, func(ctx context.Context, conn *sql.Conn) error {
return versioncontrolops.BackupRestore(ctx, conn, backupURL, database, force)
}); err != nil {
return HandleErrorRespectJSON("%s", proxiedRestoreFailureMessage(dir, false, err))
}
if err := withQuiescedProxiedProvider(ctx, beadsDir, database, root, func(ctx context.Context, conn *sql.Conn) error {
return reconcileRestoredProxiedWorkspace(ctx, conn, beadsDir, dir, force)
}); err != nil {
return HandleErrorRespectJSON("%s", proxiedRestoreFailureMessage(dir, true, err))
}
return nil
}
// proxiedRestoreFailureMessage renders what the operator is told when one of
// the two quiesced steps fails.
//
// The distinction it makes is the only one that matters here: DID THE DATA COME
// BACK. A teardown failure happens after the step succeeded, so saying "restore
// failed" would be false, and the obvious response to it — run the restore
// again — is the wrong one. Split out from the call sites so that judgement is
// something a test can hold still (TestProxiedRestoreFailureMessage).
//
// afterReconcile distinguishes the two steps, because a teardown failure costs
// the operator different follow-up work either side of the reconcile: before
// it, the backup destination has not been re-registered yet.
func proxiedRestoreFailureMessage(dir string, afterReconcile bool, err error) string {
var teardown *proxiedTeardownError
if errors.As(err, &teardown) {
if afterReconcile {
return fmt.Sprintf("restored from %s, but shutting the proxied server down afterwards failed: %v; "+
"the data is restored and reconciled — run 'bd dolt stop --force' before the next command", dir, teardown)
}
return fmt.Sprintf("restored from %s, but shutting the proxied server down afterwards failed: %v; "+
"the data is restored — run 'bd dolt stop --force', then 'bd backup init %s' to re-register the "+
"backup destination, which this run did not reach", dir, teardown, dir)
}
if afterReconcile {
return fmt.Sprintf("restored from %s, but the restored database did not reopen: %v", dir, err)
}
return fmt.Sprintf("restore failed: %v", err)
}
// quiesceProxiedTopology drops this process's provider and stops the proxy and
// its dolt child, so nothing holds the database that is about to be replaced.
func quiesceProxiedTopology(ctx context.Context, root string) error {
if uowProvider != nil {
if err := uowProvider.Close(ctx); err != nil {
return fmt.Errorf("close the proxied provider before restoring: %w", err)
}
uowProvider = nil
}
if err := proxy.Shutdown(root); err != nil {
return fmt.Errorf("stop the proxied server before restoring: %w; "+
"run 'bd dolt stop --force' and retry", err)
}
return nil
}
// proxiedTeardownError marks a failure that happened AFTER the operation
// succeeded, while putting the topology back down. It exists so the caller can
// tell the two apart in what it prints. Flattening them is what made a restore
// that completed and then failed to shut the proxy down report "restore
// failed": the operator is told their data did not come back when it did, and
// the obvious response — run the restore again — is the wrong one.
type proxiedTeardownError struct{ err error }
func (e *proxiedTeardownError) Error() string { return e.err.Error() }
func (e *proxiedTeardownError) Unwrap() error { return e.err }
// withQuiescedProxiedProvider relaunches the topology for one operation and
// puts it back down afterwards, so each step either side of the replace runs on
// a connection that was opened after it.
//
// The provider ADOPTS whatever identity the database carries: a restore is
// precisely the operation after which the workspace's recorded project id and
// the database's may legitimately differ, and asserting the old one would
// refuse the connection that is supposed to reconcile them. That makes this the
// third caller of newProxiedServerUOWProviderAdopting; its doc comment is the
// register of identity-assertion bypasses and names this one.
//
// A teardown failure is returned wrapped in *proxiedTeardownError, and only
// when the operation itself succeeded. If opening or running the operation
// failed, preserve that error and append any cleanup failure without the
// success marker. Shutdown is attempted even if construction fails after
// starting the topology but before returning a provider.
func withQuiescedProxiedProvider(ctx context.Context, beadsDir, database, root string, fn func(context.Context, *sql.Conn) error) (err error) {
var provider uow.UnitOfWorkProvider
defer func() {
var closeErr error
if provider != nil {
closeErr = provider.Close(ctx)
}
stopErr := proxy.Shutdown(root)
if teardownErr := errors.Join(closeErr, stopErr); teardownErr != nil {
if err != nil {
err = errors.Join(err, fmt.Errorf("shut down the proxied server after failure: %w", teardownErr))
} else {
err = &proxiedTeardownError{err: teardownErr}
}
}
}()
provider, err = newProxiedServerUOWProviderAdopting(ctx, beadsDir, database)
if err != nil {
return err
}
maintenance, ok := provider.(uow.MaintenanceProvider)
if !ok {
return fmt.Errorf("the proxied provider does not support maintenance operations")
}
return maintenance.RunNonTx(ctx, fn)
}
// reconcileRestoredProxiedWorkspace is the post-restore bookkeeping, mirroring
// the direct path: adopt the restored database's project identity when the
// restore overwrote one, and register the restore source as the backup
// destination so `bd backup sync` works without a separate init.
//
// Both are best-effort with a warning, as in direct mode. The restore itself
// has already succeeded by this point, and failing the command over bookkeeping
// would tell the operator their data did not come back when it did.
func reconcileRestoredProxiedWorkspace(ctx context.Context, conn *sql.Conn, beadsDir, dir string, force bool) error {
if force {
if err := syncProjectIDFromProxiedDB(ctx, conn, beadsDir); err != nil {
fmt.Fprintf(os.Stderr, "Warning: failed to sync project ID after restore: %v\n", err)
}
}
backupURL := resolveDoltBackupURL(dir)
_ = versioncontrolops.BackupRemove(ctx, conn, proxiedBackupTargetName)
if err := versioncontrolops.BackupAdd(ctx, conn, proxiedBackupTargetName, backupURL); err != nil {
fmt.Fprintf(os.Stderr, "Warning: failed to register backup remote: %v\n", err)
return nil
}
if err := saveDoltBackupConfig(backupURL); err != nil {
fmt.Fprintf(os.Stderr, "Warning: backup registered but failed to save config: %v\n", err)
}
return nil
}
// syncProjectIDFromProxiedDB copies the restored database's _project_id into
// metadata.json, so the identity check does not reject later connections. The
// direct path reads it through the store; there is no store in proxied mode, so
// it goes through the same issueops accessor the store itself uses rather than
// a second copy of the query.
func syncProjectIDFromProxiedDB(ctx context.Context, conn *sql.Conn, beadsDir string) error {
dbID, err := issueops.GetMetadataInTx(ctx, conn, "_project_id")
if err != nil || dbID == "" {
return err
}
cfg, err := configfile.Load(beadsDir)
if err != nil {
return err
}
if cfg == nil || cfg.ProjectID == dbID {
return nil
}
cfg.ProjectID = dbID
return cfg.Save(beadsDir)
}

View file

@ -30,11 +30,10 @@ Use --force to overwrite an existing database with the backup contents.
The database must already be initialized (run 'bd init' first if needed).
To initialize and restore in one step, use: bd init && bd backup restore`,
Args: cobra.MaximumNArgs(1),
Args: cobra.MaximumNArgs(1),
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
if usesProxiedServer() {
return HandleErrorRespectJSON("backup restore is not supported in proxied-server mode")
}
evt := metrics.NewCommandEvent("backup-restore")
defer func() {
if c := metrics.Global(); c != nil {
@ -61,14 +60,24 @@ To initialize and restore in one step, use: bd init && bd backup restore`,
force, _ := cmd.Flags().GetBool("force")
if err := runBackupRestore(ctx, store, dir, force); err != nil {
if usesProxiedServer() {
if err := runBackupRestoreProxied(ctx, dir, force); err != nil {
return err
}
} else if err := runBackupRestore(ctx, store, dir, force); err != nil {
return err
}
if !jsonOutput {
fmt.Printf("%s Restore complete\n", ui.RenderPass("✓"))
// One success report for both topologies. Under --json this used to
// print nothing at all, which left a caller unable to tell a completed
// restore from a silently skipped one.
if jsonOutput {
return outputJSON(map[string]interface{}{
"restored": true,
"source": dir,
})
}
fmt.Printf("%s Restore complete\n", ui.RenderPass("✓"))
return nil
},
}

View file

@ -10,6 +10,7 @@ import (
"testing"
"github.com/spf13/cobra"
"github.com/steveyegge/beads/internal/configfile"
)
func TestBackupStatusSizeErrorRespectsOutputMode(t *testing.T) {
@ -159,25 +160,62 @@ func TestBackupStatusIncludesAvailableDatabaseSize(t *testing.T) {
})
}
func TestBackupStatusProxiedGuardDoesNotMeasureSize(t *testing.T) {
prepareBackupStatusTest(t)
proxiedServerMode = true
called := false
sizeDatabase := func(context.Context) (int64, bool, error) {
called = true
return 1, true, nil
}
// TestBackupStatusProxiedGuardFollowsLocality replaces a test that asserted a
// blanket proxied refusal. Since slice S3 the guard is scoped to locality: bd
// owns the dolt process on a managed-local workspace and can honor the command
// there, while an external one is still refused — and a refused command must
// not go on to measure anything, because the directory it would measure belongs
// to a server on another host.
func TestBackupStatusProxiedGuardFollowsLocality(t *testing.T) {
for _, tc := range []struct {
name string
sidecar string
wantMeasure bool
}{
{
name: "external topology is refused without measuring",
sidecar: `{"external":{"host":"db.example.com","port":3306}}`,
wantMeasure: false,
},
{
name: "managed local is honored and measures",
sidecar: `{}`,
wantMeasure: true,
},
} {
t.Run(tc.name, func(t *testing.T) {
beadsDir := prepareBackupStatusTest(t)
proxiedServerMode = true
if err := os.WriteFile(configfile.ProxiedServerClientInfoPath(beadsDir), []byte(tc.sidecar), 0o600); err != nil {
t.Fatalf("write proxied sidecar: %v", err)
}
cmd := newBackupStatusTestRoot(sizeDatabase)
cmd.SetArgs([]string{"backup", "status"})
_, _, err := executeBackupStatusCommand(t, cmd)
assertExitCode(t, err, 1)
if called {
t.Fatal("proxied-server guard called database size provider")
measured := false
sizeDatabase := func(context.Context) (int64, bool, error) {
measured = true
return 1, true, nil
}
cmd := newBackupStatusTestRoot(sizeDatabase)
cmd.SetArgs([]string{"backup", "status"})
_, _, err := executeBackupStatusCommand(t, cmd)
if tc.wantMeasure {
if err != nil {
t.Fatalf("backup status on managed-local: %v", err)
}
} else {
assertExitCode(t, err, 1)
}
if measured != tc.wantMeasure {
t.Fatalf("size provider called = %v, want %v", measured, tc.wantMeasure)
}
})
}
}
func prepareBackupStatusTest(t *testing.T) {
// prepareBackupStatusTest builds a throwaway workspace and returns its .beads
// directory, so a caller can drop workspace files (the proxied sidecar) into it.
func prepareBackupStatusTest(t *testing.T) string {
t.Helper()
beadsDir := filepath.Join(t.TempDir(), ".beads")
@ -198,6 +236,7 @@ func prepareBackupStatusTest(t *testing.T) {
proxiedServerMode = oldProxiedServerMode
resetCommandContext()
})
return beadsDir
}
func newBackupStatusTestRoot(sizeDatabase backupSizeFunc) *cobra.Command {

View file

@ -4,6 +4,7 @@ import (
"bufio"
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"io"
@ -22,6 +23,7 @@ import (
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/doltserver"
"github.com/steveyegge/beads/internal/metrics"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/dolt"
"github.com/steveyegge/beads/internal/storage/doltutil"
"github.com/steveyegge/beads/internal/storage/embeddeddolt"
@ -51,6 +53,12 @@ type bootstrapServerDBCheck struct {
// retries. Injectable for testing.
var bootstrapRetryDelay = func(d time.Duration) { time.Sleep(d) }
// probeGitRemoteDoltData reports whether a git remote carries Dolt data
// (refs/dolt/data). Injectable for testing; the production value is a
// 10s-bounded `git ls-remote` with credential prompts suppressed. A non-nil
// error means UNKNOWN, not "no data".
var probeGitRemoteDoltData = gitRemoteHasDoltDataRefStatus
var checkBootstrapServerDB = func(probeCfg bootstrapServerProbeConfig) bootstrapServerDBCheck {
host := probeCfg.host
port := probeCfg.port
@ -113,6 +121,9 @@ Bootstrap auto-detects the right action:
• If no database exists: creates a fresh one
• If database already exists: validates and reports status
If sync.remote points at a git repository, bootstrap verifies refs/dolt/data
before cloning. Bootstrap exits non-zero when it cannot set up a database.
This is the recommended command for:
• Setting up beads on a fresh clone
• Recovering after moving to a new machine
@ -145,6 +156,19 @@ Examples:
// Resolve non-interactive mode: flag > env var > CI env > terminal detection.
nonInteractive := isNonInteractiveBootstrap(yesFlag || nonInteractiveFlag)
// dc-6jaq: "bootstrap" is in noDbCommands, so PersistentPreRunE
// returns at the skip-store early return before its freeze gate, and
// bootstrap.go calls neither CheckReadonly nor the gate itself — yet
// it writes, and the first of those writes (the metadata repair below)
// lands before its own confirmation prompt and before it takes any
// workspace gate. --dry-run is exempt because it genuinely does not
// write; every other invocation is refused while a marker is active.
if !dryRun {
if err := migrationFreezeGateFor(cmd, "bootstrap"); err != nil {
return err
}
}
// Find beads directory
beadsDir := beads.FindBeadsDir()
if beadsDir == "" {
@ -211,7 +235,7 @@ Examples:
if cfg == nil {
cfg = configfile.DefaultConfig()
}
if err := requireBootstrapDoltBackend(cfg); err != nil {
if err := requireBootstrapDoltBackend(cfg, beadsDir); err != nil {
return HandleError("%v", err)
}
@ -231,7 +255,7 @@ Examples:
return err
}
if plan.Action == "none" || dryRun {
return nil
return bootstrapPlanOutcome(plan, dryRun)
}
} else {
if repairMsg != "" {
@ -239,7 +263,7 @@ Examples:
}
printBootstrapPlan(plan)
if plan.Action == "none" || dryRun {
return nil
return bootstrapPlanOutcome(plan, dryRun)
}
}
@ -269,14 +293,73 @@ func applyBootstrapMetadataRepair(beadsDir string, cfg *configfile.Config, apply
// BootstrapPlan describes what bootstrap will do.
type BootstrapPlan struct {
Action string `json:"action"` // "sync", "restore", "jsonl-import", "init", "none"
Reason string `json:"reason"` // Human-readable explanation
BeadsDir string `json:"beads_dir"`
Database string `json:"database"`
SyncRemote string `json:"sync_remote,omitempty"`
BackupDir string `json:"backup_dir,omitempty"`
JSONLFile string `json:"jsonl_file,omitempty"`
HasExisting bool `json:"has_existing"`
Action string `json:"action"` // "sync", "restore", "jsonl-import", "init", "none"
Reason string `json:"reason"` // Human-readable explanation
BeadsDir string `json:"beads_dir"`
Database string `json:"database"`
// SyncRemote is the remote this workspace syncs with. For Action=="sync"
// it is the clone source; for the provisioning actions it is the remote to
// register as Dolt "origin" so the first `bd dolt push` has somewhere to
// go. Always the routed (doltRemoteURL) form for forge URLs, and empty on
// a Blocked plan — nothing may be cloned from or pushed to a remote
// bootstrap could not verify.
//
// This is an operational URL, so it keeps whatever credentials the user
// configured. Run it through redactRemoteURL before printing it or putting
// it anywhere a human will read. Reason and BlockedRemote are already
// display-safe.
SyncRemote string `json:"sync_remote,omitempty"`
BackupDir string `json:"backup_dir,omitempty"`
JSONLFile string `json:"jsonl_file,omitempty"`
// Blocked marks a plan that took no action and found no database. It is
// derived in detectBootstrapAction from Action+HasExisting, never set by a
// detection branch, and it is the single discriminator the exit code and
// the printed output both switch on. Additive on purpose: JSON consumers
// switch on "action", so a new action value would break them, while
// "none" + "blocked" + a non-zero exit code is backwards compatible.
Blocked bool `json:"blocked,omitempty"`
// BlockedRemote is the credential-free remote URL a Blocked plan could not
// verify, carried only so the diagnostic can name it.
BlockedRemote string `json:"blocked_remote,omitempty"`
HasExisting bool `json:"has_existing"`
}
// MarshalJSON renders the plan for `bd bootstrap --json` (and `--dry-run --json`)
// with the credential-bearing SyncRemote redacted, mirroring the human path
// (printBootstrapPlan runs it through redactRemoteURL). SyncRemote keeps whatever
// credentials the user configured — see the field comment — and `--json` writes
// to stdout, which CI logs and agent transcripts capture, so it must not be
// emitted verbatim. Redaction is on a copy; executeBootstrapPlan still consumes
// the raw plan.SyncRemote for the clone. Reason and BlockedRemote are already
// built display-safe.
func (p BootstrapPlan) MarshalJSON() ([]byte, error) {
// The local type sheds BootstrapPlan's method set so json.Marshal does not
// recurse back into this MarshalJSON.
type bootstrapPlanJSON BootstrapPlan
safe := bootstrapPlanJSON(p)
safe.SyncRemote = redactRemoteURL(p.SyncRemote)
return json.Marshal(safe)
}
// bootstrapPlanOutcome maps a printed plan to the command's exit status.
// Action=="none" was two outcomes wearing one name — "a database already
// exists, nothing to do" and "bootstrap declined and left you with no
// database" — and both returned nil, so a rejected sync.remote printed a
// success tick and exited 0 with no database (#5743). Blocked is the
// discriminator.
//
// --dry-run always succeeds. bd doctor documents `bd bootstrap --dry-run` as
// the safe inspection step, and a preview that aborts a `set -e` provisioning
// script is a worse contract than a preview that merely reports. The blocked
// state is still printed (and still in the JSON), so the preview does predict
// what a real run would refuse to do.
func bootstrapPlanOutcome(plan BootstrapPlan, dryRun bool) error {
if dryRun || !plan.Blocked {
return nil
}
// The reason and hints are already on stderr, or the plan JSON (carrying
// "blocked": true) is already on stdout.
return SilentExit()
}
func noWorkspaceBootstrapPayload() map[string]interface{} {
@ -287,8 +370,8 @@ func noWorkspaceBootstrapPayload() map[string]interface{} {
}
}
func requireBootstrapDoltBackend(cfg *configfile.Config) error {
if err := validateConfiguredBackend(cfg); err != nil {
func requireBootstrapDoltBackend(cfg *configfile.Config, beadsDir string) error {
if err := validateConfiguredBackend(cfg, beadsDir); err != nil {
return err
}
// A registered extension backend passes validateConfiguredBackend so its
@ -304,7 +387,20 @@ func requireBootstrapDoltBackend(cfg *configfile.Config) error {
return nil
}
// detectBootstrapAction builds the plan and then derives its outcome flags in
// exactly one place. Blocked is never hand-set by a detection branch: it is a
// function of the finished plan ("took no action AND found no database"), so no
// branch can leave a stale flag behind for a later branch to inherit.
func detectBootstrapAction(beadsDir string, cfg *configfile.Config) BootstrapPlan {
plan := detectBootstrapPlan(beadsDir, cfg)
plan.Blocked = plan.Action == "none" && !plan.HasExisting
if !plan.Blocked {
plan.BlockedRemote = ""
}
return plan
}
func detectBootstrapPlan(beadsDir string, cfg *configfile.Config) BootstrapPlan {
plan := BootstrapPlan{
BeadsDir: beadsDir,
Database: cfg.GetDoltDatabase(),
@ -315,10 +411,8 @@ func detectBootstrapAction(beadsDir string, cfg *configfile.Config) BootstrapPla
// sync.remote / refs/dolt/data detection run before treating an existing
// shared-server database as "nothing to do", otherwise an unrelated default
// "beads" database can mask the real recovery path.
beadsDirExists := false
if info, err := os.Stat(beadsDir); err == nil && info.IsDir() {
beadsDirExists = true
}
info, statErr := os.Stat(beadsDir)
beadsDirExists := statErr == nil && info.IsDir()
// Check for existing database (path differs between server and embedded mode).
// Determine server/shared-server mode from the target workspace itself
@ -335,68 +429,57 @@ func detectBootstrapAction(beadsDir string, cfg *configfile.Config) BootstrapPla
// and the multi-clone upgrade guide. If the local beadsDir does not exist
// yet, still prefer sync recovery first for Action=="none" so a default
// shared-server "beads" DB from another project cannot mask a real clone.
// Held aside rather than merged into plan: a "none" verdict here is only a
// fallback for when nothing else matches, and copying it into plan leaked
// its HasExisting/Blocked/Reason into every branch below — including the
// probe-error branch, which then inherited HasExisting=true and printed the
// #5743 success tick again.
var deferredExistingPlan *BootstrapPlan
if dbAction, ok := existingBootstrapDBPlan(beadsDir, cfg, isServer, isSharedServer); ok {
if beadsDirExists || dbAction.Action != "none" {
return dbAction
}
plan = dbAction
deferredExistingPlan = &dbAction
}
// Check sync.remote (primary) or sync.git-remote (deprecated fallback)
// A configured sync.remote (primary) or sync.git-remote (deprecated
// fallback) is authoritative: try it first and, when it cannot be cloned,
// remember why so an unverifiable remote can become fatal only after local
// recovery is exhausted. With no sync.remote, auto-detect Dolt data on git
// origin instead.
syncRemote := resolveSyncRemote()
unusableRemoteReason, unusableRemoteURL := "", ""
if syncRemote != "" {
if isGitCodeRepoURL(syncRemote) {
// Cloning from a git code-repo URL via DOLT_CLONE spins dolt to
// 1000% CPU and requires manual SIGKILL. Reject and
// surface the misconfiguration rather than attempting the clone.
fmt.Fprintf(os.Stderr, "error: sync.remote %q looks like a git code-repository URL, not a Dolt remote — skipping clone\n", syncRemote)
plan.Action = "none"
plan.Reason = fmt.Sprintf("sync.remote %q rejected: git code-repository URL (not a Dolt remote)", syncRemote)
done, reason, url := planConfiguredSyncRemote(&plan, syncRemote)
if done {
return plan
}
// User-provided sync.remote — trust the URL format as-is.
// normalizeRemoteURL would convert http:// to git+http://,
// breaking Dolt remotesapi endpoints (GH#3339).
plan.SyncRemote = syncRemote
plan.Action = "sync"
plan.Reason = "sync.remote configured — will clone from " + syncRemote
return plan
// Cloning is off the table; purely local recovery is not.
unusableRemoteReason, unusableRemoteURL = reason, url
} else if originPlan, ok := gitOriginSyncPlan(plan); ok {
return originPlan
}
// Auto-detect: probe git origin for Dolt data stored in git
// (refs/dolt/data). This only applies to git remotes — Dolt-native
// remotes (DoltHub, S3, etc.) must be configured via sync.remote.
if isGitRepo() && !isBareGitRepo() {
if originURL, err := gitOriginGetURL(); err == nil && originURL != "" {
if gitOriginHasDoltDataRef() {
plan.SyncRemote = normalizeRemoteURL(originURL)
plan.Action = "sync"
plan.Reason = "Found Dolt data on git origin (refs/dolt/data) — will clone from " + originURL
return plan
}
}
// On-disk recovery sources: a non-empty backup JSONL, then a git-tracked
// issues.jsonl. When a forge sync.remote had no Dolt data yet, plan already
// carries its SyncRemote so the restored/imported workspace stays wired to it.
if recoveryPlan, ok := localFileRecoveryPlan(plan, beadsDir); ok {
return recoveryPlan
}
// Check for backup JSONL files (must be non-empty to be useful)
backupDir := filepath.Join(beadsDir, "backup")
issuesFile := filepath.Join(backupDir, "issues.jsonl")
if info, err := os.Stat(issuesFile); err == nil && info.Size() > 0 {
plan.BackupDir = backupDir
plan.Action = "restore"
plan.Reason = "Backup files found — will restore from " + backupDir
return plan
if deferredExistingPlan != nil {
return *deferredExistingPlan
}
// Check for git-tracked JSONL (the portable export format)
gitJSONL := filepath.Join(beadsDir, "issues.jsonl")
if _, err := os.Stat(gitJSONL); err == nil {
plan.JSONLFile = gitJSONL
plan.Action = "jsonl-import"
plan.Reason = "Git-tracked issues.jsonl found — will import from " + gitJSONL
return plan
}
if plan.Action == "none" {
// Only now, with every local recovery source exhausted, does an
// unverifiable remote become fatal. Restoring a local backup touches no
// remote, so an offline laptop or a credential-less CI runner must not be
// denied the recovery that the same machine would get if the probe had
// merely answered "no data".
if unusableRemoteReason != "" {
plan.Action = "none"
plan.Reason = unusableRemoteReason
plan.BlockedRemote = unusableRemoteURL
return plan
}
@ -406,6 +489,116 @@ func detectBootstrapAction(beadsDir string, cfg *configfile.Config) BootstrapPla
return plan
}
// planConfiguredSyncRemote resolves a configured sync.remote into plan. done==true
// means the plan is finished — a clone plan for a trusted remote or a forge repo
// the probe confirmed carries Dolt data — and the caller returns plan as-is.
// done==false means keep looking: either the forge repo has no Dolt data yet
// (plan.SyncRemote is still recorded so a fresh workspace gets wired to it) or the
// remote could not be verified, in which case the returned reason/url describe the
// unusable remote so it can become fatal only after local recovery is exhausted.
func planConfiguredSyncRemote(plan *BootstrapPlan, syncRemote string) (done bool, unusableReason, unusableURL string) {
if !isGitCodeRepoURL(syncRemote) {
// User-provided sync.remote — trust the URL format as-is.
// normalizeRemoteURL would convert http:// to git+http://, breaking Dolt
// remotesapi endpoints (GH#3339). That concern cannot apply here: a
// remotesapi endpoint never classifies as a git code-repo URL.
plan.SyncRemote = syncRemote
plan.Action = "sync"
plan.Reason = "sync.remote configured — will clone from " + redactRemoteURL(syncRemote)
return true, "", ""
}
// A git-forge URL is a *supported* Dolt remote: `bd init` and `bd dolt
// remote add` both persist one, and `bd dolt push` stores the database under
// refs/dolt/data on it. What must never happen is cloning it blind, or in a
// form that reaches Dolt's remotesapi client. doltRemoteURL owns that routing
// rule; probe first, and clone only what the probe actually confirmed
// (#5743, #5663).
routed := doltRemoteURL(syncRemote)
hasData, probeErr := probeGitRemoteDoltData(routed)
switch {
case probeErr != nil:
// UNKNOWN, not "no data" — never clone something we could not verify
// (init's resolveRemoteHasDoltDataProbe convention).
reason := fmt.Sprintf("could not verify refs/dolt/data on sync.remote %q: %v",
redactRemoteURL(syncRemote), probeErr)
// Said here as well as in the final verdict because a local backup may
// well rescue this run, and then this is the only place the user learns
// the remote could not be reached.
fmt.Fprintf(os.Stderr, "note: not cloning — %s; checking local recovery sources\n", reason)
return false, reason, redactRemoteURL(routed)
case hasData:
plan.SyncRemote = routed
plan.Action = "sync"
plan.Reason = "sync.remote git repo has Dolt data (refs/dolt/data) — will clone from " + redactRemoteURL(routed)
return true, "", ""
default:
// Definitively no Dolt data: the repo was wired before the first `bd
// dolt push`. Say so and keep looking, the same way bd init falls back
// to a fresh local database. Carry the URL so the workspace we create
// still gets wired to it.
plan.SyncRemote = routed
fmt.Fprintf(os.Stderr, "note: sync.remote %q has no Dolt data yet (refs/dolt/data absent); checking other recovery sources\n",
redactRemoteURL(syncRemote))
return false, "", ""
}
}
// gitOriginSyncPlan returns a clone-from-git-origin plan when git origin carries
// Dolt data (refs/dolt/data), or ok==false otherwise. Only git remotes qualify —
// Dolt-native remotes (DoltHub, S3, etc.) must be configured via sync.remote.
//
// The caller invokes this only when no sync.remote is configured. sync.remote is
// the authoritative remote for this workspace, so hydrating from whatever git
// origin happens to be (a fork, an old experiment, the code repo under the
// dedicated-data-repo pattern) is not recovery — and finalizeSyncedBootstrap would
// then rewrite the team's committed sync.remote to point at it. Skipping it also
// spares the second identical ls-remote in the common case where sync.remote IS
// the git origin, which is what bd init persists.
func gitOriginSyncPlan(plan BootstrapPlan) (BootstrapPlan, bool) {
if !isGitRepo() || isBareGitRepo() {
return BootstrapPlan{}, false
}
originURL, err := gitOriginGetURL()
if err != nil || originURL == "" {
return BootstrapPlan{}, false
}
if !gitOriginHasDoltDataRef() {
return BootstrapPlan{}, false
}
plan.SyncRemote = normalizeRemoteURL(originURL)
plan.Action = "sync"
plan.Reason = "Found Dolt data on git origin (refs/dolt/data) — will clone from " + redactRemoteURL(originURL)
return plan, true
}
// localFileRecoveryPlan returns a restore or jsonl-import plan built from on-disk
// recovery sources under beadsDir — a non-empty backup/issues.jsonl (restore),
// else a git-tracked issues.jsonl (the portable export format) — or ok==false
// when neither is present. plan is taken by value so its already-populated fields
// (e.g. a forge SyncRemote with no Dolt data yet) ride into the returned plan.
func localFileRecoveryPlan(plan BootstrapPlan, beadsDir string) (BootstrapPlan, bool) {
// Backup JSONL must be non-empty to be useful.
backupDir := filepath.Join(beadsDir, "backup")
issuesFile := filepath.Join(backupDir, "issues.jsonl")
if info, err := os.Stat(issuesFile); err == nil && info.Size() > 0 {
plan.BackupDir = backupDir
plan.Action = "restore"
plan.Reason = "Backup files found — will restore from " + backupDir
return plan, true
}
gitJSONL := filepath.Join(beadsDir, "issues.jsonl")
if _, err := os.Stat(gitJSONL); err == nil {
plan.JSONLFile = gitJSONL
plan.Action = "jsonl-import"
plan.Reason = "Git-tracked issues.jsonl found — will import from " + gitJSONL
return plan, true
}
return BootstrapPlan{}, false
}
func existingBootstrapDBPlan(beadsDir string, cfg *configfile.Config, isServer, isSharedServer bool) (BootstrapPlan, bool) {
plan := BootstrapPlan{
BeadsDir: beadsDir,
@ -455,6 +648,11 @@ func existingBootstrapDBPlan(beadsDir string, cfg *configfile.Config, isServer,
bootstrapRetryDelay(retryDelays[attempt])
}
if result.Err != nil {
// Same "declined, and no database was confirmed" class as a
// failed remote probe: report it and exit non-zero rather than
// printing a success tick (#5743). Blocked is not hand-set here —
// detectBootstrapAction derives it from Action=="none" && !HasExisting
// (HasExisting is false on this branch), keeping one owner for the flag.
plan.Action = "none"
plan.Reason = fmt.Sprintf("Could not verify existing server database %s: %v", cfg.GetDoltDatabase(), result.Err)
return plan, true
@ -534,6 +732,22 @@ func bootstrapServerPort(beadsDir string, cfg *configfile.Config, isSharedServer
func printBootstrapPlan(plan BootstrapPlan) {
switch plan.Action {
case "none":
if plan.Blocked {
// Nothing was set up and nothing was found. Never print the
// success tick here — that is the #5743 false success.
fmt.Fprintf(os.Stderr, "Bootstrap did not set up a database: %s\n", plan.Reason)
if plan.BlockedRemote != "" {
// Deliberately not "delete sync.remote and re-run": the git
// origin probe on that path swallows its error, so for a
// credential failure the retry would "succeed" by creating a
// fresh database that has diverged from the team's history.
fmt.Fprintf(os.Stderr, " Reproduce: git ls-remote %s refs/dolt/data\n",
gitRemoteURLForLsRemote(plan.BlockedRemote))
fmt.Fprintf(os.Stderr, " If the remote is right, fix access (credentials, network, VPN) and re-run 'bd bootstrap'.\n")
fmt.Fprintf(os.Stderr, " If it is wrong, point sync.remote at the Dolt remote: bd dolt remote add origin <url>\n")
}
return
}
fmt.Printf("✓ Database already exists: %s\n", plan.BeadsDir)
if !usesSQLServer() {
fmt.Printf(" Nothing to do.\n")
@ -542,7 +756,7 @@ func printBootstrapPlan(plan BootstrapPlan) {
}
case "sync":
fmt.Printf("Bootstrap plan: clone from remote\n")
fmt.Printf(" Remote: %s\n", plan.SyncRemote)
fmt.Printf(" Remote: %s\n", redactRemoteURL(plan.SyncRemote))
fmt.Printf(" Database: %s\n", plan.Database)
case "restore":
fmt.Printf("Bootstrap plan: restore from backup\n")
@ -574,7 +788,7 @@ func confirmPrompt(message string, nonInteractive bool) bool {
}
func executeBootstrapPlan(plan BootstrapPlan, cfg *configfile.Config, nonInteractive bool) error {
if err := requireBootstrapDoltBackend(cfg); err != nil {
if err := requireBootstrapDoltBackend(cfg, plan.BeadsDir); err != nil {
return err
}
if !confirmPrompt("Proceed?", nonInteractive) {
@ -643,10 +857,32 @@ func executeInitAction(ctx context.Context, plan BootstrapPlan, cfg *configfile.
return fmt.Errorf("commit: %w", err)
}
wireBootstrapDoltRemote(ctx, s, plan)
fmt.Fprintf(os.Stderr, "Created fresh database with prefix %q\n", prefix)
return nil
}
// wireBootstrapDoltRemote registers plan.SyncRemote as Dolt remote "origin" on
// a workspace bootstrap just provisioned locally.
//
// The sync action has always done this (via configureInitDoltRemote after the
// clone) and bd init does it too, but the local actions never did, because a
// configured sync.remote used to short-circuit to "sync" before they could be
// reached. The refs/dolt/data fall-through makes them reachable, and without
// the remote the promise that "the first bd dolt push creates refs/dolt/data
// on that same git remote" breaks: push either refuses to adopt a remote
// without consent, or adopts the git origin — a different repository under the
// dedicated-data-repo pattern — and rewrites the committed sync.remote to
// match. Two machines bootstrapping in the pre-first-push window would then
// mint independent Dolt identities and diverge, which is the exact failure
// bootstrap exists to prevent (#5743).
func wireBootstrapDoltRemote(ctx context.Context, s storage.DoltStorage, plan BootstrapPlan) {
if plan.SyncRemote == "" {
return
}
configureInitDoltRemote(ctx, s, doltRemoteURL(plan.SyncRemote), false)
}
func executeRestoreAction(ctx context.Context, plan BootstrapPlan, cfg *configfile.Config) error {
prefix := inferPrefix(cfg)
dbName := cfg.GetDoltDatabase()
@ -674,6 +910,7 @@ func executeRestoreAction(ctx context.Context, plan BootstrapPlan, cfg *configfi
return fmt.Errorf("restore from backup: %w", err)
}
wireBootstrapDoltRemote(ctx, s, plan)
fmt.Fprintf(os.Stderr, "Restored from backup\n")
return nil
}
@ -710,6 +947,7 @@ func executeJSONLImportAction(ctx context.Context, plan BootstrapPlan, cfg *conf
return fmt.Errorf("commit import: %w", err)
}
wireBootstrapDoltRemote(ctx, s, plan)
fmt.Fprintf(os.Stderr, "Imported %d issues from %s\n", count, plan.JSONLFile)
return nil
}
@ -777,6 +1015,28 @@ func executeSyncAction(ctx context.Context, plan BootstrapPlan, cfg *configfile.
return nil
}
// printInitJoinGuidance corrects the one thing the gate's shared-store block
// gets wrong when `bd init` is the caller: it says to run `bd migrate schema`,
// but this init failed before writing metadata.json, so there is no workspace
// here to run it from (gastownhall/beads#5920). A client JOINING an existing,
// behind shared server has to be told where the command actually works.
//
// Only the shared-store arm needs this. The remote-backed refusals from init
// are already handled by printBootstrapRemoteBehindGuidance above, and their
// remedies are not workspace-local in the same way.
func printInitJoinGuidance(w io.Writer, e *schema.RemoteMigrateGateError) {
if e.Decision != "shared-no-remote" {
return
}
fmt.Fprintf(w,
"\n This workspace was NOT created — bd init stopped before writing\n"+
" .beads/metadata.json, so `%s` cannot be run from here yet.\n"+
" Either run it from a workspace already set up against this server, or\n"+
" join and migrate in one step once every client is upgraded:\n"+
" %s=1 bd init …\n",
schema.SharedConsentCommand, schema.AllowRemoteMigrateEnv)
}
// printBootstrapRemoteBehindGuidance explains a remote-migrate gate refusal in
// bootstrap terms. The gate's generic remedy ("adopt the migrated database:
// bd bootstrap") is wrong from inside a bootstrap-style clone — the database

View file

@ -3,8 +3,11 @@
package main
import (
"encoding/json"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
@ -76,13 +79,7 @@ func TestIsGitCodeRepoURL(t *testing.T) {
// Returns a cleanup function that resets config state.
func setupSyncRemoteConfig(t *testing.T, beadsDir, remote string) func() {
t.Helper()
if err := os.WriteFile(
filepath.Join(beadsDir, "config.yaml"),
[]byte("sync.remote: "+remote+"\n"),
0o644,
); err != nil {
t.Fatalf("write config.yaml: %v", err)
}
seedSyncRemote(t, beadsDir, remote)
config.ResetForTesting()
t.Setenv("BEADS_DIR", beadsDir)
t.Setenv("BEADS_TEST_IGNORE_REPO_CONFIG", "1")
@ -92,46 +89,562 @@ func setupSyncRemoteConfig(t *testing.T, beadsDir, remote string) func() {
return func() { config.ResetForTesting() }
}
// TestDetectBootstrapAction_GitCodeRepoSyncRemoteBlocked verifies that a
// sync.remote pointing at a git code-repository host is rejected (action=none)
// instead of triggering DOLT_CLONE.
func TestDetectBootstrapAction_GitCodeRepoSyncRemoteBlocked(t *testing.T) {
for _, remote := range []string{
"git+ssh://github.com/org/repo.git",
"https://github.com/org/repo.git",
"git@github.com:org/repo.git",
"https://gitlab.com/group/repo.git",
} {
t.Run(remote, func(t *testing.T) {
snapshotBootstrapEnv(t)
// stubProbeGitRemoteDoltData replaces the refs/dolt/data probe for the
// duration of a test. Every test that configures a forge sync.remote must
// install a stub: the real probe shells out to `git ls-remote` against
// whatever host the fixture URL names.
//
// A configured sync.remote also suppresses the git-origin auto-detect leg
// (gitOriginHasDoltDataRef, which is not behind this seam), so these tests
// cannot reach the network through that path either — including the
// fall-through cases, whose whole point is to keep going after the probe.
func stubProbeGitRemoteDoltData(t *testing.T, fn func(string) (bool, error)) {
t.Helper()
orig := probeGitRemoteDoltData
probeGitRemoteDoltData = fn
t.Cleanup(func() { probeGitRemoteDoltData = orig })
}
tmpDir := t.TempDir()
beadsDir := filepath.Join(tmpDir, ".beads")
if err := os.MkdirAll(beadsDir, 0o750); err != nil {
t.Fatal(err)
// forgeSyncRemoteForms enumerates the shapes a git-forge sync.remote arrives
// in, with the git+ URL normalizeRemoteURL produces for each. Every one of
// them is a value bd itself writes (bd init from a git origin, bd dolt remote
// add, a hand-edited config.yaml, or finalizeSyncedBootstrap).
var forgeSyncRemoteForms = []struct {
name string
remote string
wantClone string
}{
{"raw https with .git", "https://github.com/org/repo.git", "git+https://github.com/org/repo.git"},
{"raw https without .git", "https://github.com/org/repo", "git+https://github.com/org/repo"},
{"scp style", "git@github.com:org/repo.git", "git+ssh://git@github.com/org/repo.git"},
{"ssh scheme", "ssh://git@github.com/org/repo.git", "git+ssh://git@github.com/org/repo.git"},
{"already git+https", "git+https://github.com/org/repo.git", "git+https://github.com/org/repo.git"},
{"git+ssh without user", "git+ssh://github.com/org/repo.git", "git+ssh://github.com/org/repo.git"},
{"gitlab", "https://gitlab.com/group/repo.git", "git+https://gitlab.com/group/repo.git"},
}
// newForgeSyncRemoteWorkspace builds a workspace whose config.yaml carries
// remote as sync.remote and chdirs into it, returning the .beads path.
func newForgeSyncRemoteWorkspace(t *testing.T, remote string) string {
t.Helper()
snapshotBootstrapEnv(t)
tmpDir := t.TempDir()
beadsDir := filepath.Join(tmpDir, ".beads")
if err := os.MkdirAll(beadsDir, 0o750); err != nil {
t.Fatal(err)
}
cleanup := setupSyncRemoteConfig(t, beadsDir, remote)
t.Cleanup(cleanup)
oldWd, _ := os.Getwd()
t.Cleanup(func() { _ = os.Chdir(oldWd) })
if err := os.Chdir(tmpDir); err != nil {
t.Fatal(err)
}
return beadsDir
}
// TestDetectBootstrapAction_GitForgeSyncRemote_ProbeRouting covers the #5743 /
// #5663 fix: a git-forge sync.remote is a supported Dolt remote, so bootstrap
// probes refs/dolt/data and routes on the answer instead of rejecting the URL
// outright (which printed a success tick and exited 0 with no database).
func TestDetectBootstrapAction_GitForgeSyncRemote_ProbeRouting(t *testing.T) {
t.Run("data present clones the git+ form", func(t *testing.T) {
for _, tc := range forgeSyncRemoteForms {
t.Run(tc.name, func(t *testing.T) {
beadsDir := newForgeSyncRemoteWorkspace(t, tc.remote)
var probed []string
stubProbeGitRemoteDoltData(t, func(url string) (bool, error) {
probed = append(probed, url)
return true, nil
})
plan := detectBootstrapAction(beadsDir, configfile.DefaultConfig())
if plan.Action != "sync" {
t.Fatalf("action=%q, want %q (remote carries refs/dolt/data)", plan.Action, "sync")
}
if plan.SyncRemote != tc.wantClone {
t.Errorf("SyncRemote=%q, want %q", plan.SyncRemote, tc.wantClone)
}
// The durable #4421 pin: once isGitCodeRepoURL recognizes a
// URL as a forge, no raw http(s) form of it may reach
// DOLT_CLONE, because Dolt routes those through the
// remotesapi client and retries forever at ~1000% CPU. The
// git+ prefix is what selects the git remote factory.
//
// Scope note: this pins the routing, not the classifier. A
// forge the classifier does not recognize (self-hosted Gitea
// at https://git.example.com/org/repo, no .git suffix) still
// takes the trusted-as-is path unprobed, exactly as it did
// before this change — a pre-existing gap tracked separately.
if !strings.HasPrefix(plan.SyncRemote, "git+") {
t.Errorf("SyncRemote=%q lacks the git+ prefix; a recognized forge URL must never reach DOLT_CLONE raw (#4421)", plan.SyncRemote)
}
if plan.Blocked {
t.Error("Blocked=true on a successful sync plan")
}
if len(probed) != 1 || probed[0] != tc.wantClone {
t.Errorf("probe calls = %v, want exactly [%q]", probed, tc.wantClone)
}
})
}
})
t.Run("no data falls through to init and still wires the remote", func(t *testing.T) {
beadsDir := newForgeSyncRemoteWorkspace(t, "https://github.com/org/repo.git")
stubProbeGitRemoteDoltData(t, func(string) (bool, error) { return false, nil })
plan := detectBootstrapAction(beadsDir, configfile.DefaultConfig())
if plan.Action != "init" {
t.Fatalf("action=%q, want %q (remote wired before the first bd dolt push)", plan.Action, "init")
}
if plan.Blocked {
t.Error("Blocked=true; a clean 'no data yet' probe is not a failure")
}
// Carried so executeInitAction can register it as Dolt "origin".
// Without it the first `bd dolt push` has no remote and either
// refuses or adopts the git origin — a different repository under
// the dedicated-data-repo pattern.
if plan.SyncRemote != "git+https://github.com/org/repo.git" {
t.Errorf("SyncRemote=%q, want the routed URL so the fresh database gets wired to it", plan.SyncRemote)
}
})
t.Run("no data falls through to backup restore", func(t *testing.T) {
beadsDir := newForgeSyncRemoteWorkspace(t, "https://github.com/org/repo.git")
backupDir := filepath.Join(beadsDir, "backup")
if err := os.MkdirAll(backupDir, 0o750); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(backupDir, "issues.jsonl"), []byte(`{"id":"bd-1"}`+"\n"), 0o644); err != nil {
t.Fatal(err)
}
stubProbeGitRemoteDoltData(t, func(string) (bool, error) { return false, nil })
plan := detectBootstrapAction(beadsDir, configfile.DefaultConfig())
// Proves the fall-through runs the whole remaining detector chain,
// not just the fresh-init tail.
if plan.Action != "restore" {
t.Fatalf("action=%q, want %q (backup must still be found after the probe falls through)", plan.Action, "restore")
}
if plan.BackupDir != backupDir {
t.Errorf("BackupDir=%q, want %q", plan.BackupDir, backupDir)
}
})
// The error text is the shape the real probe now produces: git's own
// stderr, threaded out of exec.ExitError.Stderr by gitLsRemoteProbeError.
probeErr := errors.New("git ls-remote: exit status 128: fatal: Could not read from remote repository.")
t.Run("probe error blocks and never clones", func(t *testing.T) {
const remote = "git+ssh://github.com/org/repo.git"
beadsDir := newForgeSyncRemoteWorkspace(t, remote)
stubProbeGitRemoteDoltData(t, func(string) (bool, error) { return false, probeErr })
plan := detectBootstrapAction(beadsDir, configfile.DefaultConfig())
if plan.Action != "none" {
t.Fatalf("action=%q, want %q (UNKNOWN must fail closed)", plan.Action, "none")
}
if !plan.Blocked {
t.Error("Blocked=false; an unverifiable remote must exit non-zero, not print a success tick")
}
if plan.SyncRemote != "" {
t.Errorf("SyncRemote=%q, want empty (an unverified URL must not be cloned from or pushed to)", plan.SyncRemote)
}
if plan.BlockedRemote != remote {
t.Errorf("BlockedRemote=%q, want %q (needed for the ls-remote hint)", plan.BlockedRemote, remote)
}
if !strings.Contains(plan.Reason, remote) {
t.Errorf("Reason=%q does not name the offending URL %q", plan.Reason, remote)
}
if !strings.Contains(plan.Reason, "Could not read from remote repository") {
t.Errorf("Reason=%q does not carry git's own diagnosis", plan.Reason)
}
})
t.Run("probe error still restores from a local backup", func(t *testing.T) {
// Offline laptop or credential-less CI: verifying the remote failed,
// but restoring a local backup touches no remote. The same machine
// with network and an empty remote would restore from this identical
// file, so a probe failure must not be the thing that denies it.
beadsDir := newForgeSyncRemoteWorkspace(t, "https://github.com/org/repo.git")
backupDir := filepath.Join(beadsDir, "backup")
if err := os.MkdirAll(backupDir, 0o750); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(backupDir, "issues.jsonl"), []byte(`{"id":"bd-1"}`+"\n"), 0o644); err != nil {
t.Fatal(err)
}
stubProbeGitRemoteDoltData(t, func(string) (bool, error) { return false, probeErr })
plan := detectBootstrapAction(beadsDir, configfile.DefaultConfig())
if plan.Action != "restore" {
t.Fatalf("action=%q reason=%q, want %q", plan.Action, plan.Reason, "restore")
}
if plan.Blocked {
t.Error("Blocked=true on a plan that recovers locally")
}
if plan.SyncRemote != "" {
t.Errorf("SyncRemote=%q, want empty: the remote is unverified, so nothing may be wired to it", plan.SyncRemote)
}
})
t.Run("probe error still imports git-tracked jsonl", func(t *testing.T) {
beadsDir := newForgeSyncRemoteWorkspace(t, "https://github.com/org/repo.git")
jsonl := filepath.Join(beadsDir, "issues.jsonl")
if err := os.WriteFile(jsonl, []byte(`{"id":"bd-1"}`+"\n"), 0o644); err != nil {
t.Fatal(err)
}
stubProbeGitRemoteDoltData(t, func(string) (bool, error) { return false, probeErr })
plan := detectBootstrapAction(beadsDir, configfile.DefaultConfig())
if plan.Action != "jsonl-import" {
t.Fatalf("action=%q reason=%q, want %q", plan.Action, plan.Reason, "jsonl-import")
}
if plan.Blocked {
t.Error("Blocked=true on a plan that recovers locally")
}
})
t.Run("credentials are stripped from the reason and the blocked URL", func(t *testing.T) {
const token = "ghp_SUPERSECRETTOKENVALUE"
beadsDir := newForgeSyncRemoteWorkspace(t, "https://x-access-token:"+token+"@github.com/org/repo.git")
stubProbeGitRemoteDoltData(t, func(string) (bool, error) { return false, probeErr })
plan := detectBootstrapAction(beadsDir, configfile.DefaultConfig())
if !plan.Blocked {
t.Fatalf("action=%q, want a blocked plan", plan.Action)
}
// Reason and blocked_remote both land in CI logs and agent
// transcripts; the clone funnel already scrubs userinfo before
// reporting and these diagnostics must not be the hole in that.
if strings.Contains(plan.Reason, token) {
t.Errorf("Reason leaks the token: %q", plan.Reason)
}
if strings.Contains(plan.BlockedRemote, token) {
t.Errorf("BlockedRemote leaks the token: %q", plan.BlockedRemote)
}
if !strings.Contains(plan.BlockedRemote, "github.com/org/repo.git") {
t.Errorf("BlockedRemote=%q lost the part the user needs", plan.BlockedRemote)
}
})
t.Run("routing is cwd independent in a monorepo subdir", func(t *testing.T) {
remote := "https://github.com/org/repo.git"
beadsDir := newForgeSyncRemoteWorkspace(t, remote)
subDir := filepath.Join(filepath.Dir(beadsDir), "sub", "dir")
if err := os.MkdirAll(subDir, 0o750); err != nil {
t.Fatal(err)
}
if err := os.Chdir(subDir); err != nil {
t.Fatal(err)
}
var probed []string
stubProbeGitRemoteDoltData(t, func(url string) (bool, error) {
probed = append(probed, url)
return true, nil
})
plan := detectBootstrapAction(beadsDir, configfile.DefaultConfig())
if plan.Action != "sync" {
t.Fatalf("action=%q, want %q from a subdirectory", plan.Action, "sync")
}
if len(probed) != 1 || probed[0] != "git+https://github.com/org/repo.git" {
t.Errorf("probe calls = %v, want exactly [%q]", probed, "git+https://github.com/org/repo.git")
}
})
}
// TestBootstrapPlanOutcome pins the exit status of every plan shape.
// Action=="none" used to be an unconditional success, which is how a rejected
// sync.remote exited 0 with no database (#5743). Blocked is the single
// discriminator; --dry-run is always a successful preview.
func TestBootstrapPlanOutcome(t *testing.T) {
tests := []struct {
name string
plan BootstrapPlan
dryRun bool
wantExit int // 0 means "nil error"
}{
{
name: "existing database is a genuine no-op",
plan: BootstrapPlan{Action: "none", HasExisting: true, Reason: "Database already exists at /tmp/x"},
wantExit: 0,
},
{
name: "blocked plan fails",
plan: BootstrapPlan{Action: "none", Blocked: true, Reason: "could not verify refs/dolt/data"},
wantExit: 1,
},
{
name: "sync plan proceeds",
plan: BootstrapPlan{Action: "sync", SyncRemote: "git+https://github.com/org/repo.git"},
wantExit: 0,
},
{
name: "init plan proceeds",
plan: BootstrapPlan{Action: "init"},
wantExit: 0,
},
{
// bd doctor documents `bd bootstrap --dry-run` as the safe
// inspection step, so a preview must never abort a set -e script.
name: "dry run previews a blocked plan without failing",
plan: BootstrapPlan{Action: "none", Blocked: true, Reason: "could not verify refs/dolt/data"},
dryRun: true,
wantExit: 0,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := bootstrapPlanOutcome(tt.plan, tt.dryRun)
if tt.wantExit == 0 {
if err != nil {
t.Fatalf("bootstrapPlanOutcome() = %v, want nil", err)
}
return
}
cleanup := setupSyncRemoteConfig(t, beadsDir, remote)
defer cleanup()
oldWd, _ := os.Getwd()
defer func() { _ = os.Chdir(oldWd) }()
if err := os.Chdir(tmpDir); err != nil {
t.Fatal(err)
code, ok := exitCodeFromError(err)
if !ok {
t.Fatalf("bootstrapPlanOutcome() = %v, want an *exitError", err)
}
cfg := configfile.DefaultConfig()
plan := detectBootstrapAction(beadsDir, cfg)
if plan.Action != "none" {
t.Errorf("remote=%q: action=%q, want %q (git code-repo URL must be blocked)", remote, plan.Action, "none")
}
if plan.SyncRemote != "" {
t.Errorf("remote=%q: SyncRemote=%q, want empty (blocked URL must not propagate)", remote, plan.SyncRemote)
if code != tt.wantExit {
t.Errorf("exit code = %d, want %d", code, tt.wantExit)
}
})
}
}
// TestDetectBootstrapAction_BlockedIsDerivedNotInherited covers the state-leak
// class the seeded plan used to create: existingBootstrapDBPlan's "none"
// verdict was copied into the working plan, so every later branch inherited
// its HasExisting/Blocked/Reason. A probe error on top of a seeded
// HasExisting=true produced action=none + blocked=true + has_existing=true,
// which printed the success tick and exited 0 — the #5743 bug, reintroduced.
func TestDetectBootstrapAction_BlockedIsDerivedNotInherited(t *testing.T) {
// newSeededServerWorkspace builds the synthesized-beadsDir shape: the
// .beads directory does not exist, so existingBootstrapDBPlan's verdict is
// held as a fallback rather than returned outright.
newSeededServerWorkspace := func(t *testing.T, remote string, check bootstrapServerDBCheck) (string, *configfile.Config) {
t.Helper()
snapshotBootstrapEnv(t)
tmpDir := t.TempDir()
configDir := filepath.Join(tmpDir, "config-beads")
if err := os.MkdirAll(configDir, 0o750); err != nil {
t.Fatal(err)
}
cleanup := setupSyncRemoteConfig(t, configDir, remote)
t.Cleanup(cleanup)
doltDataDir := filepath.Join(tmpDir, "dolt-data")
if err := os.MkdirAll(filepath.Join(doltDataDir, "mydb"), 0o750); err != nil {
t.Fatal(err)
}
t.Setenv("BEADS_DOLT_DATA_DIR", doltDataDir)
oldWd, _ := os.Getwd()
t.Cleanup(func() { _ = os.Chdir(oldWd) })
if err := os.Chdir(tmpDir); err != nil {
t.Fatal(err)
}
origCheck := checkBootstrapServerDB
checkBootstrapServerDB = func(bootstrapServerProbeConfig) bootstrapServerDBCheck { return check }
t.Cleanup(func() { checkBootstrapServerDB = origCheck })
origDelay := bootstrapRetryDelay
bootstrapRetryDelay = func(time.Duration) {}
t.Cleanup(func() { bootstrapRetryDelay = origDelay })
cfg := configfile.DefaultConfig()
cfg.DoltMode = configfile.DoltModeServer
cfg.DoltDatabase = "mydb"
cfg.DoltDataDir = doltDataDir
// Never created: this is the "fresh clone, .beads synthesized" path.
return filepath.Join(tmpDir, ".beads"), cfg
}
t.Run("existing server database wins over a failed probe", func(t *testing.T) {
beadsDir, cfg := newSeededServerWorkspace(t, "https://github.com/org/repo.git",
bootstrapServerDBCheck{Exists: true, Reachable: true})
stubProbeGitRemoteDoltData(t, func(string) (bool, error) {
return false, errors.New("git ls-remote: exit status 128")
})
plan := detectBootstrapAction(beadsDir, cfg)
if !plan.HasExisting {
t.Fatalf("HasExisting=false; the server database exists (action=%q reason=%q)", plan.Action, plan.Reason)
}
if plan.Blocked {
t.Error("Blocked=true alongside HasExisting=true — self-contradictory, and it would print the tick AND claim a block")
}
if err := bootstrapPlanOutcome(plan, false); err != nil {
t.Errorf("outcome = %v, want nil: a database exists", err)
}
})
t.Run("failed server verification does not taint a later sync plan", func(t *testing.T) {
beadsDir, cfg := newSeededServerWorkspace(t, "https://github.com/org/repo.git",
bootstrapServerDBCheck{Reachable: true, Err: errors.New("dial tcp: connection refused")})
stubProbeGitRemoteDoltData(t, func(string) (bool, error) { return true, nil })
plan := detectBootstrapAction(beadsDir, cfg)
if plan.Action != "sync" {
t.Fatalf("action=%q reason=%q, want %q", plan.Action, plan.Reason, "sync")
}
if plan.Blocked {
t.Error(`Blocked=true on a sync plan: "nothing may clone from a blocked plan" is the field's own contract`)
}
if plan.BlockedRemote != "" {
t.Errorf("BlockedRemote=%q on a sync plan", plan.BlockedRemote)
}
})
t.Run("unverifiable server database with no other source blocks", func(t *testing.T) {
beadsDir, cfg := newSeededServerWorkspace(t, "",
bootstrapServerDBCheck{Reachable: true, Err: errors.New("dial tcp: connection refused")})
plan := detectBootstrapAction(beadsDir, cfg)
if plan.Action != "none" || !plan.Blocked {
t.Fatalf("action=%q blocked=%v, want none+blocked", plan.Action, plan.Blocked)
}
if err := bootstrapPlanOutcome(plan, false); err == nil {
t.Error("outcome = nil; an unverifiable database must not report success")
}
})
}
// TestPrintBootstrapPlan_NoneWithoutDB_NoSuccessTick pins the other half of
// the #5743 false success: the "✓ Database already exists" line must be
// reserved for plans that actually found a database.
func TestPrintBootstrapPlan_NoneWithoutDB_NoSuccessTick(t *testing.T) {
blocked := BootstrapPlan{
Action: "none",
Blocked: true,
BeadsDir: "/workspace/.beads",
BlockedRemote: "git+ssh://github.com/org/repo.git",
Reason: `could not verify refs/dolt/data on sync.remote "git+ssh://github.com/org/repo.git": git ls-remote: exit status 128: fatal: Could not read from remote repository.`,
}
t.Run("blocked plan prints nothing to stdout", func(t *testing.T) {
out := captureStdout(t, func() error { printBootstrapPlan(blocked); return nil })
if strings.TrimSpace(out) != "" {
t.Fatalf("blocked plan wrote to stdout (the success tick lives there):\n%s", out)
}
})
t.Run("blocked plan reports the reason and actionable hints on stderr", func(t *testing.T) {
out := captureStderr(t, func() { printBootstrapPlan(blocked) })
if strings.Contains(out, "Database already exists") {
t.Fatalf("blocked plan printed a success tick:\n%s", out)
}
if !strings.Contains(out, blocked.Reason) {
t.Errorf("output does not carry the reason:\n%s", out)
}
// The hint must be runnable: git ls-remote does not understand the
// git+ prefix, so it is stripped exactly as the probe strips it.
if !strings.Contains(out, "git ls-remote ssh://github.com/org/repo.git refs/dolt/data") {
t.Errorf("output lacks a runnable ls-remote hint:\n%s", out)
}
// Deleting sync.remote is NOT offered as a remedy: the git-origin
// probe swallows its error, so on a credential failure that retry
// would "succeed" by creating a database diverged from the team's.
if strings.Contains(out, "remove 'sync.remote'") {
t.Errorf("hint still suggests deleting sync.remote, which can silently diverge:\n%s", out)
}
})
t.Run("existing database keeps the success tick", func(t *testing.T) {
plan := BootstrapPlan{
Action: "none",
HasExisting: true,
BeadsDir: "/workspace/.beads",
Reason: "Database already exists at /workspace/.beads/embeddeddolt",
}
out := captureStdout(t, func() error { printBootstrapPlan(plan); return nil })
if !strings.Contains(out, "✓ Database already exists: /workspace/.beads") {
t.Fatalf("existing database lost its success line:\n%s", out)
}
if !strings.Contains(out, "Nothing to do") {
t.Errorf("existing database lost its 'Nothing to do' line:\n%s", out)
}
})
t.Run("sync plan does not print credentials", func(t *testing.T) {
const token = "ghp_SUPERSECRETTOKENVALUE"
plan := BootstrapPlan{
Action: "sync",
BeadsDir: "/workspace/.beads",
Database: "beads",
SyncRemote: "git+https://x-access-token:" + token + "@github.com/org/repo.git",
}
out := captureStdout(t, func() error { printBootstrapPlan(plan); return nil })
if strings.Contains(out, token) {
t.Fatalf("plan output leaks the token:\n%s", out)
}
if !strings.Contains(out, "github.com/org/repo.git") {
t.Errorf("plan output lost the remote entirely:\n%s", out)
}
})
}
// TestBootstrapPlanJSON_RedactsCredentials closes the JSON half of the same
// credential leak: `bd bootstrap --json` serializes the plan, and BootstrapPlan.
// MarshalJSON must redact SyncRemote the way printBootstrapPlan redacts the human
// path. The existing "sync plan does not print credentials" subtest covers only
// printBootstrapPlan, so without this the JSON gap is invisible. It also pins the
// "redact on a copy" contract — the plan executeBootstrapPlan clones from must
// keep its real credentials.
func TestBootstrapPlanJSON_RedactsCredentials(t *testing.T) {
const token = "ghp_SUPERSECRETTOKENVALUE"
plan := BootstrapPlan{
Action: "sync",
BeadsDir: "/workspace/.beads",
Database: "beads",
SyncRemote: "git+https://x-access-token:" + token + "@github.com/org/repo.git",
}
encoded, err := json.Marshal(plan)
if err != nil {
t.Fatalf("json.Marshal(plan) failed: %v", err)
}
out := string(encoded)
if strings.Contains(out, token) {
t.Fatalf("bootstrap plan JSON leaks the token:\n%s", out)
}
if !strings.Contains(out, "github.com/org/repo.git") {
t.Errorf("bootstrap plan JSON dropped the remote host/path entirely:\n%s", out)
}
// Redaction must not mutate the plan executeBootstrapPlan consumes for the
// clone, or the clone would lose the credentials it needs.
if !strings.Contains(plan.SyncRemote, token) {
t.Errorf("MarshalJSON mutated the source plan's SyncRemote: %q", plan.SyncRemote)
}
}
// TestDetectBootstrapAction_ValidDoltSyncRemoteUnchanged verifies that a valid
// Dolt remote URL is still accepted as-is (no regression from Layer 1 guard).
func TestDetectBootstrapAction_ValidDoltSyncRemoteUnchanged(t *testing.T) {
@ -141,21 +654,15 @@ func TestDetectBootstrapAction_ValidDoltSyncRemoteUnchanged(t *testing.T) {
"git+ssh://my-self-hosted-dolt.example.com/org/db",
} {
t.Run(remote, func(t *testing.T) {
snapshotBootstrapEnv(t)
beadsDir := newForgeSyncRemoteWorkspace(t, remote)
tmpDir := t.TempDir()
beadsDir := filepath.Join(tmpDir, ".beads")
if err := os.MkdirAll(beadsDir, 0o750); err != nil {
t.Fatal(err)
}
cleanup := setupSyncRemoteConfig(t, beadsDir, remote)
defer cleanup()
oldWd, _ := os.Getwd()
defer func() { _ = os.Chdir(oldWd) }()
if err := os.Chdir(tmpDir); err != nil {
t.Fatal(err)
}
// Non-forge remotes take the trusted-as-is path and must never
// pay for an ls-remote — a Dolt remotesapi endpoint has no
// refs/dolt/data to probe in the first place.
stubProbeGitRemoteDoltData(t, func(url string) (bool, error) {
t.Fatalf("probed non-forge remote %q; the trusted-as-is path must not probe", url)
return false, nil
})
cfg := configfile.DefaultConfig()
plan := detectBootstrapAction(beadsDir, cfg)

View file

@ -21,7 +21,7 @@ func TestRequireBootstrapDoltBackendRejectsRegisteredBackend(t *testing.T) {
const name = "registry-bootstrap"
registerContractBackend(t, name)
err := requireBootstrapDoltBackend(&configfile.Config{Backend: name})
err := requireBootstrapDoltBackend(&configfile.Config{Backend: name}, t.TempDir())
if err == nil {
t.Fatal("requireBootstrapDoltBackend accepted a registered backend; want fail-closed rejection")
}
@ -40,7 +40,7 @@ func TestRequireBootstrapDoltBackendRejectsRegisteredBackend(t *testing.T) {
{"empty backend", &configfile.Config{}},
{"explicit dolt", &configfile.Config{Backend: configfile.BackendDolt}},
} {
if err := requireBootstrapDoltBackend(tc.cfg); err != nil {
if err := requireBootstrapDoltBackend(tc.cfg, t.TempDir()); err != nil {
t.Errorf("requireBootstrapDoltBackend rejected the %s config: %v", tc.label, err)
}
}

View file

@ -355,9 +355,7 @@ func TestDetectBootstrapAction_ExplicitSyncRemotePreservesRemotesAPIURL(t *testi
t.Fatal(err)
}
const syncRemote = "http://myserver:7007/mydb"
if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte("sync.remote: "+syncRemote+"\n"), 0o644); err != nil {
t.Fatal(err)
}
seedSyncRemote(t, beadsDir, syncRemote)
t.Setenv("BEADS_DIR", beadsDir)
t.Setenv("BEADS_TEST_IGNORE_REPO_CONFIG", "1")
if err := config.Initialize(); err != nil {
@ -407,9 +405,7 @@ func TestDetectBootstrapAction_ExistingEmbeddedDBWithSyncRemoteIsNoOp(t *testing
t.Fatal(err)
}
const syncRemote = "http://myserver:7007/mydb"
if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte("sync.remote: "+syncRemote+"\n"), 0o644); err != nil {
t.Fatal(err)
}
seedSyncRemote(t, beadsDir, syncRemote)
t.Setenv("BEADS_DIR", beadsDir)
t.Setenv("BEADS_TEST_IGNORE_REPO_CONFIG", "1")
if err := config.Initialize(); err != nil {
@ -1327,8 +1323,18 @@ func TestFinalizeSyncedBootstrapWritesConfigFiles(t *testing.T) {
// sync.remote must be persisted so subsequent fresh clones (and
// bootstrap retries) can rediscover the remote without re-probing
// origin refs.
if !strings.Contains(yaml, "sync.remote: ") && !strings.Contains(yaml, "sync-remote: ") {
t.Errorf("config.yaml does not contain sync.remote entry:\n%s", yaml)
//
// Asserted by READING it back rather than by matching a spelling. This used
// to require a literal `sync.remote: ` line, which is a key whose name
// contains a dot — the shape config.GetStringFromDir and viper can never
// find, because both split on the dot and walk nested mappings. The writer
// now nests (bd-zj95), so the old assertion passed for exactly as long as
// the value was unreadable and failed the moment it became readable. What
// bootstrap owes its caller is a remote that can be read back, so that is
// what this checks.
if got := config.GetStringFromDir(beadsDir, "sync.remote"); got != syncRemote {
t.Errorf("config.GetStringFromDir(%q, \"sync.remote\") = %q, want %q:\n%s",
beadsDir, got, syncRemote, yaml)
}
if !strings.Contains(yaml, syncRemote) {
t.Errorf("config.yaml does not contain sync remote URL %q:\n%s", syncRemote, yaml)

View file

@ -0,0 +1,685 @@
package main
import (
"sort"
"strings"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
)
// The proxied-server capability registry: one table for every command path.
//
// Before this file the policy lived in three separate tables — a maintenance
// refusal map, a history classification map, and a transform row list — each
// with its own lookup and its own front-door validator. Nothing forced a new
// command to appear in any of them, so an unclassified command sailed past the
// gate and died deep in the store factory with an untyped error. The fix is not
// a fourth table: it is one table plus a test that fails when a command is
// missing from it (TestProxyCapabilityRegistryCoversCommandTree).
//
// WHAT A ROW MEANS. Outcome describes what the PRE-PROVIDER GATE does with the
// path on a proxied-server workspace, nothing more:
//
// ProxyOutcomeRefused the gate refuses with a typed ProxyCapabilityError
// before any workspace side effect
// ProxyOutcomeRefusedInRunE the gate lets the path through and the command
// refuses inside its own RunE with an untyped
// string; recorded here so the untyped surface is
// inventoried rather than invisible. Not enforced.
// ProxyOutcomeHonored the gate lets the path through. It does NOT
// promise a proxied route exists: a permitted
// command with no route still fails at the store
// factory with proxy.store.unrouted.
//
// Permitting a path also COSTS something, and the cost is invisible from this
// table: unless the command is on one of main.go's store-init skip lists, the
// root pre-run opens the proxied provider before RunE runs — which starts the
// proxy and its dolt child, creates the database and migrates the schema. For a
// command that genuinely uses the store that is the point. For a store-free one
// it is pure side effect, and on a workspace whose topology cannot be resolved
// it turns a clean typed refusal into a provider-open error. So "this command
// touches no store" is an argument for permitting it only once the command is
// also off the store-init path; see the `migrate hooks` row. Nothing here
// asserts that, and TestProxyCapabilityRegistryCoversCommandTree cannot see it.
//
// REASON is the load-bearing field. "design" means refusing is correct and is
// expected to stay that way — the semantics are undefined or unsafe against a
// backend that may be shared. "unimplemented" means the refusal is a capability
// gap with a named slice that closes it, and Tracking says which. Every later
// slice measures itself by turning "unimplemented" rows into honored ones; the
// count of remaining "unimplemented" rows is the completion signal for the whole
// proxied-parity effort. Assign the field row by row, from the command's actual
// semantics — a bulk assignment would destroy the only information it carries.
//
// The reason also renders in the refusal JSON, additive to the frozen
// {code, error, mutates} contract (see buildJSONCapabilityError). Codes are
// frozen; messages are informative-only.
// ProxyRefusalReason distinguishes a deliberate policy refusal from a
// not-yet-built one.
type ProxyRefusalReason string
const (
// ProxyReasonDesign: the command is refused on purpose and is expected to
// stay refused. Lifting it would need a semantic decision, not plumbing.
ProxyReasonDesign ProxyRefusalReason = "design"
// ProxyReasonUnimplemented: the command could work over the proxied
// provider; nobody has routed it yet. Tracking names the work item.
ProxyReasonUnimplemented ProxyRefusalReason = "unimplemented"
)
// Tracking values name the plan-of-record work item for an unimplemented
// refusal. These are the slices of the proxied-parity plan; a row gets a GitHub
// issue or bead ID instead once one is filed for it.
// S3 has no entry: it landed, and its rows carry Reason=design rather than a
// tracking item.
const (
trackVersionCtl = "proxied-parity S4 (dolt commit/push/pull/remote)"
trackSync = "proxied-parity S5 (bd sync)"
trackDegradation = "proxied-parity S6 (doctor, config show, degradations)"
trackLongTail = "proxied-parity S7 (long-tail policy pass)"
)
// capabilityRow is one command-path policy row. ArgSet is the sorted set of
// execution-altering flags that were set (for example "--auto-merge --dry-run"),
// empty for the bare path; a row keyed on an ArgSet wins over the bare row, which
// is how `duplicates --auto-merge --dry-run` stays honored while
// `duplicates --auto-merge` is refused.
//
// Topology overrides Rule on the proxied shapes named in it and is empty for
// every row whose answer is the same everywhere — which is all of them but the
// backup family. See honoredOn.
type capabilityRow struct {
Path string
ArgSet string
Rule proxyCapabilityRule
Topology map[ProxyTopology]proxyCapabilityRule
History HistoryCapabilityClass
Note string
}
// ruleFor resolves the row's policy on one proxied topology.
func (r capabilityRow) ruleFor(topology ProxyTopology) proxyCapabilityRule {
if rule, ok := r.Topology[topology]; ok {
return rule
}
return r.Rule
}
// honoredOn narrows a refusal to the shapes where it is actually true: the row
// goes on refusing everywhere except the topologies named here.
//
// The direction is deliberate. A row states its refusal first and then lists
// the exceptions, so a shape nobody thought about — a new transport, or the
// unknown a corrupt sidecar produces — inherits the refusal rather than the
// exemption.
func (r capabilityRow) honoredOn(topologies ...ProxyTopology) capabilityRow {
r.Topology = make(map[ProxyTopology]proxyCapabilityRule, len(topologies))
for _, topology := range topologies {
r.Topology[topology] = proxyCapabilityRule{Outcome: ProxyOutcomeHonored}
}
return r
}
// display renders the row's key the way its refusal message names it.
func (r capabilityRow) display() string {
if r.ArgSet == "" {
return r.Path
}
return r.Path + " " + r.ArgSet
}
func (r capabilityRow) withHistory(class HistoryCapabilityClass) capabilityRow {
r.History = class
return r
}
// asParentGroup marks a row whose path is a command GROUP. Cobra answers a
// non-runnable parent with its help text before PersistentPreRunE, so the gate
// never sees it; the row is kept because it is the parent every child row
// inherits its code from, and because a parent that later gains a Run must not
// silently become permitted.
func (r capabilityRow) asParentGroup() capabilityRow {
r.Note = "parent group: cobra shows help instead of reaching the gate"
return r
}
func (r capabilityRow) withMessage(message string) capabilityRow {
r.Rule.Message = message
return r
}
// refusedPath builds a typed front-door refusal for a bare command path.
func refusedPath(path, code string, reason ProxyRefusalReason, tracking string) capabilityRow {
return refusedArgs(path, "", code, reason, tracking)
}
// refusedArgs builds a typed front-door refusal keyed on a flag set.
func refusedArgs(path, argSet, code string, reason ProxyRefusalReason, tracking string) capabilityRow {
row := capabilityRow{Path: path, ArgSet: argSet}
row.Rule = proxyCapabilityRule{
Outcome: ProxyOutcomeRefused,
Code: code,
Message: row.display() + " is not supported in proxied-server mode",
ExitCode: 1,
Reason: reason,
Tracking: tracking,
}
return row
}
// refusedInRunE records a command that the gate permits and that then refuses
// itself with an untyped string from RunE. The row carries no message: the
// string lives in the command, and duplicating it here would rot. The scan in
// TestProxyCapabilityRegistryCoversInlineRefusals keeps the two in step.
//
// The reason is always "unimplemented" and the owner is always the long-tail
// slice, because an untyped string inside RunE is not a considered policy
// statement: a refusal decided on purpose belongs in the gate, typed, where a
// consumer can read it. Anything here is a family nobody has routed yet.
func refusedInRunE(path, site string) capabilityRow {
return capabilityRow{
Path: path,
Rule: proxyCapabilityRule{
Outcome: ProxyOutcomeRefusedInRunE, ExitCode: 1,
Reason: ProxyReasonUnimplemented, Tracking: trackLongTail,
},
Note: "untyped refusal in " + site,
}
}
// permitted records that the gate lets a path through. See the header: this is
// a statement about the gate, not a promise that a proxied route exists.
func permitted(path string) capabilityRow {
return capabilityRow{Path: path, Rule: proxyCapabilityRule{Outcome: ProxyOutcomeHonored}}
}
// backupRefusal builds one backup-family row: honored where bd owns the Dolt
// server, refused by design everywhere else. The code stays
// proxy.backup.unsupported — consumers have been keying on it since 1.3.0 and
// the refusal it names has not changed, only its reason and its scope — and the
// message keeps the frozen "<path> is not supported in proxied-server mode"
// prefix, with the reason appended.
//
// The message leads with SERVER-GLOBAL STATE rather than with the filesystem,
// because only the filesystem half is scheme-specific. `bd backup init` accepts
// https/aws/gs as well as file:// (resolveDoltBackupURL) and the help text
// recommends DoltHub; for those a server-side push over the network with the
// server's own credentials is a coherent thing to ask for, and "the destination
// is on the server's disk" says nothing about it. What is true of every scheme
// is that `CALL DOLT_BACKUP('add', …)` registers the remote ON THE SERVER,
// where it is global to every client of that server — one workspace's backup
// decision silently becomes everyone's, which is the shape of the auto-backup
// storm this command's own help text warns about.
//
// backupRemoteSchemeTracking is why these rows carry a Tracking item despite
// being Reason=design: the outcome is settled, the remote-scheme CASE is not.
func backupRefusal(path string) capabilityRow {
return refusedPath(path, "proxy.backup.unsupported", ProxyReasonDesign, backupRemoteSchemeTracking).
withMessage(path + " is not supported in proxied-server mode against a Dolt server bd does not own: " +
"the backup remote is registered on the server, where it is global to every client, " +
"and a file:// destination names a filesystem this client cannot see").
honoredOn(ProxyTopologyManagedLocal)
}
// backupRemoteSchemeTracking records the one part of the backup refusal that is
// a question rather than a verdict, so refusing the family off-host does not
// silently foreclose it. A remote-scheme destination (DoltHub, aws://, gs://)
// needs no filesystem bd can see — the server pushes it over the network from
// its own environment, which is exactly the credential story S4 has to settle
// for `dolt push` on these same topologies. Whether a per-client backup remote
// on a shared server is ever acceptable is the open part; the plumbing is not.
//
// Note also that the DoltHub story is already half-built in DIRECT mode:
// `bd backup init https://…` and `bd backup sync` work, but
// `bd backup restore https://…` does not, because validateBackupRestoreDir
// os.Stats its argument and rejects anything that is not an existing directory
// on every topology. Pre-existing, not something this slice changed.
const backupRemoteSchemeTracking = "open question, not a verdict: remote-scheme (DoltHub/aws/gs) backup on a server " +
"bd does not own — revisit with S4, which settles the same server-env credential question for dolt push"
// proxyCapabilityRegistry is the whole path-keyed policy. Rows for permitted
// paths are appended from proxyPermittedPaths by init.
var proxyCapabilityRegistry = []capabilityRow{
// --- doctor -------------------------------------------------------------
// Diagnosis needs no version-control surface; the checks simply have no
// proxied dual yet. S6 splits them into client-local and store-backed.
refusedPath("doctor", "proxy.doctor.unsupported", ProxyReasonUnimplemented, trackDegradation),
// --- backup / restore ---------------------------------------------------
// CALL DOLT_BACKUP(...) is pure SQL over a connection proxied mode already
// owns, so routing this family needed no proxy work at all — see
// backup_proxied_server.go.
//
// What it did need is the ownership policy, which is the first genuinely
// topology-differentiated rule in this table. A backup remote belongs to the
// SERVER, not to the client that asked for it: `CALL DOLT_BACKUP('add', …)`
// registers it in the server's own configuration, where it is global to
// every client connected to that server. On managed-local "every client" is
// this workspace, because bd spawned the child itself. Anywhere else one
// operator's backup decision silently becomes everyone's — the same shape as
// the auto-backup storm this command's help text already warns about, and
// the reason team-server is the sharpest case: bts owns the store.
//
// A file:// destination adds a second, scheme-specific problem on top: it
// names a path on the machine running dolt, so off-host the command either
// fails at the point of use or, far worse, reports success having written
// the backup somewhere the operator will never look for it.
//
// Hence: honored on managed-local, refused by design everywhere else. The
// OUTCOME is design — server-global state is not something plumbing on this
// side can make per-client, and a backup story for those shapes belongs to
// whoever runs the server. The remote-scheme case (DoltHub, aws://, gs://,
// which resolveDoltBackupURL accepts and backup.go recommends) is the part
// that is a question rather than a verdict, and it is not silently
// foreclosed: see backupRemoteSchemeTracking, which every row below carries.
backupRefusal("backup").asParentGroup(),
backupRefusal("backup init"),
backupRefusal("backup sync"),
backupRefusal("backup remove"),
backupRefusal("backup status"),
backupRefusal("backup restore"),
// --- version control over a shared history ------------------------------
// Refused by design: these mutate or traverse a history that several
// clients share through one connection, and the per-client semantics
// (whose working set? whose branch head?) are undefined. `diff` is the one
// plausible 1.4 candidate, via the dolt_diff system tables.
refusedPath("branch", "proxy.branch.unsupported", ProxyReasonDesign, "").
withHistory(HistoryDirectOnly),
refusedPath("diff", "proxy.diff.unsupported", ProxyReasonDesign, "1.4 candidate: dolt_diff system tables"),
refusedPath("flatten", "proxy.flatten.unsupported", ProxyReasonDesign, "").
withHistory(HistoryDirectOnly),
refusedPath("conflicts", "proxy.conflicts.unsupported", ProxyReasonDesign, "").
withHistory(HistoryDirectOnly).
asParentGroup(),
refusedPath("conflicts list", "proxy.conflicts.unsupported", ProxyReasonDesign, ""),
refusedPath("conflicts show", "proxy.conflicts.unsupported", ProxyReasonDesign, ""),
refusedPath("conflicts resolve", "proxy.conflicts.unsupported", ProxyReasonDesign, ""),
refusedPath("vc", "proxy.vc.unsupported", ProxyReasonDesign, "").
withHistory(HistoryDirectOnly).
asParentGroup(),
refusedPath("vc merge", "proxy.vc.unsupported", ProxyReasonDesign, ""),
refusedPath("vc commit", "proxy.vc.unsupported", ProxyReasonDesign, ""),
refusedPath("vc status", "proxy.vc.unsupported", ProxyReasonDesign, ""),
refusedPath("federation", "proxy.federation.unsupported", ProxyReasonDesign, "").
withHistory(HistoryDirectOnly).
asParentGroup(),
refusedPath("federation sync", "proxy.federation.unsupported", ProxyReasonDesign, ""),
refusedPath("federation status", "proxy.federation.unsupported", ProxyReasonDesign, ""),
refusedPath("federation add-peer", "proxy.federation.unsupported", ProxyReasonDesign, ""),
refusedPath("federation remove-peer", "proxy.federation.unsupported", ProxyReasonDesign, ""),
refusedPath("federation list-peers", "proxy.federation.unsupported", ProxyReasonDesign, ""),
// --- multi-repo routing -------------------------------------------------
// Routing to another workspace bypasses the proxied root entirely, which is
// the same reason --repo is N/A mode-wide.
refusedPath("repo", "proxy.repo.unsupported", ProxyReasonDesign, "").
withHistory(HistoryDirectOnly).
asParentGroup(),
refusedPath("repo add", "proxy.repo.unsupported", ProxyReasonDesign, ""),
refusedPath("repo remove", "proxy.repo.unsupported", ProxyReasonDesign, ""),
refusedPath("repo list", "proxy.repo.unsupported", ProxyReasonDesign, ""),
refusedPath("repo sync", "proxy.repo.unsupported", ProxyReasonDesign, ""),
// --- dolt version-control verbs -----------------------------------------
// CALL DOLT_PUSH/DOLT_PULL/DOLT_COMMIT are SQL procedures and the generic
// repository that wraps them is already written; these need a use case and
// a route, not proxy work. `dolt remote remove` is already routed, which is
// the precedent.
refusedPath("dolt commit", "proxy.dolt_commit.unsupported", ProxyReasonUnimplemented, trackVersionCtl).
withHistory(HistoryDirectOnly),
refusedPath("dolt push", "proxy.dolt_push.unsupported", ProxyReasonUnimplemented, trackVersionCtl).
withHistory(HistoryDirectOnly),
refusedPath("dolt pull", "proxy.dolt_pull.unsupported", ProxyReasonUnimplemented, trackVersionCtl).
withHistory(HistoryDirectOnly),
refusedPath("dolt remote", "proxy.dolt_remote.unsupported", ProxyReasonUnimplemented, trackVersionCtl).
withHistory(HistoryDirectOnly).
asParentGroup(),
refusedPath("dolt remote add", "proxy.dolt_remote.unsupported", ProxyReasonUnimplemented, trackVersionCtl).
withHistory(HistoryDirectOnly),
refusedPath("dolt remote list", "proxy.dolt_remote.unsupported", ProxyReasonUnimplemented, trackVersionCtl).
withHistory(HistoryDirectOnly),
refusedPath("dolt remote reset-data", "proxy.dolt_remote.unsupported", ProxyReasonUnimplemented, trackVersionCtl).
withHistory(HistoryDirectOnly),
refusedPath("sync", "proxy.sync.unsupported", ProxyReasonUnimplemented, trackSync).
withHistory(HistoryDirectOnly),
// --- migration and workspace surgery ------------------------------------
// Storage-format and workspace surgery against a store other clients may be
// holding open. `migrate schema` is the exception and is already routed:
// the provider open runs it under the migration gate.
//
// S7 MUST RE-EXAMINE THE ROWS BELOW PER TOPOLOGY. "A store other clients may
// be holding open" describes external and team-server workspaces; it does
// not describe managed-local, which is 1:1 and bd-owned — bd spawned the
// dolt child itself. A row cannot yet say "refused, except on managed-local"
// (S3's backup locality policy is the slice that adds per-topology rules),
// so the blanket refusal is the only thing this table can express today.
// These rows keep it because nobody has done the semantic work to narrow
// them, not because the work has been done and come out this way.
refusedPath("migrate", "proxy.migrate.unsupported", ProxyReasonDesign, ""),
refusedPath("migrate sync", "proxy.migrate.unsupported", ProxyReasonDesign, ""),
// `migrate hooks` is the one row in this block the surgery rationale never
// described: it plans and applies GIT HOOK FILE migration (migrate_hooks.go
// -> doctor.PlanHookMigration) on the local filesystem and opens no store at
// all. It is refused for a structural reason instead — the command is wired
// into the store-opening path it does not need. It is on none of main.go's
// store-init skip lists, so permitting it makes the root pre-run open the
// proxied provider before RunE ever runs. Measured, not assumed: with this
// row flipped to permitted, `bd migrate hooks --dry-run` on a managed-local
// workspace started the proxy AND its dolt child (both pidfiles present) to
// print a hook plan, and on a workspace with an unreadable sidecar it exited
// 1 with "failed to open uow provider: corrupt proxied-server sidecar ..."
// where it had cleanly refused before. Trading a typed refusal for a
// connection error is not a lift. The honest fix is to take the command off
// the store path first, which is S7's job, not this slice's.
refusedPath("migrate hooks", "proxy.migrate.unsupported", ProxyReasonUnimplemented,
trackLongTail+" (specifically: put `migrate hooks` on main.go's store-init skip list, then permit it)"),
refusedPath("migrate issues", "proxy.migrate.unsupported", ProxyReasonDesign, ""),
refusedPath("migrate-issues", "proxy.migrate.unsupported", ProxyReasonDesign, ""),
refusedPath("migrate-personal", "proxy.migrate.unsupported", ProxyReasonDesign, ""),
// --- destructive admin --------------------------------------------------
// Same topology-blind rationale as the migrate block above, and the same
// instruction to S7: "destructive against a possibly-shared store" is a
// claim about external and team-server shapes. On managed-local the store is
// this workspace's alone, and `bd admin reset` there is no more shared than
// it is in embedded mode, where it is honored. Re-examine per topology.
refusedPath("admin cleanup", "proxy.admin.unsupported", ProxyReasonDesign, ""),
refusedPath("admin reset", "proxy.admin.unsupported", ProxyReasonDesign, ""),
// --- issue compaction ---------------------------------------------------
// `bd admin compact` without --dolt is the description-rewriting pass; only
// the --dolt arm has a proxied route (runCompactDoltProxiedServer). The
// --dolt arm is decided by flag VALUE, so the gate keeps a value-aware
// branch for it and lands here when the flag is off.
// The wording names the real command path. "compact --dolt" reads as the
// root `bd compact`, a different command with no --dolt flag and its own
// proxied route; compact.go's RunE fallback carries the same corrected
// string so the gate and the fallback cannot drift apart.
refusedPath("admin compact", "proxy.compact.unsupported", ProxyReasonUnimplemented, trackLongTail).
withMessage("only 'bd admin compact --dolt' is supported in proxied-server mode"),
// `bd restore` recovers a COMPACTED ISSUE's original text, not a backup —
// the backup verb is `bd backup restore`. Its snapshot read is ordinary
// CRUD; only its fallback path (reconstructing from Dolt history when no
// snapshot was archived) touches version control.
refusedPath("restore", "proxy.restore.unsupported", ProxyReasonUnimplemented, trackLongTail),
// --- gates, formulas, swarm, merge slots --------------------------------
// Ordinary CRUD over a store with no version-control surface. They stay
// refused because nothing downstream has asked for them, which is this
// codebase's standing rule for lifting a refusal — not because the proxied
// topology makes them wrong.
refusedPath("gate discover", "proxy.gate.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("cook", "proxy.formula.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("ship", "proxy.formula.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("swarm create", "proxy.swarm.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("swarm list", "proxy.swarm.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("merge-slot create", "proxy.merge_slot.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("merge-slot check", "proxy.merge_slot.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("merge-slot acquire", "proxy.merge_slot.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("merge-slot release", "proxy.merge_slot.unsupported", ProxyReasonUnimplemented, trackLongTail),
// --- transforms ---------------------------------------------------------
// ID rewrites walk every reference in the store. They need a routed
// multi-statement transaction, which the UOW can express; nobody has
// written it. The arg-set rows are what let the read-only preview of a
// duplicate merge stay honored while the merge itself is refused.
refusedPath("rename", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("rename-prefix", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedArgs("rename-prefix", "--dry-run", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedArgs("rename-prefix", "--repair", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedArgs("rename-prefix", "--dry-run --repair", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("duplicate", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedArgs("duplicate", "--of", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedPath("supersede", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedArgs("supersede", "--with", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
refusedArgs("duplicates", "--auto-merge", "proxy.transform.unsupported", ProxyReasonUnimplemented, trackLongTail),
// The dry run reports what the merge WOULD do without writing anything, so
// it rides the read path plain `duplicates` already uses.
{Path: "duplicates", ArgSet: "--auto-merge --dry-run", Rule: proxyCapabilityRule{Outcome: ProxyOutcomeHonored}},
// --- routed history -----------------------------------------------------
// The two paths that already have a proxied route, recorded so the class is
// asserted rather than remembered.
permitted("history").withHistory(HistoryProxySupported),
permitted("dolt remote remove").withHistory(HistoryProxySupported),
// --- untyped refusals inside RunE ---------------------------------------
// The gate permits these paths and the command refuses itself with a bare
// string, so a JSON consumer gets no code and no mutates flag. They are
// listed to inventory that surface, not to enforce it: converting them to
// typed refusals changes observable output and belongs to the slice that
// owns each family. Tracker verbs are ordinary store reads plus a network
// call; nothing about the proxied topology makes them wrong.
refusedInRunE("ado status", "ado.go"),
refusedInRunE("ado projects", "ado.go"),
refusedInRunE("ado sync", "ado.go"),
refusedInRunE("github status", "github.go"),
refusedInRunE("github repos", "github.go"),
refusedInRunE("github sync", "github.go"),
refusedInRunE("gitlab status", "gitlab.go"),
refusedInRunE("gitlab projects", "gitlab.go"),
refusedInRunE("gitlab sync", "gitlab.go"),
refusedInRunE("notion status", "notion.go"),
refusedInRunE("notion init", "notion.go"),
refusedInRunE("notion connect", "notion.go"),
refusedInRunE("notion sync", "notion.go"),
refusedInRunE("create-form", "create_form.go"),
}
// proxyPermittedPaths lists every command path the proxied front door lets
// through. It exists so that a command added without a policy decision fails
// TestProxyCapabilityRegistryCoversCommandTree instead of shipping — the whole
// point of the registry. Being listed here says the GATE permits the path; a
// permitted command with no proxied route still fails at the store factory with
// proxy.store.unrouted.
var proxyPermittedPaths = []string{
// issue CRUD and queries
"assign", "batch", "blocked", "children", "close", "comment", "count", "create",
"defer", "delete", "duplicates", "edit", "export", "find-duplicates", "forget",
"gc", "graph", "graph check", "heartbeat", "import", "info", "lint", "link",
"list", "note", "orphans", "priority", "promote", "prune", "purge", "q", "query",
"ready", "recompute-blocked", "reclaim", "reopen", "search", "set-state", "show",
"stale", "status", "statuses", "tag", "types", "unclaim", "undefer", "update",
// comments, dependencies, labels, key-value
"comments", "comments add", "comments list",
"dep", "dep add", "dep cycles", "dep list", "dep relate", "dep remove", "dep tree", "dep unrelate",
"kv clear", "kv get", "kv list", "kv set",
"label add", "label list", "label list-all", "label propagate", "label remove",
// molecules, epics, todos, state
"epic close-eligible", "epic status",
"mol bond", "mol burn", "mol current", "mol distill", "mol last-activity", "mol pour",
"mol progress", "mol ready", "mol seed", "mol show", "mol squash", "mol stale",
"mol wisp", "mol wisp create", "mol wisp gc", "mol wisp list",
"state", "state list", "todo", "todo add", "todo done", "todo list",
// gates, rules, provenance, audit, upgrade review
"audit label", "audit record",
"gate add-waiter", "gate check", "gate create", "gate list", "gate resolve", "gate show",
"provenance by-ref", "provenance log", "provenance record",
"rules audit", "rules compact",
"upgrade ack", "upgrade review", "upgrade status",
// workspace lifecycle and configuration
"bootstrap", "config apply", "config drift", "config get", "config list", "config set",
"config set-many", "config show", "config unset", "config validate", "context",
"init", "init-safety", "onboard", "preflight", "quickstart", "schema", "setup", "where",
"version", "worktree create", "worktree info", "worktree list", "worktree remove",
// storage, history and migration paths that are already routed or store-free
"compact", "events export", "events prune", "events tail",
"dolt clean-databases", "dolt killall", "dolt set", "dolt show", "dolt status",
"dolt start", "dolt stop", "dolt test",
"migrate from-proxied-server-to-server", "migrate from-proxied-server-to-shared-server",
"migrate from-server-to-proxied-server", "migrate from-shared-server-to-proxied-server",
"migrate legacy-sqlite", "migrate schema",
// formulas (parser-only), agent memory, swarm reads
"formula convert", "formula list", "formula schema", "formula show",
"mail", "memories", "ping", "prime", "recall", "remember",
"swarm status", "swarm validate",
// human-in-the-loop queue
"human", "human dismiss", "human list", "human respond", "human stats",
// trackers: the verbs with no proxied refusal of any kind
"ado pull", "ado push", "github pull", "github push", "gitlab pull", "gitlab push",
"jira pull", "jira push", "jira status", "jira sync",
"linear pull", "linear push", "linear status", "linear sync", "linear teams",
"notion pull", "notion push",
// hooks, metrics, serving and internal entry points
"hooks install", "hooks list", "hooks run", "hooks uninstall",
"metrics", "metrics example", "metrics off", "metrics on",
"serve", "sql",
"codex-hook", "cursor-hook", "db-proxy-child", "send-metrics",
}
func init() {
for _, path := range proxyPermittedPaths {
proxyCapabilityRegistry = append(proxyCapabilityRegistry, permitted(path))
}
proxyCapabilityIndex = make(map[string]capabilityRow, len(proxyCapabilityRegistry))
for _, row := range proxyCapabilityRegistry {
proxyCapabilityIndex[capabilityKey(row.Path, row.ArgSet)] = row
}
}
// proxyCapabilityIndex is proxyCapabilityRegistry keyed for lookup. Duplicate
// keys are rejected by TestProxyCapabilityRegistryHasNoDuplicateRows rather
// than by a panic here: a typo must fail the build, not every bd invocation.
var proxyCapabilityIndex map[string]capabilityRow
func capabilityKey(path, argSet string) string { return path + "\x00" + argSet }
// LookupCapabilityRow returns the row for an exact path/arg-set key.
func LookupCapabilityRow(path, argSet string) (capabilityRow, bool) {
row, ok := proxyCapabilityIndex[capabilityKey(path, argSet)]
return row, ok
}
// capabilityRowFor resolves a command to its row, preferring the row keyed on
// the flags the invocation actually set over the bare path.
//
// The fallback to the bare row is what lets one table serve both halves of the
// old policy: a path row refuses whatever flags were passed unless a more
// specific row says otherwise, so `backup sync --dry-run` is still refused by
// the `backup sync` row. The predecessor of the arg-set half matched exactly or
// not at all, which meant an unlisted flag combination on a refused path
// slipped through; no such combination exists among the commands that carry
// these flags, so nothing changes today, and the fallback is why none can
// appear later.
func capabilityRowFor(cmd *cobra.Command) (capabilityRow, bool) {
path := commandRegistryPath(cmd)
if argSet := commandPolicyArgSet(cmd); argSet != "" {
if row, ok := LookupCapabilityRow(path, argSet); ok {
return row, true
}
}
return LookupCapabilityRow(path, "")
}
// commandRegistryPath is the registry key for a command: its full path with the
// root name removed, for example "dolt remote add", "ready" for `bd ready`, and
// "" for the bare root.
//
// Both halves of the front door key on this rather than on cobra's Name(),
// which is only the leaf: two commands in different subtrees can share one, and
// `bd ready` and `bd mol ready --gated` do. A Name()-keyed rule silently applies
// to both, so a refusal written for one command lands on an unrelated command
// that no policy row authorizes.
func commandRegistryPath(cmd *cobra.Command) string {
if cmd == nil {
return ""
}
return strings.TrimSpace(strings.TrimPrefix(cmd.CommandPath(), cmd.Root().Name()))
}
// proxyPolicyArgFlags are the flags that take part in a capability key. Only
// flags that alter a command's execution belong here: output, tracing and other
// inherited root flags must never bypass a refusal by changing the key.
var proxyPolicyArgFlags = map[string]bool{
"auto-merge": true, "dry-run": true, "repair": true, "of": true, "with": true,
}
// commandPolicyArgSet renders the set flags that take part in the key, sorted so
// the key does not depend on the order the user typed them.
func commandPolicyArgSet(cmd *cobra.Command) string {
var args []string
cmd.Flags().Visit(func(f *pflag.Flag) {
if proxyPolicyArgFlags[f.Name] {
args = append(args, "--"+f.Name)
}
})
sort.Strings(args)
return strings.Join(args, " ")
}
// validateProxyRegistryBeforeProvider is the path-keyed half of the proxied
// front door: it rejects direct-only commands before migrations, auto-start or
// provider construction, so a refusal leaves the workspace untouched. The
// flag-keyed half is validateProxyCapabilitiesBeforeProvider.
//
// topology is the workspace's proxied shape (resolveProxiedTopology). It is a
// parameter rather than a lookup inside so the gate can be exercised for every
// shape without a workspace on disk.
func validateProxyRegistryBeforeProvider(cmd *cobra.Command, topology ProxyTopology) error {
if cmd == nil {
return nil
}
// `compact` is the one rule that a flag VALUE decides rather than a flag's
// presence, so it cannot be expressed as an arg-set row. `bd compact` and
// `bd rules compact` have no --dolt flag at all and are never refused here;
// `bd admin compact --dolt` has a proxied route and falls through to it.
if cmd.Name() == "compact" {
if cmd.Flags().Lookup("dolt") == nil {
return nil
}
if dolt, _ := cmd.Flags().GetBool("dolt"); dolt {
return nil
}
}
row, ok := capabilityRowFor(cmd)
if !ok {
return nil
}
rule := row.ruleFor(topology)
if rule.Outcome != ProxyOutcomeRefused {
return nil
}
return HandleProxyCapabilityError(proxyCapabilityErrorFor(rule))
}
func proxyCapabilityErrorFor(rule proxyCapabilityRule) *ProxyCapabilityError {
return &ProxyCapabilityError{
Code: rule.Code,
Message: rule.Message,
ExitCode: rule.ExitCode,
Mutates: rule.Mutates,
Reason: rule.Reason,
}
}
// HistoryCapabilityClass classifies history/version-control commands on the
// proxied topology without coupling them to a particular backend engine.
type HistoryCapabilityClass string
const (
HistoryProxySupported HistoryCapabilityClass = "proxy-supported"
HistoryDirectOnly HistoryCapabilityClass = "direct-only"
)
// LookupHistoryCapability classifies an exact command path. Paths with no
// history classification are intentionally absent so callers cannot
// accidentally advertise support.
func LookupHistoryCapability(commandPath string) (HistoryCapabilityClass, bool) {
row, ok := LookupCapabilityRow(commandPath, "")
if !ok || row.History == "" {
return "", false
}
return row.History, true
}

View file

@ -0,0 +1,143 @@
package main
import (
"encoding/json"
"maps"
"slices"
"strings"
"testing"
"github.com/spf13/cobra"
)
func TestHistoryCapabilityMatrixExactPaths(t *testing.T) {
if got, ok := LookupHistoryCapability("history"); !ok || got != HistoryProxySupported {
t.Fatalf("history = %q, %v; want proxy-supported", got, ok)
}
for _, path := range []string{"branch", "conflicts", "repo", "federation", "vc", "flatten", "dolt push", "dolt pull", "dolt commit", "dolt remote", "sync"} {
if got, ok := LookupHistoryCapability(path); !ok || got != HistoryDirectOnly {
t.Errorf("%q = %q, %v; want direct-only", path, got, ok)
}
}
if got, ok := LookupHistoryCapability("dolt remote remove"); !ok || got != HistoryProxySupported {
t.Fatalf("dolt remote remove = %q, %v; want proxy-supported", got, ok)
}
for _, path := range []string{"dolt remote add", "dolt remote list", "dolt remote reset-data"} {
if got, ok := LookupHistoryCapability(path); !ok || got != HistoryDirectOnly {
t.Errorf("%q = %q, %v; want direct-only", path, got, ok)
}
}
}
func TestSyncRefusesBeforeProvider(t *testing.T) {
root := &cobra.Command{Use: "bd"}
cmd := &cobra.Command{Use: "sync"}
root.AddCommand(cmd)
err := validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal)
if err == nil {
t.Fatalf("sync refusal = %#v", err)
}
if code, ok := exitCodeFromError(err); !ok || code != 1 {
t.Fatalf("sync exit = %#v", err)
}
}
func TestHistoryDirectOnlyRefusalContract(t *testing.T) {
expected := map[string]struct{ code, message string }{
"branch": {"proxy.branch.unsupported", "branch is not supported in proxied-server mode"},
"conflicts": {"proxy.conflicts.unsupported", "conflicts is not supported in proxied-server mode"},
"repo": {"proxy.repo.unsupported", "repo is not supported in proxied-server mode"},
"federation": {"proxy.federation.unsupported", "federation is not supported in proxied-server mode"},
"vc": {"proxy.vc.unsupported", "vc is not supported in proxied-server mode"},
"flatten": {"proxy.flatten.unsupported", "flatten is not supported in proxied-server mode"},
"dolt push": {"proxy.dolt_push.unsupported", "dolt push is not supported in proxied-server mode"},
"dolt pull": {"proxy.dolt_pull.unsupported", "dolt pull is not supported in proxied-server mode"},
"dolt commit": {"proxy.dolt_commit.unsupported", "dolt commit is not supported in proxied-server mode"},
"dolt remote add": {"proxy.dolt_remote.unsupported", "dolt remote add is not supported in proxied-server mode"},
"dolt remote list": {"proxy.dolt_remote.unsupported", "dolt remote list is not supported in proxied-server mode"},
"dolt remote reset-data": {"proxy.dolt_remote.unsupported", "dolt remote reset-data is not supported in proxied-server mode"},
"sync": {"proxy.sync.unsupported", "sync is not supported in proxied-server mode"},
}
oldProvider := uowProvider
oldJSON := jsonOutput
t.Cleanup(func() { uowProvider = oldProvider; jsonOutput = oldJSON })
// Driven off the expectation table itself: a second hand-written path list
// let "dolt remote list" and "dolt remote reset-data" sit in `expected`
// while nothing asserted them, which reads as coverage that is not there.
for _, path := range slices.Sorted(maps.Keys(expected)) {
parts := strings.Split(path, " ")
root := &cobra.Command{Use: "bd"}
cmd := &cobra.Command{Use: parts[0]}
root.AddCommand(cmd)
for _, part := range parts[1:] {
child := &cobra.Command{Use: part}
cmd.AddCommand(child)
cmd = child
}
uowProvider = nil
jsonOutput = true
out := captureStdout(t, func() error { _ = validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal); return nil })
var got map[string]any
want := expected[path]
if err := json.Unmarshal([]byte(out), &got); err != nil || got["code"] != want.code || got["error"] != want.message || got["mutates"] != false {
t.Fatalf("%s refusal = %q (%v)", path, out, err)
}
if uowProvider != nil {
t.Fatal("refusal initialized provider")
}
}
}
func TestHistoryNestedFrontDoorsRefuseAndSupportedPathsPass(t *testing.T) {
oldJSON := jsonOutput
jsonOutput = true
t.Cleanup(func() { jsonOutput = oldJSON })
expected := map[string]struct{ code, message string }{
"vc merge": {"proxy.vc.unsupported", "vc merge is not supported in proxied-server mode"},
"vc commit": {"proxy.vc.unsupported", "vc commit is not supported in proxied-server mode"},
"repo add": {"proxy.repo.unsupported", "repo add is not supported in proxied-server mode"},
"conflicts resolve": {"proxy.conflicts.unsupported", "conflicts resolve is not supported in proxied-server mode"},
"federation sync": {"proxy.federation.unsupported", "federation sync is not supported in proxied-server mode"},
}
for _, path := range []string{"vc merge", "vc commit", "repo add", "conflicts resolve", "federation sync", "dolt remote remove"} {
parts := strings.Split(path, " ")
root := &cobra.Command{Use: "bd"}
cmd := &cobra.Command{Use: parts[0]}
root.AddCommand(cmd)
for _, part := range parts[1:] {
child := &cobra.Command{Use: part}
cmd.AddCommand(child)
cmd = child
}
err := validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal)
if path == "dolt remote remove" {
if err != nil {
t.Fatalf("supported %s refused: %v", path, err)
}
continue
}
if err == nil {
t.Fatalf("direct-only %s unexpectedly allowed", path)
}
if code, ok := exitCodeFromError(err); !ok || code != 1 {
t.Fatalf("%s exit=%v, want 1", path, err)
}
var got map[string]any
// validateProxyRegistryBeforeProvider renders the typed refusal to
// stdout in JSON mode; the command must retain its nested path.
out := captureStdout(t, func() error { _ = validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal); return nil })
if err := json.Unmarshal([]byte(out), &got); err != nil {
t.Fatalf("%s refusal JSON: %v (%q)", path, err, out)
}
want := expected[path]
if got["code"] != want.code || got["error"] != want.message || got["mutates"] != false {
t.Fatalf("%s refusal = %#v, want code=%q message=%q mutates=false", path, got, want.code, want.message)
}
}
root := &cobra.Command{Use: "bd"}
history := &cobra.Command{Use: "history"}
root.AddCommand(history)
if err := validateProxyRegistryBeforeProvider(history, ProxyTopologyManagedLocal); err != nil {
t.Fatalf("history --events supported path refused: %v", err)
}
}

View file

@ -0,0 +1,423 @@
package main
import (
"errors"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"testing"
"github.com/spf13/cobra"
)
// cobraBuiltinCommands are added to the tree by cobra itself, not by bd, and
// only once something calls Execute — which another test in this package does
// (completions_test.go runs __complete). Without this set the completeness test
// below would pass or fail depending on test ORDER, which is the worst possible
// property for a policy gate.
var cobraBuiltinCommands = map[string]bool{
"help": true, "completion": true, "__complete": true, "__completeNoDesc": true,
}
// walkCommandPaths returns every command path in the tree, with the runnable
// ones flagged. Runnable is the property that matters: cobra returns ErrHelp
// for a non-runnable parent before PersistentPreRunE, so a group command never
// reaches the capability gate at all.
func walkCommandPaths(root *cobra.Command) map[string]bool {
paths := map[string]bool{}
var walk func(c *cobra.Command)
walk = func(c *cobra.Command) {
if cobraBuiltinCommands[c.Name()] {
return
}
if path := commandRegistryPath(c); path != "" {
paths[path] = c.Runnable()
}
for _, sub := range c.Commands() {
walk(sub)
}
}
walk(root)
return paths
}
// TestProxyCapabilityRegistryCoversCommandTree is the reason this registry
// exists. Every command that can actually run must state what the proxied front
// door does with it. A new command with no row fails here, at build time, rather
// than shipping and dying in the store factory with an untyped error — which is
// how every scattered refusal in this codebase got there.
//
// Adding a row is not a formality: choose the outcome deliberately.
// - the command works over the proxied provider, or never opens a store at
// all => add its path to proxyPermittedPaths
// - the command must not run on a shared/proxied store => add a refusedPath
// row with Reason=design and say why in a comment
// - the command could work but nobody has routed it => add a refusedPath row
// with Reason=unimplemented and a Tracking item
func TestProxyCapabilityRegistryCoversCommandTree(t *testing.T) {
var missing []string
for path, runnable := range walkCommandPaths(rootCmd) {
if !runnable {
continue
}
if _, ok := LookupCapabilityRow(path, ""); !ok {
missing = append(missing, path)
}
}
sort.Strings(missing)
if len(missing) > 0 {
t.Fatalf("%d command(s) have no proxied-server capability row:\n %s\n\n"+
"Add each to cmd/bd/capability_registry.go: proxyPermittedPaths if the proxied "+
"front door should let it through, or a refusedPath row with a Reason if it should not.",
len(missing), strings.Join(missing, "\n "))
}
}
// TestProxyCapabilityRegistryHasNoStaleRows is the other direction: a row whose
// command no longer exists is policy for nothing, and would hide the renamed
// command's own missing row behind a green test.
func TestProxyCapabilityRegistryHasNoStaleRows(t *testing.T) {
tree := walkCommandPaths(rootCmd)
var stale []string
for _, row := range proxyCapabilityRegistry {
if _, ok := tree[row.Path]; !ok {
stale = append(stale, row.display())
}
}
sort.Strings(stale)
if len(stale) > 0 {
t.Fatalf("registry rows for commands that do not exist:\n %s", strings.Join(stale, "\n "))
}
}
func TestProxyCapabilityRegistryHasNoDuplicateRows(t *testing.T) {
seen := map[string]bool{}
for _, row := range proxyCapabilityRegistry {
key := capabilityKey(row.Path, row.ArgSet)
if seen[key] {
t.Errorf("duplicate registry row for %q: one of them is dead and the lookup silently picks the last", row.display())
}
seen[key] = true
}
}
// TestProxyCapabilityRegistryRowInvariants pins the contract every row must
// satisfy. The Reason rules are the load-bearing ones: a refusal with no reason
// is exactly the tribal knowledge this registry replaces, and an unimplemented
// refusal with no tracking item is a gap nobody owns.
func TestProxyCapabilityRegistryRowInvariants(t *testing.T) {
for _, row := range proxyCapabilityRegistry {
assertCapabilityRuleInvariants(t, row.display(), row.Rule)
for topology, rule := range row.Topology {
assertCapabilityRuleInvariants(t, row.display()+" on "+string(topology), rule)
if topology == ProxyTopologyUnknown {
t.Errorf("%s: a row must not name the unknown topology; unknown exists so an unclassifiable "+
"workspace INHERITS the refusal, and exempting it would invert that", row.display())
}
}
if row.History == HistoryDirectOnly && row.Rule.Outcome != ProxyOutcomeRefused {
t.Errorf("%s: classified direct-only but the gate does not refuse it", row.display())
}
if row.History == HistoryProxySupported && row.Rule.Outcome != ProxyOutcomeHonored {
t.Errorf("%s: classified proxy-supported but the gate refuses it", row.display())
}
}
}
// assertCapabilityRuleInvariants checks one rule — a row's default or one of
// its per-topology overrides. Both have to satisfy the same contract: a
// topology override is policy a consumer sees, not a footnote.
func assertCapabilityRuleInvariants(t *testing.T, where string, rule proxyCapabilityRule) {
t.Helper()
switch rule.Outcome {
case ProxyOutcomeRefused:
if rule.Code == "" || rule.Message == "" || rule.ExitCode != 1 {
t.Errorf("%s: refusal must carry a code, a message and exit 1, got %#v", where, rule)
}
if rule.Mutates {
t.Errorf("%s: a refusal never mutates", where)
}
if !strings.HasPrefix(rule.Code, "proxy.") {
t.Errorf("%s: code %q must be in the stable proxy.* namespace", where, rule.Code)
}
case ProxyOutcomeRefusedInRunE:
if rule.Code != "" {
t.Errorf("%s: an untyped RunE refusal has no code; if it has one it belongs in the gate", where)
}
case ProxyOutcomeHonored:
if rule.Reason != "" {
t.Errorf("%s: Reason explains a refusal; an honored row must not carry one", where)
}
default:
t.Errorf("%s: unexpected outcome %q", where, rule.Outcome)
}
if rule.Outcome == ProxyOutcomeRefused || rule.Outcome == ProxyOutcomeRefusedInRunE {
switch rule.Reason {
case ProxyReasonDesign, ProxyReasonUnimplemented:
default:
t.Errorf("%s: refusal must declare Reason design or unimplemented, got %q", where, rule.Reason)
}
if rule.Reason == ProxyReasonUnimplemented && rule.Tracking == "" {
t.Errorf("%s: an unimplemented refusal needs a Tracking item naming who closes it", where)
}
}
}
// inlineProxiedRefusal matches the untyped refusals commands raise from their
// own RunE, e.g. HandleErrorRespectJSON("gitlab sync is not supported in
// proxied-server mode").
var inlineProxiedRefusal = regexp.MustCompile(`HandleErrorRespectJSON\("([^"]*proxied-server mode)"\)`)
// inlineProxiedMessages classifies the messages that do not name their command
// path in the standard "<path> is not supported ..." shape. A path maps the
// message to its registry row; an empty string records a message that mentions
// proxied mode without being a capability refusal at all. Keeping both explicit
// is what lets the scan below be exhaustive rather than best-effort.
var inlineProxiedMessages = map[string]string{
"only 'bd admin compact --dolt' is supported in proxied-server mode": "admin compact",
// Not a refusal: --database is the flag that only MAKES SENSE on a proxied
// workspace, so this fires on the direct topologies.
"--database (or a --db value naming a database) is only supported in proxied-server mode": "",
}
// TestProxyCapabilityRegistryCoversInlineRefusals scans the package for
// refusals raised inside RunE and requires each one to resolve to a registry
// row. Those strings are the untyped half of the refusal surface — no code, no
// mutates flag, nothing a JSON consumer can classify — and before the registry
// they were discoverable only by grep.
//
// The scan runs in both directions on purpose: a new inline refusal must be
// registered, and a ProxyOutcomeRefusedInRunE row whose string has been deleted
// must be flipped to its new outcome by the slice that deleted it.
func TestProxyCapabilityRegistryCoversInlineRefusals(t *testing.T) {
entries, err := os.ReadDir(".")
if err != nil {
t.Fatalf("read package dir: %v", err)
}
found := map[string]string{} // registry path -> file it was found in
for _, entry := range entries {
name := entry.Name()
if entry.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
continue
}
src, err := os.ReadFile(filepath.Clean(name))
if err != nil {
t.Fatalf("read %s: %v", name, err)
}
for _, match := range inlineProxiedRefusal.FindAllStringSubmatch(string(src), -1) {
message := match[1]
path, ok := inlineRefusalPath(message)
if !ok {
t.Errorf("%s: %q mentions proxied-server mode but does not name a command path; "+
"add it to inlineProxiedMessages so it can be checked against the registry "+
"(map it to \"\" if it is not a capability refusal)", name, message)
continue
}
if path == "" {
continue // documented non-refusal
}
found[path] = name
}
}
for path, file := range found {
argSet := ""
if base, args, split := strings.Cut(path, " --"); split {
argSet = "--" + args
path = base
}
row, ok := LookupCapabilityRow(path, argSet)
if !ok {
t.Errorf("%s refuses %q inside RunE with no registry row; the front door and the command disagree about what bd supports",
file, strings.TrimSpace(path+" "+argSet))
continue
}
if row.Rule.Outcome == ProxyOutcomeHonored {
t.Errorf("%s refuses %q inside RunE, but the registry says the front door honors it", file, row.display())
}
}
for _, row := range proxyCapabilityRegistry {
if row.Rule.Outcome != ProxyOutcomeRefusedInRunE {
continue
}
if _, ok := found[row.Path]; !ok {
t.Errorf("%s is recorded as an untyped RunE refusal but no such refusal exists in the package any more (%s); "+
"flip the row to its new outcome", row.display(), row.Note)
}
}
}
func inlineRefusalPath(message string) (string, bool) {
if path, ok := inlineProxiedMessages[message]; ok {
return path, true
}
path, ok := strings.CutSuffix(message, " is not supported in proxied-server mode")
if !ok {
return "", false
}
// conflicts.go says "bd conflicts ..."; the registry keys on the path.
return strings.TrimPrefix(path, "bd "), true
}
// TestProxyCapabilityRegistryReasonsAreAssignedIndividually guards the one way
// this registry could become useless while staying green: reasons assigned in
// bulk. Every later slice reads these to decide what is left to do, so both
// values have to be genuinely represented, and the design refusals — the claim
// a reviewer will challenge — are listed here by name.
func TestProxyCapabilityRegistryReasonsAreAssignedIndividually(t *testing.T) {
wantDesign := map[string]bool{
"branch": true, "diff": true, "flatten": true,
"conflicts": true, "conflicts list": true, "conflicts show": true, "conflicts resolve": true,
"vc": true, "vc merge": true, "vc commit": true, "vc status": true,
"federation": true, "federation sync": true, "federation status": true,
"federation add-peer": true, "federation remove-peer": true, "federation list-peers": true,
"repo": true, "repo add": true, "repo remove": true, "repo list": true, "repo sync": true,
// `migrate hooks` is deliberately NOT here: it migrates git hook files
// and opens no store, so the surgery rationale the rest of this block
// rests on does not describe it. Its refusal is structural — the
// command is on none of main.go's store-init skip lists — which makes
// it unimplemented with a tracking item, not design.
"migrate": true, "migrate sync": true, "migrate issues": true,
"migrate-issues": true, "migrate-personal": true,
"admin cleanup": true, "admin reset": true,
// The backup family's DEFAULT rule is the design refusal; managed-local
// overrides it to honored (TestProxiedBackupHonoredOnlyOnManagedLocal).
// It is design because a backup remote is registered on the SERVER,
// where it is global to every client of that server — true of every
// destination scheme, unlike the filesystem half, which is a file://
// property only. These rows carry a Tracking item anyway
// (backupRemoteSchemeTracking): the outcome is settled, the
// remote-scheme case is an open question and must not read as closed.
"backup": true, "backup init": true, "backup sync": true,
"backup remove": true, "backup status": true, "backup restore": true,
}
gotDesign := map[string]bool{}
unimplemented := 0
for _, row := range proxyCapabilityRegistry {
switch row.Rule.Reason {
case ProxyReasonDesign:
gotDesign[row.display()] = true
case ProxyReasonUnimplemented:
unimplemented++
}
}
for path := range wantDesign {
if !gotDesign[path] {
t.Errorf("%q is no longer refused by design; if that is intended, update this test and say why in the PR", path)
}
}
for path := range gotDesign {
if !wantDesign[path] {
t.Errorf("%q claims Reason=design and is not in the reviewed list; a design refusal is a semantic claim that needs review", path)
}
}
if unimplemented == 0 {
t.Error("no unimplemented refusals left — either the work is done or the reasons were bulk-assigned")
}
}
// TestProxyCapabilityArgSetFallsBackToBarePath pins the lookup rule that lets
// one table carry both the path policy and the flag policy: a flag combination
// with no row of its own inherits the bare path's verdict rather than escaping
// it. Written against the real command tree so it exercises the flags these
// commands actually declare.
func TestProxyCapabilityArgSetFallsBackToBarePath(t *testing.T) {
for _, tc := range []struct {
path string
flags []string
want ProxyCapabilityOutcome
wantMsg string
}{
// Covered by an explicit row: the message names the flags.
{"rename-prefix", []string{"dry-run"}, ProxyOutcomeRefused, "rename-prefix --dry-run is not supported in proxied-server mode"},
// No row for this combination: it inherits the refused bare row rather
// than slipping past the gate.
{"rename-prefix", []string{"dry-run", "auto-merge"}, ProxyOutcomeRefused, "rename-prefix is not supported in proxied-server mode"},
// Inheriting an honored bare row is equally load-bearing: this is the
// read-only preview and it must keep working.
{"duplicates", []string{"dry-run"}, ProxyOutcomeHonored, ""},
} {
t.Run(tc.path+" "+strings.Join(tc.flags, " "), func(t *testing.T) {
root := &cobra.Command{Use: "bd"}
cmd := &cobra.Command{Use: tc.path}
root.AddCommand(cmd)
for _, flag := range tc.flags {
cmd.Flags().Bool(flag, false, "")
if err := cmd.Flags().Set(flag, "true"); err != nil {
t.Fatal(err)
}
}
row, ok := capabilityRowFor(cmd)
if !ok {
t.Fatalf("no row resolved for %q with %v", tc.path, tc.flags)
}
if row.Rule.Outcome != tc.want || row.Rule.Message != tc.wantMsg {
t.Fatalf("row = outcome %q message %q, want %q / %q",
row.Rule.Outcome, row.Rule.Message, tc.want, tc.wantMsg)
}
})
}
}
// TestProxyRefusalJSONCarriesReason pins the additive half of the JSON
// contract. {code, error, mutates} is frozen and parsed downstream; reason is
// new, and it must appear without disturbing the three fields that were already
// there.
func TestProxyRefusalJSONCarriesReason(t *testing.T) {
t.Setenv("BD_JSON_ENVELOPE", "")
oldJSON := jsonOutput
jsonOutput = true
t.Cleanup(func() { jsonOutput = oldJSON })
for _, tc := range []struct {
path, code string
reason ProxyRefusalReason
}{
{"branch", "proxy.branch.unsupported", ProxyReasonDesign},
{"sync", "proxy.sync.unsupported", ProxyReasonUnimplemented},
} {
t.Run(tc.path, func(t *testing.T) {
root := &cobra.Command{Use: "bd"}
cmd := &cobra.Command{Use: tc.path}
root.AddCommand(cmd)
out := captureStdout(t, func() error {
_ = validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal)
return nil
})
for _, want := range []string{
`"code": "` + tc.code + `"`,
`"error": "` + tc.path + ` is not supported in proxied-server mode"`,
`"mutates": false`,
`"reason": "` + string(tc.reason) + `"`,
} {
if !strings.Contains(out, want) {
t.Fatalf("refusal JSON missing %s:\n%s", want, out)
}
}
})
}
}
// TestProxiedStoreUnroutedIsTyped covers the backstop: a permitted command with
// no proxied route used to die with a bare string that no consumer could tell
// from a real store failure.
func TestProxiedStoreUnroutedIsTyped(t *testing.T) {
var typed *ProxyCapabilityError
err := errProxiedStoreUnrouted()
if !errors.As(err, &typed) {
t.Fatalf("store backstop error = %#v, want *ProxyCapabilityError", err)
}
if typed.Code != "proxy.store.unrouted" || typed.ExitCode != 1 || typed.Mutates {
t.Fatalf("store backstop = %#v", typed)
}
if typed.Reason != ProxyReasonUnimplemented {
t.Fatalf("store backstop reason = %q, want unimplemented", typed.Reason)
}
}

View file

@ -0,0 +1,188 @@
package main
import (
"bytes"
"encoding/json"
"errors"
"os"
"strings"
"testing"
"github.com/spf13/cobra"
)
func transformTestCommand(path string, flags ...string) *cobra.Command {
root := &cobra.Command{Use: "bd"}
cmd := &cobra.Command{Use: path}
for _, flag := range flags {
if flag == "of" || flag == "with" {
cmd.Flags().String(flag, "", "")
} else {
cmd.Flags().Bool(flag, false, "")
}
}
root.AddCommand(cmd)
return cmd
}
func TestProxyTransformCapabilityRows(t *testing.T) {
cases := []struct {
name, path, argument string
flags []string
outcome ProxyCapabilityOutcome
code, message string
exit int
mutates bool
}{
{name: "rename", path: "rename", outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "rename is not supported in proxied-server mode", exit: 1},
{name: "rename-prefix", path: "rename-prefix", outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "rename-prefix is not supported in proxied-server mode", exit: 1},
{name: "rename-prefix dry-run", path: "rename-prefix", argument: "--dry-run", flags: []string{"dry-run"}, outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "rename-prefix --dry-run is not supported in proxied-server mode", exit: 1},
{name: "rename-prefix repair", path: "rename-prefix", argument: "--repair", flags: []string{"repair"}, outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "rename-prefix --repair is not supported in proxied-server mode", exit: 1},
{name: "rename-prefix repair dry-run", path: "rename-prefix", argument: "--dry-run --repair", flags: []string{"dry-run", "repair"}, outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "rename-prefix --dry-run --repair is not supported in proxied-server mode", exit: 1},
{name: "duplicate", path: "duplicate", outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "duplicate is not supported in proxied-server mode", exit: 1},
{name: "duplicate of", path: "duplicate", argument: "--of", flags: []string{"of"}, outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "duplicate --of is not supported in proxied-server mode", exit: 1},
{name: "supersede", path: "supersede", outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "supersede is not supported in proxied-server mode", exit: 1},
{name: "supersede with", path: "supersede", argument: "--with", flags: []string{"with"}, outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "supersede --with is not supported in proxied-server mode", exit: 1},
{name: "duplicates", path: "duplicates", flags: []string{"auto-merge", "dry-run"}, outcome: ProxyOutcomeHonored},
{name: "duplicates auto-merge", path: "duplicates", argument: "--auto-merge", flags: []string{"auto-merge", "dry-run"}, outcome: ProxyOutcomeRefused, code: "proxy.transform.unsupported", message: "duplicates --auto-merge is not supported in proxied-server mode", exit: 1},
{name: "duplicates auto-merge dry-run", path: "duplicates", argument: "--auto-merge --dry-run", flags: []string{"auto-merge", "dry-run"}, outcome: ProxyOutcomeHonored},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
cmd := transformTestCommand(tc.path, tc.flags...)
if tc.argument != "" {
for _, flag := range strings.Fields(tc.argument) {
if err := cmd.Flags().Set(strings.TrimPrefix(flag, "--"), "true"); err != nil {
t.Fatal(err)
}
}
}
row, ok := capabilityRowFor(cmd)
if !ok {
t.Fatalf("lookup(%q, %q) missing row", tc.path, tc.argument)
}
if row.Path != tc.path || row.ArgSet != tc.argument || row.Rule.Outcome != tc.outcome {
t.Fatalf("row = %#v, want path=%q argument=%q outcome=%q", row, tc.path, tc.argument, tc.outcome)
}
if tc.outcome == ProxyOutcomeRefused {
if row.Rule.Code != tc.code || row.Rule.Message != tc.message || row.Rule.ExitCode != tc.exit || row.Rule.Mutates != tc.mutates {
t.Fatalf("refusal fields = code=%q message=%q exit=%d mutates=%t",
row.Rule.Code, row.Rule.Message, row.Rule.ExitCode, row.Rule.Mutates)
}
var typed error = proxyCapabilityErrorFor(row.Rule)
var got *ProxyCapabilityError
if !errors.As(typed, &got) || got.Code != tc.code || got.ExitCode != 1 || got.Mutates {
t.Fatalf("typed refusal = %#v", typed)
}
}
if err := validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal); tc.outcome == ProxyOutcomeHonored && err != nil {
t.Fatalf("honored transform refused: %v", err)
}
})
}
}
func TestProxyDuplicatesAutoMergeDryRunMatrix(t *testing.T) {
cmd := transformTestCommand("duplicates", "auto-merge", "dry-run")
_ = cmd.Flags().Set("auto-merge", "true")
if err := validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal); err == nil {
t.Fatalf("non-dry auto-merge error = %v", err)
}
_ = cmd.Flags().Set("dry-run", "true")
if err := validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal); err != nil {
t.Fatalf("dry-run auto-merge refused: %v", err)
}
}
func TestProxyTransformRefusalRendering(t *testing.T) {
t.Setenv("BD_JSON_ENVELOPE", "")
cmd := transformTestCommand("rename")
oldJSON := jsonOutput
t.Cleanup(func() { jsonOutput = oldJSON })
jsonOutput = false
var textErr string
stderr := captureStderr(t, func() {
err := validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal)
if code, ok := exitCodeFromError(err); !ok || code != 1 {
t.Fatalf("exit = %v, want 1", err)
}
textErr = err.Error()
})
if textErr != "exit code 1" || stderr != "Error: rename is not supported in proxied-server mode\n" {
t.Fatalf("text refusal = %q (err %q)", stderr, textErr)
}
jsonOutput = true
stdout := captureStdout(t, func() error {
_ = validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal)
return nil
})
var envelope map[string]any
if err := json.Unmarshal([]byte(stdout), &envelope); err != nil {
t.Fatalf("JSON refusal: %v (%q)", err, stdout)
}
if envelope["code"] != "proxy.transform.unsupported" || envelope["error"] != "rename is not supported in proxied-server mode" {
t.Fatalf("JSON refusal = %#v", envelope)
}
}
func TestProxyTransformMatcherIgnoresInheritedOutputFlags(t *testing.T) {
root := &cobra.Command{Use: "bd"}
root.PersistentFlags().Bool("json", false, "output JSON")
cmd := &cobra.Command{Use: "duplicates"}
cmd.Flags().Bool("auto-merge", false, "")
cmd.Flags().Bool("dry-run", false, "")
root.AddCommand(cmd)
if err := cmd.Flags().Set("auto-merge", "true"); err != nil {
t.Fatal(err)
}
if err := root.PersistentFlags().Set("json", "true"); err != nil {
t.Fatal(err)
}
row, ok := capabilityRowFor(cmd)
if !ok || row.ArgSet != "--auto-merge" || row.Rule.Outcome != ProxyOutcomeRefused {
t.Fatalf("row with inherited --json = %#v, ok=%v", row, ok)
}
}
func TestProxyTransformRefusalHasNoProviderOrFileMutation(t *testing.T) {
dir := t.TempDir()
version := []byte("version-before\n")
issues := []byte(`{"id":"before","title":"untouched"}` + "\n")
versionPath := dir + "/.local_version"
issuesPath := dir + "/issues.jsonl"
if err := os.WriteFile(versionPath, version, 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(issuesPath, issues, 0o600); err != nil {
t.Fatal(err)
}
oldProvider := uowProvider
uowProvider = nil
t.Cleanup(func() { uowProvider = oldProvider })
cmd := transformTestCommand("supersede")
err := validateProxyRegistryBeforeProvider(cmd, ProxyTopologyManagedLocal)
if err == nil {
t.Fatal("expected transform refusal")
}
if code, ok := exitCodeFromError(err); !ok || code != 1 {
t.Fatalf("exit = %v, want 1", err)
}
if uowProvider != nil {
t.Fatal("transform refusal initialized a provider")
}
gotVersion, err := os.ReadFile(versionPath)
if err != nil {
t.Fatal(err)
}
gotIssues, err := os.ReadFile(issuesPath)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(gotVersion, version) || !bytes.Equal(gotIssues, issues) {
t.Fatal("transform refusal mutated workspace files")
}
}

View file

@ -49,7 +49,7 @@ the flags appear in the command line.`,
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
CheckReadonly("close") // also covers CheckMigrationFreeze (dc-6jaq)
CheckReadonly("close") // also covers the migration freeze check (dc-6jaq)
evt := metrics.NewCommandEvent("close")
defer func() {

View file

@ -87,7 +87,10 @@ Examples:
if compactDolt {
return runCompactDoltProxiedServer(rootCtx)
}
return HandleErrorRespectJSON("only 'compact --dolt' is supported in proxied-server mode")
// Same string the "admin compact" refusal row carries; the
// pre-provider gate short-circuits this path, so the two are kept
// identical rather than left to drift.
return HandleErrorRespectJSON("only 'bd admin compact --dolt' is supported in proxied-server mode")
}
// Block mutating operations in embedded mode; allow --stats, --analyze, --dry-run read-only paths.
if !compactStats && !compactAnalyze && !compactDryRun {

View file

@ -35,7 +35,13 @@ How it works:
4. Swaps main branch to the compacted version
5. Prunes remote-tracking refs (they would keep the old history alive;
the next push or fetch re-creates them at the new tip)
6. Runs Dolt GC to reclaim space
6. Runs a full Dolt GC (all storage generations) to reclaim the old history
The GC pass is a full collection: Dolt storage is generational, and a default
GC never revisits data an earlier GC moved to the old generation. On any store
that has been GC'd before (bd gc, or a previous flatten or compact), only a
full collection reclaims the squashed history. A full GC can take minutes on
multi-gigabyte stores.
Examples:
bd compact --dry-run # Preview: show commit breakdown
@ -187,11 +193,7 @@ Examples:
}
// Reclaim disk space from orphaned old history
if gc, ok := storage.UnwrapStore(store).(storage.GarbageCollector); ok {
if err := gc.DoltGC(ctx); err != nil {
WarnError("dolt gc after compact failed: %v", err)
}
}
gcMode := runPostRewriteGC(ctx, "compact")
sizeAfter := storeSizeBytes(ctx)
elapsed := time.Since(start)
@ -208,6 +210,9 @@ Examples:
"tags_anchoring": tags,
"elapsed_ms": elapsed.Milliseconds(),
}
if gcMode != "" {
result["gc_mode"] = gcMode
}
addGCSizeJSON(result, sizeBefore, sizeAfter)
return outputJSON(result)
}

View file

@ -143,7 +143,13 @@ func runCompactProxiedServer(ctx context.Context) error {
if tags, perr = versioncontrolops.ListTags(ctx, conn); perr != nil {
WarnError("listing tags before GC: %v", perr)
}
if perr := versioncontrolops.DoltGC(ctx, conn); perr != nil {
// Full pass: the squash orphans a commit chain that any earlier GC
// already promoted to the old generation, which a default pass never
// revisits.
if !jsonOutput {
fmt.Println(" Running full Dolt GC (all generations; can take minutes on large stores)...")
}
if perr := versioncontrolops.DoltGCFull(ctx, conn); perr != nil {
WarnError("dolt gc after compact failed: %v", perr)
}
return nil
@ -167,6 +173,7 @@ func runCompactProxiedServer(ctx context.Context) error {
"recent_kept": recentCommits,
"remote_refs_pruned": pruned,
"tags_anchoring": tags,
"gc_mode": gcModeFull,
"elapsed_ms": elapsed.Milliseconds(),
})
}

View file

@ -645,10 +645,14 @@ var configUnsetCmd = &cobra.Command{
// specific false claim than the message this fix removed, and, with no
// project config.yaml, failed the command after the database row was
// already gone.
//
// A config.yaml the unset refuses to edit (a flow-style mapping, a
// block value) is reported as exactly that: the row is already gone,
// but the key is still effective from the file, so this still fails.
location := "database"
yamlCleared, err := config.UnsetYamlConfig(result.Key)
if err != nil {
return HandleError("deleting config from config.yaml: %v", err)
return HandleError("unset %s in the database, but it is still set in config.yaml: %v", result.Key, err)
}
if yamlCleared {
location = "database, config.yaml"

View file

@ -138,13 +138,18 @@ func runConcludeCommand(t *testing.T, commandStore storage.DoltStorage, asJSON b
defer func() { os.Stdout, os.Stderr = oldStdout, oldStderr }()
store, rootCtx, jsonOutput = commandStore, context.Background(), asJSON
conflictsConclude = true
// Deferred, like the stdio restore above it: RunE below can panic or
// t.Fatal, and a plain statement after the call would then leave jsonOutput
// set to asJSON — which callers pass as true — for the rest of the package.
defer func() {
store, rootCtx, jsonOutput = oldStore, oldRootCtx, oldJSON
conflictsConclude = oldConclude
}()
commandErr := conflictsResolveCmd.RunE(conflictsResolveCmd, nil)
_ = wOut.Close()
_ = wErr.Close()
store, rootCtx, jsonOutput = oldStore, oldRootCtx, oldJSON
conflictsConclude = oldConclude
var stdout, stderr bytes.Buffer
_, _ = stdout.ReadFrom(rOut)

View file

@ -52,7 +52,7 @@ var createCmd = &cobra.Command{
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
CheckReadonly("create") // also covers CheckMigrationFreeze (dc-6jaq)
CheckReadonly("create") // also covers the migration freeze check (dc-6jaq)
evt := metrics.NewCommandEvent("create")
defer func() {
@ -197,33 +197,10 @@ var createCmd = &cobra.Command{
return HandleError("--ephemeral and --no-history are mutually exclusive")
}
storageClassFlag, _ := cmd.Flags().GetString("storage-class")
storageClass, err := resolveStorageClass(storageClassFlag, types.IssueType(issueType).Normalize())
storageClass, wisp, err := resolveCreateStorageClass(storageClassFlag, types.IssueType(issueType), wisp, noHistory)
if err != nil {
return HandleError("%v", err)
}
// --storage-class ephemeral is the spelled-out spelling of --ephemeral
// (Protocol v0.1 C1.4: the wisp plane is today's ephemeral-class
// implementation). It routes to the wisp path exactly like the flag;
// the --no-history mutual exclusion above still applies.
if storageClass == types.StorageClassEphemeral {
if noHistory {
return HandleError("--storage-class ephemeral and --no-history are mutually exclusive")
}
wisp = true
storageClass = "" // wisp-plane rows derive ephemeral class (C1.2); no marker cell needed
}
// Reconcile the requested durable class with the effective wisp plane
// (flag > config, Protocol v0.1 §C1.3): an explicit --storage-class
// contradicts --ephemeral/--no-history and is rejected, so the durable
// intent is preserved rather than silently collapsed into an
// effective-ephemeral record; a per-type config default yields to the
// explicit flag. versioned normalizes to the unset marker only after the
// check (C2.4).
var storageClassConflict bool
storageClass, storageClassConflict = reconcileStorageClassPlane(storageClass, storageClassFlag != "", wisp || noHistory)
if storageClassConflict {
return HandleError("--storage-class %s conflicts with --ephemeral/--no-history: wisp-plane records are storage class ephemeral", storageClass)
}
molTypeStr, _ := cmd.Flags().GetString("mol-type")
var molType types.MolType
if molTypeStr != "" {
@ -759,6 +736,49 @@ func reconcileStorageClassPlane(class types.StorageClass, explicit, wispPlane bo
return class.Normalize(), false
}
// resolveCreateStorageClass is the ONE create-time storage-class decision every
// `bd create` CLI door makes: parse the flag (or fall back to the per-type
// storage-class.<type> config default), route the ephemeral spelling to the wisp
// plane, and reject a durable class that contradicts the plane. Every door goes
// through it so the direct, proxied, and --file routes cannot answer differently;
// graphApplyNodeStorageClass is the per-node mirror, which additionally honors a
// node's own pointer overrides.
//
// flagValue is the raw --storage-class spelling ("" when unset). wisp/noHistory
// are the plane flags as the caller read them; the returned wisp is the plane
// AFTER the ephemeral spelling has been folded in (Protocol v0.1 C1.4). The
// returned class is already normalized (C2.4), so it can go straight onto the
// issue.
func resolveCreateStorageClass(flagValue string, issueType types.IssueType, wisp, noHistory bool) (types.StorageClass, bool, error) {
class, err := resolveStorageClass(flagValue, issueType.Normalize())
if err != nil {
return "", wisp, err
}
// --storage-class ephemeral is the spelled-out spelling of --ephemeral
// (Protocol v0.1 C1.4: the wisp plane is today's ephemeral-class
// implementation). It routes to the wisp path exactly like the flag; the
// --ephemeral/--no-history mutual exclusion still applies.
if class == types.StorageClassEphemeral {
if noHistory {
return "", wisp, errors.New("--storage-class ephemeral and --no-history are mutually exclusive")
}
wisp = true
class = "" // wisp-plane rows derive ephemeral class (C1.2); no marker cell needed
}
// Reconcile the requested durable class with the effective wisp plane
// (flag > config, Protocol v0.1 §C1.3): an explicit --storage-class
// contradicts --ephemeral/--no-history and is rejected, so the durable
// intent is preserved rather than silently collapsed into an
// effective-ephemeral record; a per-type config default yields to the
// explicit flag. versioned normalizes to the unset marker only after the
// check (C2.4).
class, conflict := reconcileStorageClassPlane(class, flagValue != "", wisp || noHistory)
if conflict {
return "", wisp, fmt.Errorf("--storage-class %s conflicts with --ephemeral/--no-history: wisp-plane records are storage class ephemeral", class)
}
return class, wisp, nil
}
func buildCreateIssue(params createIssueParams) *types.Issue {
var externalRefPtr *string
if params.ExternalRef != "" {

View file

@ -46,6 +46,8 @@ type createInput struct {
validate bool
ephemeral bool
noHistory bool
storageClass types.StorageClass
storageClassFlag string
molType types.MolType
wispType types.WispType
eventCategory string
@ -180,6 +182,21 @@ func gatherCreateInput(cmd *cobra.Command, args []string) (createInput, error) {
in.priority = priority
in.issueType, _ = cmd.Flags().GetString("type")
// The raw flag rides along for the --file batch, which resolves the class
// per template (the per-type config default needs a type to key on, and only
// a template has one). --graph rejects the flag outright, so the only route
// that resolves it here is the single-issue create, on both transports.
in.storageClassFlag, _ = cmd.Flags().GetString("storage-class")
if in.markdownFile == "" && in.graphFile == "" {
class, wisp, err := resolveCreateStorageClass(in.storageClassFlag, types.IssueType(in.issueType), in.ephemeral, in.noHistory)
if err != nil {
return in, HandleError("%v", err)
}
in.storageClass = class
in.ephemeral = wisp
}
in.status, _ = cmd.Flags().GetString("status")
in.assignee, _ = cmd.Flags().GetString("assignee")
in.externalRef, _ = cmd.Flags().GetString("external-ref")
@ -334,6 +351,12 @@ func rejectSingleIssueFlagsForGraph(cmd *cobra.Command) error {
if cmd.Flags().Changed("mol-type") {
return HandleError("--mol-type is not valid with --graph (set mol_type per node in the plan instead)")
}
// Same shape as --mol-type: a plan-wide class would duplicate the per-node
// storage_class field (plus its per-type config default) that graph-apply
// already honors, so the flag is refused rather than accepted and ignored.
if cmd.Flags().Changed("storage-class") {
return HandleError("--storage-class is not valid with --graph (set storage_class per node in the plan instead)")
}
return nil
}

View file

@ -30,7 +30,11 @@ func resolveProxiedCustomTypes(dbTypes []string) []string {
func runCreateProxiedServer(cmd *cobra.Command, ctx context.Context, in createInput) error {
if in.repoOverrideSet {
return HandleError("--repo is not supported with --proxied-server")
// Defense in depth: validateProxyCapabilitiesBeforeProvider already
// refuses `create --repo` before this route is reachable. Typed anyway,
// so the day a command slips past the gate the refusal is still the
// same shape rather than silently degrading to prose.
return HandleProxyCapabilityError(AssertProxyCommandCapability("create", ProxyModeProxied, ProxyCapRepo))
}
switch {
case in.graphFile != "":
@ -226,6 +230,7 @@ func buildCreateIssueFromInput(in createInput) *types.Issue {
EstimatedMinutes: in.estimatedMinutes,
Ephemeral: in.ephemeral,
NoHistory: in.noHistory,
StorageClass: in.storageClass,
CreatedBy: in.createdBy,
Owner: in.owner,
MolType: in.molType,

View file

@ -0,0 +1,121 @@
//go:build cgo
package main
import (
"encoding/json"
"os"
"strings"
"testing"
"time"
)
// TestEmbeddedCreateJSONEchoesSubSecondTimestamps pins that `bd create --json`
// prints the instant the row was written, fraction included.
//
// created_at is a DATETIME(0) column, so anything that renders a create by
// re-reading the row it just wrote prints whole seconds. `bd create --json` has
// never done that: it echoes the stamp the write carried, and agents that create
// several issues in a burst order them by exactly that fraction. A build that
// echoed the re-read made every same-second create tie.
//
// THE ASSERTION IS THAT A FRACTION IS THERE AT ALL, across two creates. A
// truncating build prints a whole second every time, so it fails on the first
// one; a correct build prints a whole second only when the clock lands exactly
// on a second boundary, which is a billion-to-one per create and so a billion
// billion-to-one across the pair. Note that "two creates tie" is NOT usable as
// the assertion here even though it is the property that matters: each create is
// a subprocess that boots embedded Dolt, so consecutive runs are seconds apart
// and would not tie even when truncated. The executor-level proof of the tie
// lives in conformance.RunLifecycleCreateEchoesSubSecondTimestamps.
func TestEmbeddedCreateJSONEchoesSubSecondTimestamps(t *testing.T) {
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt create tests")
}
t.Parallel()
bd := buildEmbeddedBD(t)
dir, _, _ := bdInit(t, bd, "--prefix", "cts")
first := createdAtEcho(t, bd, dir, "first issue")
second := createdAtEcho(t, bd, dir, "second issue")
firstAt := parseRFC3339Nano(t, "first", first.createdAt)
parseRFC3339Nano(t, "second", second.createdAt)
fractional := false
for _, echo := range []createEcho{first, second} {
for _, member := range []struct{ name, value string }{
{"created_at", echo.createdAt},
{"updated_at", echo.updatedAt},
} {
at := parseRFC3339Nano(t, echo.id+" "+member.name, member.value)
if at.Nanosecond() != 0 {
fractional = true
}
}
}
if !fractional {
t.Errorf("no timestamp `bd create --json` echoed across two creates carries a sub-second part "+
"(%s created_at=%q updated_at=%q, %s created_at=%q updated_at=%q); the echo is being rounded to the "+
"DATETIME(0) column, which is what makes same-second creates unorderable",
first.id, first.createdAt, first.updatedAt, second.id, second.createdAt, second.updatedAt)
}
// The echo still names the row it created. It is not the SAME text: the
// column is DATETIME(0) and the engine ROUNDS to it rather than truncating,
// so a read can land up to half a second either side of the echo. That skew
// is v1.2.2 behavior and stays — `bd show` and `bd list` answer whole
// seconds, as they always have. What must hold is that the two name one
// instant, which a second of slack states without pinning either the digits
// or the rounding mode.
shown := bdShow(t, bd, dir, first.id)
if skew := shown.CreatedAt.UTC().Sub(firstAt.UTC()); skew > time.Second || skew < -time.Second {
t.Errorf("%s: create echoed created_at %v but show reports %v (%v apart); the two must name the same instant, "+
"give or take the second-granular column",
first.id, firstAt, shown.CreatedAt.UTC(), skew)
}
}
type createEcho struct {
id string
createdAt string
updatedAt string
}
// createdAtEcho runs `bd create --json` and reads the timestamps as the RAW
// wire strings, because the defect is about the text a consumer parses: a
// decode into time.Time would accept both spellings silently.
func createdAtEcho(t *testing.T, bd, dir, title string) createEcho {
t.Helper()
out, err := bdRunWithFlockRetry(t, bd, dir, "create", "--json", title)
if err != nil {
t.Fatalf("bd create --json %q: %v\n%s", title, err, out)
}
s := string(out)
start := strings.Index(s, "{")
if start < 0 {
t.Fatalf("no JSON object in `bd create --json` output:\n%s", s)
}
var raw struct {
ID string `json:"id"`
CreatedAt string `json:"created_at"`
UpdatedAt string `json:"updated_at"`
}
if err := json.NewDecoder(strings.NewReader(s[start:])).Decode(&raw); err != nil {
t.Fatalf("parse `bd create --json` output: %v\nraw: %s", err, s[start:])
}
if raw.ID == "" {
t.Fatalf("`bd create --json` echoed no id:\n%s", s)
}
return createEcho{id: raw.ID, createdAt: raw.CreatedAt, updatedAt: raw.UpdatedAt}
}
func parseRFC3339Nano(t *testing.T, label, value string) time.Time {
t.Helper()
parsed, err := time.Parse(time.RFC3339Nano, value)
if err != nil {
t.Fatalf("%s created_at %q does not parse as RFC3339Nano: %v", label, value, err)
}
return parsed
}

View file

@ -364,6 +364,14 @@ func deleteBatch(_ *cobra.Command, issueIDs []string, force bool, dryRun bool, c
commandDidWrite.Store(true)
// Tell this command's own auto-export that these ids are supposed to be
// gone. Without it the export's orphan guard sees them as JSONL-only
// records absent from the store, refuses to overwrite, and never saves
// state — which is the permanent wedge of GH#5896. Recorded for the
// requested ids; anything cascade removed alongside them is proven later
// by the store's own history instead.
commandDeletedIssueIDs.add(resolvedIDs...)
// NO COMMIT COMPENSATION HERE, for the reason the single-id path gives:
// the role versions the deletion itself, on the store the rows were
// deleted from, and defers it in batch and off modes.

View file

@ -212,17 +212,15 @@ Examples:
if err != nil {
return HandleError("failed to resolve path: %v", err)
}
if err := checkDoctorMutationGate(absPath); err != nil {
return err
}
if err := validateDoctorWorkspaceBackend(absPath); isLegacyUpgradeRefusal(err) {
return printLegacyUpgradeDiagnostic(err)
} else if err != nil {
return HandleError("%v", err)
}
if usesProxiedServer() {
fmt.Fprintln(os.Stderr, "Note: 'bd doctor' is not yet supported in proxied-server mode.")
return nil
}
if doctorFix && isOrchestratorRoot(absPath) {
return HandleErrorWithHint(
"refusing to run 'bd doctor --fix' at orchestrator workspace root",
@ -354,6 +352,57 @@ func init() {
doctorCmd.Flags().BoolVar(&doctorAgent, "agent", false, "Agent-facing diagnostic mode: rich context for AI agents (ZFC-compliant)")
}
// doctorMutationOp returns the operation label for a doctor invocation that
// will mutate the workspace, or "" when this run is diagnosis-only. --fix and
// --clean are the complete trigger set: every mutating doctor surface
// (applyFixes, applyFixesInteractive, applyValidateFixes, the pollution and
// artifacts cleaners) is reached only through one of those two flags.
func doctorMutationOp() string {
switch {
case doctorFix:
return "doctor --fix"
case doctorClean:
return "doctor --clean"
}
return ""
}
// checkDoctorMutationGate is doctor's stand-in for the CheckReadonly call every
// other write command makes at the top of its RunE (#6028). Doctor never made
// that call, and its skipStoreAnnotation opt-out also skips the root
// PersistentPreRunE's freeze gate, so both `--readonly` and an active
// MIGRATION-FREEZE were bypassed structurally rather than deliberately.
//
// One call, once, at the flag-level flip point — not per fixer: there are 40+
// fixers and any check sprinkled among them rots the moment one is added. It
// runs before every mode dispatch (including the embedded-mode gate), so a
// refused run never half-executes and never needs a store to refuse.
//
// There is deliberately no doctor-side override. A migration freeze is exactly
// when a shared store has mixed-version clients and `bd doctor --fix` is the
// mid-incident reflex command; the operator clears the freeze first.
func checkDoctorMutationGate(absPath string) error {
op := doctorMutationOp()
if op == "" {
return nil
}
if readonlyMode {
// Wording matches CheckReadonly (errors.go) exactly; doctor returns the
// error rather than calling that void helper so its deferred metrics
// CloseEventAndAdd still runs.
fmt.Fprintf(os.Stderr, "Error: operation '%s' is not allowed in read-only mode\n", op)
return &exitError{Code: 1}
}
// Doctor is the one write-capable command that takes a target path, so the
// freeze has to be looked up against that target as well as against the
// directory bd was launched in: `bd doctor /frozen/repo --fix` from an
// unfrozen cwd would otherwise walk the wrong tree entirely. ...ErrorFor
// covers both (FindFrom(absPath) alone would miss a freeze in the caller's
// own cwd), and doctorCmd already sets SilenceErrors/SilenceUsage
// statically, so the cobra-silencing gate variant buys nothing here.
return migrationFreezeErrorFor(op, absPath)
}
func shouldSkipDoctorNetworkChecks() bool {
return jsonOutput || !ui.IsTerminal()
}
@ -371,7 +420,7 @@ func validateDoctorWorkspaceBackend(path string) error {
if err != nil {
return fmt.Errorf("failed to load %s: %w; no storage database was opened or modified; fix or restore metadata.json and retry", configfile.ConfigPath(beadsDir), err)
}
return validateConfiguredBackend(cfg)
return validateConfiguredBackend(cfg, beadsDir)
}
// printLegacyUpgradeDiagnostic preserves doctor as a store-free repair path:
@ -528,18 +577,32 @@ func runDiagnostics(path string) doctorResult {
// Since doctor skips PersistentPreRun DB init (via skipStoreAnnotation),
// trackBdVersion() and autoMigrateOnVersionBump() haven't run yet.
//
// Scope version tracking to the doctor target. Without this, `bd doctor <path>`
// can accidentally touch the caller's current repo .beads state.
origBeadsDir, hadBeadsDir := os.LookupEnv("BEADS_DIR")
_ = os.Setenv("BEADS_DIR", beadsDir)
trackBdVersion()
if hadBeadsDir {
_ = os.Setenv("BEADS_DIR", origBeadsDir)
} else {
_ = os.Unsetenv("BEADS_DIR")
}
// #6028: skipping that hook also skipped its guards on these exact two
// calls — main.go runs them only when policy.runMaintenance is set and the
// workspace is not frozen for maintenance. Doctor replicates the calls, so
// it must replicate the guard, or plain `bd doctor` (no --fix needed)
// rewrites .beads/.local_version and silently applies a schema migration to
// a read-only or mid-freeze store: precisely the torn-upgrade write the
// freeze exists to prevent. Under either gate doctor now *reports* the
// version/migration mismatch in the checks below instead of healing it.
//
// The probe takes beadsDir for the same reason the mutation gate takes
// absPath: these writes land in the doctor target, so a freeze on that tree
// must stop them even when bd was launched somewhere unfrozen.
if !readonlyMode && !migrationFreezeActiveFor(beadsDir) {
// Scope version tracking to the doctor target. Without this, `bd doctor <path>`
// can accidentally touch the caller's current repo .beads state.
origBeadsDir, hadBeadsDir := os.LookupEnv("BEADS_DIR")
_ = os.Setenv("BEADS_DIR", beadsDir)
trackBdVersion()
if hadBeadsDir {
_ = os.Setenv("BEADS_DIR", origBeadsDir)
} else {
_ = os.Unsetenv("BEADS_DIR")
}
autoMigrateOnVersionBump(beadsDir)
autoMigrateOnVersionBump(beadsDir)
}
// Check 1b: Dolt format compatibility (GH#2137)
// Must run before opening the database — old noms formats cause server panics.

View file

@ -13,6 +13,7 @@ import (
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/storage/dolt"
"gopkg.in/yaml.v3"
)
// validRoutingModes are the allowed values for routing.mode
@ -188,6 +189,9 @@ func checkYAMLConfigValues(repoPath string) []string {
issues = append(issues, fmt.Sprintf("config.yaml: failed to parse: %v", err))
return issues
}
if data, err := os.ReadFile(configPath); err == nil { //nolint:gosec // resolved workspace config
issues = append(issues, findDualSpelledConfigKeys(data)...)
}
// Validate issue-prefix (should be alphanumeric with dashes/underscores, reasonably short)
if v.IsSet("issue-prefix") {
@ -296,6 +300,59 @@ func checkYAMLConfigValues(repoPath string) []string {
return issues
}
func findDualSpelledConfigKeys(data []byte) []string {
var root yaml.Node
if err := yaml.Unmarshal(data, &root); err != nil || len(root.Content) == 0 {
return nil
}
mapping := root.Content[0]
if mapping.Kind != yaml.MappingNode {
return nil
}
var issues []string
for i := 0; i+1 < len(mapping.Content); i += 2 {
key := mapping.Content[i].Value
parts := strings.Split(key, ".")
if len(parts) < 2 || !yamlPathExists(mapping, parts) {
continue
}
issues = append(issues, fmt.Sprintf(
"config.yaml: %q is present in both flat and nested spelling; remove one spelling so all readers use the same value",
key,
))
}
return issues
}
func yamlPathExists(mapping *yaml.Node, parts []string) bool {
current := mapping
for i, part := range parts {
if current.Kind != yaml.MappingNode {
return false
}
idx := yamlMappingChild(current, part)
if idx == -1 {
return false
}
if i == len(parts)-1 {
return true
}
current = current.Content[idx+1]
}
return false
}
func yamlMappingChild(mapping *yaml.Node, name string) int {
for i := 0; i+1 < len(mapping.Content); i += 2 {
key := mapping.Content[i]
if key.Kind == yaml.ScalarNode && key.Value == name {
return i
}
}
return -1
}
// isValidBoolString checks if a string represents a valid boolean value
func isValidBoolString(s string) bool {
lower := strings.ToLower(strings.TrimSpace(s))

View file

@ -4,6 +4,7 @@ import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
@ -136,6 +137,26 @@ func TestCheckConfigValues(t *testing.T) {
})
}
func TestCheckConfigValuesWarnsOnFlatAndNestedSpellings(t *testing.T) {
tmpDir := t.TempDir()
beadsDir := filepath.Join(tmpDir, ".beads")
if err := os.MkdirAll(beadsDir, 0o755); err != nil {
t.Fatalf("create .beads: %v", err)
}
configContent := "dolt.host: 127.0.0.1\ndolt:\n host: 10.0.0.1\n"
if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte(configContent), 0o644); err != nil {
t.Fatalf("write config.yaml: %v", err)
}
check := CheckConfigValues(tmpDir)
if check.Status != StatusWarning {
t.Fatalf("expected warning, got %s: %s", check.Status, check.Detail)
}
if !strings.Contains(check.Detail, "dolt.host") || !strings.Contains(check.Detail, "flat and nested") {
t.Fatalf("warning does not identify the dual spelling: %s", check.Detail)
}
}
func TestCheckConfigValues_WorktreeFallbackUsesSharedConfig(t *testing.T) {
mainRepoDir, worktreeDir := setupDoctorSharedWorktree(t)
mainBeadsDir := filepath.Join(mainRepoDir, ".beads")

View file

@ -62,10 +62,18 @@ func testMainInner(m *testing.M) int {
// doctor_pkg_shared and doctest_*-prefixed databases through.
os.Setenv("BEADS_TEST_SERVER", "1")
// Pin t.TempDir() under a suite-owned root so SweepOrphanedTestServers
// Clear out the roots of earlier runs of this suite whose process is
// gone, before claiming one of our own. A `go test -timeout` panic skips
// every defer here AND the post-run sweep, so the servers such a run
// started outlive every cleanup this process installs and nothing ever
// looks at that run's tree again (wy-j2zc8q). Roots with no owner marker,
// and roots whose owner is still running, are left untouched.
doltserver.SweepDeadSuiteRoots(os.TempDir(), suiteRootPrefix)
// Pin t.TempDir() under a suite-owned root so SweepSuiteTestServers
// can reap AutoStart leftovers whose t.TempDir() cleanup failed because
// the live child still holds the tree (gastownhall/beads#5631).
root, pinErr := testutil.PinSuiteTempRoot("beads-doctor-tests-*")
root, pinErr := testutil.PinSuiteTempRoot(suiteRootPrefix + "*")
if pinErr != nil {
fmt.Fprintf(os.Stderr, "FATAL: suite temp root: %v\n", pinErr)
return 1
@ -73,6 +81,12 @@ func testMainInner(m *testing.M) int {
suiteTempRoot = root
defer os.RemoveAll(root)
// Claim the root for this process so the NEXT run can tell our debris
// from a concurrent run's live tree.
if err := doltserver.WriteSuiteOwnerMarker(root); err != nil {
fmt.Fprintf(os.Stderr, "Warning: could not claim suite temp root %s: %v\n", root, err)
}
if err := testutil.EnsureDoltContainerForTestMain(); err != nil {
fmt.Fprintf(os.Stderr, "WARN: %v, skipping Dolt tests\n", err)
} else {
@ -101,7 +115,8 @@ func testMainInner(m *testing.M) int {
// Best-effort reap of any dolt sql-server left running under a temp dir
// this suite created (e.g. a SIGKILLed run) — see
// gastownhall/beads mybd-q6cz / #5631.
doltserver.SweepOrphanedTestServers(root, testBDDir)
swept := doltserver.SweepSuiteTestServers(root, testBDDir)
code = doltserver.ApplyLeakPolicy("cmd/bd/doctor", code, swept)
os.Unsetenv("BEADS_DOLT_PORT")
os.Unsetenv("BEADS_TEST_MODE")

View file

@ -1,39 +1,25 @@
package fix
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/steveyegge/beads/internal/testutil"
)
// suiteTempRoot is the TestMain-owned temp directory that t.TempDir must
// land under so SweepOrphanedTestServers can reap leaked sql-servers.
// land under so SweepSuiteTestServers can reap leaked sql-servers.
var suiteTempRoot string
// suiteRootPrefix is this suite's PinSuiteTempRoot pattern without its
// random tail. SweepDeadSuiteRoots globs for it, so the two must not drift.
const suiteRootPrefix = "beads-fix-tests-"
func TestTempDirLandsUnderSuiteSweepRoot(t *testing.T) {
if suiteTempRoot == "" {
t.Fatal("TestMain did not pin suiteTempRoot; leaked dolt sql-server processes cannot be swept")
}
tmp := t.TempDir()
if !pathUnderRoot(tmp, suiteTempRoot) {
if !testutil.PathUnderSuiteRoot(tmp, suiteTempRoot) {
t.Fatalf("t.TempDir() %q is not under suiteTempRoot %q", tmp, suiteTempRoot)
}
}
func pathUnderRoot(dir, root string) bool {
dir = evalOrSelf(dir)
root = evalOrSelf(root)
if dir == root {
return true
}
sep := string(os.PathSeparator)
return strings.HasPrefix(dir, strings.TrimRight(root, sep)+sep)
}
func evalOrSelf(p string) string {
if r, err := filepath.EvalSymlinks(p); err == nil {
return r
}
return p
}

View file

@ -31,11 +31,19 @@ func testMainInner(m *testing.M) int {
defer testutil.TerminateDoltContainer()
}
// Clear out the roots of earlier runs of this suite whose process is
// gone, before claiming one of our own. A `go test -timeout` panic skips
// every defer here AND the post-run sweep, so the servers such a run
// started outlive every cleanup this process installs and nothing ever
// looks at that run's tree again (wy-j2zc8q). Roots with no owner marker,
// and roots whose owner is still running, are left untouched.
doltserver.SweepDeadSuiteRoots(os.TempDir(), suiteRootPrefix)
// Pin t.TempDir() under a suite-owned root so the sweep below can reap
// AutoStart leftovers whose directory cleanup failed because the live
// child still holds the tree (gastownhall/beads#5631). Must never be a
// shared/global temp dir (see SweepOrphanedTestServers).
root, pinErr := testutil.PinSuiteTempRoot("beads-fix-tests-*")
// shared/global temp dir (see SweepSuiteTestServers).
root, pinErr := testutil.PinSuiteTempRoot(suiteRootPrefix + "*")
if pinErr != nil {
fmt.Fprintf(os.Stderr, "FATAL: suite temp root: %v\n", pinErr)
return 1
@ -43,12 +51,19 @@ func testMainInner(m *testing.M) int {
suiteTempRoot = root
defer os.RemoveAll(root)
// Claim the root for this process so the NEXT run can tell our debris
// from a concurrent run's live tree.
if err := doltserver.WriteSuiteOwnerMarker(root); err != nil {
fmt.Fprintf(os.Stderr, "Warning: could not claim suite temp root %s: %v\n", root, err)
}
code := m.Run()
// Best-effort reap of any dolt sql-server left running under this
// suite's own temp root (e.g. a SIGKILLed run) — see
// gastownhall/beads mybd-q6cz / #5631.
doltserver.SweepOrphanedTestServers(root)
swept := doltserver.SweepSuiteTestServers(root)
code = doltserver.ApplyLeakPolicy("cmd/bd/doctor/fix", code, swept)
os.Unsetenv("BEADS_DOLT_PORT")
os.Unsetenv("BEADS_TEST_MODE")
os.Unsetenv("BEADS_TEST_SERVER")

View file

@ -19,7 +19,15 @@ func TestMain(m *testing.M) {
}
func testMainInner(m *testing.M) int {
root, err := testutil.PinSuiteTempRoot("beads-fix-tests-*")
// Clear out the roots of earlier runs of this suite whose process is
// gone, before claiming one of our own. A `go test -timeout` panic skips
// every defer here AND the post-run sweep, so the servers such a run
// started outlive every cleanup this process installs and nothing ever
// looks at that run's tree again (wy-j2zc8q). Roots with no owner marker,
// and roots whose owner is still running, are left untouched.
doltserver.SweepDeadSuiteRoots(os.TempDir(), suiteRootPrefix)
root, err := testutil.PinSuiteTempRoot(suiteRootPrefix + "*")
if err != nil {
fmt.Fprintf(os.Stderr, "FATAL: suite temp root: %v\n", err)
return 1
@ -27,7 +35,14 @@ func testMainInner(m *testing.M) int {
suiteTempRoot = root
defer os.RemoveAll(root)
// Claim the root for this process so the NEXT run can tell our debris
// from a concurrent run's live tree.
if err := doltserver.WriteSuiteOwnerMarker(root); err != nil {
fmt.Fprintf(os.Stderr, "Warning: could not claim suite temp root %s: %v\n", root, err)
}
code := m.Run()
doltserver.SweepOrphanedTestServers(root)
swept := doltserver.SweepSuiteTestServers(root)
code = doltserver.ApplyLeakPolicy("cmd/bd/doctor/fix", code, swept)
return code
}

View file

@ -1,39 +1,25 @@
package doctor
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/steveyegge/beads/internal/testutil"
)
// suiteTempRoot is the TestMain-owned temp directory that t.TempDir must
// land under so SweepOrphanedTestServers can reap leaked sql-servers.
// land under so SweepSuiteTestServers can reap leaked sql-servers.
var suiteTempRoot string
// suiteRootPrefix is this suite's PinSuiteTempRoot pattern without its
// random tail. SweepDeadSuiteRoots globs for it, so the two must not drift.
const suiteRootPrefix = "beads-doctor-tests-"
func TestTempDirLandsUnderSuiteSweepRoot(t *testing.T) {
if suiteTempRoot == "" {
t.Fatal("TestMain did not pin suiteTempRoot; leaked dolt sql-server processes cannot be swept")
}
tmp := t.TempDir()
if !pathUnderRoot(tmp, suiteTempRoot) {
if !testutil.PathUnderSuiteRoot(tmp, suiteTempRoot) {
t.Fatalf("t.TempDir() %q is not under suiteTempRoot %q", tmp, suiteTempRoot)
}
}
func pathUnderRoot(dir, root string) bool {
dir = evalOrSelf(dir)
root = evalOrSelf(root)
if dir == root {
return true
}
sep := string(os.PathSeparator)
return strings.HasPrefix(dir, strings.TrimRight(root, sep)+sep)
}
func evalOrSelf(p string) string {
if r, err := filepath.EvalSymlinks(p); err == nil {
return r
}
return p
}

View file

@ -19,7 +19,15 @@ func TestMain(m *testing.M) {
}
func testMainInner(m *testing.M) int {
root, err := testutil.PinSuiteTempRoot("beads-doctor-tests-*")
// Clear out the roots of earlier runs of this suite whose process is
// gone, before claiming one of our own. A `go test -timeout` panic skips
// every defer here AND the post-run sweep, so the servers such a run
// started outlive every cleanup this process installs and nothing ever
// looks at that run's tree again (wy-j2zc8q). Roots with no owner marker,
// and roots whose owner is still running, are left untouched.
doltserver.SweepDeadSuiteRoots(os.TempDir(), suiteRootPrefix)
root, err := testutil.PinSuiteTempRoot(suiteRootPrefix + "*")
if err != nil {
fmt.Fprintf(os.Stderr, "FATAL: suite temp root: %v\n", err)
return 1
@ -27,7 +35,14 @@ func testMainInner(m *testing.M) int {
suiteTempRoot = root
defer os.RemoveAll(root)
// Claim the root for this process so the NEXT run can tell our debris
// from a concurrent run's live tree.
if err := doltserver.WriteSuiteOwnerMarker(root); err != nil {
fmt.Fprintf(os.Stderr, "Warning: could not claim suite temp root %s: %v\n", root, err)
}
code := m.Run()
doltserver.SweepOrphanedTestServers(root)
swept := doltserver.SweepSuiteTestServers(root)
code = doltserver.ApplyLeakPolicy("cmd/bd/doctor", code, swept)
return code
}

View file

@ -16,6 +16,17 @@ var latestGitHubReleaseFetcher = fetchLatestGitHubRelease
// CheckCLIVersion checks if the CLI version is up to date.
// Takes cliVersion parameter since it can't access the Version variable from main package.
func CheckCLIVersion(cliVersion string) DoctorCheck {
// A Homebrew --HEAD build carries no orderable version — CompareVersions
// would read it as 0.0.0 and warn "update available" forever, and the
// suggested upgrade would actually move the user off HEAD.
if IsBrewHeadVersion(cliVersion) {
return DoctorCheck{
Name: "CLI Version",
Status: StatusOK,
Message: fmt.Sprintf("%s (Homebrew --HEAD build; release comparison skipped)", cliVersion),
}
}
latestVersion, err := latestGitHubReleaseFetcher()
if err != nil {
// Network error or API issue - don't fail, just warn
@ -151,6 +162,20 @@ func CheckMetadataVersionTracking(path string, currentVersion string) DoctorChec
}
}
trackingActive := DoctorCheck{
Name: "Version Tracking",
Status: StatusOK,
Message: fmt.Sprintf("Version tracking active (last: %s, current: %s)", lastVersion, currentVersion),
}
// A Homebrew --HEAD stamp carries no ordering, so the staleness heuristic
// below cannot apply — but it is a healthy marker, not a malformed one.
// Warning about it would be permanent: the offered fix rewrites the same
// stamp back.
if IsBrewHeadVersion(lastVersion) {
return trackingActive
}
// Validate that version is a valid semver-like string
if !IsValidSemver(lastVersion) {
return DoctorCheck{
@ -171,11 +196,7 @@ func CheckMetadataVersionTracking(path string, currentVersion string) DoctorChec
// Guard against short version strings (e.g., "5" → [5] has no [1])
if len(currentParts) < 2 || len(lastParts) < 2 {
return DoctorCheck{
Name: "Version Tracking",
Status: StatusOK,
Message: fmt.Sprintf("Version tracking active (last: %s, current: %s)", lastVersion, currentVersion),
}
return trackingActive
}
// Simple heuristic: warn if minor version is 10+ behind or major version differs by 1+
@ -193,11 +214,7 @@ func CheckMetadataVersionTracking(path string, currentVersion string) DoctorChec
}
// Version is behind but not too old - this is normal after upgrade
return DoctorCheck{
Name: "Version Tracking",
Status: StatusOK,
Message: fmt.Sprintf("Version tracking active (last: %s, current: %s)", lastVersion, currentVersion),
}
return trackingActive
}
// Version is current or ahead
@ -289,6 +306,49 @@ func CompareVersions(v1, v2 string) int {
return 0
}
// IsBrewHeadVersion recognizes the version Homebrew stamps into --HEAD
// installs of the core beads formula: HEAD-<shortsha>, a bare HEAD when the
// head spec resolves no commit, and either shape carrying Homebrew's
// _<revision> suffix. Such a stamp is a healthy current-era version marker,
// not a malformed one — the binary rewrites it identically on every run, so
// nothing the user does can "fix" it into semver.
//
// This is the canonical definition; every consumer of a version stamp that
// needs to tell a --HEAD build apart from a release delegates here so the
// ends cannot drift.
func IsBrewHeadVersion(version string) bool {
rest, ok := strings.CutPrefix(version, "HEAD")
if !ok {
return false
}
// Homebrew's pkg_version appends _<revision> once the formula carries a
// revision, so a revision bump must not read as malformed. The revision is
// an unsigned decimal; a digits-only check keeps Atoi's tolerance of
// signed forms ("+1", "-0") from admitting shapes brew never emits.
if cut := strings.IndexByte(rest, '_'); cut >= 0 {
revision := rest[cut+1:]
if revision == "" || strings.Trim(revision, "0123456789") != "" {
return false
}
rest = rest[:cut]
}
// Homebrew leaves the version as a bare "HEAD" whenever the head spec
// cannot resolve a commit.
if rest == "" {
return true
}
sha, ok := strings.CutPrefix(rest, "-")
if !ok {
return false
}
// Bound the tail to a plausible git abbreviation so that arbitrary hex
// cannot stand in for a commit; git never abbreviates below 7 characters.
if len(sha) < 7 || len(sha) > 40 {
return false
}
return strings.Trim(sha, "0123456789abcdefABCDEF") == ""
}
// IsValidSemver checks if a version string is valid semver-like format (X.Y.Z)
func IsValidSemver(version string) bool {
if version == "" {

View file

@ -69,6 +69,41 @@ func TestIsValidSemver(t *testing.T) {
}
}
func TestIsBrewHeadVersion(t *testing.T) {
tests := []struct {
name string
version string
want bool
}{
{name: "short sha", version: "HEAD-f925f3f", want: true},
{name: "full sha", version: "HEAD-" + strings.Repeat("a", 40), want: true},
{name: "uppercase sha", version: "HEAD-F925F3F", want: true},
{name: "bare HEAD", version: "HEAD", want: true},
{name: "revision suffix", version: "HEAD-f925f3f_1", want: true},
{name: "bare HEAD with revision suffix", version: "HEAD_2", want: true},
{name: "empty sha", version: "HEAD-"},
{name: "non-hex sha", version: "HEAD-zzzzzzz"},
{name: "sha below git abbreviation floor", version: "HEAD-abc"},
{name: "sha beyond object id length", version: "HEAD-" + strings.Repeat("a", 41)},
{name: "dirty suffix", version: "HEAD-f925f3f-dirty"},
{name: "non-numeric revision", version: "HEAD-f925f3f_x"},
{name: "signed revision", version: "HEAD-f925f3f_+1"},
{name: "negative zero revision", version: "HEAD_-0"},
{name: "empty revision", version: "HEAD-f925f3f_"},
{name: "unrelated prefix", version: "HEADLESS-f925f3f"},
{name: "semver", version: "1.1.2"},
{name: "empty", version: ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := IsBrewHeadVersion(tt.version); got != tt.want {
t.Fatalf("IsBrewHeadVersion(%q) = %v, want %v", tt.version, got, tt.want)
}
})
}
}
func TestUpgradeCommandForPath(t *testing.T) {
tests := []struct {
name string
@ -119,6 +154,39 @@ func TestCheckCLIVersionUsesFetcher(t *testing.T) {
}
}
// TestCheckCLIVersionBrewHeadBuild verifies that a --HEAD build is not told to
// run `brew upgrade beads`. CompareVersions reads the stamp as 0.0.0, so every
// release looks newer forever, and taking the advice would move the user off
// HEAD entirely.
func TestCheckCLIVersionBrewHeadBuild(t *testing.T) {
orig := latestGitHubReleaseFetcher
fetched := false
latestGitHubReleaseFetcher = func() (string, error) {
fetched = true
return "1.3.0", nil
}
t.Cleanup(func() { latestGitHubReleaseFetcher = orig })
for _, stamp := range []string{"HEAD-f925f3f", "HEAD", "HEAD-f925f3f_1"} {
t.Run(stamp, func(t *testing.T) {
check := CheckCLIVersion(stamp)
if check.Status != StatusOK {
t.Fatalf("Status = %q, want %q (message: %q)", check.Status, StatusOK, check.Message)
}
if check.Fix != "" {
t.Fatalf("Fix = %q, want no upgrade advice for a --HEAD build", check.Fix)
}
if !strings.Contains(check.Message, stamp) {
t.Fatalf("Message = %q, want it to name the stamp", check.Message)
}
})
}
if fetched {
t.Error("CheckCLIVersion fetched the latest release for a --HEAD build; there is nothing to compare against")
}
}
func TestParseVersionParts(t *testing.T) {
tests := []struct {
name string
@ -191,6 +259,34 @@ func TestCheckMetadataVersionTracking_EmptyFile(t *testing.T) {
}
}
// TestCheckMetadataVersionTracking_BrewHeadStamp verifies that the version
// stamp a Homebrew --HEAD install writes is reported as healthy rather than
// permanently warning as an invalid format the user cannot fix: the offered
// fix, running any bd command, writes the same stamp straight back.
func TestCheckMetadataVersionTracking_BrewHeadStamp(t *testing.T) {
for _, stamp := range []string{"HEAD-f925f3f", "HEAD", "HEAD-f925f3f_1"} {
t.Run(stamp, func(t *testing.T) {
tmpDir := t.TempDir()
beadsDir := filepath.Join(tmpDir, ".beads")
if err := os.MkdirAll(beadsDir, 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(beadsDir, ".local_version"), []byte(stamp+"\n"), 0644); err != nil {
t.Fatal(err)
}
check := CheckMetadataVersionTracking(tmpDir, "HEAD-aaaaaaa")
if check.Status != StatusOK {
t.Errorf("Status = %q, want %q (message: %q)", check.Status, StatusOK, check.Message)
}
if check.Fix != "" {
t.Errorf("Fix = %q, want no fix advice for a --HEAD stamp", check.Fix)
}
})
}
}
// TestCheckMetadataVersionTracking_InvalidVersion verifies handling of
// malformed version strings in .local_version.
func TestCheckMetadataVersionTracking_InvalidVersion(t *testing.T) {

View file

@ -1,73 +0,0 @@
package main
import (
"os"
"path/filepath"
)
// isOrchestratorRoot returns true when path looks like a multi-project
// orchestrator workspace root (not a single-project beads repo).
//
// Detection: presence of both:
// 1. .beads/routes.jsonl (cross-project routing config)
// 2. mayor/town.json (orchestrator configuration)
//
// This prevents bd doctor --fix from running at the workspace root,
// where repairs should go through the orchestrator's own doctor command.
func isOrchestratorRoot(path string) bool {
if path == "" {
return false
}
routes := filepath.Join(path, ".beads", "routes.jsonl")
townJSON := filepath.Join(path, "mayor", "town.json")
if _, err := os.Stat(routes); err != nil {
return false
}
if _, err := os.Stat(townJSON); err != nil {
return false
}
return true
}
// findTownRoot walks up from the current working directory looking for
// mayor/town.json — the same primary marker gt's own workspace package uses
// (see gastownhall/gastown internal/cmd/handoff.go's detectTownRootFromCwd).
// Falls back to GT_TOWN_ROOT then GT_ROOT (gt's env-var fallback chain,
// already how the rest of this codebase detects an orchestrator — see
// formula.go, molecules.go, doltserver.go) when cwd detection fails, e.g. a
// detached worktree or a cwd outside the town tree entirely.
//
// Used by CheckMigrationFreeze (dc-6jaq) to find the MIGRATION-FREEZE
// sentinel; unlike isOrchestratorRoot above this only needs the mayor/
// marker, not also .beads/routes.jsonl, since a rig-level bd invocation's
// cwd is never itself the orchestrator root.
func findTownRoot() string {
if cwd, err := os.Getwd(); err == nil {
path := cwd
for {
if _, statErr := os.Stat(filepath.Join(path, "mayor", "town.json")); statErr == nil {
return path
}
parent := filepath.Dir(path)
if parent == path {
break
}
path = parent
}
}
for _, envName := range []string{"GT_TOWN_ROOT", "GT_ROOT"} {
envRoot := os.Getenv(envName)
if envRoot == "" {
continue
}
if _, err := os.Stat(filepath.Join(envRoot, "mayor", "town.json")); err == nil {
return envRoot
}
}
return ""
}

View file

@ -0,0 +1,226 @@
// Tests for doctor's mutation gate (#6028): `bd doctor --fix` / `--clean` must
// refuse to run under strict --readonly, exactly like the ~120 write commands
// that call CheckReadonly at the top of their RunE. Doctor never made that call
// — it opts out of the root PersistentPreRunE store init via
// skipStoreAnnotation, which also opted it out of that hook's gates — so both
// --readonly and an active MIGRATION-FREEZE were bypassed structurally.
//
// The freeze half of the same gate lives in migration_freeze_gate_test.go,
// beside the rest of the freeze suite; both halves share the helpers below and
// the hermetic subprocess harness declared there.
//
// This file MUST NOT carry a cgo build tag: it drives a bd binary built with
// the gms_pure_go tag via subprocess, and asserts only on files and output.
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
// doctorMutationSurface is one class of doctor invocation that mutates the
// workspace. Section 2 of the #6028 design enumerates the complete set: every
// mutating path is reached through --fix or --clean and nothing else, so these
// five rows cover the whole write surface, including fixers added later.
type doctorMutationSurface struct {
name string
args []string
// op is the operation label the gate reports, derived from which flag made
// the run mutating (--fix wins when both are set).
op string
}
func doctorMutationSurfaces() []doctorMutationSurface {
return []doctorMutationSurface{
{name: "fix", args: []string{"doctor", "--fix", "--yes"}, op: "doctor --fix"},
{name: "fix-interactive", args: []string{"doctor", "--fix", "-i"}, op: "doctor --fix"},
{name: "validate-fix", args: []string{"doctor", "--check=validate", "--fix", "--yes"}, op: "doctor --fix"},
{name: "pollution-clean", args: []string{"doctor", "--check=pollution", "--clean", "--yes"}, op: "doctor --clean"},
{name: "artifacts-clean", args: []string{"doctor", "--check=artifacts", "--clean", "--yes"}, op: "doctor --clean"},
}
}
// doctorWorkspaceFingerprint captures the two workspace files a refused doctor
// run is most likely to have touched: .local_version (trackBdVersion's target)
// and metadata.json (the Metadata Config / Database fixers' target).
func doctorWorkspaceFingerprint(t *testing.T, dir string) map[string][]byte {
t.Helper()
fingerprint := make(map[string][]byte)
for _, name := range []string{localVersionFile, "metadata.json"} {
path := filepath.Join(dir, ".beads", name)
data, err := os.ReadFile(path)
if err != nil {
if !os.IsNotExist(err) {
t.Fatalf("fingerprinting %s: %v", path, err)
}
continue
}
fingerprint[name] = data
}
return fingerprint
}
// assertDoctorWorkspaceUnchanged proves the refusal happened before any
// mutation, not merely that the refusal message was printed.
func assertDoctorWorkspaceUnchanged(t *testing.T, dir string, before map[string][]byte) {
t.Helper()
after := doctorWorkspaceFingerprint(t, dir)
if len(after) != len(before) {
t.Errorf("workspace file set changed across a refused doctor run: had %d tracked files, now %d", len(before), len(after))
}
for name, want := range before {
got, ok := after[name]
if !ok {
t.Errorf(".beads/%s was deleted by a refused doctor run", name)
continue
}
if string(got) != string(want) {
t.Errorf(".beads/%s was rewritten by a refused doctor run:\nbefore:\n%s\nafter:\n%s", name, want, got)
}
}
}
// assertDoctorRefusalIsClean checks the shared shape of every gate refusal:
// nothing on stdout, and none of the output a fix or clean run would produce.
func assertDoctorRefusalIsClean(t *testing.T, stdout string) {
t.Helper()
if strings.TrimSpace(stdout) != "" {
t.Errorf("stdout should be empty when doctor is refused, got:\n%s", stdout)
}
for _, marker := range []string{"Applying fixes", "Deleting", "Verifying fixes"} {
if strings.Contains(stdout, marker) {
t.Errorf("refused doctor run produced %q on stdout — it started mutating before refusing:\n%s", marker, stdout)
}
}
}
// TestDoctorMutationBlockedInReadonlyMode is the readonly half of the gate.
// One workspace serves every row: a refused run must leave it untouched, so a
// row that leaks a mutation is caught by the next row's fingerprint too.
func TestDoctorMutationBlockedInReadonlyMode(t *testing.T) {
bd, dir := setupMigrationFreezeWorkspace(t)
before := doctorWorkspaceFingerprint(t, dir)
for _, tt := range doctorMutationSurfaces() {
t.Run(tt.name, func(t *testing.T) {
args := append(append([]string{}, tt.args...), "--readonly")
stdout, stderr, code := runBDMigrationFreeze(t, bd, dir, args...)
if code != 1 {
t.Fatalf("exit code = %d, want 1\nstdout:\n%s\nstderr:\n%s", code, stdout, stderr)
}
want := "operation '" + tt.op + "' is not allowed in read-only mode"
if !strings.Contains(stderr, want) {
t.Errorf("stderr missing %q:\n%s", want, stderr)
}
assertDoctorRefusalIsClean(t, stdout)
assertDoctorWorkspaceUnchanged(t, dir, before)
})
}
}
// TestDoctorDiagnosisWorksInReadonlyMode pins the other half of the contract:
// the gate keys on --fix/--clean alone, so every diagnosis mode still runs
// under --readonly. Diagnosing a sandboxed workspace is the whole point.
func TestDoctorDiagnosisWorksInReadonlyMode(t *testing.T) {
bd, dir := setupMigrationFreezeWorkspace(t)
for _, args := range [][]string{
{"doctor"},
{"doctor", "--dry-run"},
{"doctor", "--check=pollution"},
{"doctor", "--check=artifacts"},
} {
t.Run(strings.Join(args, "_"), func(t *testing.T) {
stdout, stderr, _ := runBDMigrationFreeze(t, bd, dir, append(args, "--readonly")...)
if strings.Contains(stderr, "not allowed in read-only mode") {
t.Errorf("bd %v was refused under --readonly but mutates nothing:\nstderr:\n%s\nstdout:\n%s", args, stderr, stdout)
}
})
}
}
// TestDoctorMaintenanceSkippedInReadonlyMode is the readonly twin of
// TestDoctorMaintenanceSkippedDuringMigrationFreeze — see that test for why
// plain `bd doctor` had hidden writes at all, and for the shared-server setup.
func TestDoctorMaintenanceSkippedInReadonlyMode(t *testing.T) {
bd, dir := setupMigrationFreezeWorkspace(t)
seedStaleLocalVersion(t, dir)
stdout, stderr, _ := runBDMigrationFreezeWithEnv(t, bd, dir, doctorMaintenanceEnv(), "doctor", "--readonly")
assertDoctorReportedStaleVersionWithoutHealing(t, dir, stdout, stderr)
if strings.Contains(stderr, "not allowed in read-only mode") {
t.Errorf("plain 'bd doctor --readonly' must diagnose, not refuse:\nstderr:\n%s", stderr)
}
}
// TestDoctorMutationNotBlockedWithoutGates is the regression-safety control:
// with no freeze sentinel and no --readonly, the gate must be invisible. The
// fix outcome itself is not this test's business — only that neither refusal
// fires and neither exit path is the gate's.
func TestDoctorMutationNotBlockedWithoutGates(t *testing.T) {
bd, dir := setupMigrationFreezeWorkspace(t)
for _, tt := range doctorMutationSurfaces() {
t.Run(tt.name, func(t *testing.T) {
stdout, stderr, code := runBDMigrationFreeze(t, bd, dir, tt.args...)
if code == ExitMigrationFrozen || strings.Contains(stderr, "frozen for migration") {
t.Errorf("bd %v hit the freeze gate with no marker present (exit %d):\nstderr:\n%s", tt.args, code, stderr)
}
if strings.Contains(stderr, "not allowed in read-only mode") {
t.Errorf("bd %v hit the readonly gate without --readonly (exit %d):\nstderr:\n%s", tt.args, code, stderr)
}
_ = stdout
})
}
}
// TestDoctorMaintenanceStillRunsWithoutGates is the companion control for the
// diagnosis-path change: unfrozen and not read-only, plain `bd doctor` must
// still reconcile a stale .local_version exactly as it did before #6028.
func TestDoctorMaintenanceStillRunsWithoutGates(t *testing.T) {
bd, dir := setupMigrationFreezeWorkspace(t)
seedStaleLocalVersion(t, dir)
stdout, stderr, _ := runBDMigrationFreezeWithEnv(t, bd, dir, doctorMaintenanceEnv(), "doctor")
if !strings.Contains(stderr, "auto-migrate:") {
t.Errorf("expected autoMigrateOnVersionBump to run (an 'auto-migrate:' debug line) with no gate active, got none:\nstderr:\n%s", stderr)
}
got := readWorkspaceLocalVersion(t, dir)
if got == staleLocalVersion {
t.Errorf("%s = %q — trackBdVersion did not run with no gate active; the #6028 skip is too broad:\nstdout:\n%s",
localVersionFile, got, stdout)
}
}
// TestDoctorReadonlyWinsOverMigrationFreeze pins the branch order inside the
// gate: --readonly is checked first, so a workspace that is both frozen and
// sandboxed reports the readonly refusal. This mirrors CheckReadonly, which
// also answers readonly before delegating to the freeze check. Either refusal
// stops the run, but they carry different exit codes — 1 vs ExitMigrationFrozen
// — so which one wins is observable, and the precedence has to be deliberate.
func TestDoctorReadonlyWinsOverMigrationFreeze(t *testing.T) {
bd, dir := setupMigrationFreezeWorkspace(t)
writeFreezeMarker(t, dir, "migrator", "dolt v2 migration")
before := doctorWorkspaceFingerprint(t, dir)
stdout, stderr, code := runBDFrozen(t, bd, dir, "doctor", "--fix", "--yes", "--readonly")
if code != 1 {
t.Fatalf("exit code = %d, want 1 (not %d — readonly is answered before the freeze)\nstdout:\n%s\nstderr:\n%s",
code, ExitMigrationFrozen, stdout, stderr)
}
if !strings.Contains(stderr, "operation 'doctor --fix' is not allowed in read-only mode") {
t.Errorf("stderr missing the readonly refusal (readonly must be checked before the freeze):\n%s", stderr)
}
if strings.Contains(stderr, "frozen for migration") {
t.Errorf("stderr carries the freeze refusal too; the gate should report exactly one reason:\n%s", stderr)
}
assertDoctorRefusalIsClean(t, stdout)
assertDoctorWorkspaceUnchanged(t, dir, before)
}

View file

@ -0,0 +1,33 @@
package main
import (
"os"
"path/filepath"
)
// isOrchestratorRoot returns true when path looks like a multi-project
// orchestrator workspace root (not a single-project beads repo).
//
// Detection: presence of both:
// 1. .beads/routes.jsonl (cross-project routing config)
// 2. mayor/town.json (orchestrator configuration)
//
// This prevents bd doctor --fix from running at the workspace root,
// where repairs should go through the orchestrator's own doctor command.
func isOrchestratorRoot(path string) bool {
if path == "" {
return false
}
routes := filepath.Join(path, ".beads", "routes.jsonl")
townJSON := filepath.Join(path, "mayor", "town.json")
if _, err := os.Stat(routes); err != nil {
return false
}
if _, err := os.Stat(townJSON); err != nil {
return false
}
return true
}

View file

@ -773,7 +773,12 @@ The server runs in the background on a per-project port derived from the
project path. PID and logs are stored in .beads/.
The server auto-starts transparently when needed, so manual start is rarely
required. Use this command for explicit control or diagnostics.`,
required. Use this command for explicit control or diagnostics.
Not available in proxied-server mode: there the proxy owns its dolt backend's
lifecycle and starts it on demand, so a server started here would be a second,
unsupervised one over the same data directory. Use 'bd dolt status' to see what
is running and 'bd dolt stop' to shut it down.`,
RunE: func(cmd *cobra.Command, args []string) error {
beadsDir := selectedDoltBeadsDir()
if beadsDir == "" {
@ -783,6 +788,26 @@ required. Use this command for explicit control or diagnostics.`,
if err != nil {
return HandleError("%v", err)
}
// A proxied workspace already has a server manager. The proxy spawns
// its dolt child on demand and reaps the whole tree when idle, so
// there is no state for bd to add here — only a second one to
// collide with. With the tree down, doltserver.Start resolves the
// proxied root's own configured port and launches an unsupervised
// sql-server directly over .beads/dolt, and the next ordinary bd
// command then fails because the relaunched proxy finds a foreign
// server on its port; with the tree up, Start instead adopts the
// proxy's child into bd's classic PID/port records, leaving two
// managers for one process. Refused rather than routed: the proxy
// owns its child's lifecycle, and an idle reaper means a "started"
// postcondition bd cannot honor for more than the idle window.
//
// Both the process-wide mode and the selected workspace's own
// metadata are consulted, and this runs before the embedded guard:
// over-refusing costs an operator one error message, under-refusing
// costs them the datastore.
if usesProxiedServer() || fileCfg.IsDoltProxiedServerMode() {
return HandleProxyCapabilityError(proxiedDoltStartRefusal())
}
if !usesSQLServer() {
return HandleError("'bd dolt start' is not supported in embedded mode (no Dolt server)")
}
@ -790,7 +815,7 @@ required. Use this command for explicit control or diagnostics.`,
// server lifecycle (GH#3545/GH#3518): starting a repo-local
// server here would write local PID/port state that shadows the
// configured remote endpoint.
if host := fileCfg.GetDoltServerHost(); !usesProxiedServer() && !configfile.IsLocalHostString(host) {
if host := fileCfg.GetDoltServerHost(); !configfile.IsLocalHostString(host) {
return HandleError("the configured Dolt server host is remote (%s); 'bd dolt start' only manages a local server.\nStart the server on that host, or clear dolt_server_host / dolt.host / BEADS_DOLT_SERVER_HOST to run one locally", host)
}
serverDir := doltserver.ResolveServerDir(beadsDir)
@ -1067,7 +1092,12 @@ PID, port, and data directory from the local PID file. For externally-
managed servers — a shared server (dolt.shared-server: true), a remote
dolt_server_host, or a local server managed outside bd (dolt.auto-start:
false, e.g. an orchestrator-shared sql-server) — pings the configured
endpoint via SQL and reports reachability, server version, and database.`,
endpoint via SQL and reports reachability, server version, and database.
In proxied-server mode, reports the proxy (the endpoint every bd command
connects through) and the dolt backend behind it separately, read from the
proxy's own process records. Reporting starts nothing: a quiesced workspace
is shown as not running, and the next bd command launches it.`,
RunE: func(cmd *cobra.Command, args []string) error {
beadsDir := selectedDoltBeadsDir()
if beadsDir == "" {
@ -1080,7 +1110,7 @@ endpoint via SQL and reports reachability, server version, and database.`,
if cfg == nil {
cfg = configfile.DefaultConfig()
}
if err := validateConfiguredBackend(cfg); err != nil {
if err := validateConfiguredBackend(cfg, beadsDir); err != nil {
return HandleError("%v", err)
}
// A non-Dolt backend (SQLite or a removed-backend tombstone) has no Dolt engine;
@ -1090,6 +1120,21 @@ endpoint via SQL and reports reachability, server version, and database.`,
fmt.Printf("Backend: %s (no Dolt engine)\n", cfg.GetBackend())
return nil
}
// Neither branch below describes a proxied workspace. It writes no
// classic PID file, so the bd-managed path reports "not running"
// while the proxy is serving CRUD, and shouldUseExternalDoltStatus
// deliberately excludes proxied so the SQL-probe path never sees it
// either. Keyed on the loaded config rather than the process-wide
// mode so `bd --db <proxied workspace> dolt status` describes the
// workspace it was pointed at.
if cfg.IsDoltProxiedServerMode() {
status, err := collectProxiedDoltStatus(beadsDir)
if err != nil {
return HandleError("%v", err)
}
renderProxiedDoltStatus(status)
return nil
}
if !usesSQLServer() {
showEmbeddedDoltStatus(beadsDir)
return nil
@ -1920,7 +1965,7 @@ func showDoltConfig(testConnection bool) error {
if cfg == nil {
cfg = configfile.DefaultConfig()
}
if err := validateConfiguredBackend(cfg); err != nil {
if err := validateConfiguredBackend(cfg, beadsDir); err != nil {
return HandleError("%v", err)
}

View file

@ -0,0 +1,108 @@
//go:build cgo
package main
// The external-tcp half of the `bd dolt status` / `bd dolt start` coverage.
// It lives in the TestProxiedServer* lane, which runs on every PR, so the
// corruption guard is enforced there and not only in the path-filtered
// managed-local lane.
//
// The distinction this topology exercises: bd runs a proxy but spawns no dolt,
// so "the backend is not running" is a claim bd has no standing to make.
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/steveyegge/beads/internal/doltserver"
"github.com/steveyegge/beads/internal/storage/dbproxy/proxy"
)
func TestProxiedServerDoltLifecycle(t *testing.T) {
bd := buildEmbeddedBD(t)
p := newSharedProxiedProject(t, bd, "extdolt")
t.Cleanup(func() {
serverDir := doltserver.ResolveServerDir(p.beadsDir)
if _, err := os.Stat(filepath.Join(serverDir, doltserver.PIDFileName)); err != nil {
return
}
t.Logf("stray classic dolt-server.pid under %s; stopping it", serverDir)
if err := doltserver.Stop(serverDir); err != nil {
t.Logf("doltserver.Stop(%s): %v", serverDir, err)
}
})
bdProxiedCreate(t, bd, p.dir, "external lifecycle sentinel")
proxyPid := readProxyPidFileOrFail(t, p)
t.Run("status reports the proxy and disclaims the backend", func(t *testing.T) {
stdout, stderr, err := bdProxiedRunBuffersWithEnv(t, bd, p.dir, []string{"BEADS_JSON=1"}, "--json", "dolt", "status")
if err != nil {
t.Fatalf("bd dolt status failed: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr)
}
var got struct {
Mode string `json:"mode"`
Running bool `json:"running"`
ProxyPID int `json:"proxy_pid"`
BackendManaged bool `json:"backend_managed"`
BackendRunning bool `json:"backend_running"`
BackendEndpoint string `json:"backend_endpoint"`
}
if jsonErr := json.Unmarshal([]byte(strings.TrimSpace(stdout)), &got); jsonErr != nil {
t.Fatalf("decode dolt status JSON: %v\nstdout:\n%s", jsonErr, stdout)
}
if got.Mode != "proxied-server" || !got.Running {
t.Errorf("status=%+v, want mode=proxied-server running=true", got)
}
if got.ProxyPID != proxyPid {
t.Errorf("proxy_pid=%d, want %d", got.ProxyPID, proxyPid)
}
if got.BackendManaged {
t.Error("backend_managed=true on an external topology; bd spawns no dolt there")
}
if got.BackendRunning {
t.Error("backend_running=true on an external topology; bd has no record to claim that from")
}
if !strings.HasPrefix(got.BackendEndpoint, "127.0.0.1:") {
t.Errorf("backend_endpoint=%q, want the configured external endpoint", got.BackendEndpoint)
}
})
t.Run("start is refused", func(t *testing.T) {
stdout, stderr, err := bdProxiedRunBuffersWithEnv(t, bd, p.dir, []string{"BEADS_JSON=1"}, "--json", "dolt", "start")
if err == nil {
t.Errorf("bd dolt start succeeded on an external proxied workspace:\nstdout:\n%s\nstderr:\n%s", stdout, stderr)
}
var got struct {
Code string `json:"code"`
Mutates bool `json:"mutates"`
}
if jsonErr := json.Unmarshal([]byte(strings.TrimSpace(stdout)), &got); jsonErr != nil {
t.Fatalf("refusal is not typed JSON (%v)\nstdout:\n%s\nstderr:\n%s", jsonErr, stdout, stderr)
}
if got.Code != proxyDoltStartConflictCode || got.Mutates {
t.Errorf("refusal=%+v, want code=%q mutates=false", got, proxyDoltStartConflictCode)
}
for _, name := range []string{doltserver.PIDFileName, doltserver.PortFileName} {
if _, statErr := os.Stat(filepath.Join(p.beadsDir, name)); statErr == nil {
t.Errorf("%s exists after a refused start", name)
}
}
})
if out, err := bdProxiedRun(t, bd, p.dir, "create", "--json", "post-refusal write"); err != nil {
t.Fatalf("workspace unusable after the lifecycle probes: %v\n%s", err, out)
}
}
func readProxyPidFileOrFail(t *testing.T, p proxiedProject) int {
t.Helper()
running, pid := proxy.IsRunning(p.proxyRoot)
if !running {
t.Fatalf("expected a live proxy under %s after a create", p.proxyRoot)
}
return pid
}

View file

@ -0,0 +1,144 @@
package main
// `bd dolt start` and `bd dolt status` for proxied-server workspaces.
//
// Both commands predate proxied mode and assume bd owns the sql-server: they
// read and write .beads/dolt-server.pid. A proxied workspace never has one —
// the proxy writes proxy.pid and proxy-child.pid under the proxied root — so
// status used to report "not running" against a healthy backend, and start
// used to act on that by putting a second sql-server over the live data
// directory. This file is the proxied-aware half of both.
import (
"fmt"
"net"
"os"
"strconv"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/storage/dbproxy/proxy"
)
// Stable identity of the `bd dolt start` refusal on a proxied workspace.
// Codes are the part of the refusal contract downstream consumers branch on,
// so this one is frozen; the message beside it is informative.
const proxyDoltStartConflictCode = "proxy.dolt_start.conflict"
const proxyDoltStartConflictMessage = "dolt start is not supported in proxied-server mode: " +
"the proxy owns its dolt backend, and a second sql-server over the same data directory risks corrupting it; " +
"the proxy starts on demand — run 'bd dolt status' to see what is running, or 'bd dolt stop' to shut it down"
// proxiedDoltStartRefusal is enforced inside the command rather than by the
// pre-provider capability gate because `bd dolt` is a noDbCommands entry: its
// subcommands skip store init and so never reach the gate in main.go. Folding
// this row into the consolidated registry is S2's job; the corruption guard
// should not wait for it.
func proxiedDoltStartRefusal() *ProxyCapabilityError {
return &ProxyCapabilityError{
Code: proxyDoltStartConflictCode,
Message: proxyDoltStartConflictMessage,
ExitCode: 1,
}
}
// proxiedDoltStatus is the `bd dolt status --json` payload for a proxied
// workspace. It reports the two processes separately because they fail
// separately and an operator's next move differs: Running describes the proxy,
// which is the endpoint every bd command connects through, while the Backend*
// fields describe the dolt server behind it.
type proxiedDoltStatus struct {
Mode string `json:"mode"`
Root string `json:"root"`
Running bool `json:"running"`
ProxyPID int `json:"proxy_pid,omitempty"`
ProxyPort int `json:"proxy_port,omitempty"`
// BackendManaged is false on an external proxied topology, where the
// dolt server is somebody else's process and BackendRunning says nothing
// about it. Without this flag a caller cannot tell "the backend is down"
// from "bd never had a backend to report".
BackendManaged bool `json:"backend_managed"`
BackendRunning bool `json:"backend_running"`
BackendPID int `json:"backend_pid,omitempty"`
BackendPort int `json:"backend_port,omitempty"`
BackendEndpoint string `json:"backend_endpoint,omitempty"`
IdleTimeout string `json:"idle_timeout,omitempty"`
}
func collectProxiedDoltStatus(beadsDir string) (proxiedDoltStatus, error) {
root, err := resolveProxiedServerRootPath(beadsDir)
if err != nil {
return proxiedDoltStatus{}, err
}
live := proxy.ReadStatus(root)
status := proxiedDoltStatus{
Mode: configfile.DoltModeProxiedServer,
Root: root,
Running: live.ProxyRunning,
ProxyPID: live.ProxyPID,
ProxyPort: live.ProxyPort,
BackendManaged: true,
BackendRunning: live.BackendRunning,
BackendPID: live.BackendPID,
BackendPort: live.BackendPort,
}
info, err := configfile.LoadProxiedServerClientInfo(beadsDir)
if err != nil {
return proxiedDoltStatus{}, err
}
// A missing sidecar means the workspace predates it or was hand-edited;
// managed-local is the default topology, and the pid records already read
// above are the authority on what is actually running either way.
if info != nil {
if info.IdleTimeout > 0 {
status.IdleTimeout = info.IdleTimeout.String()
}
if info.External != nil {
status.BackendManaged = false
status.BackendEndpoint = externalDoltEndpoint(*info.External)
}
}
return status, nil
}
func externalDoltEndpoint(cfg configfile.ExternalDoltConfig) string {
if cfg.Socket != "" {
return "unix:" + cfg.Socket
}
return net.JoinHostPort(cfg.Host, strconv.Itoa(cfg.Port))
}
func renderProxiedDoltStatus(status proxiedDoltStatus) {
if jsonOutput {
if err := outputJSON(status); err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
}
return
}
if status.Running {
fmt.Printf("Dolt server: running (%s)\n", status.Mode)
fmt.Printf(" Proxy PID: %d\n", status.ProxyPID)
fmt.Printf(" Proxy port: %d\n", status.ProxyPort)
} else {
fmt.Printf("Dolt server: not running (%s)\n", status.Mode)
}
fmt.Printf(" Root: %s\n", status.Root)
switch {
case !status.BackendManaged:
fmt.Printf(" Backend: external at %s (not managed by bd)\n", status.BackendEndpoint)
case status.BackendRunning:
fmt.Printf(" Backend: running (dolt PID %d, port %d)\n", status.BackendPID, status.BackendPort)
default:
fmt.Println(" Backend: not running")
}
if status.IdleTimeout != "" {
fmt.Printf(" Idle timeout: %s\n", status.IdleTimeout)
}
if !status.Running {
fmt.Println(" The proxy starts on demand; the next bd command launches it.")
}
if isDoltLocalOnly() {
fmt.Println(" Remote sync: disabled (dolt.local-only=true)")
}
}

View file

@ -0,0 +1,187 @@
package main
import (
"encoding/json"
"path/filepath"
"strings"
"testing"
"time"
"github.com/steveyegge/beads/internal/configfile"
)
// TestProxiedDoltStartRefusalContract pins the parts downstream consumers
// branch on. The code is frozen; the message is informative and may be
// reworded, but it has to keep naming the mode and pointing somewhere useful,
// because "start refused" with no onward path is what sends an operator
// looking for a manual workaround.
func TestProxiedDoltStartRefusalContract(t *testing.T) {
err := proxiedDoltStartRefusal()
if err.Code != "proxy.dolt_start.conflict" {
t.Errorf("code = %q, want %q", err.Code, "proxy.dolt_start.conflict")
}
if err.ExitCode != 1 {
t.Errorf("exit code = %d, want 1", err.ExitCode)
}
if err.Mutates {
t.Error("mutates = true; a refused start changes nothing")
}
for _, want := range []string{"proxied-server mode", "bd dolt status", "bd dolt stop"} {
if !strings.Contains(err.Message, want) {
t.Errorf("message does not mention %q: %q", want, err.Message)
}
}
}
func TestExternalDoltEndpoint(t *testing.T) {
for _, tc := range []struct {
name string
cfg configfile.ExternalDoltConfig
want string
}{
{"tcp", configfile.ExternalDoltConfig{Host: "127.0.0.1", Port: 3310}, "127.0.0.1:3310"},
{"ipv6", configfile.ExternalDoltConfig{Host: "::1", Port: 3310}, "[::1]:3310"},
{"socket wins", configfile.ExternalDoltConfig{Socket: "/tmp/dolt.sock"}, "unix:/tmp/dolt.sock"},
} {
t.Run(tc.name, func(t *testing.T) {
if got := externalDoltEndpoint(tc.cfg); got != tc.want {
t.Errorf("externalDoltEndpoint = %q, want %q", got, tc.want)
}
})
}
}
// TestCollectProxiedDoltStatusQuiescentRoot covers the case an operator hits
// most: nothing is running yet. The answer has to be "not running" WITHOUT
// starting anything — a status command that spawns the topology it is
// reporting on would defeat the idle reaper it is reporting about.
func TestCollectProxiedDoltStatusQuiescentRoot(t *testing.T) {
beadsDir := t.TempDir()
root := filepath.Join(beadsDir, "dolt")
t.Setenv("BEADS_PROXIED_SERVER_ROOT_PATH", root)
status, err := collectProxiedDoltStatus(beadsDir)
if err != nil {
t.Fatalf("collectProxiedDoltStatus: %v", err)
}
if status.Mode != "proxied-server" {
t.Errorf("mode = %q, want %q", status.Mode, "proxied-server")
}
if status.Root != root {
t.Errorf("root = %q, want %q", status.Root, root)
}
if status.Running || status.BackendRunning {
t.Errorf("reported something running for an empty root: %+v", status)
}
if !status.BackendManaged {
t.Error("backend_managed = false with no external sidecar; managed-local is the default topology")
}
if entries, _ := filepath.Glob(filepath.Join(root, "*")); len(entries) != 0 {
t.Errorf("reading status created %v under the proxied root", entries)
}
}
func TestCollectProxiedDoltStatusExternalTopology(t *testing.T) {
beadsDir := t.TempDir()
root := filepath.Join(beadsDir, "dolt")
t.Setenv("BEADS_PROXIED_SERVER_ROOT_PATH", root)
if err := configfile.SaveProxiedServerClientInfo(beadsDir, &configfile.ProxiedServerClientInfo{
IdleTimeout: 45 * time.Second,
External: &configfile.ExternalDoltConfig{Host: "10.0.0.5", Port: 3306},
}); err != nil {
t.Fatalf("SaveProxiedServerClientInfo: %v", err)
}
status, err := collectProxiedDoltStatus(beadsDir)
if err != nil {
t.Fatalf("collectProxiedDoltStatus: %v", err)
}
if status.BackendManaged {
t.Error("backend_managed = true for an external topology; bd spawns no dolt there")
}
if status.BackendEndpoint != "10.0.0.5:3306" {
t.Errorf("backend_endpoint = %q, want %q", status.BackendEndpoint, "10.0.0.5:3306")
}
if status.IdleTimeout != "45s" {
t.Errorf("idle_timeout = %q, want %q", status.IdleTimeout, "45s")
}
}
func TestRenderProxiedDoltStatus(t *testing.T) {
running := proxiedDoltStatus{
Mode: "proxied-server", Root: "/w/.beads/dolt", Running: true,
ProxyPID: 111, ProxyPort: 40001,
BackendManaged: true, BackendRunning: true, BackendPID: 222, BackendPort: 40002,
IdleTimeout: "30s",
}
t.Run("text reports both processes", func(t *testing.T) {
defer restoreJSONOutput(t)()
jsonOutput = false
out := captureStdout(t, func() error { renderProxiedDoltStatus(running); return nil })
for _, want := range []string{
"Dolt server: running (proxied-server)",
"Proxy PID: 111",
"Proxy port: 40001",
"Root: /w/.beads/dolt",
"Backend: running (dolt PID 222, port 40002)",
"Idle timeout: 30s",
} {
if !strings.Contains(out, want) {
t.Errorf("missing %q in:\n%s", want, out)
}
}
})
t.Run("text says how to start it when down", func(t *testing.T) {
defer restoreJSONOutput(t)()
jsonOutput = false
down := running
down.Running, down.ProxyPID, down.ProxyPort = false, 0, 0
down.BackendRunning, down.BackendPID, down.BackendPort = false, 0, 0
out := captureStdout(t, func() error { renderProxiedDoltStatus(down); return nil })
for _, want := range []string{
"Dolt server: not running (proxied-server)",
"Backend: not running",
"The proxy starts on demand",
} {
if !strings.Contains(out, want) {
t.Errorf("missing %q in:\n%s", want, out)
}
}
})
t.Run("text marks an external backend as somebody else's", func(t *testing.T) {
defer restoreJSONOutput(t)()
jsonOutput = false
external := running
external.BackendManaged, external.BackendRunning = false, false
external.BackendEndpoint = "10.0.0.5:3306"
out := captureStdout(t, func() error { renderProxiedDoltStatus(external); return nil })
if !strings.Contains(out, "Backend: external at 10.0.0.5:3306 (not managed by bd)") {
t.Errorf("external backend not described:\n%s", out)
}
if strings.Contains(out, "Backend: not running") {
t.Errorf("external backend reported as down; bd has no record to say that from:\n%s", out)
}
})
t.Run("json carries the process records", func(t *testing.T) {
defer restoreJSONOutput(t)()
jsonOutput = true
out := captureStdout(t, func() error { renderProxiedDoltStatus(running); return nil })
var got proxiedDoltStatus
if err := json.Unmarshal([]byte(strings.TrimSpace(out)), &got); err != nil {
t.Fatalf("decode %q: %v", out, err)
}
if got != running {
t.Errorf("round-trip mismatch:\ngot %+v\nwant %+v", got, running)
}
})
}
func restoreJSONOutput(t *testing.T) func() {
t.Helper()
orig := jsonOutput
return func() { jsonOutput = orig }
}

View file

@ -8,9 +8,9 @@ import (
"path/filepath"
"runtime"
"strconv"
"strings"
"testing"
"github.com/steveyegge/beads/internal/config"
"github.com/steveyegge/beads/internal/testutil"
)
@ -32,6 +32,12 @@ func TestDoltRemoteAddPersistsSyncRemoteToSharedWorktreeConfig(t *testing.T) {
"BEADS_DOLT_SERVER_PORT="+strconv.Itoa(port),
)
// `bd init` in shared-server mode DAEMONIZES a dolt sql-server under
// $HOME/.beads/shared-server and returns; nothing here used to stop it,
// so it outlived the suite that deleted its tree (wy-j2zc8q). Registered
// before the first subprocess so the t.Fatalf paths are covered too.
stopSharedServerCleanup(t)
initCmd := exec.Command(bd, "init", "--prefix", "remote-sync", "--skip-hooks", "--quiet")
initCmd.Dir = worktreeDir
initCmd.Env = sharedEnv
@ -47,13 +53,21 @@ func TestDoltRemoteAddPersistsSyncRemoteToSharedWorktreeConfig(t *testing.T) {
t.Fatalf("bd dolt remote add from bare-parent worktree failed: %v\n%s", err, out)
}
// Asserted by reading it back, not by matching YAML text. The literal
// `sync.remote: "<url>"` this used to require is a key whose NAME contains
// a dot — the shape config.GetStringFromDir can never find, because it
// splits on the dot and walks nested mappings. Viper finds a dotted key
// either way, so this assertion was green while the value was invisible to
// bd's other reader; it passed BECAUSE bd-zj95 was present. What the remote
// add owes its caller is a remote that reads back, so that is what this
// checks.
configPath := filepath.Join(bareBeadsDir, "config.yaml")
content, err := os.ReadFile(configPath)
if err != nil {
t.Fatalf("failed to read shared config.yaml: %v", err)
}
if !strings.Contains(string(content), `sync.remote: "`+remoteURL+`"`) {
t.Fatalf("expected shared config.yaml to contain sync.remote, got:\n%s", string(content))
if got := config.GetStringFromDir(bareBeadsDir, "sync.remote"); got != remoteURL {
t.Fatalf("shared config.yaml sync.remote = %q, want %q; contents:\n%s", got, remoteURL, content)
}
if _, err := os.Stat(filepath.Join(worktreeDir, ".beads")); !os.IsNotExist(err) {

View file

@ -28,8 +28,8 @@ Examples:
var supersedeCmd = &cobra.Command{
Use: "supersede <id> --with <new>",
GroupID: "deps",
Short: "Mark an issue as superseded by a newer one",
Long: `Mark an issue as superseded by a newer version.
Short: "Mark an issue as superseded by a replacement issue",
Long: `Mark an issue as superseded by a different (replacement) issue.
The superseded issue is automatically closed with a reference to the replacement.
Useful for design docs, specs, and evolving artifacts.

View file

@ -6,8 +6,12 @@ import (
"fmt"
"os"
"github.com/spf13/cobra"
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/metrics"
"github.com/steveyegge/beads/internal/migration"
"github.com/steveyegge/beads/internal/ui"
)
type exitError struct {
@ -72,6 +76,52 @@ func buildJSONError(message, hint string) interface{} {
return inner
}
// buildJSONCapabilityError renders a typed refusal. {code, error, mutates} is a
// frozen contract that downstream consumers parse; `reason` is additive and
// appears only on refusals that came from the capability registry, which is why
// it is omitted rather than emitted empty — a consumer that keys on its presence
// must not be told "" by a runtime state error that has no policy reason.
func buildJSONCapabilityError(e *ProxyCapabilityError) interface{} {
inner := map[string]interface{}{"error": e.Message, "code": e.Code, "mutates": e.Mutates}
if e.Reason != "" {
inner["reason"] = string(e.Reason)
}
if jsonEnvelopeEnabled() {
return map[string]interface{}{"schema_version": JSONSchemaVersion, "data": inner}
}
inner["schema_version"] = JSONSchemaVersion
return inner
}
// HandleProxyCapabilityError renders a stable capability refusal while
// preserving the normal text/JSON front-door conventions.
//
// The allow path of every AssertProxy*Capability returns nil, and this wraps
// that return value directly, so nil must pass through: without the fast path
// errors.As(nil, ...) is false and an allowed capability would render as
// "Error: <nil>" with exit 1 at a front door nothing downstream catches.
func HandleProxyCapabilityError(err error) error {
if err == nil {
return nil
}
var capErr *ProxyCapabilityError
if !errors.As(err, &capErr) {
return HandleErrorRespectJSON("%v", err)
}
code := capErr.ExitCode
if code == 0 {
code = 1
}
if jsonOutput {
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
_ = enc.Encode(buildJSONCapabilityError(capErr))
} else {
fmt.Fprintf(os.Stderr, "Error: %s\n", capErr.Message)
}
return &exitError{Code: code}
}
func jsonStderrError(message, hint string) {
encoder := json.NewEncoder(os.Stderr)
encoder.SetIndent("", " ")
@ -125,44 +175,75 @@ func WarnError(format string, args ...interface{}) {
fmt.Fprintf(os.Stderr, "Warning: "+format+"\n", args...)
}
// ExitMigrationFrozen is the exit code for a write refused because a
// migration freeze marker is active (dc-6jaq). Stable value so scripts can
// branch on "someone is migrating this workspace, come back later" without
// grep'ing stderr, instead of reading it as a generic failure (1) worth
// retrying immediately.
const ExitMigrationFrozen = 14
// CheckReadonly aborts the command when bd is running in read-only mode (the
// worker-sandbox posture, see readonlyMode), or when a MIGRATION-FREEZE
// sentinel is active at the town root (dc-6jaq, folded in here rather than
// requiring every write command to remember a second call — see
// CheckMigrationFreeze below). This is the chokepoint essentially every
// write command already calls first, so folding the freeze check in here
// covers the whole write surface at once, including commands added after
// this comment is written — not just a hand-picked list that rots (one
// concrete instance the hand-picked list missed: "bd q", cmd/bd/quick.go,
// the create alias). It exits via os.Exit and so cannot run the per-command
// deferred CloseEventAndAdd — a command blocked here records no
// cli_command event of its own (it never actually ran). It does flush
// metrics first, so events already queued earlier in this run are still
// written and scheduled for upload rather than stranded until the next
// clean exit.
// worker-sandbox posture, see readonlyMode), or when a migration freeze
// marker is active (dc-6jaq, folded in here rather than requiring every write
// command to remember a second call — see migrationFreezeError below). This
// is the chokepoint essentially every write command already calls first, so
// folding the freeze check in here covers the whole write surface at once,
// including commands added after this comment is written — not just a
// hand-picked list that rots (one concrete instance the hand-picked list
// missed: "bd q", cmd/bd/quick.go, the create alias). It exits via os.Exit
// and so cannot run the per-command deferred CloseEventAndAdd — a command
// blocked here records no cli_command event of its own (it never actually
// ran). It does flush metrics first, so events already queued earlier in this
// run are still written and scheduled for upload rather than stranded until
// the next clean exit. Callers that CAN return an error (the root
// PersistentPreRunE, runImport) call migrationFreezeError directly instead,
// so their deferred cleanup still runs.
func CheckReadonly(operation string) {
if readonlyMode {
fmt.Fprintf(os.Stderr, "Error: operation '%s' is not allowed in read-only mode\n", operation)
metrics.CloseAndFlush()
os.Exit(1)
}
CheckMigrationFreeze(operation)
if err := migrationFreezeError(operation); err != nil {
metrics.CloseAndFlush()
os.Exit(ExitMigrationFrozen)
}
}
// CheckMigrationFreeze aborts the command when a MIGRATION-FREEZE sentinel is
// present at the town root (dc-6jaq). The gt CLI already refuses to write
// under the same sentinel (gt mail send, gt nudge, gt sling, gt assign);
// mail-poller/daemon patrols that also write via bd are stopped separately
// via the migration playbook's plist-unload step, so this closes the
// remaining gap: a human typing 'bd create'/'bd update' etc. mid-migration,
// bypassing the gt-layer gate. Same exit-via-os.Exit tradeoff as
// CheckReadonly above — no cli_command event for a command blocked here,
// but queued metrics are still flushed.
// freezeSearchRoots returns the directories whose ancestry the freeze lookup
// must cover in addition to the working directory: the resolved workspace, so
// the gate keys on the store being written rather than on the caller's shell.
// Without it, `BEADS_DIR=/work/repo/.beads bd create` run from $HOME, `bd -C
// /work/repo create` (which sets BEADS_DIR and never chdirs), and any daemon
// or cron job with an unrelated cwd all walk the wrong tree and write straight
// through a freeze.
//
// PersistentPreRunE exports BEADS_DIR as part of selecting the workspace
// (prepareSelectedCommandContext), so by the time a command's RunE reaches
// CheckReadonly this is a cheap env read. The FindBeadsDir fallback covers the
// callers that run before that — and returns "" rather than erroring when
// there is no workspace at all, which Find skips.
func freezeSearchRoots() []string {
if dir := os.Getenv("BEADS_DIR"); dir != "" {
return []string{dir}
}
if dir := beads.FindBeadsDir(); dir != "" {
return []string{dir}
}
return nil
}
// migrationFreezeError reports the active migration freeze, if any: it
// returns nil when writes are not frozen, and otherwise prints the refusal to
// stderr and returns an ExitMigrationFrozen exit error. The refusal names the
// marker's own path, so the recovery instruction ("remove that file") is
// always actionable without knowing what created it.
//
// Three callers, each closing a different hole:
// 1. CheckReadonly above — the per-command chokepoint covering the write
// surface at large (create/update/close/... and everything that calls
// CheckReadonly, ~120 sites).
// CheckReadonly, ~120 sites). Only that caller turns this error into an
// os.Exit, because its void signature leaves it nothing else to do.
// 2. The root PersistentPreRunE in main.go, before autoMigrateOnVersionBump
// and maybeAutoImportJSONL — those run as store-open side effects
// *before* any RunE, so waiting for a write command's own CheckReadonly
@ -173,25 +254,118 @@ func CheckReadonly(operation string) {
// CheckReadonly at all today (a pre-existing, separate gap: readonlyMode
// doesn't block it either), so it cannot inherit the freeze check from
// caller 1 and needs its own explicit call.
func CheckMigrationFreeze(operation string) {
townRoot := findTownRoot()
if !migration.IsFrozen(townRoot) {
return
func migrationFreezeError(operation string) error {
return migrationFreezeRefusal(operation, migration.Find(freezeSearchRoots()...))
}
// migrationFreezeRefusal renders an already-resolved lookup. Callers that
// need the same Result for more than the refusal — PersistentPreRunE, which
// also derives frozenForMaintenance from it — resolve once and pass it here,
// so a single walk answers the whole invocation and the two readings cannot
// disagree about a marker that appeared or vanished between them.
func migrationFreezeRefusal(operation string, res migration.Result) error {
if !res.Frozen() {
return nil
}
info := migration.Read(townRoot)
operator := "unknown"
if res.Err != nil {
// Fail closed: bd could not tell whether a marker is present (a
// permission error on the marker or a directory above it). An
// undeterminable gate is not an open gate, and saying so beats
// writing into a workspace that may be mid-migration.
fmt.Fprint(os.Stderr, "⛔ ERROR: cannot determine whether this workspace is frozen for migration.\n")
fmt.Fprintf(os.Stderr, " %s\n", ui.SanitizeForTerminal(res.Err.Error()))
fmt.Fprintf(os.Stderr, " bd %s is blocked until the marker can be read.\n", operation)
fmt.Fprintf(os.Stderr, " Fix the permissions, or set %s to a path bd can stat.\n", migration.EnvFreezeFile)
return &exitError{Code: ExitMigrationFrozen}
}
// The marker can live in a directory bd does not control, so its payload
// is untrusted input on its way to a terminal. parse already keeps it to
// one line; sanitizing strips ANSI and control bytes on top of that.
operator := ""
reason := ""
if info != nil {
operator = info.Operator
reason = info.Reason
if info := migration.ReadFile(res.Path); info != nil {
operator = ui.SanitizeForTerminal(info.Operator)
reason = ui.SanitizeForTerminal(info.Reason)
}
fmt.Fprintf(os.Stderr, "⛔ ERROR: town is frozen for migration (by %s).\n", operator)
if operator != "" {
fmt.Fprintf(os.Stderr, "⛔ ERROR: workspace is frozen for migration (by %s).\n", operator)
} else {
// An empty or unparseable marker (a bare `touch`) records no
// operator; say nothing rather than printing "(by )".
fmt.Fprint(os.Stderr, "⛔ ERROR: workspace is frozen for migration.\n")
}
if reason != "" {
fmt.Fprintf(os.Stderr, " Reason: %s\n", reason)
}
fmt.Fprintf(os.Stderr, " bd %s is blocked. Clear the freeze: gt migrate thaw\n", operation)
metrics.CloseAndFlush()
os.Exit(1)
fmt.Fprintf(os.Stderr, " bd %s is blocked by the freeze marker at %s.\n", operation, res.Path)
if res.FromEnv {
fmt.Fprintf(os.Stderr, " To resume writes, remove that file or unset %s.\n", migration.EnvFreezeFile)
} else {
// Name the variable on this path too: when the marker sits in a
// directory the caller cannot write to, pointing the variable
// elsewhere is the only recovery available to them.
fmt.Fprint(os.Stderr, " To resume writes, remove that file.\n")
fmt.Fprintf(os.Stderr, " If you cannot remove it, set %s to a path bd should check instead.\n", migration.EnvFreezeFile)
}
return &exitError{Code: ExitMigrationFrozen}
}
// migrationFreezeGate is the RunE-side entry point: it renders the refusal and,
// because the refusal has already said everything worth saying, suppresses
// cobra's own error and usage rendering for this invocation.
//
// Without the suppression a freeze refusal on any command that does not set
// SilenceErrors/SilenceUsage statically — duplicate, supersede, dep
// relate/unrelate, backup init/sync/remove/restore, ado sync, migrate-personal,
// batch — is followed by "Error: exit code 14" (exitError's placeholder text)
// and a full usage dump, so a clean refusal reads like a syntax error. Setting
// the flags here rather than on each command covers the ones added after this
// comment too. main() exits on the exitError before its own SilenceErrors
// branch, so nothing prints the placeholder.
func migrationFreezeGate(cmd *cobra.Command, operation string, res migration.Result) error {
err := migrationFreezeRefusal(operation, res)
if err != nil && cmd != nil {
cmd.SilenceErrors = true
cmd.SilenceUsage = true
}
return err
}
// migrationFreezeGateFor resolves the freeze and gates in one step, for the
// callers that have no Result to thread: the skip-store commands (bd init,
// bd bootstrap), which return from PersistentPreRunE long before it resolves
// one, and any command handed an explicit target path.
func migrationFreezeGateFor(cmd *cobra.Command, operation string, extraRoots ...string) error {
return migrationFreezeGate(cmd, operation, migration.Find(freezeRootsWith(extraRoots)...))
}
// migrationFreezeErrorFor is migrationFreezeError against extra roots, for
// callers that hold a target path but no *cobra.Command to silence — a command
// given a workspace to operate on that is not the one it was launched in.
func migrationFreezeErrorFor(operation string, extraRoots ...string) error {
return migrationFreezeRefusal(operation, migration.Find(freezeRootsWith(extraRoots)...))
}
// migrationFreezeActive is the print-nothing probe for code that must SKIP a
// write during a freeze rather than refuse the whole command — the diagnosis
// paths that keep running while frozen but must not apply version tracking or
// schema auto-migration.
func migrationFreezeActive() bool {
return migrationFreezeActiveFor()
}
// migrationFreezeActiveFor is migrationFreezeActive against extra roots — the
// probe half of the migrationFreezeErrorFor pair, for a diagnosis path handed a
// workspace that is not the one bd was launched in. `bd doctor /frozen/repo`
// skips its own maintenance writes on the strength of this, so it has to look
// at the tree it is about to write to and not just at the caller's cwd.
func migrationFreezeActiveFor(extraRoots ...string) bool {
return migration.Find(freezeRootsWith(extraRoots)...).Frozen()
}
func freezeRootsWith(extraRoots []string) []string {
return append(freezeSearchRoots(), extraRoots...)
}

View file

@ -10,6 +10,7 @@ import (
"github.com/spf13/cobra"
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/config"
"github.com/steveyegge/beads/internal/eventsjournal"
"github.com/steveyegge/beads/internal/storage"
@ -103,10 +104,12 @@ Record contract (stable for external consumers):
issue_id string the mutated issue's id
actor string the acting identity that performed the mutation, as resolved
for the audit-events table (on a comment row: the comment's
author); empty (omitted) when the mutation path has no
actor — derived maintenance, deletes (other than a rename's
synthetic delete), and rows older than the column. Never
user attribution when empty.
author). A delete — and the dep_remove rows a cascading
delete produces — carries the identity that REQUESTED it.
Empty (omitted) only when the path genuinely has no actor:
derived maintenance, system cleanup with no request behind
it, and rows older than the column. Never user attribution
when empty.
issue object full issue state AFTER the mutation; null on delete
dep object {"kind","target","metadata"} for dep_add / dep_remove; omitted otherwise
comment object {"id","author","text","created_at","source"} for comment; omitted otherwise
@ -246,7 +249,32 @@ func reportEventsTruncated(err error, streaming bool) error {
fmt.Sprintf("resume with --since %d to continue from the oldest retained record (accepting the gap), or re-import from scratch", trunc.Floor-1))
}
// noticeEventsJournalDisabled warns, on stderr, that this workspace is not
// recording — so an empty or stale read is not mistaken for "caught up".
//
// It exists because zero rows is INDISTINGUISHABLE from a journal that has
// nothing new: a consumer polling a workspace that never enabled the journal
// sees a successful, empty read forever and concludes it is up to date. The
// HTTP API refuses that read outright (EventsJournalDisabled, 409); the CLI
// cannot, because exporting the ledger a now-disabled workspace recorded while
// it WAS enabled is legitimate. So it serves the rows and says what it is
// serving.
//
// stderr, not stdout, and not the JSON envelope: stdout is a JSONL stream a
// line reader consumes, and a notice about the reader's own configuration is
// not a record. Exit status is unchanged.
func noticeEventsJournalDisabled(beadsDir string) {
if eventsjournal.EnabledFor(beadsDir) {
return
}
fmt.Fprintf(os.Stderr,
"note: the events journal is disabled for this workspace (enable with 'bd config set %s true'); "+
"any records shown were written while it was enabled, and new mutations are not being recorded\n",
eventsjournal.ConfigKey)
}
func runEventsTail(ctx context.Context, since int64, limit int, follow bool) error {
noticeEventsJournalDisabled(beads.FindBeadsDir())
enc := json.NewEncoder(os.Stdout)
emit := func(from int64) (int64, error) {
rows, err := readJournal(ctx, from, limit)

View file

@ -0,0 +1,129 @@
//go:build cgo && unix
package main
import (
"bytes"
"os"
"os/exec"
"strings"
"testing"
)
// The two halves of #5985 at the front door: what `bd delete --actor` records,
// and what `bd events export` tells you when there is nothing to record into.
// Both are end-to-end because both failures are invisible from inside the
// process — the command succeeds either way, and only a later read shows the
// attribution missing or the reader polling a journal that was never on.
// runBDOut runs bd in dir and returns stdout and stderr separately. The notice
// under test is on stderr precisely so it does not corrupt the JSONL on stdout,
// so a CombinedOutput helper cannot check it.
func runBDOut(t *testing.T, bd, dir string, env []string, args ...string) (string, string) {
t.Helper()
cmd := exec.Command(bd, args...)
cmd.Dir = dir
cmd.Env = env
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
t.Fatalf("bd %s in %s: %v\nstdout:\n%s\nstderr:\n%s",
strings.Join(args, " "), dir, err, stdout.String(), stderr.String())
}
return stdout.String(), stderr.String()
}
// TestDeleteJournalsTheRequestingActor is #5985's repro, verbatim: create two
// beads with an edge between them, delete the blocker with an explicit --actor,
// and read the journal back. Both the delete record and the cascade dep_remove
// record must name that actor.
//
// The whole route is exercised — cobra flag to journal column — because the
// actor is resolved, threaded through the role, and written by the storage
// seam, and a unit test at any one of those cannot see the drop at the next.
func TestDeleteJournalsTheRequestingActor(t *testing.T) {
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests")
}
t.Parallel()
bd := buildEmbeddedBD(t)
dir, _, _ := bdInit(t, bd, "--prefix", "dja", "--skip-hooks", "--skip-agents")
env := append(envWithout(bdEnv(dir), "BD_EVENTS_JOURNAL"), "BD_EVENTS_JOURNAL=1")
run := func(args ...string) string {
t.Helper()
stdout, _ := runBDOut(t, bd, dir, env, args...)
return stdout
}
run("create", "--silent", "--id", "dja-blocker", "the blocker")
run("create", "--silent", "--id", "dja-dependent", "the dependent")
run("dep", "add", "dja-dependent", "dja-blocker")
// --force is what makes this a real delete rather than a preview, and it is
// also what lets it orphan the dependent instead of refusing.
run("delete", "dja-blocker", "--force", "--actor", "alice")
records := decodeEventRecords(t, run("events", "export"))
var sawDelete, sawCascadeDepRemove bool
for _, rec := range records {
switch {
case rec.Op == "delete" && rec.IssueID == "dja-blocker":
sawDelete = true
if rec.Actor != "alice" {
t.Errorf("delete record actor = %q, want %q (#5985)", rec.Actor, "alice")
}
// The cascade edge removal is journaled under the edge's SOURCE — the
// surviving dependent — and belongs to the identity that asked for the
// delete, not to whoever created the edge.
case rec.Op == "dep_remove" && rec.IssueID == "dja-dependent":
sawCascadeDepRemove = true
if rec.Actor != "alice" {
t.Errorf("cascade dep_remove record actor = %q, want %q (#5985)", rec.Actor, "alice")
}
}
}
if !sawDelete {
t.Errorf("no delete record for dja-blocker: %+v", records)
}
if !sawCascadeDepRemove {
t.Errorf("no cascade dep_remove record for dja-dependent: %+v", records)
}
}
// TestEventsReadNoticesADisabledJournal pins the "nothing to read and no way to
// tell why" fix. A workspace that never enabled the journal serves an empty
// export forever, which a consumer cannot distinguish from being caught up, so
// the read says so on stderr — and keeps stdout clean and the exit status zero,
// because exporting a historical ledger from a now-disabled workspace is
// legitimate.
func TestEventsReadNoticesADisabledJournal(t *testing.T) {
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests")
}
t.Parallel()
bd := buildEmbeddedBD(t)
dir, _, _ := bdInit(t, bd, "--prefix", "dnn", "--skip-hooks", "--skip-agents")
off := envWithout(bdEnv(dir), "BD_EVENTS_JOURNAL")
for _, sub := range []string{"export", "tail"} {
stdout, stderr := runBDOut(t, bd, dir, off, "events", sub)
if !strings.Contains(stderr, "events journal is disabled") {
t.Errorf("bd events %s on a disabled workspace printed no notice\nstderr:\n%s", sub, stderr)
}
if !strings.Contains(stderr, "events-journal") {
t.Errorf("bd events %s notice does not name the setting to flip\nstderr:\n%s", sub, stderr)
}
if strings.TrimSpace(stdout) != "" {
t.Errorf("bd events %s put the notice on stdout, which carries the JSONL stream:\n%s", sub, stdout)
}
}
// Enabled: no notice, so the message stays a signal rather than noise every
// consumer learns to filter out.
on := append(envWithout(bdEnv(dir), "BD_EVENTS_JOURNAL"), "BD_EVENTS_JOURNAL=1")
if _, stderr := runBDOut(t, bd, dir, on, "events", "export"); strings.Contains(stderr, "events journal is disabled") {
t.Errorf("an enabled workspace still printed the disabled notice:\n%s", stderr)
}
}

View file

@ -153,8 +153,14 @@ func renderGoldenLines(t *testing.T) []byte {
// actor pins both halves of the attribution contract: an attributed row
// emits the acting identity (the same one the audit-events table resolves),
// and an actorless row — derived maintenance, actorless delete plumbing,
// and an actorless row — derived maintenance, the system delete surfaces,
// records from before the column existed — omits the member entirely.
//
// A `delete` appears on BOTH sides on purpose. A user's `bd delete --actor A`
// is attributed to A, cascade rows included; the actorless surfaces (gc,
// molecule cleanup, `bd repo remove`) still emit none. A consumer that read
// the empty case as "deletes are never attributed" — which the fixture used
// to be the only evidence for — would mis-resolve every delete conflict.
records := []eventsjournal.Record{
goldenRecord(1, ts, string(issueops.EventCreate), minimal.ID, "author", mustJSON(t, minimal), "", ""),
goldenRecord(2, ts, string(issueops.EventCreate), full.ID, "author", mustJSON(t, full), "", ""),
@ -169,7 +175,11 @@ func renderGoldenLines(t *testing.T) []byte {
goldenRecord(8, ts, string(issueops.EventCommentWrite), claimed.ID, "worker-1", mustJSON(t, claimed), "", mustJSON(t, auditComment)),
goldenRecord(9, ts, string(issueops.EventClose), closedIssue.ID, "worker-1", mustJSON(t, closedIssue), "", ""),
goldenRecord(10, ts, string(issueops.EventCreate), wisp.ID, "author", mustJSON(t, wisp), "", ""),
goldenRecord(11, ts, string(issueops.EventDelete), "bd-100", "", "", "", ""), // null issue on delete; actorless delete plumbing
// The cascade edge removal a delete emits, attributed to the requester.
goldenRecord(11, ts, string(issueops.EventDepRemove), "bd-101", "deleter-1", mustJSON(t, full),
mustJSON(t, &issueops.EventDep{Kind: string(types.DepBlocks), Target: "bd-100"}), ""),
goldenRecord(12, ts, string(issueops.EventDelete), "bd-100", "deleter-1", "", "", ""), // null issue on delete; requested delete
goldenRecord(13, ts, string(issueops.EventDelete), "bd-wisp-1", "", "", "", ""), // actorless system delete surface
}
var buf bytes.Buffer

View file

@ -109,8 +109,13 @@ func TestRoutedCreateJournalsIntoTheTargetWorkspace(t *testing.T) {
}
// The launching workspace never enabled the journal, so it records nothing
// — the target's setting must not leak back, in either direction.
if got := strings.TrimSpace(run(sourceDir, "events", "export")); got != "" {
// — the target's setting must not leak back, in either direction. Read
// STDOUT only: a disabled workspace also prints the "journal is disabled"
// notice, and that notice is on stderr precisely so it cannot be mistaken
// for a record.
sourceEnv := envWithout(bdEnv(sourceDir), "BD_EVENTS_JOURNAL")
stdout, _ := runBDOut(t, bd, sourceDir, sourceEnv, "events", "export")
if got := strings.TrimSpace(stdout); got != "" {
t.Errorf("the launching workspace journaled %q despite never enabling the journal", got)
}
}

View file

@ -58,6 +58,7 @@ var storeConstructors = map[string]map[string]bool{
"OpenForReadOnlyCommand": true,
"OpenForPreviewCommand": true,
"OpenForWorkingSetReconcile": true,
"OpenForRemoteSync": true,
},
"github.com/steveyegge/beads/internal/storage/uow": {
"NewDoltServerUOWProvider": true,

View file

@ -11,6 +11,7 @@ import (
"path/filepath"
"sort"
"strings"
"sync"
"time"
"github.com/steveyegge/beads/internal/atomicfile"
@ -39,8 +40,27 @@ const doltWorkingRef = "WORKING"
// exceeds this prints a one-line stderr tip pointing at the fix levers.
const slowExportWarnThreshold = 3 * time.Second
// exportAutoStateVersion is the format version stamped into
// .beads/export-state.json. BUMP IT whenever the MEANING of a persisted
// field changes — LastDiffAnchor's semantics above all — so a newer binary's
// state file is discarded by an older one instead of being reinterpreted.
// Adding a purely additive optional field does not need a bump.
//
// Discarding is only safe because of the deletion proofs in
// proveDeletedIssueIDs: dropping the state drops the diff anchor, and before
// GH#5896 an absent anchor plus a pending delete was itself a permanent
// wedge, which made "invalidate the state file" a cure worse than the
// disease. Now an anchorless cycle just proves what it can from history and
// re-anchors on the way out.
const exportAutoStateVersion = 1
// exportAutoState tracks auto-export state to avoid redundant work.
type exportAutoState struct {
// FormatVersion is exportAutoStateVersion at write time. Absent (0) means
// a file written before versioning existed; those are field-for-field
// identical to v1 and load as-is.
FormatVersion int `json:"v,omitempty"`
LastDoltCommit string `json:"last_dolt_commit"`
Timestamp time.Time `json:"timestamp"`
Issues int `json:"issues"`
@ -128,21 +148,31 @@ func maybeAutoExport(ctx context.Context, allowEmptyOverwrite bool) error {
return nil
}
// Reconcile the existing JSONL against the store ONCE, and let BOTH
// fail-safe guards below read the same answer. A deletion this store can
// prove is not divergence for either of them: the empty-overwrite guard
// fires first, so without a shared verdict `bd delete <last issue>`
// wedges there before the orphan guard ever gets to prove anything
// (GH#5896).
var provenDeleted []string
if !allowEmptyOverwrite {
if skip, existingCount, err := shouldSkipEmptyAutoExport(ctx, fullPath); err != nil {
rec, err := reconcileAutoExportJSONL(ctx, fullPath, state.LastDiffAnchor)
if err != nil {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: failed to compare existing JSONL against local store: %v\n", err)
return nil
}
provenDeleted = rec.provenDeleted
if skip, existingCount, err := shouldSkipEmptyAutoExport(ctx, rec); err != nil {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: failed to check existing JSONL: %v\n", err)
return nil
} else if skip {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: current database would export 0 issues, but %s already contains %d issue(s); refusing to overwrite. Run `bd init --from-jsonl` to import the JSONL file, or move it aside and retry.\n", fullPath, existingCount)
return nil
}
}
if !allowEmptyOverwrite {
if missingIDs, err := missingJSONLIssueIDsInStore(ctx, fullPath, state.LastDiffAnchor); err != nil {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: failed to compare existing JSONL against local store: %v\n", err)
return nil
} else if len(missingIDs) > 0 {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: %s contains %d JSONL-only issue record(s) absent from the local Dolt store (%s); refusing to overwrite. Run `bd init --from-jsonl` to import the JSONL file, or move it aside and retry.\n", fullPath, len(missingIDs), strings.Join(sampleStrings(missingIDs, 5), ", "))
if len(rec.unproven) > 0 {
fmt.Fprintf(os.Stderr, "Warning: auto-export skipped: %s contains %d JSONL-only issue record(s) absent from the local Dolt store (%s); refusing to overwrite. Run `bd init --from-jsonl` to import the JSONL file, or move it aside and retry.\n", fullPath, len(rec.unproven), strings.Join(sampleStrings(rec.unproven, 5), ", "))
return nil
}
}
@ -167,8 +197,14 @@ func maybeAutoExport(ctx context.Context, allowEmptyOverwrite bool) error {
debug.Logf("auto-export: failed to resolve diff anchor (%v); preserving previous anchor\n", anchorErr)
anchorCommit = state.LastDiffAnchor
}
// provenDeleted rides along as FORCED removals: an id proved by the
// in-process handoff or by store history is invisible to
// dolt_diff(anchor, WORKING), so an incremental patch would faithfully
// preserve its stale JSONL line forever even though the guard just
// cleared the export to run. A full export needs nothing — it rewrites
// from the store.
issueCount, memoryCount, dirtyIDs, didIncremental, err := tryIncrementalExport(
ctx, fullPath, state.LastDiffAnchor, doltWorkingRef, state.LastDirtyIDs,
ctx, fullPath, state.LastDiffAnchor, doltWorkingRef, state.LastDirtyIDs, provenDeleted,
)
if err != nil {
debug.Logf("auto-export: incremental failed (%v); falling back to full\n", err)
@ -182,6 +218,13 @@ func maybeAutoExport(ctx context.Context, allowEmptyOverwrite bool) error {
dirtyIDs = nil
}
// Never heal silently. The user asked for a delete and is getting a line
// removed from a git-tracked file; say so, and name what went.
if len(provenDeleted) > 0 {
fmt.Fprintf(os.Stderr, "auto-export: reconciled %d deleted issue(s) out of %s (deleted from store: %s)\n",
len(provenDeleted), fullPath, strings.Join(sampleStrings(provenDeleted, 5), ", "))
}
if dur := time.Since(exportStart); dur > slowExportWarnThreshold && !didIncremental {
// Only warn on full exports — the whole point of incremental is to
// get us under the threshold, so a slow incremental is noise.
@ -265,12 +308,15 @@ func shouldExport(state *exportAutoState, interval time.Duration) bool {
return time.Since(state.Timestamp) >= interval
}
func shouldSkipEmptyAutoExport(ctx context.Context, path string) (bool, int, error) {
existingCount, err := countIssueRecordsInJSONL(path)
if err != nil {
return false, 0, err
}
if existingCount == 0 {
// shouldSkipEmptyAutoExport is the "store would write 0 issues over a
// populated file" fail-safe. It reads the shared reconciliation rather than
// re-parsing the JSONL so that ids already proven deleted don't count as
// records worth protecting — deleting the last exported issue is the shape
// that used to wedge here, one guard earlier than the orphan guard everyone
// blamed (GH#5896).
func shouldSkipEmptyAutoExport(ctx context.Context, rec *autoExportReconciliation) (bool, int, error) {
existingCount := rec.jsonlIssueCount - len(rec.provenDeleted)
if existingCount <= 0 {
return false, 0, nil
}
@ -282,15 +328,37 @@ func shouldSkipEmptyAutoExport(ctx context.Context, path string) (bool, int, err
return len(issues) == 0, existingCount, nil
}
func countIssueRecordsInJSONL(path string) (int, error) {
ids, err := issueIDsInJSONL(path)
if err != nil {
return 0, err
}
return len(ids), nil
// historyProofMaxCandidates caps how many JSONL-only ids the store-history
// proof will ask about in one export. A divergence wider than this is not a
// delete pattern — it is GH#4988 territory (a torn, stale, or foreign JSONL),
// where the conservative refusal is the right answer and a 100-way history
// scan is wasted work on the way to it.
const historyProofMaxCandidates = 100
// autoExportReconciliation is the single comparison of the existing JSONL
// against the live store that both pre-export guards read.
type autoExportReconciliation struct {
// jsonlIssueCount is every issue record in the file, in auto-export
// scope or not — the number the empty-overwrite guard reports.
jsonlIssueCount int
// provenDeleted are in-scope JSONL ids absent from the store that some
// proof source accounted for as real deletions. The export must drop
// their lines; neither guard may refuse over them.
provenDeleted []string
// unproven are in-scope JSONL ids absent from the store that nothing
// could account for. The orphan guard refuses over these, unchanged.
unproven []string
}
func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) ([]string, error) {
// reconcileAutoExportJSONL compares the on-disk export against the store and
// splits the JSONL-only issue ids into proven deletions and unexplained
// divergence.
//
// fromCommit is the previous cycle's diff anchor, used by one of the three
// proof sources; the other two need no anchor, which is what lets a store
// wedged by an earlier binary (no anchor, or an anchor that no longer
// resolves) recover on its first run under this code.
func reconcileAutoExportJSONL(ctx context.Context, path, fromCommit string) (*autoExportReconciliation, error) {
// GH#4988: only refuse when *in-scope* JSONL issue records are absent from
// the store. Ephemeral wisps, templates, and infra types are outside
// auto-export scope (buildAutoExportFilter / GH#3649). Compaction can
@ -300,8 +368,9 @@ func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) (
if err != nil {
return nil, err
}
rec := &autoExportReconciliation{jsonlIssueCount: len(existing)}
if len(existing) == 0 {
return nil, nil
return rec, nil
}
// Store-side query stays unfiltered and MaxRows: 0 (opts out of
@ -321,24 +390,49 @@ func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) (
}
candidates := make([]string, 0)
for _, rec := range existing {
if rec.Ephemeral || rec.IsTemplate || infraTypeSet[string(rec.IssueType)] {
for _, record := range existing {
if record.Ephemeral || record.IsTemplate || infraTypeSet[string(record.IssueType)] {
continue
}
if _, ok := localIDs[rec.ID]; !ok {
candidates = append(candidates, rec.ID)
if _, ok := localIDs[record.ID]; !ok {
candidates = append(candidates, record.ID)
}
}
if len(candidates) == 0 {
return nil, nil
return rec, nil
}
// Before treating a JSONL-only id as an orphan/corruption, ask dolt_diff
// whether the deletion is real and already recorded — a normal `bd
// delete` since the last export's diff anchor, not data loss. Without an
// anchor (cold start / no successful export yet) there is nothing to
// diff against, so every candidate falls through to the conservative
// "missing" verdict below, same as before this check existed.
rec.provenDeleted, rec.unproven = proveDeletedIssueIDs(ctx, candidates, fromCommit)
return rec, nil
}
// proveDeletedIssueIDs splits JSONL-only candidate ids into the ones a proof
// source accounts for as real deletions and the ones nothing can explain.
//
// Sources run cheapest-first, and each one only sees what the previous could
// not prove:
//
// 1. the in-process delete handoff — this very command deleted the id a few
// milliseconds ago. Free, and the only source that can see a create and a
// delete that both sit uncommitted in the working set.
// 2. dolt_diff against the previous export's anchor (GH#5806) — a DoltStore
// fast path that also covers deletions that arrived from a peer.
// 3. the store's own committed history (GH#5896) — anchor-free, works on
// both stores, and the only source that can heal a store some earlier
// binary already wedged.
//
// Whatever survives all three is unexplained divergence, and the caller keeps
// refusing to overwrite: that is GH#4988's fail-safe, unchanged.
func proveDeletedIssueIDs(ctx context.Context, candidates []string, fromCommit string) (proven, unproven []string) {
unproven = candidates
// P1 — same-process delete handoff.
proven, unproven = splitProvenDeletions(proven, unproven, commandDeletedIssueIDs.has)
if len(unproven) == 0 {
return proven, unproven
}
// P2 — dolt_diff against the last export's anchor.
//
// The anchor must still be on the CURRENT HEAD's history for that proof
// to mean anything. dolt_diff(anchor, WORKING) reports a row as "removed"
@ -349,14 +443,11 @@ func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) (
// there would silently drop a live record from issues.jsonl, which is
// exactly #4988's corruption class this guard exists to prevent. So the
// deletion proof is accepted only when the anchor is reachable from HEAD
// (CommitExists queries dolt_log, i.e. HEAD's forward history); otherwise
// we keep refusing, which is the conservative pre-existing behavior.
// (CommitExists queries dolt_log, i.e. HEAD's forward history).
//
// Residual, deliberately not fixed here (tracked by #5896): an anchor
// that has become unresolvable — after a history squash or GC — combined
// with a pending delete is a permanent wedge. The guard refuses, so the
// export skips, so state is never saved, so the anchor never refreshes.
// Recovery is `bd init --from-jsonl` or moving issues.jsonl aside.
// A missing anchor, an off-history anchor, a diff error, or a store with
// no DiffStore at all (embedded mode — the default) all just fall through
// to P3 now, instead of ending the search.
if fromCommit != "" && diffAnchorOnCurrentHistory(ctx, fromCommit) {
if ds, ok := storage.UnwrapStore(store).(storage.DiffStore); ok {
if changed, diffErr := ds.ChangedIssueIDs(ctx, fromCommit, doltWorkingRef); diffErr == nil {
@ -364,25 +455,101 @@ func missingJSONLIssueIDsInStore(ctx context.Context, path, fromCommit string) (
for _, id := range changed.Removed {
removed[id] = struct{}{}
}
proven := candidates[:0]
for _, id := range candidates {
if _, ok := removed[id]; !ok {
proven = append(proven, id)
}
}
candidates = proven
proven, unproven = splitProvenDeletions(proven, unproven, func(id string) bool {
_, ok := removed[id]
return ok
})
} else {
debug.Logf("auto-export: anchor diff %s..%s failed (%v); falling through to the history proof\n",
fromCommit, doltWorkingRef, diffErr)
}
// A diff error (e.g. anchor unreachable after history rewrite)
// leaves candidates untouched — we can't prove the deletion, so
// fall back to refusing, exactly as before this check existed.
}
}
if len(unproven) == 0 {
return proven, unproven
}
return candidates, nil
// P3 — the store's own committed history. "Was here, is gone" is a
// deletion this store recorded; "was never here" is data this store never
// had, which stays unproven. See storage.HistoryPresence for why that
// asymmetry is what keeps #4988's guarantee intact.
if len(unproven) > historyProofMaxCandidates {
debug.Logf("auto-export: %d unproven JSONL-only ids exceeds the history-proof cap %d; refusing without asking\n",
len(unproven), historyProofMaxCandidates)
return proven, unproven
}
hp, ok := storage.UnwrapStore(store).(storage.HistoryPresence)
if !ok {
debug.Logf("auto-export: store does not implement HistoryPresence; JSONL-only ids stay unproven\n")
return proven, unproven
}
historical, err := hp.HistoricalIssueIDs(ctx, unproven)
if err != nil {
debug.Logf("auto-export: history proof failed (%v); treating JSONL-only ids as unproven\n", err)
return proven, unproven
}
proven, unproven = splitProvenDeletions(proven, unproven, func(id string) bool {
_, ok := historical[id]
return ok
})
return proven, unproven
}
// splitProvenDeletions moves the ids isProven accepts from unproven onto
// proven, preserving order on both sides.
func splitProvenDeletions(proven, unproven []string, isProven func(string) bool) ([]string, []string) {
remaining := make([]string, 0, len(unproven))
for _, id := range unproven {
if isProven(id) {
proven = append(proven, id)
continue
}
remaining = append(remaining, id)
}
return proven, remaining
}
// deletedIssueIDSet is the in-process handoff from the delete commands to
// auto-export: "this process just hard-deleted these ids, they are supposed to
// be gone". Deliberately NOT persisted — a cross-process ledger would mean the
// delete path read-modify-writes export-state.json underneath a concurrent
// exporter, and a lost entry there silently re-wedges. When the handoff misses
// (throttled export, separate invocation, peer's delete), the history proof
// picks the id up as soon as the deletion is committed.
type deletedIssueIDSet struct {
mu sync.Mutex
ids map[string]struct{}
}
func (s *deletedIssueIDSet) add(ids ...string) {
if len(ids) == 0 {
return
}
s.mu.Lock()
defer s.mu.Unlock()
if s.ids == nil {
s.ids = make(map[string]struct{}, len(ids))
}
for _, id := range ids {
s.ids[id] = struct{}{}
}
}
func (s *deletedIssueIDSet) reset() {
s.mu.Lock()
defer s.mu.Unlock()
s.ids = nil
}
func (s *deletedIssueIDSet) has(id string) bool {
s.mu.Lock()
defer s.mu.Unlock()
_, ok := s.ids[id]
return ok
}
// commitHistoryChecker is the narrow slice of storage.VersionControl that
// missingJSONLIssueIDsInStore needs. It is declared locally and asserted
// reconcileAutoExportJSONL needs. It is declared locally and asserted
// separately from storage.DiffStore so a store implementing one but not the
// other still degrades to the conservative verdict instead of panicking.
type commitHistoryChecker interface {
@ -392,7 +559,7 @@ type commitHistoryChecker interface {
// diffAnchorOnCurrentHistory reports whether anchor is still reachable from
// the current HEAD. It is the ancestry precondition on trusting a dolt_diff
// "removed" verdict as proof of a real `bd delete` rather than a history
// rewind — see missingJSONLIssueIDsInStore. Fails closed: a store that
// rewind — see reconcileAutoExportJSONL. Fails closed: a store that
// cannot answer, or an error while asking, yields false so the caller keeps
// refusing to overwrite.
func diffAnchorOnCurrentHistory(ctx context.Context, anchor string) bool {
@ -495,18 +662,6 @@ func issueRecordsInJSONL(path string) ([]jsonlIssueRecord, error) {
return records, nil
}
func issueIDsInJSONL(path string) ([]string, error) {
records, err := issueRecordsInJSONL(path)
if err != nil {
return nil, err
}
ids := make([]string, len(records))
for i, rec := range records {
ids[i] = rec.ID
}
return ids, nil
}
func sampleStrings(values []string, limit int) []string {
if len(values) <= limit {
return values
@ -693,7 +848,7 @@ func exportToFile(ctx context.Context, path string, includeMemories bool) (issue
// safe to drop when its id is no longer in the store (a TTL-compacted wisp,
// GH#4988); if the store still has it, dropping it repeats #4069's data
// loss. Deliberately unfiltered + MaxRows: 0, for the same reason as
// missingJSONLIssueIDsInStore's store-side query: this guard's failure mode
// reconcileAutoExportJSONL's store-side query: this guard's failure mode
// is a permanent wedge, so the query must stay maximally permissive.
func storeKnownIssueIDs(ctx context.Context) (map[string]struct{}, error) {
issues, err := store.SearchIssues(ctx, "", types.IssueFilter{Limit: 0, MaxRows: 0})
@ -809,11 +964,19 @@ func loadExportAutoState(beadsDir string) *exportAutoState {
path, len(data), err, string(data[:min(len(data), 200)]))
return &exportAutoState{}
}
if state.FormatVersion > exportAutoStateVersion {
debug.Logf("auto-export: state file %s is format v%d, this binary understands v%d; starting from empty state\n",
path, state.FormatVersion, exportAutoStateVersion)
return &exportAutoState{}
}
return &state
}
func saveExportAutoState(beadsDir string, state *exportAutoState) {
path := filepath.Join(beadsDir, exportAutoStateFile)
// Stamped here rather than at the call sites so there is exactly one
// place that decides what version a written file claims to be.
state.FormatVersion = exportAutoStateVersion
data, err := json.Marshal(state)
if err != nil {
fmt.Fprintf(os.Stderr, "Warning: auto-export: failed to marshal state: %v\n", err)
@ -1034,12 +1197,19 @@ func pathInsideDir(path, dir string) bool {
// didIncremental=false when any precondition fails so the caller can fall
// back to a full export.
//
// forcedRemovals are ids the caller has already proven deleted by some means
// the diff cannot see (the in-process delete handoff, or the store's committed
// history — see proveDeletedIssueIDs). They are unioned into the diff's own
// removals, because a proven-deleted id is by definition absent at BOTH diff
// endpoints in the cases that matter, so the patch would otherwise carry its
// stale line forward forever.
//
// dirtyIDs returns the RAW diff's upserted ids (never carryIDs) for the
// caller to persist as next cycle's carryIDs. Carrying only the
// freshly-diffed ids — not the ones merely carried-in — makes a stable
// no-op cycle self-heal: a carried id that produces no new diff simply
// drops out instead of accumulating forever.
func tryIncrementalExport(ctx context.Context, fullPath, fromCommit, toCommit string, carryIDs []string) (issueCount, memoryCount int, dirtyIDs []string, didIncremental bool, err error) {
func tryIncrementalExport(ctx context.Context, fullPath, fromCommit, toCommit string, carryIDs, forcedRemovals []string) (issueCount, memoryCount int, dirtyIDs []string, didIncremental bool, err error) {
if fromCommit == "" {
debug.Logf("auto-export: incremental skipped — no prior commit hash recorded\n")
return 0, 0, nil, false, nil
@ -1069,8 +1239,16 @@ func tryIncrementalExport(ctx context.Context, fullPath, fromCommit, toCommit st
// from last cycle (e.g. a working-set value that reverted between two
// diff anchors nets to "no change" against the anchor, but the live row
// still needs re-fetching to correct a stale patch from last cycle).
removed := make(map[string]bool, len(changed.Removed)+len(forcedRemovals))
for _, id := range changed.Removed {
removed[id] = true
}
for _, id := range forcedRemovals {
removed[id] = true
}
upsertIDs := unionStrings(changed.Upserted, carryIDs)
total := len(upsertIDs) + len(changed.Removed)
total := len(upsertIDs) + len(removed)
if total == 0 {
// Nothing changed and nothing carried over. Still a valid
// "incremental" outcome: no issues to rewrite, just refresh the
@ -1135,10 +1313,6 @@ func tryIncrementalExport(ctx context.Context, fullPath, fromCommit, toCommit st
// changed.Removed from the issues-table diff; rely on that.
}
removed := make(map[string]bool, len(changed.Removed)+len(droppedByFilter))
for _, id := range changed.Removed {
removed[id] = true
}
for id := range droppedByFilter {
removed[id] = true
}

View file

@ -13,21 +13,22 @@ import (
"github.com/steveyegge/beads/internal/types"
)
// TestMissingJSONLIssueIDsInStore_IgnoresCompactedWisp pins the actual
// GH#4988 behaviour at the layer that wedged auto-export permanently:
// missingJSONLIssueIDsInStore itself. maphew's review noted that
// TestReconcileAutoExportJSONL_IgnoresCompactedWisp pins the actual GH#4988
// behaviour at the layer that wedged auto-export permanently: the
// JSONL-vs-store reconciliation itself (formerly missingJSONLIssueIDsInStore,
// now reconcileAutoExportJSONL). maphew's review noted that
// TestMissingJSONLIssueIDsInStore is named in the original guard commit's
// validation list but doesn't exist in the tree; this closes that gap.
//
// Seeds one persistent issue (present in the store) and a JSONL file that
// additionally lists an ephemeral wisp id that is NOT in the store — the
// TTL-compaction scenario, where a wisp was written to issues.jsonl and
// later deleted from Dolt by compaction. Asserts `missing` comes back
// empty: an out-of-scope (ephemeral) row's absence from the store must not
// be treated as a hard orphan, or auto-export wedges forever.
// later deleted from Dolt by compaction. Asserts the reconciliation reports
// nothing at all: an out-of-scope (ephemeral) row's absence from the store
// must not be treated as a hard orphan, or auto-export wedges forever.
//
// Requires a live Dolt test server/container (cgo build); skips otherwise.
func TestMissingJSONLIssueIDsInStore_IgnoresCompactedWisp(t *testing.T) {
func TestReconcileAutoExportJSONL_IgnoresCompactedWisp(t *testing.T) {
if testDoltServerPort == 0 {
t.Skip("Dolt test server not available")
}
@ -85,11 +86,16 @@ func TestMissingJSONLIssueIDsInStore_IgnoresCompactedWisp(t *testing.T) {
t.Fatal(err)
}
missing, err := missingJSONLIssueIDsInStore(ctx, jsonlPath, "")
rec, err := reconcileAutoExportJSONL(ctx, jsonlPath, "")
if err != nil {
t.Fatalf("missingJSONLIssueIDsInStore: %v", err)
t.Fatalf("reconcileAutoExportJSONL: %v", err)
}
if len(missing) != 0 {
t.Fatalf("missing = %v, want empty (compacted wisp is out-of-scope and must not be a hard orphan)", missing)
if len(rec.unproven) != 0 {
t.Fatalf("unproven = %v, want empty (compacted wisp is out-of-scope and must not be a hard orphan)", rec.unproven)
}
// And it must be out of scope, not merely proven: an ephemeral row never
// reaches a proof source, so nothing about it is "reconciled" either.
if len(rec.provenDeleted) != 0 {
t.Fatalf("provenDeleted = %v, want empty (an out-of-scope wisp is never a candidate in the first place)", rec.provenDeleted)
}
}

View file

@ -314,6 +314,18 @@ func (s *spyDiffStore) GetStateHash(ctx context.Context) (string, error) {
return sh.GetStateHash(ctx)
}
// Forwarded for the same reason as the two above: the spy hides the concrete
// store from UnwrapStore, so any capability it does not re-declare simply
// vanishes from the guard's point of view. Without this the orphan guard's
// history proof silently degrades to "unproven" under the spy (GH#5896).
func (s *spyDiffStore) HistoricalIssueIDs(ctx context.Context, ids []string) (map[string]struct{}, error) {
hp, ok := storage.UnwrapStore(s.DoltStorage).(storage.HistoryPresence)
if !ok {
return nil, fmt.Errorf("spyDiffStore: wrapped store does not implement HistoryPresence")
}
return hp.HistoricalIssueIDs(ctx, ids)
}
func autoExportTestDir(t *testing.T) string {
t.Helper()
dir := t.TempDir()
@ -1222,7 +1234,11 @@ func TestShouldExport(t *testing.T) {
}
}
func TestCountIssueRecordsInJSONL(t *testing.T) {
// TestIssueRecordsInJSONL_CountsAndOrdersInScopeIDs pins the parse the
// pre-export guards read: exactly one record per id, memories and tombstones
// and id-less lines skipped, sorted by id. Both guards now take their counts
// and their candidate list from this one call.
func TestIssueRecordsInJSONL_CountsAndOrdersInScopeIDs(t *testing.T) {
tmpDir := t.TempDir()
path := filepath.Join(tmpDir, "issues.jsonl")
data := strings.Join([]string{
@ -1238,20 +1254,19 @@ func TestCountIssueRecordsInJSONL(t *testing.T) {
t.Fatal(err)
}
got, err := countIssueRecordsInJSONL(path)
records, err := issueRecordsInJSONL(path)
if err != nil {
t.Fatalf("countIssueRecordsInJSONL: %v", err)
t.Fatalf("issueRecordsInJSONL: %v", err)
}
if got != 2 {
t.Fatalf("countIssueRecordsInJSONL = %d, want 2", got)
if len(records) != 2 {
t.Fatalf("issueRecordsInJSONL returned %d records, want 2: %v", len(records), records)
}
ids, err := issueIDsInJSONL(path)
if err != nil {
t.Fatalf("issueIDsInJSONL: %v", err)
ids := make([]string, len(records))
for i, rec := range records {
ids[i] = rec.ID
}
if got := strings.Join(ids, ","); got != "bd-1,bd-2" {
t.Fatalf("issueIDsInJSONL = %q, want bd-1,bd-2", got)
t.Fatalf("issueRecordsInJSONL ids = %q, want bd-1,bd-2", got)
}
}
@ -1686,7 +1701,7 @@ func TestTryIncrementalExport_PatchesChangedIssuesAndDropsRemoved(t *testing.T)
}
c2 := h.mustCommit(t, ctx, "mutate")
issueCount, memoryCount, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
issueCount, memoryCount, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport returned error: %v", err)
}
@ -1742,7 +1757,7 @@ func TestTryIncrementalExport_DropsIssueWhenFlippedToTemplate(t *testing.T) {
}
c2 := h.mustCommit(t, ctx, "promote to template")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}
@ -1770,7 +1785,7 @@ func TestTryIncrementalExport_FallsBackWhenFileMissing(t *testing.T) {
// No existing file → must return didIncremental=false and leave the
// disk untouched so the caller falls back to the full-export path.
exportPath := filepath.Join(h.beadsDir, "issues.jsonl")
issueCount, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
issueCount, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@ -1810,7 +1825,7 @@ func TestTryIncrementalExport_ThresholdExceededFallsBack(t *testing.T) {
h.mustCreateBatch(t, ctx, incrementalExportThreshold+1, "thr-")
c2 := h.mustCommit(t, ctx, "flood")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@ -1875,7 +1890,7 @@ func TestMaybeAutoExport_SecondRunTakesIncrementalPath_ServerMode(t *testing.T)
}
// Both ChangedIssueIDs call sites must have fired: the orphan guard's
// proof-of-deletion probe (missingJSONLIssueIDsInStore, which runs
// proof-of-deletion probe (reconcileAutoExportJSONL, which runs
// because deleting e2e-b leaves it JSONL-only) and tryIncrementalExport's
// own diff. A bare "called at least once" check is NOT sufficient and was
// the PR #5806 round-5 review finding: the guard's own call satisfies it
@ -2055,7 +2070,7 @@ func TestTryIncrementalExport_NeverLeaksMemoriesIntoAutoExport(t *testing.T) {
}
c2 := h.mustCommit(t, ctx, "mutate")
_, memCount, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, memCount, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}
@ -2105,7 +2120,7 @@ func TestTryIncrementalExport_PreservesPreExistingMemoryAcrossPatch(t *testing.T
}
c2 := h.mustCommit(t, ctx, "mutate")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}
@ -2206,7 +2221,7 @@ func TestTryIncrementalExport_ExcludesConfiguredOwnerFromPatchedIssues(t *testin
}
c2 := h.mustCommit(t, ctx, "mutate")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}
@ -2239,7 +2254,7 @@ func TestTryIncrementalExport_PatchedLinesIncludeTypeField(t *testing.T) {
}
c2 := h.mustCommit(t, ctx, "mutate")
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil)
_, _, _, didIncremental, err := tryIncrementalExport(ctx, exportPath, c1, c2, nil, nil)
if err != nil {
t.Fatalf("tryIncrementalExport: %v", err)
}

View file

@ -0,0 +1,793 @@
//go:build cgo
package main
import (
"context"
"encoding/json"
"os"
"path/filepath"
"reflect"
"sort"
"strings"
"testing"
"time"
"github.com/steveyegge/beads/internal/config"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/types"
)
// These tests cover GH#5896: `bd delete` used to leave a JSONL-only issue id
// that no pre-export guard could account for, so auto-export refused, never
// saved state, and refused again on every later command — forever, with the
// documented recovery (`bd init --from-jsonl`) undoing the delete.
//
// They run against a fake store that models EMBEDDED mode's capability set —
// the default open path, and the one the in-window #5806 fix could not reach:
// no StateHasher, no DiffStore, but CommitExists and (new) HistoryPresence.
// That is the same deliberate split TestMaybeAutoExport_EmbeddedModeFallsBack
// CleanlyToFullExport makes: what these assert is the GUARD's wiring — which
// proof sources it consults, what it does with the answers, what reaches the
// file — while the capability's own semantics against a real Dolt engine are
// pinned by the cross-store contract in internal/storage/storagecontract.
// fakeEmbeddedStore models EmbeddedDoltStore's capability surface for
// auto-export: HEAD-only change detection, no dolt_diff, but able to answer
// "was this id ever in my history".
type fakeEmbeddedStore struct {
storage.DoltStorage
issues []*types.Issue
historical map[string]bool
// knownCommits is what CommitExists says yes to. An anchor that isn't
// here models a rewind, a squash, or a state file from another clone.
knownCommits map[string]bool
headCommit string
// historyAsked records every id handed to HistoricalIssueIDs, so a test
// can assert what the guard did NOT ask about.
historyAsked []string
historyCalls int
historyFailed bool
}
func (f *fakeEmbeddedStore) GetCurrentCommit(_ context.Context) (string, error) {
if f.headCommit == "" {
return "head-1", nil
}
return f.headCommit, nil
}
func (f *fakeEmbeddedStore) GetInfraTypes(_ context.Context) map[string]bool { return nil }
func (f *fakeEmbeddedStore) GetConfig(_ context.Context, _ string) (string, error) {
return "", nil
}
func (f *fakeEmbeddedStore) SearchIssues(_ context.Context, _ string, _ types.IssueFilter) ([]*types.Issue, error) {
return f.issues, nil
}
func (f *fakeEmbeddedStore) GetLabelsForIssues(_ context.Context, _ []string) (map[string][]string, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) GetDependencyRecordsForIssues(_ context.Context, _ []string) (map[string][]*types.Dependency, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) GetCommentsForIssues(_ context.Context, _ []string) (map[string][]*types.Comment, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) GetCommentCounts(_ context.Context, _ []string) (map[string]int, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) GetDependencyCounts(_ context.Context, _ []string) (map[string]*types.DependencyCounts, error) {
return nil, nil
}
func (f *fakeEmbeddedStore) CommitExists(_ context.Context, hash string) (bool, error) {
return f.knownCommits[hash], nil
}
func (f *fakeEmbeddedStore) HistoricalIssueIDs(_ context.Context, ids []string) (map[string]struct{}, error) {
f.historyCalls++
f.historyAsked = append(f.historyAsked, ids...)
if f.historyFailed {
return nil, context.DeadlineExceeded
}
present := make(map[string]struct{})
for _, id := range ids {
if f.historical[id] {
present[id] = struct{}{}
}
}
return present, nil
}
// deleteFromStore removes an id from the store's live rows the way a hard
// delete does, leaving its history intact.
func (f *fakeEmbeddedStore) deleteFromStore(id string) {
kept := f.issues[:0]
for _, iss := range f.issues {
if iss.ID != id {
kept = append(kept, iss)
}
}
f.issues = kept
}
// fakeHistorylessStore is fakeEmbeddedStore without HistoryPresence: the
// degradation case (an older or third-party store), and the fixture the P1
// handoff test needs so nothing but the handoff can prove anything.
type fakeHistorylessStore struct {
*fakeEmbeddedStore
}
func (f *fakeHistorylessStore) HistoricalIssueIDs(context.Context, []string) (map[string]struct{}, error) {
panic("HistoryPresence must not be reached on a store that does not implement it")
}
func newWedgeHealTestStore(t *testing.T, issues ...*types.Issue) *fakeEmbeddedStore {
t.Helper()
historical := make(map[string]bool, len(issues))
for _, iss := range issues {
historical[iss.ID] = true
}
return &fakeEmbeddedStore{
issues: issues,
historical: historical,
knownCommits: map[string]bool{},
headCommit: "head-1",
}
}
func wedgeHealIssue(id, title string) *types.Issue {
return &types.Issue{
ID: id,
Title: title,
Status: types.StatusOpen,
Priority: 2,
IssueType: types.TypeTask,
}
}
// setupWedgeHealTest wires config, globals and a .beads dir, and returns the
// beads dir plus the export path.
func setupWedgeHealTest(t *testing.T, store2 storage.DoltStorage) (beadsDir, exportPath string) {
t.Helper()
initConfigForTest(t)
config.Set("export.auto", true)
// The throttle is not what these tests are about, and a 1ms interval is
// long enough to swallow a second export that lands in the same
// millisecond as the first.
config.Set("export.interval", "1ns")
saveAndRestoreGlobals(t)
dir := autoExportTestDir(t)
store = store2
storeMutex.Lock()
storeActive = true
storeMutex.Unlock()
t.Cleanup(func() {
storeMutex.Lock()
storeActive = false
storeMutex.Unlock()
})
beadsDir = filepath.Join(dir, ".beads")
return beadsDir, filepath.Join(beadsDir, "issues.jsonl")
}
func jsonlIDs(t *testing.T, path string) []string {
t.Helper()
records, err := issueRecordsInJSONL(path)
if err != nil {
t.Fatalf("issueRecordsInJSONL(%s): %v", path, err)
}
ids := make([]string, len(records))
for i, rec := range records {
ids[i] = rec.ID
}
sort.Strings(ids)
return ids
}
// writeWedgedExportState writes the on-disk state a pre-fix binary leaves
// behind: a last-seen commit, an open throttle window, and whatever anchor
// the caller wants to model.
func writeWedgedExportState(t *testing.T, beadsDir, lastCommit, anchor string) {
t.Helper()
saveExportAutoState(beadsDir, &exportAutoState{
LastDoltCommit: lastCommit,
LastDiffAnchor: anchor,
Timestamp: time.Time{},
Issues: 2,
})
}
// TestMaybeAutoExport_DeleteHealsInsteadOfWedging_Embedded is the issue's own
// repro on the default backend: create → export → delete → export must
// SUCCEED and the export must reflect the deletion.
func TestMaybeAutoExport_DeleteHealsInsteadOfWedging_Embedded(t *testing.T) {
fake := newWedgeHealTestStore(t,
wedgeHealIssue("wh-a", "Alpha"),
wedgeHealIssue("wh-b", "Beta"),
)
beadsDir, exportPath := setupWedgeHealTest(t, fake)
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
if got, want := jsonlIDs(t, exportPath), []string{"wh-a", "wh-b"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Fatalf("setup: exported ids = %v, want %v", got, want)
}
// Delete wh-b the way a pre-existing store would have it — gone from the
// live rows, still in history — and touch wh-a so the test can also tell
// a healed export from a skipped one by its CONTENT.
fake.deleteFromStore("wh-b")
fake.issues[0].Title = "Alpha renamed"
fake.headCommit = "head-2"
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
})
if got, want := jsonlIDs(t, exportPath), []string{"wh-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("after delete, exported ids = %v, want %v — the deletion was not reflected", got, want)
}
if titles := loadIssueTitles(t, exportPath); titles["wh-a"] != "Alpha renamed" {
t.Errorf("wh-a title = %q, want %q — the export was skipped, not healed", titles["wh-a"], "Alpha renamed")
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "head-2" {
t.Errorf("LastDoltCommit = %q, want %q — state must advance or the next command wedges on the same comparison", state.LastDoltCommit, "head-2")
}
if !strings.Contains(stderr, "wh-b") || !strings.Contains(stderr, "reconciled") {
t.Errorf("stderr = %q, want a heal notice naming wh-b", stderr)
}
if strings.Contains(stderr, "refusing to overwrite") {
t.Errorf("stderr = %q, want no refusal", stderr)
}
}
// TestMaybeAutoExport_DeleteLastIssueHealsEmptyGuard covers the trap the
// single-issue repro actually hits: with nothing left in the store, the
// EMPTY-OVERWRITE guard fires before the orphan guard, and refuses on "0
// issues vs 1 record" unless it too knows the record is a proven deletion.
func TestMaybeAutoExport_DeleteLastIssueHealsEmptyGuard(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-only", "The only one"))
beadsDir, exportPath := setupWedgeHealTest(t, fake)
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
if got := jsonlIDs(t, exportPath); len(got) != 1 {
t.Fatalf("setup: exported ids = %v, want exactly wh-only", got)
}
fake.deleteFromStore("wh-only")
fake.headCommit = "head-2"
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "would export 0 issues") {
t.Fatalf("empty-overwrite guard refused a fully-proven deletion: %q", stderr)
}
if _, err := os.Stat(exportPath); !os.IsNotExist(err) {
t.Errorf("issues.jsonl still exists after the last issue was deleted (stat err = %v); the empty export removes the file", err)
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "head-2" {
t.Errorf("LastDoltCommit = %q, want %q", state.LastDoltCommit, "head-2")
}
}
// TestMaybeAutoExport_PreWedgedStoreRecovers is the upgrade path: the delete
// already happened under a binary that had no proof for it, so there is no
// in-process handoff to lean on and the anchor is useless. The first export
// under this code must heal anyway, from history alone.
func TestMaybeAutoExport_PreWedgedStoreRecovers(t *testing.T) {
for _, tc := range []struct {
name string
anchor string
}{
// R4: the state file was lost, or no export ever succeeded.
{name: "AnchorLost", anchor: ""},
// R2 stand-in: a well-formed anchor that no longer resolves, as
// after a history squash, a GC, or a branch checkout.
{name: "AnchorNoLongerResolves", anchor: "abcdefghijklmnopqrstuvwxyz012345"},
} {
t.Run(tc.name, func(t *testing.T) {
// The store never knew wh-b as a live row in this process; only
// its history remembers it. That IS the pre-fix on-disk shape.
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
fake.historical["wh-b"] = true
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-b", "title": "Beta", "issue_type": "task"},
)
writeWedgedExportState(t, beadsDir, "head-0", tc.anchor)
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "refusing to overwrite") {
t.Fatalf("still wedged: %q", stderr)
}
if got, want := jsonlIDs(t, exportPath), []string{"wh-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("exported ids = %v, want %v", got, want)
}
if !strings.Contains(stderr, "wh-b") {
t.Errorf("stderr = %q, want a heal notice naming wh-b", stderr)
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "head-1" {
t.Errorf("LastDoltCommit = %q, want head-1 (state must be re-saved or the wedge survives the upgrade)", state.LastDoltCommit)
}
})
}
}
// TestMaybeAutoExport_UnprovableJSONLIDStillRefuses is the fail-safe half.
// An id this store has no history of is indistinguishable from GH#4988's torn
// or replaced store, and must keep refusing with the unchanged message —
// including after a rewind, where the id's commits are no longer reachable
// from HEAD and so are no longer in dolt_history_issues.
func TestMaybeAutoExport_UnprovableJSONLIDStillRefuses(t *testing.T) {
for _, tc := range []struct {
name string
anchor string
}{
{name: "NeverExisted", anchor: ""},
{name: "RewoundAwayWithStaleAnchor", anchor: "abcdefghijklmnopqrstuvwxyz012345"},
} {
t.Run(tc.name, func(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-ghost", "title": "Ghost", "issue_type": "task"},
)
writeWedgedExportState(t, beadsDir, "head-0", tc.anchor)
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if !strings.Contains(stderr, "refusing to overwrite") {
t.Errorf("stderr = %q, want the unchanged orphan refusal", stderr)
}
if !strings.Contains(stderr, "wh-ghost") {
t.Errorf("stderr = %q, want the refusal to name wh-ghost", stderr)
}
if got := jsonlIDs(t, exportPath); len(got) != 2 {
t.Errorf("exported ids = %v, want both records preserved — an unprovable divergence must never lose a line", got)
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "head-0" {
t.Errorf("LastDoltCommit = %q, want head-0 (a refusal must not advance state)", state.LastDoltCommit)
}
})
}
}
// TestMaybeAutoExport_HistoryProofFailureRefuses: a store that cannot answer
// the history question is not a store that answered "no". The guard must fall
// back to refusing, not to overwriting.
func TestMaybeAutoExport_HistoryProofFailureRefuses(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
fake.historical["wh-b"] = true
fake.historyFailed = true
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-b", "title": "Beta", "issue_type": "task"},
)
writeWedgedExportState(t, beadsDir, "head-0", "")
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if !strings.Contains(stderr, "refusing to overwrite") {
t.Errorf("stderr = %q, want a refusal when the history proof itself failed", stderr)
}
if got := jsonlIDs(t, exportPath); len(got) != 2 {
t.Errorf("exported ids = %v, want both records preserved", got)
}
}
// TestMaybeAutoExport_DeleteHandoffProvesWhatHistoryCannot covers the one
// shape no store-side proof can ever reach: an issue created AND deleted
// without either write reaching a commit. The in-process handoff from
// deleteBatch is the only witness, and the store here has no HistoryPresence
// at all, so nothing else can cover for it.
func TestMaybeAutoExport_DeleteHandoffProvesWhatHistoryCannot(t *testing.T) {
inner := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
fake := &fakeHistorylessStore{fakeEmbeddedStore: inner}
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-uncommitted", "title": "Never committed", "issue_type": "task"},
)
writeWedgedExportState(t, beadsDir, "head-0", "")
commandDeletedIssueIDs.reset()
t.Cleanup(commandDeletedIssueIDs.reset)
commandDeletedIssueIDs.add("wh-uncommitted")
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "refusing to overwrite") {
t.Fatalf("the delete handoff did not prove the deletion: %q", stderr)
}
if got, want := jsonlIDs(t, exportPath), []string{"wh-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("exported ids = %v, want %v", got, want)
}
}
// TestMaybeAutoExport_ScopeSkipsNeverReachAProofSource: out-of-scope JSONL
// rows (ephemeral wisps, templates, infra types) are not candidates in the
// first place — GH#4988's own fix — so they must not cost a history query
// either.
func TestMaybeAutoExport_ScopeSkipsNeverReachAProofSource(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
_, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-wisp", "title": "Compacted wisp", "issue_type": "task", "ephemeral": true},
map[string]any{"_type": "issue", "id": "wh-tpl", "title": "Template", "issue_type": "task", "is_template": true},
)
rec, err := reconcileAutoExportJSONL(context.Background(), exportPath, "")
if err != nil {
t.Fatalf("reconcileAutoExportJSONL: %v", err)
}
if len(rec.unproven) != 0 || len(rec.provenDeleted) != 0 {
t.Errorf("unproven = %v, provenDeleted = %v, want both empty", rec.unproven, rec.provenDeleted)
}
if fake.historyCalls != 0 {
t.Errorf("HistoricalIssueIDs called %d times with %v; out-of-scope rows must never reach a proof source", fake.historyCalls, fake.historyAsked)
}
if rec.jsonlIssueCount != 3 {
t.Errorf("jsonlIssueCount = %d, want 3 (the empty-overwrite guard counts every record, in scope or not)", rec.jsonlIssueCount)
}
}
// TestProveDeletedIssueIDs_HistoryProofIsCapped: past the cap, a divergence is
// not a delete pattern, and the guard refuses without paying for the scan.
func TestProveDeletedIssueIDs_HistoryProofIsCapped(t *testing.T) {
fake := newWedgeHealTestStore(t)
setupWedgeHealTest(t, fake)
candidates := make([]string, historyProofMaxCandidates+1)
for i := range candidates {
candidates[i] = "wh-" + string(rune('a'+i%26)) + string(rune('0'+i/26))
fake.historical[candidates[i]] = true
}
proven, unproven := proveDeletedIssueIDs(context.Background(), candidates, "")
if len(proven) != 0 {
t.Errorf("proven = %d ids, want 0 past the cap", len(proven))
}
if len(unproven) != len(candidates) {
t.Errorf("unproven = %d ids, want all %d", len(unproven), len(candidates))
}
if fake.historyCalls != 0 {
t.Errorf("HistoricalIssueIDs called %d times; past the cap the guard must not ask", fake.historyCalls)
}
}
// TestExportAutoState_VersionRoundTripAndFutureDiscard is the GH#5896 R5
// acceptance: today's unversioned files load unchanged, a file from a future
// binary is discarded rather than reinterpreted, and — the part that makes
// discarding safe at all — discarding does not wedge, because the deletion
// proofs no longer need the anchor it threw away.
func TestExportAutoState_VersionRoundTripAndFutureDiscard(t *testing.T) {
t.Run("StampsTheCurrentVersion", func(t *testing.T) {
dir := t.TempDir()
saveExportAutoState(dir, &exportAutoState{LastDoltCommit: "c1", LastDiffAnchor: "a1"})
data, err := os.ReadFile(filepath.Join(dir, exportAutoStateFile))
if err != nil {
t.Fatalf("read state: %v", err)
}
var raw map[string]any
if err := json.Unmarshal(data, &raw); err != nil {
t.Fatalf("unmarshal state: %v", err)
}
if got := raw["v"]; got != float64(exportAutoStateVersion) {
t.Errorf("state file v = %v, want %d", got, exportAutoStateVersion)
}
state := loadExportAutoState(dir)
if state.LastDoltCommit != "c1" || state.LastDiffAnchor != "a1" {
t.Errorf("round trip lost fields: %+v", state)
}
})
t.Run("LoadsAnUnversionedFileAsIs", func(t *testing.T) {
dir := t.TempDir()
legacy := `{"last_dolt_commit":"c1","last_diff_anchor":"a1","issues":7}`
if err := os.WriteFile(filepath.Join(dir, exportAutoStateFile), []byte(legacy), 0o600); err != nil {
t.Fatal(err)
}
state := loadExportAutoState(dir)
if state.LastDoltCommit != "c1" || state.LastDiffAnchor != "a1" || state.Issues != 7 {
t.Errorf("pre-versioning state file did not load as-is: %+v", state)
}
})
t.Run("DiscardsAFutureVersionWithoutWedging", func(t *testing.T) {
fake := newWedgeHealTestStore(t, wedgeHealIssue("wh-a", "Alpha"))
fake.historical["wh-b"] = true
beadsDir, exportPath := setupWedgeHealTest(t, fake)
writeJSONLLines(t, exportPath,
map[string]any{"_type": "issue", "id": "wh-a", "title": "Alpha", "issue_type": "task"},
map[string]any{"_type": "issue", "id": "wh-b", "title": "Beta", "issue_type": "task"},
)
future := `{"v":999,"last_dolt_commit":"head-1","last_diff_anchor":"future-anchor"}`
if err := os.WriteFile(filepath.Join(beadsDir, exportAutoStateFile), []byte(future), 0o600); err != nil {
t.Fatal(err)
}
if state := loadExportAutoState(beadsDir); state.LastDoltCommit != "" || state.LastDiffAnchor != "" {
t.Fatalf("future-version state was not discarded: %+v", state)
}
// Discarding drops the anchor AND the last-seen commit, so this run
// looks like a cold start with a stale JSONL — pre-#5896 the exact
// R4 permanent wedge. It must export and re-anchor instead.
stderr := captureStderr(t, func() {
if err := maybeAutoExport(context.Background(), false); err != nil {
t.Fatalf("maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "refusing to overwrite") {
t.Fatalf("discarding a future-version state file wedged the export: %q", stderr)
}
if got, want := jsonlIDs(t, exportPath), []string{"wh-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("exported ids = %v, want %v", got, want)
}
state := loadExportAutoState(beadsDir)
if state.FormatVersion != exportAutoStateVersion || state.LastDoltCommit != "head-1" {
t.Errorf("state after re-anchor = %+v, want v%d at head-1", state, exportAutoStateVersion)
}
})
}
// The tests below run against a REAL DoltStore over the test sql-server —
// server mode, where the store does implement DiffStore and the incremental
// path is live. What the fake-store tests cannot reach is exactly what these
// are for: dolt_history_issues answering for itself, and proven-deleted ids
// having to survive a dolt_diff-driven incremental PATCH rather than a full
// rewrite from the store.
// TestMaybeAutoExport_DeleteHealsInsteadOfWedging_ServerMode is the issue's
// repro on the server-backed store, with the anchor deliberately blanked so
// the #5806 anchor-diff proof cannot fire and the history proof has to carry
// it — the R2/R4 residue this fix is for.
func TestMaybeAutoExport_DeleteHealsInsteadOfWedging_ServerMode(t *testing.T) {
h, ctx := setupIncrementalExportTest(t)
initConfigForTest(t)
config.Set("export.auto", true)
config.Set("export.interval", "1ns")
h.mustCreate(t, ctx, "sw-a", "Alpha")
h.mustCreate(t, ctx, "sw-b", "Beta")
h.mustCommit(t, ctx, "baseline")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
exportPath := filepath.Join(h.beadsDir, "issues.jsonl")
if got := countIssueLines(t, exportPath); got != 2 {
t.Fatalf("setup: %d issue lines, want 2", got)
}
// Delete through the store directly — no in-process handoff — and blank
// the anchor, which is what a squashed history or a lost state file
// leaves behind. Only dolt_history_issues can answer now.
if err := h.store.DeleteIssue(ctx, "sw-b"); err != nil {
t.Fatalf("DeleteIssue: %v", err)
}
h.mustCommit(t, ctx, "delete sw-b")
state := loadExportAutoState(h.beadsDir)
state.LastDiffAnchor = ""
state.Timestamp = time.Time{}
saveExportAutoState(h.beadsDir, state)
stderr := captureStderr(t, func() {
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
})
if strings.Contains(stderr, "refusing to overwrite") {
t.Fatalf("still wedged on the server-backed store: %q", stderr)
}
if got, want := jsonlIDs(t, exportPath), []string{"sw-a"}; strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("exported ids = %v, want %v", got, want)
}
if !strings.Contains(stderr, "sw-b") {
t.Errorf("stderr = %q, want a heal notice naming sw-b", stderr)
}
if loadExportAutoState(h.beadsDir).LastDiffAnchor == "" {
t.Error("the export re-anchored to nothing; the next cycle would take the same anchorless path forever")
}
}
// TestMaybeAutoExport_ProvenDeletionReachesTheIncrementalPatch is plumbing
// consequence #1, and the one that fails SILENTLY if missed: the guard can
// clear the export while the incremental patch still carries the deleted
// record forward, because a row absent at BOTH diff endpoints produces no
// diff row at all.
//
// The setup makes that shape exactly: sx-x is created and deleted entirely
// AFTER the anchor, so dolt_diff(anchor, WORKING) never mentions it, while
// sx-y is modified so the run genuinely stays on the incremental path.
func TestMaybeAutoExport_ProvenDeletionReachesTheIncrementalPatch(t *testing.T) {
h, ctx := setupIncrementalExportTest(t)
initConfigForTest(t)
config.Set("export.auto", true)
config.Set("export.interval", "1ns")
spy := &spyDiffStore{DoltStorage: h.store}
store = spy
h.mustCreate(t, ctx, "sx-y", "Yankee")
h.mustCommit(t, ctx, "baseline")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
exportPath := filepath.Join(h.beadsDir, "issues.jsonl")
anchor := loadExportAutoState(h.beadsDir).LastDiffAnchor
if anchor == "" {
t.Fatal("setup: no diff anchor recorded, the incremental path can't be under test")
}
// sx-x exists only between two commits that both come after the anchor,
// and its JSONL line is written by an export that ran while it was live.
h.mustCreate(t, ctx, "sx-x", "X-ray")
h.mustCommit(t, ctx, "add sx-x")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
if _, ok := loadIssueTitles(t, exportPath)["sx-x"]; !ok {
t.Fatalf("setup: sx-x must be in the export before it is deleted")
}
// Pin the anchor back to before sx-x ever existed. Now the diff's two
// endpoints both lack it, exactly like a create+delete inside one
// throttle window.
state := loadExportAutoState(h.beadsDir)
state.LastDiffAnchor = anchor
state.Timestamp = time.Time{}
saveExportAutoState(h.beadsDir, state)
if err := h.store.DeleteIssue(ctx, "sx-x"); err != nil {
t.Fatalf("DeleteIssue: %v", err)
}
h.mustCommit(t, ctx, "delete sx-x")
if err := h.store.UpdateIssue(ctx, "sx-y", map[string]interface{}{"title": "Yankee renamed"}, "tester"); err != nil {
t.Fatalf("UpdateIssue: %v", err)
}
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("third maybeAutoExport: %v", err)
}
// LastDirtyIDs reaches state only from a successful incremental patch —
// the full-export fallback nils it — and sx-y is the only upserted id, so
// exactly {sx-y} is what says "this run was incremental AND it ran".
if got, want := loadExportAutoState(h.beadsDir).LastDirtyIDs, []string{"sx-y"}; !reflect.DeepEqual(got, want) {
t.Fatalf("LastDirtyIDs = %v, want %v — the run was skipped or fell back to a full export, and this test only means something on the incremental path", got, want)
}
titles := loadIssueTitles(t, exportPath)
if _, stillThere := titles["sx-x"]; stillThere {
t.Error("sx-x survived the incremental patch: a deletion proven outside dolt_diff never reached the file as a forced removal")
}
if titles["sx-y"] != "Yankee renamed" {
t.Errorf("sx-y title = %q, want %q", titles["sx-y"], "Yankee renamed")
}
// Content-equivalence control: an incrementally-patched file must describe
// the same issue set, field for field, as a fresh full export of the same
// state — including in what it OMITS.
controlPath := filepath.Join(t.TempDir(), "control.jsonl")
if _, _, err := exportToFile(ctx, controlPath, false); err != nil {
t.Fatalf("control exportToFile: %v", err)
}
got := jsonlRecordsByID(t, exportPath)
want := jsonlRecordsByID(t, controlPath)
if len(got) != len(want) {
t.Fatalf("incremental export has %d records, control full export has %d", len(got), len(want))
}
for id, wantRec := range want {
gotRec, ok := got[id]
if !ok {
t.Errorf("record %s present in control export, missing from incremental export", id)
continue
}
if !reflect.DeepEqual(gotRec, wantRec) {
t.Errorf("record %s differs between incremental and full export:\n incremental: %v\n full: %v", id, gotRec, wantRec)
}
}
}
// TestMaybeAutoExport_HistoryProofDoesNotSurviveARewind is the safety sibling
// of TestMaybeAutoExport_HistoryRewindDoesNotProveDeletion, aimed at the new
// proof source rather than the anchor one. After a hard reset past an id's
// creation, dolt_history_issues no longer reaches those commits, so the
// history proof must come up empty and the guard must keep refusing — with
// the line still in the file.
func TestMaybeAutoExport_HistoryProofDoesNotSurviveARewind(t *testing.T) {
h, ctx := setupIncrementalExportTest(t)
initConfigForTest(t)
config.Set("export.auto", true)
config.Set("export.interval", "1ns")
h.mustCreate(t, ctx, "sr-a", "Alpha")
c1 := h.mustCommit(t, ctx, "baseline")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("first maybeAutoExport: %v", err)
}
exportPath := filepath.Join(h.beadsDir, "issues.jsonl")
h.mustCreate(t, ctx, "sr-x", "Rewound")
h.mustCommit(t, ctx, "add sr-x")
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("second maybeAutoExport: %v", err)
}
if titles := loadIssueTitles(t, exportPath); titles["sr-x"] != "Rewound" {
t.Fatalf("setup: sr-x must be exported before the rewind, got %v", titles)
}
raw, ok := storage.UnwrapStore(h.store).(storage.RawDBAccessor)
if !ok {
t.Skip("store does not expose raw DB access")
}
if _, err := raw.DB().ExecContext(ctx, "CALL DOLT_RESET('--hard', ?)", c1); err != nil {
t.Fatalf("CALL DOLT_RESET('--hard', %s): %v", c1, err)
}
// Blank the anchor so the anchor-diff proof cannot even be consulted:
// whatever happens next is the history proof's answer alone.
state := loadExportAutoState(h.beadsDir)
state.LastDiffAnchor = ""
state.Timestamp = time.Time{}
saveExportAutoState(h.beadsDir, state)
stderr := captureStderr(t, func() {
if err := maybeAutoExport(ctx, false); err != nil {
t.Fatalf("third maybeAutoExport: %v", err)
}
})
if !strings.Contains(stderr, "refusing to overwrite") {
t.Errorf("stderr = %q, want the unchanged refusal: a rewound-away id is data this store LOST, not a deletion it recorded", stderr)
}
if _, stillThere := loadIssueTitles(t, exportPath)["sr-x"]; !stillThere {
t.Error("sr-x was silently dropped from issues.jsonl after a history rewind: the history proof accepted a rewound-away row as a proven deletion")
}
}

View file

@ -107,6 +107,15 @@ func seedProxiedExportFixture(t *testing.T, bd string, p proxiedProject) exportP
// Ephemeral wisp: excluded from default export, included with --all.
fx.wisp = bdProxiedCreateSilent(t, bd, p.dir, "Ephemeral heartbeat", "--ephemeral", "--wisp-type", "heartbeat")
// Relations on the wisp: the --all export must carry them, which is what
// pins the UOW plane partition (only wisp ids reach the wisp readers,
// wy-237yfi) against the classic-mode oracle.
if out, err := bdProxiedRun(t, bd, p.dir, "label", "add", fx.wisp, "wisp-label"); err != nil {
t.Fatalf("label add on wisp: %v\n%s", err, out)
}
if out, err := bdProxiedRun(t, bd, p.dir, "comment", fx.wisp, "wisp comment"); err != nil {
t.Fatalf("comment on wisp: %v\n%s", err, out)
}
// Infra-typed bead (message): auto-routed to the ephemeral plane and an
// infra type, so default export must exclude it and --all include it.
@ -409,6 +418,21 @@ func TestProxiedServerExport(t *testing.T) {
if len(keys) != 2 || keys[0] != "aardvark" || keys[1] != "zebra" {
t.Errorf("--all export memories = %v, want [aardvark zebra] (sorted)", keys)
}
// The wisp's own relations ride along: a misclassified plane would
// drop them silently while the id above still passes.
for _, rec := range records {
if rec["id"] != fx.wisp {
continue
}
labels, _ := rec["labels"].([]any)
if len(labels) != 1 || labels[0] != "wisp-label" {
t.Errorf("--all export wisp %s labels = %v, want [wisp-label]", fx.wisp, rec["labels"])
}
comments, _ := rec["comments"].([]any)
if len(comments) != 1 {
t.Errorf("--all export wisp %s comments = %v, want one", fx.wisp, rec["comments"])
}
}
})
t.Run("include_memories_after_issues_sorted", func(t *testing.T) {

View file

@ -207,10 +207,14 @@ func (s *uowExportSource) GetAllConfig(ctx context.Context) (map[string]string,
// by wisps-table membership (issueops.PartitionWispIDsInTx). Row flags are
// NOT a substitute for that partition — a promoted no-history wisp is a
// durable issues-table row that (as wild data from the pre-bd-r9uce
// promote) still carries NoHistory=true — so both
// planes are queried with the FULL ID set and merged. The planes are
// ID-disjoint (GH#4455 cross-table guard), so at most one plane returns
// rows for any id and the merge cannot collide.
// promote) still carries NoHistory=true — so membership is read from the
// wisps table itself (WispPlaneIDs), never inferred. The durable plane is
// then queried with the FULL ID set and the wisp plane only with that
// membership subset (wispReaderIDs), and the two are merged. Unreadable
// membership fails OPEN to the full id list, so a nil set is never
// mistaken for "no wisps" (wy-237yfi). The planes are ID-disjoint
// (GH#4455 cross-table guard), so at most one plane returns rows for any
// id and the merge cannot collide.
// - Dependency records: domain GetIssueDependencyRecords already partitions
// internally (same issueops helper), so one call covers both planes.
// - Dependency counts: classic GetDependencyCountsInTx queries BOTH dep
@ -219,7 +223,12 @@ func (s *uowExportSource) GetAllConfig(ctx context.Context) (map[string]string,
// queried for the full ID set and summed here too.
//
// Wisp-plane reads tolerate a rig without the wisp tables (IsTableNotExist),
// mirroring the dependency-counts leg.
// mirroring the dependency-counts leg. The three wisp-relation readers below
// keep that blanket check: each is a single-table read, so blanket and
// table-specific are the same test there. The membership probe is the
// exception — WispPlaneIDs renders FROM wisps LEFT JOIN leases, so its
// tolerance is pinned to a missing `wisps` table and a broken database
// surfaces as an error instead of an empty wisp plane.
func (s *uowExportSource) LoadExportRelations(ctx context.Context, issues []*types.Issue) (exportRelations, error) {
rel := exportRelations{
labels: map[string][]string{},
@ -259,7 +268,24 @@ func (s *uowExportSource) LoadExportRelations(ctx context.Context, issues []*typ
}
mergeExportMap(rel.commentCounts, counts)
wispLabels, err := s.uw.LabelUseCase().GetLabelsForWisps(ctx, allIDs) //nolint:forbidigo // bulk relation load; see GetLabelsForIssues above
// The wisp_labels / wisp_comments rows of a wisp are keyed by an issue_id
// that lives in the WISPS table (promotion moves them to the durable
// tables, issueops.PromoteFromEphemeralInTx), so the three wisp readers
// below only need the ids that are wisps right now. Handing them the
// whole workspace was pure waste — on a 19k-issue rig with no wisps it
// was three 19k-placeholder statements answering nothing (wy-237yfi).
// CountsByWispIDs is NOT narrowed: its inbound leg counts wisp -> durable
// edges under the durable target's id.
// A nil set means membership could NOT be read (the wisps table is
// absent, the tolerated case) — never "no wisps": fail open to the full
// id list so the readers below keep their own table-not-exist tolerance.
wispSet, err := s.WispPlaneIDs(ctx, allIDs)
if err != nil {
return exportRelations{}, err
}
wispIDs := wispReaderIDs(allIDs, wispSet)
wispLabels, err := s.uw.LabelUseCase().GetLabelsForWisps(ctx, wispIDs) //nolint:forbidigo // bulk relation load; see GetLabelsForIssues above
if err != nil {
if !dberrors.IsTableNotExist(err) {
return exportRelations{}, fmt.Errorf("load wisp labels: %w", err)
@ -267,7 +293,7 @@ func (s *uowExportSource) LoadExportRelations(ctx context.Context, issues []*typ
} else {
mergeExportMap(rel.labels, wispLabels)
}
wispComments, err := s.uw.CommentUseCase().GetCommentsForWisps(ctx, allIDs)
wispComments, err := s.uw.CommentUseCase().GetCommentsForWisps(ctx, wispIDs)
if err != nil {
if !dberrors.IsTableNotExist(err) {
return exportRelations{}, fmt.Errorf("load wisp comments: %w", err)
@ -275,7 +301,7 @@ func (s *uowExportSource) LoadExportRelations(ctx context.Context, issues []*typ
} else {
mergeExportMap(rel.comments, wispComments)
}
wispCounts, err := s.uw.CommentUseCase().GetWispCommentCounts(ctx, allIDs)
wispCounts, err := s.uw.CommentUseCase().GetWispCommentCounts(ctx, wispIDs)
if err != nil {
if !dberrors.IsTableNotExist(err) {
return exportRelations{}, fmt.Errorf("load wisp comment counts: %w", err)
@ -319,17 +345,47 @@ func (s *uowExportSource) LoadExportRelations(ctx context.Context, issues []*typ
return rel, nil
}
// wispPlaneSubset returns, in input order, the ids that wispSet marks as
// wisp-plane; none marked yields nil, and the bulk readers answer nil with no
// statement at all. The caller decides what a nil SET means (see
// LoadExportRelations: unreadable membership fails open, never empty).
func wispPlaneSubset(ids []string, wispSet map[string]bool) []string {
var out []string
for _, id := range ids {
if wispSet[id] {
out = append(out, id)
}
}
return out
}
// wispReaderIDs is the caller-side policy wispPlaneSubset defers: a nil set
// means membership was UNREADABLE (WispPlaneIDs tolerated a missing wisps
// table), so every id goes to the wisp readers and they keep their own
// table-not-exist tolerance; a non-nil set — even an empty one — is the
// answer, and only the ids in it go.
func wispReaderIDs(ids []string, wispSet map[string]bool) []string {
if wispSet == nil {
return ids
}
return wispPlaneSubset(ids, wispSet)
}
// WispPlaneIDs classifies by wisps-table membership through the plane-pinned
// GetWispsByIDs read (row flags are NOT a substitute — see the exportSource
// interface comment). A rig without the wisp tables has no wisp-plane rows,
// mirroring the tolerance of the wisp-plane legs in LoadExportRelations.
// interface comment). A rig without the wisps table has no wisp-plane rows
// (nil set), mirroring the tolerance of the wisp-plane legs in
// LoadExportRelations. Only the WISPS table is optional: the read joins
// leases, and a blanket table-not-exist check would report a broken
// database as an empty wisp plane (the issue_search.go missingOptionalWispTable
// lesson), so the tolerance is pinned to that one table name.
func (s *uowExportSource) WispPlaneIDs(ctx context.Context, ids []string) (map[string]bool, error) {
if len(ids) == 0 {
return nil, nil
}
wisps, err := s.uw.IssueUseCase().GetWispsByIDs(ctx, ids)
if err != nil {
if dberrors.IsTableNotExist(err) {
if dberrors.IsMissingTable(err, "wisps") {
return nil, nil
}
return nil, fmt.Errorf("wisp plane membership: %w", err)

View file

@ -0,0 +1,43 @@
package main
import (
"reflect"
"testing"
)
// wispPlaneSubset feeds the wisp-plane relation readers in
// uowExportSource.LoadExportRelations: only ids that live in the wisps table
// go to them, in input order; a set that marks none sends nothing, so those
// readers issue no statement at all (wy-237yfi).
func TestWispPlaneSubset(t *testing.T) {
ids := []string{"a", "b", "c", "d"}
if got := wispPlaneSubset(ids, map[string]bool{}); got != nil {
t.Fatalf("empty set: want nil, got %v", got)
}
got := wispPlaneSubset(ids, map[string]bool{"d": true, "b": true, "zz": true, "a": false})
if !reflect.DeepEqual(got, []string{"b", "d"}) {
t.Fatalf("want [b d] in input order, got %v", got)
}
if got := wispPlaneSubset(nil, map[string]bool{"a": true}); got != nil {
t.Fatalf("no ids: want nil, got %v", got)
}
}
// wispReaderIDs is the policy on top: a nil set is "membership unreadable"
// and must FAIL OPEN to every id — narrowing on it would silently export
// every wisp with no labels and no comments — while a non-nil empty set is a
// real answer (no wisps) and sends nothing.
func TestWispReaderIDs(t *testing.T) {
ids := []string{"a", "b", "c"}
if got := wispReaderIDs(ids, nil); !reflect.DeepEqual(got, ids) {
t.Fatalf("nil set must fail open to all ids, got %v", got)
}
if got := wispReaderIDs(ids, map[string]bool{}); got != nil {
t.Fatalf("empty non-nil set means no wisps: want nil, got %v", got)
}
if got := wispReaderIDs(ids, map[string]bool{"c": true}); !reflect.DeepEqual(got, []string{"c"}) {
t.Fatalf("want [c], got %v", got)
}
}

View file

@ -27,7 +27,13 @@ This uses the Tim Sehn recipe:
4. Swap main branch to the new flattened branch
5. Prune remote-tracking refs (they would keep the old history alive;
the next push or fetch re-creates them at the new tip)
6. Run Dolt GC to reclaim space from old history
6. Run a full Dolt GC (all storage generations) to reclaim the old history
The GC pass is a full collection: Dolt storage is generational, and a default
GC never revisits data an earlier GC moved to the old generation. On any store
that has been GC'd before (bd gc, or a previous flatten or compact), only a
full collection reclaims the squashed history. A full GC can take minutes on
multi-gigabyte stores.
This is irreversible — all commit history is lost. The resulting database
has exactly one commit containing all current data.
@ -143,11 +149,7 @@ Examples:
printPruneReport(pruned, tags)
}
if gc, ok := storage.UnwrapStore(store).(storage.GarbageCollector); ok {
if err := gc.DoltGC(ctx); err != nil {
WarnError("dolt gc after flatten failed: %v", err)
}
}
gcMode := runPostRewriteGC(ctx, "flatten")
sizeAfter := storeSizeBytes(ctx)
elapsed := time.Since(start)
@ -161,6 +163,9 @@ Examples:
"tags_anchoring": tags,
"elapsed_ms": elapsed.Milliseconds(),
}
if gcMode != "" {
result["gc_mode"] = gcMode
}
addGCSizeJSON(result, sizeBefore, sizeAfter)
return outputJSON(result)
}

View file

@ -0,0 +1,126 @@
//go:build cgo
package main
import (
"encoding/json"
"os/exec"
"strings"
"testing"
)
type gateFrontDoorRunner struct {
name string
dir string
env func(string) []string
}
func (r gateFrontDoorRunner) run(t *testing.T, bd string, args ...string) (string, string, error) {
t.Helper()
cmd := exec.Command(bd, args...)
cmd.Dir = r.dir
cmd.Env = r.env(r.dir)
stdout, stderr, err := runCommandBuffers(t, cmd)
return stdout.String(), stderr.String(), err
}
func (r gateFrontDoorRunner) mustRun(t *testing.T, bd string, args ...string) string {
t.Helper()
stdout, stderr, err := r.run(t, bd, args...)
if err != nil {
t.Fatalf("[%s] bd %s failed: %v\nstdout:\n%s\nstderr:\n%s",
r.name, strings.Join(args, " "), err, stdout, stderr)
}
return stdout
}
// TestProxiedServerGateFrontDoorDirectServerProxyParity keeps gate lifecycle writes on the
// same UOW front in direct SQL and proxied-server modes. Gate IDs are generated
// independently, so assertions compare stable lifecycle and blocking facts.
func TestProxiedServerGateFrontDoorDirectServerProxyParity(t *testing.T) {
requireSharedProxiedServer(t)
bd := buildEmbeddedBD(t)
directProject := newServerModeProject(t, bd, "mg")
proxiedProject := newSharedProxiedProject(t, bd, "mg")
modes := []gateFrontDoorRunner{
{name: "direct-sql", dir: directProject.dir, env: func(string) []string { return directProject.env }},
{name: "proxied", dir: proxiedProject.dir, env: bdProxiedEnv},
}
for _, r := range modes {
r.mustRun(t, bd, "create", "Gate target", "--id", "mg-target")
}
for _, r := range modes {
created := r.mustRun(t, bd, "gate", "create", "--type=human", "--blocks", "mg-target", "--reason", "review required")
gateID := parseCreatedGateID(t, created)
for _, want := range []string{"Blocks: mg-target", "Reason: review required", "Resolve with: bd gate resolve " + gateID} {
if !strings.Contains(created, want) {
t.Errorf("[%s] gate create missing %q:\n%s", r.name, want, created)
}
}
show := r.mustRun(t, bd, "gate", "show", gateID)
for _, want := range []string{gateID, "Status: open", "Await Type: human", "Reason: review required"} {
if !strings.Contains(show, want) {
t.Errorf("[%s] gate show missing %q:\n%s", r.name, want, show)
}
}
// A waiter is persisted by the same gate UOW and remains visible in
// list output before resolution.
r.mustRun(t, bd, "gate", "add-waiter", gateID, "worker1")
var rows []map[string]any
list := r.mustRun(t, bd, "gate", "list", "--all", "--json")
if err := json.Unmarshal([]byte(strings.TrimSpace(list)), &rows); err != nil {
t.Fatalf("[%s] parse gate list: %v\n%s", r.name, err, list)
}
var row map[string]any
for _, candidate := range rows {
if candidate["id"] == gateID {
row = candidate
break
}
}
if row == nil || row["status"] != "open" {
t.Fatalf("[%s] gate list row = %v, want open gate %s", r.name, row, gateID)
}
waiters, ok := row["waiters"].([]any)
if !ok || len(waiters) != 1 || waiters[0] != "worker1" {
t.Errorf("[%s] gate list waiters = %v, want [worker1]", r.name, row["waiters"])
}
resolved := r.mustRun(t, bd, "gate", "resolve", gateID, "--reason", "approved")
if !strings.Contains(resolved, "Gate resolved: "+gateID) {
t.Errorf("[%s] gate resolve output missing confirmation:\n%s", r.name, resolved)
}
closed := r.mustRun(t, bd, "gate", "show", gateID)
if !strings.Contains(closed, "Status: closed") {
t.Errorf("[%s] gate remains open after resolve:\n%s", r.name, closed)
}
// `gate check` is the watcher-facing read/transaction boundary. A
// resolved human gate is a clean no-op and must still exit successfully.
checked := r.mustRun(t, bd, "gate", "check", "--type", "human")
const wantGateCheck = "No open gates of type 'human' found."
if strings.TrimSpace(checked) != wantGateCheck {
t.Errorf("[%s] gate check output = %q, want %q", r.name, checked, wantGateCheck)
}
// A bead gate exercises the transactional watcher path: once the
// awaited bead closes, gate check must close the gate and report one
// resolution (rather than merely observing an already-closed gate).
r.mustRun(t, bd, "create", "Awaited bead", "--id", "mg-awaited")
beadCreated := r.mustRun(t, bd, "gate", "create", "--type=bead", "--blocks", "mg-target", "--await-id", "mg-awaited", "--reason", "awaited")
beadGateID := parseCreatedGateID(t, beadCreated)
r.mustRun(t, bd, "close", "mg-awaited", "--reason", "awaited bead complete")
beadChecked := r.mustRun(t, bd, "gate", "check", "--type", "bead")
if !strings.Contains(strings.ToLower(beadChecked), "1 resolved") {
t.Errorf("[%s] bead gate check output = %q, want one resolved gate", r.name, beadChecked)
}
beadShow := r.mustRun(t, bd, "gate", "show", beadGateID)
if !strings.Contains(beadShow, "Status: closed") {
t.Errorf("[%s] bead gate %s remains open after check:\n%s", r.name, beadGateID, beadShow)
}
}
}

View file

@ -16,6 +16,7 @@ var (
gcOlderThan int
gcSkipDecay bool
gcSkipDolt bool
gcFull bool
)
var gcCmd = &cobra.Command{
@ -32,12 +33,19 @@ Runs three phases in sequence:
Each phase can be skipped individually. Use --dry-run to preview all phases
without making changes.
Phase 3 runs Dolt's default, generational GC: it only examines data written
since the last GC. Data that survived an earlier GC lives in the old generation
and is never revisited, so on a long-lived store the space freed by decay may
not be reclaimed. Use --full to collect all generations (slower on large
stores).
Examples:
bd gc # Full GC with defaults (90 day decay)
bd gc --dry-run # Preview what would happen
bd gc --older-than 30 # Decay issues closed 30+ days ago
bd gc --skip-decay # Skip issue deletion, just compact+GC
bd gc --skip-dolt # Skip Dolt GC, just decay+compact
bd gc --full # Collect all storage generations
bd gc --force # Skip confirmation prompt`,
SilenceUsage: true,
SilenceErrors: true,
@ -190,7 +198,11 @@ Examples:
results = append(results, phaseResult{name: "Dolt GC", detail: "not supported"})
} else if gcDryRun {
if !jsonOutput {
fmt.Println(" Would run DOLT_GC()")
if gcFull {
fmt.Println(" Would run a full DOLT_GC() (all storage generations)")
} else {
fmt.Println(" Would run DOLT_GC()")
}
}
results = append(results, phaseResult{name: "Dolt GC", detail: "dry-run"})
} else {
@ -200,7 +212,11 @@ Examples:
// (bd-agctw). Sizes are reported so a no-op reclaim is visible.
sizeBefore := storeSizeBytes(ctx)
remoteRefs, tags := listRemoteRefsAndTags(ctx)
if err := gc.DoltGC(ctx); err != nil {
if gcFull && !jsonOutput {
fmt.Println(" Running full Dolt GC (all generations; can take minutes on large stores)...")
}
gcMode, err := runDoltGCPass(ctx, gc, gcFull)
if err != nil {
WarnError("dolt gc failed: %v", err)
results = append(results, phaseResult{name: "Dolt GC", detail: "failed"})
} else {
@ -211,6 +227,9 @@ Examples:
}
if !jsonOutput {
fmt.Printf(" Done (%s)\n", detail)
if gcMode != gcModeFull && suggestFullGC(sizeBefore, sizeAfter) {
printFullGCHint()
}
if len(remoteRefs)+len(tags) > 0 {
fmt.Printf(" Note: %d remote-tracking ref(s) and %d tag(s) anchor history;\n", len(remoteRefs), len(tags))
fmt.Printf(" after a history squash, use bd flatten / bd compact so they are pruned first.\n")
@ -220,6 +239,7 @@ Examples:
gcSizeInfo = map[string]interface{}{
"remote_refs": len(remoteRefs),
"tags": len(tags),
"mode": gcMode,
}
addGCSizeJSON(gcSizeInfo, sizeBefore, sizeAfter)
}
@ -275,6 +295,7 @@ func init() {
gcCmd.Flags().IntVar(&gcOlderThan, "older-than", 90, "Delete closed issues older than N days")
gcCmd.Flags().BoolVar(&gcSkipDecay, "skip-decay", false, "Skip issue deletion phase")
gcCmd.Flags().BoolVar(&gcSkipDolt, "skip-dolt", false, "Skip Dolt garbage collection phase")
gcCmd.Flags().BoolVar(&gcFull, "full", false, "Run a full Dolt GC (all generations; slower, reclaims space default passes cannot)")
rootCmd.AddCommand(gcCmd)
}

View file

@ -126,6 +126,34 @@ func TestEmbeddedGC(t *testing.T) {
}
})
// ===== Full GC =====
t.Run("gc_full", func(t *testing.T) {
gcDir, _, _ := bdInit(t, bd, "--prefix", "gu")
for i := 0; i < 3; i++ {
bdCreate(t, bd, gcDir, fmt.Sprintf("Full GC issue %d", i), "--type", "task")
}
// A default pass first, so the second one has an old generation to
// collect — the case a generational pass cannot reclaim (#5907).
bdGC(t, bd, gcDir, "--force", "--skip-decay")
out := bdGC(t, bd, gcDir, "--full", "--force", "--skip-decay")
if !strings.Contains(out, "GC complete") {
t.Errorf("expected 'GC complete' from bd gc --full: %s", out)
}
if !strings.Contains(out, "full Dolt GC") {
t.Errorf("expected the full-GC notice: %s", out)
}
})
t.Run("gc_full_dry_run", func(t *testing.T) {
out := bdGC(t, bd, dir, "--full", "--dry-run")
if !strings.Contains(out, "full DOLT_GC()") {
t.Errorf("dry run does not say it would run a full GC: %s", out)
}
})
// ===== No --force prompts =====
t.Run("gc_no_force_prompts", func(t *testing.T) {

View file

@ -155,12 +155,23 @@ func runGCProxiedServer(ctx context.Context) error {
}
if gcDryRun {
if !jsonOutput {
fmt.Println(" Would run DOLT_GC() and DOLT_STATS_GC()")
if gcFull {
fmt.Println(" Would run a full DOLT_GC() (all storage generations) and DOLT_STATS_GC()")
} else {
fmt.Println(" Would run DOLT_GC() and DOLT_STATS_GC()")
}
}
results = append(results, phaseResult{name: "Dolt GC", detail: "dry-run"})
} else {
if gcFull && !jsonOutput {
fmt.Println(" Running full Dolt GC (all generations; can take minutes on large stores)...")
}
err := runProxiedNonTx(ctx, func(ctx context.Context, conn *sql.Conn) error {
if err := versioncontrolops.DoltGC(ctx, conn); err != nil {
doltGC := versioncontrolops.DoltGC
if gcFull {
doltGC = versioncontrolops.DoltGCFull
}
if err := doltGC(ctx, conn); err != nil {
return err
}
if _, err := conn.ExecContext(ctx, "CALL DOLT_STATS_GC()"); err != nil {

Some files were not shown because too many files have changed in this diff Show more