beads/cmd/bd/shared_store_migrate_gate_test.go
Julian Knutsen 8bb2781a8b
fix(storage): shared stores refuse version-bump migration without explicit consent (#5920, #5043) (#6048)
* feat(schema): add the shared-store migration gate and a locked migration gate hook

Grows checkRemoteMigrateGate with a `shared` flag, exposed as
CheckSharedStoreMigrateGate. On a shared store (a database served to
co-resident bd clients) the gate:

  - refuses a version-bump migration when there is NO remote and the
    operator has not consented, with the new "shared-no-remote" decision
    on the existing RemoteMigrateGateError. Promoting the schema cursor
    on a shared server locks out every client still on an older bd —
    gastownhall/beads#5920;
  - suppresses the smart gate's two auto-EXECUTE arms with a remote
    configured. The safe-first-mover argument is about clones, not
    clients; it firing on a shared server IS the #5920 mechanism.

A fresh database (current == 0) still migrates: creating a database is
consent for its schema. Consent surfaces are the migrate verb (new
process-local SetSharedMigrateConsent, unlocking only the no-remote arm),
--force, and the reused BD_ALLOW_REMOTE_MIGRATE.

schema.WithMigrationGate gives MigrateUpWithLock a pre-migration hook for
callers whose migration runs entirely inside it. It sits after locked
preparation and before MigrateUp, so the 4a3c02abc converged fast path
still costs it zero statements.

No call site changes behavior yet: every existing exported variant passes
shared=false, so embedded recompiles untouched.

* fix(storage): refuse version-bump migration on shared dolt sql-server stores

Switches internal/storage/dolt's writable-open gate to the shared variant.
A DoltStore is always a sql-server client, and a sql-server accepts
co-resident clients by construction — the property that distinguishes it
from the flock-guarded single-writer embedded store — so this open must
never promote the schema cursor on its own.

This closes gastownhall/beads#5920 at its mechanism: the version-bump
reconciliation (autoMigrateOnVersionBump) opens its OWN writable store for
every non-preview command, including read-only ones, and that open is what
migrated a shared database out from under a team of co-resident v1.2.2
clients. It now refuses, and the refusal — previously swallowed to
debug.Logf with everything else — prints a one-shot stderr notice saying
nothing was applied and what to run instead.

`bd migrate` / `bd migrate schema` set the new verb consent in the root
pre-run (withheld under a preview flag), so the typed verb still migrates.

Tests: a full-CLI #5920 regression (server mode, no remote, stale
.local_version, `bd list` must not promote MAX(version) — it does promote
it, 65 -> 66, with this commit's one-line gate swap reverted), the
server-mode open matrix on a real sql-server (writable refuses, read-only
succeeds, verb consent migrates), refusal permanence in the retry loop,
and the agent-facing JSON block.

* docs(upgrading): document the shared-server upgrade path

Adds a "Shared servers" section: a version bump never auto-migrates a
database served to co-resident clients, upgrade every client first, then
consent once with `bd migrate schema`, and `bd serve` refuses to start
until the schema is reconciled.

Two corrections to what was already there:

  - the smart gate's "auto-migrates as a safe first-mover" arm is now
    embedded-only, so the bullet says so and links to the new section;
  - the literal `BD_ALLOW_REMOTE_MIGRATE=1 bd migrate` recipe is removed
    in favor of "run bd migrate as an explicit step in exactly one job; if
    the gate blocks a run it prints the options and the scripted
    override". Per settled decision 4
    (engdocs/decisions/2026-07-17-docs-release-pin.md), the escape hatch
    is not documented as a recipe — the gate prints its own guidance and
    the docs defer to it. This brings the page back into compliance
    rather than amending the decision.

* fix(storage): address review of the shared-store migration gate

CI blocker: cmd/bd/shared_store_migrate_gate_test.go uses cgo-tagged
helpers but carried no build tag, so the required pure-Go compile lane
(CGO_ENABLED=0 go test -tags gms_pure_go -c ./cmd/bd) failed on undefined
symbols. Tagged //go:build cgo, like every sibling Dolt-server test.

Truth at the merge point. This PR gates the sql-server store; the proxied /
`bd serve` plane is #6055. So its text no longer claims that plane: the
shared-no-remote body, the gate's doc comment, the decision constant, and
WithMigrationGate's doc all describe a dolt sql-server, and upgrading.md
drops the `bd serve` paragraph. #6055 adds them back with the code.

Remedies now match the refusal they are attached to:

  - the one-shot version-bump notice keyed only on the error TYPE, so an
    "adopt" refusal (another clone already migrated) told the operator to
    run `bd migrate schema` — a command the verb consent deliberately does
    not unlock for that arm. It now picks the line per Decision, and stays
    silent under --json, where it was corrupting the gate's own JSON block
    on the same stream;
  - a new Shared flag marks refusals that came from a shared store, so the
    adopt and adopt-ff options can say that adopting promotes the schema
    for every co-resident client. adopt-ff previously advertised
    "Risk: none", which is true only of local data;
  - handleRemoteMigrateGateJSON gains the missing adopt-ff case (now a
    routine outcome, since the auto-execute arm is suppressed), and
    shared-no-remote points at the #shared-servers docs anchor instead of
    inheriting the migrate-or-adopt one;
  - `bd init` joining a behind shared server exits before writing
    metadata.json, so its refusal now says where the prescribed command
    can actually be run;
  - the beads_global refusal names `bd migrate schema --global`, in the
    init warning and under --global at the store open — the unflagged
    command migrates the project database and leaves the refusal in place;
  - a consented `bd migrate schema` that hits the dirty-table guard names
    the recovery that terminates, instead of pointing at `bd dolt commit`,
    which is itself gated in server mode and points back.

Behavior: consent is now scoped to `bd migrate schema` alone. Bare
`bd migrate` never applies a schema migration itself, and
`bd migrate --update-repo-id` is repo-fingerprint surgery — neither should
promote the schema for a whole server as a side effect. A consented
`bd migrate schema` now also stamps the version marker the refused
reconciliation could not, so `bd doctor` stops reporting a permanent
mismatch after the operator has done exactly what the refusal asked.

Library API: beads.AllowSharedSchemaMigration is the programmatic consent
embedders had no way to give except os.Setenv, which is process-global and
inherited by every child.

Cleanup: the force/env consent ladder was hand-copied into both arms; it is
now one helper that also names the surface that granted consent, so the
warning can stop hardcoding a command the operator may not have typed. Two
gate wrappers left with no callers at all are deleted.

* fix(cli): make --json shared-store migrate gate --global-aware

The shared-no-remote refusal's JSON gate block hardcoded the project-scoped
`bd migrate schema` consent command, so a --json/agent consumer operating on
the global database (--global) was handed a command that consents the wrong
database and leaves the refusal in place. The human/text path was already
--global-aware (printGlobalDatabaseConsentHint, noticeSharedMigrateRefusal),
but the notice self-suppresses under --json, making the gate block the sole
channel for that consumer.

Mirror the human-path consent idiom in handleRemoteMigrateGateJSON: retarget
both the top-level `expected` directive and the runnable migrate-shared option
command to `bd migrate schema --global` when the refused open targeted the
global database, scoped strictly to the shared-no-remote decision. Add a JSON
--global regression test at parity with TestPrintGlobalDatabaseConsentHint.

Also tighten the AllowSharedSchemaMigration doc to state it authorizes only the
no-remote shared arm (a remote-backed shared store still needs --force /
AllowRemoteMigrateEnv), matching the exported API's actual reach.

Maintainer review fixup for PR #6048.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 21:47:24 -07:00

202 lines
7.2 KiB
Go

//go:build cgo
package main
import (
"context"
"database/sql"
"errors"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/storage/dolt"
"github.com/steveyegge/beads/internal/storage/doltutil"
"github.com/steveyegge/beads/internal/storage/schema"
"github.com/steveyegge/beads/internal/types"
)
// TestSharedServerVersionBumpDoesNotPromoteCursor reproduces
// gastownhall/beads#5920 end to end, through the real CLI.
//
// The incident: a shared dolt sql-server with no remote, a team of co-resident
// bd clients, one of them upgraded. That client's first command was a plain
// `bd list` — a read — but the root pre-run's version-bump reconciliation
// (autoMigrateOnVersionBump) opens its OWN writable store, which migrated the
// database and promoted schema_migrations for everyone. Every co-resident
// client still on the old binary then refused the database until it was
// upgraded too, for ~14 minutes, with nothing in the output saying what had
// happened.
//
// The assertion that matters is MAX(version): the read must succeed and the
// cursor must not move.
func TestSharedServerVersionBumpDoesNotPromoteCursor(t *testing.T) {
if testDoltServerPort == 0 {
t.Skip("Dolt test server not available, skipping")
}
repo := t.TempDir()
beadsDir := filepath.Join(repo, ".beads")
if err := os.MkdirAll(beadsDir, 0o755); err != nil {
t.Fatalf("create .beads: %v", err)
}
database := uniqueTestDBName(t)
if err := (&configfile.Config{
Backend: configfile.BackendDolt,
DoltMode: configfile.DoltModeServer,
DoltServerHost: "127.0.0.1",
DoltServerPort: testDoltServerPort,
DoltDatabase: database,
}).Save(beadsDir); err != nil {
t.Fatalf("save metadata: %v", err)
}
ctx := context.Background()
// SetForceAllowRemoteMigrate is deliberately NOT set: this fixture creates
// the database from scratch, so the gate sees version 0 and allows it —
// creating a database is consent for its schema.
store, err := dolt.New(ctx, &dolt.Config{
Path: filepath.Join(beadsDir, "dolt"),
BeadsDir: beadsDir,
ServerHost: "127.0.0.1",
ServerPort: testDoltServerPort,
Database: database,
CreateIfMissing: true,
})
if err != nil {
t.Fatalf("create test store: %v", err)
}
if err := store.SetConfig(ctx, "issue_prefix", "shr"); err != nil {
t.Fatalf("set issue_prefix: %v", err)
}
now := time.Now()
if err := store.CreateIssue(ctx, &types.Issue{
ID: "shr-1",
Title: "Readable through the upgrade window",
Status: types.StatusOpen,
Priority: 1,
IssueType: types.TypeTask,
CreatedAt: now,
UpdatedAt: now,
}, "test-user"); err != nil {
t.Fatalf("create issue: %v", err)
}
if err := store.Close(); err != nil {
t.Fatalf("close setup store: %v", err)
}
// The version-bump reconciliation only runs when the workspace has a local
// Dolt data dir (it stats cfg.DatabasePath before opening anything) — the
// shape of the auto-started localhost server the incident was reported on.
// The bytes live in the test container, but the directory is what arms the
// code path under test.
if err := os.MkdirAll(filepath.Join(beadsDir, "dolt"), 0o755); err != nil {
t.Fatalf("create local dolt data dir: %v", err)
}
t.Cleanup(func() { dropTestDatabase(database, testDoltServerPort) })
admin, err := sql.Open("mysql", doltutil.ServerDSN{
Host: "127.0.0.1", Port: testDoltServerPort, User: "root", Database: database,
}.String())
if err != nil {
t.Fatalf("connect to test dolt server: %v", err)
}
t.Cleanup(func() { admin.Close() })
// Put the database one migration behind the candidate binary by regressing
// only the recorded cursor — the schema change itself stays applied, so
// reads keep working, exactly as they did throughout the incident.
if _, err := admin.ExecContext(ctx,
"DELETE FROM schema_migrations WHERE version = ?", schema.LatestVersion()); err != nil {
t.Fatalf("regress schema_migrations: %v", err)
}
cursor := func() int {
var v int
if err := admin.QueryRowContext(ctx,
"SELECT COALESCE(MAX(version), 0) FROM schema_migrations").Scan(&v); err != nil {
t.Fatalf("read schema cursor: %v", err)
}
return v
}
behind := cursor()
if behind != schema.LatestVersion()-1 {
t.Fatalf("precondition failed: cursor = %d, want %d", behind, schema.LatestVersion()-1)
}
// The one-shot upgrade signal: a stale .local_version makes trackBdVersion
// report a version bump, which is what arms autoMigrateOnVersionBump. The
// value has to be strictly below the candidate binary's Version and still
// in the 1.x era — a pre-1.0 witness is refused outright by the cross-era
// legacy guard long before any of this runs.
if err := os.WriteFile(filepath.Join(beadsDir, localVersionFile), []byte("1.0.0\n"), 0o600); err != nil {
t.Fatalf("write stale %s: %v", localVersionFile, err)
}
bd := buildBDForInitTests(t)
run := func(args ...string) (string, string, int) {
t.Helper()
cmd := exec.Command(bd, args...)
cmd.Dir = repo
cmd.Env = append(filteredEnvForContextBinding("BEADS_DIR", "BEADS_DB", "BD_DB",
"BEADS_DOLT_SERVER_PORT", "BEADS_DOLT_SERVER_DATABASE", schema.AllowRemoteMigrateEnv),
"HOME="+t.TempDir(),
"XDG_CONFIG_HOME="+t.TempDir(),
"BEADS_TEST_MODE=1",
"BEADS_TEST_SERVER=1",
"BEADS_DIR="+beadsDir,
schema.AllowRemoteMigrateEnv+"=0",
)
var stdout, stderr strings.Builder
cmd.Stdout = &stdout
cmd.Stderr = &stderr
code := 0
if err := cmd.Run(); err != nil {
var exitErr *exec.ExitError
if !errors.As(err, &exitErr) {
t.Fatalf("running bd %v: %v", args, err)
}
code = exitErr.ExitCode()
}
return stdout.String(), stderr.String(), code
}
stdout, stderr, code := run("list")
if code != 0 {
t.Fatalf("bd list exit = %d, want 0\nstdout:\n%s\nstderr:\n%s", code, stdout, stderr)
}
if !strings.Contains(stdout, "shr-1") {
t.Errorf("bd list did not render the issue; reads must keep working on the old schema:\n%s", stdout)
}
if got := cursor(); got != behind {
t.Fatalf("bd list promoted the schema cursor from %d to %d — this IS gastownhall/beads#5920", behind, got)
}
// The #5920 UX gap, inverted: nothing advanced, and the one-shot line says so.
if !strings.Contains(stderr, "not auto-applying") || !strings.Contains(stderr, "bd migrate schema") {
t.Errorf("expected the one-shot shared-store notice on stderr, got:\n%s", stderr)
}
// A write is refused outright, with the full guidance block rather than a
// bare error.
stdout, stderr, code = run("create", "Should be refused", "-p", "2")
if code == 0 {
t.Fatalf("bd create succeeded on a behind shared database\nstdout:\n%s\nstderr:\n%s", stdout, stderr)
}
if !strings.Contains(stderr, "co-resident") {
t.Errorf("bd create refusal missing the shared-store guidance:\n%s", stderr)
}
if got := cursor(); got != behind {
t.Fatalf("the refused write promoted the schema cursor from %d to %d", behind, got)
}
// Consent completes the upgrade.
stdout, stderr, code = run("migrate", "schema")
if code != 0 {
t.Fatalf("bd migrate schema exit = %d, want 0\nstdout:\n%s\nstderr:\n%s", code, stdout, stderr)
}
if got := cursor(); got != schema.LatestVersion() {
t.Fatalf("cursor = %d after bd migrate schema, want %d", got, schema.LatestVersion())
}
}