beads/cmd/bd/migrate_dolt_mode.go
matt wilkie 6d81e1c73a
feat(workspacegate): shared physical-root resolver, beads.OpenGated, and CLI gate wiring (part 2) (#5093)
* feat(workspacegate): wire the gate through a shared physical-root resolver, OpenGated, and the CLI (part 2)

Part 2 of the workspace-gate work started in #5046, which landed the
internal/workspacegate primitive and deliberately held back OpenGated
because its private physical-root derivation was wrong in exactly the
ways review found: it skipped the store's real mode resolution and
silently skipped the physical gate on metadata-less workspaces.

internal/doltserver.ResolvePhysicalRoots is now the one shared answer
to 'which local directory will this workspace's store actually open':
side-effect free (never triggers configfile.Load's legacy-migration
write, never mkdirs the shared-server tree), central-config aware
(the dolt_mode inheritance that previously existed only inside
storage/dolt/open.go), correct for embedded (embeddeddolt, where
ResolveDoltDir answers dolt/), proxied (env > client-info sidecar >
default, hoisted out of package main so it cannot diverge), shared
(no-mkdir path), remote backends (workspace gate only), and the
supported no-metadata layouts (embeddeddolt-then-dolt stat probe).

beads.OpenGated gives library consumers a store that holds SHARED
workspace+physical gates for its lifetime. The CLI acquires at one
chokepoint after workspace resolution - SHARED for store-opening
commands, EXCLUSIVE for backup restore - and in-command for the
noDbCommands that replace state: init, migrate from-*-to-* (both
source and destination roots, ahead of migrate.lock per the normative
lock order), and bootstrap.

Posture is deliberate and asymmetric: SHARED acquisition failures that
are not contention warn once and continue ungated (the gate is
cooperative; normal commands must not brick existing deployments),
while contention aborts with the holder's identity, and EXCLUSIVE
failures of any kind refuse - maintenance must not pretend to a fence
it does not hold. Known residual: discovery-path config loads before
the chokepoint can still fire the legacy-migration write; deferring
that is follow-up scope alongside doctor, secondary-target opens, and
bd dolt set.

Agent-Signature: claude-fable-5-medium on behalf of matt wilkie
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AQHiBWVPSXDCcUSG9EGw9d

* fix(workspacegate): nil store on failed factory open before gate-release cleanup

newDoltStore's concrete factory branches (e.g. server-mode dolt.New) return
a typed-nil pointer on error; assigning it to the storage.DoltStorage
interface makes 'store != nil' true in closeStoreBeforeGateRelease, which
then calls Close on a nil receiver and panics (SIGSEGV observed in
TestPersistentPreRunHonorsSkipStoreAnnotation, verify run 20260727T000727Z).
No store was opened on that path, so drop the value before the error return.

Agent-Signature: claude-fable-5-high on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P3JtQ3ukoVmfg6LCKcKBBh

* fix(workspacegate): address review — unparam, gate-busy lock classification, bootstrap dolt-dir gating

Three fixes from the adversarial review of gastownhall/beads#5093
(see reports/2026-07-26-reviewer-dispatch-sweep.md, "#5093" section):

- unparam (CI lint fail): drop the dead PhysicalRoots return value from
  buildWorkspaceGateSet; both call sites already discarded it, and the
  code comments already document that provenance is deliberately not
  surfaced in user-facing errors. Updated both callers.

- TestEmbeddedInitConcurrent regression (CI test fail): the new EXCLUSIVE
  workspace gate in cmd/bd/init.go short-circuits before the embedded
  Dolt flock is ever attempted, so a losing concurrent `bd init` now
  reports gate-busy output that isEmbeddedLockOutput did not recognize.
  Taught isEmbeddedLockOutput to also match workspacegate.ErrBusy's
  "workspace gate busy" text, and added a deterministic
  TestInitGateBusyClassifiedAsLockContention that holds the gate
  in-process and exercises the exact classification path, independent of
  subprocess race timing.

- Bootstrap dolt-dir gating gap: executeBootstrapPlan gated
  plan.BeadsDir's resolved physical roots but not
  doltserver.ResolveDoltDir(plan.BeadsDir), the directory
  executeInitAction/executeRestoreAction/executeJSONLImportAction
  actually open. Pass it as an extraRoot, mirroring how bd init passes
  its own resolved db path.

Also fixes a locally-discovered flake in
TestChokepointSharedExcludesMigrateExclusive: acquireMigrateGates now
threads the package-global rootCtx into acquireExclusiveWorkspaceGates,
so a prior in-process test that ran the full command path (which sets
rootCtx via setupGracefulShutdown and cancels it via rootCancel() without
resetting the var) can leak an already-canceled context into this test.
Pin rootCtx to nil for the test's duration, matching the isolation
pattern already used elsewhere in this file. Reproduced deterministically
by injecting a canceled context before the fix, confirmed neutralized
after.

TestEmbeddedInitConcurrent itself still does not reproduce the CI
gate-busy symptom on this dev machine (10/10 processes succeed within
the 5s exclusive-gate wait, so the test's own "at least 1 lock error"
assertion still fails locally) — that is a machine-speed-dependent
property of the test's contention window, not the classification bug;
the classification fix is independently verified by the new deterministic
test and by code inspection of workspacegate.ErrBusy's text.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 01:41:20 -07:00

485 lines
17 KiB
Go

package main
import (
"fmt"
"os"
"path/filepath"
"time"
"github.com/spf13/cobra"
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/config"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/doltserver"
"github.com/steveyegge/beads/internal/lockfile"
"github.com/steveyegge/beads/internal/metrics"
"github.com/steveyegge/beads/internal/storage/dbproxy/proxy"
"github.com/steveyegge/beads/internal/storage/dbproxy/server"
"github.com/steveyegge/beads/internal/storage/dbproxy/util"
"github.com/steveyegge/beads/internal/ui"
)
const migrateLockFileName = "migrate.lock"
func migrateToProxiedRunE(metricName, checkName string, shared bool) func(*cobra.Command, []string) error {
return func(cmd *cobra.Command, _ []string) error {
evt := metrics.NewCommandEvent(metricName)
defer func() {
if c := metrics.Global(); c != nil {
c.CloseEventAndAdd(evt)
}
}()
dryRun, _ := cmd.Flags().GetBool("dry-run")
if !dryRun {
CheckReadonly(checkName)
}
idleTimeout, err := resolveMigrateIdleTimeout(cmd)
if err != nil {
return err
}
return runMigrateToProxiedServer(dryRun, idleTimeout, shared)
}
}
func migrateFromProxiedRunE(metricName, checkName string, shared bool) func(*cobra.Command, []string) error {
return func(cmd *cobra.Command, _ []string) error {
evt := metrics.NewCommandEvent(metricName)
defer func() {
if c := metrics.Global(); c != nil {
c.CloseEventAndAdd(evt)
}
}()
dryRun, _ := cmd.Flags().GetBool("dry-run")
if !dryRun {
CheckReadonly(checkName)
}
return runMigrateFromProxiedServer(dryRun, shared)
}
}
var migrateToProxiedServerCmd = &cobra.Command{
Use: "from-server-to-proxied-server",
Short: "[EXPERIMENTAL] Switch a server-mode repo to proxied-server mode",
SilenceUsage: true,
SilenceErrors: true,
Long: `Switch a repo from server mode (bd init --server) to proxied-server mode.
Both modes root their dolt sql-server at the same .beads/dolt directory, so this
only rewrites .beads/metadata.json (dolt_mode) and writes the proxied-server
sidecar — no Dolt data is copied or moved. Stop the running server first with
'bd dolt stop'.
Note: dolt_mode lives in the committed metadata.json, so this change propagates
to clones on the next push.`,
Args: cobra.NoArgs,
RunE: migrateToProxiedRunE("migrate-to-proxied-server", "migrate from-server-to-proxied-server", false),
}
var migrateSharedToProxiedServerCmd = &cobra.Command{
Use: "from-shared-server-to-proxied-server",
Short: "[EXPERIMENTAL] Switch a shared-server repo to proxied-server mode",
SilenceUsage: true,
SilenceErrors: true,
Long: `Switch a repo from shared-server mode to proxied-server mode.
The proxied server is rooted at the shared dolt directory
(~/.beads/shared-server/dolt), so no Dolt data is copied or moved; this rewrites
.beads/metadata.json (dolt_mode), turns off dolt.shared-server for this repo, and
writes the proxied-server sidecar. Stop the running shared server first with
'bd dolt stop' — note that stops it for every project sharing it.`,
Args: cobra.NoArgs,
RunE: migrateToProxiedRunE("migrate-shared-to-proxied-server", "migrate from-shared-server-to-proxied-server", true),
}
var migrateToServerCmd = &cobra.Command{
Use: "from-proxied-server-to-server",
Short: "[EXPERIMENTAL] Switch a proxied-server repo to server mode",
SilenceUsage: true,
SilenceErrors: true,
Long: `Switch a repo from proxied-server mode to server mode (bd init --server).
Both modes root their dolt sql-server at the same .beads/dolt directory, so this
only rewrites .beads/metadata.json (dolt_mode) and removes the proxied-server
sidecar — no Dolt data is copied or moved. Stop the running proxy first with
'bd dolt stop'.
Note: dolt_mode lives in the committed metadata.json, so this change propagates
to clones on the next push.`,
Args: cobra.NoArgs,
RunE: migrateFromProxiedRunE("migrate-to-server", "migrate from-proxied-server-to-server", false),
}
var migrateToSharedServerCmd = &cobra.Command{
Use: "from-proxied-server-to-shared-server",
Short: "[EXPERIMENTAL] Switch a proxied-server repo back to shared-server mode",
SilenceUsage: true,
SilenceErrors: true,
Long: `Switch a repo from proxied-server mode back to shared-server mode.
Only applies to a proxied-server repo rooted at the shared dolt directory
(~/.beads/shared-server/dolt) — the reverse of from-shared-server-to-proxied-server.
This rewrites .beads/metadata.json (dolt_mode), re-enables dolt.shared-server, and
removes the proxied-server sidecar; no Dolt data is copied or moved. Stop the
running proxy first with 'bd dolt stop'.`,
Args: cobra.NoArgs,
RunE: migrateFromProxiedRunE("migrate-to-shared-server", "migrate from-proxied-server-to-shared-server", true),
}
func resolveMigrateIdleTimeout(cmd *cobra.Command) (time.Duration, error) {
if !cmd.Flags().Changed("idle-timeout") {
return 0, nil
}
v, _ := cmd.Flags().GetDuration("idle-timeout")
if v < 0 {
return 0, HandleError("--idle-timeout must be 0 (never) or a positive duration, got %s", v)
}
if v == 0 {
return proxy.IdleTimeoutNever, nil
}
return v, nil
}
func migrateModeBeadsDir() (string, error) {
beadsDir := beads.FindBeadsDir()
if beadsDir == "" {
return "", HandleErrorWithHint(activeWorkspaceNotFoundError(), diagHint())
}
return beadsDir, nil
}
func loadMigrateModeConfig(beadsDir string) (*configfile.Config, error) {
cfg, err := configfile.Load(beadsDir)
if err != nil {
return nil, HandleError("failed to load config: %v", err)
}
if cfg == nil {
return nil, HandleError("no beads database found in %s — run 'bd init' first", beadsDir)
}
return cfg, nil
}
// migrateGateDestRoot resolves the physical root the DESTINATION mode of a
// dolt-mode migration will use, so the migration can gate it alongside the
// source's roots. Both proxied-server migration directions keep the data
// directory in place, so this is the shared dolt dir for --shared flows and
// the per-project dolt data dir otherwise — resolved side-effect-free (no
// mkdir) because gate planning must not create the tree it is guarding.
func migrateGateDestRoot(shared bool, beadsDir string) (string, error) {
if shared {
return doltserver.SharedDoltPath()
}
return doltserver.DoltDirPath(beadsDir), nil
}
// acquireMigrateGates takes the workspace gate plus the physical-root gates
// for BOTH the source mode's roots (via ResolvePhysicalRoots inside
// acquireExclusiveWorkspaceGates) and the destination mode's root, all
// EXCLUSIVE in one AcquireAll. It must run BEFORE acquireMigrateLock:
// the normative lock ordering is workspace gate(s) → physical-root gate(s)
// → migrate.lock → embedded .lock → proxy locks → dolt-server.lock.
func acquireMigrateGates(beadsDir string, shared bool, reason string) (func(), error) {
destRoot, err := migrateGateDestRoot(shared, beadsDir)
if err != nil {
return nil, HandleError("failed to resolve migration destination root: %v", err)
}
h, err := acquireExclusiveWorkspaceGates(rootCtx, beadsDir, reason, destRoot)
if err != nil {
return nil, HandleErrorWithHint(
fmt.Sprintf("cannot migrate while other bd activity holds this workspace: %v", err),
"wait for running bd commands to finish, then retry")
}
return func() { _ = h.Release() }, nil
}
// acquireMigrateLock takes the legacy per-workspace migrate.lock. Known
// hazard, deliberately left as-is: this lock file lives INSIDE .beads and is
// removed on release, which is exactly the split-inode pattern the
// workspacegate package documents as unsafe for operations that replace
// directories. The workspace/physical-root gates acquired above (see
// acquireMigrateGates) are the durable fence; replacing migrate.lock itself
// is out of scope here (PR-B2+).
func acquireMigrateLock(beadsDir string) (func(), error) {
lockPath := filepath.Join(beadsDir, migrateLockFileName)
lock, err := util.TryLock(lockPath)
if err != nil {
if lockfile.IsLocked(err) {
return nil, HandleErrorWithHint("another bd migrate is in progress on this workspace", "wait for it to finish, then retry")
}
return nil, HandleError("failed to acquire migration lock: %v", err)
}
return func() {
lock.Unlock()
_ = os.Remove(lockPath)
}, nil
}
func migrateLockErr(what string, err error) error {
if lockfile.IsLocked(err) {
return HandleErrorWithHint(fmt.Sprintf("%s is still running", what), "stop it first: bd dolt stop")
}
return HandleError("failed to acquire %s lock: %v", what, err)
}
func runMigrateToProxiedServer(dryRun bool, idleTimeout time.Duration, shared bool) error {
beadsDir, err := migrateModeBeadsDir()
if err != nil {
return err
}
if !dryRun {
releaseGates, err := acquireMigrateGates(beadsDir, shared, "bd migrate to proxied-server")
if err != nil {
return err
}
defer releaseGates()
releaseMigrateLock, err := acquireMigrateLock(beadsDir)
if err != nil {
return err
}
defer releaseMigrateLock()
}
cfg, err := loadMigrateModeConfig(beadsDir)
if err != nil {
return err
}
if cfg.IsDoltProxiedServerMode() {
fmt.Printf("%s\n", ui.RenderPass("✓ Already in proxied-server mode"))
return nil
}
var rootPath string
if shared {
if !doltserver.IsSharedServerMode() {
return HandleError("repo is not in shared-server mode; this command only migrates shared-server repos")
}
rootPath, err = doltserver.SharedDoltDir()
if err != nil {
return HandleError("failed to resolve shared dolt directory: %v", err)
}
} else {
if !cfg.IsDoltServerMode() {
return HandleError("repo is not in server mode (dolt_mode=%q); this command only migrates server-mode repos", cfg.GetDoltMode())
}
if doltserver.IsSharedServerMode() {
return HandleErrorWithHint("repo is in shared-server mode", "use 'bd migrate from-shared-server-to-proxied-server'")
}
}
serverDir := doltserver.ResolveServerDir(beadsDir)
if state, _ := doltserver.IsRunning(serverDir); state != nil && state.Running {
return HandleErrorWithHint("dolt server is still running", "stop it first: bd dolt stop")
}
if dryRun {
fmt.Println("Dry run mode - no changes will be made")
fmt.Printf("Would set dolt_mode: %s → %s\n", configfile.DoltModeServer, configfile.DoltModeProxiedServer)
if shared {
fmt.Println("Would disable dolt.shared-server")
fmt.Printf("Would root the proxy at %s\n", rootPath)
}
fmt.Printf("Would write %s\n", configfile.ProxiedServerClientInfoFileName)
for _, p := range doltserver.StateFilePaths(serverDir) {
fmt.Printf("Would remove %s\n", p)
}
return nil
}
cfg.DoltMode = configfile.DoltModeProxiedServer
if err := cfg.Save(beadsDir); err != nil {
return HandleError("failed to save metadata.json: %v", err)
}
if shared {
if err := config.SetYamlConfigInDir(beadsDir, "dolt.shared-server", "false"); err != nil {
fmt.Fprintf(os.Stderr, "Warning: could not disable dolt.shared-server: %v\n", err)
}
}
info := &configfile.ProxiedServerClientInfo{RootPath: rootPath, IdleTimeout: idleTimeout}
if err := configfile.SaveProxiedServerClientInfo(beadsDir, info); err != nil {
return HandleError("failed to write %s: %v", configfile.ProxiedServerClientInfoFileName, err)
}
warnMigrateRemovalErrors(doltserver.RemoveStateFiles(serverDir))
dataDir := proxiedServerRoot(beadsDir)
if shared {
dataDir = rootPath
}
commandDidWrite.Store(true)
fmt.Printf("%s\n\n", ui.RenderPass("✓ Switched to proxied-server mode"))
fmt.Printf(" Data directory unchanged: %s\n", dataDir)
fmt.Println(" The proxy starts automatically on the next bd command.")
return nil
}
func runMigrateFromProxiedServer(dryRun bool, shared bool) error {
beadsDir, err := migrateModeBeadsDir()
if err != nil {
return err
}
if !dryRun {
releaseGates, err := acquireMigrateGates(beadsDir, shared, "bd migrate from proxied-server")
if err != nil {
return err
}
defer releaseGates()
releaseMigrateLock, err := acquireMigrateLock(beadsDir)
if err != nil {
return err
}
defer releaseMigrateLock()
}
cfg, err := loadMigrateModeConfig(beadsDir)
if err != nil {
return err
}
if shared {
if doltserver.IsSharedServerMode() {
fmt.Printf("%s\n", ui.RenderPass("✓ Already in shared-server mode"))
return nil
}
} else if cfg.IsDoltServerMode() && !doltserver.IsSharedServerMode() {
fmt.Printf("%s\n", ui.RenderPass("✓ Already in server mode"))
return nil
}
if !cfg.IsDoltProxiedServerMode() {
return HandleError("repo is not in proxied-server mode (dolt_mode=%q); this command only migrates proxied-server repos", cfg.GetDoltMode())
}
// Leaving proxied mode would make dolt_team_server inert and resume
// bd-driven schema migrations on the bts-owned database.
if cfg.DoltTeamServer {
return HandleErrorWithHint("workspace is team-server managed (dolt_team_server in metadata.json); the shared database's schema is owned by beads-team-server",
"if the database is no longer bts-managed, remove dolt_team_server from .beads/metadata.json first")
}
rootDir, err := resolveProxiedServerRootPath(beadsDir)
if err != nil {
return HandleError("%v", err)
}
sharedDolt, sharedErr := doltserver.SharedDoltDir()
if shared {
if sharedErr != nil {
return HandleError("failed to resolve shared dolt directory: %v", sharedErr)
}
if rootDir != sharedDolt {
return HandleErrorWithHint(fmt.Sprintf("proxied-server root %s is not the shared dolt directory", rootDir), "use 'bd migrate from-proxied-server-to-server'")
}
} else if sharedErr == nil && rootDir == sharedDolt {
return HandleErrorWithHint("proxied-server is rooted at the shared dolt directory", "use 'bd migrate from-proxied-server-to-shared-server'")
}
if running, _ := proxy.IsRunning(rootDir); running {
return HandleErrorWithHint("proxied-server is still running", "stop it first: bd dolt stop")
}
logAssets, err := proxiedLogAssets(beadsDir)
if err != nil {
return HandleError("%v", err)
}
if dryRun {
fmt.Println("Dry run mode - no changes will be made")
fmt.Printf("Would set dolt_mode: %s → %s\n", configfile.DoltModeProxiedServer, configfile.DoltModeServer)
if shared {
fmt.Println("Would enable dolt.shared-server")
}
fmt.Printf("Would remove %s\n", configfile.ProxiedServerClientInfoFileName)
for _, p := range proxy.ControlFilePaths(rootDir) {
fmt.Printf("Would remove %s\n", p)
}
for _, p := range logAssets {
fmt.Printf("Would remove %s\n", p)
}
return nil
}
serverStateDir := beadsDir
if shared {
serverStateDir, err = doltserver.SharedServerDir()
if err != nil {
return HandleError("failed to resolve shared server directory: %v", err)
}
}
proxyLock, err := util.TryLock(filepath.Join(rootDir, proxy.LockFileName))
if err != nil {
return migrateLockErr("proxy", err)
}
defer proxyLock.Unlock()
childLock, err := util.TryLock(filepath.Join(rootDir, server.LockFileName))
if err != nil {
return migrateLockErr("proxied dolt sql-server", err)
}
defer childLock.Unlock()
serverLock, err := util.TryLock(doltserver.LockPath(serverStateDir))
if err != nil {
return migrateLockErr("dolt sql-server", err)
}
defer serverLock.Unlock()
if err := doltserver.MarkDoltDirCompatible(rootDir); err != nil {
return HandleError("failed to mark dolt directory compatible: %v", err)
}
cfg.DoltMode = configfile.DoltModeServer
if err := cfg.Save(beadsDir); err != nil {
return HandleError("failed to save metadata.json: %v", err)
}
if shared {
if err := config.SetYamlConfigInDir(beadsDir, "dolt.shared-server", "true"); err != nil {
fmt.Fprintf(os.Stderr, "Warning: could not enable dolt.shared-server: %v\n", err)
}
}
if err := os.Remove(configfile.ProxiedServerClientInfoPath(beadsDir)); err != nil && !os.IsNotExist(err) {
return HandleError("failed to remove %s: %v", configfile.ProxiedServerClientInfoFileName, err)
}
warnMigrateRemovalErrors(proxy.PurgeControlFiles(rootDir))
warnMigrateRemovalErrors(removeMigrateAssets(logAssets))
commandDidWrite.Store(true)
if shared {
fmt.Printf("%s\n\n", ui.RenderPass("✓ Switched to shared-server mode"))
} else {
fmt.Printf("%s\n\n", ui.RenderPass("✓ Switched to server mode"))
}
fmt.Printf(" Data directory unchanged: %s\n", rootDir)
fmt.Println(" The dolt sql-server starts automatically on the next bd command.")
return nil
}
func proxiedLogAssets(beadsDir string) ([]string, error) {
logPath, isCustomLog, err := resolveProxiedServerLogPath(beadsDir)
if err != nil {
return nil, err
}
if isCustomLog {
return nil, nil
}
return []string{logPath}, nil
}
func removeMigrateAssets(paths []string) []error {
var errs []error
for _, p := range paths {
if err := os.Remove(p); err != nil && !os.IsNotExist(err) {
errs = append(errs, err)
}
}
return errs
}
func warnMigrateRemovalErrors(errs []error) {
for _, err := range errs {
fmt.Fprintf(os.Stderr, "Warning: could not remove migration asset: %v\n", err)
}
}