beads/internal/storage/dolt/open.go
Julian Knutsen f3c47390e7
fix(storage): refuse graph_mode workspaces on library opens (#7721)
#7671 fences graph workspaces in cmd/bd: command admission, the store
factories and the other CLI paths refuse one, and #7722 extends that
fence to a .beads holding the graph-preview-format marker whatever its
metadata says. But the public library opens (beads.OpenBestAvailable,
OpenBestAvailableWith, OpenFromConfig, OpenGated) and the Dolt store's
metadata-driven constructors (NewFromConfig, NewFromConfigWithOptions,
NewFromConfigWithCLIOptions) still opened either kind of graph workspace
as an Issue store and ran Issue schema migrations into the graph
database (or, with no metadata beside the marker, created an Issue
database inside the graph workspace).

- Move the typed refusal into internal/configfile
  (UnsupportedWorkspaceFormatError with BeadsDir, GraphMode and
  FormatMarker; CheckWorkspaceFormat; GraphFormatMarkerFile;
  HasGraphFormatMarker) with byte-identical text and hints for both the
  graph_mode and the marker refusal, and move cmd/bd's discovery
  pre-checks there as LoadAndCheckWorkspaceFormat and
  CheckDiscoveredWorkspaceFormat (LoadForDiscovery, which never migrates
  a legacy config.json; unreadable metadata beside the marker is refused
  as the marker's format). cmd/bd's checkWorkspaceFormat,
  loadAndCheckWorkspaceFormat, hasGraphFormatMarker and
  checkDiscoveredWorkspaceFormat become thin delegates, so the CLI keeps
  its behaviour (exit 16 with the same hints) and every caller of them is
  unchanged; bootstrap's ancestor walk builds the configfile type.
- Every public metadata-driven open runs CheckDiscoveredWorkspaceFormat
  as its first statement: before OpenGated takes its gates and before
  OpenBestAvailableWith's Load can migrate config.json. A refused open
  leaves the workspace byte-identical. The type is exported as
  beads.UnsupportedWorkspaceFormatError.
- The Dolt constructors run CheckDiscoveredWorkspaceFormat before
  configfile.Load, so a graph workspace marked only by a legacy
  config.json, or by the marker beside stripped, missing or unparseable
  metadata, is refused before Load can migrate it into metadata.json
  (and drop graph_* keys this build does not know). requireDoltBackend
  re-checks the loaded config as a backstop, so they also refuse before
  resolving a server or writing a port file or Dolt root.

Tests: configfile unit tests for both refusals (exact marker text and
hint, graph_mode winning over the marker, unparseable metadata with and
without the marker); the public opens and the three Dolt constructors
against graph_mode workspaces and against marker-only, stripped
(metadata.json and legacy config.json), server, explicit dependency and
unparseable-metadata marker workspaces, each refused with the typed
error and the workspace left unchanged.

Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 09:29:54 +00:00

425 lines
18 KiB
Go

package dolt
import (
"context"
"fmt"
"os"
"strconv"
"strings"
"github.com/steveyegge/beads/internal/config"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/doltserver"
)
// ServerMode is re-exported from doltserver for convenience.
type ServerMode = doltserver.ServerMode
// Re-export ServerMode constants for callers that import storage/dolt.
const (
ServerModeOwned = doltserver.ServerModeOwned
ServerModeExternal = doltserver.ServerModeExternal
ServerModeEmbedded = doltserver.ServerModeEmbedded
)
// ApplyCLIAutoStart sets the standalone auto-start policy used by the
// normal CLI path. Honors the actual server mode resolved from
// metadata.json + env: when External (e.g. metadata.json has explicit
// dolt_server_port), suppresses fallback auto-spawn — the user has
// configured an external server; if it's transiently unreachable, bd
// errors out rather than silently spawning a different server from
// .beads/dolt/ (the shadow database bug).
//
// Cold standalone setups remain unaffected: bd init writes the port and
// starts the server in one shot, so subsequent commands find a running
// server and don't need fallback auto-start. If the user explicitly
// stops the server, External mode's "you manage the lifecycle" semantics
// asks the user to run `bd dolt start`.
func ApplyCLIAutoStart(beadsDir string, cfg *Config) {
if cfg.DisableAutoStart {
cfg.AutoStart = false
return
}
autoStartCfg := config.GetString("dolt.auto-start")
if autoStartCfg == "" {
autoStartCfg = config.GetStringFromDir(beadsDir, "dolt.auto-start")
}
mode := doltserver.ResolveServerMode(beadsDir)
cfg.AutoStart = resolveAutoStart(true, autoStartCfg, mode)
}
// ApplyResolvedServerPort fills cfg's server port from doltserver's
// precedence chain (env > port file > dolt config.yaml > beads config.yaml >
// metadata.json), and — the part that is easy to drop — records WHICH of those
// produced it.
//
// The source is not decoration. applyConfigDefaults infers "the caller
// explicitly asserted this port" from "ServerPort nonzero, ServerPortSource
// unset" (be-wf9a.1). A site that copies doltserver.DefaultConfig(dir).Port on
// its own therefore launders bd's own bookkeeping — the gitignored port file,
// or the shared-server default 3308 — into a deliberate user assertion, and
// that mislabel has two visible consequences:
//
// - the legacy BEADS_DOLT_PORT override becomes a silent no-op, because an
// authoritative source outranks the env read (be-9tju); and
// - auto-start's benign "configured port is stale, here is the new one"
// retarget turns into a hard failure telling the user to fix a port they
// never configured (GH#4052).
//
// So every hand-built dolt.Config that resolves its port this way goes through
// here rather than reaching for .Port. Callers that want to resolve only when
// unset keep their own `if cfg.ServerPort == 0` guard; this function is
// unconditional.
func ApplyResolvedServerPort(beadsDir string, cfg *Config) {
resolved := doltserver.DefaultConfig(beadsDir)
cfg.ServerPort = resolved.Port
cfg.ServerPortSource = resolved.PortSource
cfg.ServerPortSharedServer = resolved.PortSharedServer
}
// requireDoltBackend keeps metadata-driven callers from bypassing the storage
// factory and interpreting another backend's workspace as Dolt. Removed backend
// identifiers deliberately remain recognizable in metadata so this check can fail
// closed instead of opening a new, empty Dolt database. beadsDir is the workspace
// the config came from, so the refusal can offer the metadata heal when that
// workspace already holds a Dolt database.
//
// It first refuses a workspace whose metadata or graph format marker declares a
// data model this build cannot open (configfile.CheckWorkspaceFormat): such a
// workspace keeps another model's database behind an ordinary Dolt backend,
// and opening it here would run Issue migrations into that database.
func requireDoltBackend(fileCfg *configfile.Config, beadsDir string) error {
if err := configfile.CheckWorkspaceFormat(fileCfg, beadsDir); err != nil {
return err
}
switch fileCfg.Backend {
case configfile.BackendPostgres, configfile.BackendMySQL, configfile.BackendSQLite:
return fmt.Errorf("configured storage backend %q is no longer supported and cannot be opened as Dolt: %s", fileCfg.Backend, configfile.RemovedBackendDetailAt(fileCfg.Backend, beadsDir, fileCfg))
}
if !configfile.IsSupportedBackend(fileCfg.Backend) {
return fmt.Errorf("configured storage backend %q in metadata.json is not recognized and cannot be opened as Dolt; %s", fileCfg.Backend, configfile.BackendNotOpenedGuarantee)
}
backend := fileCfg.GetBackend()
if backend != configfile.BackendDolt {
return fmt.Errorf("configured storage backend %q cannot be opened as Dolt", backend)
}
return nil
}
// NewFromConfig creates a DoltStore based on the metadata.json configuration.
// beadsDir is the path to the .beads directory.
func NewFromConfig(ctx context.Context, beadsDir string) (*DoltStore, error) {
return NewFromConfigWithOptions(ctx, beadsDir, nil)
}
// NewFromConfigWithCLIOptions creates a DoltStore using the standalone CLI
// auto-start policy from cmd/bd/main.go. This is for CLI helper paths like
// `bd doctor` that should behave the same way as normal top-level CLI commands
// while still honoring externally managed server mode.
func NewFromConfigWithCLIOptions(ctx context.Context, beadsDir string, cfg *Config) (*DoltStore, error) {
// Refuse a graph workspace before Load can migrate a legacy config.json
// into metadata.json; requireDoltBackend re-checks the loaded config.
if err := configfile.CheckDiscoveredWorkspaceFormat(beadsDir); err != nil {
return nil, err
}
fileCfg, err := configfile.Load(beadsDir)
if err != nil {
return nil, fmt.Errorf("loading config: %w", err)
}
if fileCfg == nil {
fileCfg = configfile.DefaultConfig()
}
if err := requireDoltBackend(fileCfg, beadsDir); err != nil {
return nil, err
}
// Apply central server config as defaults for any server fields not
// set in the per-project metadata.json. This eliminates the need to
// duplicate host/port/user across 30+ project configs.
applyCentralConfigDefaults(fileCfg)
if cfg == nil {
cfg = &Config{}
}
if err := applyResolvedConfig(ctx, beadsDir, fileCfg, cfg); err != nil {
return nil, err
}
ApplyCLIAutoStart(beadsDir, cfg)
return New(ctx, cfg)
}
// NewFromConfigWithOptions creates a DoltStore with options from metadata.json.
// Options in cfg override those from the config file. Pass nil for default options.
func NewFromConfigWithOptions(ctx context.Context, beadsDir string, cfg *Config) (*DoltStore, error) {
// Refuse a graph workspace before Load can migrate a legacy config.json
// into metadata.json; requireDoltBackend re-checks the loaded config.
if err := configfile.CheckDiscoveredWorkspaceFormat(beadsDir); err != nil {
return nil, err
}
fileCfg, err := configfile.Load(beadsDir)
if err != nil {
return nil, fmt.Errorf("loading config: %w", err)
}
if fileCfg == nil {
fileCfg = configfile.DefaultConfig()
}
if err := requireDoltBackend(fileCfg, beadsDir); err != nil {
return nil, err
}
// Apply central server config as defaults for any server fields not
// set in the per-project metadata.json.
applyCentralConfigDefaults(fileCfg)
// Build config from metadata.json, allowing overrides from caller
if cfg == nil {
cfg = &Config{}
}
if err := applyResolvedConfig(ctx, beadsDir, fileCfg, cfg); err != nil {
return nil, err
}
// Enable auto-start for standalone users (similar to main.go's auto-start
// handling), with additional support for BEADS_TEST_MODE and a config.yaml
// fallback for library consumers that never call config.Initialize().
// Disabled under orchestrator (which manages its own server), by explicit config,
// or in test mode (tests manage their own server lifecycle via testdoltserver).
// Note: cfg.ReadOnly refers to the store's read-only mode, not the server —
// the server must be running regardless of whether the store is read-only.
//
// Prefer the global viper config (populated when config.Initialize() has been
// called, i.e. all CLI paths). Fall back to a direct read of the project
// config.yaml for library consumers that never call config.Initialize().
autoStartCfg := config.GetString("dolt.auto-start")
if autoStartCfg == "" {
autoStartCfg = config.GetStringFromDir(beadsDir, "dolt.auto-start")
}
// When the server is externally managed (explicit port in metadata.json,
// shared server mode, etc.), suppress auto-start. This prevents bd from
// launching a different server when the user's configured server is
// temporarily unreachable — the root cause of the shadow database bug.
mode := doltserver.ResolveServerMode(beadsDir)
if cfg.DisableAutoStart {
cfg.AutoStart = false
} else {
cfg.AutoStart = resolveAutoStart(cfg.AutoStart, autoStartCfg, mode)
}
return New(ctx, cfg)
}
// resolveAutoStart computes the effective AutoStart value, respecting a
// caller-provided value (current) while applying system-level overrides.
//
// Priority (highest to lowest):
// 1. BEADS_TEST_MODE=1 → always false (tests own the server lifecycle)
// 2. BEADS_DOLT_AUTO_START=0 → always false (explicit env opt-out)
// 3. mode == ServerModeExternal → always false (server is externally managed;
// auto-starting a different server would create shadow databases)
// 4. doltAutoStartCfg == "false"/"0"/"off" → false (config.yaml explicit opt-out;
// user intent to disable auto-start must be respected even when callers
// like ApplyCLIAutoStart or bootstrap pass current=true — GH#autostart-bug)
// 5. current == true → true (caller option wins over default)
// 6. default → true (standalone user; safe default)
//
// doltAutoStartCfg is the raw value of the "dolt.auto-start" key from config.yaml
// (pass config.GetString("dolt.auto-start") at the call site).
//
// Note: because AutoStart is a plain bool, a zero value (false) cannot be
// distinguished from an explicit "opt-out" by the caller. Callers that need
// to suppress auto-start should use one of the environment-variable or
// config-file overrides above.
func resolveAutoStart(current bool, doltAutoStartCfg string, mode ServerMode) bool {
if os.Getenv("BEADS_TEST_MODE") == "1" {
return false
}
if os.Getenv("BEADS_DOLT_AUTO_START") == "0" {
return false
}
// When the server is externally managed, never auto-start.
// The user has configured a specific server — if it's down, error out
// rather than silently starting a different server from .beads/dolt/.
if mode == ServerModeExternal {
return false
}
// Config.yaml explicit opt-out takes precedence over caller-provided
// current=true. Without this, ApplyCLIAutoStart (which passes current=true)
// and bootstrap paths (which hardcode AutoStart=true) would ignore the
// user's dolt.auto-start: false setting, spawning rogue dolt servers that
// overwrite port files and cause DB lock conflicts.
if strings.EqualFold(doltAutoStartCfg, "false") || doltAutoStartCfg == "0" || strings.EqualFold(doltAutoStartCfg, "off") {
return false
}
// Caller option wins over default.
if current {
return true
}
// Default: auto-start for standalone users.
return true
}
// GetBackendFromConfig returns the backend type from metadata.json.
// Returns "dolt" if no config exists or backend is not specified.
func GetBackendFromConfig(beadsDir string) string {
cfg, err := configfile.Load(beadsDir)
if err != nil || cfg == nil {
return configfile.BackendDolt
}
return cfg.GetBackend()
}
// applyResolvedConfig merges metadata.json-derived defaults into a store config.
// Server connection fields are always populated because the storage layer is
// server-backed even when older metadata.json files omit dolt_mode. It returns an
// error only when a configured server credential command fails (fail-closed).
func applyResolvedConfig(ctx context.Context, beadsDir string, fileCfg *configfile.Config, cfg *Config) error {
cfg.Path = fileCfg.DatabasePath(beadsDir)
if cfg.BeadsDir == "" {
cfg.BeadsDir = beadsDir
}
// GH#2438: Warn if data-dir is set in server mode — it has no effect on
// which database the server uses and can cause silent DB context switches.
if fileCfg.DoltDataDir != "" && fileCfg.IsDoltServerMode() {
fmt.Fprintf(os.Stderr, "Warning: dolt_data_dir is set (%s) but Dolt is in server mode.\n", fileCfg.DoltDataDir)
fmt.Fprintf(os.Stderr, "In server mode, data-dir does not control which database is used.\n")
fmt.Fprintf(os.Stderr, "This may cause commands to operate on the wrong database.\n")
fmt.Fprintf(os.Stderr, "Fix: bd dolt set data-dir '' (clear the data-dir setting)\n\n")
}
// Always apply database name from metadata.json (prefix-based naming, bd-u8rda).
if cfg.Database == "" {
cfg.Database = fileCfg.GetDoltDatabase()
}
if cfg.ServerHost == "" {
cfg.ServerHost = fileCfg.GetDoltServerHost()
}
if cfg.ServerPort == 0 {
// fileCfg.GetDoltServerPort() falls back to 3307, which is wrong for
// standalone repos; ApplyResolvedServerPort walks doltserver's real
// precedence chain and carries the source along with the port.
ApplyResolvedServerPort(beadsDir, cfg)
}
// Resolve the server-mode credential (the connection username). In server mode a
// configured credential command takes precedence over the static user; it fails
// closed (see ApplyGatewayCredential). The command runs only in server mode — an
// embedded store never presents a username, so a command exported in the environment
// must not run (or fail) an embedded open. The server-mode test mirrors main.go's
// (metadata dolt_mode=server, or shared-server via env/config.yaml) so both the CLI
// and library/doctor open paths agree on whether to run the command.
applied := false
if fileCfg.IsDoltServerMode() || doltserver.IsSharedServerMode() {
got, err := ApplyGatewayCredential(ctx, fileCfg, cfg)
if err != nil {
return fmt.Errorf("resolving dolt credential command: %w", err)
}
applied = got
}
if !applied && cfg.ServerUser == "" {
cfg.ServerUser = fileCfg.GetDoltServerUser()
}
// Populate password and TLS the same way the CLI CRUD path does. Without
// this, callers that rely on NewFromConfigWithOptions (e.g. doctor's
// SharedStore) fail to reach externally-hosted Dolt servers that keep
// credentials in ~/.config/beads/credentials, while bd create/list/close
// succeed (bd-h5k7). GetDoltServerPasswordForPort checks BEADS_DOLT_PASSWORD
// env first, then credentials file keyed by [host:resolved-port].
if cfg.ServerPassword == "" {
cfg.ServerPassword = fileCfg.GetDoltServerPasswordForPort(cfg.ServerPort)
}
if !cfg.ServerTLS {
cfg.ServerTLS = fileCfg.GetDoltServerTLS()
}
applyPoolKnobs(cfg)
return nil
}
// applyPoolKnobs fills the shared pool's operator knobs when the caller left
// them unset, each on the ladder caller override > env var > config.yaml >
// built-in default:
//
// - pool size: BEADS_DOLT_MAX_CONNS / dolt.max-conns (default 10, see
// applyPoolLimits). Useful for shared-server setups with many worktrees
// (GH#3140).
// - per-I/O deadlines: BEADS_DOLT_POOL_READ_TIMEOUT / dolt.pool-read-timeout
// and BEADS_DOLT_POOL_WRITE_TIMEOUT / dolt.pool-write-timeout (default 10s
// each, see buildServerDSN). The default fast-fail is right for healthy
// local servers; overloaded shared-server deployments raise it so ordinary
// queries stop dying with "i/o timeout" under load (bd-vz0y9).
//
// It is idempotent, and it runs from New (applyConfigDefaults) so every
// DoltStore open honors the knobs. The CLI's main store open hand-builds its
// Config and never passes through applyResolvedConfig, so a knob read only
// there reaches the CLI's NewFromConfig* opens (routed stores, bd doctor) but
// not ordinary bd commands: that is how the deadlines shipped in #5089 stayed
// inert in server mode (gastownhall/beads#6144), and the pool size the same
// way. A new pool knob belongs here, not in applyResolvedConfig.
func applyPoolKnobs(cfg *Config) {
if cfg.MaxOpenConns == 0 {
if n, err := strconv.Atoi(os.Getenv("BEADS_DOLT_MAX_CONNS")); err == nil && n > 0 {
cfg.MaxOpenConns = n
}
}
if cfg.MaxOpenConns == 0 {
if n, err := strconv.Atoi(poolKnobFromConfig(cfg, "dolt.max-conns")); err == nil && n > 0 {
cfg.MaxOpenConns = n
}
}
if cfg.PoolReadTimeout == 0 {
cfg.PoolReadTimeout = timeoutFromEnv("BEADS_DOLT_POOL_READ_TIMEOUT", 0)
}
if cfg.PoolReadTimeout == 0 {
cfg.PoolReadTimeout = parseTimeout(poolKnobFromConfig(cfg, "dolt.pool-read-timeout"), 0)
}
if cfg.PoolWriteTimeout == 0 {
cfg.PoolWriteTimeout = timeoutFromEnv("BEADS_DOLT_POOL_WRITE_TIMEOUT", 0)
}
if cfg.PoolWriteTimeout == 0 {
cfg.PoolWriteTimeout = parseTimeout(poolKnobFromConfig(cfg, "dolt.pool-write-timeout"), 0)
}
}
// poolKnobFromConfig reads a pool-knob key from the initialized config and,
// like the auto-start ladder above, falls back to the .beads directory's own
// config.yaml. It needs both reads: config.GetString reads a package-global
// viper populated only by cmd/bd's config.Initialize(), so for a library
// consumer it always returns "" and the project's configured values would be
// silently ignored. The fallback follows GetStringFromDir's ladder, so when
// cfg.BeadsDir is not the process CWD a user-level ~/.config/bd/config.yaml
// value outranks that project's file — same behavior as dolt.auto-start.
func poolKnobFromConfig(cfg *Config, key string) string {
if v := config.GetString(key); v != "" {
return v
}
if cfg.BeadsDir == "" {
return ""
}
return config.GetStringFromDir(cfg.BeadsDir, key)
}
// applyCentralConfigDefaults loads the central server config from
// ~/.config/beads/server.json (or BEADS_CENTRAL_CONFIG env var) and
// applies its server fields as defaults to the per-project config.
// A missing central config file is silently ignored.
func applyCentralConfigDefaults(fileCfg *configfile.Config) {
centralPath := os.Getenv("BEADS_CENTRAL_CONFIG")
if centralPath == "" {
centralPath = configfile.DefaultCentralConfigPath()
}
if centralPath == "" {
return
}
centralCfg, err := configfile.LoadCentralConfig(centralPath)
if err != nil {
// Log but don't fail — a broken central config shouldn't block operations.
fmt.Fprintf(os.Stderr, "Warning: failed to load central config %s: %v\n", centralPath, err)
return
}
configfile.ApplyCentralDefaults(fileCfg, centralCfg)
}