beads/cmd/bd/store_factory.go
Julian Knutsen 1fcd715e75
fix(cli): refuse graph_mode workspaces and keep their metadata marker on save (#7671)
* fix(configfile): carry graph_mode and graph_* binding through Save

Graph-preview builds of bd persist a per-workspace format in
metadata.json: "graph_mode": "link" plus graph_scope_url,
graph_authority_id, graph_workspace, graph_schema_version and
graph_ready, beside an ordinary Dolt backend. configfile.Config had no
such fields, so every Save, and Load's config.json -> metadata.json
migration, rewrote the file without them, turning a graph workspace's
metadata into an ordinary Issue workspace's.

Config now carries the marker and the binding as omitempty fields, so
a Save round-trips them unchanged and metadata.json written for an
ordinary workspace is byte-for-byte what it was. GetGraphMode resolves
the marker: absent or "dependency" is the historical default, "link"
is recognized, and any other value is an error rather than a fallback
to the Issue store. Nothing in this build opens a non-default mode;
the CLI refusal follows in the next commit.

Ported from the donnabox fork commit deea8d57df (graph_mode.go,
graph_mode_test.go and the configfile.go fields), without any graph
code. The binding-survives-Save and omitted-from-ordinary-metadata
tests are new.

Co-authored-by: Donna Box <dbox@boxenllc.com>
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cli): refuse graph_mode workspaces before opening them

A workspace whose metadata.json declares a graph_mode other than
"dependency" holds a graph database behind an ordinary Dolt backend,
and nothing else in its metadata says so. bd admitted it like any
current workspace: `bd list` ran the Issue schema initialization
against the graph database (beads_graph_<id>), `bd init --force` wrote
a .gitignore and a .beads/dolt root into it, `bd dolt start` started a
server for it, and `bd doctor` reported on it as an Issue store.

guardLegacyUpgradeWorkspace, the admission check every store-opening
command, store-less command with workspace effects, init, doctor and
bootstrap already run before touching a workspace, now refuses such a
workspace first, before the legacy classifications that would admit
it or report it as a legacy store to migrate. An unknown graph_mode
value is refused the same way. The refusal is a typed error naming the
workspace and the format, rendered with a hint (as JSON under --json)
and exit code 16, ExitUnsupportedWorkspaceFormat, so a wrapper can
tell "needs a different bd build" from a generic failure.
`bd version`, `bd --version`, `bd help` and completion stay exempt, as
they are for legacy workspaces. validateConfiguredBackend refuses the
same metadata, which covers the store-open paths that skip command
admission (routed writes, cross-repository reads, create --repo).

Tests: guard and store-factory unit tests; a subprocess e2e test that
show/list/create/context/where/dolt start/doctor/bootstrap/init are
refused with exit 16 and leave .beads byte-identical (metadata.json and
legacy config.json, link and unknown markers, embedded and server
layouts), and that store-less commands still run; an embedded test
that an ordinary workspace, with the marker absent or "dependency",
still creates and lists issues and keeps the marker through a
metadata.json rewrite.

The guard hunk and graph_mode_admission_e2e_test.go are ported from
the donnabox fork commit deea8d57df; the exit code, rendering, store
factory backstop and remaining tests are new.

Co-authored-by: Donna Box <dbox@boxenllc.com>
Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cli): refuse graph workspaces reached outside command admission

Command admission refuses a graph_mode workspace a command runs in, but
bd also reaches workspaces before or without admission: a create --repo
target, the path --db or BEADS_DB names, and the stores the factories
open for routed and cross-repository work. On those paths a workspace
marked only by a legacy config.json was changed before anything refused
it: configfile.Load migrates config.json into metadata.json before the
format check sees the loaded metadata, and create --repo initialized
such a target as a new workspace. Where the refusal did come, create
reported it with exit 1 rather than 16.

- checkDiscoveredWorkspaceFormat checks a workspace's metadata as
  configfile.LoadForDiscovery reads it, which never rewrites it.
  ensureBeadsDirForPath, the dry-run target open and the four store
  factories (cgo and nocgo builds) run it before their own load.
- resolveBeadsDirForDBPath and findDatabaseInBeadsDir locate a
  workspace with LoadForDiscovery, so resolving --db, BEADS_DB or a
  create target no longer migrates it.
- bd create renders a refused target as the workspace refusal, exit 16
  with its hint, on the initializing and both dry-run paths. A dry run
  without --parent opens no target, so it now checks a local one before
  rendering the preview (checkRepoTargetFormat).
- bd bootstrap, when discovery finds no workspace, reads each .beads
  from the working directory up to the repository root without
  migrating it and refuses one in a format this build cannot open, the
  way commands without a workspace already do. A .beads marked only by
  config.json is passed over by discovery, so bootstrap reported "no
  workspace" there instead of exit 16. The walk now returns its
  candidates (undiscoveredWorkspaceCandidates) to both checks: handing
  it the check as a func(string) error made gosec's G702 taint analysis
  connect BEADS_DIR to every func(string) error that runs git.
- backendNameForErrorFraming reads with LoadForDiscovery, and
  loadDoltBackendConfig, bd dolt status and bd dolt show refuse a graph
  workspace before their migrating configfile.Load. Admission refuses
  first on every path that reaches them today; this keeps a new caller
  from migrating a legacy config.json before the refusal.
- The CHANGELOG entry and the configfile comment now claim what holds:
  a routed read fails without touching a graph workspace but not with
  code 16, and a Save keeps only the known graph_* fields, with the
  tags graph-preview builds declare. workspace_gate.go no longer lists
  the pre-gate discovery reads as migrating.
- New e2e rows run create --repo from a workspace and from outside one,
  create --dry-run with and without --parent, --db, BEADS_DB, init
  --proxied-server, bootstrap in a workspace nested in a graph one, and
  the hook shim's bd hooks run pre-commit against every fixture; each
  refusal must exit 16 and name the format and the hint. bootstrap in a
  workspace marked only by config.json now refuses like the other
  commands. The store factory test also runs on a config.json graph
  workspace, TestWorkspaceMetadataReadersLeaveGraphWorkspaceUnchanged
  covers the framing, bd dolt loader and diagnostic reads, and
  TestNocgoStoreFactoriesRefuseGraphWorkspace covers the CGO_ENABLED=0
  factories; bazel.yml's pure lane runs it by name (PURE_CMD_BD_TESTS,
  pinned in scripts/ci_workflow_test.go).

Measured with gocyclo/gocognit: every function this changes stays below
the PR's worst changed function (guardLegacyUpgradeWorkspaceForInit
29/28, resolveBeadsDirForDBPath 26/36); showDoltConfig is 19/31 (was
18/30) and guardUndiscoveredLegacyWorkspace 3/3 (was 9/12).

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Donna Box <dbox@boxenllc.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
2026-10-20 00:05:39 +00:00

344 lines
15 KiB
Go

//go:build cgo
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/doltserver"
"github.com/steveyegge/beads/internal/lockfile"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/backends"
"github.com/steveyegge/beads/internal/storage/dbproxy/util"
"github.com/steveyegge/beads/internal/storage/dolt"
"github.com/steveyegge/beads/internal/storage/embeddeddolt"
)
func usesSQLServer() bool {
if shouldUseGlobals() {
if serverMode || proxiedServerMode {
return true
}
} else if cmdCtx != nil && (cmdCtx.ServerMode || cmdCtx.ProxiedServerMode) {
return true
}
if doltserver.IsSharedServerMode() {
return true
}
return false // default: embedded
}
// isEmbeddedMode reports whether the command is using embedded Dolt storage.
func isEmbeddedMode() bool {
return !usesSQLServer()
}
func usesProxiedServer() bool {
if shouldUseGlobals() {
return proxiedServerMode
}
return cmdCtx != nil && cmdCtx.ProxiedServerMode
}
// newDoltStore creates a storage backend from an explicit config.
// Used by bd init and PersistentPreRun.
//
// Events-journal activation is applied HERE rather than by the caller: see the
// note at the top of events_journal.go for why every store construction has to
// go through an activating factory.
// newRegisteredBackendStore opens a store from the pluggable backend registry,
// so the registry arm of the root pre-run's open is not a second, unactivated
// construction path. A read-only open still goes through activation: a
// registered backend decides for itself what read-only means, and refusing to
// offer it the setting would be this factory guessing on its behalf.
func newRegisteredBackendStore(ctx context.Context, name, beadsDir string, readOnly bool) (s storage.DoltStorage, err error) {
defer func() { s, err = activateVersionedHistoryStore(ctx, s, err) }()
defer func() { s, err = activateEventsJournalStore(beadsDir, s, err) }()
backend, ok := backends.Lookup(name)
if !ok {
return nil, fmt.Errorf("storage backend %q is not registered", name)
}
if readOnly {
return backend.OpenReadOnly(ctx, beadsDir)
}
return backend.Open(ctx, beadsDir)
}
func newDoltStore(ctx context.Context, cfg *dolt.Config) (s storage.DoltStorage, err error) {
defer func() { s, err = activateVersionedHistoryStore(ctx, s, err) }()
defer func() { s, err = activateEventsJournalStore(cfg.BeadsDir, s, err) }()
if cfg.ProxiedServer {
return nil, errProxiedStoreUnrouted()
}
if cfg.ServerMode {
return dolt.New(ctx, cfg)
}
if cfg.Preview {
// Preview commands are stricter than ordinary read commands: they
// must not run schema initialization or permit incidental writes
// before their RunE honors --dry-run/--inspect.
return embeddeddolt.OpenForPreviewCommand(ctx, cfg.BeadsDir, cfg.Database, "main")
}
if cfg.ReadOnly {
if cfg.DisableAutoStart {
// Strict --readonly (cfg.DisableAutoStart is the strict-only
// signal threaded from policy.disableAutoStart): the command
// must not write anything, not even incidentally (schema
// init, migrations, the post-command autocommit net). Use the
// genuinely write-refusing open — same one used for cross-repo
// hydration of foreign projects (GH#3231, bd-6dnrw.32) — instead
// of OpenForReadOnlyCommand, which is "otherwise a normal
// writable store".
return embeddeddolt.OpenReadOnly(ctx, cfg.BeadsDir, cfg.Database, "main")
}
// Ordinary classified-read commands (bd show, bd list, ...) must
// not be bricked by the #4259 remote-migrate gate (bd-578h9.5);
// server mode's ReadOnly opens already skip migration entirely.
return embeddeddolt.OpenForReadOnlyCommand(ctx, cfg.BeadsDir, cfg.Database, "main")
}
if cfg.LenientOpen {
// Working-set-reconcile commands (bd dolt commit, bd vc commit) must
// not be bricked by a pending-migration dirty-table refusal: that
// refusal's documented recovery is exactly the commit these commands
// run, so failing the open here would deadlock (#4566). The same goes
// for the migration-consent refusal that precedes it. The server
// arm above honors the same cfg.LenientOpen inside dolt.New; this
// branch is the embedded half of one policy, not the whole of it.
return embeddeddolt.OpenForWorkingSetReconcile(ctx, cfg.BeadsDir, cfg.Database, "main")
}
if cfg.RemoteSyncOpen {
// `bd dolt pull` must not be bricked by the #6575 data-behind gate
// refusal, because that refusal's entire remedy IS this pull: the gate
// stops a clone that is behind the remote from migrating and tells the
// operator to pull first, and an embedded clone has no external dolt
// binary to do it with. Same deadlock as #4566, and this open tolerates
// only that one gate reason (see openRemoteSync). The server arm above
// honors the same cfg.RemoteSyncOpen inside dolt.New.
return embeddeddolt.OpenForRemoteSync(ctx, cfg.BeadsDir, cfg.Database, "main")
}
return embeddeddolt.Open(ctx, cfg.BeadsDir, cfg.Database, "main")
}
// acquireEmbeddedLock acquires an exclusive flock on the embeddeddolt data
// directory derived from beadsDir. The caller must defer lock.Unlock().
// Returns a no-op lock when serverMode is true (the server handles its own
// concurrency).
func acquireEmbeddedLock(beadsDir string, serverMode bool) (util.Unlocker, error) {
if serverMode {
return util.NoopLock{}, nil
}
dataDir := filepath.Join(beadsDir, "embeddeddolt")
lock, err := util.TryLock(filepath.Join(dataDir, ".lock"))
if err != nil {
if lockfile.IsLocked(err) {
return nil, fmt.Errorf("embeddeddolt: another process holds the exclusive lock on %s; "+
"the embedded backend supports only one writer at a time — "+
"use the dolt server backend for concurrent access", dataDir)
}
return nil, fmt.Errorf("embeddeddolt: acquiring lock: %w", err)
}
return lock, nil
}
// newDoltStoreFromConfig creates a storage backend from the beads directory's
// persisted metadata.json configuration. Uses embedded Dolt by default;
// connects to dolt sql-server when dolt_mode is "server".
//
// For embedded mode, legacy hyphenated database names (pre-GH#2142) are
// auto-sanitized to underscores and the fix is persisted to metadata.json.
//
// This is the factory the CROSS-WORKSPACE opens use — routed creates,
// remote-cache hydration — so activation is resolved from beadsDir's own
// config, not the launching workspace's. effectiveServerMode below is what
// keeps that true for the dolt.shared-server layer; the one deliberate
// exception is BEADS_DOLT_SHARED_SERVER, which is machine-global in every
// resolver in the tree (see sharedServerModeForWorkspace).
func newDoltStoreFromConfig(ctx context.Context, beadsDir string) (storage.DoltStorage, error) {
return openDoltStoreFromConfig(ctx, beadsDir, false)
}
// newExistingDoltStoreFromConfig is newDoltStoreFromConfig for a write routed
// to another workspace (a prefix route or an auto-route, see
// openRoutedTargetStore): it opens that workspace's database writable, but
// refuses an embedded workspace that holds no database yet instead of creating
// one there. The routed open runs before the lookup it serves, so a created
// database would outlive an id that misses, and `bd bootstrap` would then take
// the empty database for the real one and skip the clone from sync.remote. The
// probe is embeddeddolt.HasRepository, the adapter's own marker check. A
// server-mode open already refuses a missing database: it does not set
// CreateIfMissing.
func newExistingDoltStoreFromConfig(ctx context.Context, beadsDir string) (storage.DoltStorage, error) {
return openDoltStoreFromConfig(ctx, beadsDir, true)
}
// openDoltStoreFromConfig is the body of newDoltStoreFromConfig and
// newExistingDoltStoreFromConfig; requireExisting selects the latter.
func openDoltStoreFromConfig(ctx context.Context, beadsDir string, requireExisting bool) (s storage.DoltStorage, err error) {
defer func() { s, err = activateVersionedHistoryStore(ctx, s, err) }()
defer func() { s, err = activateEventsJournalStore(beadsDir, s, err) }()
if err := checkDiscoveredWorkspaceFormat(beadsDir); err != nil {
return nil, err
}
cfg, err := configfile.Load(beadsDir)
if err != nil {
// A present-but-unloadable metadata.json must not degrade to the
// embedded default: on server-mode deployments the embedded
// directory is an empty relic, and opening it silently turns every
// query into an empty result set with exit 0 (false-empty). Absent
// metadata.json (cfg == nil, err == nil) keeps the embedded default.
return nil, fmt.Errorf("load %s: %w (refusing to fall back to the embedded store)", configfile.ConfigPath(beadsDir), err)
}
if err := validateConfiguredBackend(cfg, beadsDir); err != nil {
return nil, err
}
cfg = normalizeLoadedConfig(cfg)
if backend, ok := backends.Lookup(cfg.GetBackend()); ok {
return backend.Open(ctx, beadsDir)
}
if cfg != nil && cfg.IsDoltProxiedServerMode() {
return nil, errProxiedStoreUnrouted()
}
if effectiveServerMode(beadsDir, cfg) {
return dolt.NewFromConfig(ctx, beadsDir)
}
// Refuse before the sanitize migration below, which renames the legacy
// directory and rewrites metadata.json: a refused open writes nothing.
if requireExisting && !embeddeddolt.HasRepository(beadsDir) {
return nil, fmt.Errorf("embeddeddolt: no embedded database at %s; a routed write does not create one (run 'bd bootstrap' in that repository first)",
filepath.Join(beadsDir, "embeddeddolt"))
}
database := configfile.DefaultDoltDatabase
if cfg != nil {
database = cfg.GetDoltDatabase()
}
if sanitized := sanitizeDBName(database); sanitized != database {
if err := migrateHyphenatedDB(beadsDir, cfg, database, sanitized); err != nil {
return nil, fmt.Errorf("auto-sanitize database name %q → %q: %w", database, sanitized, err)
}
database = sanitized
}
return embeddeddolt.Open(ctx, beadsDir, database, "main")
}
// migrateHyphenatedDB renames a legacy hyphenated database directory and
// persists the sanitized name to metadata.json so subsequent opens use it.
// This handles projects initialized before GH#2142 that upgrade to
// embedded-mode-default builds (GH#3231).
func migrateHyphenatedDB(beadsDir string, cfg *configfile.Config, oldName, newName string) error {
dataDir := filepath.Join(beadsDir, "embeddeddolt")
oldDir := filepath.Join(dataDir, oldName)
newDir := filepath.Join(dataDir, newName)
oldExists := false
if info, err := os.Stat(oldDir); err == nil && info.IsDir() {
oldExists = true
}
if oldExists {
_, newErr := os.Stat(newDir)
switch {
case newErr == nil:
return fmt.Errorf("cannot auto-migrate database: both %q and %q exist under %s; remove one manually and retry",
oldName, newName, dataDir)
case !os.IsNotExist(newErr):
return fmt.Errorf("checking target directory %q: %w", newDir, newErr)
default:
if err := os.Rename(oldDir, newDir); err != nil {
return fmt.Errorf("renaming database directory: %w", err)
}
fmt.Fprintf(os.Stderr, "bd: migrated database directory %q → %q (GH#3231)\n", oldName, newName)
}
}
if cfg != nil && cfg.DoltDatabase != newName {
cfg.DoltDatabase = newName
if err := cfg.Save(beadsDir); err != nil {
return fmt.Errorf("persisting sanitized database name to metadata.json: %w", err)
}
fmt.Fprintf(os.Stderr, "bd: updated metadata.json dolt_database %q → %q (GH#3231)\n", oldName, newName)
}
return nil
}
// newReadOnlyStoreFromConfig creates a read-only storage backend from the beads
// directory's persisted metadata.json configuration.
//
// For embedded mode, invalid characters (hyphens, dots) are sanitized in-memory
// only — no directory renames or metadata.json writes. This prevents cross-repo
// hydration from mutating foreign projects (GH#3231).
func newReadOnlyStoreFromConfig(ctx context.Context, beadsDir string) (storage.DoltStorage, error) {
return openNonMutatingStoreFromConfig(ctx, beadsDir, false)
}
// newPreviewStoreFromConfig is newReadOnlyStoreFromConfig for a preview
// command (--dry-run, --inspect) reading a repository it was only pointed at:
// the same non-mutating open, but a BEHIND schema cursor is tolerated rather
// than refused, matching embeddeddolt.OpenForPreviewCommand and the preview
// policy the root pre-run applies to the command's own store. A preview that
// only reads a parent issue out of another repo should not be the thing that
// refuses because that repo has not been migrated yet — and it must certainly
// not migrate it.
func newPreviewStoreFromConfig(ctx context.Context, beadsDir string) (storage.DoltStorage, error) {
return openNonMutatingStoreFromConfig(ctx, beadsDir, true)
}
// openNonMutatingStoreFromConfig deliberately does NOT activate the events
// journal: every arm below opens a store that refuses writes (OpenReadOnly,
// OpenForPreviewCommand, ReadOnly server config), so there is no mutation for a
// journal row to accompany. Registered in the construction guard's exemption
// list with that reason.
//
// The registry arm is the exception: it returns whatever the backend's
// OpenReadOnly means, and that one hook also serves the root pre-run's
// classified reads, `bd ready --claim` included, so it may be writable. The
// http client backend's read-only open is (httpclient.NewReadOnlyFromConfig),
// and a write through it reaches its bd serve. The journal exemption still
// holds there: recording that write is the business of the server that applies
// it, and the client has no journal of its own to activate. A separate
// OpenNonMutating hook on backends.Backend, which this function would call
// instead, is the follow-up that closes the exception.
func openNonMutatingStoreFromConfig(ctx context.Context, beadsDir string, preview bool) (storage.DoltStorage, error) {
if err := checkDiscoveredWorkspaceFormat(beadsDir); err != nil {
return nil, err
}
cfg, err := configfile.Load(beadsDir)
if err != nil {
// Same contract as newDoltStoreFromConfig: a present-but-unloadable
// metadata.json is a hard error, not a silent embedded fallback —
// and the error must name the real cause rather than the downstream
// "database not found" the embedded open would produce.
return nil, fmt.Errorf("load %s: %w (refusing to fall back to the embedded store)", configfile.ConfigPath(beadsDir), err)
}
if err := validateConfiguredBackend(cfg, beadsDir); err != nil {
return nil, err
}
cfg = normalizeLoadedConfig(cfg)
if backend, ok := backends.Lookup(cfg.GetBackend()); ok {
return backend.OpenReadOnly(ctx, beadsDir)
}
if cfg != nil && cfg.IsDoltProxiedServerMode() {
return nil, errProxiedStoreUnrouted()
}
if effectiveServerMode(beadsDir, cfg) {
return dolt.NewFromConfigWithOptions(ctx, beadsDir, &dolt.Config{ReadOnly: true})
}
database := configfile.DefaultDoltDatabase
if cfg != nil {
database = cfg.GetDoltDatabase()
}
if sanitized := sanitizeDBName(database); sanitized != database {
database = sanitized
}
if preview {
return embeddeddolt.OpenForPreviewCommand(ctx, beadsDir, database, "main")
}
// OpenReadOnly, not Open: a read-only open of a foreign project must not
// run the remote-migrate gate (a behind, remote-backed database would fail
// hard) and must not write migrations into the target's history
// (bd-6dnrw.32, GH#3231).
return embeddeddolt.OpenReadOnly(ctx, beadsDir, database, "main")
}