From 686bc798cb5a5c44a70303a92c9c99dfd2b19ebc Mon Sep 17 00:00:00 2001 From: Rongjun GENG Date: Sun, 27 Sep 2026 10:32:56 -0700 Subject: [PATCH] fix(prime): honor shared-server mode consistently across bd prime's store resolution paths (#6589) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(store-factory): honor shared-server mode in prime's store resolution newDoltStoreFromConfig and its read-only sibling openNonMutatingStoreFromConfig checked only cfg.IsDoltServerMode(), which deliberately does not read dolt.shared-server from config.yaml (to avoid a circular import with the doltserver package). A workspace with config.yaml tracked but metadata.json gitignored as machine-local state — the common shape of a linked git worktree — left configfile.Load returning (nil, nil), so both paths silently fell through to the embedded backend and created a phantom database named "beads" instead of reaching the real shared server. cmd/bd/main.go's own top-level command resolution already compensates for this exact gap (GH#3817), but bd prime resolves its store through a separate path (formatMemoriesForPrime -> ensureStoreActiveForPrime -> ensureStoreActiveWithContext -> newDoltStoreFromConfig) that never got the same fix. Centralized the compensation into effectiveServerMode so the two internal paths and main.go's own resolution cannot drift from each other again. Fixes #6551. * fix(store-factory): route read-only shared mode * fix(store-factory): reach the !cgo twins and scope shared-server per workspace Maintainer review fixes for PR #6589 (GH#6551). Addresses the three majors in the review synthesis, plus the two non-gating items, and corrects two false claims in the PR's own new prose. 1. major — the !cgo twins kept the unfixed gate, and "centralizing the check here keeps them from drifting again" was structurally false. effectiveServerMode sat behind //go:build cgo, so store_factory_nocgo.go's own copies of newDoltStoreFromConfig / newReadOnlyStoreFromConfig could not call it: the released CGO_ENABLED=0 binaries (.goreleaser.yml ships three) still ignored shared-server in the exact GH#6551 shape, and there the fallthrough is the "reinstall with embedded support" refusal rather than a phantom database — actively misleading, in the one build where a shared server is the only usable backend. Moved the helper to a new untagged file, store_factory_server_mode.go, and called it at all four sites. The centralization claim is now true rather than reworded, and the new file's header records why it must stay untagged. 2. major — TestPrimeCfgNilHonorsSharedServer was vacuous. Two separate causes, both fixed: - the beads dir had no files at all, so hasBeadsProjectFiles was false, FindBeadsDir() returned "" and ensureStoreActiveWithContext refused with ErrNoBeadsDatabase before the factory was reached. It now gets the config.yaml the reported shape actually has, which is simultaneously discoverable and still metadata.json-less. - it exported BEADS_DOLT_SHARED_SERVER=1, which the OLD gate already honored, so the A/B stayed green on unfixed code even after the first fix. The env var is now deliberately empty: the config.yaml layer is the whole of the base gap. Renamed to TestPrimeHonorsSharedServerConfigYaml, because the "cfg == nil" mechanism the old name asserts does not happen at this call site — normalizeLoadedConfig substitutes DefaultConfig() before the gate, so the gate's `cfg != nil` conjunct never short-circuited here. The same false mechanism is corrected in TestEffectiveServerMode's comment. Also added the positive assertion the test lacked: it now requires the captured output to name the port it pinned, so a refusal before store resolution can no longer pass. Confirmed by A/B against the reverted gate. 3. major — cross-workspace opens resolved activation from the launcher's process-global config, contradicting the invariant three lines above the gate. doltserver.IsSharedServerMode() reads process-global state, but routed.go:214, create.go:408 and init_contributor.go:330 pass a foreign beadsDir, so a launcher enabling shared-server in its own config.yaml could retarget a workspace that explicitly asked for embedded — a connect failure or a same-named database holding someone else's rows. effectiveServerMode now takes beadsDir and resolves the yaml layer against it via config.WorkspaceYamlValueStrictWithLocal (config.yaml only in the first iteration; see item 4 below). The env arm stays machine-global (configfile.IsDoltServerMode already honors it above an explicit dolt_mode, so that is base-equivalent), and the process-wide merge is still consulted when the target declares no dolt_mode of its own — which preserves both the GH#6551 headline fix and base behavior, since IsDoltServerMode falls through to config.yaml only when c.DoltMode == "". Purely workspace-scoping the read as prescribed would have dropped the user-global layer, which viper merges, and regressed the fix it was meant to protect. Shared-server mode still wins over a stale metadata.json dolt_mode=embedded for the bound workspace (main.go:1710 / shouldUseExternalDoltStatus, GH#2946); TestEffectiveServerModeResolvesYamlPerWorkspace pins both halves of that pair, and each half was mutation-proved to fail independently. Non-gating, taken because they were adjacent: - TestEffectiveServerMode's false-case assertions are now hermetic (config.ResetForTesting + HOME/BEADS_DIR scoping), matching its sibling. - the new assertion no longer matches on internal/storage/dolt's connection prose; it uses errors.Is(err, syscall.ECONNREFUSED) and the port the test itself pins, both owned outside the package this PR does not own. Required by fix 1 rather than optional: routing the !cgo factories through effectiveServerMode makes them consult ambient config, which reds the two pre-existing TestNocgoNew*FromConfig flag-suggestion tests on any machine with dolt.shared-server enabled. Both now isolate those signals first, verified by running them with BEADS_DOLT_SHARED_SERVER=1. Tests: TestNocgoFactoriesHonorSharedServer is new and is the missing behavioral half of fix 1 — the cgo-tagged tests cannot observe the !cgo twins and the pure-Go CI job only checks compilation. Each of its two arms was proved by reverting one !cgo call site at a time. Validated: gofmt, git diff --check, go build and go vet under both CGO_ENABLED settings, and every touched test individually in both lanes under a private temp root (the shared /tmp suite-root sweeper races sibling agent sessions; see the pre-review CI note). Iteration 2 of the maintainer review found one new major, introduced by fix 3 itself, plus four non-gating items. All five are addressed here. 4. major -- the workspace-scoped read dropped the config.local.yaml layer, so the four factories resolved dolt.shared-server differently from main.go, bootstrap, doctor and migrate-dolt-mode: the same resolver-divergence class GH#6551 is an instance of, newly introduced by scoping the read. Initialize merges .beads/config.local.yaml LAST (the file-loading block at the end of Initialize), i.e. at the highest priority, and it is the documented place for machine-specific settings that must not be committed (docs/reference/configuration.md). dolt.shared-server is exactly that kind of state while .beads/config.yaml is git-tracked, so "the repo tracks it, this machine opts out" is the ordinary use of the escape hatch -- this repo is itself that shape. The divergence ran both ways and the enabling direction was the silent one: config.yaml false (what `bd dolt shared-server off` persists) plus a local true fell through to embeddeddolt.Open and re-created the GH#6551 phantom database with no error. Added config.WorkspaceYamlValueStrictWithLocal beside WorkspaceYamlValueStrict so the layering lives in the package that owns it, and pointed the helper at it. Each layer keeps the strict contract -- absent file or absent key falls through, unreadable/unparseable/non-scalar is an error -- which matches Initialize, since it fails outright on a config.local.yaml it cannot merge. config.local.yaml's name is now a config constant used by both readers, so the merged and workspace-scoped paths cannot drift onto different files. TestEffectiveServerModeHonorsWorkspaceLocalYaml pins both directions and uses the merged config.GetBool as its oracle, so the assertion cannot drift from the layer order it pins; reverting to the config.yaml-only reader fails both arms independently. The config package's own contract test additionally pins the order, the fallthrough when the local file is silent on the key, and the error on a malformed local file; inverting the two layers fails it. The prescription asked for cast.ToBool parsing. Used strconv.ParseBool instead: spf13/cast is an indirect dependency, ParseBool is exactly what cast.ToBool calls for a string (so "1"/"t"/"T"/"TRUE" resolve here as they do everywhere else, which was the point), and it matches migrationSharedTopology, the in-tree precedent for parsing this key from this reader. An unparseable value is viper's false too, and now says so. Corrected the parity claim the finding named: for the bound workspace these are the same two files the process-wide merge reads, which is true only now. Non-gating items from iteration 2, taken because the iteration was already forced and all four live in code this fixup owns: - the read-only factory's positive discriminator was a near-tautology: the pinned port was "1", OR'd with errors.Is(ECONNREFUSED), so the string half matched any errno, timestamp or port containing that digit and held up wherever the dial error was not ECONNREFUSED. Pinned 59998 instead, five distinctive digits like the prime sibling. Verified both halves now fire independently, so neither is carrying the other. - the new !cgo behavioral pin ran in no CI lane: the pure-Go cmd/bd test step is a name allowlist and omitted it, so the exact single-call-site regression this commit mutation-simulated would have passed every job. Added the three Nocgo test names to that -run group in both pr.yml and main.yml. The three pass under -tags gms_pure_go, CGO_ENABLED=0, -short in 0.4s with no server, and reverting store_factory_nocgo.go's first call site now reds that lane -- failing only the mutated arm, so per-site discrimination is preserved. (This also retires the "the pure-Go CI job only checks compilation" note above: that job does run a named subset, and now it runs these.) - the unreadable-config.yaml arm degraded silently and its comment claimed the opposite: it said the configfile.Load error path "refuses to degrade to the embedded default", but returning false IS the degrade to embedded here. Kept the conservative false (main.go degrades identically on the same corrupt input, so it is parity, and a target whose own config is unreadable must not be retargeted by the LAUNCHER's config), added a debug.Logf so the discarded parse error is recoverable, and corrected the comment to describe what the code does. - TestNocgoFactoriesHonorSharedServer's discriminator was negative-only, the one-sidedness class this commit fixed twice elsewhere: any future pre-gate refusal would have passed it without the shared-server arm running. Added the positive half the siblings use, against its own distinctive pinned port. Iteration 2 validation: gofmt, git diff --check, go build ./... and go vet under both CGO_ENABLED settings, the full internal/config suite, the cmd/bd store-factory and prime shared-server tests (including the 30s flagship binary test), and the exact pure-Go CI step command with the widened allowlist. Every fix above was mutation-proved, each arm separately. Iteration 3 of the maintainer review returned approve with no blocker or major finding: one minor and five nits. All but one are addressed here, amended into this same fixup rather than stacked. 5. minor -- the layered reader honored a lone config.local.yaml, so the fixed divergence survived at the edge of the fix. Initialize hangs its local merge off the project config.yaml it actually found (primaryConfigPath, assigned only where an os.Stat on a config.yaml succeeded), so a .beads holding config.local.yaml ALONE is never merged: config.GetBool answers from defaults while the four patched factories answered from the local file. Same resolver-divergence class as item 4, one-directional instead of two, and loud (a split-brain connect failure) rather than a phantom database. Took the "require the pair" arm rather than only narrowing the prose. A reader whose entire contract is "resolve the way Initialize's merged reader does" should satisfy that contract instead of documenting where it does not, and the permissive read left it the single resolver in the tree that honors a file nothing else reads -- the divergence pointed the other way. Two lines in WorkspaceYamlValueStrictWithLocal; a stat error other than not-exist drops the layer too, matching Initialize, whose own probes for a project config.yaml fail the same way, and the tracked read then surfaces that error instead of swallowing it. Added the missing table arm (local only, no tracked config.yaml) and mutation- proved the guard in both directions: neutralizing it reds that arm alone, with the divergent ("true", true) answer, while all seven incumbent arms stay green; always dropping the local layer reds the two override arms and the malformed- local arm while the new one stays green. The guard fires on exactly the population it was written for. Both parity claims now state Initialize's condition as well as its order (store_factory_server_mode.go, yaml_config.go). Nits, taken because amending this fixup costs nothing extra: - the !cgo test's own comment still told the reader the pure-Go job "only checks that the package compiles" -- in the commit that allowlisted that test by name in both workflows. It reads as license to drop the name, which would retire exactly the coverage iteration 2 bought. It now says the job runs this test by name in pr.yml and main.yml, and that the test name and both allowlists must move together. - three new doc comments cited loadConfigFiles, which does not exist; the merge is inline at the end of Initialize. All three now cite Initialize, as does item 4 of this message. - the new constants' justification claimed they keep the merged reader and the workspace-scoped reader from drifting onto different files, but only the local path was switched -- Initialize still spells config.yaml literally everywhere it probes for one. Narrowed the comment to what the constants actually bind, rather than rewriting config-path literals this PR has no other reason to move. - the three pinned "nothing listens here" ports (59997/59998/59999) sat inside the kernel's ephemeral range (32768-60999 here and on the GitHub runners). That only became load-bearing because the two discriminator fixes worked: the assertions now mean "we dialed THIS port and were refused", so a process that bound :0 and landed there would answer the dial and red a correct implementation. Moved all three to 19997/19998/19999. Each move is proved live by the assertion that names its constant -- the prime test still finds its port in the captured output. Declined: switching migrationSharedTopology (migrate_dolt_mode.go:1238) to the layered reader. Both lanes graded it optional and explicitly non-regressing -- migrate has always read config.yaml only -- and it is a production behavior change to `bd migrate dolt-mode` that no further review pass would see, on an iteration that is otherwise approve. It belongs with the five other unlayered IsDoltServerMode resolvers iteration 2 recorded (where.go:143, context_cmd.go:140, doctor/database.go:311, doctor/shared_store.go:45, doctor/server.go:73) as follow-up. Iteration 3 validation: gofmt, git diff --check, go vet in the default, CGO_ENABLED=0/gms_pure_go and GOOS=windows modes, the full internal/config suite, the five cmd/bd resolver and call-site tests including the 33s prime binary test at its new port, the migrate-dolt-mode topology test, and the exact pure-Go CI step command (ok, 0.44s). Co-Authored-By: Claude Opus 5 --------- Co-authored-by: Julian Knutsen Co-authored-by: Claude Opus 5 --- .github/workflows/main.yml | 2 +- .github/workflows/pr.yml | 2 +- cmd/bd/prime_shared_server_test.go | 128 +++++++++++++ cmd/bd/store_factory.go | 9 +- cmd/bd/store_factory_nocgo.go | 4 +- cmd/bd/store_factory_nocgo_test.go | 136 ++++++++++++++ cmd/bd/store_factory_server_mode.go | 134 ++++++++++++++ cmd/bd/store_factory_test.go | 266 ++++++++++++++++++++++++++++ internal/config/config.go | 18 +- internal/config/yaml_config.go | 63 ++++++- internal/config/yaml_config_test.go | 104 +++++++++++ 11 files changed, 855 insertions(+), 11 deletions(-) create mode 100644 cmd/bd/prime_shared_server_test.go create mode 100644 cmd/bd/store_factory_server_mode.go diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 68181e7ad..93ce257fa 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -181,7 +181,7 @@ jobs: # is available; the goal here is to catch compile-time contamination, # not run the full suite. run: | - go test -tags gms_pure_go -count=1 -short -run '^Test(Help|CheckRemoteSafety|FormatDestroyToken|ShouldWireInitRemote|ExtractPrefix|IsNumericID|GetWorktreeGitDir|DriftItemStatuses|RunDriftChecks|IsServerProbablyRunning|CheckHooksDriftNotGitRepo|CheckServerDriftNoBeadsDir|CheckRemoteDriftNoBeadsDir)' ./cmd/bd + go test -tags gms_pure_go -count=1 -short -run '^Test(Help|CheckRemoteSafety|FormatDestroyToken|ShouldWireInitRemote|ExtractPrefix|IsNumericID|GetWorktreeGitDir|DriftItemStatuses|RunDriftChecks|IsServerProbablyRunning|CheckHooksDriftNotGitRepo|CheckServerDriftNoBeadsDir|CheckRemoteDriftNoBeadsDir|NocgoFactoriesHonorSharedServer|NocgoNewDoltStoreFromConfig_ErrorSuggestsCorrectFlag|NocgoNewReadOnlyStoreFromConfig_ErrorSuggestsCorrectFlag)' ./cmd/bd # Fast check to ensure all version files are in sync check-version-consistency: diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index 3f90bca75..0600e4d51 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -251,7 +251,7 @@ jobs: # is available; the goal here is to catch compile-time contamination, # not run the full suite. run: | - go test -tags gms_pure_go -count=1 -short -run '^Test(Help|CheckRemoteSafety|FormatDestroyToken|ShouldWireInitRemote|ExtractPrefix|IsNumericID|GetWorktreeGitDir|DriftItemStatuses|RunDriftChecks|IsServerProbablyRunning|CheckHooksDriftNotGitRepo|CheckServerDriftNoBeadsDir|CheckRemoteDriftNoBeadsDir)' ./cmd/bd + go test -tags gms_pure_go -count=1 -short -run '^Test(Help|CheckRemoteSafety|FormatDestroyToken|ShouldWireInitRemote|ExtractPrefix|IsNumericID|GetWorktreeGitDir|DriftItemStatuses|RunDriftChecks|IsServerProbablyRunning|CheckHooksDriftNotGitRepo|CheckServerDriftNoBeadsDir|CheckRemoteDriftNoBeadsDir|NocgoFactoriesHonorSharedServer|NocgoNewDoltStoreFromConfig_ErrorSuggestsCorrectFlag|NocgoNewReadOnlyStoreFromConfig_ErrorSuggestsCorrectFlag)' ./cmd/bd # The lingering-handle liveness regression test is //go:build windows: the # Ubuntu job above selects it by name but compiles it out, and main.yml's diff --git a/cmd/bd/prime_shared_server_test.go b/cmd/bd/prime_shared_server_test.go new file mode 100644 index 000000000..bd4213ccb --- /dev/null +++ b/cmd/bd/prime_shared_server_test.go @@ -0,0 +1,128 @@ +//go:build cgo + +package main + +import ( + "context" + "os" + "path/filepath" + "runtime" + "strings" + "testing" + "time" +) + +// sharedServerPrimePort is pinned so the env var the run is given and the +// assertion on the captured output cannot drift apart. BEADS_DOLT_SERVER_PORT is +// the highest-priority port source (internal/doltserver), so it wins over +// DefaultSharedServerPort, and nothing listens here. +// +// Below 32768, i.e. outside the kernel's ephemeral range (32768-60999 here and +// on the GitHub runners), so a process that bound :0 cannot be handed this port +// and answer the dial the assertion below expects to fail. +const sharedServerPrimePort = "19999" + +// TestPrimeHonorsSharedServerConfigYaml is a regression test for GH#6551. +// +// bd prime resolves its store through ensureStoreActiveForPrime -> +// ensureStoreActiveWithContext -> newDoltStoreFromConfig, a different path +// from every other command's resolution in cmd/bd/main.go. That path did not +// compensate for the gap configfile.IsDoltServerMode() leaves on purpose: it +// does not read dolt.shared-server from config.yaml, to avoid a circular +// import with the doltserver package. A linked git worktree commonly has +// config.yaml tracked but metadata.json gitignored as machine-local, so the +// workspace's only statement of shared-server mode lives in the one file this +// resolution ignored, and prime silently fell through to embeddeddolt.Open, +// creating a phantom embedded database named "beads" instead of reaching the +// real shared server. Same shape as the GH#3817 fix already applied to +// main.go's own resolution (see TestSharedServerCfgNilHonorsSharedServer) and +// the same centralizing fix (effectiveServerMode). +// +// The config.yaml layer is the ONLY thing this test may rely on, which is why +// BEADS_DOLT_SHARED_SERVER is deliberately absent from the env below. +// configfile.IsDoltServerMode honors that env var itself (step 2 of its +// precedence chain), and newDoltStoreFromConfig runs normalizeLoadedConfig +// before the gate, so an absent metadata.json arrives as a non-nil +// DefaultConfig() rather than nil: exporting the env var made the OLD gate +// select server mode too, and the A/B went green on unfixed code. This test +// was named for a `cfg == nil` short-circuit that does not happen at this call +// site. +// +// Hermetic: no container required. Auto-start is disabled and the server +// port points nowhere, so the honored path fails fast instead of silently +// creating the phantom store — the observable signal this test checks for. +func TestPrimeHonorsSharedServerConfigYaml(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("not supported on Windows") + } + + bdBinary := buildSharedServerTestBinary(t) + + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + + // metadata.json-less beads dir — the configfile.Load -> (nil, nil) case + // that left the compensation gap. + beadsDir := filepath.Join(t.TempDir(), ".beads", "shared-server-prime") + if err := os.MkdirAll(beadsDir, 0o755); err != nil { + t.Fatalf("mkdir metadata-less beads dir: %v", err) + } + // The dir needs a config.yaml, and not only for realism. It is + // simultaneously the reported GH#6551 shape — config.yaml tracked, + // metadata.json gitignored as machine-local, so configfile.Load still + // returns (nil, nil) because it reads only metadata.json — and what makes + // the workspace DISCOVERABLE. With an entirely empty dir, + // internal/beads.hasBeadsProjectFiles is false, so FindBeadsDir() ignores + // BEADS_DIR and returns "", and ensureStoreActiveWithContext refuses with + // ErrNoBeadsDatabase before newDoltStoreFromConfig is ever reached. This + // test then passed identically with and without the fix: no phantom store + // can be created on either, so the assertion below proved nothing. + if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), + []byte("dolt:\n shared-server: true\n auto-start: false\n"), 0o600); err != nil { + t.Fatalf("write config.yaml: %v", err) + } + + env := []string{ + "PATH=" + os.Getenv("PATH"), + "HOME=" + t.TempDir(), + "BEADS_DIR=" + beadsDir, + // BEADS_DOLT_SHARED_SERVER is intentionally NOT set here; see the note + // on the test. The workspace's config.yaml is the signal under test. + "BEADS_DOLT_SHARED_SERVER=", + // Disable auto-start and point at a port nothing listens on so the + // shared-server path fails fast instead of spinning up a server. + "BEADS_DOLT_AUTO_START=0", + "BEADS_DOLT_SERVER_PORT=" + sharedServerPrimePort, + "BD_DISABLE_EVENT_FLUSH=1", + "BEADS_TEST_MODE=1", + "GIT_TERMINAL_PROMPT=0", + "GIT_ASKPASS=", + "SSH_ASKPASS=", + "GT_ROOT=", + } + + neutralCwd := t.TempDir() + // A connection error from the honored shared-server attempt is expected and + // fine — formatMemoriesForPrime degrades gracefully to a "memory + // unavailable" banner rather than failing the command — so the exit status + // carries no signal. The OUTPUT does, and it is checked below. + out, _ := ssExec(ctx, bdBinary, neutralCwd, env, "prime", "--memories-only") + + // Positive half: prime must have REACHED shared-server resolution. The + // phantom-directory check below is one-sided on its own — any earlier + // failure (a rename of `prime --memories-only`, a bad env pin, any refusal + // before store resolution) also leaves no embeddeddolt directory, so the + // test would go green without ever exercising the honored path. Asserting + // the pinned endpoint appears is what makes it able to fail for the reason + // it was written. + if !strings.Contains(out, sharedServerPrimePort) { + t.Fatalf("bd prime never reached shared-server resolution: output does not name the pinned endpoint %s.\n"+ + "Output:\n%s", sharedServerPrimePort, out) + } + + phantom := filepath.Join(beadsDir, "embeddeddolt") + if info, statErr := os.Stat(phantom); statErr == nil && info.IsDir() { + t.Fatalf("bd prime created a phantom embedded database at %s for a workspace with "+ + "dolt.shared-server: true in its config.yaml (GH#6551 regression)", phantom) + } +} diff --git a/cmd/bd/store_factory.go b/cmd/bd/store_factory.go index 0bf5fe521..4b7a0c588 100644 --- a/cmd/bd/store_factory.go +++ b/cmd/bd/store_factory.go @@ -150,7 +150,10 @@ func acquireEmbeddedLock(beadsDir string, serverMode bool) (util.Unlocker, error // // This is the factory the CROSS-WORKSPACE opens use — routed creates, // remote-cache hydration — so activation is resolved from beadsDir's own -// config, not the launching workspace's. +// config, not the launching workspace's. effectiveServerMode below is what +// keeps that true for the dolt.shared-server layer; the one deliberate +// exception is BEADS_DOLT_SHARED_SERVER, which is machine-global in every +// resolver in the tree (see sharedServerModeForWorkspace). func newDoltStoreFromConfig(ctx context.Context, beadsDir string) (s storage.DoltStorage, err error) { defer func() { s, err = activateEventsJournalStore(beadsDir, s, err) }() cfg, err := configfile.Load(beadsDir) @@ -172,7 +175,7 @@ func newDoltStoreFromConfig(ctx context.Context, beadsDir string) (s storage.Dol if cfg != nil && cfg.IsDoltProxiedServerMode() { return nil, errProxiedStoreUnrouted() } - if cfg != nil && cfg.IsDoltServerMode() { + if effectiveServerMode(beadsDir, cfg) { return dolt.NewFromConfig(ctx, beadsDir) } database := configfile.DefaultDoltDatabase @@ -274,7 +277,7 @@ func openNonMutatingStoreFromConfig(ctx context.Context, beadsDir string, previe if cfg != nil && cfg.IsDoltProxiedServerMode() { return nil, errProxiedStoreUnrouted() } - if cfg != nil && cfg.IsDoltServerMode() { + if effectiveServerMode(beadsDir, cfg) { return dolt.NewFromConfigWithOptions(ctx, beadsDir, &dolt.Config{ReadOnly: true}) } database := configfile.DefaultDoltDatabase diff --git a/cmd/bd/store_factory_nocgo.go b/cmd/bd/store_factory_nocgo.go index 152ceb294..f50303512 100644 --- a/cmd/bd/store_factory_nocgo.go +++ b/cmd/bd/store_factory_nocgo.go @@ -83,7 +83,7 @@ func newDoltStoreFromConfig(ctx context.Context, beadsDir string) (s storage.Dol if cfg != nil && cfg.IsDoltProxiedServerMode() { return nil, errProxiedStoreUnrouted() } - if cfg != nil && cfg.IsDoltServerMode() { + if effectiveServerMode(beadsDir, cfg) { return dolt.NewFromConfig(ctx, beadsDir) } return nil, fmt.Errorf("%s", nocgoEmbeddedErrMsg) @@ -108,7 +108,7 @@ func newReadOnlyStoreFromConfig(ctx context.Context, beadsDir string) (storage.D if cfg != nil && cfg.IsDoltProxiedServerMode() { return nil, errProxiedStoreUnrouted() } - if cfg != nil && cfg.IsDoltServerMode() { + if effectiveServerMode(beadsDir, cfg) { return dolt.NewFromConfigWithOptions(ctx, beadsDir, &dolt.Config{ReadOnly: true}) } return nil, fmt.Errorf("%s", nocgoEmbeddedErrMsg) diff --git a/cmd/bd/store_factory_nocgo_test.go b/cmd/bd/store_factory_nocgo_test.go index 384078acb..b20a8cacc 100644 --- a/cmd/bd/store_factory_nocgo_test.go +++ b/cmd/bd/store_factory_nocgo_test.go @@ -3,12 +3,52 @@ package main import ( + "context" + "errors" + "os" + "path/filepath" "strings" + "syscall" "testing" + "time" + "github.com/steveyegge/beads/internal/config" "github.com/steveyegge/beads/internal/storage/dolt" ) +// nocgoSharedServerPort is a distinctive port nothing listens on, pinned through +// BEADS_DOLT_SERVER_PORT (the highest-priority port source in +// internal/doltserver) and reused by the assertion so the two cannot drift +// apart. Distinct from the cgo half's readOnlySharedServerPort and the prime +// test's sharedServerPrimePort only so a failure names which test dialed. +// +// Below 32768, i.e. outside the kernel's ephemeral range (32768-60999 here and +// on the GitHub runners). That matters now that the assertion means "we dialed +// THIS port and were refused": a process that bound :0 and happened to land +// here would answer the dial, turning the expected ECONNREFUSED into a +// handshake error and reddening a correct implementation. +const nocgoSharedServerPort = "19997" + +// isolateSharedServerSignals makes a test's shared-server answer come only from +// what the test itself sets up. +// +// The two *FromConfig tests below assert the ABSENCE of server mode, and both +// now resolve it through effectiveServerMode, whose last fallback is +// doltserver.IsSharedServerMode() — the process-global env var plus the +// config.yaml layer on the config singleton. Without this, an empty beadsDir +// inherits the answer from whatever machine the suite runs on, and on any +// developer box or agent rig with dolt.shared-server enabled the two tests red +// while reporting a connection error instead of the flag-suggestion string they +// are about. Same env-bleed class, and same recipe, as TestEffectiveServerMode +// in the cgo half of this package. +func isolateSharedServerSignals(t *testing.T) { + t.Helper() + config.ResetForTesting() + t.Cleanup(config.ResetForTesting) + t.Setenv("HOME", t.TempDir()) + t.Setenv("BEADS_DOLT_SHARED_SERVER", "") +} + // TestNocgoNewDoltStore_ErrorSuggestsCorrectFlag verifies that newDoltStore // suggests "bd init --server" (not the old "--mode server") when ServerMode // is false. @@ -31,6 +71,7 @@ func TestNocgoNewDoltStore_ErrorSuggestsCorrectFlag(t *testing.T) { // newDoltStoreFromConfig suggests "bd init --server" when no server-mode // config exists. func TestNocgoNewDoltStoreFromConfig_ErrorSuggestsCorrectFlag(t *testing.T) { + isolateSharedServerSignals(t) beadsDir := t.TempDir() // empty dir — no config.json _, err := newDoltStoreFromConfig(t.Context(), beadsDir) if err == nil { @@ -49,6 +90,7 @@ func TestNocgoNewDoltStoreFromConfig_ErrorSuggestsCorrectFlag(t *testing.T) { // newReadOnlyStoreFromConfig suggests "bd init --server" when no server-mode // config exists. func TestNocgoNewReadOnlyStoreFromConfig_ErrorSuggestsCorrectFlag(t *testing.T) { + isolateSharedServerSignals(t) beadsDir := t.TempDir() // empty dir — no config.json _, err := newReadOnlyStoreFromConfig(t.Context(), beadsDir) if err == nil { @@ -62,3 +104,97 @@ func TestNocgoNewReadOnlyStoreFromConfig_ErrorSuggestsCorrectFlag(t *testing.T) t.Errorf("error should NOT contain '--mode' (old flag), got: %s", msg) } } + +// TestNocgoFactoriesHonorSharedServer is the !cgo half of the GH#6551 fix. +// +// store_factory_nocgo.go defines its OWN copies of newDoltStoreFromConfig and +// newReadOnlyStoreFromConfig, so compensation added only to the cgo factories +// did not reach the CGO_ENABLED=0 binaries .goreleaser.yml ships — the build +// where a shared server is the only usable backend. There the embedded +// fallthrough is a refusal rather than a phantom database, so the symptom is +// different and worse: the user is told to reinstall with embedded support +// (nocgoEmbeddedErrMsg) even though they already configured a mode this binary +// can serve. +// +// Every test covering the cgo factories carries //go:build cgo and therefore +// cannot observe this, and before this test the pure-Go CI job only proved the +// package compiles. This test is the missing behavioral half, and it runs in CI +// only because it is allowlisted BY NAME in the "Run pure-Go cmd/bd test subset +// (CGO_ENABLED=0)" step of both .github/workflows/pr.yml and +// .github/workflows/main.yml. The test name and those two allowlists must move +// together: renaming this function without editing both files, or trimming the +// name out of them, retires this coverage silently and restores the +// compile-only lane. Locally: +// `CGO_ENABLED=0 go test ./cmd/bd -run TestNocgoFactoriesHonorSharedServer`. +// It is the exact inverse of the two flag-suggestion tests above: same two +// factories, same empty-metadata.json workspace, opposite expectation once the +// workspace's config.yaml asks for a shared server. +func TestNocgoFactoriesHonorSharedServer(t *testing.T) { + isolateSharedServerSignals(t) + + beadsDir := filepath.Join(t.TempDir(), ".beads") + if err := os.MkdirAll(beadsDir, 0o755); err != nil { + t.Fatal(err) + } + // The GH#6551 shape: config.yaml present (tracked), metadata.json absent + // (gitignored machine-local state), so the workspace's only statement of + // shared-server mode is in the file configfile.IsDoltServerMode does not read. + if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte("dolt:\n shared-server: true\n auto-start: false\n"), 0o600); err != nil { + t.Fatal(err) + } + + t.Setenv("BEADS_DIR", beadsDir) + // BEADS_DOLT_SHARED_SERVER stays empty (isolateSharedServerSignals): the env + // arm was never the broken one, and configfile.IsDoltServerMode honors it + // already. The config.yaml layer is what the !cgo twins still ignored. + t.Setenv("BEADS_DOLT_AUTO_START", "0") + t.Setenv("BEADS_DOLT_SERVER_PORT", nocgoSharedServerPort) + if err := config.Initialize(); err != nil { + t.Fatalf("config.Initialize: %v", err) + } + if !effectiveServerMode(beadsDir, nil) { + t.Fatal("test setup: config.yaml did not enable shared-server mode") + } + + factories := map[string]func(context.Context, string) (interface{ Close() error }, error){ + "newDoltStoreFromConfig": func(ctx context.Context, dir string) (interface{ Close() error }, error) { + return newDoltStoreFromConfig(ctx, dir) + }, + "newReadOnlyStoreFromConfig": func(ctx context.Context, dir string) (interface{ Close() error }, error) { + return newReadOnlyStoreFromConfig(ctx, dir) + }, + } + + for name, open := range factories { + t.Run(name, func(t *testing.T) { + ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second) + defer cancel() + store, err := open(ctx, beadsDir) + if err == nil { + if store != nil { + _ = store.Close() + } + t.Fatalf("%s unexpectedly succeeded without a server", name) + } + // Negative half: the "reinstall with CGO" refusal is what the + // unfixed twin produces, so its absence is what the GH#6551 fix + // buys here. + if strings.Contains(err.Error(), "requires a CGO build") { + t.Fatalf("%s fell through to the embedded refusal with shared-server enabled (GH#6551, !cgo twin); got: %v", name, err) + } + // Positive half: absence alone is one-sided — any future pre-gate + // refusal (say a new validation that errors on a metadata-less dir) + // would satisfy it without the shared-server arm ever running. Only + // that arm dials, so a refused connection to the port this test + // pinned is proof it was reached. Both discriminators are owned + // outside internal/storage/dolt — a syscall sentinel and this test's + // own distinctive port — so a reworded connection error in a package + // this PR does not own cannot turn a correct implementation red. The + // two sibling fixes in this commit added the same half for the same + // reason (prime_shared_server_test.go, store_factory_test.go). + if !errors.Is(err, syscall.ECONNREFUSED) && !strings.Contains(err.Error(), nocgoSharedServerPort) { + t.Fatalf("%s did not dial the shared server at the pinned port %s; got: %v", name, nocgoSharedServerPort, err) + } + }) + } +} diff --git a/cmd/bd/store_factory_server_mode.go b/cmd/bd/store_factory_server_mode.go new file mode 100644 index 000000000..9e9c8cc33 --- /dev/null +++ b/cmd/bd/store_factory_server_mode.go @@ -0,0 +1,134 @@ +package main + +import ( + "os" + "strconv" + "strings" + + "github.com/steveyegge/beads/internal/config" + "github.com/steveyegge/beads/internal/configfile" + "github.com/steveyegge/beads/internal/debug" + "github.com/steveyegge/beads/internal/doltserver" +) + +// This file carries NO build tag on purpose. +// +// Every factory call site with the GH#6551 gap lives in a build-tagged file: +// newDoltStoreFromConfig and its read-only sibling in store_factory.go +// (//go:build cgo), and their twins newDoltStoreFromConfig / +// newReadOnlyStoreFromConfig in store_factory_nocgo.go (//go:build !cgo). A +// helper placed behind either tag could only ever reach half of them, so +// "centralizing the check" would have been a claim the build tags falsify -- +// and .goreleaser.yml ships three CGO_ENABLED=0 targets, the build where a +// shared server is the only usable backend. Keeping the helper untagged is +// what actually stops the four call sites from drifting apart again. Nothing +// here needs cgo: configfile, config and doltserver all build under +// CGO_ENABLED=0. + +// effectiveServerMode reports whether the workspace at beadsDir selects Dolt +// server-mode storage, compensating for the same gap main.go's own resolution +// already compensates for (see the comment on doltCfg.ServerMode in main.go): +// configfile.IsDoltServerMode() deliberately does not read dolt.shared-server +// from config.yaml, to avoid a circular import with the doltserver package. A +// workspace (typically a linked git worktree) that has config.yaml but no +// metadata.json otherwise falls through to the embedded backend and silently +// creates a phantom database -- GH#6551, the same shape +// newDoltStoreFromConfig's caller in main.go was already fixed for under +// GH#3817. +func effectiveServerMode(beadsDir string, cfg *configfile.Config) bool { + if cfg != nil && cfg.IsDoltServerMode() { + return true + } + return sharedServerModeForWorkspace(beadsDir, cfg) +} + +// sharedServerModeForWorkspace resolves dolt.shared-server for the workspace at +// beadsDir rather than for the directory bd happened to be launched from. The +// cross-workspace factories -- routed creates, remote-cache hydration, +// contributor init -- pass a beadsDir that is by construction NOT the bound +// target, and doltserver.IsSharedServerMode() reads process-global state, so +// consulting it directly would resolve activation from the launching workspace +// and contradict the contract documented on newDoltStoreFromConfig. +// legacy_upgrade_guard.go records the same hazard for the same key: "Do not +// rely on the binding from a caller that walks ancestors." +// +// Layering, highest priority first: +// +// 1. BEADS_DOLT_SHARED_SERVER. Machine-global by design, and already what +// configfile.IsDoltServerMode honors ABOVE an explicit dolt_mode (step 2 of +// its documented precedence chain), so honoring it for every workspace here +// is base-equivalent. +// 2. beadsDir's own config.local.yaml, then its config.yaml -- the two +// project-level layers config.Initialize merges, in Initialize's own order +// (local last there, so local wins here) and under Initialize's own +// condition (the local file counts only when a config.yaml sits beside it, +// which is what Initialize hangs its local merge off; see +// config.WorkspaceYamlValueStrictWithLocal). For the bound workspace these +// are the same two files the process-wide merge already read, and config.yaml +// is where the key is actually persisted: `bd init --shared-server` +// (init.go) and `bd dolt shared-server on` (dolt.go) both route through +// config.SetYamlConfig, which resolves to a .beads/config.yaml and never to +// the user-global file, and the mode migrations write it with +// SetYamlConfigInDir(beadsDir, ...). config.local.yaml has no bd writer; it +// is the documented hand-authored escape hatch for holding machine-specific +// state out of the tracked file, and dolt.shared-server is exactly that kind +// of state, so a workspace can track the team's setting and opt this machine +// out. Reading only config.yaml here would make these four factories answer +// that shape differently from main.go, bootstrap, doctor and +// migrate-dolt-mode -- the same resolver divergence GH#6551 is an instance +// of. For a cross-workspace open this pair is the only config that answers +// for the target. This also keeps shared-server mode winning over a stale +// metadata.json that still pins dolt_mode="embedded", which main.go:1710 and +// shouldUseExternalDoltStatus both rely on (GH#2946). +// 3. The process-wide merge, but only when the target declares no dolt_mode of +// its own. That keeps machine-wide enablement working for a workspace that +// has not spoken -- including the GH#6551 shape, where metadata.json is +// absent entirely -- while preserving base behavior for one that has: at +// base the config.yaml layer could not override an explicit dolt_mode +// either, because IsDoltServerMode falls through to config.yaml only when +// c.DoltMode == "". Without this gate a launcher whose config.yaml enables +// shared-server would silently retarget a foreign workspace that explicitly +// asked for embedded onto the shared server -- a connect failure or, worse, +// a same-named database holding someone else's rows. +func sharedServerModeForWorkspace(beadsDir string, cfg *configfile.Config) bool { + if v := os.Getenv("BEADS_DOLT_SHARED_SERVER"); v == "1" || strings.EqualFold(v, "true") { + return true + } + raw, present, err := config.WorkspaceYamlValueStrictWithLocal(beadsDir, "dolt.shared-server") + switch { + case err != nil: + // The target has a config.yaml (or config.local.yaml) we cannot read or + // parse. Returning false here is a deliberate degrade to the embedded + // default -- the same thing main.go's resolution does on the same + // corrupt input, so it is parity rather than a new refusal. What it must + // NOT be is silent: this is the one arm where the answer is "we do not + // know", and an operator whose workspace does enable shared-server gets + // the embedded store (or, under !cgo, the misleading "reinstall with + // CGO" refusal) with the parse error otherwise discarded. Declining + // rather than consulting layer 3 is still the right policy: a target + // whose own configuration is unreadable must not be retargeted onto the + // shared server by the LAUNCHER's configuration. + debug.Logf("Debug: dolt.shared-server unresolved for %s, assuming embedded: %v\n", beadsDir, err) + return false + case present: + // strconv.ParseBool is what the merged reader ends at for this key: + // config.GetBool -> cast.ToBool, which for a string value is + // strconv.ParseBool, so "1"/"t"/"T"/"TRUE" resolve here exactly as they + // do everywhere else. An unparseable value is viper's false too, but + // unlike viper we can say so. Same parse as migrationSharedTopology + // applies to this key from this reader. + parsed, parseErr := strconv.ParseBool(strings.TrimSpace(raw)) + if parseErr != nil { + debug.Logf("Debug: dolt.shared-server in %s is not a boolean (%q), assuming embedded\n", beadsDir, raw) + return false + } + return parsed + } + // cfg.DoltMode, not cfg.GetDoltMode(): the getter substitutes the embedded + // default for an absent mode, and this gate must fire only on an explicit + // statement, exactly as IsDoltServerMode's own `c.DoltMode != ""` check does. + if cfg != nil && cfg.DoltMode != "" { + return false + } + return doltserver.IsSharedServerMode() +} diff --git a/cmd/bd/store_factory_test.go b/cmd/bd/store_factory_test.go index 4a76db204..6f76dd92c 100644 --- a/cmd/bd/store_factory_test.go +++ b/cmd/bd/store_factory_test.go @@ -3,12 +3,17 @@ package main import ( + "context" "encoding/json" + "errors" "os" "path/filepath" "strings" + "syscall" "testing" + "time" + "github.com/steveyegge/beads/internal/config" "github.com/steveyegge/beads/internal/configfile" "github.com/steveyegge/beads/internal/storage/dolt" "github.com/steveyegge/beads/internal/storage/embeddeddolt" @@ -42,6 +47,267 @@ func TestNewDoltStoreFromConfig_NoMetadata(t *testing.T) { defer store.Close() } +// TestEffectiveServerMode is a regression test for GH#6551: newDoltStoreFromConfig +// and its read-only sibling openNonMutatingStoreFromConfig checked only +// cfg.IsDoltServerMode(), which does not read dolt.shared-server from +// config.yaml (deliberately, to avoid a circular import with doltserver). +// A workspace with config.yaml but no metadata.json — the common shape of a +// linked git worktree, since metadata.json is commonly gitignored as +// machine-local state — therefore had its only statement of shared-server mode +// silently ignored on these two paths, even though cmd/bd/main.go's own +// resolution already compensates for exactly this gap (GH#3817). +// effectiveServerMode centralizes that compensation so the paths cannot drift +// from main.go's again. +// +// The config.yaml layer is the whole of the gap. BEADS_DOLT_SHARED_SERVER was +// never ignored at these call sites: configfile.IsDoltServerMode honors it +// itself, and normalizeLoadedConfig replaces an absent metadata.json with a +// non-nil DefaultConfig() before the gate runs, so the old gate's `cfg != nil` +// conjunct never short-circuited here. This test covers the env arm because it +// is part of the helper's contract, not because it was broken. +func TestEffectiveServerMode(t *testing.T) { + // The false cases assert the ABSENCE of a shared-server signal, and + // doltserver.IsSharedServerMode falls through to config.GetBool on the + // process-global config singleton, so without this they red spuriously on + // any machine or agent rig that enables dolt.shared-server in config.yaml + // — or after any earlier test in this package leaves the singleton + // initialized with it. The hermetic recipe is the sibling test's below. + config.ResetForTesting() + t.Cleanup(config.ResetForTesting) + t.Setenv("HOME", t.TempDir()) + emptyDir := t.TempDir() + t.Setenv("BEADS_DIR", emptyDir) + + t.Setenv("BEADS_DOLT_SHARED_SERVER", "") + if effectiveServerMode(emptyDir, nil) { + t.Error("effectiveServerMode(nil) = true with no shared-server signal, want false") + } + if effectiveServerMode(emptyDir, &configfile.Config{}) { + t.Error("effectiveServerMode(cfg not naming server) = true with no shared-server signal, want false") + } + + t.Setenv("BEADS_DOLT_SHARED_SERVER", "1") + if !effectiveServerMode(emptyDir, nil) { + t.Error("effectiveServerMode(nil) = false under BEADS_DOLT_SHARED_SERVER=1, want true (GH#6551)") + } + if !effectiveServerMode(emptyDir, &configfile.Config{}) { + t.Error("effectiveServerMode(cfg not naming server) = false under BEADS_DOLT_SHARED_SERVER=1, want true (GH#6551)") + } +} + +// TestEffectiveServerModeResolvesYamlPerWorkspace pins which config.yaml the +// dolt.shared-server layer is resolved against. newDoltStoreFromConfig's +// contract says activation is resolved from beadsDir's own config rather than +// the launching workspace's, and it is the factory the CROSS-WORKSPACE opens +// use (routed.go, create.go, init_contributor.go all pass a foreign beadsDir). +// doltserver.IsSharedServerMode() reads process-global state, so consulting it +// directly let a launcher whose own config.yaml enables shared-server retarget +// a foreign workspace that explicitly asked for embedded onto the shared server +// — a connect failure, or a same-named database holding someone else's rows. +// +// The two halves are a matched pair and both are load-bearing: the bound +// workspace must keep winning over a stale metadata.json that still pins +// dolt_mode="embedded" (main.go:1710 / shouldUseExternalDoltStatus, GH#2946), +// so the guard cannot simply be "an explicit dolt_mode always wins". +func TestEffectiveServerModeResolvesYamlPerWorkspace(t *testing.T) { + config.ResetForTesting() + t.Cleanup(config.ResetForTesting) + + root := t.TempDir() + launcherDir := filepath.Join(root, "launcher", ".beads") + foreignDir := filepath.Join(root, "foreign", ".beads") + for _, dir := range []string{launcherDir, foreignDir} { + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + } + + // The launcher enables shared-server in its OWN config.yaml. This is the + // durable shape: `bd dolt shared-server on` and `bd init --shared-server` + // both persist the key through config.SetYamlConfig, which resolves to a + // .beads/config.yaml. + if err := os.WriteFile(filepath.Join(launcherDir, "config.yaml"), []byte("dolt:\n shared-server: true\n auto-start: false\n"), 0o600); err != nil { + t.Fatal(err) + } + + // Both workspaces carry the same explicit metadata.json statement, so the + // ONLY thing that differs between the two assertions below is which + // workspace the yaml layer is read from. + embeddedMeta := []byte(`{"dolt_mode":"embedded","dolt_database":"beads"}`) + for _, dir := range []string{launcherDir, foreignDir} { + if err := os.WriteFile(filepath.Join(dir, "metadata.json"), embeddedMeta, 0o600); err != nil { + t.Fatal(err) + } + } + + t.Setenv("HOME", t.TempDir()) + t.Setenv("BEADS_DIR", launcherDir) + t.Setenv("BEADS_DOLT_SHARED_SERVER", "") + if err := config.Initialize(); err != nil { + t.Fatalf("config.Initialize: %v", err) + } + + launcherCfg, err := configfile.Load(launcherDir) + if err != nil { + t.Fatalf("load launcher metadata.json: %v", err) + } + foreignCfg, err := configfile.Load(foreignDir) + if err != nil { + t.Fatalf("load foreign metadata.json: %v", err) + } + if launcherCfg == nil || foreignCfg == nil { + t.Fatal("test setup: metadata.json did not load") + } + + if !effectiveServerMode(launcherDir, launcherCfg) { + t.Error("effectiveServerMode(bound workspace) = false; shared-server in the workspace's own config.yaml must win over its stale dolt_mode=embedded (GH#2946)") + } + if effectiveServerMode(foreignDir, foreignCfg) { + t.Error("effectiveServerMode(foreign workspace) = true; the LAUNCHER's config.yaml must not retarget a workspace whose own metadata.json asks for embedded") + } +} + +// TestEffectiveServerModeHonorsWorkspaceLocalYaml pins the SECOND project-level +// layer. config.Initialize merges .beads/config.local.yaml last — the documented +// place for machine-specific settings that must not be committed — so every +// config.GetBool consumer in the tree (main.go, bootstrap, doctor, +// migrate-dolt-mode) lets it override the tracked config.yaml. dolt.shared-server +// is exactly that kind of state and .beads/config.yaml is git-tracked, so "the +// repo enables it, this machine opts out" is the ordinary use of the escape +// hatch, not an exotic shape. +// +// Resolving these four factories against config.yaml alone would answer that +// shape differently from every other resolver — the same resolver-divergence +// class GH#6551 itself is an instance of. Both directions are load-bearing and +// the enabling one is the silent one: config.yaml false (what `bd dolt +// shared-server off` persists) plus a local true would fall through to +// embeddeddolt.Open and re-create the GH#6551 phantom database with no error at +// all. +func TestEffectiveServerModeHonorsWorkspaceLocalYaml(t *testing.T) { + for _, tc := range []struct { + name string + tracked string + local string + want bool + }{ + {name: "local opts this machine out", tracked: "true", local: "false", want: false}, + {name: "local opts this machine in", tracked: "false", local: "true", want: true}, + } { + t.Run(tc.name, func(t *testing.T) { + // Same hermetic recipe as the siblings: these arms are answered by + // the workspace's own files, but the recipe is what keeps that true + // if the layering below them ever changes. + config.ResetForTesting() + t.Cleanup(config.ResetForTesting) + t.Setenv("HOME", t.TempDir()) + t.Setenv("BEADS_DOLT_SHARED_SERVER", "") + + // The GH#6551 shape: config.yaml tracked, no metadata.json, so cfg + // arrives nil and the yaml layer is the workspace's only statement. + beadsDir := filepath.Join(t.TempDir(), ".beads") + if err := os.MkdirAll(beadsDir, 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("BEADS_DIR", beadsDir) + if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), + []byte("dolt:\n shared-server: "+tc.tracked+"\n auto-start: false\n"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(beadsDir, "config.local.yaml"), + []byte("dolt:\n shared-server: "+tc.local+"\n"), 0o600); err != nil { + t.Fatal(err) + } + + // The merged reader is the oracle: whatever config.GetBool answers + // for this workspace is what these factories must answer, so the + // assertion cannot drift from the layer order it is pinning. + if err := config.Initialize(); err != nil { + t.Fatalf("config.Initialize: %v", err) + } + if got := config.GetBool("dolt.shared-server"); got != tc.want { + t.Fatalf("test setup: merged config.GetBool = %v, want %v (config.yaml %s + config.local.yaml %s)", + got, tc.want, tc.tracked, tc.local) + } + + if got := effectiveServerMode(beadsDir, nil); got != tc.want { + t.Errorf("effectiveServerMode = %v, want %v: .beads/config.local.yaml (%s) must override the tracked config.yaml (%s), as config.Initialize merges it last", + got, tc.want, tc.local, tc.tracked) + } + }) + } +} + +// TestOpenNonMutatingStoreHonorsSharedServerConfig pins the read-only factory +// call site, not just effectiveServerMode in isolation. A linked worktree can +// have config.yaml tracked while metadata.json is absent; in that shape the +// active shared server must win over the embedded read-only fallback. +func TestOpenNonMutatingStoreHonorsSharedServerConfig(t *testing.T) { + config.ResetForTesting() + t.Cleanup(config.ResetForTesting) + + beadsDir := filepath.Join(t.TempDir(), ".beads") + if err := os.MkdirAll(beadsDir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte("dolt:\n shared-server: true\n auto-start: false\n"), 0o600); err != nil { + t.Fatal(err) + } + + t.Setenv("BEADS_DIR", beadsDir) + t.Setenv("BEADS_DOLT_SHARED_SERVER", "") + t.Setenv("BEADS_DOLT_AUTO_START", "0") + t.Setenv("BEADS_DOLT_SERVER_PORT", readOnlySharedServerPort) + t.Setenv("HOME", t.TempDir()) + if err := config.Initialize(); err != nil { + t.Fatalf("config.Initialize: %v", err) + } + if !effectiveServerMode(beadsDir, nil) { + t.Fatal("test setup: config.yaml did not enable shared-server mode") + } + + ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second) + defer cancel() + store, err := openNonMutatingStoreFromConfig(ctx, beadsDir, false) + if err == nil { + if store != nil { + _ = store.Close() + } + t.Fatal("openNonMutatingStoreFromConfig unexpectedly succeeded without a server") + } + // Positive half: only the server arm dials, so a refused connection to the + // port this test pinned is proof the shared-server branch was taken. Both + // discriminators are owned outside internal/storage/dolt — a syscall + // sentinel and this test's own port — so a reworded connection error in a + // package this PR does not own cannot turn a correct implementation red. + if !errors.Is(err, syscall.ECONNREFUSED) && !strings.Contains(err.Error(), readOnlySharedServerPort) { + t.Fatalf("read-only factory did not dial the shared server; got: %v", err) + } + // Negative half: the embedded fallback must not have run. + if strings.Contains(err.Error(), "embeddeddolt") { + t.Fatalf("read-only factory fell through to the embedded store; got: %v", err) + } +} + +// readOnlySharedServerPort is a port nothing listens on, pinned through +// BEADS_DOLT_SERVER_PORT (the highest-priority port source in +// internal/doltserver, so it wins over DefaultSharedServerPort) and reused by +// the assertion, so the two cannot drift apart. +// +// It must be DISTINCTIVE, not merely unused: the assertion below ORs this +// substring with errors.Is(ECONNREFUSED), so a short or common value makes that +// half unfalsifiable — "1" matched any errno, timestamp, or port like 3306, and +// the OR then held up wherever the dial error was not ECONNREFUSED (a context +// deadline from the 2s timeout, or a Windows ETIMEDOUT shape), silently +// collapsing a two-discriminator proof to a tautology. Five distinctive digits, +// like the prime sibling's sharedServerPrimePort. +// +// It must also sit BELOW 32768, outside the kernel's ephemeral range (32768-60999 +// here and on the GitHub runners). Now that the substring half carries real +// weight, a process that bound :0 and happened to land on this port would answer +// the dial and turn the expected ECONNREFUSED into a handshake error, reddening +// a correct implementation. +const readOnlySharedServerPort = "19998" + // TestEmbeddedOpen_EmptyDatabaseRejected verifies that embeddeddolt.Open fails // with a clear error when called with an empty database name, rather than // deferring to a confusing "no database selected" SQL error. diff --git a/internal/config/config.go b/internal/config/config.go index 5d37cbd51..152935554 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -14,6 +14,22 @@ import ( "gopkg.in/yaml.v3" ) +const ( + // ProjectConfigFileName is the tracked per-workspace config file, read from + // a .beads directory. + ProjectConfigFileName = "config.yaml" + // LocalConfigFileName sits beside ProjectConfigFileName and is merged LAST + // by Initialize, so it overrides the tracked file. It is the documented + // place for machine-specific settings that must not be committed + // (docs/reference/configuration.md). Named here so Initialize's own local + // merge (in the file-loading block at the end of Initialize) and the + // workspace-scoped reader in WorkspaceYamlValueStrictWithLocal cannot + // drift onto different files. ProjectConfigFileName binds the two + // workspace-scoped strict readers to each other; Initialize still spells + // config.yaml literally in the several places it probes for one. + LocalConfigFileName = "config.local.yaml" +) + var v *viper.Viper // overriddenKeys tracks keys explicitly set via Set() at runtime, so @@ -377,7 +393,7 @@ func Initialize() error { // Merge local config overrides if present (config.local.yaml) // This allows machine-specific settings without polluting tracked config - localConfigPath := filepath.Join(filepath.Dir(primaryConfigPath), "config.local.yaml") + localConfigPath := filepath.Join(filepath.Dir(primaryConfigPath), LocalConfigFileName) if _, err := os.Stat(localConfigPath); err == nil { v.SetConfigFile(localConfigPath) if err := v.MergeInConfig(); err != nil { diff --git a/internal/config/yaml_config.go b/internal/config/yaml_config.go index b6d752cf3..eaa960b86 100644 --- a/internal/config/yaml_config.go +++ b/internal/config/yaml_config.go @@ -375,20 +375,77 @@ func WorkspaceYamlValue(beadsDir, key string) (string, bool) { // without conflating a malformed or unreadable file with an absent key. A // missing file or key returns present=false and no error; all other I/O and // YAML shape errors are returned to the caller. +// +// It reads config.yaml ONLY. Callers that must agree with the merged reader +// (config.GetBool and friends) want WorkspaceYamlValueStrictWithLocal below, +// which also honors the machine-local override layer Initialize merges. func WorkspaceYamlValueStrict(beadsDir, key string) (value string, present bool, err error) { if beadsDir == "" { return "", false, nil } - data, err := os.ReadFile(filepath.Join(beadsDir, "config.yaml")) //nolint:gosec // beadsDir is caller-resolved workspace state + return strictYamlValueAtPath(filepath.Join(beadsDir, ProjectConfigFileName), key) +} + +// WorkspaceYamlValueStrictWithLocal reads one dotted key from ONE workspace's +// project-level config the way Initialize's merged reader resolves it: a +// config.local.yaml beside config.yaml is merged LAST there (the file-loading +// block at the end of Initialize), i.e. it has the highest priority of the two, +// so it is consulted first here. +// +// "Beside config.yaml" is a condition, not scenery. Initialize hangs its local +// merge off the project config.yaml it actually found, so a .beads holding +// config.local.yaml ALONE is never merged and config.GetBool answers from +// defaults. This reader mirrors that and ignores a lone local file, because +// honoring one would put it in the same position the layered read exists to +// fix — the single resolver in the tree that answers from a file nothing else +// reads — just pointed the other way. +// +// That layer is the documented way to hold machine-specific state out of the +// git-tracked config.yaml (docs/reference/configuration.md), which makes it +// load-bearing for exactly the settings a cross-workspace open has to resolve +// against the target — a workspace can track `dolt.shared-server: true` for the +// team and opt one machine out locally. A workspace-scoped reader that saw only +// config.yaml would answer differently from every merged-config consumer in the +// tree for that ordinary shape. +// +// Each layer keeps WorkspaceYamlValueStrict's strict contract: an absent file or +// an absent key falls through to the next layer, while an unreadable file, an +// unparseable one, or a non-scalar value at the key is returned as an error +// rather than silently treated as absent. Initialize is equally strict — it +// fails outright on a config.local.yaml it cannot merge. +func WorkspaceYamlValueStrictWithLocal(beadsDir, key string) (value string, present bool, err error) { + if beadsDir == "" { + return "", false, nil + } + names := []string{LocalConfigFileName, ProjectConfigFileName} + if _, statErr := os.Stat(filepath.Join(beadsDir, ProjectConfigFileName)); statErr != nil { + // No project config.yaml here, so Initialize would have nothing to hang + // the local merge off: drop the local layer to stay in step with it. A + // stat error other than "not exist" drops it too, matching Initialize, + // whose own os.Stat probes for a project config.yaml fail the same way; + // the read below then surfaces that error rather than swallowing it. + names = names[1:] + } + for _, name := range names { + value, present, err = strictYamlValueAtPath(filepath.Join(beadsDir, name), key) + if err != nil || present { + return value, present, err + } + } + return "", false, nil +} + +func strictYamlValueAtPath(path, key string) (value string, present bool, err error) { + data, err := os.ReadFile(path) //nolint:gosec // path is caller-resolved workspace state if os.IsNotExist(err) { return "", false, nil } if err != nil { - return "", false, fmt.Errorf("reading workspace config.yaml: %w", err) + return "", false, fmt.Errorf("reading workspace %s: %w", filepath.Base(path), err) } var root map[string]interface{} if err := yaml.Unmarshal(data, &root); err != nil { - return "", false, fmt.Errorf("parsing workspace config.yaml: %w", err) + return "", false, fmt.Errorf("parsing workspace %s: %w", filepath.Base(path), err) } if root == nil { return "", false, nil diff --git a/internal/config/yaml_config_test.go b/internal/config/yaml_config_test.go index 8603e1604..781503eee 100644 --- a/internal/config/yaml_config_test.go +++ b/internal/config/yaml_config_test.go @@ -419,6 +419,110 @@ func TestWorkspaceYamlValueStrictReadsNestedScalar(t *testing.T) { } } +// TestWorkspaceYamlValueStrictWithLocalMirrorsInitializeMergeOrder pins the one +// thing the layered reader exists for: it must answer the way the merged +// singleton does, and Initialize merges config.local.yaml LAST, so the +// local file wins. A workspace-scoped reader that saw only config.yaml would +// diverge from every config.GetBool consumer for the ordinary shape of a tracked +// config.yaml plus a machine-local override. +func TestWorkspaceYamlValueStrictWithLocalMirrorsInitializeMergeOrder(t *testing.T) { + write := func(t *testing.T, beadsDir, name, body string) { + t.Helper() + if err := os.MkdirAll(beadsDir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(beadsDir, name), []byte(body), 0o600); err != nil { + t.Fatal(err) + } + } + + t.Run("local overrides tracked", func(t *testing.T) { + beadsDir := filepath.Join(t.TempDir(), ".beads") + write(t, beadsDir, ProjectConfigFileName, "dolt:\n shared-server: true\n") + write(t, beadsDir, LocalConfigFileName, "dolt:\n shared-server: false\n") + value, present, err := WorkspaceYamlValueStrictWithLocal(beadsDir, "dolt.shared-server") + if err != nil || !present || value != "false" { + t.Fatalf("= (%q, %v, %v), want the config.local.yaml answer \"false\"/true/nil", value, present, err) + } + }) + + t.Run("local overrides tracked in the enabling direction", func(t *testing.T) { + beadsDir := filepath.Join(t.TempDir(), ".beads") + write(t, beadsDir, ProjectConfigFileName, "dolt:\n shared-server: false\n") + write(t, beadsDir, LocalConfigFileName, "dolt:\n shared-server: true\n") + value, present, err := WorkspaceYamlValueStrictWithLocal(beadsDir, "dolt.shared-server") + if err != nil || !present || value != "true" { + t.Fatalf("= (%q, %v, %v), want the config.local.yaml answer \"true\"/true/nil", value, present, err) + } + }) + + // MergeInConfig only overrides the keys the local file actually states, so a + // local file that is silent about this key must fall through rather than + // mask the tracked answer. + t.Run("local silent on the key falls through", func(t *testing.T) { + beadsDir := filepath.Join(t.TempDir(), ".beads") + write(t, beadsDir, ProjectConfigFileName, "dolt:\n shared-server: true\n") + write(t, beadsDir, LocalConfigFileName, "dolt:\n auto-start: false\n") + value, present, err := WorkspaceYamlValueStrictWithLocal(beadsDir, "dolt.shared-server") + if err != nil || !present || value != "true" { + t.Fatalf("= (%q, %v, %v), want the config.yaml answer \"true\"/true/nil", value, present, err) + } + }) + + t.Run("no local file at all", func(t *testing.T) { + beadsDir := filepath.Join(t.TempDir(), ".beads") + write(t, beadsDir, ProjectConfigFileName, "dolt:\n shared-server: true\n") + value, present, err := WorkspaceYamlValueStrictWithLocal(beadsDir, "dolt.shared-server") + if err != nil || !present || value != "true" { + t.Fatalf("= (%q, %v, %v), want the config.yaml answer \"true\"/true/nil", value, present, err) + } + }) + + // The mirror of the case above, and the corner where being MORE permissive + // than Initialize would reopen the divergence: Initialize hangs its local + // merge off a project config.yaml it found, so a .beads holding only + // config.local.yaml is never merged and config.GetBool answers from + // defaults. This reader has to say "absent" too, or it becomes the one + // resolver in the tree that honors that file. + t.Run("local only, no tracked config.yaml", func(t *testing.T) { + beadsDir := filepath.Join(t.TempDir(), ".beads") + write(t, beadsDir, LocalConfigFileName, "dolt:\n shared-server: true\n") + value, present, err := WorkspaceYamlValueStrictWithLocal(beadsDir, "dolt.shared-server") + if err != nil || present || value != "" { + t.Fatalf("= (%q, %v, %v), want empty/false/nil: Initialize does not merge a config.local.yaml with no config.yaml beside it", value, present, err) + } + }) + + t.Run("neither file states the key", func(t *testing.T) { + beadsDir := filepath.Join(t.TempDir(), ".beads") + write(t, beadsDir, ProjectConfigFileName, "dolt:\n auto-start: false\n") + value, present, err := WorkspaceYamlValueStrictWithLocal(beadsDir, "dolt.shared-server") + if err != nil || present || value != "" { + t.Fatalf("= (%q, %v, %v), want empty/false/nil", value, present, err) + } + }) + + // Initialize fails outright on a config.local.yaml it cannot merge, so the + // strict contract has to apply to the local layer too rather than skipping + // it and quietly answering from the tracked file. + t.Run("malformed local is an error, not a fallthrough", func(t *testing.T) { + beadsDir := filepath.Join(t.TempDir(), ".beads") + write(t, beadsDir, ProjectConfigFileName, "dolt:\n shared-server: true\n") + write(t, beadsDir, LocalConfigFileName, "dolt: [") + if _, _, err := WorkspaceYamlValueStrictWithLocal(beadsDir, "dolt.shared-server"); err == nil { + t.Fatal("malformed config.local.yaml should return an error") + } else if !strings.Contains(err.Error(), LocalConfigFileName) { + t.Fatalf("error should name the file that failed, got: %v", err) + } + }) + + t.Run("empty beadsDir", func(t *testing.T) { + if value, present, err := WorkspaceYamlValueStrictWithLocal("", "dolt.shared-server"); err != nil || present || value != "" { + t.Fatalf("= (%q, %v, %v), want empty/false/nil", value, present, err) + } + }) +} + func TestWorkspaceYamlValueStrictRejectsNullAndWrongParent(t *testing.T) { for _, body := range []string{"dolt:\n shared-server: null\n", "dolt: false\n", "dolt: []\n"} { t.Run(strings.ReplaceAll(body, "\n", "_"), func(t *testing.T) {