* fix(storage): an env-pointed server is shared, not owned — close the migrate-gate bypass
A workspace pointed at an externally-managed dolt sql-server purely through
BEADS_DOLT_SERVER_PORT was classified ServerModeOwned, so the shared-store
consent gate (#5920/#6048/#6055) never ran. bd then migrated that server's
database in place — no prompt, no warning, exit 0 — and every co-resident bd
still on the older schema was hard-refused with "schema version mismatch". The
gate added specifically to stop this had a hole straight through it.
The asymmetry. Two resolvers disagree about what names the server:
- the CONNECTION path takes the env var first. configfile.GetDoltServerPort
reads BEADS_DOLT_SERVER_PORT, then BEADS_DOLT_PORT, then metadata.json;
doltserver's precedence chain behind ApplyResolvedServerPort does the same
and records which step won as Config.ServerPortSource.
- the OWNERSHIP path ignores it. ResolveServerMode's step 4 reads
dolt_server_port from metadata.json ONLY.
So a workspace whose metadata.json says dolt_mode=server with no port, and
whose port arrives entirely from the environment, connects to a server it never
started while believing it owns a private one. Reproduced on a scratch server
shaped exactly like the reported deployment: identical topology, identical
database — with the port in metadata.json the gate refuses (exit 1); with the
same port supplied only by the env var, bd migrated v65 -> v66 silently
(exit 0). The only difference is where the port came from.
The fix. sharedServerDatabase stops borrowing the lifecycle resolver's answer
for a question it was not written to answer. ResolveServerMode's contract is
"may bd manage this server's lifecycle" — deliberately conservative about env
ports, because flipping it to External also disables auto-start. The gate needs
a stricter question, asked directly: did THIS workspace's own bookkeeping
produce the endpoint we connected to? Config.ServerPortSource already carries
the answer, stamped by applyConfigDefaults on every open, library path
included.
Workspace bookkeeping (owned): the port file doltserver.Start wrote, the dolt
config.yaml inside the workspace's own dolt dir, an in-process caller assertion
(be-wf9a.1), or no port resolved at all. Everything else — the env vars, beads'
config.yaml, metadata.json, the two documented fallback defaults — is an
operator writing down where a server already lives, and is shared. One escape
hatch: .beads/dolt-server.port naming the same port bd is connected on is bd's
own proof that it started that server, even when the env read won the chain.
Two neighbouring asymmetries in the same family, both found by auditing every
signal the connection path honors against what ownership resolution reads:
- a gateway credential command mints a token for an authenticating server;
bd's auto-start brings up a plain local root@localhost dolt and never
issues tokens, so cfg.Gateway now asserts shared on its own rather than
depending on port provenance.
- applyResolvedConfig never copied dolt_server_socket out of metadata.json,
although cmd/bd's resolveDoltServerConnection — which its own doc says it
mirrors — always has. The library open path and the CLI therefore chose
different transports for the same workspace, and it blinded the predicate's
socket arm, which exists precisely because auto-start only ever creates a
TCP listener. A configured-but-unreachable socket stays harmless:
newServerMode normalizes it back to TCP via ResolveSocketTransport.
Not changed: ResolveServerMode itself, and so auto-start. Making an env-named
port imply External would also suppress auto-start, which is arguably right —
it is the shadow-database hazard External mode exists for — but it is a
behavior change on a path the reported blocker does not need, and this is an
RC. Left as a documented follow-up.
Why #6088's narrowing survives. Every check added here is an early `return
true`, so it can only ADD a shared classification, never remove one: any
topology gated before still returns at the same place. The auto-started
single-workspace server keeps every signal that made it owned — local host, no
TLS, no socket, not shared-server mode, ServerModeOwned — and now also resolves
its port from its own port file. TestServerModeLenientOpenDirtyTableGate, the
#4566/#5781 recovery whose whole shape is refuse-then-commit-then-migrate on
exactly such a server, passes unmodified; end to end, bd init --server on a
fresh workspace rolled back to v65 still migrates to v66 on the next write,
exit 0, no gate.
Tests: TestSharedServerDatabaseEndpointProvenance walks all nine port sources
plus the port-file hatch in both directions, the disagreeing-port-file case,
and the reported shape end to end (including an assertion that
ResolveServerMode still says Owned there — if that ever changes, the test has
stopped covering the bug). TestSharedServerDatabase gains the gateway case.
Reverting the new arm turns seven of them red, including the reported shape.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(storage): prove server ownership instead of inferring it from the endpoint
Review of the first cut found the bypass still open through two more doors, and
two false positives the "can only ADD shared" argument did not cover. All four
have the same cause: the predicate inferred ownership from how the endpoint was
NAMED, and every such inference is a separate silent bypass.
Still-open bypasses:
- PortSourceCallerExplicit was treated as workspace bookkeeping, but
applyConfigDefaults stamps it onto ANY caller-preset port, and
`bd init --server-port` is the documented way to attach a workspace to an
existing server. So `bd init --server-port 3307` against a shared server
still resolved owned and migrated it in place, exit 0 — the reported
failure through a different door. Same for any library embedder building
dolt.Config{ServerHost: "127.0.0.1", ServerPort: 3307}.
- the port-file escape hatch had no liveness check. A port file records that
bd once bound a port, not that the process answering it now is bd's. A
workspace whose own server died, with an external server later occupying
that port, laundered the foreign endpoint into ownership. 3307 is both the
common external port and bd's own default, so the collision is likely
rather than exotic.
False positives:
- PortSourceConfigYaml was classified shared, which hard-refuses a
single-user workspace that pinned dolt.port in config.yaml with bd
auto-starting its own server on it — the configuration bd's own auto-start
warning tells users to adopt ("To pin a port: set dolt.port in
.beads/config.yaml"). Newly-gated topologies are exactly where the #6088
class of regression lives, and the additive argument says nothing about
them.
- reading dolt_server_socket in applyResolvedConfig disabled auto-start on
the library path whenever the server was cold: ResolveSocketTransport
returns the socket unchanged when neither socket nor TCP answers, and
serverOpenCanAutoStart requires an empty socket. The claim that a
non-connectable socket stays harmless held only while TCP was up — never in
the case where auto-start matters.
So ownership is no longer inferred at all. doltserver.ManagesLiveServerOnPort
PROVES it from the state files bd writes when it starts a server: a live PID
beside a port file naming the port this store is actually connected on.
Everything else is shared. Enumerating the ways an endpoint can be foreign is
unbounded — an env var, a config.yaml pin, a flag, a hand-built Config, a stale
file all produce the same local TCP endpoint — while there is exactly one way
to be sure, and it is two cheap file reads and a signal 0.
That single rule closes both bypasses and removes both false positives: the
config.yaml pin and the env port bd itself bound both carry the proof, because
bd started those servers.
ManagesLiveServerOnPort deliberately stops short of IsRunning. IsRunning
repairs as it goes (deletes stale state, can stop an orphan), and a predicate
classifying a connection must not mutate workspace state as a side effect. It
also omits IsRunning's isDoltProcess() command-name check, which shells out to
`ps` — measured at 765ms on a loaded machine, and this runs on every writable
open. The residual gap needs a recycled PID AND a stale port file AND a foreign
server on that same port; strictly narrower than the gap left by trusting the
port file alone, and documented at the definition.
Also dropped: the cfg.Gateway arm, which was unreachable — newServerMode only
calls initSchema when !cfg.Gateway, so a gateway config never reaches this
predicate. It was listed as a closed asymmetry in the PR body and closed
nothing.
The socket read in applyResolvedConfig is reverted entirely: besides breaking
cold auto-start it never armed the predicate it was written for, because
newServerMode overwrites cfg.ServerSocket with the normalized transport before
initSchema runs. open.go is byte-identical to the base branch again. The
socket topology is covered by the ownership proof instead, without touching the
transport.
Tests: TestManagesLiveServerOnPort covers the proof directly, including that it
does not mutate the state it reads. TestSharedServerDatabaseOwnershipProof
replaces the port-source matrix — three of whose nine rows asserted
source/host combinations that cannot coexist in production — with the real
topologies, both bypasses among them. TestSharedServerDatabaseThroughResolvedConfig
drives applyResolvedConfig + applyConfigDefaults so the link between
BEADS_DOLT_SERVER_PORT and the classification is pinned by behavior rather than
by a hand-stamped field. The duplicated workspace/env helpers are hoisted to
package level, one env list.
Verified end to end on a scratch server for both new bypasses: the previous
build migrated v65 -> v66 exit 0 through each door; this one refuses both. The
#6088 auto-start case still migrates on open, exit 0.
Refs #6123 for the four-resolver reconciliation this defers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(doltserver): name the adopted-server residual gap in the ownership proof
Maintainer review fixup for PR #6118.
`ManagesLiveServerOnPort`'s doc stated the invariant as "a workspace whose
server bd brought up", but `Start()` writes the same PID and port files for a
server it *adopts*: `reclaimPort` returns the PID of a dolt process whose CWD
is this workspace's dolt dir, and `Start` records that foreign PID before
reporting `Running`. An operator running `dolt sql-server` over `.beads/dolt`
under systemd and then typing `bd dolt start` mints the proof for a server bd
does not manage — a second, wider residual gap than the recycled-PID one the
doc named, and one an auditor was told did not exist.
Comment-only. Soften the stated invariant to "started — or adopted — for
itself", document both residual gaps explicitly (adopted-server is unix-only
in practice: `isProcessInDir` is false on Windows), and point both at #6123,
which owns reconciling the four resolvers.
Deliberately not changed: no `isDoltProcess` check in the helper — it would
close only the recycled-PID gap, since `reclaimPort` already requires
`isDoltProcess` before adopting, at hundreds of ms on every writable open. And
no change to `Start`'s adopt branch: recording adoption distinctly is the right
fix but is a behavior change on the auto-start path, not an RC gate-hardening
patch. Both are tracked in #6123.
Neither gap is a regression: at base both topologies resolved `ServerModeOwned`
and were ungated, and `store.go` is +35/-0 with the new check as an early
return ahead of base's unchanged final return, so this PR can only add
refusals.
* docs(doltserver): the adopted-server gap is incidental and permanent
Amend the ManagesLiveServerOnPort ownership-proof comment so the residual
adopted-server gap is described the way it actually behaves: typing
`bd dolt start` is not required (a 500 ms dial timeout reaches the same
adopt branch with no operator action), the two state files the adopt
branch writes persist, and `dolt.auto-start: false` /
BEADS_DOLT_AUTO_START=0 is the one setting that closes the incidental
route while #6123 is open — the deliberate `bd dolt start` route is not
affected by it.
Also rename the positive proof subtest to say what it pins: any live PID
beside an agreeing port file is accepted, dolt process or not. The PID it
writes is the test binary's own, which parseDoltProcessPIDs would never
match, so the subtest is the live demonstration of the liveness-only
contract and is expected to go red when #6123 tightens the predicate.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
121 lines
4.3 KiB
Go
121 lines
4.3 KiB
Go
package doltserver
|
|
|
|
import (
|
|
"os"
|
|
"path/filepath"
|
|
"strconv"
|
|
"testing"
|
|
)
|
|
|
|
// TestManagesLiveServerOnPort pins the ownership proof: both state files, a
|
|
// live PID, and a port file agreeing with the port the caller is connected on.
|
|
//
|
|
// Every "false" case below is a silent in-place migration of somebody else's
|
|
// database if it ever flips, so each one names the topology it stands for.
|
|
func TestManagesLiveServerOnPort(t *testing.T) {
|
|
const port = 3307
|
|
|
|
// dead is a PID no live process answers to: above every default pid_max.
|
|
const dead = 2147483646
|
|
|
|
newDir := func(t *testing.T) string {
|
|
t.Helper()
|
|
dir := filepath.Join(t.TempDir(), ".beads")
|
|
if err := os.MkdirAll(dir, 0o700); err != nil {
|
|
t.Fatalf("mkdir: %v", err)
|
|
}
|
|
return dir
|
|
}
|
|
write := func(t *testing.T, dir, name string, v int) {
|
|
t.Helper()
|
|
if err := os.WriteFile(filepath.Join(dir, name), []byte(strconv.Itoa(v)), 0o600); err != nil {
|
|
t.Fatalf("write %s: %v", name, err)
|
|
}
|
|
}
|
|
|
|
// The PID recorded here is this test binary's, which is emphatically not a
|
|
// dolt sql-server: parseDoltProcessPIDs requires "dolt" followed by
|
|
// "sql-server" in the command line, so an identity check would never return
|
|
// it. That is the point — this subtest pins the predicate's liveness-only
|
|
// contract and is the live demonstration that any live PID beside an
|
|
// agreeing port file is accepted today, dolt or not. #6123 is expected to
|
|
// tighten that (a distinct adoption marker, or an identity check); when it
|
|
// does, this subtest goes red, and that is the signal to re-derive #6088's
|
|
// guarantee from whatever the new proof is, not to relax the assertion.
|
|
t.Run("any live pid with an agreeing port file is proof", func(t *testing.T) {
|
|
dir := newDir(t)
|
|
write(t, dir, PortFileName, port)
|
|
write(t, dir, PIDFileName, os.Getpid())
|
|
if !ManagesLiveServerOnPort(dir, port) {
|
|
t.Fatal("a workspace with a live recorded server on this port owns it (#6088 must keep migrating)")
|
|
}
|
|
})
|
|
|
|
t.Run("dead pid is not proof", func(t *testing.T) {
|
|
// The workspace once ran its own server on this port; it died, and an
|
|
// externally managed server now answers there. 3307 is both the common
|
|
// external port and bd's own default, so the collision is likely.
|
|
dir := newDir(t)
|
|
write(t, dir, PortFileName, port)
|
|
write(t, dir, PIDFileName, dead)
|
|
if ManagesLiveServerOnPort(dir, port) {
|
|
t.Fatal("a port file records that bd once bound a port, not that bd answers on it now")
|
|
}
|
|
})
|
|
|
|
t.Run("port file naming a different port is not proof", func(t *testing.T) {
|
|
dir := newDir(t)
|
|
write(t, dir, PortFileName, 40083)
|
|
write(t, dir, PIDFileName, os.Getpid())
|
|
if ManagesLiveServerOnPort(dir, port) {
|
|
t.Fatal("bd's server is on another port; this endpoint is somebody else's")
|
|
}
|
|
})
|
|
|
|
t.Run("missing state files are not proof", func(t *testing.T) {
|
|
dir := newDir(t)
|
|
if ManagesLiveServerOnPort(dir, port) {
|
|
t.Fatal("bd never started a server here")
|
|
}
|
|
write(t, dir, PortFileName, port)
|
|
if ManagesLiveServerOnPort(dir, port) {
|
|
t.Fatal("a port file with no pid file proves nothing about liveness")
|
|
}
|
|
})
|
|
|
|
t.Run("garbage pid file is not proof", func(t *testing.T) {
|
|
dir := newDir(t)
|
|
write(t, dir, PortFileName, port)
|
|
if err := os.WriteFile(filepath.Join(dir, PIDFileName), []byte("not-a-pid"), 0o600); err != nil {
|
|
t.Fatalf("write pid: %v", err)
|
|
}
|
|
if ManagesLiveServerOnPort(dir, port) {
|
|
t.Fatal("an unparseable pid file must fail closed")
|
|
}
|
|
})
|
|
|
|
t.Run("degenerate inputs fail closed", func(t *testing.T) {
|
|
if ManagesLiveServerOnPort("", port) {
|
|
t.Fatal("no workspace, no proof")
|
|
}
|
|
if ManagesLiveServerOnPort(newDir(t), 0) {
|
|
t.Fatal("port 0 means unresolved; there is no endpoint to own")
|
|
}
|
|
})
|
|
|
|
// Read-only: unlike IsRunning, classifying a connection must never repair
|
|
// or delete a workspace's server state as a side effect.
|
|
t.Run("does not mutate workspace state", func(t *testing.T) {
|
|
dir := newDir(t)
|
|
write(t, dir, PortFileName, port)
|
|
write(t, dir, PIDFileName, dead)
|
|
if ManagesLiveServerOnPort(dir, port) {
|
|
t.Fatal("dead pid must not be proof")
|
|
}
|
|
for _, name := range []string{PortFileName, PIDFileName} {
|
|
if _, err := os.Stat(filepath.Join(dir, name)); err != nil {
|
|
t.Fatalf("%s was removed: %v — the predicate must not repair state it merely reads", name, err)
|
|
}
|
|
}
|
|
})
|
|
}
|