* feat(dbproxy): add procid birth-token package and pidfile schema v2 fields Foundation for verified proxy/backend adoption and safe stale-process cleanup (mybd-psxg.5, PR-A stage 1): internal/procid provides versioned process-birth tokens (Linux boot_id+starttime, Windows creation FILETIME, Darwin p_starttime) with TOCTOU-safe signaling handles (pidfd on Linux, re-verified process handle on Windows); pidfile gains additive schema-v2 fields (schema, kind, birth, root_id, control_port) and sentinel-error validation. No behavior changes; existing v1 pidfiles still parse. The procid Token/Verify shape intentionally matches the API requested in gastownhall/beads#4513 so internal/doltserver can adopt it later. Agent-Signature: codex-gpt-5.6-terra-medium on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi * feat(dbproxy): publish verifiable proxy identity (secret, control handshake, v2 pidfiles) Stage 2 of mybd-psxg.5 (PR-A): the managed proxy now proves who it is. New leaf package internal/storage/dbproxy/identity provides the workspace RootID (sha256 of the resolved root path), a rotated 0600 proxy.secret, and the IDENT control protocol (client Identify + IdentReply). The proxy opens a loopback control listener that answers only to the workspace secret, and both pidfile writers publish schema-v2 records (kind, birth token, root_id, control_port) captured via procid. A proxy or backend that cannot prove its own identity fails startup instead of publishing. Adoption, cleanup, and kill behavior are unchanged in this stage; the consumers land next so each step stays reviewable. Agent-Signature: codex-gpt-5.6-terra-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi * fix(dbproxy): verified adoption, fail-closed cleanup, and handle-safe kills Stage 3 of mybd-psxg.5 (PR-A), the security core. Endpoint discovery now classifies records with typed statuses instead of adopting any listener that answers a TCP dial: adoption requires a schema-v2 pidfile, a live birth-token match, the authenticated IDENT handshake, and root/pid/birth/ port cross-checks. Legacy and malformed records fail closed (quarantined under proxy.lock, never adopted, never killed); quarantines are timestamped renames with a 30-day sweep, so no forensic record is ever unlinked. Every kill site (spawn-path backend cleanup and bd dolt stop's Shutdown) signals only through a procid.Handle opened against the recorded birth token, refusing unverifiable live processes with an actionable error. Orphaned Dolt backends left by a SIGKILLed proxy supervisor are now detected regardless of child-lock state and reaped only after verification. A proxy.spawn marker plus a persistent stop epoch closes the window where bd dolt stop could report success while a fork already in flight published a listener afterwards. Agent-Signature: codex-gpt-5.6-sol-xhigh on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi * fix(dbproxy): harden identity fixes from cross-vendor adversarial review Fixes every accepted finding from paired independent reviews of the first three stages (reports in the coordination repo, 2026-07-24): - darwin: classify wrapped ESRCH and the zero-length/EIO missing-pid sysctl result as process-gone; stale pidfiles self-heal again. - fallback signaling (darwin, non-pidfd linux): a target that exits after a fatal signal is a successful kill, not an error. - Shutdown verifies root_id on the proxy record too (copied-workspace kill hazard); positional bool pair replaced by killRecordChecks. - birth-token mismatch now classifies as provably-dead everywhere; dead legacy/malformed records quarantine and self-heal; refusal messages name the record file and manual remedy. - discovery-time verify/secret/root errors map to a non-adopting status routed through quarantine-under-lock instead of wedging before the lock probe. - boot_id read failures are environmental errors, never process-gone (closes the one fail-open path found in review). - ForceStopUnverified: executable-identity-checked operator escape for pre-upgrade/unverifiable records, quarantines after, reports truthfully; CLI wiring intentionally deferred. - handshake replies are HMAC'd against a client nonce; constant-time secret compare; bounded accept failures and handshake concurrency; migration purge covers proxy.secret and quarantine files; spawn marker cleared via defer; unix-only constraints on sleep tests. Agent-Signature: codex-gpt-5.6-sol-xhigh on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi * feat(dbproxy): wire bd dolt stop --force and prove the lifecycle lane end to end Stage 4 of mybd-psxg.5 (PR-A). bd dolt stop gains --force, routed through proxy.ForceStopUnverified only after the backend is confirmed stopped: it verifies executable identity before signaling, quarantines the record afterward, refuses foreign executables and verifiable v2 records, and reports text/JSON truthfully. The managed-local lane (BEADS_TEST_PROXIED_LOCAL=1) now exercises the whole security model against a real dolt sql-server: foreign-listener rejection, reused-PID no-kill, legacy fail-closed (free lock, held lock refusing a foreign executable, held lock force-stopping a genuine bd), orphan-backend reaping after a SIGKILLed supervisor, 5-process cold-start convergence, and stop/start interleaving. Windows/macOS lane equivalents are defined in the lane header, implementation staged later. Agent-Signature: codex-gpt-5.6-sol-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi * fix(dbproxy): close cross-vendor review findings — handle lifetimes, pidfd races, force-stop scoping - B1: cleanupOrphanBackend closes the verified handle before waiting for exit, mirroring shutdown.go; an open Windows handle kept the dead PID allocated and made every orphan-backend recovery time out. - B2: Windows procid checks GetExitCodeProcess after OpenProcess, so a terminated process kept resolvable by a third-party handle no longer verifies; "Verify == true implies running" now holds on all platforms. - B3: Linux procid.Open opens the pidfd BEFORE token verification; the pidfd pins the PID number, closing the reuse window where a later Signal could kill an unrelated replacement process. - B4: PidfdSendSignal ESRCH on a fatal signal is treated as successful termination, matching the fallback path (same for Windows terminate). - B5: ForceStopUnverified now covers the backend (proxy-child) record with the same safety checks, and CanForceStopUnverified accepts refusals where the backend is also (or only) unverifiable — the paired-legacy pre-v2 topology was locked out of its own advertised recovery. bd dolt stop --force reports the backend record too. - B6a: an unverified live PID is only signaled when its command line ties it to this workspace (Linux /proc cmdline, Darwin ps args); Windows cannot read another process's command line and now refuses, which beats basename-only killing. Clear errors name the manual remedy. - B6b: one stable OS handle (Linux pidfd, Windows process handle) spans inspection and signaling, removing the inspect/kill TOCTOU. - S1: control-listener accept errors are non-fatal — log and retry with capped exponential backoff instead of tearing down a healthy proxy. - S2: requireUnverifiableRecord surfaces RootID resolution errors instead of failing open into the destructive branch. - S3: killSpawnedChild falls back to cmd.Process.Kill for our own un-reaped child when no verified handle exists, instead of leaking it. - S4: adoption handshake accepts reply schema v2 or newer, matching pidfile.ValidateV2 forward-compat policy. - S5: errors.Is for ERROR_INVALID_PARAMETER. - Tests: pidfd ESRCH-after-exit and Open-after-reap classification; paired-legacy force recovery end to end; foreign-workspace bd process refused; Windows Verify-after-terminate (compile-checked); schema-v3 handshake adoption; accept-loop survival under persistent errors. Agent-Signature: claude-fable-5-medium on behalf of maphew * fix(dbproxy): race-free proxy port allocation and managed loopback listener policy Companion to the identity PR (stacks on it). The parent no longer pre-picks the proxy data port: port 0 flows to the child, which binds 127.0.0.1:0 and publishes the OS-assigned port through its verified pidfile, eliminating the bind-close-launch race; explicitly configured ports keep fixed-port failure semantics. PickFreePort survives only for the dolt-config path, annotated for the follow-up RFC on backend port collisions. Managed dolt configs are now validated by a single numeric-loopback listener policy at init and at runtime resolution, covering all config paths (generated, BEADS_PROXIED_SERVER_CONFIG, sidecar, pre-marker, hand-edited marker, interpolation fallback). Hostnames including localhost are rejected with remediation text naming the --proxied-server-external-* flags; an omitted host is accepted because dolt servercfg defaults it. There is deliberately no bypass env: managed mode assumes loopback for its root/empty-password account. External proxied topology is exempt by construction. Agent-Signature: codex-gpt-5.6-sol-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi * fix(dbproxy): whole-config managed listener policy, epoch-fenced startup, diagnosis fixes Review findings from two independent passes over the listener-policy commit (gastownhall/beads#5024), rebased onto the rewritten #5013 head: - F1: the managed policy now covers the whole sql-server YAML, not just listener.host. validateManagedListenerHost is renamed validateManagedServerConfigPolicy and additionally rejects listener.socket (incl. the empty-string form, which maps to Dolt's default socket path), remotesapi.port, and any cluster block — all of which open non-loopback listeners for the root/empty-password child. - F2: managed mode requires an explicit numeric loopback listener.host. An omitted host is no longer trusted: servercfg resolves it to "localhost", which CheckForUnixSocket also treats as license to open the shared-namespace default unix socket. This removes the dead net.ParseIP("localhost") branch and the omitted/explicit asymmetry. - F3: one interpolation-aware policy check instead of five scattered raw-bytes ones. ensureProxiedServerConfig now delegates resolution to resolveOrCreateProxiedServerConfig and runs a single validateProxiedServerConfig (YamlConfigFromFile, ${VAR}-expanding) on the resolved path, so a marker config with an interpolated host is judged on what it expands to. - F4: the proxy child re-checks the stop epoch immediately before publishing proxy.pid and aborts if it advanced (the spawn marker is cleared at lock acquisition, so a `bd dolt stop` racing a slow backend start otherwise has no record to wait on). The parent's observed epoch is threaded through a new db-proxy-child --stop-epoch flag into ProxyOpts.StopEpoch. - F5a/b: child-exit diagnosis distinguishes a LockHeldExitCode spawn race from real failures, points at the child's log path otherwise, and timeout messages say "its OS-assigned port" instead of "port 0". - F5c/d: validateManagedProxiedServerConfigAtInit now honors --proxied-server-root-path (the sidecar is not persisted yet at validation time) and drops the redundant explicit-config re-validation already done at flag parse. - F5e: --proxied-server-config-path flag help states the hard managed constraint (docs/cli-reference regenerates from the release pin, so only the source help string changes here). - F5f: the listener-policy lifecycle test documents its accepted PickFreePort EADDRINUSE flake window. Tests: whole-config policy table (socket/remotesapi/cluster/omitted/ localhost/loopback forms), marker ${VAR} host interpolation accept and reject, root-path-flag validation at init (unit + gated integration), and a deterministic stop-epoch-advance-during-startup abort test. Agent-Signature: claude-fable-5-medium on behalf of maphew --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
243 lines
9 KiB
Go
243 lines
9 KiB
Go
//go:build cgo && unix
|
|
|
|
package main
|
|
|
|
// Helpers for the MANAGED-LOCAL proxied-server lane: bd launches its own
|
|
// loopback proxy (a detached `bd db-proxy-child` process) and that proxy
|
|
// launches a locally installed `dolt sql-server`. No external host/port and
|
|
// no testcontainer are involved, which is the production local-first
|
|
// topology that the external-harness shards (BEADS_TEST_PROXIED_SERVER)
|
|
// deliberately do not cover.
|
|
//
|
|
// The lane is gated by its own environment variable so CI can run the two
|
|
// proxied lanes independently. These helpers are written to be reusable by
|
|
// lifecycle/failure-injection tests (stale artifacts, process identity,
|
|
// port safety) beyond the initial smoke test.
|
|
|
|
import (
|
|
"context"
|
|
"database/sql"
|
|
"fmt"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"syscall"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/steveyegge/beads/internal/configfile"
|
|
"github.com/steveyegge/beads/internal/storage/dbproxy/pidfile"
|
|
"github.com/steveyegge/beads/internal/storage/dbproxy/proxy"
|
|
"github.com/steveyegge/beads/internal/storage/dbproxy/server"
|
|
)
|
|
|
|
const managedLocalProxiedEnvVar = "BEADS_TEST_PROXIED_LOCAL"
|
|
|
|
// requireManagedLocalProxiedEnv gates the managed-local proxied lane.
|
|
// Unlike requireProxiedServerEnv, a missing dolt binary FAILS the test when
|
|
// the lane is explicitly requested: this lane exists to prove that bd can
|
|
// launch and supervise a local Dolt child, so skipping on a broken
|
|
// prerequisite would report a green run without testing anything.
|
|
func requireManagedLocalProxiedEnv(t *testing.T) {
|
|
t.Helper()
|
|
if os.Getenv(managedLocalProxiedEnvVar) != "1" {
|
|
t.Skipf("set %s=1 to run managed-local proxied lifecycle tests", managedLocalProxiedEnvVar)
|
|
}
|
|
if _, err := exec.LookPath("dolt"); err != nil {
|
|
t.Fatalf("%s=1 but dolt is not in PATH; this lane must fail rather than skip: %v",
|
|
managedLocalProxiedEnvVar, err)
|
|
}
|
|
}
|
|
|
|
// bdManagedLocalInit initializes a disposable managed-local proxied project:
|
|
// `bd init --proxied-server` with NO external host/port flags, so command
|
|
// dispatch launches the loopback proxy and the proxy launches dolt.
|
|
// idleTimeout is passed through --proxied-server-idle-timeout so tests can
|
|
// observe idle shutdown without waiting out the 30s production default.
|
|
func bdManagedLocalInit(t *testing.T, bd, prefix string, idleTimeout time.Duration, extraInitArgs ...string) proxiedProject {
|
|
t.Helper()
|
|
args := append([]string{"--proxied-server-idle-timeout", idleTimeout.String()}, extraInitArgs...)
|
|
p := bdProxiedInit(t, bd, prefix, args...)
|
|
|
|
info, err := configfile.LoadProxiedServerClientInfo(p.beadsDir)
|
|
if err != nil {
|
|
t.Fatalf("LoadProxiedServerClientInfo(%s): %v", p.beadsDir, err)
|
|
}
|
|
if info == nil {
|
|
t.Fatalf("missing %s in %s after managed-local init", configfile.ProxiedServerClientInfoFileName, p.beadsDir)
|
|
}
|
|
if info.External != nil {
|
|
t.Fatalf("expected managed-local topology (no External block), got %+v", info.External)
|
|
}
|
|
if info.IdleTimeout != idleTimeout {
|
|
t.Fatalf("persisted managed-local idle timeout: got %s, want %s", info.IdleTimeout, idleTimeout)
|
|
}
|
|
if info.Port != 0 {
|
|
t.Fatalf("default managed proxy port: got %d, want 0 so the child binds an OS-assigned port", info.Port)
|
|
}
|
|
return p
|
|
}
|
|
|
|
// readManagedProxyPidFile returns the loopback proxy's pidfile (proxy.pid:
|
|
// the detached `bd db-proxy-child` process and the port the proxy listens
|
|
// on), or nil when it does not exist.
|
|
func readManagedProxyPidFile(t *testing.T, p proxiedProject) *pidfile.PidFile {
|
|
t.Helper()
|
|
pf, err := pidfile.Read(p.proxyRoot, proxy.PIDFileName)
|
|
if err != nil && !os.IsNotExist(err) {
|
|
t.Fatalf("read %s in %s: %v", proxy.PIDFileName, p.proxyRoot, err)
|
|
}
|
|
return pf
|
|
}
|
|
|
|
// readManagedBackendPidFile returns the supervised Dolt backend's pidfile
|
|
// (proxy-child.pid: the `dolt sql-server` process and its listener port),
|
|
// or nil when it does not exist.
|
|
func readManagedBackendPidFile(t *testing.T, p proxiedProject) *pidfile.PidFile {
|
|
t.Helper()
|
|
pf, err := pidfile.Read(p.proxyRoot, server.PIDFileName)
|
|
if err != nil && !os.IsNotExist(err) {
|
|
t.Fatalf("read %s in %s: %v", server.PIDFileName, p.proxyRoot, err)
|
|
}
|
|
return pf
|
|
}
|
|
|
|
// processAlive reports whether pid refers to a live process we can signal
|
|
// (kill -0). A pid that exists but is owned by another user would report
|
|
// alive=false here; in this lane every process is spawned by the test user.
|
|
func processAlive(pid int) bool {
|
|
if pid <= 0 {
|
|
return false
|
|
}
|
|
proc, err := os.FindProcess(pid)
|
|
if err != nil {
|
|
return false
|
|
}
|
|
return proc.Signal(syscall.Signal(0)) == nil
|
|
}
|
|
|
|
// managedProxiedArtifacts is a point-in-time snapshot of which lifecycle
|
|
// artifacts exist under the proxy root. Lock files legitimately persist on
|
|
// disk after a clean shutdown (the flock is what matters, not the file), so
|
|
// assertions should usually key on the pid files.
|
|
type managedProxiedArtifacts struct {
|
|
ProxyLock bool
|
|
ProxyPid bool
|
|
BackendLock bool
|
|
BackendPid bool
|
|
ConfigYAML bool
|
|
}
|
|
|
|
func snapshotManagedProxiedArtifacts(p proxiedProject) managedProxiedArtifacts {
|
|
exists := func(name string) bool {
|
|
_, err := os.Stat(filepath.Join(p.proxyRoot, name))
|
|
return err == nil
|
|
}
|
|
return managedProxiedArtifacts{
|
|
ProxyLock: exists(proxy.LockFileName),
|
|
ProxyPid: exists(proxy.PIDFileName),
|
|
BackendLock: exists(server.LockFileName),
|
|
BackendPid: exists(server.PIDFileName),
|
|
ConfigYAML: exists(proxiedServerConfigName),
|
|
}
|
|
}
|
|
|
|
// waitForManagedProxied polls cond until it reports done or timeout elapses,
|
|
// failing the test with the condition's last reported state.
|
|
func waitForManagedProxied(t *testing.T, timeout time.Duration, desc string, cond func() (bool, string)) {
|
|
t.Helper()
|
|
deadline := time.Now().Add(timeout)
|
|
last := "(condition never evaluated)"
|
|
for {
|
|
done, state := cond()
|
|
if done {
|
|
return
|
|
}
|
|
last = state
|
|
if time.Now().After(deadline) {
|
|
t.Fatalf("timed out after %s waiting for %s; last state: %s", timeout, desc, last)
|
|
}
|
|
time.Sleep(100 * time.Millisecond)
|
|
}
|
|
}
|
|
|
|
// waitForManagedProxiedShutdown waits until both pid files are gone and the
|
|
// previously observed proxy and backend processes are dead — the observable
|
|
// contract of a clean idle shutdown.
|
|
func waitForManagedProxiedShutdown(t *testing.T, p proxiedProject, proxyPID, backendPID int, timeout time.Duration) {
|
|
t.Helper()
|
|
waitForManagedProxied(t, timeout, "idle shutdown of managed proxy and dolt backend", func() (bool, string) {
|
|
arts := snapshotManagedProxiedArtifacts(p)
|
|
proxyDead := !processAlive(proxyPID)
|
|
backendDead := !processAlive(backendPID)
|
|
if !arts.ProxyPid && !arts.BackendPid && proxyDead && backendDead {
|
|
return true, ""
|
|
}
|
|
return false, fmt.Sprintf(
|
|
"proxy.pid exists=%v proxy-child.pid exists=%v proxy(pid %d) alive=%v dolt(pid %d) alive=%v",
|
|
arts.ProxyPid, arts.BackendPid, proxyPID, !proxyDead, backendPID, !backendDead)
|
|
})
|
|
}
|
|
|
|
// heldProxiedConn is a single checked-out SQL connection through the
|
|
// managed proxy. While it is held, the proxy's idle watcher sees an active
|
|
// connection and will not shut the topology down — the intended way to keep
|
|
// the process tree stable while inspecting it. Release() must close the
|
|
// entire pool, not just return the conn to it: a pooled-but-idle connection
|
|
// keeps its TCP session to the proxy open, which still counts as active and
|
|
// blocks idle shutdown indefinitely.
|
|
type heldProxiedConn struct {
|
|
Conn *sql.Conn
|
|
db *sql.DB
|
|
}
|
|
|
|
// Release drops every TCP connection this holder has to the proxy, allowing
|
|
// the idle countdown to begin. Safe to call more than once.
|
|
func (h *heldProxiedConn) Release() {
|
|
if h.Conn != nil {
|
|
_ = h.Conn.Close()
|
|
h.Conn = nil
|
|
}
|
|
if h.db != nil {
|
|
_ = h.db.Close()
|
|
h.db = nil
|
|
}
|
|
}
|
|
|
|
// openHeldManagedProxiedConn ensures the managed proxy is running (issuing a
|
|
// cheap bd command to start or restart it when needed), then returns a held
|
|
// connection through it. Callers must Release() it to let the topology idle
|
|
// out; a t.Cleanup Release is registered as a backstop.
|
|
func openHeldManagedProxiedConn(t *testing.T, bd string, p proxiedProject) *heldProxiedConn {
|
|
t.Helper()
|
|
|
|
var held *heldProxiedConn
|
|
waitForManagedProxied(t, 60*time.Second, "held connection through managed proxy", func() (bool, string) {
|
|
pf := readManagedProxyPidFile(t, p)
|
|
if pf == nil || !processAlive(pf.Pid) {
|
|
// Proxy not up (yet, or idled out between commands): any data
|
|
// command restarts it transparently.
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "list", "--json"); err != nil {
|
|
return false, fmt.Sprintf("bd list to (re)start proxy failed: %v\n%s", err, out)
|
|
}
|
|
return false, "proxy pidfile absent; issued bd list to start it"
|
|
}
|
|
|
|
dsn := fmt.Sprintf("root:@tcp(127.0.0.1:%d)/%s?parseTime=true", pf.Port, p.database)
|
|
db, err := sql.Open("mysql", dsn)
|
|
if err != nil {
|
|
return false, fmt.Sprintf("sql.Open %s: %v", dsn, err)
|
|
}
|
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
|
c, err := db.Conn(ctx)
|
|
cancel()
|
|
if err != nil {
|
|
_ = db.Close()
|
|
return false, fmt.Sprintf("checkout conn via %s: %v", dsn, err)
|
|
}
|
|
held = &heldProxiedConn{Conn: c, db: db}
|
|
t.Cleanup(held.Release)
|
|
return true, ""
|
|
})
|
|
return held
|
|
}
|