* fix(dbproxy): prove the managed dolt owns its port; move a taken Beads-chosen port
The managed-local backend's dolt port comes from a bind-and-close
(proxy.PickFreePort) when bd writes the generated config. The dolt
sql-server binds it only later, after the proxy child has run dolt
config and init and booted the server. If another process takes the port
in that gap, dolt logs "Port N already in use." and exits. But
DoltServer.waitReady only dialed the port, so it accepted the other
process's listener as ready. The proxy then published proxy.pid, its
backend health check saw dolt gone, and it shut down. bd init's first
ping got "unexpected EOF" and every retry after that got "connection
refused". Under parallel tests this hit any test that starts its own
managed backend.
Readiness now requires proof that the process Start launched owns the
port:
- DoltServer sends the child's stdout/stderr through a watcher that
forwards to the server log. A dial counts only after dolt's own
"Server ready. Accepting connections." line has appeared on that
output. That line is info level, and every generated config uses
info. A config with a quieter log_level has no such signal and keeps
the dial-only check.
- A child that exits first is reported as the new server.ErrPortInUse
when its output said its port was taken.
Recovery: DoltServer.SetPortRepicker lets Start move the config to a
fresh port, re-read it, and relaunch, up to 5 ports. dolt config/init
run only on the first attempt. bd's repicker,
repickManagedProxiedServerPort, needs to tell a port Beads chose from
one an operator set, so generated configs now record the chosen port
on a "# beads-chosen-listener-port: N" line after the managed marker.
The repicker moves the port only when the file is Beads-generated,
carries that record, and its listener.port is still the recorded port.
Otherwise the port is treated as pinned and Start fails with
ErrPortInUse. That includes configs generated before this change. It
rewrites only listener.port and the record, through a yaml.Node so the
rest of the file is kept, and writes the file atomically.
db-proxy-child installs it for the local backend.
The proxy's waitForServerReady also re-checks Running after its dial.
Start has already proved ownership, and a running dolt holds its
listener, so only a backend that exited between the two checks could
let a dial reach another process.
Tests:
- server: a held port is not accepted as ready (ErrPortInUse; this
test fails on main), a repicker brings the server up on a new port
that the pid record and DSN both use, and a declining repicker ends
Start with ErrPortInUse.
- cmd/bd: the generated config records its port, a Beads-chosen port
moves while hand edits survive, and a hand-set port, a config
without the record, an operator config, and a stale in-use port are
all refused with the file left untouched.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(dbproxy): move a taken managed port via a runtime config, never config.yaml
This reworks cd00c5acb after review. That commit recovered from a taken
Beads-chosen port by rewriting the managed config.yaml. That broke the
file's own contract ("Never rewritten afterward; hand edits are
preserved", #4986), and #5024 had deferred that contract change to an
RFC.
dolt sql-server has no command-line way around the file: with --config,
-P/--port is silently ignored (checked against dolt 2.1.8 and 2.2.0, in
both argument orders). So on ErrPortInUse the proxy child now writes
proxy-child.runtime-config.yaml (server.RuntimeConfigFileName) in the
server root. That file is the operator config with only listener.port
moved to a free port. The child relaunches dolt from it and records the
effective port where runtime readers already look: the backend pid
record and the DoltServer's dial/DSN config. The runtime config is
removed when the server stops, when Start fails, and by the next Start.
config.yaml is never written.
- server.PortRepicker becomes server.PortConflictPolicy. It only decides
whether the port may move; the server does the move.
- bd's policy (managedPortConflictPolicy) allows a move when the file
carries the managed marker and either:
- has the port record and listener.port still equals it, or
- is byte-identical to the config bd generated before the record
existed, so existing unedited workspaces recover too.
- Everything else is pinned: operator configs, hand-set ports,
hand-edited legacy files, and configs with ${...} placeholders (a
runtime copy would put the expanded values on disk).
- The record line now says plainly that Beads runs on a free port
without editing the file, and that deleting the line or changing
listener.port pins the port. The record parser now requires the
digits to end at a space or line end and to be in 1..65535.
- ErrPortInUse messages say what to do: free port P, set listener.port
in <path> to a free port, or (for a generated file) delete it so
Beads regenerates it. The give-up message names the last port tried.
- A dolt that never logs the ready line is still not accepted. A
grace-period fallback would let a foreign listener through while dolt
is still starting. The timeout error now names the missing line, says
whether anything answered, and points at the escape hatch: at
log_level warning, readiness is dial-only.
- Port-conflict detection matches go-mysql-server's "Port host:port
already in use." and the Windows bind error, and requires the SQL port
in the kernel's bind error, so a metrics or MCP listener failure is
not taken for the SQL port.
- Spawn error paths reap with cmd.Wait(). cmd.WaitDelay is set, and a
comment notes that the backend's output is now a pipe into the proxy
child.
Tests:
- Output-watcher table tests: split, one-byte and CRLF writes, output
past the 4 KiB tail, the MCP ready line, port boundaries, the
go-mysql-server and Windows forms, another listener's bind error, the
unix-socket warning, and nil log.
- A real dolt at log_level warning starts through dial-only readiness.
- With a stand-in dolt (the test binary in fake mode, wired in
TestMain): Start gives up after maxStartPortAttempts, a missing ready
line is diagnosed, plus a positive control.
- With real dolt: a move keeps config.yaml byte-identical, runs from the
runtime config, and the runtime config is removed on Stop. A declining
policy and the no-policy remedy are covered.
- The bd policy's allow and pin cases.
- The managed-local lifecycle test's config/pid port check accepts the
runtime config.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(dbproxy): ask the port policy once, about the operator's port; copy raw config text
Re-review of the runtime-config rework found a regression. After the
first move, Start asked the PortConflictPolicy about the runtime port.
bd's policy re-reads config.yaml, whose listener.port is still the
operator's port, so it refused with "listener.port is A, not the in-use
port B". The "up to 5 ports" retry therefore never ran in production;
the fake-dolt test missed it because its policy always allowed.
Start now asks the policy once per Start, on the first conflict, about
the operator config's port. Ports Start picked itself are its own to move
again. Start also re-reads the operator config up front, so every Start
begins from that config's port.
The runtime config is now built from config.yaml's raw text
(servercfg.NewYamlConfig) instead of the env-interpolated
YamlConfigFromFile result. dolt interpolates the runtime file when it
reads it, so the old way expanded placeholders twice: "$$" escapes lost a
level and placeholder values landed on disk. The server reads the
runtime file back with interpolation for its own Dial/DSN view. Since
placeholders no longer expand on disk, bd's policy stops refusing
configs that contain them.
Tests:
- bd's real policy driving a real DoltServer with a stand-in dolt that
reports every port taken: Start makes five launches on five distinct
ports, starting with config.yaml's, then gives up. It is not refused
on the second move, and config.yaml stays unchanged.
- A frozen byte-for-byte literal of the config bd v1.3.0/v1.3.1
generates. The legacy render must equal it, and the policy must find
it eligible.
- The runtime config keeps placeholders and "$$" verbatim and is mode
0600, while the server dials the interpolated host.
- The give-up test asserts the policy is asked once, about the
operator's port.
- The missing-ready-line test timeout is raised to 30s for the slow
fake-dolt exec under race.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(dbproxy): classify a child that exits before its identity is captured
TestManagedPortConflictPolicy_ConsecutiveCollisions failed on the farm
with "capture child birth identity: procid: process is no longer
running", after 2 launches instead of 5. Its stand-in dolt reports
"Port N already in use." and exits immediately, and it exited before
Start captured its birth identity. Start treated that capture failure as
fatal, so an ErrPortInUse that Start could recover from became a hard
error. A real dolt that finds its port taken can exit just as fast, so
this race exists in the product, not only in the test.
Every spawn step before the child is handed to its supervising
goroutine (identity capture, root identity, pid record) now fails
through failSpawn. failSpawn kills and reaps the child; reaping also
drains its output into the watcher. The child's own exit then decides
the error:
- the child reported its port taken: ErrPortInUse, so the normal policy
and retry path runs;
- the child exited by itself for another reason: "exited (status N)
before startup completed";
- the child was alive and we killed it: the step's own error, as
before.
On Windows, a killed child reports an ordinary exit code, so there an
exit cannot be told from a kill. Only the port-in-use classification
applies there.
procid.Capture is reached through a package var so tests can make the
capture lose the race deterministically.
Tests, using a shell-script stand-in dolt that starts in milliseconds:
- a fast port-in-use exit that beats the capture is ErrPortInUse, and
with a policy it is retried up to the 5-port budget;
- a fast exit for another reason reports its status;
- a capture failure while the child is alive stays fatal and kills the
child;
- ten immediate port-in-use exits with the real capture are all
ErrPortInUse.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
118 lines
5 KiB
Go
118 lines
5 KiB
Go
package main
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/spf13/cobra"
|
|
|
|
"github.com/steveyegge/beads/internal/configfile"
|
|
"github.com/steveyegge/beads/internal/storage/dbproxy/proxy"
|
|
"github.com/steveyegge/beads/internal/storage/dbproxy/server"
|
|
)
|
|
|
|
var (
|
|
dbProxyChildRoot string
|
|
dbProxyChildPort int
|
|
dbProxyChildIdleTimeout time.Duration
|
|
dbProxyChildBackend string
|
|
dbProxyChildConfig string
|
|
dbProxyChildLogPath string
|
|
dbProxyChildDoltBin string
|
|
dbProxyChildDatabase string
|
|
dbProxyChildExternalHost string
|
|
dbProxyChildExternalPort int
|
|
dbProxyChildExternalSocketPath string
|
|
dbProxyChildExternalKeepAlive time.Duration
|
|
dbProxyChildStopEpoch string
|
|
)
|
|
|
|
var dbProxyChildCmd = &cobra.Command{
|
|
Use: "db-proxy-child",
|
|
Hidden: true,
|
|
Short: "Internal: run as the database proxy child process",
|
|
Long: `db-proxy-child runs the long-lived per-rootDir TCP proxy that fronts a
|
|
DatabaseServer. It is spawned by the parent bd process via fork+exec and is
|
|
not intended to be invoked directly by users.`,
|
|
|
|
PersistentPreRun: func(cmd *cobra.Command, args []string) {},
|
|
PersistentPostRun: func(cmd *cobra.Command, args []string) {},
|
|
|
|
RunE: func(cmd *cobra.Command, _ []string) error {
|
|
backend := proxy.Backend(dbProxyChildBackend)
|
|
if err := backend.Validate(); err != nil {
|
|
return err
|
|
}
|
|
|
|
external := configfile.ExternalDoltConfig{
|
|
Host: dbProxyChildExternalHost,
|
|
Port: dbProxyChildExternalPort,
|
|
Socket: dbProxyChildExternalSocketPath,
|
|
KeepAlivePeriod: dbProxyChildExternalKeepAlive,
|
|
}
|
|
|
|
srv, err := newDatabaseServer(backend, dbProxyChildRoot, dbProxyChildConfig, dbProxyChildLogPath, dbProxyChildDoltBin, dbProxyChildDatabase, external)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
defer func() { _ = srv.Stop(context.Background()) }()
|
|
|
|
p := proxy.NewProxyServer(proxy.ProxyOpts{
|
|
RootDir: dbProxyChildRoot,
|
|
Port: dbProxyChildPort,
|
|
IdleTimeout: dbProxyChildIdleTimeout,
|
|
Server: srv,
|
|
StopEpoch: dbProxyChildStopEpoch,
|
|
})
|
|
if err := p.ListenAndServe(cmd.Context()); err != nil {
|
|
if errors.Is(err, proxy.ErrLockHeld) {
|
|
return &exitError{Code: proxy.LockHeldExitCode}
|
|
}
|
|
return err
|
|
}
|
|
return nil
|
|
},
|
|
}
|
|
|
|
func newDatabaseServer(backend proxy.Backend, rootDir, configPath, logPath, doltBin, database string, external configfile.ExternalDoltConfig) (server.DatabaseServer, error) {
|
|
switch backend {
|
|
case proxy.BackendLocalServer:
|
|
s, err := server.NewDoltServer(doltBin, rootDir, configPath, logPath, 0, database)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
// A Beads-generated config's port was picked before dolt binds it;
|
|
// if another process got there first, run on a free port instead.
|
|
s.SetPortConflictPolicy(managedPortConflictPolicy)
|
|
return s, nil
|
|
case proxy.BackendExternal:
|
|
return server.NewExternalDoltServer(external)
|
|
case proxy.BackendLocalSharedServer:
|
|
return nil, fmt.Errorf("backend %q: not yet implemented", backend)
|
|
}
|
|
return nil, fmt.Errorf("unknown backend %q", backend)
|
|
}
|
|
|
|
func init() {
|
|
dbProxyChildCmd.Flags().StringVar(&dbProxyChildRoot, "root", "", "root directory holding proxy.lock, proxy.pid, proxy.log")
|
|
dbProxyChildCmd.Flags().IntVar(&dbProxyChildPort, "port", 0, "port to listen on")
|
|
dbProxyChildCmd.Flags().DurationVar(&dbProxyChildIdleTimeout, "idle-timeout", 0, "idle timeout before shutdown (0 or negative = never shut down)")
|
|
dbProxyChildCmd.Flags().StringVar(&dbProxyChildBackend, "backend", "",
|
|
"backend kind: "+strings.Join(proxy.KnownBackendNames(), " | "))
|
|
dbProxyChildCmd.Flags().StringVar(&dbProxyChildConfig, "config", "", "path to backend server config (e.g. dolt sql-server YAML)")
|
|
dbProxyChildCmd.Flags().StringVar(&dbProxyChildLogPath, "logpath", "", "path the backend server should write its stdout/stderr to")
|
|
dbProxyChildCmd.Flags().StringVar(&dbProxyChildDoltBin, "dolt-bin", "", "path to the dolt executable")
|
|
dbProxyChildCmd.Flags().StringVar(&dbProxyChildDatabase, "database", "", "database to select when running shutdown maintenance (local-server backend)")
|
|
dbProxyChildCmd.Flags().StringVar(&dbProxyChildExternalHost, "external-host", "", "external backend: hostname or IP of the dolt sql-server")
|
|
dbProxyChildCmd.Flags().IntVar(&dbProxyChildExternalPort, "external-port", 0, "external backend: TCP port of the dolt sql-server")
|
|
dbProxyChildCmd.Flags().StringVar(&dbProxyChildExternalSocketPath, "external-socket-path", "", "external backend: absolute path to a unix domain socket (overrides host/port)")
|
|
dbProxyChildCmd.Flags().DurationVar(&dbProxyChildExternalKeepAlive, "external-keep-alive", 0, "external backend: TCP keepalive period (default 30s)")
|
|
dbProxyChildCmd.Flags().StringVar(&dbProxyChildStopEpoch, "stop-epoch", "", "stop epoch the parent observed at spawn; the proxy aborts before publishing if it advances")
|
|
_ = dbProxyChildCmd.MarkFlagRequired("root")
|
|
_ = dbProxyChildCmd.MarkFlagRequired("port")
|
|
_ = dbProxyChildCmd.MarkFlagRequired("backend")
|
|
rootCmd.AddCommand(dbProxyChildCmd)
|
|
}
|