beads/internal/procid/procid_linux_test.go
matt wilkie 21669eca60
fix(dbproxy): verified identity for the managed proxied-server lifecycle (adoption, cleanup, kills) (#5013)
* feat(dbproxy): add procid birth-token package and pidfile schema v2 fields

Foundation for verified proxy/backend adoption and safe stale-process
cleanup (mybd-psxg.5, PR-A stage 1): internal/procid provides versioned
process-birth tokens (Linux boot_id+starttime, Windows creation FILETIME,
Darwin p_starttime) with TOCTOU-safe signaling handles (pidfd on Linux,
re-verified process handle on Windows); pidfile gains additive schema-v2
fields (schema, kind, birth, root_id, control_port) and sentinel-error
validation. No behavior changes; existing v1 pidfiles still parse.

The procid Token/Verify shape intentionally matches the API requested in
gastownhall/beads#4513 so internal/doltserver can adopt it later.

Agent-Signature: codex-gpt-5.6-terra-medium on behalf of matt wilkie
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi

* feat(dbproxy): publish verifiable proxy identity (secret, control handshake, v2 pidfiles)

Stage 2 of mybd-psxg.5 (PR-A): the managed proxy now proves who it is.
New leaf package internal/storage/dbproxy/identity provides the workspace
RootID (sha256 of the resolved root path), a rotated 0600 proxy.secret,
and the IDENT control protocol (client Identify + IdentReply). The proxy
opens a loopback control listener that answers only to the workspace
secret, and both pidfile writers publish schema-v2 records (kind, birth
token, root_id, control_port) captured via procid. A proxy or backend
that cannot prove its own identity fails startup instead of publishing.

Adoption, cleanup, and kill behavior are unchanged in this stage; the
consumers land next so each step stays reviewable.

Agent-Signature: codex-gpt-5.6-terra-high on behalf of matt wilkie
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi

* fix(dbproxy): verified adoption, fail-closed cleanup, and handle-safe kills

Stage 3 of mybd-psxg.5 (PR-A), the security core. Endpoint discovery now
classifies records with typed statuses instead of adopting any listener
that answers a TCP dial: adoption requires a schema-v2 pidfile, a live
birth-token match, the authenticated IDENT handshake, and root/pid/birth/
port cross-checks. Legacy and malformed records fail closed (quarantined
under proxy.lock, never adopted, never killed); quarantines are
timestamped renames with a 30-day sweep, so no forensic record is ever
unlinked.

Every kill site (spawn-path backend cleanup and bd dolt stop's Shutdown)
signals only through a procid.Handle opened against the recorded birth
token, refusing unverifiable live processes with an actionable error.
Orphaned Dolt backends left by a SIGKILLed proxy supervisor are now
detected regardless of child-lock state and reaped only after
verification. A proxy.spawn marker plus a persistent stop epoch closes
the window where bd dolt stop could report success while a fork already
in flight published a listener afterwards.

Agent-Signature: codex-gpt-5.6-sol-xhigh on behalf of matt wilkie
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi

* fix(dbproxy): harden identity fixes from cross-vendor adversarial review

Fixes every accepted finding from paired independent reviews of the
first three stages (reports in the coordination repo, 2026-07-24):

- darwin: classify wrapped ESRCH and the zero-length/EIO missing-pid
  sysctl result as process-gone; stale pidfiles self-heal again.
- fallback signaling (darwin, non-pidfd linux): a target that exits
  after a fatal signal is a successful kill, not an error.
- Shutdown verifies root_id on the proxy record too (copied-workspace
  kill hazard); positional bool pair replaced by killRecordChecks.
- birth-token mismatch now classifies as provably-dead everywhere;
  dead legacy/malformed records quarantine and self-heal; refusal
  messages name the record file and manual remedy.
- discovery-time verify/secret/root errors map to a non-adopting
  status routed through quarantine-under-lock instead of wedging
  before the lock probe.
- boot_id read failures are environmental errors, never process-gone
  (closes the one fail-open path found in review).
- ForceStopUnverified: executable-identity-checked operator escape for
  pre-upgrade/unverifiable records, quarantines after, reports
  truthfully; CLI wiring intentionally deferred.
- handshake replies are HMAC'd against a client nonce; constant-time
  secret compare; bounded accept failures and handshake concurrency;
  migration purge covers proxy.secret and quarantine files; spawn
  marker cleared via defer; unix-only constraints on sleep tests.

Agent-Signature: codex-gpt-5.6-sol-xhigh on behalf of matt wilkie
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi

* feat(dbproxy): wire bd dolt stop --force and prove the lifecycle lane end to end

Stage 4 of mybd-psxg.5 (PR-A). bd dolt stop gains --force, routed through
proxy.ForceStopUnverified only after the backend is confirmed stopped:
it verifies executable identity before signaling, quarantines the record
afterward, refuses foreign executables and verifiable v2 records, and
reports text/JSON truthfully. The managed-local lane
(BEADS_TEST_PROXIED_LOCAL=1) now exercises the whole security model
against a real dolt sql-server: foreign-listener rejection, reused-PID
no-kill, legacy fail-closed (free lock, held lock refusing a foreign
executable, held lock force-stopping a genuine bd), orphan-backend
reaping after a SIGKILLed supervisor, 5-process cold-start convergence,
and stop/start interleaving. Windows/macOS lane equivalents are defined
in the lane header, implementation staged later.

Agent-Signature: codex-gpt-5.6-sol-high on behalf of matt wilkie
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WZHiYc3wjcdy1wrUP2JiQi

* fix(dbproxy): close cross-vendor review findings — handle lifetimes, pidfd races, force-stop scoping

- B1: cleanupOrphanBackend closes the verified handle before waiting for
  exit, mirroring shutdown.go; an open Windows handle kept the dead PID
  allocated and made every orphan-backend recovery time out.
- B2: Windows procid checks GetExitCodeProcess after OpenProcess, so a
  terminated process kept resolvable by a third-party handle no longer
  verifies; "Verify == true implies running" now holds on all platforms.
- B3: Linux procid.Open opens the pidfd BEFORE token verification; the
  pidfd pins the PID number, closing the reuse window where a later
  Signal could kill an unrelated replacement process.
- B4: PidfdSendSignal ESRCH on a fatal signal is treated as successful
  termination, matching the fallback path (same for Windows terminate).
- B5: ForceStopUnverified now covers the backend (proxy-child) record
  with the same safety checks, and CanForceStopUnverified accepts
  refusals where the backend is also (or only) unverifiable — the
  paired-legacy pre-v2 topology was locked out of its own advertised
  recovery. bd dolt stop --force reports the backend record too.
- B6a: an unverified live PID is only signaled when its command line
  ties it to this workspace (Linux /proc cmdline, Darwin ps args);
  Windows cannot read another process's command line and now refuses,
  which beats basename-only killing. Clear errors name the manual remedy.
- B6b: one stable OS handle (Linux pidfd, Windows process handle) spans
  inspection and signaling, removing the inspect/kill TOCTOU.
- S1: control-listener accept errors are non-fatal — log and retry with
  capped exponential backoff instead of tearing down a healthy proxy.
- S2: requireUnverifiableRecord surfaces RootID resolution errors
  instead of failing open into the destructive branch.
- S3: killSpawnedChild falls back to cmd.Process.Kill for our own
  un-reaped child when no verified handle exists, instead of leaking it.
- S4: adoption handshake accepts reply schema v2 or newer, matching
  pidfile.ValidateV2 forward-compat policy.
- S5: errors.Is for ERROR_INVALID_PARAMETER.
- Tests: pidfd ESRCH-after-exit and Open-after-reap classification;
  paired-legacy force recovery end to end; foreign-workspace bd process
  refused; Windows Verify-after-terminate (compile-checked); schema-v3
  handshake adoption; accept-loop survival under persistent errors.

Agent-Signature: claude-fable-5-medium on behalf of maphew

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 12:52:51 -07:00

206 lines
5.2 KiB
Go

//go:build linux
package procid
import (
"errors"
"os"
"os/exec"
"syscall"
"testing"
"time"
"golang.org/x/sys/unix"
)
func TestParseStartTime(t *testing.T) {
tests := []struct {
name string
stat string
want string
}{
{
name: "ordinary command",
stat: "123 (sleep) S 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 987654 20",
want: "987654",
},
{
name: "spaces and parentheses in command",
stat: "123 (a) b) S 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 42 20",
want: "42",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := parseStartTime(tt.stat)
if err != nil {
t.Fatalf("parseStartTime: %v", err)
}
if got != tt.want {
t.Errorf("parseStartTime = %q, want %q", got, tt.want)
}
})
}
}
func TestCaptureBootIDFailureIsNotProcessGone(t *testing.T) {
previous := bootIDPath
bootIDPath = "/definitely-not-a-real-procid-boot-id"
t.Cleanup(func() { bootIDPath = previous })
_, err := Capture(os.Getpid())
if err == nil {
t.Fatal("Capture succeeded with missing boot_id, want error")
}
if IsProcessGone(err) {
t.Fatalf("IsProcessGone(%v) = true, want false for environmental boot_id failure", err)
}
if matched, verifyErr := Verify(os.Getpid(), "linux-v1:irrelevant:1"); verifyErr == nil || matched {
t.Fatalf("Verify with missing boot_id = (%v, %v), want (false, error)", matched, verifyErr)
}
}
func TestFallbackFatalSignalAcceptsExitedTarget(t *testing.T) {
previous := pidfdOpen
pidfdOpen = func(int, int) (int, error) { return -1, unix.ENOSYS }
t.Cleanup(func() { pidfdOpen = previous })
cmd := exec.Command("sleep", "30")
if err := cmd.Start(); err != nil {
t.Fatalf("start child: %v", err)
}
waited := make(chan error, 1)
go func() { waited <- cmd.Wait() }()
tok, err := Capture(cmd.Process.Pid)
if err != nil {
_ = cmd.Process.Kill()
<-waited
t.Fatalf("Capture(child): %v", err)
}
handle, err := Open(cmd.Process.Pid, tok)
if err != nil {
_ = cmd.Process.Kill()
<-waited
t.Fatalf("Open(child): %v", err)
}
defer func() { _ = handle.Close() }()
if handle.pidfd != -1 {
t.Fatalf("Open fallback pidfd = %d, want -1", handle.pidfd)
}
if err := handle.Kill(); err != nil {
t.Fatalf("fallback Kill: %v", err)
}
select {
case <-waited:
case <-time.After(5 * time.Second):
t.Fatal("fallback-signaled child did not exit")
}
}
func requirePidfdSupport(t *testing.T) {
t.Helper()
fd, err := pidfdOpen(os.Getpid(), 0)
if errors.Is(err, unix.ENOSYS) {
t.Skip("kernel has no pidfd support")
}
if err != nil {
t.Fatalf("pidfd open self: %v", err)
}
_ = unix.Close(fd)
}
// A fatal signal whose target exited naturally between Open and Signal must
// count as success on the pidfd path, matching the fallback path: the process
// is gone, which is the goal.
func TestPidfdFatalSignalAcceptsExitedTarget(t *testing.T) {
requirePidfdSupport(t)
cmd := exec.Command("sleep", "30")
if err := cmd.Start(); err != nil {
t.Fatalf("start child: %v", err)
}
tok, err := Capture(cmd.Process.Pid)
if err != nil {
_ = cmd.Process.Kill()
_ = cmd.Wait()
t.Fatalf("Capture(child): %v", err)
}
handle, err := Open(cmd.Process.Pid, tok)
if err != nil {
_ = cmd.Process.Kill()
_ = cmd.Wait()
t.Fatalf("Open(child): %v", err)
}
defer func() { _ = handle.Close() }()
if handle.pidfd < 0 {
t.Fatalf("Open pidfd = %d, want >= 0", handle.pidfd)
}
if err := cmd.Process.Kill(); err != nil {
t.Fatalf("kill child: %v", err)
}
_ = cmd.Wait() // reap, so the target is fully gone, not a zombie
if err := handle.Kill(); err != nil {
t.Fatalf("Kill after natural exit = %v, want nil", err)
}
}
// Open must not hand back a handle for an exited process, and the failure has
// to classify as gone so lifecycle callers quarantine instead of refusing.
func TestOpenReapedChildReportsGone(t *testing.T) {
requirePidfdSupport(t)
cmd := exec.Command("sleep", "0.05")
if err := cmd.Start(); err != nil {
t.Fatalf("start child: %v", err)
}
tok, err := Capture(cmd.Process.Pid)
if err != nil {
_ = cmd.Wait()
t.Fatalf("Capture(child): %v", err)
}
if err := cmd.Wait(); err != nil {
t.Fatalf("wait child: %v", err)
}
_, err = Open(cmd.Process.Pid, tok)
if err == nil {
t.Fatal("Open(reaped child) succeeded, want error")
}
if !IsProcessGone(err) {
t.Fatalf("IsProcessGone(Open(reaped child)) = false for %v", err)
}
}
func TestVerifyZombieChildIsGone(t *testing.T) {
cmd := exec.Command("sleep", "0.05")
if err := cmd.Start(); err != nil {
t.Fatalf("start child: %v", err)
}
tok, err := Capture(cmd.Process.Pid)
if err != nil {
_ = cmd.Wait()
t.Fatalf("Capture(child): %v", err)
}
defer func() { _ = cmd.Wait() }()
deadline := time.Now().Add(5 * time.Second)
for {
matched, verifyErr := Verify(cmd.Process.Pid, tok)
if verifyErr != nil {
t.Fatalf("Verify(zombie child): %v", verifyErr)
}
if !matched {
break
}
if time.Now().After(deadline) {
t.Fatal("Verify(zombie child) remained true")
}
time.Sleep(10 * time.Millisecond)
}
// Signal 0 still sees an unreaped zombie's PID, proving Verify classified
// process state rather than relying only on /proc/<pid> disappearing.
if err := syscall.Kill(cmd.Process.Pid, 0); err != nil && !errors.Is(err, unix.EPERM) {
t.Fatalf("signal 0 to unreaped zombie: %v", err)
}
}