feat: add journaled direct-local ownership handoff API (#6281)

* feat: add journaled direct-local ownership handoff API

* fix: harden ownership handoff retries and identity

* fix(ownershiphandoff): durable checkpoints, stated retry contract, decomposed Execute

Maintainer review fixups for PR #6281, closing the two correctness majors and
both measured-complexity band majors raised by the quality scorecard.

save() now fsyncs the journal's parent directory after the atomic rename,
mirroring configfile.SaveProxiedServerClientInfo. Without it, power loss could
lose the commit reservation while the Commit hook's external effects survived,
so a retry would re-invoke Commit — exactly what the reservation's own comment
promises to prevent.

Hooks now states the retry contract it always relied on: Execute resumes from
the last durable phase, so Snapshot/Configure/StopLegacy/Verify must be
idempotent, an already-absent legacy owner is success rather than a refusal,
and Configure must not mutate the legacy-owned scope. Commit remains the sole
at-most-once hook. The stop leg gains a pre-invocation reservation
(legacy_stop_in_progress) so a partial or interrupted stop is reported as a
mutation instead of an untouched legacy owner, with a matching Load invariant.

Execute is decomposed into an unlocked preflight and a handoffRun phase
machine; ValidateRequest and Load split along their existing seams. Worst
changed-function complexity drops from cyclomatic 44 / cognitive 87 to 18 / 16.

Also folds in the non-gating cleanups: cumulative mutation reporting on both
identity-conflict paths, journal_unreadable typing, a truthful dry-run phase,
lock contention distinguished from lock I/O failure (lock_unavailable), a
bounded commit-recovery message, the ambiguous-commit contract sentence, the
dead root guard and empty-host message, and new tests for the corrupt journal,
mid-flight dry run, stop reservation and retry, and message growth.

The second review round reproduced three operator-surface defects, all fixed
here. commitRecoveryMessage is now a fixed point rather than regenerative: an
ambiguous-commit journal that already carries the explanation is returned
unchanged, so from the third retry onward the original commit failure survives
instead of being overwritten with bare boilerplate — it used to vanish from both
the journal and the returned error precisely when a recovery loop had set in.
resolveRoot splits its precondition branch: a root that exists but is not a
directory no longer wraps a nil error, which rendered the literal
"root must be an existing directory: %!w(<nil>)" on the public ValidateRequest
path, while a missing root still carries its real cause. Result.Mutates gains
the doc sentence its cumulative-on-refusal, attempt-scoped-on-no-op semantics
needed. Each behavior is pinned: the recovery test drives a fourth attempt and
asserts the cause survives in both the journal and the returned error, and the
rejection table gains a non-directory root case beside a message-asserting test.

The Windows directory-fsync exposure raised that round is deliberately not
addressed here. The identical unconditional os.Open+Sync shape already ships at
base in configfile.SaveProxiedServerClientInfo, so guarding only the new site
would fix an inert copy and leave the shipping one broken while making the two
diverge. It is tracked as one two-site follow-up to be settled on a real Windows
runner before a consumer makes this path reachable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Julian Knutsen 2026-09-11 16:43:53 -07:00 • committed by GitHub
commit 74e25cba15
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 1373 additions and 43 deletions

View file

@ -1,45 +1,27 @@
# Direct-local ownership handoff contract
This contract defines the Beads-side boundary required before Gas City can
retire a legacy GC-managed direct-local Dolt server. It is intentionally an
explicit operation; normal `bd` or `gc` startup never invokes it.
`internal/ownershiphandoff` is the explicit Beads-side boundary for handing a
GC-managed direct-local Dolt server to `bd`. Normal startup never invokes it.
## Request identity
The handoff request must name the canonical, real-path Dolt data root, Dolt
database, Beads project/workspace identity, endpoint (host/port or socket),
and current owner (`legacy-gc`). The provider rejects missing, symlinked,
non-canonical, or conflicting identity. It must never create a replacement
root or database when identity cannot be proved.
## Journaled phases
The operation persists an atomic journal beside the Beads metadata. Every
retry resumes from the last durable phase and a committed retry is a no-op.
1. `prepared`: snapshot metadata, config, endpoint, ownership, and sentinel.
2. `target_configured`: validate that bd can open the exact root/database.
3. `old_owner_stopped`: stop only the positively identified GC owner through
its existing lifecycle interface; a live or unknown process fails closed.
4. `verified`: start through bd and verify endpoint, database identity, and
sentinel rows.
5. `committed`: atomically record owner `bd` and retire only GC artifacts whose
ownership matches the snapshot.
Any failure before `committed` leaves `legacy-gc` authoritative. Rollback
restores the byte/row/config snapshot and never kills an unverified process.
The journal records the error and phase so operators can retry or inspect it.
## Front-door shape
The eventual CLI/API must expose dry-run, execute, and resume behavior with a
typed result containing `phase`, `owner`, `root`, `database`, `endpoint`,
`mutates`, and `error_code`. Text and JSON responses share the same values;
non-zero exits are required for every refusal. Dry-run and all refusals have
`mutates=false` and must not open a provider.
External TCP/Unix endpoints, embedded stores, wrong-root/database requests,
missing or stale runtime state, concurrent starts, and duplicate-owner
conditions are refusal boundaries. Existing rows, event history, metadata,
locks, PID records, and unrelated files are no-mutation artifacts in those
cases.
Requests identify a canonical real-path Dolt root, database, workspace, local
endpoint, and current owner (`legacy-gc`). Symlinked, missing, non-canonical,
remote, or conflicting identities are rejected before any provider hook runs.
The operation records `prepared`, `target_configured`, `old_owner_stopped`,
`verified`, and `committed` in an atomic journal; each write is fsynced and its
containing directory is fsynced after the rename, so a checkpoint cannot be
lost while the effect it guards survives. Retries resume from the last phase
and committed replays are no-ops. Hooks own provider-specific snapshots,
validation, lifecycle, and artifact retirement; the package never guesses at
or kills an external process. Because a retry resumes from the last durable
phase, every hook except `Commit` must be idempotent under re-invocation, and
the legacy-owner stop is reserved in the journal before it is attempted so a
partial stop is reported as a mutation. Failures remain owned by `legacy-gc`
and are journaled with a stable error code. The commit hook has a durable
in-progress checkpoint and a post-hook completion checkpoint; a commit is
ambiguous whenever it was reserved but not recorded as complete — including
when the hook itself returned an error, which may have applied part of its
effect — and an ambiguous commit refuses to advance unless the provider
supplies an explicitly idempotent replay hook.
The per-journal advisory lock is persistent and kernel-released rather than an
`O_EXCL` marker, so a crashed process cannot leave a stale file that wedges
future retries.

View file

@ -0,0 +1,704 @@
// Package ownershiphandoff implements the explicit, journaled handoff from a
// legacy GC-owned direct-local Dolt server to bd. It deliberately has no
// process-management policy: callers must provide a positively identifying
// legacy-owner stop hook.
package ownershiphandoff
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/steveyegge/beads/internal/lockfile"
)
// Owner identifies the lifecycle authority for a Dolt scope.
type Owner string
const (
OwnerLegacyGC Owner = "legacy-gc"
OwnerBD Owner = "bd"
)
// Phase is a durable checkpoint in the handoff journal.
type Phase string
const (
PhasePrepared Phase = "prepared"
PhaseTargetConfigured Phase = "target_configured"
PhaseOldOwnerStopped Phase = "old_owner_stopped"
PhaseVerified Phase = "verified"
PhaseCommitted Phase = "committed"
)
// Endpoint identifies the local server endpoint. Exactly one of Port or
// Socket may be set. Host must be loopback; ownership handoff never adopts a
// remotely managed server.
type Endpoint struct {
Host string `json:"host,omitempty"`
Port int `json:"port,omitempty"`
Socket string `json:"socket,omitempty"`
}
// String returns a stable display form for the endpoint.
func (e Endpoint) String() string {
if e.Socket != "" {
return "unix://" + e.Socket
}
return fmt.Sprintf("%s:%d", e.Host, e.Port)
}
// Request identifies the exact scope being handed off.
type Request struct {
Root string `json:"root"`
Database string `json:"database"`
Workspace string `json:"workspace"`
Endpoint Endpoint `json:"endpoint"`
Owner Owner `json:"owner"`
}
// Snapshot contains provider metadata captured before mutation.
type Snapshot struct {
Metadata []byte `json:"metadata,omitempty"`
Config []byte `json:"config,omitempty"`
Sentinel string `json:"sentinel,omitempty"`
}
// Journal is the atomically persisted handoff state.
type Journal struct {
Request Request `json:"request"`
Snapshot Snapshot `json:"snapshot,omitempty"`
SnapshotCaptured bool `json:"snapshot_captured,omitempty"`
// LegacyStopInProgress records that StopLegacy was invoked but has not yet
// reached its own checkpoint. It makes a partial or interrupted stop a
// durable mutation instead of leaving the journal claiming the legacy owner
// was never touched.
LegacyStopInProgress bool `json:"legacy_stop_in_progress,omitempty"`
CommitHookInProgress bool `json:"commit_hook_in_progress,omitempty"`
CommitHookRan bool `json:"commit_hook_ran,omitempty"`
Phase Phase `json:"phase"`
Owner Owner `json:"owner"`
ErrorCode string `json:"error_code,omitempty"`
Error string `json:"error,omitempty"`
UpdatedAt time.Time `json:"updated_at"`
}
// Result is the typed front-door outcome shared by text and JSON callers.
type Result struct {
Phase Phase `json:"phase"`
Owner Owner `json:"owner"`
Root string `json:"root"`
Database string `json:"database"`
Endpoint Endpoint `json:"endpoint"`
// Mutates reports whether handoff execution has touched provider-visible
// state, cumulatively across resumed attempts: a refusal or failure inherits
// the mutation an earlier attempt made durable. No-op outcomes report false
// even against a journal whose phase proves prior mutation — a dry run and a
// committed replay both change nothing themselves — so read Phase, not
// Mutates, for how far the handoff has progressed.
Mutates bool `json:"mutates"`
ErrorCode string `json:"error_code,omitempty"`
}
// Hooks are provider-owned operations.
//
// Retry contract: Execute resumes from the last durable journal phase, so a
// crash between a hook returning and its checkpoint reaching disk replays that
// hook. Snapshot, Configure, StopLegacy, and Verify must therefore be
// idempotent under re-invocation, and an effect that is already applied is
// success rather than a conflict. Commit is the sole exception: it is bracketed
// by durable checkpoints so it is invoked at most once, and an ambiguous
// outcome is recovered only through CommitReplay.
type Hooks struct {
// Snapshot captures provider metadata before any mutation. It runs at most
// once per journal because the captured snapshot is made durable before
// Configure.
Snapshot func(context.Context, Request) (Snapshot, error)
// Configure prepares the bd-owned target. It must not mutate the
// legacy-owned scope: until Commit, legacy-gc stays authoritative and every
// failure path must be able to leave the scope untouched. Re-running it
// against an already-configured target must succeed.
Configure func(context.Context, Request, Snapshot) error
// StopLegacy stops the legacy owner. It must refuse unless it can
// positively identify the process as the legacy owner; it must never kill an
// unknown process. An already-absent legacy owner is success: the refusal
// duty is about never killing an unidentified *live* process, so a retry
// after a stop that was durable but uncheckpointed must not wedge the
// handoff.
StopLegacy func(context.Context, Request, Snapshot) error
// Verify confirms the bd-owned target serves the scope. It is read-only and
// re-runnable.
Verify func(context.Context, Request, Snapshot) error
// Commit retires the legacy owner's artifacts and transfers authority.
Commit func(context.Context, Request, Snapshot) error
// CommitReplay is an optional idempotent recovery operation for a commit
// attempt whose outcome is unknown: a process crash after the hook ran but
// before its completion checkpoint was durable, or any error returned by
// Commit, which may have applied part of its effect before failing. Without
// this explicit provider guarantee, Execute refuses to guess or invoke
// Commit a second time.
CommitReplay func(context.Context, Request, Snapshot) error
}
// ValidateRequest rejects incomplete, non-canonical, or remotely managed identities.
func ValidateRequest(r Request) error {
if !filepath.IsAbs(r.Root) || filepath.Clean(r.Root) != r.Root {
return errors.New("root must be an absolute canonical path")
}
if r.Owner != OwnerLegacyGC {
return errors.New("owner must be legacy-gc")
}
if r.Database == "" || r.Workspace == "" {
return errors.New("database and workspace are required")
}
root, err := resolveRoot(r.Root)
if err != nil {
return err
}
return validateEndpoint(root, r.Endpoint)
}
// resolveRoot requires root to exist as a directory and to already be its own
// canonical path, and returns the resolved form used for socket containment.
func resolveRoot(root string) (string, error) {
info, err := os.Stat(root)
if err != nil {
return "", fmt.Errorf("root must be an existing directory: %w", err)
}
// A root that exists but is not a directory is a precondition failure with no
// underlying error, so it must not be wrapped: %w over a nil error renders a
// format artifact where the operator expects the reason.
if !info.IsDir() {
return "", errors.New("root must be an existing directory")
}
real, err := filepath.EvalSymlinks(root)
if err != nil {
return "", fmt.Errorf("resolve root: %w", err)
}
if filepath.Clean(root) != filepath.Clean(real) {
return "", errors.New("root must be canonical and not symlinked")
}
return real, nil
}
// validateEndpoint accepts only a socket contained by root or a loopback TCP
// port. root must already be resolved by resolveRoot.
func validateEndpoint(root string, e Endpoint) error {
if e.Socket != "" && e.Port != 0 {
return errors.New("endpoint may specify socket or port, not both")
}
if e.Socket != "" {
return validateSocketEndpoint(root, e)
}
if e.Port <= 0 || e.Port > 65535 {
return errors.New("endpoint port is invalid")
}
if e.Host == "" {
return errors.New("endpoint host is required")
}
host := strings.ToLower(e.Host)
if host != "127.0.0.1" && host != "localhost" && host != "::1" {
return errors.New("external endpoint is not eligible for handoff")
}
return nil
}
// validateSocketEndpoint requires an absolute socket path that stays inside
// root once symlinked parent directories are resolved.
func validateSocketEndpoint(root string, e Endpoint) error {
if e.Host != "" {
return errors.New("socket endpoint must not specify a host")
}
if !filepath.IsAbs(e.Socket) {
return errors.New("socket must be absolute")
}
resolved, err := resolvePathForContainment(e.Socket)
if err != nil {
return fmt.Errorf("resolve socket: %w", err)
}
rel, err := filepath.Rel(root, resolved)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return errors.New("external unix endpoint is not eligible for handoff")
}
return nil
}
// resolvePathForContainment resolves every existing component of path. The
// final socket is commonly created by the provider after validation, so a
// missing final component is retained while symlinked parent directories are
// still resolved. This prevents a socket beneath root from escaping through a
// symlink introduced after the lexical Rel check.
func resolvePathForContainment(path string) (string, error) {
path = filepath.Clean(path)
var suffix []string
for {
if _, err := os.Lstat(path); err == nil {
resolved, err := filepath.EvalSymlinks(path)
if err != nil {
return "", err
}
for i := len(suffix) - 1; i >= 0; i-- {
resolved = filepath.Join(resolved, suffix[i])
}
return filepath.Clean(resolved), nil
} else if !os.IsNotExist(err) {
return "", err
}
parent := filepath.Dir(path)
if parent == path {
return "", os.ErrNotExist
}
suffix = append(suffix, filepath.Base(path))
path = parent
}
}
// Load reads and validates a handoff journal from path.
func Load(path string) (Journal, error) {
b, err := os.ReadFile(path) //nolint:gosec // path is the operator-selected journal path
if err != nil {
return Journal{}, err
}
var j Journal
if err := json.Unmarshal(b, &j); err != nil {
return Journal{}, fmt.Errorf("decode handoff journal: %w", err)
}
if err := validateJournal(j); err != nil {
return Journal{}, err
}
return j, nil
}
// validateJournal rejects any journal state the phase machine cannot have
// written. Each checkpoint is only meaningful in the phase that reserves it, so
// an impossible combination means corruption or an out-of-band edit rather than
// a resumable handoff.
func validateJournal(j Journal) error {
if j.Owner != OwnerLegacyGC && j.Owner != OwnerBD {
return errors.New("handoff journal has unknown owner")
}
switch j.Phase {
case PhasePrepared, PhaseTargetConfigured, PhaseOldOwnerStopped, PhaseVerified, PhaseCommitted:
default:
return errors.New("handoff journal has unknown phase")
}
if j.Phase != PhaseCommitted && j.Owner != OwnerLegacyGC {
return errors.New("uncommitted handoff journal must remain owned by legacy-gc")
}
if j.Phase == PhaseCommitted && j.Owner != OwnerBD {
return errors.New("committed handoff journal must be owned by bd")
}
if (j.CommitHookInProgress || j.CommitHookRan) && j.Phase != PhaseVerified && j.Phase != PhaseCommitted {
return errors.New("handoff journal has commit checkpoint in an invalid phase")
}
if j.CommitHookInProgress && j.CommitHookRan {
return errors.New("handoff journal has conflicting commit checkpoints")
}
if j.Phase == PhaseCommitted && j.CommitHookInProgress {
return errors.New("committed handoff journal has an in-progress commit")
}
if j.LegacyStopInProgress && j.Phase != PhaseTargetConfigured {
return errors.New("handoff journal has a stop checkpoint in an invalid phase")
}
return nil
}
func save(path string, j Journal) error {
b, err := json.MarshalIndent(j, "", " ")
if err != nil {
return err
}
tmp, err := os.CreateTemp(filepath.Dir(path), ".handoff-*")
if err != nil {
return err
}
tmpName := tmp.Name()
defer func() { _ = os.Remove(tmpName) }()
if err = tmp.Chmod(0600); err == nil {
_, err = tmp.Write(append(b, '\n'))
}
if err == nil {
err = tmp.Sync()
}
if closeErr := tmp.Close(); err == nil {
err = closeErr
}
if err != nil {
return err
}
if err := os.Rename(tmpName, path); err != nil {
return err
}
// The rename itself must be durable, not just the temp file's contents. A
// checkpoint that a hook's external effects outlive is worse than no
// checkpoint: on power loss the pre-reservation journal would be resurrected
// and a retry would re-invoke a hook this journal already promised was
// reserved.
return syncDir(filepath.Dir(path))
}
// syncDir flushes a directory entry so a completed rename survives power loss.
func syncDir(dir string) error {
d, err := os.Open(dir) //nolint:gosec // dir is the operator-selected journal's directory
if err != nil {
return fmt.Errorf("sync handoff journal directory: %w", err)
}
if err := d.Sync(); err != nil {
_ = d.Close()
return fmt.Errorf("sync handoff journal directory: %w", err)
}
if err := d.Close(); err != nil {
return fmt.Errorf("close handoff journal directory: %w", err)
}
return nil
}
func result(j Journal, mutates bool) Result {
return Result{Phase: j.Phase, Owner: j.Owner, Root: j.Request.Root, Database: j.Request.Database, Endpoint: j.Request.Endpoint, Mutates: mutates, ErrorCode: j.ErrorCode}
}
// requestResult reports an outcome refused before any journal was adopted.
func requestResult(r Request, code string) Result {
return Result{Phase: PhasePrepared, Owner: OwnerLegacyGC, Root: r.Root, Database: r.Database, Endpoint: r.Endpoint, ErrorCode: code}
}
func mutationOccurred(j Journal) bool {
return j.Phase == PhaseOldOwnerStopped || j.Phase == PhaseVerified ||
j.Phase == PhaseCommitted || j.CommitHookRan || j.CommitHookInProgress ||
j.LegacyStopInProgress
}
func snapshotCaptured(j Journal) bool {
return j.SnapshotCaptured || len(j.Snapshot.Metadata) != 0 ||
len(j.Snapshot.Config) != 0 || j.Snapshot.Sentinel != ""
}
func journalSaveError(j Journal, saveErr error) (Result, error) {
j.ErrorCode = "journal_save_failed"
j.Error = saveErr.Error()
return result(j, mutationOccurred(j)), fmt.Errorf("save handoff journal: %w", saveErr)
}
// acquireLock opens a persistent advisory lock beside the journal. The lock
// inode is never removed: the kernel releases the lock when the process exits,
// so a crash cannot wedge retries or let stale reclaimers unlink a live lock.
func acquireLock(path string) (*os.File, error) {
lock, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE, 0600) //nolint:gosec // path is derived from the operator-selected journal
if err != nil {
return nil, err
}
if err := lockfile.FlockExclusiveNonBlocking(lock); err != nil {
_ = lock.Close()
return nil, err
}
return lock, nil
}
// lockErrorCode separates a genuinely concurrent handoff from a lock file that
// could not be opened at all. Reporting ENOENT or EACCES as concurrent_handoff
// sends an operator hunting a process that does not exist.
func lockErrorCode(err error) string {
if lockfile.IsLocked(err) {
return "concurrent_handoff"
}
return "lock_unavailable"
}
// Execute validates and performs a handoff. Dry-run validates identity only
// and never invokes a hook or opens a provider. A committed journal replays as
// a no-op. Failures are journaled and leave legacy-gc authoritative.
func Execute(ctx context.Context, r Request, journalPath string, h Hooks, dryRun bool) (Result, error) {
if err := ValidateRequest(r); err != nil {
return requestResult(r, "invalid_request"), err
}
if out := preflight(r, journalPath, dryRun); out != nil {
return out.res, out.err
}
lock, err := acquireLock(journalPath + ".lock")
if err != nil {
return requestResult(r, lockErrorCode(err)), fmt.Errorf("acquire handoff lock: %w", err)
}
defer func() {
_ = lockfile.FlockUnlock(lock)
_ = lock.Close()
}()
return newRun(r, journalPath, h).run(ctx)
}
// stepOutcome is a terminal answer from a handoff step. A nil *stepOutcome
// means the step is satisfied and the run continues to the next one.
type stepOutcome struct {
res Result
err error
}
func done(res Result, err error) *stepOutcome { return &stepOutcome{res: res, err: err} }
// preflight answers from an unlocked journal read. Identity conflicts,
// committed replays, unreadable journals, and dry runs never need the lock.
func preflight(r Request, journalPath string, dryRun bool) *stepOutcome {
j, err := Load(journalPath)
if err != nil {
if !os.IsNotExist(err) {
return done(requestResult(r, "journal_unreadable"), err)
}
if dryRun {
return done(requestResult(r, ""), nil)
}
return nil
}
if out := refuseOrReplay(j, r); out != nil {
return out
}
if dryRun {
// Report the adopted journal's real phase: an operator probing a
// mid-flight handoff must not be told the scope is still prepared.
return done(result(j, false), nil)
}
return nil
}
// refuseOrReplay settles the two questions a durable journal answers on its
// own: whether it belongs to a different scope, and whether the handoff already
// committed. A nil outcome means the journal matches and is still in flight.
func refuseOrReplay(j Journal, r Request) *stepOutcome {
if j.Request != r {
j.ErrorCode = "identity_conflict"
return done(result(j, mutationOccurred(j)), errors.New("handoff journal identity conflicts with request"))
}
if j.Phase == PhaseCommitted {
return done(result(j, false), nil)
}
return nil
}
// handoffRun holds the journal state for one lock-held handoff attempt.
type handoffRun struct {
journalPath string
hooks Hooks
j Journal
}
func newRun(r Request, journalPath string, h Hooks) *handoffRun {
return &handoffRun{
journalPath: journalPath,
hooks: h,
j: Journal{Request: r, Owner: OwnerLegacyGC, Phase: PhasePrepared, UpdatedAt: time.Now().UTC()},
}
}
// run adopts any durable journal and drives the phase machine from wherever
// that journal left off.
func (x *handoffRun) run(ctx context.Context) (Result, error) {
steps := []func(context.Context) *stepOutcome{
x.adopt, x.stepSnapshot, x.stepConfigure, x.stepStopLegacy, x.stepVerify, x.stepCommit,
}
for _, step := range steps {
if out := step(ctx); out != nil {
return out.res, out.err
}
}
return result(x.j, true), nil
}
// adopt re-reads the journal under the lock. The preflight read is only an
// optimization and may have raced with another handoff.
func (x *handoffRun) adopt(context.Context) *stepOutcome {
old, err := Load(x.journalPath)
if err != nil {
if os.IsNotExist(err) {
return nil
}
x.j.ErrorCode = "journal_unreadable"
return done(result(x.j, false), err)
}
if out := refuseOrReplay(old, x.j.Request); out != nil {
return out
}
x.j = old
return nil
}
func (x *handoffRun) save() error { return save(x.journalPath, x.j) }
// advance records the next durable phase.
func (x *handoffRun) advance(p Phase) *stepOutcome {
x.j.Phase = p
x.j.UpdatedAt = time.Now().UTC()
if err := x.save(); err != nil {
return done(journalSaveError(x.j, err))
}
return nil
}
// fail journals a typed refusal and leaves legacy-gc authoritative.
func (x *handoffRun) fail(code string, err error) *stepOutcome {
x.j.ErrorCode, x.j.Error, x.j.UpdatedAt = code, err.Error(), time.Now().UTC()
if saveErr := x.save(); saveErr != nil {
return done(journalSaveError(x.j, errors.Join(err, saveErr)))
}
return done(result(x.j, mutationOccurred(x.j)), err)
}
func (x *handoffRun) clearError() {
x.j.ErrorCode = ""
x.j.Error = ""
}
func (x *handoffRun) stepSnapshot(ctx context.Context) *stepOutcome {
if x.j.Phase != PhasePrepared || snapshotCaptured(x.j) {
return nil
}
if x.hooks.Snapshot == nil {
return x.fail("snapshot_unavailable", errors.New("snapshot hook is required"))
}
s, err := x.hooks.Snapshot(ctx, x.j.Request)
if err != nil {
return x.fail("snapshot_failed", err)
}
x.j.Snapshot = s
x.j.SnapshotCaptured = true
if err := x.save(); err != nil {
return done(journalSaveError(x.j, err))
}
return nil
}
func (x *handoffRun) stepConfigure(ctx context.Context) *stepOutcome {
if x.j.Phase != PhasePrepared {
return nil
}
if x.hooks.Configure == nil {
return x.fail("configure_unavailable", errors.New("configure hook is required"))
}
if err := x.hooks.Configure(ctx, x.j.Request, x.j.Snapshot); err != nil {
return x.fail("target_configure_failed", err)
}
return x.advance(PhaseTargetConfigured)
}
func (x *handoffRun) stepStopLegacy(ctx context.Context) *stepOutcome {
if x.j.Phase != PhaseTargetConfigured {
return nil
}
if x.hooks.StopLegacy == nil {
return x.fail("owner_stop_unavailable", errors.New("legacy owner stop hook is required"))
}
// Reserve the stop durably before invoking it. Once the hook has been
// entered its effect cannot be assumed absent, so a failed or interrupted
// stop must report a mutation rather than an untouched legacy owner.
if !x.j.LegacyStopInProgress {
x.j.LegacyStopInProgress = true
x.j.UpdatedAt = time.Now().UTC()
if err := x.save(); err != nil {
return done(journalSaveError(x.j, err))
}
}
if err := x.hooks.StopLegacy(ctx, x.j.Request, x.j.Snapshot); err != nil {
return x.fail("owner_stop_failed", err)
}
x.j.LegacyStopInProgress = false
return x.advance(PhaseOldOwnerStopped)
}
func (x *handoffRun) stepVerify(ctx context.Context) *stepOutcome {
if x.j.Phase != PhaseOldOwnerStopped {
return nil
}
if x.hooks.Verify == nil {
return x.fail("verify_unavailable", errors.New("verify hook is required"))
}
if err := x.hooks.Verify(ctx, x.j.Request, x.j.Snapshot); err != nil {
return x.fail("verification_failed", err)
}
return x.advance(PhaseVerified)
}
func (x *handoffRun) stepCommit(ctx context.Context) *stepOutcome {
if x.j.Phase != PhaseVerified {
return nil
}
if out := x.replayAmbiguousCommit(ctx); out != nil {
return out
}
if out := x.invokeCommit(ctx); out != nil {
return out
}
x.j.Owner = OwnerBD
x.clearError()
return x.advance(PhaseCommitted)
}
// replayAmbiguousCommit resolves a commit that was reserved but never reached
// its completion checkpoint. Only an explicitly idempotent provider hook may
// advance it.
func (x *handoffRun) replayAmbiguousCommit(ctx context.Context) *stepOutcome {
if !x.j.CommitHookInProgress || x.j.CommitHookRan {
return nil
}
if x.hooks.CommitReplay == nil {
return x.fail("commit_recovery_required", errors.New(commitRecoveryMessage(x.j)))
}
if err := x.hooks.CommitReplay(ctx, x.j.Request, x.j.Snapshot); err != nil {
return x.fail("commit_recovery_failed", err)
}
return x.recordCommitRan()
}
// commitRecoveryMessage explains an ambiguous commit, keeping the original
// commit failure as context. It builds the message once from the journal's own
// commit_failed error and is then a fixed point: a journal that already carries
// the explanation is returned unchanged, so repeated retries neither grow the
// message without bound nor discard the cause that started the recovery. An
// operator reading a wedged journal needs that cause most on the retry where a
// regenerated message would have lost it.
func commitRecoveryMessage(j Journal) string {
const message = "commit hook outcome is unknown; an idempotent commit replay hook is required"
if j.ErrorCode == "commit_failed" && j.Error != "" {
return j.Error + "; " + message
}
if strings.HasSuffix(j.Error, message) {
return j.Error
}
return message
}
func (x *handoffRun) invokeCommit(ctx context.Context) *stepOutcome {
if x.j.CommitHookRan {
return nil
}
if x.hooks.Commit == nil {
return x.fail("commit_unavailable", errors.New("commit hook is required"))
}
// Reserve the commit hook durably before invoking it. If the process
// crashes after this checkpoint, a retry refuses to invoke Commit a
// second time unless the provider supplies CommitReplay.
x.j.CommitHookInProgress = true
x.clearError()
x.j.UpdatedAt = time.Now().UTC()
if err := x.save(); err != nil {
return done(journalSaveError(x.j, err))
}
if err := x.hooks.Commit(ctx, x.j.Request, x.j.Snapshot); err != nil {
return x.fail("commit_failed", err)
}
return x.recordCommitRan()
}
// recordCommitRan durably retires the in-progress reservation once the commit
// effect is known to have been applied exactly once.
func (x *handoffRun) recordCommitRan() *stepOutcome {
x.j.CommitHookRan = true
x.j.CommitHookInProgress = false
x.clearError()
x.j.UpdatedAt = time.Now().UTC()
if err := x.save(); err != nil {
return done(journalSaveError(x.j, err))
}
return nil
}

View file

@ -0,0 +1,644 @@
package ownershiphandoff
import (
"context"
"encoding/json"
"errors"
"io/fs"
"os"
"path/filepath"
"strings"
"testing"
"github.com/steveyegge/beads/internal/lockfile"
)
func validRequest(t *testing.T) Request {
t.Helper()
root := t.TempDir()
// macOS exposes temporary directories through a symlinked /var prefix;
// callers of the handoff API must provide the canonical real path.
canonical, err := filepath.EvalSymlinks(root)
if err != nil {
t.Fatal(err)
}
root = canonical
return Request{Root: root, Database: "beads", Workspace: "ws-1", Endpoint: Endpoint{Host: "127.0.0.1", Port: 3307}, Owner: OwnerLegacyGC}
}
func TestValidateRejectsSymlinkRootAndExternalEndpoint(t *testing.T) {
tests := []struct {
name string
mutate func(*testing.T, *Request)
}{
{name: "symlinked root", mutate: func(t *testing.T, r *Request) {
link := filepath.Join(t.TempDir(), "link")
if err := os.Symlink(r.Root, link); err != nil {
t.Fatal(err)
}
r.Root = link
}},
{name: "non-canonical root", mutate: func(_ *testing.T, r *Request) {
r.Root += string(filepath.Separator) + "."
}},
{name: "root is a regular file", mutate: func(t *testing.T, r *Request) {
file := filepath.Join(r.Root, "root-file")
if err := os.WriteFile(file, nil, 0600); err != nil {
t.Fatal(err)
}
r.Root = file
}},
{name: "external host", mutate: func(_ *testing.T, r *Request) { r.Endpoint.Host = "10.0.0.8" }},
{name: "empty host", mutate: func(_ *testing.T, r *Request) { r.Endpoint.Host = "" }},
{name: "port zero", mutate: func(_ *testing.T, r *Request) { r.Endpoint.Port = 0 }},
{name: "port negative", mutate: func(_ *testing.T, r *Request) { r.Endpoint.Port = -1 }},
{name: "port above range", mutate: func(_ *testing.T, r *Request) { r.Endpoint.Port = 65536 }},
{name: "external unix endpoint", mutate: func(_ *testing.T, r *Request) {
r.Endpoint = Endpoint{Socket: filepath.Join(filepath.Dir(r.Root), "outside.sock")}
}},
{name: "socket with host", mutate: func(_ *testing.T, r *Request) {
r.Endpoint = Endpoint{Host: "127.0.0.1", Socket: filepath.Join(r.Root, "beads.sock")}
}},
{name: "socket through symlinked directory", mutate: func(t *testing.T, r *Request) {
if err := os.Symlink(t.TempDir(), filepath.Join(r.Root, "run")); err != nil {
t.Fatal(err)
}
r.Endpoint = Endpoint{Socket: filepath.Join(r.Root, "run", "beads.sock")}
}},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
r := validRequest(t)
tc.mutate(t, &r)
if err := ValidateRequest(r); err == nil {
t.Fatalf("%s accepted: %+v", tc.name, r)
}
})
}
}
// TestValidateNamesTheIncompleteEndpoint pins the message, not just the
// rejection: an endpoint with no host at all is incomplete, and reporting it as
// "external" misdirects the reader.
func TestValidateNamesTheIncompleteEndpoint(t *testing.T) {
r := validRequest(t)
r.Endpoint.Host = ""
err := ValidateRequest(r)
if err == nil || !strings.Contains(err.Error(), "host is required") {
t.Fatalf("empty host error = %v, want an incomplete-endpoint message", err)
}
}
// TestValidateNamesTheNonDirectoryRoot pins both halves of the root
// precondition. A root that exists but is not a directory has no underlying
// error to report, so it must read as a plain rejection instead of a formatting
// artifact; a root that is simply missing must still carry its real cause.
func TestValidateNamesTheNonDirectoryRoot(t *testing.T) {
base := validRequest(t)
file := filepath.Join(base.Root, "root-file")
if err := os.WriteFile(file, nil, 0600); err != nil {
t.Fatal(err)
}
r := base
r.Root = file
err := ValidateRequest(r)
if err == nil || err.Error() != "root must be an existing directory" {
t.Fatalf("non-directory root error = %v, want a plain existing-directory message", err)
}
r = base
r.Root = filepath.Join(base.Root, "absent")
if err := ValidateRequest(r); !errors.Is(err, fs.ErrNotExist) {
t.Fatalf("missing root error = %v, want the underlying cause wrapped", err)
}
}
func TestValidateRejectsIncompleteIdentity(t *testing.T) {
tests := []struct {
name string
mutate func(*Request)
}{
{name: "owner", mutate: func(r *Request) { r.Owner = OwnerBD }},
{name: "database", mutate: func(r *Request) { r.Database = "" }},
{name: "workspace", mutate: func(r *Request) { r.Workspace = "" }},
{name: "root", mutate: func(r *Request) { r.Root = "" }},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
r := validRequest(t)
tc.mutate(&r)
if err := ValidateRequest(r); err == nil {
t.Fatalf("invalid %s identity accepted: %+v", tc.name, r)
}
})
}
}
func TestDryRunDoesNotInvokeHooks(t *testing.T) {
r := validRequest(t)
called := false
h := Hooks{Snapshot: func(context.Context, Request) (Snapshot, error) { called = true; return Snapshot{}, nil }}
got, err := Execute(context.Background(), r, filepath.Join(r.Root, "handoff.json"), h, true)
if err != nil {
t.Fatal(err)
}
if called || got.Mutates || got.Phase != PhasePrepared {
t.Fatalf("dry run result=%+v called=%v", got, called)
}
}
func TestExecutePersistsPhasesAndCommittedReplayIsNoop(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
var calls int
h := Hooks{
Snapshot: func(context.Context, Request) (Snapshot, error) { calls++; return Snapshot{Sentinel: "s"}, nil },
Configure: func(context.Context, Request, Snapshot) error { calls++; return nil },
StopLegacy: func(context.Context, Request, Snapshot) error { calls++; return nil },
Verify: func(context.Context, Request, Snapshot) error { calls++; return nil },
Commit: func(context.Context, Request, Snapshot) error { calls++; return nil },
}
got, err := Execute(context.Background(), r, path, h, false)
if err != nil || got.Phase != PhaseCommitted || !got.Mutates {
t.Fatalf("result=%+v err=%v", got, err)
}
before := calls
got, err = Execute(context.Background(), r, path, h, false)
if err != nil || got.Phase != PhaseCommitted || calls != before {
t.Fatalf("replay result=%+v calls %d->%d err=%v", got, before, calls, err)
}
}
func TestExecuteHookOrderIsExact(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
var order []string
h := Hooks{
Snapshot: func(context.Context, Request) (Snapshot, error) {
order = append(order, "snapshot")
return Snapshot{Sentinel: "s"}, nil
},
Configure: func(context.Context, Request, Snapshot) error {
order = append(order, "configure")
return nil
},
StopLegacy: func(context.Context, Request, Snapshot) error {
order = append(order, "stop")
return nil
},
Verify: func(context.Context, Request, Snapshot) error {
order = append(order, "verify")
return nil
},
Commit: func(context.Context, Request, Snapshot) error {
order = append(order, "commit")
return nil
},
}
if _, err := Execute(context.Background(), r, path, h, false); err != nil {
t.Fatal(err)
}
want := []string{"snapshot", "configure", "stop", "verify", "commit"}
if strings.Join(order, ",") != strings.Join(want, ",") {
t.Fatalf("hook order = %v, want %v", order, want)
}
}
func TestConfigureRetryPreservesDurableSnapshot(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
var snapshots, configures int
first := Hooks{
Snapshot: func(context.Context, Request) (Snapshot, error) {
snapshots++
return Snapshot{Sentinel: "before-mutation"}, nil
},
Configure: func(context.Context, Request, Snapshot) error {
configures++
return os.ErrPermission
},
}
if _, err := Execute(context.Background(), r, path, first, false); err == nil {
t.Fatal("first configure failure unexpectedly succeeded")
}
second := first
second.Configure = func(_ context.Context, _ Request, s Snapshot) error {
if s.Sentinel != "before-mutation" {
t.Fatalf("retry snapshot = %+v, want durable pre-mutation snapshot", s)
}
return nil
}
second.StopLegacy = func(context.Context, Request, Snapshot) error { return nil }
second.Verify = func(context.Context, Request, Snapshot) error { return nil }
second.Commit = func(context.Context, Request, Snapshot) error { return nil }
if _, err := Execute(context.Background(), r, path, second, false); err != nil {
t.Fatal(err)
}
if snapshots != 1 || configures != 1 {
t.Fatalf("hooks snapshot=%d configure=%d, want one snapshot and one failed configure", snapshots, configures)
}
}
func TestStaleEmptyLockIsRecovered(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
if err := os.WriteFile(path+".lock", nil, 0600); err != nil {
t.Fatal(err)
}
h := Hooks{
Snapshot: func(context.Context, Request) (Snapshot, error) { return Snapshot{}, nil },
Configure: func(context.Context, Request, Snapshot) error { return nil },
StopLegacy: func(context.Context, Request, Snapshot) error { return nil },
Verify: func(context.Context, Request, Snapshot) error { return nil },
Commit: func(context.Context, Request, Snapshot) error { return nil },
}
got, err := Execute(context.Background(), r, path, h, false)
if err != nil || got.Phase != PhaseCommitted {
t.Fatalf("stale lock recovery result=%+v err=%v", got, err)
}
}
func TestConcurrentLockHoldersHaveSingleWinner(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json.lock")
if err := os.WriteFile(path, nil, 0600); err != nil {
t.Fatal(err)
}
held, err := acquireLock(path)
if err != nil {
t.Fatal(err)
}
defer func() {
_ = lockfile.FlockUnlock(held)
_ = held.Close()
}()
if _, err := acquireLock(path); err == nil {
t.Fatalf("second stale reclaimer err=%v, want concurrent refusal", err)
}
}
func TestVerifyFailureReportsMutation(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
h := Hooks{
Snapshot: func(context.Context, Request) (Snapshot, error) { return Snapshot{}, nil },
Configure: func(context.Context, Request, Snapshot) error { return nil },
StopLegacy: func(context.Context, Request, Snapshot) error { return nil },
Verify: func(context.Context, Request, Snapshot) error { return os.ErrClosed },
}
got, err := Execute(context.Background(), r, path, h, false)
if err == nil || !got.Mutates || got.Phase != PhaseOldOwnerStopped {
t.Fatalf("verify failure result=%+v err=%v, want mutation=true at old_owner_stopped", got, err)
}
}
func TestPostHookCompletionCheckpointAvoidsDuplicateHook(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
j := Journal{Request: r, Snapshot: Snapshot{Sentinel: "s"}, SnapshotCaptured: true,
CommitHookRan: true, Phase: PhaseVerified, Owner: OwnerLegacyGC}
b, err := json.Marshal(j)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, append(b, '\n'), 0600); err != nil {
t.Fatal(err)
}
calls := 0
h := Hooks{Commit: func(context.Context, Request, Snapshot) error { calls++; return nil }}
got, err := Execute(context.Background(), r, path, h, false)
if err != nil || got.Phase != PhaseCommitted || calls != 0 {
t.Fatalf("checkpoint replay result=%+v err=%v commit calls=%d", got, err, calls)
}
}
func TestCommitInProgressRequiresIdempotentReplay(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
j := Journal{Request: r, Snapshot: Snapshot{Sentinel: "s"}, SnapshotCaptured: true,
CommitHookInProgress: true, Phase: PhaseVerified, Owner: OwnerLegacyGC}
b, err := json.Marshal(j)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, append(b, '\n'), 0600); err != nil {
t.Fatal(err)
}
calls := 0
h := Hooks{Commit: func(context.Context, Request, Snapshot) error { calls++; return nil }}
got, err := Execute(context.Background(), r, path, h, false)
if err == nil || got.Owner != OwnerLegacyGC || got.Phase != PhaseVerified || got.ErrorCode != "commit_recovery_required" || calls != 0 {
t.Fatalf("ambiguous commit result=%+v err=%v commit calls=%d", got, err, calls)
}
h.CommitReplay = func(context.Context, Request, Snapshot) error { calls++; return nil }
got, err = Execute(context.Background(), r, path, h, false)
if err != nil || got.Phase != PhaseCommitted || calls != 1 {
t.Fatalf("replayed commit result=%+v err=%v commit calls=%d", got, err, calls)
}
}
func TestCommitErrorDoesNotRetryNonIdempotently(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
calls := 0
h := Hooks{
Snapshot: func(context.Context, Request) (Snapshot, error) { return Snapshot{}, nil },
Configure: func(context.Context, Request, Snapshot) error { return nil },
StopLegacy: func(context.Context, Request, Snapshot) error { return nil },
Verify: func(context.Context, Request, Snapshot) error { return nil },
Commit: func(context.Context, Request, Snapshot) error {
calls++
return os.ErrPermission
},
}
first, err := Execute(context.Background(), r, path, h, false)
if err == nil || first.ErrorCode != "commit_failed" || calls != 1 {
t.Fatalf("first commit error result=%+v err=%v calls=%d", first, err, calls)
}
second, err := Execute(context.Background(), r, path, h, false)
if err == nil || second.Owner != OwnerLegacyGC || second.Phase != PhaseVerified || calls != 1 {
t.Fatalf("retry commit error result=%+v err=%v calls=%d", second, err, calls)
}
}
func TestLiveLockRefusesConcurrentHandoff(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
lock, err := acquireLock(path + ".lock")
if err != nil {
t.Fatal(err)
}
defer func() {
_ = lockfile.FlockUnlock(lock)
_ = lock.Close()
}()
got, err := Execute(context.Background(), r, path, Hooks{}, false)
if err == nil || got.ErrorCode != "concurrent_handoff" {
t.Fatalf("live lock result=%+v err=%v", got, err)
}
}
func TestFailureLeavesLegacyOwnerAndRecordsError(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
h := Hooks{Snapshot: func(context.Context, Request) (Snapshot, error) { return Snapshot{}, nil }, Configure: func(context.Context, Request, Snapshot) error { return os.ErrPermission }}
got, err := Execute(context.Background(), r, path, h, false)
if err == nil || got.Owner != OwnerLegacyGC || got.Mutates {
t.Fatalf("result=%+v err=%v", got, err)
}
j, loadErr := Load(path)
if loadErr != nil || j.Phase != PhasePrepared || j.ErrorCode == "" {
t.Fatalf("journal=%+v err=%v", j, loadErr)
}
}
func TestJournalIdentityConflictIsTyped(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
h := Hooks{Snapshot: func(context.Context, Request) (Snapshot, error) {
return Snapshot{Sentinel: "s"}, nil
}}
if _, err := Execute(context.Background(), r, path, h, false); err == nil {
t.Fatal("incomplete handoff unexpectedly succeeded")
}
other := r
other.Database = "other"
got, err := Execute(context.Background(), other, path, Hooks{}, true)
if err == nil || got.ErrorCode != "identity_conflict" || got.Mutates {
t.Fatalf("identity conflict result=%+v err=%v", got, err)
}
}
func TestLoadRejectsUnknownPhase(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
if err := os.WriteFile(path, []byte(`{"request":{},"phase":"bogus","owner":"legacy-gc"}`), 0600); err != nil {
t.Fatal(err)
}
if _, err := Load(path); err == nil {
t.Fatal("accepted unknown journal phase")
}
}
func TestLoadRejectsOwnerPhaseMismatch(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
for _, raw := range []string{
`{"request":{},"phase":"prepared","owner":"bd"}`,
`{"request":{},"phase":"committed","owner":"legacy-gc"}`,
} {
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
t.Fatal(err)
}
if _, err := Load(path); err == nil {
t.Fatalf("accepted owner/phase mismatch %s", raw)
}
}
}
func TestLoadRejectsCommitCheckpointInEarlyPhase(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
for _, raw := range []string{
`{"request":{},"phase":"prepared","owner":"legacy-gc","commit_hook_ran":true}`,
`{"request":{},"phase":"target_configured","owner":"legacy-gc","commit_hook_in_progress":true}`,
`{"request":{},"phase":"verified","owner":"legacy-gc","commit_hook_ran":true,"commit_hook_in_progress":true}`,
} {
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
t.Fatal(err)
}
if _, err := Load(path); err == nil {
t.Fatalf("accepted impossible commit checkpoint %s", raw)
}
}
}
func TestLoadRejectsStopCheckpointInInvalidPhase(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
for _, raw := range []string{
`{"request":{},"phase":"prepared","owner":"legacy-gc","legacy_stop_in_progress":true}`,
`{"request":{},"phase":"old_owner_stopped","owner":"legacy-gc","legacy_stop_in_progress":true}`,
`{"request":{},"phase":"committed","owner":"bd","legacy_stop_in_progress":true}`,
} {
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
t.Fatal(err)
}
if _, err := Load(path); err == nil {
t.Fatalf("accepted impossible stop checkpoint %s", raw)
}
}
}
// TestStopFailureReportsMutation pins the pre-stop reservation: once StopLegacy
// has been entered its effect cannot be assumed absent, so a failed stop must
// not report the legacy owner as untouched.
func TestStopFailureReportsMutation(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
h := Hooks{
Snapshot: func(context.Context, Request) (Snapshot, error) { return Snapshot{}, nil },
Configure: func(context.Context, Request, Snapshot) error { return nil },
StopLegacy: func(context.Context, Request, Snapshot) error { return os.ErrPermission },
}
got, err := Execute(context.Background(), r, path, h, false)
if err == nil || !got.Mutates || got.Phase != PhaseTargetConfigured || got.ErrorCode != "owner_stop_failed" {
t.Fatalf("stop failure result=%+v err=%v, want mutation=true at target_configured", got, err)
}
j, loadErr := Load(path)
if loadErr != nil || !j.LegacyStopInProgress {
t.Fatalf("journal=%+v err=%v, want a durable stop reservation", j, loadErr)
}
}
// TestStopRetryAfterUncheckpointedStopSucceeds covers the crash window between
// a successful StopLegacy and its checkpoint: the retry re-invokes the hook
// against an already-absent legacy owner, which the contract defines as
// success rather than a wedge.
func TestStopRetryAfterUncheckpointedStopSucceeds(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
j := Journal{Request: r, Snapshot: Snapshot{Sentinel: "s"}, SnapshotCaptured: true,
LegacyStopInProgress: true, Phase: PhaseTargetConfigured, Owner: OwnerLegacyGC}
b, err := json.Marshal(j)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, append(b, '\n'), 0600); err != nil {
t.Fatal(err)
}
stops := 0
h := Hooks{
StopLegacy: func(context.Context, Request, Snapshot) error { stops++; return nil },
Verify: func(context.Context, Request, Snapshot) error { return nil },
Commit: func(context.Context, Request, Snapshot) error { return nil },
}
got, execErr := Execute(context.Background(), r, path, h, false)
if execErr != nil || got.Phase != PhaseCommitted || stops != 1 {
t.Fatalf("stop retry result=%+v err=%v stops=%d", got, execErr, stops)
}
final, loadErr := Load(path)
if loadErr != nil || final.LegacyStopInProgress {
t.Fatalf("journal=%+v err=%v, want the stop reservation retired", final, loadErr)
}
}
func TestExecuteRefusesCorruptJournal(t *testing.T) {
for _, dryRun := range []bool{false, true} {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
if err := os.WriteFile(path, []byte("{not json"), 0600); err != nil {
t.Fatal(err)
}
called := false
h := Hooks{Snapshot: func(context.Context, Request) (Snapshot, error) { called = true; return Snapshot{}, nil }}
got, err := Execute(context.Background(), r, path, h, dryRun)
if err == nil || got.ErrorCode != "journal_unreadable" || called {
t.Fatalf("corrupt journal (dryRun=%v) result=%+v err=%v called=%v", dryRun, got, err, called)
}
}
}
// TestDryRunReportsMidFlightJournalPhase keeps the dry-run probe truthful: an
// operator inspecting a handoff that already stopped the legacy owner must not
// be told the scope is still prepared.
func TestDryRunReportsMidFlightJournalPhase(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
j := Journal{Request: r, Snapshot: Snapshot{Sentinel: "s"}, SnapshotCaptured: true,
Phase: PhaseOldOwnerStopped, Owner: OwnerLegacyGC}
b, err := json.Marshal(j)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, append(b, '\n'), 0600); err != nil {
t.Fatal(err)
}
called := false
h := Hooks{Verify: func(context.Context, Request, Snapshot) error { called = true; return nil }}
got, execErr := Execute(context.Background(), r, path, h, true)
if execErr != nil || called || got.Mutates || got.Phase != PhaseOldOwnerStopped {
t.Fatalf("dry run result=%+v err=%v called=%v, want the journal's real phase", got, execErr, called)
}
}
// TestIdentityConflictReportsAccumulatedMutation pins one refusal semantic: a
// conflict against a journal that already stopped the legacy owner reports that
// mutation, whichever side of the lock the conflict is detected on.
func TestIdentityConflictReportsAccumulatedMutation(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
j := Journal{Request: r, Snapshot: Snapshot{Sentinel: "s"}, SnapshotCaptured: true,
Phase: PhaseOldOwnerStopped, Owner: OwnerLegacyGC}
b, err := json.Marshal(j)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, append(b, '\n'), 0600); err != nil {
t.Fatal(err)
}
other := r
other.Database = "other"
got, execErr := Execute(context.Background(), other, path, Hooks{}, false)
if execErr == nil || got.ErrorCode != "identity_conflict" || !got.Mutates {
t.Fatalf("identity conflict result=%+v err=%v, want mutation=true", got, execErr)
}
}
// TestUnopenableLockIsNotReportedAsConcurrency keeps an incident honest: only
// real contention may send an operator hunting a concurrent handoff.
func TestUnopenableLockIsNotReportedAsConcurrency(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "missing-dir", "handoff.json")
got, err := Execute(context.Background(), r, path, Hooks{}, false)
if err == nil || got.ErrorCode != "lock_unavailable" {
t.Fatalf("unopenable lock result=%+v err=%v, want lock_unavailable", got, err)
}
}
// TestCommitRecoveryMessageDoesNotAccumulate pins both bounds of the recovery
// message: it must not re-concatenate onto the journaled error across retries,
// and it must not buy that bound by discarding the original commit failure. A
// fourth attempt reads a journal the recovery path itself wrote, so it proves
// the message is a fixed point rather than merely bounded on the first rebuild.
// The journal is what an operator reads when a handoff is wedged at `verified`,
// so the cause has to survive exactly when a recovery loop has set in.
func TestCommitRecoveryMessageDoesNotAccumulate(t *testing.T) {
r := validRequest(t)
path := filepath.Join(r.Root, "handoff.json")
h := Hooks{
Snapshot: func(context.Context, Request) (Snapshot, error) { return Snapshot{}, nil },
Configure: func(context.Context, Request, Snapshot) error { return nil },
StopLegacy: func(context.Context, Request, Snapshot) error { return nil },
Verify: func(context.Context, Request, Snapshot) error { return nil },
Commit: func(context.Context, Request, Snapshot) error { return os.ErrPermission },
}
var lastErr error
for i := 0; i < 4; i++ {
if _, lastErr = Execute(context.Background(), r, path, h, false); lastErr == nil {
t.Fatalf("attempt %d unexpectedly succeeded", i)
}
}
j, err := Load(path)
if err != nil {
t.Fatal(err)
}
if n := strings.Count(j.Error, "commit hook outcome is unknown"); n != 1 {
t.Fatalf("journaled error repeated the recovery message %d times: %q", n, j.Error)
}
if !strings.Contains(j.Error, os.ErrPermission.Error()) {
t.Fatalf("journaled error dropped the original commit failure cause: %q", j.Error)
}
if !strings.Contains(lastErr.Error(), os.ErrPermission.Error()) {
t.Fatalf("returned error dropped the original commit failure cause: %v", lastErr)
}
}
func TestJournalSaveErrorIsTypedAndTruthful(t *testing.T) {
j := Journal{Phase: PhaseOldOwnerStopped, Owner: OwnerLegacyGC}
got, err := journalSaveError(j, os.ErrPermission)
if err == nil || got.ErrorCode != "journal_save_failed" || !got.Mutates {
t.Fatalf("journal save failure result=%+v err=%v", got, err)
}
}