beads/cmd/bd/backup_lock_test.go
Julian Knutsen 4e4ebdb443
feat(backup): proxied auto-backup on managed-local, serialized with bd backup sync (#6879)
* feat(backup): run opted-in auto-backup on a managed-local proxied server

#6584 routed the five `bd backup` verbs over the proxied provider on
managed-local, and #6694 made `bd backup status` admit the remaining gap:
the proxied arm of PersistentPostRunE never called runPostRunAutoBackup,
so backup.enabled=true was inert on every proxied workspace. Close it.

The proxied arm now calls the auto-backup hook before it closes the
provider, under the direct arm's maintenance gate (strict --readonly,
`bd serve`, a migration freeze) plus previews. maybeAutoBackup picks a
backend for the command: the direct store as before, or the proxied
provider's RunNonTx seam, which `bd backup sync` already uses.

The proxied backend is honored exactly where `bd backup sync` is: the
"backup sync" capability row's rule for this workspace's topology, so
managed-local only. Auto-backup registers a server-side backup remote,
which on a server bd does not own is global to every client; an implicit
trigger must not get further than the explicit verb. It never opens a
provider of its own: a command that opened none (`bd dolt stop`) or
closed it (restore) backs nothing up rather than relaunching the server.
The default stays OFF, as on every server-mode workspace.

The register/sync-to-a-local-dir sequence that DoltStore and
EmbeddedDoltStore each carried a copy of moves to
versioncontrolops.BackupToDir (and HEAD lookup to CurrentCommit), so the
proxied backend reuses it instead of adding a third copy.

Tests: TestManagedLocalProxiedAutoBackup (default off, opt-in writes a
restorable backup, change detection, `bd dolt stop` stays quiescent,
restore round trip with issue-set equality; fails against origin/main);
TestManagedLocalProxiedBackupRestoreRefusedWhileAttached pins the restore
design (refused while another bd client holds the workspace, serving
child untouched, succeeds once it detaches); unit tests for the backend
locality gate, the post-run wiring, the status note and BackupToDir.
Both managed-local tests, and the existing round trip, are added to the
proxied-local smoke lane's compiled-in list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(backup): serialize auto-backup and bd backup sync with a workspace backup lock

Review follow-up on #6879. Auto-backup and an explicit bd backup sync
both run under the command's SHARED workspace gate, so nothing stopped
them overlapping, and auto-backup's remove -> add -> sync on one fixed
remote name races a concurrent copy of itself.

A per-workspace flock (.beads/backup.lock, already gitignored by
*.lock) now serializes them in every mode. Auto-backup takes it
non-blocking and skips when held (the throttle is left alone, so a
later command retries). bd backup sync waits up to exclusiveGateWait -
the bound restore's exclusive gate acquisition uses - then fails with
"another backup is running for this workspace" before touching the
store. Restore needs no part in it: its exclusive workspace gate
already excludes every command that could hold the lock.

Tests: lock semantics (refusal while held, bounded wait, release),
auto-backup skipping while held and running once released, and sync
failing loudly without reaching the store.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 02:45:58 -07:00

133 lines
3.8 KiB
Go

package main
import (
"context"
"errors"
"os"
"strings"
"testing"
"time"
)
// TestBackupLockSerializesBackups pins the lock itself: a second holder is
// refused while the first holds it, a bounded wait gives up with errBackupBusy,
// and releasing makes it available again.
//
// Cannot be parallel: prepareBackupStatusTest mutates package globals.
func TestBackupLockSerializesBackups(t *testing.T) {
prepareBackupStatusTest(t)
release, err := acquireBackupLock(0)
if err != nil {
t.Fatalf("first acquisition: %v", err)
}
if _, err := acquireBackupLock(0); !errors.Is(err, errBackupBusy) {
t.Fatalf("non-blocking acquisition while held = %v, want errBackupBusy", err)
}
start := time.Now()
if _, err := acquireBackupLock(150 * time.Millisecond); !errors.Is(err, errBackupBusy) {
t.Fatalf("bounded wait while held = %v, want errBackupBusy", err)
}
if waited := time.Since(start); waited < 100*time.Millisecond {
t.Fatalf("bounded wait returned after %s; it must poll for the wait it was given", waited)
}
release()
again, err := acquireBackupLock(0)
if err != nil {
t.Fatalf("acquisition after release: %v", err)
}
again()
}
// TestAutoBackupSkipsWhileAnotherBackupRuns: auto-backup never waits on the
// lock. With it held the backup does not run and the throttle state is left
// alone, so the next command retries; once released, the same call backs up.
func TestAutoBackupSkipsWhileAnotherBackupRuns(t *testing.T) {
prepareBackupStatusTest(t)
t.Setenv("BD_BACKUP_ENABLED", "true")
t.Setenv("BD_BACKUP_INTERVAL", "1ms")
t.Setenv("BD_GIT_HOOK", "")
initConfigForTest(t)
oldStore := store
fake := &failingBackupStore{commit: "c1"}
store = fake
t.Cleanup(func() { store = oldStore })
release, err := acquireBackupLock(0)
if err != nil {
t.Fatalf("hold the backup lock: %v", err)
}
maybeAutoBackup(context.Background())
if fake.backupCalls != 0 {
t.Fatalf("auto-backup ran %d backup(s) while another backup held the lock", fake.backupCalls)
}
dir, err := backupDir()
if err != nil {
t.Fatal(err)
}
if st, _ := loadBackupState(dir); !st.Timestamp.IsZero() {
t.Fatalf("a skipped auto-backup advanced the throttle to %v", st.Timestamp)
}
release()
maybeAutoBackup(context.Background())
if fake.backupCalls != 1 {
t.Fatalf("auto-backup ran %d backup(s) after the lock was released, want 1", fake.backupCalls)
}
}
// TestBackupSyncFailsWhenAnotherBackupHoldsTheLock: an explicit sync waits
// backupLockWait for another backup and then fails loudly, before touching the
// store — it never reports a sync it did not do.
func TestBackupSyncFailsWhenAnotherBackupHoldsTheLock(t *testing.T) {
prepareBackupStatusTest(t)
oldWait := backupLockWait
backupLockWait = 100 * time.Millisecond
t.Cleanup(func() { backupLockWait = oldWait })
oldStore := store
fake := &failingBackupStore{commit: "c1"}
store = fake
t.Cleanup(func() { store = oldStore })
release, err := acquireBackupLock(0)
if err != nil {
t.Fatalf("hold the backup lock: %v", err)
}
defer release()
var runErr error
stderr := captureStderrForBackupLockTest(t, func() {
runErr = backupSyncCmd.RunE(backupSyncCmd, nil)
})
if runErr == nil {
t.Fatal("bd backup sync succeeded while another backup held the lock")
}
if !strings.Contains(stderr, "another backup is running") {
t.Fatalf("sync refusal does not say why; stderr:\n%s", stderr)
}
if fake.backupCalls != 0 {
t.Fatalf("refused sync still reached the store (%d backup calls)", fake.backupCalls)
}
}
func captureStderrForBackupLockTest(t *testing.T, fn func()) string {
t.Helper()
stdioMutex.Lock()
defer stdioMutex.Unlock()
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
old := os.Stderr
os.Stderr = w
fn()
os.Stderr = old
_ = w.Close()
buf := make([]byte, 1<<16)
n, _ := r.Read(buf)
_ = r.Close()
return string(buf[:n])
}