beads/cmd/bd/backup_export.go
Julian Knutsen 4e4ebdb443
feat(backup): proxied auto-backup on managed-local, serialized with bd backup sync (#6879)
* feat(backup): run opted-in auto-backup on a managed-local proxied server

#6584 routed the five `bd backup` verbs over the proxied provider on
managed-local, and #6694 made `bd backup status` admit the remaining gap:
the proxied arm of PersistentPostRunE never called runPostRunAutoBackup,
so backup.enabled=true was inert on every proxied workspace. Close it.

The proxied arm now calls the auto-backup hook before it closes the
provider, under the direct arm's maintenance gate (strict --readonly,
`bd serve`, a migration freeze) plus previews. maybeAutoBackup picks a
backend for the command: the direct store as before, or the proxied
provider's RunNonTx seam, which `bd backup sync` already uses.

The proxied backend is honored exactly where `bd backup sync` is: the
"backup sync" capability row's rule for this workspace's topology, so
managed-local only. Auto-backup registers a server-side backup remote,
which on a server bd does not own is global to every client; an implicit
trigger must not get further than the explicit verb. It never opens a
provider of its own: a command that opened none (`bd dolt stop`) or
closed it (restore) backs nothing up rather than relaunching the server.
The default stays OFF, as on every server-mode workspace.

The register/sync-to-a-local-dir sequence that DoltStore and
EmbeddedDoltStore each carried a copy of moves to
versioncontrolops.BackupToDir (and HEAD lookup to CurrentCommit), so the
proxied backend reuses it instead of adding a third copy.

Tests: TestManagedLocalProxiedAutoBackup (default off, opt-in writes a
restorable backup, change detection, `bd dolt stop` stays quiescent,
restore round trip with issue-set equality; fails against origin/main);
TestManagedLocalProxiedBackupRestoreRefusedWhileAttached pins the restore
design (refused while another bd client holds the workspace, serving
child untouched, succeeds once it detaches); unit tests for the backend
locality gate, the post-run wiring, the status note and BackupToDir.
Both managed-local tests, and the existing round trip, are added to the
proxied-local smoke lane's compiled-in list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(backup): serialize auto-backup and bd backup sync with a workspace backup lock

Review follow-up on #6879. Auto-backup and an explicit bd backup sync
both run under the command's SHARED workspace gate, so nothing stopped
them overlapping, and auto-backup's remove -> add -> sync on one fixed
remote name races a concurrent copy of itself.

A per-workspace flock (.beads/backup.lock, already gitignored by
*.lock) now serializes them in every mode. Auto-backup takes it
non-blocking and skips when held (the throttle is left alone, so a
later command retries). bd backup sync waits up to exclusiveGateWait -
the bound restore's exclusive gate acquisition uses - then fails with
"another backup is running for this workspace" before touching the
store. Restore needs no part in it: its exclusive workspace gate
already excludes every command that could hold the lock.

Tests: lock semantics (refusal while held, bounded wait, release),
auto-backup skipping while held and running once released, and sync
failing loudly without reaching the store.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 02:45:58 -07:00

214 lines
7.2 KiB
Go

package main
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/config"
"github.com/steveyegge/beads/internal/debug"
"github.com/steveyegge/beads/internal/storage"
)
// backupState tracks watermarks for backup change detection.
type backupState struct {
LastDoltCommit string `json:"last_dolt_commit"`
Timestamp time.Time `json:"timestamp"`
}
// backupDir returns the backup directory path, creating it if needed.
// When backup.git-repo is set to a valid git repo, returns a backup/ subdirectory
// inside that repo. Otherwise it requires an active beads workspace and uses its
// backup/ subdirectory.
func backupDir() (string, error) {
gitRepo := config.GetString("backup.git-repo")
if gitRepo != "" {
if strings.HasPrefix(gitRepo, "~/") {
home, _ := os.UserHomeDir()
gitRepo = filepath.Join(home, gitRepo[2:])
}
if _, err := os.Stat(filepath.Join(gitRepo, ".git")); err != nil {
fmt.Fprintf(os.Stderr, "Warning: backup.git-repo %s is not a git repo, falling back to .beads/backup\n", gitRepo)
} else {
dir := filepath.Join(gitRepo, "backup")
if err := os.MkdirAll(dir, 0700); err != nil {
return "", fmt.Errorf("failed to create backup dir in git-repo: %w", err)
}
return dir, nil
}
}
beadsDir := beads.FindBeadsDir()
if beadsDir == "" {
return "", fmt.Errorf("%s; %s", activeWorkspaceNotFoundError(), diagHint())
}
dir := filepath.Join(beadsDir, "backup")
if err := os.MkdirAll(dir, 0700); err != nil {
return "", fmt.Errorf("failed to create backup directory: %w", err)
}
return dir, nil
}
// loadBackupState reads the backup state file, returning a zero state if missing.
func loadBackupState(dir string) (*backupState, error) {
path := filepath.Join(dir, "backup_state.json")
data, err := os.ReadFile(path) //nolint:gosec // path is constructed internally
if os.IsNotExist(err) {
return &backupState{}, nil
}
if err != nil {
return nil, fmt.Errorf("failed to read backup state: %w", err)
}
var state backupState
if err := json.Unmarshal(data, &state); err != nil {
return nil, fmt.Errorf("failed to parse backup state: %w", err)
}
return &state, nil
}
// saveBackupState writes the backup state file atomically.
func saveBackupState(dir string, state *backupState) error {
data, err := json.MarshalIndent(state, "", " ")
if err != nil {
return fmt.Errorf("failed to marshal backup state: %w", err)
}
return atomicWriteFile(filepath.Join(dir, "backup_state.json"), data)
}
// atomicWriteFile writes data to a same-directory temp file, fsyncs the
// temp file's own contents, then renames it into place. This avoids a
// truncated/partial file at path if the process crashes mid-write.
//
// Two caveats this does NOT cover, narrowing the "crash-safe" claim rather
// than the implementation (existing callers' behavior is unchanged here):
// - Only the temp file's contents are fsynced, not the parent directory
// entry; a crash between the rename and a subsequent directory fsync
// can still lose the rename itself on some filesystems.
// - os.Rename's atomic-replace guarantee is a POSIX/Unix property; it is
// not guaranteed on Windows. It also does not follow a symlink at
// path — it replaces whatever is there, symlink or not — so a caller
// that must preserve a symlink's target should resolve path with
// filepath.EvalSymlinks first (see cmd/bd/proxied_server.go).
func atomicWriteFile(path string, data []byte) error {
dir := filepath.Dir(path)
tmp, err := os.CreateTemp(dir, ".backup-tmp-*")
if err != nil {
return fmt.Errorf("failed to create temp file: %w", err)
}
tmpPath := tmp.Name()
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
_ = os.Remove(tmpPath)
return fmt.Errorf("failed to write temp file: %w", err)
}
if err := tmp.Sync(); err != nil {
_ = tmp.Close()
_ = os.Remove(tmpPath)
return fmt.Errorf("failed to sync temp file: %w", err)
}
if err := tmp.Close(); err != nil {
_ = os.Remove(tmpPath)
return fmt.Errorf("failed to close temp file: %w", err)
}
if err := os.Rename(tmpPath, path); err != nil {
_ = os.Remove(tmpPath)
return fmt.Errorf("failed to rename temp file: %w", err)
}
return nil
}
// localBackupBackend is what a Dolt-native backup into a local directory needs
// from storage: the commit it would capture, and the backup itself. The direct
// store supplies one (directLocalBackup) and so does the proxied provider
// (proxiedLocalBackup, backup_proxied_server.go).
type localBackupBackend interface {
CurrentCommit(ctx context.Context) (string, error)
BackupToDir(ctx context.Context, dir string) error
}
// directLocalBackup is localBackupBackend over an embedded or sql-server store.
type directLocalBackup struct {
store storage.DoltStorage
}
func (b directLocalBackup) CurrentCommit(ctx context.Context) (string, error) {
return b.store.GetCurrentCommit(ctx)
}
func (b directLocalBackup) BackupToDir(ctx context.Context, dir string) error {
bs, ok := storage.UnwrapStore(b.store).(storage.BackupStore)
if !ok {
return fmt.Errorf("storage backend does not support backup operations")
}
return bs.BackupDatabase(ctx, dir)
}
// runBackupExport performs a Dolt-native backup to .beads/backup/.
// Returns the updated state.
func runBackupExport(ctx context.Context, backend localBackupBackend, force bool) (*backupState, error) {
dir, err := backupDir()
if err != nil {
return nil, err
}
state, err := loadBackupState(dir)
if err != nil {
return nil, err
}
// Change detection: skip if nothing changed (unless forced)
if !force {
currentCommit, err := backend.CurrentCommit(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get current commit: %w", err)
}
if currentCommit == state.LastDoltCommit && state.LastDoltCommit != "" {
debug.Logf("backup: no changes since last backup (commit %s)\n", truncateHash(currentCommit))
return state, nil
}
}
if err := backend.BackupToDir(ctx, dir); err != nil {
// Persist the attempt time even on failure so the throttle
// interval (checked by maybeAutoBackup via state.Timestamp)
// applies to the next command. Without this, a sync that keeps
// failing — e.g. a slow/overloaded shared Dolt server — retries
// on EVERY bd command instead of once per interval, turning a
// transient slowdown into a self-amplifying storm (the 2026-07
// shared-dolt CPU-pin incident). LastDoltCommit is deliberately
// left unchanged so change-detection still sees pending work and
// a real backup runs once the failure clears.
state.Timestamp = time.Now().UTC()
if saveErr := saveBackupState(dir, state); saveErr != nil {
debug.Logf("backup: failed to persist throttle state after error: %v\n", saveErr)
}
return nil, err
}
// Update watermarks
currentCommit, err := backend.CurrentCommit(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get current commit for state: %w", err)
}
state.LastDoltCommit = currentCommit
state.Timestamp = time.Now().UTC()
if err := saveBackupState(dir, state); err != nil {
return nil, err
}
return state, nil
}
// truncateHash returns the first 8 characters of a hash, or the full string if shorter.
func truncateHash(h string) string {
if len(h) > 8 {
return h[:8]
}
return h
}