beads/cmd/bd/version_tracking.go
Jim Wordelman ae48b0a090
perf(dolt): eliminate 12s slow path on bd commands when .local_version is stale (#3710)
* perf(dolt): be-1he eliminate 12s `dolt remote -v` against multi-db server roots

Three layers of defense for the bd 12s slow-path bug:

- Layer 1: migrateServerRootRemotes skips when rootDir/.dolt/repo_state.json
  is missing. Multi-database server roots have only sql-server.info; running
  `dolt remote -v` there takes ~12s to fail.
- Layer 2: ListCLIRemotes wraps the dolt subprocess in a 2s context timeout.
  Backstops Layer 1 against any future variant of the broken-parent-dir case.
- Layer 3: autoMigrateOnVersionBump does a read-only probe of bd_version
  before opening writeable. Read-only stores skip syncCLIRemotesToSQL, so a
  stale .local_version (e.g. from a multi-binary install) no longer triggers
  the slow path on every command.

End-user impact pre-fix: gc mail inbox 60-80s, gc status 30s+ hangs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: release gate PASS for be-1he (3-layer slow-path fix)

* docs(repro): add be-1he slow-path repro script (be-vxngmn)

Adds scripts/repro-be-1he-slow-path/repro.sh + REPRO.md demonstrating
the 12-second `dolt remote -v` slow path on multi-DB server roots.

Satisfies coffeegoddd's review request: "We need a beads-only repro of
this to prove its a bug."

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(cli): regenerate CLI reference after slow-path fix rebase

Output of ./scripts/generate-cli-docs.sh ./bd and ./scripts/generate-llms-full.sh.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(cli): regenerate CLI reference + llms-full with pure-go bd

The previous regen (0aeb5934e) was built with a CGO-enabled bd, which
includes the CGO-only `bd federation` subcommands. CI builds the docs
binary with CGO_ENABLED=0 (scripts/ci/pr-policy.sh build_docs_binary),
so check-doc-flags rejected the extra federation subcommand pages.

Regenerated docs/CLI_REFERENCE.md, the federation cli-reference pages
(live + versioned 1.0.0/1.0.4/1.0.5), and website/static/llms-full.txt
with a CGO_ENABLED=0 binary. check-doc-flags + check-doc-freshness now
pass locally under LC_ALL=C.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(release-gate): correct be-1he layer description to match shipped diff

The PR body and this gate doc both described a "Layer 1" repo_state.json
sentinel in internal/storage/dolt/federation.go, but the actual diff never
touched that file - only cmd/bd/version_tracking.go (Layer 3) and
internal/storage/doltutil/remotes.go (Layer 2) are changed. Correct the
gate doc's commit table, acceptance-criteria row, and follow-up/out-of-scope
notes to describe only what ships. No code or test changes; Layer 2 and
Layer 3 remain independently verified as still-needed on current main
(ListCLIRemotes has no timeout, no equivalent sentinel exists).

Agent-Signature: claude-sonnet-medium on behalf of maphew

* fix(dolt): scope ListCLIRemotes timeout, fix repro.sh, purge Layer 1 narrative

Addresses the cross-vendor review's four should-fixes plus the comment nit
(gastownhall/beads#3710 review, 2026-07-24):

1. ListCLIRemotes' 2s cap could turn a slow-but-valid dolt remote -v into
   "remote absent": FindCLIRemote folds any error (including a timeout) into
   "", and EnsureCLIRemote then blind-adds on that signal, which hard-fails
   if the remote actually exists. Chose the "scope the cap" fix over
   "propagate DeadlineExceeded distinctly": it's fully contained inside
   ListCLIRemotes (no changes needed at the three call sites - remotes.go:192
   ensure-path, store.go:1958 URL match via hasMatchingCLIRemote, and
   doctor/federation.go:127, which already treats errors distinctly from "no
   remotes"), whereas threading a sentinel error type through FindCLIRemote's
   string-returning signature and its callers touches more surface for the
   same outcome. ListCLIRemotes now stats dbPath/.dolt/repo_state.json: 2s
   cap when absent (the known broken multi-DB server-root case - there's
   never a real answer coming from there), 30s cap otherwise (a real repo's
   `dolt remote -v` is ~130ms even loaded, so 30s only bites a genuinely hung
   subprocess). Added TestListCLIRemotesTimeout covering both branches,
   following the t.TempDir() pattern already used in
   persisted_remotes_test.go.

2. repro.sh's baseline `dolt remote -v` ran in the invocation directory, not
   $SERVER_ROOT, so the central "slow" measurement never exercised the
   constructed broken root. Fixed to `(cd "$SERVER_ROOT" && dolt remote -v
   2>&1) || true`, matching the correct subshell pattern already used later
   in the script. Also relabeled the "PARTIAL FIX ... Layer 2 timeout capped
   it" branch: the 2s cap lives in bd's Go code, not dolt, and this script
   calls the raw dolt binary directly - it's never capped. Relabeled to
   describe the underlying dolt slowness the script demonstrates instead of
   implying bd's timeout is in play here.

3. `dolt init --quiet` isn't supported by dolt 2.2.x and set -e killed the
   script before it reached the summary. Redirected output instead
   (`dolt init >/dev/null 2>&1`). Ran repro.sh end-to-end locally (dolt
   2.2.0): completes with exit 0, and the fixed subshell now shows real
   timing/output from inside $SERVER_ROOT.

4. Purged the unshipped "Layer 1" (federation.go sentinel) narrative from
   repro.sh and REPRO.md - same reconciliation already done for the release
   gate doc and PR body. Both files now describe only the two layers that
   ship (remotes.go timeout, version_tracking.go read-only probe), with a
   short note that an earlier draft described a Layer 1 that never landed.

5. cmd/bd/version_tracking.go:195-197 comment cited syncCLIRemotesToSQL,
   which doesn't exist anywhere in the tree, and overstated the read-only
   probe's benefit. On current main the writeable-open gate reads remotes
   via doltutil.PersistedRemotes (a fast on-disk repo_state.json probe, no
   dolt subprocess), so the probe saves an initSchema round-trip, not a 12s
   hang. Corrected.

Agent-Signature: claude-sonnet-medium on behalf of maphew

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-24 00:53:06 -07:00

281 lines
9.6 KiB
Go

package main
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/steveyegge/beads/cmd/bd/doctor"
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/debug"
"github.com/steveyegge/beads/internal/doltserver"
"github.com/steveyegge/beads/internal/storage/dolt"
)
// localVersionFile is the gitignored file that stores the last bd version used locally.
// This prevents the upgrade notification from firing repeatedly when git operations
// reset the tracked metadata.json file.
const localVersionFile = ".local_version"
// trackBdVersion checks if bd version has changed since last run and updates the local version file.
// This function is best-effort - failures are silent to avoid disrupting commands.
// Sets global variables versionUpgradeDetected and previousVersion if upgrade detected.
func trackBdVersion() {
// Find the beads directory
beadsDir := beads.FindBeadsDir()
if beadsDir == "" {
// No .beads directory found - this is fine (e.g., bd init, bd version, etc.)
return
}
// Read last version from local (gitignored) file
localVersionPath := filepath.Join(beadsDir, localVersionFile)
lastVersion := readLocalVersion(localVersionPath)
// Check if version changed (only flag actual upgrades, not downgrades)
if lastVersion != "" && lastVersion != Version {
if doctor.CompareVersions(Version, lastVersion) > 0 {
// Version upgrade detected!
versionUpgradeDetected = true
previousVersion = lastVersion
}
}
// Update local version file (best effort)
// Only write if version actually changed to minimize I/O
if lastVersion != Version {
_ = writeLocalVersion(localVersionPath, Version) // Best effort: version tracking is advisory
}
}
// readLocalVersion reads the last bd version from the local version file.
// Returns empty string if file doesn't exist or can't be read.
func readLocalVersion(path string) string {
// #nosec G304 - path is constructed from beadsDir + constant
data, err := os.ReadFile(path)
if err != nil {
return ""
}
return strings.TrimSpace(string(data))
}
// writeLocalVersion writes the current version to the local version file.
func writeLocalVersion(path, version string) error {
return os.WriteFile(path, []byte(version+"\n"), 0600)
}
// getVersionsSince returns all version changes since the given version.
// If sinceVersion is empty, returns all known versions.
// Returns changes in chronological order (oldest first).
//
// Note: versionChanges array is in reverse chronological order (newest first),
// so we return elements before the found index and reverse the slice.
func getVersionsSince(sinceVersion string) []VersionChange {
if sinceVersion == "" {
// Return all versions (already in reverse chronological, but kept for compatibility)
return versionChanges
}
// Find the index of sinceVersion
// versionChanges is ordered newest-first: [0.23.0, 0.22.1, 0.22.0, 0.21.0]
startIdx := -1
for i, vc := range versionChanges {
if vc.Version == sinceVersion {
startIdx = i
break
}
}
if startIdx == -1 {
// sinceVersion not found in our changelog - return all versions
// (user might be upgrading from a very old version)
return versionChanges
}
if startIdx == 0 {
// Already on the newest version
return []VersionChange{}
}
// Return versions before sinceVersion (those are newer)
// Then reverse to get chronological order (oldest first)
newerVersions := versionChanges[:startIdx]
// Reverse the slice to get chronological order
result := make([]VersionChange, len(newerVersions))
for i := range newerVersions {
result[i] = newerVersions[len(newerVersions)-1-i]
}
return result
}
// maybeShowUpgradeNotification displays a one-time upgrade notification if version changed.
// This is called by commands like 'bd ready' and 'bd list' to inform users of upgrades.
func maybeShowUpgradeNotification() {
// Only show if upgrade detected and not yet acknowledged
if !versionUpgradeDetected || upgradeAcknowledged {
return
}
// Mark as acknowledged so we only show once per session
upgradeAcknowledged = true
// Display notification
fmt.Printf("🔄 bd upgraded from v%s to v%s since last use\n", previousVersion, Version)
fmt.Println("💡 Run 'bd upgrade review' to see what changed")
if usesSQLServer() {
fmt.Println("💊 Run 'bd doctor' to verify upgrade completed cleanly")
}
fmt.Println()
}
// autoMigrateOnVersionBump automatically migrates the database when CLI version changes.
// This function is best-effort - failures are silent to avoid disrupting commands.
// Called from PersistentPreRun before opening DB for main operation.
//
// IMPORTANT: This must be called BEFORE opening the database to avoid opening DB twice.
//
// beadsDir is the path to the .beads directory.
func autoMigrateOnVersionBump(beadsDir string) {
// Only migrate if version upgrade was detected
if !versionUpgradeDetected {
return
}
// Validate beadsDir
if beadsDir == "" {
debug.Logf("auto-migrate: skipping migration, no beads directory")
return
}
// Load config to determine the correct database path for this backend
cfg, err := configfile.Load(beadsDir)
if err != nil {
debug.Logf("auto-migrate: failed to load config: %v", err)
return
}
if cfg == nil {
cfg = configfile.DefaultConfig()
}
if cfg.GetBackend() != configfile.BackendDolt {
debug.Logf("auto-migrate: skipping Dolt migration for backend %q", cfg.GetBackend())
return
}
if cfg.IsDoltProxiedServerMode() {
debug.Logf("auto-migrate: skipping embedded migration, proxied-server handled after UOW provider init")
return
}
// Check if database exists at the backend-appropriate path
dbPath := cfg.DatabasePath(beadsDir)
if _, err := os.Stat(dbPath); os.IsNotExist(err) {
// No database - nothing to migrate
debug.Logf("auto-migrate: skipping migration, database does not exist: %s", dbPath)
return
}
// GH#2137: If upgrading from pre-0.56, the dolt database may have been
// created by the old embedded Dolt mode. Recover by reinitializing.
if previousVersion != "" && doctor.CompareVersions(previousVersion, "0.56.0") < 0 {
recovered, recErr := doltserver.RecoverPreV56DoltDir(dbPath)
if recErr != nil {
debug.Logf("auto-migrate: pre-v56 recovery failed: %v", recErr)
}
if recovered {
debug.Logf("auto-migrate: rebuilt pre-v56 dolt database at %s", dbPath)
}
}
// Open database using factory (respects backend config from metadata.json)
// Use rootCtx if available and not canceled, otherwise use Background
ctx := rootCtx
if ctx == nil || ctx.Err() != nil {
// rootCtx is nil or canceled - use fresh background context
ctx = context.Background()
}
// Read-only probe: if the DB is already at the current version, skip the
// writeable open. On current main the writeable-open gate reads remotes
// via doltutil.PersistedRemotes, a fast on-disk repo_state.json probe —
// not a `dolt remote -v` subprocess — so this doesn't save a 12s hang;
// it saves an unnecessary initSchema round-trip when no migration is
// needed. (be-1he)
if roStore, roErr := dolt.NewFromConfigWithOptions(ctx, beadsDir, &dolt.Config{ReadOnly: true}); roErr == nil {
dbVersion, _ := roStore.GetLocalMetadata(ctx, "bd_version")
_ = roStore.Close()
if dbVersion == Version {
debug.Logf("auto-migrate: database already at version %s (ro probe)", Version)
return
}
}
store, err := dolt.NewFromConfig(ctx, beadsDir)
if err != nil {
// Failed to open database - skip migration
debug.Logf("auto-migrate: failed to open database: %v", err)
return
}
// Get current database version (clone-local, dolt-ignored)
dbVersion, err := store.GetLocalMetadata(ctx, "bd_version")
if err != nil {
// Failed to read version - skip migration
debug.Logf("auto-migrate: failed to read database version: %v", err)
_ = store.Close() // Best effort cleanup on error path
return
}
// Check if migration is needed
if dbVersion == Version {
// Database is already at current version
debug.Logf("auto-migrate: database already at version %s", Version)
_ = store.Close() // Best effort cleanup on error path
return
}
// Check for downgrade: refuse to overwrite a newer version with an older one (gt-e3uiy)
maxVersion, _ := store.GetLocalMetadata(ctx, "bd_version_max")
if dbVersion != "" && doctor.CompareVersions(Version, dbVersion) < 0 {
debug.Logf("auto-migrate: refusing downgrade from %s to %s", dbVersion, Version)
_ = store.Close() // Best effort cleanup on error path
return
}
if maxVersion != "" && doctor.CompareVersions(Version, maxVersion) < 0 {
debug.Logf("auto-migrate: refusing downgrade (max version %s > current %s)", maxVersion, Version)
_ = store.Close() // Best effort cleanup on error path
return
}
// Perform migration: update database version
debug.Logf("auto-migrate: migrating database from %s to %s", dbVersion, Version)
if err := store.SetLocalMetadata(ctx, "bd_version", Version); err != nil {
// Migration failed - log and continue
debug.Logf("auto-migrate: failed to update database version: %v", err)
_ = store.Close() // Best effort cleanup on error path
return
}
// Update max version tracking
if maxVersion == "" || doctor.CompareVersions(Version, maxVersion) > 0 {
if err := store.SetLocalMetadata(ctx, "bd_version_max", Version); err != nil {
debug.Logf("auto-migrate: failed to update max version: %v", err)
}
}
// No Dolt commit needed — local_metadata is dolt-ignored and persists
// in the working set for the lifetime of the server session.
// Close database
if err := store.Close(); err != nil {
debug.Logf("auto-migrate: warning: failed to close database: %v", err)
}
debug.Logf("auto-migrate: successfully migrated database to version %s", Version)
}