beads/internal/doltversion/cache.go
matt wilkie 21834e720d
feat(doltversion): dolt CLI discovery, hardened version probe, and warn-only version policy (contract part 1) (#5092)
* feat(doltversion): own dolt CLI discovery, hardened probe, and version policy in a leaf package

PR-1 of the dolt CLI install/version contract (design:
mybd reports/2026-07-25-psxg4-dolt-cli-contract-design.md). The dolt
executable managed proxied-server mode spawns was discovered by a bare
exec.LookPath with no override, probed nowhere, and version-checked only
by one advisory archive_level probe; a missing binary erred after
.beads/ creation and an incompatible one died opaquely.

internal/doltversion (stdlib-only) now owns:
- resolution order BEADS_DOLT_BIN env > clone-local sidecar (hook point;
  the sidecar itself lands with identity threading in PR-2) > PATH.
  Explicit values are absolutized before validation (a bare name would
  otherwise validate the cwd file but exec a PATH lookup) and an
  explicitly-named broken binary is an error, never a silent
  fall-through;
- a hardened probe: timeout + process kill, 64KiB output caps,
  unconditional symlink resolution recording given and real paths,
  errno-based exec-format diagnostics naming the likely arch mismatch,
  and file identity captured for later change detection;
- version parsing generalized from the archive_level probe, tolerating
  prerelease/dev suffixes;
- policy: NO hard version floor (adversarial review rejected 2.2.0 as
  policy-without-evidence); RecommendedMin 2.0.0 warns only, backed by
  the 2026-07-25 cross-version matrix run (cross-read of module-written
  storage requires >= 2.0.0). An unparseable version string is likewise
  a warning, not a refusal - the only hard failures are a missing,
  non-executable, or probe-failing binary.

Wired at the two entry points: the managed proxied UOW factory replaces
its bare LookPath and derives archive-level support from the probe it
already ran (no second dolt-version fork on the per-command hot path;
warning is printed once, suppressed under --quiet/--json), and bd init's
proxied preflight fails before any .beads/ write when the binary is
missing or broken (external proxied config is exempt - bd never spawns
that server). doltserver.Start's own bare LookPath is deliberately
untouched here; identity threading through every spawn site is PR-2.

Agent-Signature: claude-fable-5-medium on behalf of matt wilkie
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AQHiBWVPSXDCcUSG9EGw9d

* fix(doltversion): fail closed on prerelease archive_level; dedupe dolt advisory

Two review findings on this PR (see reports/2026-07-26-reviewer-dispatch-sweep.md
#5092 for the adversarial-review reasoning):

1. archive_level fail-open regression (major, re gastownhall/beads#4986).
   newManagedProxiedServerUOWProvider derived archiveLevelSupported straight
   from doltID.Version.AtLeast(MinDoltVersionForArchiveLevelConfig), but
   AtLeast deliberately ignores Version.Prerelease (by design, for the
   separate warn-only RecommendedMin check). That meant a prerelease build
   at or above the floor, e.g. "1.52.1-rc1", satisfied the >= 1.52.1 gate
   and had auto_gc_behavior.archive_level written into the YAML — even
   though the old doltVersionAtLeast (gc_config.go) already failed closed
   for any prerelease-suffixed string via strconv.Atoi. If that prerelease
   build predates the field, dolt sql-server's yaml.UnmarshalStrict refuses
   to start: the exact #4986 regression. Extracted the derivation into
   doltArchiveLevelSupported(id) and added a Prerelease == "" gate, plus
   TestDoltArchiveLevelSupported (table test covering above/at/below the
   floor and prerelease-at/-above-floor) and
   TestDoltArchiveLevelSupported_UnparsedVersionFailsClosed.

2. Duplicate dolt-version advisory (major, UX). `bd init --proxied-server`
   probed and printed the version-recommendation warning twice: once in its
   own preflight (init_proxied_server.go) and again ~90 lines later when
   newProxiedServerUOWProvider reaches newManagedProxiedServerUOWProvider,
   which resolved+probed again. The existing doltVersionWarnOnce didn't
   cover init's own Fprintf. Extracted one shared resolveAndProbeDolt(ctx,
   errPrefix) helper (uow_factory.go) that does resolve + probe +
   once-gated warn, called from both sites; init_proxied_server.go no
   longer needs its own copy of the Resolve/ProbeWithPolicy/error-wrapping
   sequence. This also folds the two call sites' warning gates (previously
   !in.quiet vs !quietFlag && !jsonOutput) into one JSON-mode-safe gate.

Verified: CGO_ENABLED=1 go build -tags gms_pure_go ./... clean;
CGO_ENABLED=1 go vet -tags gms_pure_go ./cmd/bd/... clean; gofmt -l on all
changed files empty; targeted tests pass (TestDoltArchiveLevelSupported,
TestDoltArchiveLevelSupported_UnparsedVersionFailsClosed,
TestNewProxiedServerUOWProvider_RoutesExternalConfigToExternalProvider,
TestNewExternalProxiedServerUOWProvider_*, TestBuildProxiedServerClientInfo,
and internal/doltversion's own suite).

Agent-Signature: builder-unknown-model-unknown-reasoning on behalf of maphew

* test(doltversion): resolve symlinks in RealPath expectations (macOS fix)

TestProbeNormalOutput and TestProbeSymlink failed on macOS CI because
t.TempDir lives under /var/folders, itself a symlink to
/private/var/folders. Probe derives RealPath via filepath.EvalSymlinks,
so it correctly returned the /private/var form while the tests compared
against the raw TempDir path.

Map the expected path through the same EvalSymlinks resolution (new
resolveSymlinks test helper). Verified against a symlinked TMPDIR on
Linux, which reproduced the failure before this change.

Agent-Signature: claude-fable-5-high on behalf of maphew

* refactor(cmd/bd): narrow resolveAndProbeDolt to drop the unused identity return

Upstream removed the archive-level gate from proxied-server mode
(68afa3d51), which was the only consumer of the probed identity at these
call sites; returning it now trips unparam on the PR-introduced line.
PR-2's revalidation consumer re-widens the signature when it lands.

Agent-Signature: claude-fable-5-high on behalf of maphew

* fix(doltversion): cross-process probe cache, PATHEXT completion, daily warning dedup

Review fixes (all three MERGE-AFTER-FIXES items from the 2026-08-10 review):

1. The per-command `dolt version` fork on the store-open hot path is gone
   for the steady state: ProbeWithPolicyCached fronts the fork with a JSON
   cache in the user cache dir, keyed by the binary's (real path, size,
   mtime) fingerprint - the exact fields Identity documents for caching.
   A fingerprint hit costs a stat; any change to the binary re-probes.
   Hard probe errors are never cached, and every cache failure direction
   fails open to the uncached behavior.

2. Windows: an explicit BEADS_DOLT_BIN spelled without its extension now
   completes via exec.LookPath's PATHEXT handling (exact spelled path wins
   when both exist), and the behavioral tests run on Windows through .cmd
   stubs instead of skipping wholesale - plus a focused windows-latest CI
   job (test-windows-liveness pattern) so those branches actually execute
   at PR time. skipOnWindows remains only where the subject genuinely
   doesn't exist there (exec-bit check, symlink creation).

3. The old/unverifiable-version advisory now repeats at most once per day
   (warn stamp in the same cache file) instead of on every bd invocation;
   troubleshooting docs state the cadence.

Nits from the same review: Ctrl-C during the probe no longer prints the
misleading install hint (call site branches on context.Canceled); Probe's
stat branch now uses validateExplicitPath's not-found/not-executable
taxonomy split; ParseVersion documents its trailing-integer laxity and the
tightening required before part-2 enforcement.

Agent-Signature: claude-fable-5-high on behalf of maphew

* fix(doltversion): honor init's local --quiet for the advisory; defer sidecar claim in docs

Cross-vendor review follow-ups: bd init defines a local --quiet that
shadows the persistent flag, so resolveAndProbeDolt now takes quiet as a
parameter (init passes in.quiet || quietFlag) instead of reading the
global; and the troubleshooting entry no longer advertises the clone-local
sidecar resolution source, which is a part-2 hook not yet wired
(SidecarValue is always empty in this PR).

Agent-Signature: claude-fable-5-high on behalf of maphew

* test: stop the Windows counting stub double-counting forks

cmd.exe includes the space before a mid-line '>>' in the written text,
so the batch stub's 'set /p =x >> file' appended two bytes ('x ') per
invocation and forkCount — which counts bytes — read every fork as two.
All four Windows failures were exactly doubled expectations; the cache
itself hit and skipped forks correctly. Lead with the redirection so
exactly one byte lands per invocation.

Agent-Signature: claude-fable-5-high on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019CwiLhLbAGZdJtKYT76CPp

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 21:18:24 -07:00

217 lines
7.9 KiB
Go

package doltversion
import (
"context"
"encoding/json"
"os"
"path/filepath"
"time"
)
// warnRepeatInterval is how long a shown advisory suppresses identical
// advisories from later processes. One day keeps the warning present enough
// that an operator on an old dolt is reminded it is still old, without
// repeating on every single bd invocation, which review found trains
// operators to ignore it entirely.
const warnRepeatInterval = 24 * time.Hour
// probeCacheFile is the on-disk shape of the probe cache. One entry only:
// a machine has one resolved dolt binary at a time in practice, and the
// fingerprint check below makes a stale entry for a different binary
// harmless (it just misses).
type probeCacheFile struct {
// RealPath/FileSize/FileModTimeUnixNano fingerprint the binary the
// cached result was probed from — exactly the Identity fields whose
// doc comment promises callers can cache on them.
RealPath string `json:"real_path"`
FileSize int64 `json:"file_size"`
FileModTimeUnixNano int64 `json:"file_mod_time_unixnano"`
// RawOutput and Version replay the probe result on a fingerprint hit.
// Version is the canonical string form; empty when Unverifiable.
RawOutput string `json:"raw_output"`
Version string `json:"version,omitempty"`
Unverifiable bool `json:"unverifiable,omitempty"`
// LastWarnedUnix is when a caller last showed the version advisory
// (via MarkWarned), in Unix seconds. Zero means never shown.
LastWarnedUnix int64 `json:"last_warned_unix,omitempty"`
}
// CachedProbeResult is what ProbeWithPolicyCached returns alongside its
// error: the probe outcome plus the cache/warning bookkeeping a CLI call
// site needs to decide whether to print the advisory this time.
type CachedProbeResult struct {
Identity Identity
Warning *Warning
// FromCache is true when the result was replayed from the cache file
// without forking the binary.
FromCache bool
// WarnDue is true when Warning is non-nil and the advisory has not
// been shown (per the cache file's stamp) within warnRepeatInterval.
// With no usable cache file it is always true — fail open toward
// showing the warning, never toward hiding it.
WarnDue bool
}
// ProbeWithPolicyCached is ProbeWithPolicy with a cross-process cache in
// front of the fork. Every bd command in managed proxied-server mode runs
// this on its hot path, and each command is a fresh process, so an
// in-process memo cannot help — the cache lives in a small JSON file at
// cachePath, keyed by the binary's (real path, size, mtime) fingerprint.
// A fingerprint hit replays the previous result for the cost of a stat;
// any change to the binary on disk misses and re-probes.
//
// The cache is strictly best-effort and fails open in every direction: an
// empty cachePath, an unreadable/corrupt cache file, or a failed write all
// degrade to exactly the uncached ProbeWithPolicy behavior. Hard probe
// errors are never cached — a broken binary is re-probed (and re-fails,
// loudly) on every command.
func ProbeWithPolicyCached(ctx context.Context, path, cachePath string, now time.Time) (CachedProbeResult, error) {
if cachePath == "" {
id, warn, err := ProbeWithPolicy(ctx, path)
return CachedProbeResult{Identity: id, Warning: warn, WarnDue: warn != nil}, err
}
if cached, ok := loadCacheEntry(cachePath); ok {
if id, ok := fingerprintMatches(path, cached); ok {
res := CachedProbeResult{Identity: id, FromCache: true}
if cached.Unverifiable {
res.Identity.RawOutput = cached.RawOutput
res.Warning = &Warning{kind: warningUnverifiable, RawOutput: cached.RawOutput}
} else {
ver, err := ParseVersion(cached.RawOutput)
if err != nil {
// Cache content no longer parses (e.g. written by a
// different beads version) — treat as a miss.
return probeAndStore(ctx, path, cachePath, cached.LastWarnedUnix, now)
}
res.Identity.Version = ver
res.Identity.RawOutput = cached.RawOutput
res.Warning = CheckRecommended(res.Identity)
}
res.WarnDue = res.Warning != nil && warnDue(cached.LastWarnedUnix, now)
return res, nil
}
}
return probeAndStore(ctx, path, cachePath, 0, now)
}
// MarkWarned records in the cache file that the version advisory was shown
// at now, so later processes suppress it for warnRepeatInterval. Best
// effort: on any error the stamp is simply not recorded and the next
// process warns again, which is the safe direction.
func MarkWarned(cachePath string, now time.Time) {
if cachePath == "" {
return
}
entry, ok := loadCacheEntry(cachePath)
if !ok {
return
}
entry.LastWarnedUnix = now.Unix()
writeCacheEntry(cachePath, entry)
}
// probeAndStore runs the real fork-and-parse probe and, on a cacheable
// outcome (parsed version or unverifiable-but-ran), persists it.
// lastWarned carries an existing warn stamp forward across a re-probe of
// what fingerprints as a changed binary at the same path — zero when there
// is nothing to carry.
func probeAndStore(ctx context.Context, path, cachePath string, lastWarned int64, now time.Time) (CachedProbeResult, error) {
id, warn, err := ProbeWithPolicy(ctx, path)
res := CachedProbeResult{Identity: id, Warning: warn}
if err != nil {
return res, err
}
entry := probeCacheFile{
RealPath: id.RealPath,
FileSize: id.FileSize,
FileModTimeUnixNano: id.FileModTime.UnixNano(),
RawOutput: id.RawOutput,
LastWarnedUnix: lastWarned,
}
if warn != nil && warn.kind == warningUnverifiable {
entry.Unverifiable = true
} else {
entry.Version = id.Version.String()
}
writeCacheEntry(cachePath, entry)
res.WarnDue = warn != nil && warnDue(lastWarned, now)
return res, nil
}
// fingerprintMatches stats path (following symlinks, the same resolution
// Probe applies) and reports whether it is byte-for-byte plausibly the
// binary entry was probed from. It also returns the fresh Identity file
// fields for the caller to build its result from — GivenPath is the path
// as asked for now, not as cached, so a symlink flip is reflected.
func fingerprintMatches(path string, entry probeCacheFile) (Identity, bool) {
if err := validateExplicitPath(path); err != nil {
return Identity{}, false
}
realPath := path
if resolved, err := filepath.EvalSymlinks(path); err == nil {
realPath = resolved
}
info, err := os.Stat(realPath)
if err != nil {
return Identity{}, false
}
id := Identity{
GivenPath: path,
RealPath: realPath,
FileSize: info.Size(),
FileModTime: info.ModTime(),
}
ok := realPath == entry.RealPath &&
info.Size() == entry.FileSize &&
info.ModTime().UnixNano() == entry.FileModTimeUnixNano
return id, ok
}
func warnDue(lastWarnedUnix int64, now time.Time) bool {
if lastWarnedUnix <= 0 {
return true
}
return now.Sub(time.Unix(lastWarnedUnix, 0)) >= warnRepeatInterval
}
func loadCacheEntry(cachePath string) (probeCacheFile, bool) {
data, err := os.ReadFile(cachePath) //nolint:gosec // G304: cachePath is caller-controlled cache-file location (user cache dir), not request input
if err != nil {
return probeCacheFile{}, false
}
var entry probeCacheFile
if err := json.Unmarshal(data, &entry); err != nil {
return probeCacheFile{}, false
}
return entry, true
}
// writeCacheEntry persists entry atomically (temp file + rename) so a
// concurrent reader never sees a torn write; two concurrent writers
// last-write-wins, which is fine for a cache. Best effort throughout.
func writeCacheEntry(cachePath string, entry probeCacheFile) {
data, err := json.Marshal(entry)
if err != nil {
return
}
dir := filepath.Dir(cachePath)
if err := os.MkdirAll(dir, 0o700); err != nil {
return
}
tmp, err := os.CreateTemp(dir, filepath.Base(cachePath)+".tmp-*")
if err != nil {
return
}
tmpName := tmp.Name()
_, werr := tmp.Write(data)
cerr := tmp.Close()
if werr != nil || cerr != nil {
_ = os.Remove(tmpName)
return
}
if err := os.Rename(tmpName, cachePath); err != nil {
_ = os.Remove(tmpName)
}
}