* Use Snappy output for Dolt garbage collection Pass archive level 0 through every Beads-owned GC path so new table files use Snappy instead of zstd archives. Existing zstd archive decoding remains unchanged. Refs #4249. Agent-Signature: amp-unknown-model-unknown-reasoning on behalf of matt wilkie Amp-Thread-ID: https://ampcode.com/threads/T-019f8d27-e480-774e-bd5a-7051e8e6e378 * fix(dolt-gc): gate archive_level on managed server auto-GC to avoid bricking older external dolt Paired review on gastownhall/beads#4986 flagged that Beads-managed dolt sql-server processes still auto-GC into zstd archives: the pinned Dolt defaults background auto-GC to archive_level 1, and neither server-launch path set auto_gc_behavior.archive_level: 0. Both managed-server launch paths now set archive_level: 0 (auto-GC stays enabled) when the resolved external dolt binary is new enough to support the key: - internal/storage/dbproxy/server (via cmd/bd/proxied_server.go's config.yaml generator) already wrote a YAML config; it now includes auto_gc_behavior when supported. - internal/doltserver's legacy launcher previously used CLI flags only (-H/-P/--loglevel) with no YAML config at all — Dolt's own commandLineServerConfig.AutoGCBehavior() is a stub that hardcodes archive_level 1 regardless of flags, so CLI-flag mode can never express this. It now optionally switches to `--config <generated yaml>` (Dolt ignores all other sql-server flags once --config is present, so host/port/log-level move into the YAML alongside archive_level). Both paths gate on internal/doltserver.SupportsArchiveLevelConfig, which probes `dolt version` and fails closed (omits the key, keeps prior behavior) on any parse/exec error. This matters because Dolt's YAML loader uses yaml.UnmarshalStrict (servercfg/yaml_config.go): an older external dolt binary whose own compiled-in YAMLConfig struct predates auto_gc_behavior.archive_level (minver 1.52.1) would otherwise hit a hard parse error and refuse to start, not silently ignore the key. Existing config.yaml files are left untouched (only fresh bootstraps set the key). Refs gastownhall/beads#4986. Agent-Signature: claude-fable-5-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(compact): fall back to plain dolt gc on older external dolt --archive-level rejection Paired review on gastownhall/beads#4986 flagged that `bd compact --dolt` shells out to the PATH `dolt` binary with no version guarantee, unlike the in-process GC paths pinned by go.mod. Passing --archive-level 0 unconditionally meant an external dolt predating the flag would abort compact with an unknown-flag error where plain `dolt gc` used to succeed. runCompactDolt now retries with plain `dolt gc` when the first attempt's combined output matches isUnknownArchiveLevelFlagError, printing a one-line notice that the older dolt may still produce zstd archives. The classifier is deliberately narrow (requires both an "unknown flag"-style phrase and the archive-level flag name in the output) so a genuine GC failure is never misclassified as a missing flag and swallowed — real failures still fail compact as before. Refs gastownhall/beads#4986. Agent-Signature: claude-fable-5-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(dolt-gc): replicate .doltcfg discovery in --config mode; memoize version probe Delta review round 2 on gastownhall/beads#4986 flagged that the legacy launcher's new --config mode silently drops Dolt's own .doltcfg discovery: Dolt's setupDoltConfig (cmd/dolt/commands/sqlserver/sqlserver.go, pinned module) returns immediately when --config is passed, so the parent ../.doltcfg lookup (privileges.db users, branch_control.db) that flag mode performs never runs. A deployment previously run in CLI-flag mode that had a parent .doltcfg would, once the resolved dolt becomes new enough to use --config mode, silently fall back to a fresh $data_dir/.doltcfg: existing users and branch controls abandoned, new passwordless root. internal/doltserver.resolveCfgDir replicates Dolt's own flag-mode discovery exactly (parent ../.doltcfg if present, else $data_dir/.doltcfg, error via ErrMultipleDoltCfgDirs if both exist rather than guess) and Start() now resolves it once per launch and writes it into the generated YAML's cfg_dir. servercfg.YAMLConfig.CfgDirStr carries no minver tag in the pinned module (i.e. always supported), so the existing 1.52.1 archive_level gate stays as the single version gate for --config mode. Also: filepath.Abs the --config path before exec (the child's cwd is the data dir, matching the sibling dbproxy/server path's existing defensive abs), and memoize SupportsArchiveLevelConfig per doltBin (sync.Map) so Start()/UOW-provider creation does not re-fork `dolt version` on every call within a process. Refs gastownhall/beads#4986. Agent-Signature: claude-fable-5-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(dolt-gc): reconcile pre-existing managed configs on load; warn on custom ones Delta review round 2 on gastownhall/beads#4986 flagged that pre-existing config.yaml files bypass the archive_level plumbing in both directions: an already-bootstrapped install's config.yaml silently keeps zstd auto-GC forever (missing key never gets added without a manual regenerate), and a config already carrying the key can hard-fail a later dolt downgrade via yaml.UnmarshalStrict on an unrecognized key. Fresh config.yaml renders now carry a one-line YAML comment marker identifying them as Beads-managed (invisible to Dolt's own parser, which treats it as a comment). ensureProxiedServerConfig uses the marker to decide how to handle an existing default-path file: - marker present (Beads-managed): reconcile at load via reconcileManagedProxiedServerConfig, which parses the existing YAML into the servercfg structs, adds or strips auto_gc_behavior to match the current archive-level probe result, and re-renders — preserving every other field (including port) via the struct round trip rather than string surgery. A no-op when already aligned, so this costs nothing on the steady-state path. - marker absent (a pre-marker file from an older Beads version, or a genuinely user-supplied custom config path): never rewritten. Emits a one-line stderr warning instead, covering both directions (zstd may still be active with a hint how to opt in; or the file may cause a refuse-to-start on this dolt). Refs gastownhall/beads#4986. Agent-Signature: claude-fable-5-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(dolt-gc): preserve sibling auto_gc fields, interpolate custom configs, write atomically Verification review of the round-2 commits on gastownhall/beads#4986 found three confirmed issues in cmd/bd/proxied_server.go: - reconcileManagedProxiedServerConfig tested cfg.BehaviorConfig.AutoGCBehavior != nil (the whole block) instead of ArchiveLevel_ specifically. A managed config whose block carried only a user-set "enable" neither gained archive_level when the binary supported it, nor kept "enable" when the whole block was stripped on an unsupported binary. Now inspects and mutates only ArchiveLevel_, creating the block if needed to add it and removing it only if clearing archive_level leaves it completely empty. - Validating a custom (env var / sidecar) config via servercfg.NewYamlConfig on raw bytes skips Dolt's own environment-variable interpolation (servercfg/env_interpolate.go, applied by YamlConfigFromFile before parsing) — confirmed by reading the pinned module source. A valid custom config using ${VAR}-style values was being rejected before the server ever started. ensureProxiedServerConfig and warnUnmanagedProxiedServerConfig now parse any unmanaged file (custom path, or a pre-marker default-path file) via YamlConfigFromFile; only a Beads-managed file, which never contains interpolation syntax, still uses the raw-bytes/NewYamlConfig path (inside reconcileManagedProxiedServerConfig). - Reconciliation rewrote config.yaml with a truncating os.WriteFile, which could leave a partial file — blocking every future dolt sql-server start — if the process crashed mid-write. Both the reconcile-rewrite and the fresh-render write paths now go through the existing atomicWriteFile helper (temp file in the same directory, fsync, close, rename), already used elsewhere in cmd/bd for exactly this reason. Tests: TestReconcileManagedProxiedServerConfig_PreservesSiblingFields (all three directions requested by the review), and TestEnsureProxiedServerConfig_CustomConfigEnvVarInterpolation. Refs gastownhall/beads#4986. Agent-Signature: claude-fable-5-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(dolt-gc): resolve .doltcfg physically through symlinks; key version cache by file identity Verification review of the round-2 commits on gastownhall/beads#4986 found two more confirmed issues: - resolveCfgDir computed the parent .doltcfg via filepath.Join(doltDir, "..", ...), which cleans ".." lexically without touching the filesystem. If doltDir is itself a symlink, that yields the symlink's own lexical parent directory, not the physical parent of whatever it actually points at. Reproduced directly: on Linux, chdir into a symlink makes the kernel track the process's cwd physically, so the real `dolt` child process's own "../.doltcfg" lookup (a bare relative path resolved against its actual, physically-tracked cwd) sees the PHYSICAL parent — diverging from the lexical one a naive Go-side filepath.Join would compute. A symlinked data directory could therefore miss the very parent .doltcfg this whole fix exists to find, re-abandoning existing privileges/branch-control. resolveCfgDir now resolves doltDir via filepath.EvalSymlinks (falling back to the lexical path on error) before computing the parent. TestResolveCfgDir_SymlinkedDataDirFindsPhysicalParent reproduces the exact layout and was confirmed to fail without the fix. - SupportsArchiveLevelConfig's memoization cache was keyed by doltBin path alone, so an in-place dolt upgrade or downgrade at that same path kept the stale verdict for the rest of the process's lifetime. The cache key now includes size+mtime (archiveLevelCacheKey), falling back to an uncached probe when the binary can't be stat'd. The test that previously codified the staleness (rewriting the binary at the same path and asserting the cached result "sticks") is replaced with TestSupportsArchiveLevelConfig_MemoizesUnchangedBinary (same identity -> still cached, proven via an embedded call counter, not just the returned bool) and TestSupportsArchiveLevelConfig_InvalidatesOnFileIdentityChange (changed mtime -> re-probed). Refs gastownhall/beads#4986. Agent-Signature: claude-fable-5-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(dolt-gc): preserve symlinked configs, field-level warn check, graceful degrade on hand-edited managed files Verification review of the round-3 commits on gastownhall/beads#4986 found three narrower issues, all in cmd/bd/proxied_server.go: - atomicWriteFile's os.Rename destination does not follow a symlink — it replaces whatever is directly at that path. Reconciling (or freshly rendering) a managed config.yaml that was itself a symlink therefore silently replaced the symlink with a regular file at the same location, losing the indirection and orphaning whatever it pointed at. Both write call sites now resolve the target through filepath.EvalSymlinks first (resolveConfigWriteTarget), falling back to the given path when it does not exist yet (which also covers a dangling symlink — self-healed by writing directly at its location). Test: TestEnsureProxiedServerConfig_ManagedConfigBehindSymlinkPreservesSymlink reconciles a config.yaml symlink and asserts the symlink itself survives while the physical target gets the reconciled content; confirmed it fails without the fix (reverted, watched fail, restored). - warnUnmanagedProxiedServerConfig tested cfg.AutoGCBehavior() != nil (the whole block), not archive_level specifically — the same bug the reconciler had before round 3, now applied to the warn path. An enable-only custom config missed the zstd-risk warning on a capable binary and got a false incompatibility warning on an incapable one. Now goes through the same field-level check via a new autoGCArchiveLevelIsSet helper (type-asserts to the concrete *servercfg.YAMLConfig the pinned module's loaders always return, falling back to the block-level check if that assertion ever fails). Test: TestWarnUnmanagedProxiedServerConfig_EnableOnlyFieldLevel covers both directions. - A managed file's marker only proves Beads generated it ORIGINALLY, not that it hasn't been hand-edited since (e.g. a ${VAR} placeholder added by an operator). reconcileManagedProxiedServerConfig's raw-bytes parse is deliberately non-interpolating, so that now fails to unmarshal. Verified the prior behavior first: that parse error propagated as a hard error out of ensureProxiedServerConfig, which newManagedProxiedServerUOWProvider (cmd/bd/uow_factory.go) returns directly — aborting the ENTIRE bd command before ever attempting to start the managed server, even though a real dolt process would happily interpolate and start against the same file. Now degrades gracefully: logs a one-line warning and falls through to the same warn-only/never-rewrite treatment as an unmanaged file. Test: TestEnsureProxiedServerConfig_ManagedFileWithPlaceholderDegradesGracefully. Refs gastownhall/beads#4986. Agent-Signature: claude-fable-5-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(backup): narrow atomicWriteFile's crash-safe claim instead of changing behavior Verification review of the round-3 commits on gastownhall/beads#4986 noted the "crash-safe" doc comment overstates: only the temp file's own contents are fsynced (not the parent directory entry, so a crash between rename and a directory fsync could still lose the rename on some filesystems), and os.Rename's atomic-replace guarantee is a POSIX/Unix property, not guaranteed on Windows. This helper has an existing caller (saveBackupState) whose expectations should not change silently, so this narrows the doc comment to state those caveats precisely rather than strengthening the implementation. Also notes the symlink-replacement behavior a caller needs to know about, pointing at cmd/bd/proxied_server.go's resolveConfigWriteTarget for a case that resolves the physical target first. No behavior change. Refs gastownhall/beads#4986. Agent-Signature: claude-fable-5-high on behalf of matt wilkie Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
71 lines
2.1 KiB
Go
71 lines
2.1 KiB
Go
package main
|
|
|
|
import "testing"
|
|
|
|
// TestIsUnknownArchiveLevelFlagError covers the classifier that decides
|
|
// whether runCompactDolt should retry `dolt gc` without --archive-level
|
|
// (gastownhall/beads#4986). It must recognize the real unknown-flag
|
|
// rejection from an older external dolt, and must NOT misclassify genuine
|
|
// GC failures — those should keep failing compact rather than being
|
|
// silently swallowed by a retry that then also fails.
|
|
func TestIsUnknownArchiveLevelFlagError(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
output string
|
|
want bool
|
|
}{
|
|
{
|
|
name: "real dolt argparser unknown-option message",
|
|
output: "error: unknown option `archive-level'\n\nNAME\n\tdolt gc - Cleans up unreferenced data from the repository.\n",
|
|
want: true,
|
|
},
|
|
{
|
|
name: "uppercase / mixed case tolerant",
|
|
output: "ERROR: Unknown Option `archive-level'",
|
|
want: true,
|
|
},
|
|
{
|
|
name: "cobra-pflag style unknown flag message",
|
|
output: "unknown flag: --archive-level",
|
|
want: true,
|
|
},
|
|
{
|
|
name: "go flag package style message",
|
|
output: "flag provided but not defined: -archive-level",
|
|
want: true,
|
|
},
|
|
{
|
|
name: "underscore spelling still matches",
|
|
output: "error: unknown option `archive_level'",
|
|
want: true,
|
|
},
|
|
{
|
|
name: "genuine GC failure must not be swallowed",
|
|
output: "error: could not acquire lock on repository; another process may be using it",
|
|
want: false,
|
|
},
|
|
{
|
|
name: "unrelated unknown-option error for a different flag",
|
|
output: "error: unknown option `full'",
|
|
want: false,
|
|
},
|
|
{
|
|
name: "mentions archive-level but is not an unknown-flag error",
|
|
output: "Specify the archive compression level garbage collection results. Default is 1, Disable with 0",
|
|
want: false,
|
|
},
|
|
{
|
|
name: "empty output",
|
|
output: "",
|
|
want: false,
|
|
},
|
|
}
|
|
|
|
for _, tc := range tests {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
if got := isUnknownArchiveLevelFlagError(tc.output); got != tc.want {
|
|
t.Errorf("isUnknownArchiveLevelFlagError(%q) = %v, want %v", tc.output, got, tc.want)
|
|
}
|
|
})
|
|
}
|
|
}
|