beads/cmd/bd/dolt_remote_reset_data.go
Rongjun GENG 8da2ae4328
fix(embeddeddolt): commit pending changes before push (#5433) (#5795)
* fix(embeddeddolt): commit pending changes before push (#5433)

Every Pull variant (Pull, PullWithStrategy, PullRemoteWithStrategy,
PullRemote, PullFrom) auto-commits pending working-set changes first
(GH#2474, so a crashed command's leftovers don't make the merge
refuse to start), but no Push variant (Push, ForcePush, PushRemote,
PushTo) did. A write that landed in the working set but was never
explicitly committed -- auto-commit configured off, or a session that
ended right after a write with no intervening commit -- silently
pushed nothing new while `bd dolt push` still printed "Push
complete." and exited 0.

This is the half of #5433's silent-stall report the maintainer traced
and confirmed in code (issue comment, quoting exact line numbers this
matches on current main): "every Pull variant calls CommitPending
before pulling... no Push variant does... push never flushes the
working set, then prints 'Push complete.' on nil error with no
remote-ref or working-set check. Strongest hook for a narrow
push-honesty PR." Reporter confirmed: "Yes, please open the
push-honesty PR. That is the surface that actually cost us."

Scoped to exactly this half -- the confirmed, traced, mechanical gap
matching an already-established pattern (copy Pull's own guard to
Push). Did NOT attempt the harder half discussed in the same thread:
verifying the remote ref actually advanced and reporting honestly
when it didn't. That needs a deliberate design call the maintainer
themselves treated as separate (distinguishing a legitimate no-op
push, already in sync, from one that had something to send but
transferred nothing) and risks false positives if done carelessly --
a bigger change than this issue's narrowed scope asked for. Flagging
in the relay rather than guessing at that design.

Also confirmed the server-mode backend (internal/storage/dolt) has
the identical asymmetry -- Push/ForcePush/PushRemote all route through
pushToRemote, which likewise never calls CommitPending -- but the
issue's repro and the maintainer's trace are both embedded-mode
specific, so left server-mode untouched pending its own verification.

Does not claim to fix the mysterious full stall itself (bd dolt
commit also returning "Nothing to commit" while HEAD stays frozen,
even when called directly) -- the reporter and maintainer both
independently concluded that has no known deterministic trigger.
This closes one real, confirmed contributing gap: push not flushing
pending changes the way pull always has.

RED: added TestPushCommitsPendingChangesFirst, table-driven over all
four Push variants. Creates an issue (a genuinely pending, uncommitted
change -- verified via GetCurrentCommit before/after CreateIssue, so
the test would fail loudly if CreateIssue ever started auto-committing
on its own), then calls each Push variant and asserts HEAD moved.
Push/ForcePush/PushRemote route through the swappable vcPush/
vcForcePush vars (remote_peer_auth_test.go's existing
captureRemoteEntryPoints seam) so the test stops with errStopRemote
before any network IO; PushTo calls versioncontrolops.Push directly
with no equivalent seam, so it actually attempts the push and fails on
"remote not found" in this remote-less test store -- still proves
CommitPending ran first, since that failure only happens after this
fix's guard clause. All four failed pre-fix (HEAD did not move).
GREEN: all four pass after the fix.

Verified: full remote-verb/peer-auth suite (TestWithPeerAuth and its
variants, TestRemoteVerbsPresentStoredPeerCredentials,
TestRemoteVerbPlainRemote*) still green -- confirms this fix doesn't
disturb credential resolution or the swappable-seam test
infrastructure it shares. go vet clean, go build ./... clean across
the repo.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018c3vLFNVgPebSpCJrd8xTD

* chore(bazel): enumerate push_commit_pending_test.go in embeddeddolt go_test srcs

Generated by `bazel run //:gazelle`. The PR predates base's addition of
internal/storage/embeddeddolt/BUILD.bazel, so its new test file was not in
the explicit go_test srcs list; bazel would have silently run zero of its
cases. Base parity at the rebase base was 118/118 enumerated.

* fix(autopush): record post-push HEAD, and cover the clean-store push path

Maintainer review fixup for #5795 (adopt-pr review iteration 1).

Push now commits pending changes before pushing (GH#5433), so HEAD can move
across the call. maybeAutoPush read HEAD only before the push and recorded that
hash as LastCommit, which is now the parent of what was actually pushed, so the
next tick saw a mismatch and pushed once redundantly before re-latching. Route
the recorded hash through pushedCommitForState, which re-reads HEAD after a
successful push and falls back to the pre-push value if the re-read fails (the
push itself already succeeded, so dropping LastCommit would lose change
detection entirely).

Also:

- Add a clean-store arm to TestPushCommitsPendingChangesFirst. The guard relies
  on commitAll(tolerateEmpty=true) mapping Dolt's "nothing to commit" to
  (false, nil) for every push against an already-committed store, and nothing
  in this package asserted it. Flipping that flag reds the new arm alone, with
  the four pending-change arms still green.

- Correct the `bd dolt remote reset-data` confirmation, which promised the
  remote is "rebuilt from local HEAD" while the push now adds a commit created
  after the operator types y.

Not taken here, tracked separately: extending auto-push change detection to also
fire on a dirty working set (that gate predates this PR, so skipping it is an
incompleteness rather than a regression this PR introduces), and the PR-body
disclosure that push now commits on the auto-push and `bd sync` cadences.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
2026-09-30 15:58:51 -07:00

346 lines
12 KiB
Go

package main
import (
"bufio"
"context"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
"github.com/spf13/cobra"
"golang.org/x/term"
"github.com/steveyegge/beads/internal/githooksenv"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/doltutil"
)
// Ref names anchoring a git-backed Dolt remote's data plane, as published by
// dolt's git blobstore (store/blobstore/git_refs.go: DoltDataRef and
// DefaultInfoBranch). Pinned locally so the binary does not import that
// package; TestResetDataRefNamesMatchDolt keeps the pin honest.
const (
gitDoltDataRef = "refs/dolt/data"
gitDoltInfoRef = "refs/heads/__dolt_remote_info__"
)
// resetDataKind classifies how reset-data can replace the data plane behind
// a remote URL.
type resetDataKind int
const (
// resetDataGitBacked: delete the Dolt data refs on the git remote, then
// force-push to rebuild a fresh store holding only live chunks.
resetDataGitBacked resetDataKind = iota
// resetDataFileStore: clear the native Dolt file-store directory, then
// force-push to rebuild it.
resetDataFileStore
// resetDataFileAbsent: the file target is missing or empty — nothing
// anchors old data; force-push alone rebuilds it.
resetDataFileAbsent
// resetDataUnsupported: a cloud or hosted store whose contents bd cannot
// safely clear — replace the remote with a fresh URL/prefix instead.
resetDataUnsupported
)
// classifyResetDataRemote decides which reset-data mechanism applies to url.
// File URLs are classified by what is actually on disk, because a file://
// remote can be either a bare git repository (Dolt normalizes those to
// git+file:// at push time) or a native Dolt file store: a bare git repo
// takes the git-backed path, a directory with an nbs manifest is a file
// store, and a missing or empty target has nothing to reset. A non-empty
// target that is neither is an error — reset-data must not clear a
// directory it cannot identify.
func classifyResetDataRemote(url string) (resetDataKind, error) {
if doltutil.IsGitProtocolURL(url) {
return resetDataGitBacked, nil
}
path, isFile := resetDataFilePath(url)
if !isFile {
return resetDataUnsupported, nil
}
entries, err := os.ReadDir(path)
if os.IsNotExist(err) {
return resetDataFileAbsent, nil
}
if err != nil {
return 0, fmt.Errorf("inspecting remote target %s: %w", path, err)
}
if len(entries) == 0 {
return resetDataFileAbsent, nil
}
if isBareGitRepoDir(path) {
return resetDataGitBacked, nil
}
if _, err := os.Stat(filepath.Join(path, "manifest")); err == nil {
return resetDataFileStore, nil
}
return 0, fmt.Errorf("remote target %s is non-empty but is neither a bare git repository nor a Dolt file store (no manifest); refusing to clear it", path)
}
// resetDataFilePath extracts the local filesystem path from a file:// URL or
// a bare absolute path, reporting whether url is file-backed at all.
func resetDataFilePath(url string) (string, bool) {
if strings.HasPrefix(url, "file://") {
return strings.TrimPrefix(url, "file://"), true
}
if filepath.IsAbs(url) {
return url, true
}
return "", false
}
// isBareGitRepoDir reports whether dir looks like a bare git repository
// (HEAD file plus an objects directory).
func isBareGitRepoDir(dir string) bool {
if fi, err := os.Stat(filepath.Join(dir, "HEAD")); err != nil || fi.IsDir() {
return false
}
fi, err := os.Stat(filepath.Join(dir, "objects"))
return err == nil && fi.IsDir()
}
// resetDataGitURL converts a Dolt git-protocol remote URL to the form the
// git CLI accepts (strip the git+ prefix; file paths gain file://).
func resetDataGitURL(url string) string {
if trimmed := strings.TrimPrefix(url, "git+"); trimmed != url {
return trimmed
}
if path, isFile := resetDataFilePath(url); isFile && !strings.HasPrefix(url, "file://") {
return "file://" + path
}
return url
}
// lsRemoteDoltDataRefs returns which of the Dolt data-plane refs currently
// exist on the git remote at gitURL.
func lsRemoteDoltDataRefs(ctx context.Context, gitURL string) ([]string, error) {
ctx, cancel := context.WithTimeout(ctx, 60*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "git", "ls-remote", gitURL, gitDoltDataRef, gitDoltInfoRef) // #nosec G204 -- URL from configured remote
out, err := cmd.CombinedOutput()
if err != nil {
return nil, fmt.Errorf("git ls-remote %s failed: %s: %w", gitURL, strings.TrimSpace(string(out)), err)
}
var refs []string
for _, line := range strings.Split(strings.TrimSpace(string(out)), "\n") {
fields := strings.Fields(line)
if len(fields) == 2 {
refs = append(refs, fields[1])
}
}
return refs, nil
}
// deleteGitDoltDataRefs deletes refs on the git remote at gitURL. Git
// client-side hooks are disabled for the push, same as bd's other internal
// git invocations (GH#3724 class: a user's templated pre-push hook must not
// break — or observe — bd's data-plane plumbing).
func deleteGitDoltDataRefs(ctx context.Context, gitURL string, refs []string) error {
args := []string{"push", gitURL}
for _, ref := range refs {
args = append(args, ":"+ref)
}
cmd := exec.CommandContext(ctx, "git", args...) // #nosec G204 -- URL from configured remote, fixed refspecs
cmd.Env = envWithNoGitHooks()
out, err := cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("git push (delete %s) failed: %s: %w", strings.Join(refs, ", "), strings.TrimSpace(string(out)), err)
}
return nil
}
// envWithNoGitHooks returns the current environment with git client-side
// hooks disabled via GIT_CONFIG_PARAMETERS, preserving any parameters the
// caller already set (mirrors applyNoGitHooksToCmd in internal/storage/dolt).
func envWithNoGitHooks() []string {
return githooksenv.DisabledEnv(os.Environ())
}
// clearDoltFileStore removes the contents of a native Dolt file-store
// directory, keeping the directory itself so the follow-up push can rebuild
// the store in place. The caller has already verified a manifest is present.
func clearDoltFileStore(dir string) error {
entries, err := os.ReadDir(dir)
if err != nil {
return fmt.Errorf("reading remote store %s: %w", dir, err)
}
for _, entry := range entries {
if err := os.RemoveAll(filepath.Join(dir, entry.Name())); err != nil {
return fmt.Errorf("clearing remote store %s: %w", dir, err)
}
}
return nil
}
var doltRemoteResetDataYes bool
var doltRemoteResetDataCmd = &cobra.Command{
Use: "reset-data <name>",
Short: "Replace a remote's data plane in place after a history squash",
Long: `Replace a Dolt remote's stored data with a fresh copy of local HEAD.
After a history squash (see the History Bloat recovery runbook), a plain
'bd dolt push --force' re-points the remote's refs but deletes nothing:
Dolt remotes accumulate chunks monotonically, so the remote keeps the full
pre-squash store. This command rebuilds the remote's data plane so it holds
only live chunks:
- Git-backed remotes (issue data riding a git remote under refs/dolt/data):
deletes the Dolt data refs on the git remote, then force-pushes to
rebuild a fresh store. Code branches are untouched.
- Native file remotes (file:// paths): clears the store directory, then
force-pushes to rebuild it.
- Cloud/hosted remotes (aws://, gs://, dolthub://, ...): bd cannot clear
the stored data safely — replace the remote with a fresh URL or prefix:
bd dolt remote remove <name>
bd dolt remote add <name> <fresh-url>
bd dolt push --force
This rewrites the remote's data plane. Every other clone must re-clone from
the reset remote (that is already true after the squash itself). Refuses to
run with uncommitted working-set changes: the rebuilt remote holds exactly
HEAD, and anything uncommitted would not be part of it.
Examples:
bd dolt remote reset-data origin # prompts for confirmation
bd dolt remote reset-data origin --yes # no prompt (scripts, agents)`,
SilenceUsage: true,
SilenceErrors: true,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
CheckReadonly("dolt remote reset-data")
name := args[0]
if usesProxiedServer() {
return HandleErrorRespectJSON("bd dolt remote reset-data is not supported over a proxied bd server; run it from the workspace that hosts the Dolt database")
}
if isDoltLocalOnly() {
fmt.Fprintln(os.Stderr, "Error: cannot reset remote data: remote sync is disabled (dolt.local-only=true).")
fmt.Fprintln(os.Stderr, "To re-enable remote sync: bd config unset dolt.local-only")
return SilentExit()
}
ctx := rootCtx
st := getStore()
if st == nil {
return HandleError("no store available")
}
remotes, err := st.ListRemotes(ctx)
if err != nil {
return HandleError("listing remotes: %v", err)
}
var url string
for _, r := range remotes {
if r.Name == name {
url = r.URL
break
}
}
if url == "" {
return HandleErrorWithHint(
fmt.Sprintf("remote %q is not configured", name),
"Use 'bd dolt remote list' to see configured remotes.")
}
// The rebuilt remote holds exactly HEAD; refuse while anything
// uncommitted would be silently left out of it.
if det, ok := storage.UnwrapStore(st).(storage.PendingChangeDetector); ok {
dirty, derr := det.HasCommittablePending(ctx)
if derr != nil {
return HandleError("checking working set: %v", derr)
}
if dirty {
return HandleErrorWithHint(
"working set has uncommitted changes; refusing to reset remote data",
"Run 'bd dolt commit' first — the rebuilt remote holds exactly HEAD.")
}
}
kind, err := classifyResetDataRemote(url)
if err != nil {
return HandleError("%v", err)
}
if kind == resetDataUnsupported {
return HandleErrorWithHint(
fmt.Sprintf("cannot clear stored data on remote %q (%s)", name, url),
fmt.Sprintf("Replace the remote with a fresh URL or prefix instead:\n"+
" bd dolt remote remove %s\n"+
" bd dolt remote add %s <fresh-url>\n"+
" bd dolt push --force", name, name))
}
if !doltRemoteResetDataYes {
if !term.IsTerminal(int(os.Stdin.Fd())) {
return HandleErrorWithHint(
fmt.Sprintf("reset-data replaces all Dolt data stored on remote %q (%s)", name, url),
"Re-run with --yes to confirm.")
}
fmt.Printf("This replaces all Dolt data stored on remote %q:\n", name)
fmt.Printf(" %s\n", url)
fmt.Println("The remote is rebuilt from local HEAD, including any pending changes; other clones must re-clone.")
fmt.Print("Proceed? (y/N): ")
reader := bufio.NewReader(os.Stdin)
response, rerr := reader.ReadString('\n')
if rerr != nil {
return HandleError("reading confirmation: %v", rerr)
}
response = strings.TrimSpace(strings.ToLower(response))
if response != "y" && response != "yes" {
fmt.Println("Canceled.")
return nil
}
}
var deletedRefs []string
var clearedStore bool
switch kind {
case resetDataGitBacked:
gitURL := resetDataGitURL(url)
refs, lerr := lsRemoteDoltDataRefs(ctx, gitURL)
if lerr != nil {
return HandleError("%v", lerr)
}
if len(refs) == 0 {
fmt.Println("No Dolt data refs found on the remote (already reset?); pushing fresh store.")
} else {
if derr := deleteGitDoltDataRefs(ctx, gitURL, refs); derr != nil {
return HandleError("%v", derr)
}
deletedRefs = refs
fmt.Printf("Deleted %s on %s\n", strings.Join(refs, ", "), gitURL)
}
case resetDataFileStore:
path, _ := resetDataFilePath(url)
if cerr := clearDoltFileStore(path); cerr != nil {
return HandleError("%v", cerr)
}
clearedStore = true
fmt.Printf("Cleared remote store %s\n", path)
case resetDataFileAbsent:
fmt.Println("Remote store is empty or absent; pushing fresh store.")
}
fmt.Printf("Force-pushing to remote %q...\n", name)
if perr := st.PushRemote(ctx, name, true); perr != nil {
return HandleError("push after reset failed (the remote's data refs were already removed — re-run 'bd dolt push --force --remote %s' once the cause is fixed): %v", name, perr)
}
if jsonOutput {
return outputJSON(map[string]interface{}{
"remote": name,
"url": url,
"deleted_refs": deletedRefs,
"cleared_store": clearedStore,
"pushed": true,
})
}
fmt.Printf("✓ Remote %q data plane reset; store rebuilt from HEAD.\n", name)
fmt.Println(" Other clones of this database must re-clone from the remote.")
return nil
},
}