beads/internal/storage/embeddeddolt/remote_git_ref_test.go
Stephen Hodges 3efcd3571c
feat(storage): carry the git data ref of a git-backed Dolt remote (#6521)
* feat(storage): carry the git data ref of a git-backed Dolt remote

Dolt lets a git-backed remote keep its data on a ref other than
refs/dolt/data (the git_ref remote parameter, --ref on dolt remote add,
dolt clone, DOLT_REMOTE, and DOLT_CLONE). The storage layer never passed
it through, so a host that only accepts pushes under refs/heads/, or one
repository holding several databases on refs such as
refs/dolt/units/<key>, could not be used. Part of #6261; the commands
and the config key follow in a stacked PR.

- storage.RemoteInfo gains Ref (empty = refs/dolt/data);
  RemoteRefsMatch, EffectiveGitDataRef, and GitRefFromParamsJSON give
  every package one reading of it, pinned against dolt's constants.
- RemoteStore.AddRemoteWithRef on both stores, and CreateRemoteWithRef
  on the proxied-server remote use case, pass a non-empty ref as
  DOLT_REMOTE's --ref. AddRemote is unchanged; UpdateRemote keeps the
  ref through replacement and restore.
- versioncontrolops.DoltCloneWithRef and dolt.BootstrapFromRemoteWithDBRef
  pass the ref to DOLT_CLONE and to the dolt clone argv.
- ListRemotes reads the ref back from dolt_remotes.params (git_ref) in
  both remote repositories; PersistedRemotes reads it from
  repo_state.json. Malformed params is an error, never a silent default.
- The server-mode CLI mirror is compared on URL and ref and
  re-materialized with dolt remote add --ref when the ref differs. Where
  the CLI is proxied to the running server: the check reads the server's
  own repo_state.json when the listing is empty, the cold-start fallback
  re-adds a ref remote over SQL (which accepts --ref) so the server knows
  it, and a throwaway probe remote proves --ref works before an existing
  mirror remote is removed; probe remotes left by runs no longer alive
  are swept before any change to the mirror. For a server this bd does
  not own, the route stops and names the DOLT_REMOTE call to run
  against that server through bd sql, as one shell-quoted word.
- A ref is checked for a space, an ASCII control character, DEL, and a
  leading dash before it becomes a dolt argument; no other ref-name rule
  applies at this layer.
- Tests carry both ref shapes; the embedded-engine test proves add, list,
  push, and clone on the real engine; sqlmock tests cover the route
  decider and the cold-start re-add at the store seam; the dolt-CLI tests
  skip without a dolt binary.

* fix(storage): make both git_ref readers fail closed; pin the edges (#6521 review)

Maintainer review fixup for PR #6521. The review found no blocker or major;
this is the non-gating minor/nit set plus the test pins the synthesis named as
the residual defects.

- One reader for git_ref. storage.GitRefFromParams is now the single decoder
  for both planes (the SQL params column through GitRefFromParamsJSON, and
  repo_state.json through doltutil.PersistedRemotes), and a git_ref that is
  present but not a string is an error naming the remote instead of silently
  the default ref. Reading it as the default is how a mirror pinned to a custom
  ref would be kept on refs/dolt/data -- the wrong-ref push this PR closes,
  reached from the opposite direction.

- CreateRemoteWithRef checks the ref against what the dolt argv boundary will
  accept, so the recordable ref set is no longer wider than the routable one
  and a bad ref is refused at add time rather than diagnosed at push time.
  UpdateRemote deliberately does not re-check: it carries and restores refs
  that are already recorded and must stay able to put one back.

- BootstrapFromRemoteWithDBRef validates the ref before the already-exists
  early return, so the same --ref is accepted or refused the same way on a
  fresh host and on a bootstrapped one.

- sweepLeftoverProbes warns when it cannot remove another run's leftover probe
  remote, following PersistedRemoteInfos' precedent: the remote stays visible
  in `bd remote list` until some run succeeds, and nothing ever said why.

- EnsureCLIRemote's doc comment records that escalating a state-file read error
  is intentional for default-ref remotes too, and
  TestCLIMirrorUnreadableStateIsAnError now pins the ref == "" direction that a
  later "helpful" relaxation would gate away. The suggested URL-match
  short-circuit is deliberately NOT applied: when the state file is unreadable
  currentRef is unknown, so returning nil on a URL match alone would route a
  push onto whatever ref the mirror is secretly pinned to.

- AddCLIRemote's doc comment records that it is kept deliberately as the
  default-ref spelling now that EnsureCLIRemote uses AddCLIRemoteWithRef.

Not applied: widening probeCLIRemoteRef to `ref != "" || currentRef != ""` and
probing the restore shape. On a proxied server that turns a working custom-ref
-> default-ref transition into a hard failure (the target add is plain and
succeeds today, while probing the restore shape would be refused), and in the
one window that survives calibration -- a non-proxied plain-add failure from
disk/lock contention -- a capability probe predicts nothing about it.

Each new pin was mutation-proved: reverting the fix it guards reds that pin,
and only that pin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julian@gastownhall.dev>
2026-09-27 15:56:18 -07:00

185 lines
6.3 KiB
Go

//go:build cgo
package embeddeddolt_test
import (
"context"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/doltutil"
"github.com/steveyegge/beads/internal/storage/embeddeddolt"
"github.com/steveyegge/beads/internal/storage/versioncontrolops"
)
// runGit runs git in dir and fails the test on error.
func runGit(t *testing.T, dir string, args ...string) string {
t.Helper()
cmd := exec.Command("git", args...)
cmd.Dir = dir
// Repository-local hooks path, so a developer's global core.hooksPath
// cannot run inside the fixture repositories (engdocs/TESTING.md).
cmd.Env = append(os.Environ(),
"GIT_TERMINAL_PROMPT=0",
"GIT_AUTHOR_NAME=bd-test", "GIT_AUTHOR_EMAIL=bd-test@example.com",
"GIT_COMMITTER_NAME=bd-test", "GIT_COMMITTER_EMAIL=bd-test@example.com",
"GIT_CONFIG_COUNT=1", "GIT_CONFIG_KEY_0=core.hooksPath", "GIT_CONFIG_VALUE_0="+filepath.Join(dir, ".git", "hooks"),
)
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("git %s: %v\n%s", strings.Join(args, " "), err, out)
}
return string(out)
}
// newBareGitRemote creates a bare repository with one commit on main, the
// shape a git host presents before any Dolt data has been pushed to it.
func newBareGitRemote(t *testing.T, name string) string {
t.Helper()
base := t.TempDir()
bare := filepath.Join(base, name)
runGit(t, base, "init", "--bare", "-b", "main", bare)
seed := filepath.Join(base, "seed")
if err := os.MkdirAll(seed, 0o755); err != nil {
t.Fatal(err)
}
runGit(t, seed, "init", "-b", "main")
runGit(t, seed, "commit", "--allow-empty", "-m", "init")
runGit(t, seed, "push", bare, "main")
return bare
}
// lsRemoteRef returns the object a ref points at on the bare repository, or
// "" when the ref does not exist there.
func lsRemoteRef(t *testing.T, bare, ref string) string {
t.Helper()
return strings.TrimSpace(runGit(t, bare, "ls-remote", bare, ref))
}
// The engine, not a mock, is the authority on the SQL shapes above: a remote
// added with a ref lists that ref back from dolt_remotes and from
// repo_state.json, for a branch and for a ref outside refs/heads/; a remote
// added without one lists the default; Dolt refuses a ref on a non-git URL
// and the refusal reaches the caller; a push lands on the custom ref and
// never on refs/dolt/data; a clone with the ref records it on origin.
func TestRemoteGitDataRefOnEngine(t *testing.T) {
if _, err := exec.LookPath("git"); err != nil {
t.Skip("git not available")
}
ctx := context.Background()
te := newTestEnv(t, "gref")
shapes := map[string]string{
"branch": "refs/heads/issue-data",
"unit": "refs/dolt/units/team-12542",
}
bares := map[string]string{}
for name, ref := range shapes {
bares[name] = newBareGitRemote(t, name)
if err := te.store.AddRemoteWithRef(ctx, name, "git+file://"+bares[name], ref); err != nil {
t.Fatalf("AddRemoteWithRef(%s, %s): %v", name, ref, err)
}
}
plainDir := t.TempDir()
if err := te.store.AddRemote(ctx, "plain", "file://"+plainDir); err != nil {
t.Fatalf("AddRemote(plain): %v", err)
}
t.Run("listed from SQL and from disk", func(t *testing.T) {
remotes, err := te.store.ListRemotes(ctx)
if err != nil {
t.Fatalf("ListRemotes: %v", err)
}
assertRefs(t, "ListRemotes", remotes, shapes)
persisted, err := doltutil.PersistedRemotes(filepath.Join(te.dataDir, te.database))
if err != nil {
t.Fatalf("PersistedRemotes: %v", err)
}
assertRefs(t, "PersistedRemotes", persisted, shapes)
})
t.Run("ref on a non-git remote is Dolt's error, not a silent default", func(t *testing.T) {
err := te.store.AddRemoteWithRef(ctx, "bad", "file://"+t.TempDir(), "refs/heads/issue-data")
if err == nil {
t.Fatal("AddRemoteWithRef on file:// with a ref = nil, want error")
}
if !strings.Contains(err.Error(), "git remotes") {
t.Errorf("error should be Dolt's --ref refusal, got: %v", err)
}
if ok, _ := te.store.HasRemote(ctx, "bad"); ok {
t.Error("refused remote must not exist")
}
})
for name, ref := range shapes {
t.Run("push and clone on "+ref, func(t *testing.T) {
bare := bares[name]
if err := te.store.PushRemote(ctx, name, false); err != nil {
t.Fatalf("PushRemote(%s): %v", name, err)
}
if got := lsRemoteRef(t, bare, ref); got == "" {
t.Fatalf("push did not create %s on the git remote", ref)
}
if got := lsRemoteRef(t, bare, storage.DefaultGitDataRef); got != "" {
t.Errorf("push created %s (%s) although the remote was added with --ref %s", storage.DefaultGitDataRef, got, ref)
}
if got := lsRemoteRef(t, bare, "refs/heads/main"); got == "" {
t.Error("refs/heads/main disappeared from the git remote")
}
cloneDir := t.TempDir()
boot, bootCleanup, err := embeddeddolt.OpenSQL(ctx, cloneDir, "", "")
if err != nil {
t.Fatalf("OpenSQL for clone: %v", err)
}
cloneErr := versioncontrolops.DoltCloneWithRef(ctx, boot, "git+file://"+bare, "cloned", "", ref)
_ = bootCleanup()
if cloneErr != nil {
t.Fatalf("DoltCloneWithRef: %v", cloneErr)
}
persisted, err := doltutil.PersistedRemotes(filepath.Join(cloneDir, "cloned"))
if err != nil {
t.Fatalf("PersistedRemotes on clone: %v", err)
}
assertRefs(t, "clone repo_state.json", persisted, map[string]string{"origin": ref})
db, cleanup, err := embeddeddolt.OpenSQL(ctx, cloneDir, "cloned", "main")
if err != nil {
t.Fatalf("OpenSQL on clone: %v", err)
}
defer func() { _ = cleanup() }()
listed, err := versioncontrolops.ListRemotes(ctx, db)
if err != nil {
t.Fatalf("ListRemotes on clone: %v", err)
}
assertRefs(t, "clone dolt_remotes", listed, map[string]string{"origin": ref})
})
}
}
// assertRefs checks that every named remote in want lists exactly that ref
// and that the remote named plain, when present, lists the default.
func assertRefs(t *testing.T, where string, remotes []storage.RemoteInfo, want map[string]string) {
t.Helper()
got := map[string]string{}
for _, r := range remotes {
got[r.Name] = r.Ref
}
for name, ref := range want {
if _, ok := got[name]; !ok {
t.Errorf("%s: remote %s missing from %+v", where, name, remotes)
continue
}
if got[name] != ref {
t.Errorf("%s: remote %s Ref = %q, want %q", where, name, got[name], ref)
}
}
if ref, ok := got["plain"]; ok && ref != "" {
t.Errorf("%s: remote plain Ref = %q, want empty", where, ref)
}
}