* feat(storage): carry the git data ref of a git-backed Dolt remote Dolt lets a git-backed remote keep its data on a ref other than refs/dolt/data (the git_ref remote parameter, --ref on dolt remote add, dolt clone, DOLT_REMOTE, and DOLT_CLONE). The storage layer never passed it through, so a host that only accepts pushes under refs/heads/, or one repository holding several databases on refs such as refs/dolt/units/<key>, could not be used. Part of #6261; the commands and the config key follow in a stacked PR. - storage.RemoteInfo gains Ref (empty = refs/dolt/data); RemoteRefsMatch, EffectiveGitDataRef, and GitRefFromParamsJSON give every package one reading of it, pinned against dolt's constants. - RemoteStore.AddRemoteWithRef on both stores, and CreateRemoteWithRef on the proxied-server remote use case, pass a non-empty ref as DOLT_REMOTE's --ref. AddRemote is unchanged; UpdateRemote keeps the ref through replacement and restore. - versioncontrolops.DoltCloneWithRef and dolt.BootstrapFromRemoteWithDBRef pass the ref to DOLT_CLONE and to the dolt clone argv. - ListRemotes reads the ref back from dolt_remotes.params (git_ref) in both remote repositories; PersistedRemotes reads it from repo_state.json. Malformed params is an error, never a silent default. - The server-mode CLI mirror is compared on URL and ref and re-materialized with dolt remote add --ref when the ref differs. Where the CLI is proxied to the running server: the check reads the server's own repo_state.json when the listing is empty, the cold-start fallback re-adds a ref remote over SQL (which accepts --ref) so the server knows it, and a throwaway probe remote proves --ref works before an existing mirror remote is removed; probe remotes left by runs no longer alive are swept before any change to the mirror. For a server this bd does not own, the route stops and names the DOLT_REMOTE call to run against that server through bd sql, as one shell-quoted word. - A ref is checked for a space, an ASCII control character, DEL, and a leading dash before it becomes a dolt argument; no other ref-name rule applies at this layer. - Tests carry both ref shapes; the embedded-engine test proves add, list, push, and clone on the real engine; sqlmock tests cover the route decider and the cold-start re-add at the store seam; the dolt-CLI tests skip without a dolt binary. * fix(storage): make both git_ref readers fail closed; pin the edges (#6521 review) Maintainer review fixup for PR #6521. The review found no blocker or major; this is the non-gating minor/nit set plus the test pins the synthesis named as the residual defects. - One reader for git_ref. storage.GitRefFromParams is now the single decoder for both planes (the SQL params column through GitRefFromParamsJSON, and repo_state.json through doltutil.PersistedRemotes), and a git_ref that is present but not a string is an error naming the remote instead of silently the default ref. Reading it as the default is how a mirror pinned to a custom ref would be kept on refs/dolt/data -- the wrong-ref push this PR closes, reached from the opposite direction. - CreateRemoteWithRef checks the ref against what the dolt argv boundary will accept, so the recordable ref set is no longer wider than the routable one and a bad ref is refused at add time rather than diagnosed at push time. UpdateRemote deliberately does not re-check: it carries and restores refs that are already recorded and must stay able to put one back. - BootstrapFromRemoteWithDBRef validates the ref before the already-exists early return, so the same --ref is accepted or refused the same way on a fresh host and on a bootstrapped one. - sweepLeftoverProbes warns when it cannot remove another run's leftover probe remote, following PersistedRemoteInfos' precedent: the remote stays visible in `bd remote list` until some run succeeds, and nothing ever said why. - EnsureCLIRemote's doc comment records that escalating a state-file read error is intentional for default-ref remotes too, and TestCLIMirrorUnreadableStateIsAnError now pins the ref == "" direction that a later "helpful" relaxation would gate away. The suggested URL-match short-circuit is deliberately NOT applied: when the state file is unreadable currentRef is unknown, so returning nil on a URL match alone would route a push onto whatever ref the mirror is secretly pinned to. - AddCLIRemote's doc comment records that it is kept deliberately as the default-ref spelling now that EnsureCLIRemote uses AddCLIRemoteWithRef. Not applied: widening probeCLIRemoteRef to `ref != "" || currentRef != ""` and probing the restore shape. On a proxied server that turns a working custom-ref -> default-ref transition into a hard failure (the target add is plain and succeeds today, while probing the restore shape would be refused), and in the one window that survives calibration -- a non-proxied plain-add failure from disk/lock contention -- a capability probe predicts nothing about it. Each new pin was mutation-proved: reverting the fix it guards reds that pin, and only that pin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Julian Knutsen <julian@gastownhall.dev>
185 lines
6.3 KiB
Go
185 lines
6.3 KiB
Go
//go:build cgo
|
|
|
|
package embeddeddolt_test
|
|
|
|
import (
|
|
"context"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/steveyegge/beads/internal/storage"
|
|
"github.com/steveyegge/beads/internal/storage/doltutil"
|
|
"github.com/steveyegge/beads/internal/storage/embeddeddolt"
|
|
"github.com/steveyegge/beads/internal/storage/versioncontrolops"
|
|
)
|
|
|
|
// runGit runs git in dir and fails the test on error.
|
|
func runGit(t *testing.T, dir string, args ...string) string {
|
|
t.Helper()
|
|
cmd := exec.Command("git", args...)
|
|
cmd.Dir = dir
|
|
// Repository-local hooks path, so a developer's global core.hooksPath
|
|
// cannot run inside the fixture repositories (engdocs/TESTING.md).
|
|
cmd.Env = append(os.Environ(),
|
|
"GIT_TERMINAL_PROMPT=0",
|
|
"GIT_AUTHOR_NAME=bd-test", "GIT_AUTHOR_EMAIL=bd-test@example.com",
|
|
"GIT_COMMITTER_NAME=bd-test", "GIT_COMMITTER_EMAIL=bd-test@example.com",
|
|
"GIT_CONFIG_COUNT=1", "GIT_CONFIG_KEY_0=core.hooksPath", "GIT_CONFIG_VALUE_0="+filepath.Join(dir, ".git", "hooks"),
|
|
)
|
|
out, err := cmd.CombinedOutput()
|
|
if err != nil {
|
|
t.Fatalf("git %s: %v\n%s", strings.Join(args, " "), err, out)
|
|
}
|
|
return string(out)
|
|
}
|
|
|
|
// newBareGitRemote creates a bare repository with one commit on main, the
|
|
// shape a git host presents before any Dolt data has been pushed to it.
|
|
func newBareGitRemote(t *testing.T, name string) string {
|
|
t.Helper()
|
|
base := t.TempDir()
|
|
bare := filepath.Join(base, name)
|
|
runGit(t, base, "init", "--bare", "-b", "main", bare)
|
|
seed := filepath.Join(base, "seed")
|
|
if err := os.MkdirAll(seed, 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
runGit(t, seed, "init", "-b", "main")
|
|
runGit(t, seed, "commit", "--allow-empty", "-m", "init")
|
|
runGit(t, seed, "push", bare, "main")
|
|
return bare
|
|
}
|
|
|
|
// lsRemoteRef returns the object a ref points at on the bare repository, or
|
|
// "" when the ref does not exist there.
|
|
func lsRemoteRef(t *testing.T, bare, ref string) string {
|
|
t.Helper()
|
|
return strings.TrimSpace(runGit(t, bare, "ls-remote", bare, ref))
|
|
}
|
|
|
|
// The engine, not a mock, is the authority on the SQL shapes above: a remote
|
|
// added with a ref lists that ref back from dolt_remotes and from
|
|
// repo_state.json, for a branch and for a ref outside refs/heads/; a remote
|
|
// added without one lists the default; Dolt refuses a ref on a non-git URL
|
|
// and the refusal reaches the caller; a push lands on the custom ref and
|
|
// never on refs/dolt/data; a clone with the ref records it on origin.
|
|
func TestRemoteGitDataRefOnEngine(t *testing.T) {
|
|
if _, err := exec.LookPath("git"); err != nil {
|
|
t.Skip("git not available")
|
|
}
|
|
ctx := context.Background()
|
|
te := newTestEnv(t, "gref")
|
|
|
|
shapes := map[string]string{
|
|
"branch": "refs/heads/issue-data",
|
|
"unit": "refs/dolt/units/team-12542",
|
|
}
|
|
bares := map[string]string{}
|
|
for name, ref := range shapes {
|
|
bares[name] = newBareGitRemote(t, name)
|
|
if err := te.store.AddRemoteWithRef(ctx, name, "git+file://"+bares[name], ref); err != nil {
|
|
t.Fatalf("AddRemoteWithRef(%s, %s): %v", name, ref, err)
|
|
}
|
|
}
|
|
plainDir := t.TempDir()
|
|
if err := te.store.AddRemote(ctx, "plain", "file://"+plainDir); err != nil {
|
|
t.Fatalf("AddRemote(plain): %v", err)
|
|
}
|
|
|
|
t.Run("listed from SQL and from disk", func(t *testing.T) {
|
|
remotes, err := te.store.ListRemotes(ctx)
|
|
if err != nil {
|
|
t.Fatalf("ListRemotes: %v", err)
|
|
}
|
|
assertRefs(t, "ListRemotes", remotes, shapes)
|
|
|
|
persisted, err := doltutil.PersistedRemotes(filepath.Join(te.dataDir, te.database))
|
|
if err != nil {
|
|
t.Fatalf("PersistedRemotes: %v", err)
|
|
}
|
|
assertRefs(t, "PersistedRemotes", persisted, shapes)
|
|
})
|
|
|
|
t.Run("ref on a non-git remote is Dolt's error, not a silent default", func(t *testing.T) {
|
|
err := te.store.AddRemoteWithRef(ctx, "bad", "file://"+t.TempDir(), "refs/heads/issue-data")
|
|
if err == nil {
|
|
t.Fatal("AddRemoteWithRef on file:// with a ref = nil, want error")
|
|
}
|
|
if !strings.Contains(err.Error(), "git remotes") {
|
|
t.Errorf("error should be Dolt's --ref refusal, got: %v", err)
|
|
}
|
|
if ok, _ := te.store.HasRemote(ctx, "bad"); ok {
|
|
t.Error("refused remote must not exist")
|
|
}
|
|
})
|
|
|
|
for name, ref := range shapes {
|
|
t.Run("push and clone on "+ref, func(t *testing.T) {
|
|
bare := bares[name]
|
|
if err := te.store.PushRemote(ctx, name, false); err != nil {
|
|
t.Fatalf("PushRemote(%s): %v", name, err)
|
|
}
|
|
if got := lsRemoteRef(t, bare, ref); got == "" {
|
|
t.Fatalf("push did not create %s on the git remote", ref)
|
|
}
|
|
if got := lsRemoteRef(t, bare, storage.DefaultGitDataRef); got != "" {
|
|
t.Errorf("push created %s (%s) although the remote was added with --ref %s", storage.DefaultGitDataRef, got, ref)
|
|
}
|
|
if got := lsRemoteRef(t, bare, "refs/heads/main"); got == "" {
|
|
t.Error("refs/heads/main disappeared from the git remote")
|
|
}
|
|
|
|
cloneDir := t.TempDir()
|
|
boot, bootCleanup, err := embeddeddolt.OpenSQL(ctx, cloneDir, "", "")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL for clone: %v", err)
|
|
}
|
|
cloneErr := versioncontrolops.DoltCloneWithRef(ctx, boot, "git+file://"+bare, "cloned", "", ref)
|
|
_ = bootCleanup()
|
|
if cloneErr != nil {
|
|
t.Fatalf("DoltCloneWithRef: %v", cloneErr)
|
|
}
|
|
persisted, err := doltutil.PersistedRemotes(filepath.Join(cloneDir, "cloned"))
|
|
if err != nil {
|
|
t.Fatalf("PersistedRemotes on clone: %v", err)
|
|
}
|
|
assertRefs(t, "clone repo_state.json", persisted, map[string]string{"origin": ref})
|
|
|
|
db, cleanup, err := embeddeddolt.OpenSQL(ctx, cloneDir, "cloned", "main")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL on clone: %v", err)
|
|
}
|
|
defer func() { _ = cleanup() }()
|
|
listed, err := versioncontrolops.ListRemotes(ctx, db)
|
|
if err != nil {
|
|
t.Fatalf("ListRemotes on clone: %v", err)
|
|
}
|
|
assertRefs(t, "clone dolt_remotes", listed, map[string]string{"origin": ref})
|
|
})
|
|
}
|
|
}
|
|
|
|
// assertRefs checks that every named remote in want lists exactly that ref
|
|
// and that the remote named plain, when present, lists the default.
|
|
func assertRefs(t *testing.T, where string, remotes []storage.RemoteInfo, want map[string]string) {
|
|
t.Helper()
|
|
got := map[string]string{}
|
|
for _, r := range remotes {
|
|
got[r.Name] = r.Ref
|
|
}
|
|
for name, ref := range want {
|
|
if _, ok := got[name]; !ok {
|
|
t.Errorf("%s: remote %s missing from %+v", where, name, remotes)
|
|
continue
|
|
}
|
|
if got[name] != ref {
|
|
t.Errorf("%s: remote %s Ref = %q, want %q", where, name, got[name], ref)
|
|
}
|
|
}
|
|
if ref, ok := got["plain"]; ok && ref != "" {
|
|
t.Errorf("%s: remote plain Ref = %q, want empty", where, ref)
|
|
}
|
|
}
|