feat(storage): carry the git data ref of a git-backed Dolt remote (#6521)
* feat(storage): carry the git data ref of a git-backed Dolt remote Dolt lets a git-backed remote keep its data on a ref other than refs/dolt/data (the git_ref remote parameter, --ref on dolt remote add, dolt clone, DOLT_REMOTE, and DOLT_CLONE). The storage layer never passed it through, so a host that only accepts pushes under refs/heads/, or one repository holding several databases on refs such as refs/dolt/units/<key>, could not be used. Part of #6261; the commands and the config key follow in a stacked PR. - storage.RemoteInfo gains Ref (empty = refs/dolt/data); RemoteRefsMatch, EffectiveGitDataRef, and GitRefFromParamsJSON give every package one reading of it, pinned against dolt's constants. - RemoteStore.AddRemoteWithRef on both stores, and CreateRemoteWithRef on the proxied-server remote use case, pass a non-empty ref as DOLT_REMOTE's --ref. AddRemote is unchanged; UpdateRemote keeps the ref through replacement and restore. - versioncontrolops.DoltCloneWithRef and dolt.BootstrapFromRemoteWithDBRef pass the ref to DOLT_CLONE and to the dolt clone argv. - ListRemotes reads the ref back from dolt_remotes.params (git_ref) in both remote repositories; PersistedRemotes reads it from repo_state.json. Malformed params is an error, never a silent default. - The server-mode CLI mirror is compared on URL and ref and re-materialized with dolt remote add --ref when the ref differs. Where the CLI is proxied to the running server: the check reads the server's own repo_state.json when the listing is empty, the cold-start fallback re-adds a ref remote over SQL (which accepts --ref) so the server knows it, and a throwaway probe remote proves --ref works before an existing mirror remote is removed; probe remotes left by runs no longer alive are swept before any change to the mirror. For a server this bd does not own, the route stops and names the DOLT_REMOTE call to run against that server through bd sql, as one shell-quoted word. - A ref is checked for a space, an ASCII control character, DEL, and a leading dash before it becomes a dolt argument; no other ref-name rule applies at this layer. - Tests carry both ref shapes; the embedded-engine test proves add, list, push, and clone on the real engine; sqlmock tests cover the route decider and the cold-start re-add at the store seam; the dolt-CLI tests skip without a dolt binary. * fix(storage): make both git_ref readers fail closed; pin the edges (#6521 review) Maintainer review fixup for PR #6521. The review found no blocker or major; this is the non-gating minor/nit set plus the test pins the synthesis named as the residual defects. - One reader for git_ref. storage.GitRefFromParams is now the single decoder for both planes (the SQL params column through GitRefFromParamsJSON, and repo_state.json through doltutil.PersistedRemotes), and a git_ref that is present but not a string is an error naming the remote instead of silently the default ref. Reading it as the default is how a mirror pinned to a custom ref would be kept on refs/dolt/data -- the wrong-ref push this PR closes, reached from the opposite direction. - CreateRemoteWithRef checks the ref against what the dolt argv boundary will accept, so the recordable ref set is no longer wider than the routable one and a bad ref is refused at add time rather than diagnosed at push time. UpdateRemote deliberately does not re-check: it carries and restores refs that are already recorded and must stay able to put one back. - BootstrapFromRemoteWithDBRef validates the ref before the already-exists early return, so the same --ref is accepted or refused the same way on a fresh host and on a bootstrapped one. - sweepLeftoverProbes warns when it cannot remove another run's leftover probe remote, following PersistedRemoteInfos' precedent: the remote stays visible in `bd remote list` until some run succeeds, and nothing ever said why. - EnsureCLIRemote's doc comment records that escalating a state-file read error is intentional for default-ref remotes too, and TestCLIMirrorUnreadableStateIsAnError now pins the ref == "" direction that a later "helpful" relaxation would gate away. The suggested URL-match short-circuit is deliberately NOT applied: when the state file is unreadable currentRef is unknown, so returning nil on a URL match alone would route a push onto whatever ref the mirror is secretly pinned to. - AddCLIRemote's doc comment records that it is kept deliberately as the default-ref spelling now that EnsureCLIRemote uses AddCLIRemoteWithRef. Not applied: widening probeCLIRemoteRef to `ref != "" || currentRef != ""` and probing the restore shape. On a proxied server that turns a working custom-ref -> default-ref transition into a hard failure (the target add is plain and succeeds today, while probing the restore shape would be refused), and in the one window that survives calibration -- a non-proxied plain-add failure from disk/lock contention -- a capability probe predicts nothing about it. Each new pin was mutation-proved: reverting the fix it guards reds that pin, and only that pin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Julian Knutsen <julian@gastownhall.dev>
This commit is contained in:
parent
70a12ba664
commit
3efcd3571c
29 changed files with 2098 additions and 63 deletions
|
|
@ -6,6 +6,7 @@ import (
|
|||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/steveyegge/beads/internal/configfile"
|
||||
|
|
@ -13,6 +14,7 @@ import (
|
|||
"github.com/steveyegge/beads/internal/gittraceenv"
|
||||
"github.com/steveyegge/beads/internal/lockfile"
|
||||
"github.com/steveyegge/beads/internal/remotecache"
|
||||
"github.com/steveyegge/beads/internal/storage/doltutil"
|
||||
)
|
||||
|
||||
// staleLockAge is the maximum age of a lock file before it's considered stale.
|
||||
|
|
@ -46,6 +48,21 @@ func BootstrapFromGitRemote(ctx context.Context, doltDir, gitRemoteURL string) (
|
|||
// callers should use cfg.GetDoltDatabase() which applies the fallback chain
|
||||
// (env var → config → default).
|
||||
func BootstrapFromRemoteWithDB(ctx context.Context, doltDir, remoteURL, database string) (bool, error) {
|
||||
return BootstrapFromRemoteWithDBRef(ctx, doltDir, remoteURL, database, "")
|
||||
}
|
||||
|
||||
// BootstrapFromRemoteWithDBRef is BootstrapFromRemoteWithDB for a git-backed
|
||||
// remote whose Dolt data lives on the git ref ref (`dolt clone --ref`). An
|
||||
// empty ref is Dolt's default, refs/dolt/data.
|
||||
func BootstrapFromRemoteWithDBRef(ctx context.Context, doltDir, remoteURL, database, ref string) (bool, error) {
|
||||
// The ref is checked before the already-exists early return so the same
|
||||
// --ref argument is accepted or refused the same way on every host, rather
|
||||
// than surfacing a typo only on a machine that has not bootstrapped yet.
|
||||
// The URL and database checks keep their existing position below it.
|
||||
if err := doltutil.ValidateGitDataRefArg(strings.TrimSpace(ref)); err != nil {
|
||||
return false, fmt.Errorf("invalid git data ref: %w", err)
|
||||
}
|
||||
|
||||
// Skip if Dolt database already exists
|
||||
if doltExists(doltDir) {
|
||||
return false, nil
|
||||
|
|
@ -78,7 +95,7 @@ func BootstrapFromRemoteWithDB(ctx context.Context, doltDir, remoteURL, database
|
|||
// from an earlier bootstrap that a stale/empty doltExists() check
|
||||
// missed) that we must not delete.
|
||||
targetPreExisted := pathExists(cloneTarget)
|
||||
cmd := bootstrapCloneCmd(ctx, remoteURL, cloneTarget)
|
||||
cmd := bootstrapCloneCmd(ctx, remoteURL, cloneTarget, ref)
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
if targetPreExisted {
|
||||
return false, fmt.Errorf("dolt clone failed: %w\nOutput: %s\nClone target %q already existed before this attempt; left untouched to avoid deleting a pre-existing Dolt repo", err, output, cloneTarget)
|
||||
|
|
@ -94,8 +111,8 @@ func BootstrapFromRemoteWithDB(ctx context.Context, doltDir, remoteURL, database
|
|||
// bootstrapCloneCmd builds the `dolt clone` for remote bootstrap. It does not
|
||||
// route through prepareDoltCLITransferCommand, so it applies the remote env
|
||||
// guards itself (see internal/gittraceenv and internal/githooksenv).
|
||||
func bootstrapCloneCmd(ctx context.Context, remoteURL, cloneTarget string) *exec.Cmd {
|
||||
cmd := exec.CommandContext(ctx, "dolt", doltCloneArgs(remoteURL, cloneTarget)...)
|
||||
func bootstrapCloneCmd(ctx context.Context, remoteURL, cloneTarget, ref string) *exec.Cmd {
|
||||
cmd := exec.CommandContext(ctx, "dolt", doltCloneArgs(remoteURL, cloneTarget, ref)...) // #nosec G204 -- validated remote URL; the ref is an option value, taken verbatim by dolt's argparser and checked by doltutil.ValidateGitDataRefArg
|
||||
cmd.Env = githooksenv.DisabledEnv(gittraceenv.ScrubEnv(os.Environ()))
|
||||
return cmd
|
||||
}
|
||||
|
|
@ -144,11 +161,17 @@ func formatFailedCloneTargetError(cloneErr error, output []byte, cloneTarget str
|
|||
return fmt.Errorf("dolt clone failed: %w\nOutput: %s\nCould not clean up failed clone target %q after retrying: %v\nOn Windows this usually means a dolt or bd process, or antivirus scanner, still has a file handle open under `.dolt/noms/LOCK`. Stop stuck dolt/bd processes, wait a moment, delete the directory manually if it remains, then retry `bd bootstrap`", cloneErr, output, cloneTarget, cleanupErr)
|
||||
}
|
||||
|
||||
func doltCloneArgs(remoteURL, target string) []string {
|
||||
// doltCloneArgs builds the `dolt clone` argv. A non-empty ref adds --ref so
|
||||
// the clone reads a git-backed remote's Dolt data from that git ref instead
|
||||
// of the default refs/dolt/data.
|
||||
func doltCloneArgs(remoteURL, target, ref string) []string {
|
||||
args := []string{"clone"}
|
||||
if user := os.Getenv("DOLT_REMOTE_USER"); user != "" {
|
||||
args = append(args, "--user", user)
|
||||
}
|
||||
if ref = strings.TrimSpace(ref); ref != "" {
|
||||
args = append(args, "--ref", ref)
|
||||
}
|
||||
return append(args, remoteURL, target)
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ func TestBootstrapCloneCmdEnvIsGuarded(t *testing.T) {
|
|||
t.Setenv("GIT_TRACE", "1")
|
||||
t.Setenv("GIT_CURL_VERBOSE", "1")
|
||||
|
||||
cmd := bootstrapCloneCmd(context.Background(), "git+https://example.com/repo.git", filepath.Join(t.TempDir(), "beads"))
|
||||
cmd := bootstrapCloneCmd(context.Background(), "git+https://example.com/repo.git", filepath.Join(t.TempDir(), "beads"), "")
|
||||
if cmd.Env == nil {
|
||||
t.Fatal("bootstrapCloneCmd() left cmd.Env nil; the clone would inherit stderr-directed git tracing")
|
||||
}
|
||||
|
|
@ -153,22 +153,70 @@ func TestBootstrapFromRemoteWithDB_PreservesPreExistingCloneTarget(t *testing.T)
|
|||
}
|
||||
}
|
||||
|
||||
// An invalid --ref is refused on a host whose database already exists, not
|
||||
// only on a fresh one: the ref check runs before the already-exists early
|
||||
// return, so the same argument is accepted or refused the same way everywhere.
|
||||
// The valid-ref control proves the early return still fires, so the refusal
|
||||
// comes from the ref and not from the call failing for some other reason.
|
||||
// Needs no dolt binary — both outcomes are decided before the CLI lookup.
|
||||
func TestBootstrapFromRemoteWithDBRef_RefusesInvalidRefOnBootstrappedHost(t *testing.T) {
|
||||
doltDir := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(doltDir, "beads", ".dolt"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !doltExists(doltDir) {
|
||||
t.Fatal("test setup: doltExists should see the pre-existing database")
|
||||
}
|
||||
|
||||
for _, ref := range []string{"-x", "refs/heads/issue data"} {
|
||||
cloned, err := BootstrapFromRemoteWithDBRef(context.Background(), doltDir, "git+https://example.com/repo.git", "beads", ref)
|
||||
if err == nil || !strings.Contains(err.Error(), "invalid git data ref") {
|
||||
t.Errorf("BootstrapFromRemoteWithDBRef(ref=%q) = %v, %v; want the invalid-ref error", ref, cloned, err)
|
||||
}
|
||||
}
|
||||
|
||||
cloned, err := BootstrapFromRemoteWithDBRef(context.Background(), doltDir, "git+https://example.com/repo.git", "beads", "refs/dolt/units/team-12542")
|
||||
if cloned || err != nil {
|
||||
t.Fatalf("BootstrapFromRemoteWithDBRef with a valid ref on an existing database = %v, %v; want false, nil", cloned, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDoltCloneArgs(t *testing.T) {
|
||||
t.Setenv("DOLT_REMOTE_USER", "")
|
||||
got := doltCloneArgs("https://example.com/repo", "/tmp/clone")
|
||||
got := doltCloneArgs("https://example.com/repo", "/tmp/clone", "")
|
||||
want := []string{"clone", "https://example.com/repo", "/tmp/clone"}
|
||||
if strings.Join(got, "\x00") != strings.Join(want, "\x00") {
|
||||
t.Fatalf("doltCloneArgs() = %q, want %q", got, want)
|
||||
}
|
||||
|
||||
t.Setenv("DOLT_REMOTE_USER", "alice")
|
||||
got = doltCloneArgs("https://example.com/repo", "/tmp/clone")
|
||||
got = doltCloneArgs("https://example.com/repo", "/tmp/clone", "")
|
||||
want = []string{"clone", "--user", "alice", "https://example.com/repo", "/tmp/clone"}
|
||||
if strings.Join(got, "\x00") != strings.Join(want, "\x00") {
|
||||
t.Fatalf("doltCloneArgs() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// A git data ref reaches `dolt clone` as --ref, verbatim, whether it is a
|
||||
// branch under refs/heads/ or a ref elsewhere in the namespace.
|
||||
func TestDoltCloneArgsWithRef(t *testing.T) {
|
||||
t.Setenv("DOLT_REMOTE_USER", "")
|
||||
for _, ref := range []string{"refs/heads/issue-data", "refs/dolt/units/team-12542"} {
|
||||
got := doltCloneArgs("git+https://example.com/repo.git", "/tmp/clone", ref)
|
||||
want := []string{"clone", "--ref", ref, "git+https://example.com/repo.git", "/tmp/clone"}
|
||||
if strings.Join(got, "\x00") != strings.Join(want, "\x00") {
|
||||
t.Fatalf("doltCloneArgs(ref=%q) = %q, want %q", ref, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
t.Setenv("DOLT_REMOTE_USER", "alice")
|
||||
got := doltCloneArgs("git+https://example.com/repo.git", "/tmp/clone", " refs/heads/issue-data ")
|
||||
want := []string{"clone", "--user", "alice", "--ref", "refs/heads/issue-data", "git+https://example.com/repo.git", "/tmp/clone"}
|
||||
if strings.Join(got, "\x00") != strings.Join(want, "\x00") {
|
||||
t.Fatalf("doltCloneArgs() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRemoveFailedCloneTargetWithRetryRemovesPartialClone(t *testing.T) {
|
||||
oldDelays := failedCloneCleanupRetryDelays
|
||||
failedCloneCleanupRetryDelays = []time.Duration{0}
|
||||
|
|
|
|||
267
internal/storage/dolt/cli_mirror_ref_test.go
Normal file
267
internal/storage/dolt/cli_mirror_ref_test.go
Normal file
|
|
@ -0,0 +1,267 @@
|
|||
package dolt
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
sqlmock "github.com/DATA-DOG/go-sqlmock"
|
||||
)
|
||||
|
||||
// writeCLIRepoState gives cliDir a .dolt/repo_state.json with one remote,
|
||||
// the state a served database directory holds, without a working Dolt
|
||||
// repository behind it: `dolt remote -v` there fails or lists nothing, which
|
||||
// is also what the proxied CLI reports at a server's cold start.
|
||||
func writeCLIRepoState(t *testing.T, cliDir, name, url, ref string) {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(filepath.Join(cliDir, ".dolt"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
params := "{}"
|
||||
if ref != "" {
|
||||
params = `{"git_ref":"` + ref + `"}`
|
||||
}
|
||||
body := `{"head":"refs/heads/main","remotes":{"` + name + `":{"name":"` + name + `","url":"` + url + `","fetch_specs":["refs/heads/*:refs/remotes/` + name + `/*"],"params":` + params + `}},"backups":{},"branches":{}}`
|
||||
if err := os.WriteFile(filepath.Join(cliDir, ".dolt", "repo_state.json"), []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func newServedStoreForCLIMirror(t *testing.T) (*DoltStore, string) {
|
||||
t.Helper()
|
||||
// These tests stand in for a proxied CLI that lists nothing; they must
|
||||
// not depend on what a real dolt binary does in a directory that holds
|
||||
// only a state file, so no dolt is reachable from PATH here.
|
||||
t.Setenv("PATH", t.TempDir())
|
||||
root := t.TempDir()
|
||||
store := &DoltStore{serverMode: true, dbPath: root, database: "beads"}
|
||||
cliDir := store.CLIDir()
|
||||
if cliDir != filepath.Join(root, "beads") {
|
||||
t.Fatalf("CLIDir() = %q, want %q", cliDir, filepath.Join(root, "beads"))
|
||||
}
|
||||
return store, cliDir
|
||||
}
|
||||
|
||||
// When the CLI in cliDir lists nothing, the mirror check for a remote on a
|
||||
// custom ref consults the state file in cliDir: matching URL and ref is a
|
||||
// match and nothing is re-materialized; a different ref is not a match. A
|
||||
// remote on the default ref keeps the previous behavior and does not consult
|
||||
// the file. Both ref shapes.
|
||||
func TestHasMatchingCLIRemoteFallsBackToPersistedStateForRef(t *testing.T) {
|
||||
const url = "git+https://example.com/repo.git"
|
||||
for _, ref := range []string{"refs/heads/issue-data", "refs/dolt/units/team-12542"} {
|
||||
t.Run(ref, func(t *testing.T) {
|
||||
store, cliDir := newServedStoreForCLIMirror(t)
|
||||
writeCLIRepoState(t, cliDir, "origin", url, ref)
|
||||
|
||||
if !store.hasMatchingCLIRemote("origin", url, ref) {
|
||||
t.Fatal("same URL and ref recorded in cliDir should match")
|
||||
}
|
||||
if store.hasMatchingCLIRemote("origin", url, "refs/heads/elsewhere") {
|
||||
t.Fatal("a different ref must not match")
|
||||
}
|
||||
if store.hasMatchingCLIRemote("origin", "git+https://example.com/other.git", ref) {
|
||||
t.Fatal("a different URL must not match")
|
||||
}
|
||||
if err := store.ensureMatchingCLIRemote("origin", url, ref); err != nil {
|
||||
t.Fatalf("ensureMatchingCLIRemote on a matching mirror should not touch dolt: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("default ref does not consult the state file", func(t *testing.T) {
|
||||
store, cliDir := newServedStoreForCLIMirror(t)
|
||||
writeCLIRepoState(t, cliDir, "origin", url, "")
|
||||
if store.hasMatchingCLIRemote("origin", url, "") {
|
||||
t.Fatal("without a ref the check must stay on the CLI listing, which is empty here")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
const listRemotesQuery = "SELECT name, url, params FROM dolt_remotes"
|
||||
|
||||
// The git-protocol route decider carries the ref from the SQL listing into
|
||||
// the mirror check: a served directory whose state file records the same
|
||||
// URL and ref routes over the CLI; the same URL on another ref does not
|
||||
// (the re-materialization then needs the dolt binary, absent here, and the
|
||||
// error says so instead of a silent default). Dropping the ref at the call
|
||||
// site would pass the first case on the default ref and fail this test.
|
||||
func TestPrepareCLIRouteForGitProtocolCarriesRef(t *testing.T) {
|
||||
const url = "git+ssh://git@example.com/org/repo.git"
|
||||
for _, ref := range []string{"refs/heads/issue-data", "refs/dolt/units/team-12542"} {
|
||||
t.Run(ref, func(t *testing.T) {
|
||||
store, cliDir := newServedStoreForCLIMirror(t)
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = db.Close() })
|
||||
store.db = db
|
||||
store.remote = "origin"
|
||||
mock.MatchExpectationsInOrder(false)
|
||||
// One listing per prepareCLIRouteForGitProtocol call below.
|
||||
for i := 0; i < 2; i++ {
|
||||
mock.ExpectQuery(regexp.QuoteMeta(listRemotesQuery)).WillReturnRows(
|
||||
sqlmock.NewRows([]string{"name", "url", "params"}).AddRow("origin", url, `{"git_ref":"`+ref+`"}`))
|
||||
}
|
||||
|
||||
writeCLIRepoState(t, cliDir, "origin", url, ref)
|
||||
useCLI, err := store.prepareCLIRouteForGitProtocol(context.Background(), "origin")
|
||||
if err != nil || !useCLI {
|
||||
t.Fatalf("matching state file: useCLI=%v err=%v, want true, nil", useCLI, err)
|
||||
}
|
||||
|
||||
writeCLIRepoState(t, cliDir, "origin", url, "refs/heads/elsewhere")
|
||||
useCLI, err = store.prepareCLIRouteForGitProtocol(context.Background(), "origin")
|
||||
if err == nil || useCLI {
|
||||
t.Fatalf("state file on another ref: useCLI=%v err=%v, want an error from the re-materialization", useCLI, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// In the GH#2118 cold-start window (SQL lists nothing, the state file has the
|
||||
// remote) a remote on a custom ref is re-added over bd's SQL connection with
|
||||
// --ref before the CLI route is taken, so the server the proxied transfer
|
||||
// delegates to knows the remote; "already exists" from the server is the
|
||||
// state wanted. That write happens only when this process owns the served
|
||||
// directory (localActiveDatabaseDir is the CLI directory); for any other
|
||||
// server the route fails with the remedy instead of registering a remote
|
||||
// read from a client-local file. A remote on the default ref keeps the
|
||||
// previous path and issues no SQL add.
|
||||
func TestPrepareCLIRouteForGitProtocolColdStartReaddsRefRemoteOverSQL(t *testing.T) {
|
||||
const url = "git+ssh://git@example.com/org/repo.git"
|
||||
const ref = "refs/dolt/units/team-12542"
|
||||
newStore := func(t *testing.T) (*DoltStore, string, sqlmock.Sqlmock) {
|
||||
t.Helper()
|
||||
store, cliDir := newServedStoreForCLIMirror(t)
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = db.Close() })
|
||||
store.db = db
|
||||
store.remote = "origin"
|
||||
store.localActiveDatabaseDir = cliDir
|
||||
mock.MatchExpectationsInOrder(false)
|
||||
// One listing per prepareCLIRouteForGitProtocol call.
|
||||
mock.ExpectQuery(regexp.QuoteMeta(listRemotesQuery)).WillReturnRows(sqlmock.NewRows([]string{"name", "url", "params"}))
|
||||
|
||||
return store, cliDir, mock
|
||||
}
|
||||
|
||||
t.Run("ref remote is re-added with --ref", func(t *testing.T) {
|
||||
store, cliDir, mock := newStore(t)
|
||||
writeCLIRepoState(t, cliDir, "origin", url, ref)
|
||||
mock.ExpectExec(regexp.QuoteMeta("CALL DOLT_REMOTE('add', '--ref', ?, ?, ?)")).
|
||||
WithArgs(ref, "origin", url).WillReturnResult(sqlmock.NewResult(0, 1))
|
||||
useCLI, err := store.prepareCLIRouteForGitProtocol(context.Background(), "origin")
|
||||
if err != nil || !useCLI {
|
||||
t.Fatalf("useCLI=%v err=%v, want true, nil", useCLI, err)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("already exists is accepted", func(t *testing.T) {
|
||||
store, cliDir, mock := newStore(t)
|
||||
writeCLIRepoState(t, cliDir, "origin", url, ref)
|
||||
mock.ExpectExec(regexp.QuoteMeta("CALL DOLT_REMOTE('add', '--ref', ?, ?, ?)")).
|
||||
WithArgs(ref, "origin", url).WillReturnError(errRemoteAlreadyExistsForTest)
|
||||
useCLI, err := store.prepareCLIRouteForGitProtocol(context.Background(), "origin")
|
||||
if err != nil || !useCLI {
|
||||
t.Fatalf("useCLI=%v err=%v, want true, nil", useCLI, err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("unowned server is not written to", func(t *testing.T) {
|
||||
store, cliDir, mock := newStore(t)
|
||||
store.localActiveDatabaseDir = ""
|
||||
writeCLIRepoState(t, cliDir, "origin", url, ref)
|
||||
// No ExpectExec: an add would surface as a sqlmock error wrapped as
|
||||
// the SQL re-add failure, which the assertion below rejects.
|
||||
_, err := store.prepareCLIRouteForGitProtocol(context.Background(), "origin")
|
||||
if err == nil {
|
||||
t.Fatal("an unowned server must not be written to; the route must fail")
|
||||
}
|
||||
// The remedy must be something that works at this commit and
|
||||
// repairs the condition: the procedure call over bd's SQL
|
||||
// connection to that server.
|
||||
for _, want := range []string{"does not own", `bd sql 'CALL DOLT_REMOTE("add", "--ref", "` + ref + `", "origin", "` + url + `")'`} {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("error should contain %q: %v", want, err)
|
||||
}
|
||||
}
|
||||
if strings.Contains(err.Error(), "over SQL") {
|
||||
t.Fatalf("no SQL add may be attempted for an unowned server: %v", err)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("other SQL failure is reported", func(t *testing.T) {
|
||||
store, cliDir, mock := newStore(t)
|
||||
writeCLIRepoState(t, cliDir, "origin", url, ref)
|
||||
mock.ExpectExec(regexp.QuoteMeta("CALL DOLT_REMOTE('add', '--ref', ?, ?, ?)")).
|
||||
WithArgs(ref, "origin", url).WillReturnError(errSQLDownForTest)
|
||||
_, err := store.prepareCLIRouteForGitProtocol(context.Background(), "origin")
|
||||
if err == nil || !strings.Contains(err.Error(), "GH#2118") || !strings.Contains(err.Error(), ref) {
|
||||
t.Fatalf("err = %v, want the cold-start explanation naming the ref", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("default ref issues no SQL add", func(t *testing.T) {
|
||||
store, cliDir, mock := newStore(t)
|
||||
writeCLIRepoState(t, cliDir, "origin", url, "")
|
||||
// No ExpectExec: an unexpected add comes back from sqlmock as an
|
||||
// error that the code would wrap as the SQL re-add failure. The
|
||||
// error seen must instead be the CLI re-materialization failing
|
||||
// (no dolt repository behind the state file), the previous behavior.
|
||||
_, err := store.prepareCLIRouteForGitProtocol(context.Background(), "origin")
|
||||
if err == nil {
|
||||
t.Fatal("without a usable CLI mirror the route must fail")
|
||||
}
|
||||
if strings.Contains(err.Error(), "over SQL") {
|
||||
t.Fatalf("a default-ref remote must not be re-added over SQL: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "CLI") {
|
||||
t.Fatalf("expected the CLI materialization failure, got: %v", err)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// The remedy is pasted into a shell, so user data in it must not change the
|
||||
// command: an apostrophe in the URL is escaped for the shell, a dollar sign
|
||||
// and a backtick sit inside single quotes, and a double quote or backslash in
|
||||
// a value is escaped for the SQL literal.
|
||||
func TestColdStartReaddRemedyQuotes(t *testing.T) {
|
||||
got := coldStartReaddRemedy("origin", "git+https://example.com/o'hare/$HOME/`id`/repo.git", "refs/heads/issue-data")
|
||||
want := `bd sql 'CALL DOLT_REMOTE("add", "--ref", "refs/heads/issue-data", "origin", "git+https://example.com/o'\''hare/$HOME/` + "`id`" + `/repo.git")'`
|
||||
if got != want {
|
||||
t.Fatalf("coldStartReaddRemedy =\n%s\nwant\n%s", got, want)
|
||||
}
|
||||
if !strings.HasPrefix(got, "bd sql '") || !strings.HasSuffix(got, "'") {
|
||||
t.Fatalf("the statement must be one single-quoted shell word: %s", got)
|
||||
}
|
||||
got = coldStartReaddRemedy("origin", `git+https://example.com/a"b\c.git`, "refs/dolt/units/team-12542")
|
||||
if !strings.Contains(got, `"git+https://example.com/a\"b\\c.git"`) {
|
||||
t.Fatalf("a double quote and a backslash must be escaped inside the SQL literal: %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
var (
|
||||
errRemoteAlreadyExistsForTest = &testSQLError{"error: remote already exists"}
|
||||
errSQLDownForTest = &testSQLError{"driver: bad connection"}
|
||||
)
|
||||
|
||||
type testSQLError struct{ msg string }
|
||||
|
||||
func (e *testSQLError) Error() string { return e.msg }
|
||||
|
|
@ -637,7 +637,7 @@ func (s *DoltStore) prepareCLIRouteForPeerCredentials(ctx context.Context, peer
|
|||
}
|
||||
for _, r := range remotes {
|
||||
if r.Name == peer {
|
||||
if err := s.ensureMatchingCLIRemote(peer, r.URL); err != nil {
|
||||
if err := s.ensureMatchingCLIRemote(peer, r.URL, r.Ref); err != nil {
|
||||
return false, fmt.Errorf("peer remote %q has credentials and requires CLI routing: %w", peer, err)
|
||||
}
|
||||
return true, nil
|
||||
|
|
@ -687,7 +687,7 @@ func (s *DoltStore) prepareCLIRouteForCredentials(ctx context.Context, remote st
|
|||
}
|
||||
for _, r := range remotes {
|
||||
if r.Name == remote {
|
||||
if err := s.ensureMatchingCLIRemote(remote, r.URL); err != nil {
|
||||
if err := s.ensureMatchingCLIRemote(remote, r.URL, r.Ref); err != nil {
|
||||
return false, fmt.Errorf("remote %q has credentials and requires CLI routing: %w", remote, err)
|
||||
}
|
||||
return true, nil
|
||||
|
|
@ -713,7 +713,7 @@ func (s *DoltStore) shouldUseCLIForLocalRemoteWithError(ctx context.Context, rem
|
|||
}
|
||||
for _, r := range sqlRemotes {
|
||||
if r.Name == remote {
|
||||
return s.hasMatchingCLIRemote(remote, r.URL), nil
|
||||
return s.hasMatchingCLIRemote(remote, r.URL, r.Ref), nil
|
||||
}
|
||||
}
|
||||
return false, nil
|
||||
|
|
@ -806,10 +806,11 @@ func (s *DoltStore) prepareCLIRouteForCloudAuth(ctx context.Context, remote stri
|
|||
if err != nil {
|
||||
return false, fmt.Errorf("list Dolt remotes before cloud-auth routing for remote %q: %w", remote, err)
|
||||
}
|
||||
var remoteURL string
|
||||
var remoteURL, remoteRef string
|
||||
for _, r := range remotes {
|
||||
if r.Name == remote {
|
||||
remoteURL = r.URL
|
||||
remoteRef = r.Ref
|
||||
break
|
||||
}
|
||||
}
|
||||
|
|
@ -821,7 +822,7 @@ func (s *DoltStore) prepareCLIRouteForCloudAuth(ctx context.Context, remote stri
|
|||
return false, nil // unknown scheme — not a cloud remote
|
||||
}
|
||||
if execenv.ContainsKeyWithPrefix(os.Environ(), prefixes...) {
|
||||
if err := s.ensureMatchingCLIRemote(remote, remoteURL); err != nil {
|
||||
if err := s.ensureMatchingCLIRemote(remote, remoteURL, remoteRef); err != nil {
|
||||
return false, fmt.Errorf("remote %q has cloud credentials and requires CLI routing: %w", remote, err)
|
||||
}
|
||||
return true, nil
|
||||
|
|
|
|||
|
|
@ -636,7 +636,7 @@ func (s *DoltStore) prepareCLIRouteForPeerGitProtocol(ctx context.Context, peer
|
|||
if !doltutil.IsGitProtocolURL(r.URL) {
|
||||
return false, nil
|
||||
}
|
||||
if err := s.ensureMatchingCLIRemote(peer, r.URL); err != nil {
|
||||
if err := s.ensureMatchingCLIRemote(peer, r.URL, r.Ref); err != nil {
|
||||
return false, fmt.Errorf("peer remote %q uses git protocol and requires CLI routing: %w", peer, err)
|
||||
}
|
||||
return true, nil
|
||||
|
|
|
|||
|
|
@ -1646,7 +1646,7 @@ func TestCredentialCLIRoutingE2E(t *testing.T) {
|
|||
require.False(t, store.isGitProtocolRemote(ctx, store.remote), "file:// is not git-protocol")
|
||||
if !store.shouldUseCLIForCredentials(ctx, store.remote, store.mainRemoteCredentials()) {
|
||||
remotes, listErr := store.ListRemotes(ctx)
|
||||
ensureErr := doltutil.EnsureCLIRemote(clientTestdbDir, store.remote, remoteURL)
|
||||
ensureErr := doltutil.EnsureCLIRemote(clientTestdbDir, store.remote, remoteURL, "")
|
||||
t.Fatalf("should route through CLI for credentials; serverMode=%v remotes=%v listErr=%v cliRemote=%q ensureErr=%v",
|
||||
store.serverMode, remotes, listErr, doltutil.FindCLIRemote(clientTestdbDir, store.remote), ensureErr)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ func TestEnsureMatchingCLIRemoteSurfacesValidationErrors(t *testing.T) {
|
|||
database: "beads",
|
||||
}
|
||||
|
||||
err := store.ensureMatchingCLIRemote("origin", "ftp://server/path")
|
||||
err := store.ensureMatchingCLIRemote("origin", "ftp://server/path", "")
|
||||
if err == nil {
|
||||
t.Fatal("expected invalid remote URL to be returned as an error")
|
||||
}
|
||||
|
|
@ -201,8 +201,8 @@ func TestPrepareCLIRouteForGitProtocolColdStartWindow(t *testing.T) {
|
|||
// dolt_remotes.
|
||||
mock.MatchExpectationsInOrder(false)
|
||||
for i := 0; i < 4; i++ {
|
||||
mock.ExpectQuery("SELECT name, url FROM dolt_remotes").
|
||||
WillReturnRows(sqlmock.NewRows([]string{"name", "url"}))
|
||||
mock.ExpectQuery("SELECT name, url, params FROM dolt_remotes").
|
||||
WillReturnRows(sqlmock.NewRows([]string{"name", "url", "params"}))
|
||||
}
|
||||
store := &DoltStore{
|
||||
serverMode: true,
|
||||
|
|
|
|||
|
|
@ -3977,9 +3977,11 @@ func (s *DoltStore) buildBatchCommitMessage(ctx context.Context, actor string) s
|
|||
}
|
||||
|
||||
// hasMatchingCLIRemote reports whether the local CLI directory contains the
|
||||
// same remote URL that SQL reports. CLI push/pull/fetch run from CLIDir, so
|
||||
// SQL visibility alone is not enough to route safely.
|
||||
func (s *DoltStore) hasMatchingCLIRemote(remote, expectedURL string) bool {
|
||||
// same remote URL, on the same git data ref, that SQL reports. CLI
|
||||
// push/pull/fetch run from CLIDir, so SQL visibility alone is not enough to
|
||||
// route safely; a mirror on a different ref would move the data to the wrong
|
||||
// ref of the git remote. An empty expectedRef is Dolt's default.
|
||||
func (s *DoltStore) hasMatchingCLIRemote(remote, expectedURL, expectedRef string) bool {
|
||||
if expectedURL == "" {
|
||||
return false
|
||||
}
|
||||
|
|
@ -3990,7 +3992,38 @@ func (s *DoltStore) hasMatchingCLIRemote(remote, expectedURL string) bool {
|
|||
if !s.hasCLIDatabase() {
|
||||
return false
|
||||
}
|
||||
return doltutil.RemoteURLsMatch(doltutil.FindCLIRemote(cliDir, remote), expectedURL)
|
||||
cliURL := doltutil.FindCLIRemote(cliDir, remote)
|
||||
cliRef, err := doltutil.FindCLIRemoteRef(cliDir, remote)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
if cliURL == "" && expectedRef != "" {
|
||||
// A proxied dolt CLI can list nothing at cold start (GH#2118) and
|
||||
// refuses remote parameters, so for a ref the server's own state
|
||||
// file in cliDir is what says whether the mirror is in place.
|
||||
if persisted, ok := persistedCLIRemote(cliDir, remote); ok {
|
||||
cliURL, cliRef = persisted.URL, persisted.Ref
|
||||
}
|
||||
}
|
||||
if !doltutil.RemoteURLsMatch(cliURL, expectedURL) {
|
||||
return false
|
||||
}
|
||||
return storage.RemoteRefsMatch(cliRef, expectedRef)
|
||||
}
|
||||
|
||||
// persistedCLIRemote returns the named remote as recorded in cliDir's
|
||||
// .dolt/repo_state.json.
|
||||
func persistedCLIRemote(cliDir, remote string) (storage.RemoteInfo, bool) {
|
||||
remotes, err := doltutil.PersistedRemotes(cliDir)
|
||||
if err != nil {
|
||||
return storage.RemoteInfo{}, false
|
||||
}
|
||||
for _, r := range remotes {
|
||||
if r.Name == remote {
|
||||
return r, true
|
||||
}
|
||||
}
|
||||
return storage.RemoteInfo{}, false
|
||||
}
|
||||
|
||||
// hasCLIDatabase reports whether CLIDir points at an initialized Dolt database.
|
||||
|
|
@ -4008,8 +4041,8 @@ func (s *DoltStore) hasCLIDatabase() bool {
|
|||
// ensureMatchingCLIRemote materializes the local CLI remote needed before
|
||||
// subprocess push/pull/fetch routing. SQL remains the source of truth; the CLI
|
||||
// remote is only the local transport surface that dolt subprocesses read.
|
||||
func (s *DoltStore) ensureMatchingCLIRemote(remote, expectedURL string) error {
|
||||
if s.hasMatchingCLIRemote(remote, expectedURL) {
|
||||
func (s *DoltStore) ensureMatchingCLIRemote(remote, expectedURL, expectedRef string) error {
|
||||
if s.hasMatchingCLIRemote(remote, expectedURL, expectedRef) {
|
||||
return nil
|
||||
}
|
||||
cliDir := s.CLIDir()
|
||||
|
|
@ -4019,11 +4052,11 @@ func (s *DoltStore) ensureMatchingCLIRemote(remote, expectedURL string) error {
|
|||
if cliDir == "" {
|
||||
return fmt.Errorf("remote %q (%s) requires CLI routing but no CLI directory is configured", remote, expectedURL)
|
||||
}
|
||||
if err := doltutil.EnsureCLIRemote(cliDir, remote, expectedURL); err != nil {
|
||||
if err := doltutil.EnsureCLIRemote(cliDir, remote, expectedURL, expectedRef); err != nil {
|
||||
return fmt.Errorf("materialize CLI remote %q (%s) in %s: %w", remote, expectedURL, cliDir, err)
|
||||
}
|
||||
if !s.hasMatchingCLIRemote(remote, expectedURL) {
|
||||
return fmt.Errorf("materialized CLI remote %q in %s, but its URL does not match SQL URL %q", remote, cliDir, expectedURL)
|
||||
if !s.hasMatchingCLIRemote(remote, expectedURL, expectedRef) {
|
||||
return fmt.Errorf("materialized CLI remote %q in %s, but its URL or git data ref does not match SQL (%s on %s)", remote, cliDir, expectedURL, storage.EffectiveGitDataRef(expectedRef))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
|
@ -4075,7 +4108,7 @@ func (s *DoltStore) prepareCLIRouteForGitProtocol(ctx context.Context, remote st
|
|||
if !doltutil.IsGitProtocolURL(r.URL) {
|
||||
return false, nil
|
||||
}
|
||||
if err := s.ensureMatchingCLIRemote(remote, r.URL); err != nil {
|
||||
if err := s.ensureMatchingCLIRemote(remote, r.URL, r.Ref); err != nil {
|
||||
return false, fmt.Errorf("remote %q uses git protocol and requires CLI routing: %w", remote, err)
|
||||
}
|
||||
return true, nil
|
||||
|
|
@ -4095,7 +4128,22 @@ func (s *DoltStore) prepareCLIRouteForGitProtocol(ctx context.Context, remote st
|
|||
if !doltutil.IsGitProtocolURL(r.URL) {
|
||||
return false, fmt.Errorf("remote %q (%s) is persisted on disk but not yet visible to this sql-server (GH#2118 cold start); retry shortly, or restart the dolt sql-server if it persists", remote, r.URL)
|
||||
}
|
||||
if err := s.ensureMatchingCLIRemote(remote, r.URL); err != nil {
|
||||
if r.Ref != "" {
|
||||
// The proxied CLI refuses remote parameters, so a ref remote is
|
||||
// given to the server over SQL, and only when this process owns
|
||||
// the served directory: a client-local state file proves nothing
|
||||
// about an external server. The remedy names the procedure the
|
||||
// server accepts (the proxied CLI in its directory would refuse
|
||||
// the same --ref, and bd's own remote add finds the persisted
|
||||
// remote and returns without a SQL add).
|
||||
if s.localActiveDatabaseDir == "" || s.localActiveDatabaseDir != s.CLIDir() {
|
||||
return false, fmt.Errorf("remote %q (%s on %s) is persisted on disk but not visible to this sql-server (GH#2118 cold start), and this bd does not own the server's data directory; re-add it on that server with: %s", remote, r.URL, r.Ref, coldStartReaddRemedy(remote, r.URL, r.Ref))
|
||||
}
|
||||
if err := versioncontrolops.AddRemote(ctx, s.db, remote, r.URL, r.Ref); err != nil && !strings.Contains(strings.ToLower(err.Error()), "already exists") {
|
||||
return false, fmt.Errorf("remote %q (%s on %s) is persisted on disk but not visible to this sql-server (GH#2118 cold start), and re-adding it over SQL failed: %w", remote, r.URL, r.Ref, err)
|
||||
}
|
||||
}
|
||||
if err := s.ensureMatchingCLIRemote(remote, r.URL, r.Ref); err != nil {
|
||||
return false, fmt.Errorf("remote %q uses git protocol and requires CLI routing: %w", remote, err)
|
||||
}
|
||||
return true, nil
|
||||
|
|
@ -4103,6 +4151,19 @@ func (s *DoltStore) prepareCLIRouteForGitProtocol(ctx context.Context, remote st
|
|||
return false, nil
|
||||
}
|
||||
|
||||
// coldStartReaddRemedy is the command an operator runs to register a ref
|
||||
// remote on a sql-server this bd does not own: bd sql with a DOLT_REMOTE call.
|
||||
// The values are user data that lands in a message meant to be pasted into a
|
||||
// shell, so the SQL literals are double-quoted with their escapes and the
|
||||
// whole statement is single-quoted for the shell, which keeps an apostrophe,
|
||||
// a dollar sign, or a backtick in a URL from changing the command.
|
||||
func coldStartReaddRemedy(remote, url, ref string) string {
|
||||
stmt := fmt.Sprintf("CALL DOLT_REMOTE(%s, %s, %s, %s, %s)",
|
||||
doltutil.SQLDoubleQuoted("add"), doltutil.SQLDoubleQuoted("--ref"),
|
||||
doltutil.SQLDoubleQuoted(ref), doltutil.SQLDoubleQuoted(remote), doltutil.SQLDoubleQuoted(url))
|
||||
return "bd sql " + doltutil.ShellQuote(stmt)
|
||||
}
|
||||
|
||||
// shouldUseCLIForGitProtocol is a compatibility wrapper for tests and older
|
||||
// call sites. Prefer prepareCLIRouteForGitProtocol so mutation is explicit.
|
||||
func (s *DoltStore) shouldUseCLIForGitProtocol(ctx context.Context, remote string) (bool, error) {
|
||||
|
|
@ -5500,8 +5561,13 @@ func (s *DoltStore) HasRemote(ctx context.Context, name string) (bool, error) {
|
|||
|
||||
// AddRemote adds a Dolt remote
|
||||
func (s *DoltStore) AddRemote(ctx context.Context, name, url string) error {
|
||||
_, err := s.db.ExecContext(ctx, "CALL DOLT_REMOTE('add', ?, ?)", name, url)
|
||||
if err != nil {
|
||||
return s.AddRemoteWithRef(ctx, name, url, "")
|
||||
}
|
||||
|
||||
// AddRemoteWithRef adds a remote whose Dolt data lives on the git ref ref;
|
||||
// see storage.RemoteStore.
|
||||
func (s *DoltStore) AddRemoteWithRef(ctx context.Context, name, url, ref string) error {
|
||||
if err := versioncontrolops.AddRemote(ctx, s.db, name, url, ref); err != nil {
|
||||
return fmt.Errorf("failed to add remote %s: %w", name, err)
|
||||
}
|
||||
return nil
|
||||
|
|
|
|||
393
internal/storage/doltutil/cli_mirror_ref_test.go
Normal file
393
internal/storage/doltutil/cli_mirror_ref_test.go
Normal file
|
|
@ -0,0 +1,393 @@
|
|||
package doltutil
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
)
|
||||
|
||||
// initDoltRepo creates an empty Dolt repository in a temp directory with the
|
||||
// dolt CLI. Skips when the CLI is absent.
|
||||
func initDoltRepo(t *testing.T) string {
|
||||
t.Helper()
|
||||
if _, err := exec.LookPath("dolt"); err != nil {
|
||||
t.Skip("dolt CLI not available")
|
||||
}
|
||||
dir := t.TempDir()
|
||||
cmd := exec.Command("dolt", "init", "--name", "bd-test", "--email", "bd-test@example.com")
|
||||
cmd.Dir = dir
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("dolt init: %v\n%s", err, out)
|
||||
}
|
||||
return dir
|
||||
}
|
||||
|
||||
func readRepoState(t *testing.T, dir string) string {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(filepath.Join(dir, ".dolt", "repo_state.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return string(data)
|
||||
}
|
||||
|
||||
// The CLI mirror that server-mode push, pull, and fetch shell out through
|
||||
// must sit on the same git data ref as the SQL-visible remote. EnsureCLIRemote
|
||||
// re-materializes the mirror when the ref differs, including a change to or
|
||||
// from the default, and leaves the state file untouched when URL and ref
|
||||
// already match.
|
||||
func TestCLIMirrorFollowsGitDataRef(t *testing.T) {
|
||||
for _, ref := range []string{"refs/heads/issue-data", "refs/dolt/units/team-12542"} {
|
||||
t.Run(ref, func(t *testing.T) {
|
||||
dir := initDoltRepo(t)
|
||||
url := "git+https://example.com/repo.git"
|
||||
|
||||
if err := EnsureCLIRemote(dir, "origin", url, ""); err != nil {
|
||||
t.Fatalf("EnsureCLIRemote(default ref): %v", err)
|
||||
}
|
||||
if got, _ := FindCLIRemoteRef(dir, "origin"); got != "" {
|
||||
t.Fatalf("default-ref mirror records Ref %q, want empty", got)
|
||||
}
|
||||
|
||||
if err := EnsureCLIRemote(dir, "origin", url, ref); err != nil {
|
||||
t.Fatalf("EnsureCLIRemote(%s): %v", ref, err)
|
||||
}
|
||||
if got, _ := FindCLIRemoteRef(dir, "origin"); got != ref {
|
||||
t.Fatalf("after ref change, mirror Ref = %q, want %q", got, ref)
|
||||
}
|
||||
if got := FindCLIRemote(dir, "origin"); !RemoteURLsMatch(got, url) {
|
||||
t.Fatalf("after ref change, mirror URL = %q, want %q", got, url)
|
||||
}
|
||||
|
||||
before := readRepoState(t, dir)
|
||||
if err := EnsureCLIRemote(dir, "origin", url, ref); err != nil {
|
||||
t.Fatalf("EnsureCLIRemote(same URL and ref): %v", err)
|
||||
}
|
||||
if after := readRepoState(t, dir); after != before {
|
||||
t.Errorf("matching URL and ref must leave repo_state.json untouched:\nbefore: %s\nafter: %s", before, after)
|
||||
}
|
||||
|
||||
if err := EnsureCLIRemote(dir, "origin", url, ""); err != nil {
|
||||
t.Fatalf("EnsureCLIRemote(back to the default): %v", err)
|
||||
}
|
||||
if got, _ := FindCLIRemoteRef(dir, "origin"); got != "" {
|
||||
t.Errorf("back on the default, mirror should record no git_ref, got %q", got)
|
||||
}
|
||||
if probes := probeRemotesLeft(t, dir); probes != 0 {
|
||||
t.Errorf("%d probe remote(s) left behind", probes)
|
||||
}
|
||||
|
||||
if err := EnsureCLIRemote(dir, "origin", url, "refs/dolt/data"); err != nil {
|
||||
t.Fatalf("EnsureCLIRemote(explicit default): %v", err)
|
||||
}
|
||||
// Dolt records the explicit value; it is the comparison that treats it
|
||||
// as the default.
|
||||
if got, _ := FindCLIRemoteRef(dir, "origin"); !storage.RemoteRefsMatch(got, "") {
|
||||
t.Errorf("explicit default ref should compare equal to the default, got %q", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Dolt refuses --ref for a remote that is not git-backed; the CLI mirror
|
||||
// surfaces that refusal instead of adding the remote on the default ref.
|
||||
func TestCLIMirrorRefRefusedForNonGitURL(t *testing.T) {
|
||||
dir := initDoltRepo(t)
|
||||
err := AddCLIRemoteWithRef(dir, "backup", "file://"+t.TempDir(), "refs/heads/issue-data")
|
||||
if err == nil {
|
||||
t.Fatal("AddCLIRemoteWithRef on file:// with a ref = nil, want error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "git remotes") {
|
||||
t.Errorf("error should carry Dolt's --ref refusal, got: %v", err)
|
||||
}
|
||||
if got := FindCLIRemote(dir, "backup"); got != "" {
|
||||
t.Errorf("refused remote must not exist, found %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// probeRemotesLeft counts remotes named like EnsureCLIRemote's ref probe
|
||||
// still recorded in dir; the probe must always be removed again.
|
||||
func probeRemotesLeft(t *testing.T, dir string) int {
|
||||
t.Helper()
|
||||
remotes, err := PersistedRemotes(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
n := 0
|
||||
for _, r := range remotes {
|
||||
if strings.HasPrefix(r.Name, cliRefProbePrefix) {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// The probe runs before the real remote is removed, so a refused --ref (here
|
||||
// dolt's own refusal on a file:// URL stands in for the proxied server's)
|
||||
// leaves the remote exactly as it was and no probe behind. Reordering the
|
||||
// probe after the removal, or dropping it, fails this test.
|
||||
func TestCLIMirrorRefusedProbeLeavesRemoteInPlace(t *testing.T) {
|
||||
dir := initDoltRepo(t)
|
||||
url := "file://" + t.TempDir()
|
||||
if err := AddCLIRemote(dir, "origin", url); err != nil {
|
||||
t.Fatalf("AddCLIRemote: %v", err)
|
||||
}
|
||||
err := EnsureCLIRemote(dir, "origin", url, "refs/heads/issue-data")
|
||||
if err == nil || !strings.Contains(err.Error(), "cannot record git data ref") {
|
||||
t.Fatalf("EnsureCLIRemote with a ref on file:// = %v, want the probe's refusal", err)
|
||||
}
|
||||
if got := FindCLIRemote(dir, "origin"); !RemoteURLsMatch(got, url) {
|
||||
t.Fatalf("origin must survive the refused probe, found %q", got)
|
||||
}
|
||||
if probes := probeRemotesLeft(t, dir); probes != 0 {
|
||||
t.Errorf("%d probe remote(s) left behind", probes)
|
||||
}
|
||||
}
|
||||
|
||||
// deadPID returns the pid of a process that has exited: the test binary run
|
||||
// with a pattern no test matches. The kernel does not reuse a pid within the
|
||||
// time this test takes.
|
||||
func deadPID(t *testing.T) int {
|
||||
t.Helper()
|
||||
cmd := exec.Command(os.Args[0], "-test.run=^$")
|
||||
if err := cmd.Run(); err != nil {
|
||||
t.Fatalf("run a short-lived process: %v", err)
|
||||
}
|
||||
return cmd.Process.Pid
|
||||
}
|
||||
|
||||
// A probe left behind by a run killed between its add and its remove is
|
||||
// swept before the next change to the mirror, so it does not stay in the
|
||||
// state file (and in every remote listing) until removed by hand: this
|
||||
// process's own leftover always, a dead run's where the platform can tell it
|
||||
// from a live one (sweepsOtherRunsProbes). A remote that only shares the
|
||||
// prefix, and the probe of a run still alive, are not touched.
|
||||
func TestCLIMirrorSweepsLeftoverProbes(t *testing.T) {
|
||||
dir := initDoltRepo(t)
|
||||
url := "git+https://example.com/repo.git"
|
||||
if err := AddCLIRemote(dir, "origin", url); err != nil {
|
||||
t.Fatalf("AddCLIRemote: %v", err)
|
||||
}
|
||||
ownLeftover := fmt.Sprintf("%s%d", cliRefProbePrefix, os.Getpid())
|
||||
deadLeftover := fmt.Sprintf("%s%d", cliRefProbePrefix, deadPID(t))
|
||||
liveProbe := fmt.Sprintf("%s%d", cliRefProbePrefix, os.Getppid())
|
||||
userRemote := cliRefProbePrefix + "backup"
|
||||
for _, name := range []string{ownLeftover, deadLeftover, liveProbe, userRemote} {
|
||||
if err := AddCLIRemoteWithRef(dir, name, url, "refs/heads/issue-data"); err != nil {
|
||||
t.Fatalf("plant %s: %v", name, err)
|
||||
}
|
||||
}
|
||||
if err := EnsureCLIRemote(dir, "origin", url, "refs/dolt/units/team-12542"); err != nil {
|
||||
t.Fatalf("EnsureCLIRemote: %v", err)
|
||||
}
|
||||
remotes, err := PersistedRemotes(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
left := map[string]bool{}
|
||||
for _, r := range remotes {
|
||||
left[r.Name] = true
|
||||
}
|
||||
if left[ownLeftover] {
|
||||
t.Errorf("%s should have been swept", ownLeftover)
|
||||
}
|
||||
if left[deadLeftover] == sweepsOtherRunsProbes {
|
||||
t.Errorf("%s left=%v, want the opposite: a dead run's probe is swept only where the platform can tell it is dead", deadLeftover, left[deadLeftover])
|
||||
}
|
||||
for _, kept := range []string{liveProbe, userRemote, "origin"} {
|
||||
if !left[kept] {
|
||||
t.Errorf("%s must not be swept", kept)
|
||||
}
|
||||
}
|
||||
if got, _ := FindCLIRemoteRef(dir, "origin"); got != "refs/dolt/units/team-12542" {
|
||||
t.Errorf("origin Ref = %q after re-materialization", got)
|
||||
}
|
||||
|
||||
// The sweep runs before any change to the mirror, a move back to the
|
||||
// default ref included, and not only ahead of a probe. The own name is
|
||||
// used so the case holds on every platform.
|
||||
if err := AddCLIRemoteWithRef(dir, ownLeftover, url, "refs/heads/issue-data"); err != nil {
|
||||
t.Fatalf("plant %s again: %v", ownLeftover, err)
|
||||
}
|
||||
if err := EnsureCLIRemote(dir, "origin", url, ""); err != nil {
|
||||
t.Fatalf("EnsureCLIRemote(back to the default): %v", err)
|
||||
}
|
||||
remotes, err = PersistedRemotes(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, r := range remotes {
|
||||
if r.Name == ownLeftover {
|
||||
t.Errorf("%s should have been swept by the move to the default ref", ownLeftover)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// An invalid ref is refused by EnsureCLIRemote before the mirror is touched
|
||||
// even when a remote exists, the case where the check used to run inside the
|
||||
// probe and come back worded as the server refusing parameters; the existing
|
||||
// remote is left as it was.
|
||||
func TestCLIMirrorInvalidRefIsRefusedBeforeProbe(t *testing.T) {
|
||||
dir := initDoltRepo(t)
|
||||
url := "git+https://example.com/repo.git"
|
||||
if err := AddCLIRemote(dir, "origin", url); err != nil {
|
||||
t.Fatalf("AddCLIRemote: %v", err)
|
||||
}
|
||||
err := EnsureCLIRemote(dir, "origin", url, "refs/heads/issue data")
|
||||
if err == nil || !strings.Contains(err.Error(), "invalid git data ref") || strings.Contains(err.Error(), "refuses remote parameters") {
|
||||
t.Errorf("EnsureCLIRemote with an invalid ref over an existing remote = %v, want the argument check's own error", err)
|
||||
}
|
||||
if got := FindCLIRemote(dir, "origin"); got != url {
|
||||
t.Errorf("origin URL after the refusal = %q, want %q untouched", got, url)
|
||||
}
|
||||
if got, err := FindCLIRemoteRef(dir, "origin"); err != nil || got != "" {
|
||||
t.Errorf("origin Ref after the refusal = %q, %v, want the default ref untouched", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
// leftoverProbe admits this process's own name and, where the platform can
|
||||
// tell (sweepsOtherRunsProbes), the exact probe shape for a pid that is not
|
||||
// running; a prefix alone, a padded or signed number, and a live pid are not
|
||||
// leftovers. Needs no dolt binary.
|
||||
func TestLeftoverProbe(t *testing.T) {
|
||||
own := fmt.Sprintf("%s%d", cliRefProbePrefix, os.Getpid())
|
||||
dead := deadPID(t)
|
||||
if !leftoverProbe(own, own) {
|
||||
t.Errorf("leftoverProbe(%q) = false, want true", own)
|
||||
}
|
||||
if deadName := fmt.Sprintf("%s%d", cliRefProbePrefix, dead); leftoverProbe(deadName, own) != sweepsOtherRunsProbes {
|
||||
t.Errorf("leftoverProbe(%q) = %v, want %v on this platform", deadName, !sweepsOtherRunsProbes, sweepsOtherRunsProbes)
|
||||
}
|
||||
for _, name := range []string{
|
||||
cliRefProbePrefix + "backup", cliRefProbePrefix, cliRefProbePrefix + "0", cliRefProbePrefix + "-5",
|
||||
fmt.Sprintf("%s0%d", cliRefProbePrefix, dead), fmt.Sprintf("%s%d ", cliRefProbePrefix, dead),
|
||||
fmt.Sprintf("%s%d", cliRefProbePrefix, os.Getppid()), "origin",
|
||||
} {
|
||||
if leftoverProbe(name, own) {
|
||||
t.Errorf("leftoverProbe(%q) = true, want false", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// When the state file cannot be read at sweep time, the sweep reports that
|
||||
// read, not a refusal of remote parameters by the server. The sweep is called
|
||||
// directly because through EnsureCLIRemote the same file is read a moment
|
||||
// earlier by FindCLIRemoteRef, which refuses first. Needs no dolt binary: the
|
||||
// error is returned before any dolt invocation.
|
||||
func TestCLIMirrorSweepReportsUnreadableState(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(dir, ".dolt"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, ".dolt", "repo_state.json"), []byte(`{"remotes": not-json`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err := sweepLeftoverProbes(dir)
|
||||
if err == nil {
|
||||
t.Fatal("sweepLeftoverProbes on a corrupt state file = nil, want the read error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "recorded remotes") || strings.Contains(err.Error(), "refuses remote parameters") {
|
||||
t.Fatalf("sweepLeftoverProbes error must name the state-file read, not the server's refusal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A leftover under this process's own name that cannot be removed is reported
|
||||
// as that removal, never as the server refusing parameters: here the state
|
||||
// file names it but no dolt repository backs it, so the removal fails.
|
||||
// Reached through EnsureCLIRemote: the failed CLI listing is discarded, the
|
||||
// state file is read and shows no origin, and the sweep runs before the add.
|
||||
// Needs no dolt binary (without one, the removal fails on the missing
|
||||
// binary).
|
||||
func TestCLIMirrorOwnLeftoverRemovalFailureIsReported(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(dir, ".dolt"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
own := ownCLIRefProbe()
|
||||
state := `{"head":"refs/heads/main","remotes":{"` + own + `":{"name":"` + own + `","url":"git+https://example.com/repo.git","fetch_specs":[],"params":{"git_ref":"refs/heads/issue-data"}}},"backups":{},"branches":{}}`
|
||||
if err := os.WriteFile(filepath.Join(dir, ".dolt", "repo_state.json"), []byte(state), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err := EnsureCLIRemote(dir, "origin", "git+https://example.com/repo.git", "refs/heads/issue-data")
|
||||
if err == nil || !strings.Contains(err.Error(), "remove leftover ref probe remote "+own) || strings.Contains(err.Error(), "refuses remote parameters") {
|
||||
t.Fatalf("EnsureCLIRemote = %v, want the own-leftover removal failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
// An invalid ref is refused by EnsureCLIRemote itself, before the state file
|
||||
// is read and before any dolt invocation, and not under the probe's wording
|
||||
// about a server refusing parameters. The state file is made unreadable so
|
||||
// that only the check at the top of EnsureCLIRemote can produce this error:
|
||||
// without it, the read of the recorded ref fails first. Needs no dolt binary.
|
||||
func TestCLIMirrorInvalidRefIsRefusedUnwrapped(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(dir, ".dolt"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, ".dolt", "repo_state.json"), []byte(`{"remotes": not-json`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err := EnsureCLIRemote(dir, "origin", "git+https://example.com/repo.git", "refs/heads/issue data")
|
||||
if err == nil || !strings.Contains(err.Error(), "invalid git data ref") || strings.Contains(err.Error(), "recorded ref") || strings.Contains(err.Error(), "refuses remote parameters") {
|
||||
t.Fatalf("EnsureCLIRemote with an invalid ref = %v, want the argument check's own error before the state file is read", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The argument check refuses the bytes git's ref rules refuse, a space, an
|
||||
// ASCII control character, DEL, plus a leading dash so the value cannot be
|
||||
// mistaken for an option; dolt accepts all of them through exec, so this is
|
||||
// bd policy at the argv boundary. A non-ASCII space (U+00A0, U+2003, U+0085)
|
||||
// is a valid ref byte for git check-ref-format and for dolt remote add
|
||||
// --ref, and passes: this layer adds no ref-name rules of its own. Both ref
|
||||
// shapes and the empty default pass.
|
||||
func TestValidateGitDataRefArg(t *testing.T) {
|
||||
for _, ok := range []string{
|
||||
"", "refs/heads/issue-data", "refs/dolt/units/team-12542", "refs/dolt/data",
|
||||
"refs/heads/issue\u00a0data", "refs/heads/issue\u2003data", "refs/heads/issue\u0085data",
|
||||
} {
|
||||
if err := ValidateGitDataRefArg(ok); err != nil {
|
||||
t.Errorf("ValidateGitDataRefArg(%q) = %v, want nil", ok, err)
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{
|
||||
"-refs/heads/x", "--ref", "refs/heads/issue data", "refs/heads/issue\tdata", "refs/heads/issue\ndata",
|
||||
"refs/heads/issue\x7fdata", "refs/heads/issue\x1bdata", "refs/heads/issue\x00data",
|
||||
} {
|
||||
if err := ValidateGitDataRefArg(bad); err == nil {
|
||||
t.Errorf("ValidateGitDataRefArg(%q) = nil, want an error", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A state file that cannot be parsed is an error out of EnsureCLIRemote,
|
||||
// before any dolt mutation: treating it as the default ref would remove a
|
||||
// ref remote and re-add it on refs/dolt/data. Needs no dolt binary.
|
||||
//
|
||||
// The empty-ref case is asserted too, because that is the one a later
|
||||
// "helpful" relaxation would gate the read error on (only escalate when a ref
|
||||
// was asked for). It must stay an error: an unreadable file cannot rule out a
|
||||
// ref on the mirror, so the default-ref route has no more proof that the
|
||||
// mirror is already correct than the ref route does.
|
||||
func TestCLIMirrorUnreadableStateIsAnError(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(dir, ".dolt"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, ".dolt", "repo_state.json"), []byte(`{"remotes": not-json`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := FindCLIRemoteRef(dir, "origin"); err == nil {
|
||||
t.Fatal("FindCLIRemoteRef on a corrupt state file = nil error, want error")
|
||||
}
|
||||
for _, ref := range []string{"refs/dolt/units/team-12542", ""} {
|
||||
err := EnsureCLIRemote(dir, "origin", "git+https://example.com/repo.git", ref)
|
||||
if err == nil || !strings.Contains(err.Error(), "recorded ref") {
|
||||
t.Fatalf("EnsureCLIRemote(ref=%q) on a corrupt state file = %v, want the read error", ref, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -3,6 +3,7 @@ package doltutil
|
|||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
|
|
@ -54,10 +55,59 @@ func TestPersistedRemotes(t *testing.T) {
|
|||
}
|
||||
})
|
||||
|
||||
t.Run("git_ref parameter is surfaced as Ref", func(t *testing.T) {
|
||||
dir := writeState(t, `{"remotes":{
|
||||
"branch":{"name":"branch","url":"git+https://example.com/repo.git","fetch_specs":["refs/heads/*:refs/remotes/branch/*"],"params":{"git_ref":"refs/heads/issue-data"}},
|
||||
"plain":{"name":"plain","url":"file:///tmp/p","fetch_specs":["refs/heads/*:refs/remotes/plain/*"],"params":{}},
|
||||
"unit":{"name":"unit","url":"git+file:///srv/ledgers","fetch_specs":["refs/heads/*:refs/remotes/unit/*"],"params":{"git_ref":"refs/dolt/units/team-12542"}}
|
||||
}}`)
|
||||
remotes, err := PersistedRemotes(dir)
|
||||
if err != nil {
|
||||
t.Fatalf("PersistedRemotes: %v", err)
|
||||
}
|
||||
want := map[string]string{"branch": "refs/heads/issue-data", "plain": "", "unit": "refs/dolt/units/team-12542"}
|
||||
if len(remotes) != len(want) {
|
||||
t.Fatalf("PersistedRemotes = %+v, want %d remotes", remotes, len(want))
|
||||
}
|
||||
for _, r := range remotes {
|
||||
if r.Ref != want[r.Name] {
|
||||
t.Errorf("remote %s: Ref = %q, want %q", r.Name, r.Ref, want[r.Name])
|
||||
}
|
||||
}
|
||||
if got, _ := FindCLIRemoteRef(dir, "unit"); got != "refs/dolt/units/team-12542" {
|
||||
t.Errorf("FindCLIRemoteRef(unit) = %q", got)
|
||||
}
|
||||
if got, _ := FindCLIRemoteRef(dir, "plain"); got != "" {
|
||||
t.Errorf("FindCLIRemoteRef(plain) = %q, want empty", got)
|
||||
}
|
||||
if got, _ := FindCLIRemoteRef(dir, "absent"); got != "" {
|
||||
t.Errorf("FindCLIRemoteRef(absent) = %q, want empty", got)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("corrupt state file is an error, not silently none", func(t *testing.T) {
|
||||
dir := writeState(t, `{"remotes": not-json`)
|
||||
if _, err := PersistedRemotes(dir); err == nil {
|
||||
t.Fatal("PersistedRemotes on corrupt repo_state.json = nil error, want error")
|
||||
}
|
||||
})
|
||||
|
||||
// A git_ref of the wrong JSON type is "could not tell" too, not "no ref":
|
||||
// silently reading it as the default is what would re-add a remote that is
|
||||
// pinned to a custom ref onto refs/dolt/data.
|
||||
t.Run("a git_ref that is not a string is an error", func(t *testing.T) {
|
||||
dir := writeState(t, `{"remotes":{
|
||||
"unit":{"name":"unit","url":"git+file:///srv/ledgers","fetch_specs":[],"params":{"git_ref":["refs/dolt/units/team-12542"]}}
|
||||
}}`)
|
||||
remotes, err := PersistedRemotes(dir)
|
||||
if err == nil {
|
||||
t.Fatalf("PersistedRemotes with a non-string git_ref = %+v, nil; want an error", remotes)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "unit") {
|
||||
t.Errorf("error %q does not name the remote", err)
|
||||
}
|
||||
if _, err := FindCLIRemoteRef(dir, "unit"); err == nil {
|
||||
t.Error("FindCLIRemoteRef with a non-string git_ref = nil error, want the read error")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
|
|
|||
13
internal/storage/doltutil/probe_pid_other.go
Normal file
13
internal/storage/doltutil/probe_pid_other.go
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
//go:build !unix && !windows
|
||||
|
||||
package doltutil
|
||||
|
||||
// sweepsOtherRunsProbes is false here: with no liveness check implemented,
|
||||
// the sweep removes only this process's own leftover.
|
||||
const sweepsOtherRunsProbes = false
|
||||
|
||||
// probeProcessAlive has no liveness check implemented for this platform, so
|
||||
// every other pid is treated as alive: the sweep then removes only this
|
||||
// process's own leftover, never another run's, which errs on the side of
|
||||
// leaving a remote in place.
|
||||
func probeProcessAlive(_ int) bool { return true }
|
||||
28
internal/storage/doltutil/probe_pid_unix.go
Normal file
28
internal/storage/doltutil/probe_pid_unix.go
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
//go:build unix
|
||||
|
||||
package doltutil
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// sweepsOtherRunsProbes says the sweep can tell a dead run's probe from a
|
||||
// live one on this platform, so it removes another run's leftover.
|
||||
const sweepsOtherRunsProbes = true
|
||||
|
||||
// probeProcessAlive reports whether a process with pid is running, with the
|
||||
// kill(pid, 0) that internal/doltserver uses, read more strictly: there any
|
||||
// error means dead, here only ESRCH ("no such process") does. EPERM means the
|
||||
// process exists under another user, and any other errno means the check
|
||||
// itself failed, and both report alive so the caller leaves the remote in
|
||||
// place on an inconclusive read. kill(0, sig) would signal the caller's own
|
||||
// process group and kill(-1, sig) everything it may signal, so a pid that is
|
||||
// not positive never reaches the syscall and reports alive.
|
||||
func probeProcessAlive(pid int) bool {
|
||||
if pid <= 0 {
|
||||
return true
|
||||
}
|
||||
err := syscall.Kill(pid, 0)
|
||||
return !errors.Is(err, syscall.ESRCH)
|
||||
}
|
||||
37
internal/storage/doltutil/probe_pid_windows.go
Normal file
37
internal/storage/doltutil/probe_pid_windows.go
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
//go:build windows
|
||||
|
||||
package doltutil
|
||||
|
||||
import (
|
||||
"errors"
|
||||
|
||||
"golang.org/x/sys/windows"
|
||||
)
|
||||
|
||||
// sweepsOtherRunsProbes says the sweep can tell a dead run's probe from a
|
||||
// live one on this platform, so it removes another run's leftover.
|
||||
const sweepsOtherRunsProbes = true
|
||||
|
||||
// probeProcessAlive reports whether a process with pid is running, the way
|
||||
// internal/doltserver does on Windows: opening the process is not enough,
|
||||
// since an exited process stays openable while another handle refers to it,
|
||||
// so a zero-timeout wait tells a signaled (exited) process from a running
|
||||
// one. A pid that cannot be opened reads as dead, except for access denied,
|
||||
// which means the process exists under another user and reads as alive; a
|
||||
// wait that fails reads as alive too, so the caller leaves the remote in
|
||||
// place on an inconclusive read.
|
||||
func probeProcessAlive(pid int) bool {
|
||||
if pid <= 0 {
|
||||
return true
|
||||
}
|
||||
h, err := windows.OpenProcess(windows.SYNCHRONIZE, false, uint32(pid))
|
||||
if err != nil {
|
||||
return errors.Is(err, windows.ERROR_ACCESS_DENIED)
|
||||
}
|
||||
defer func() { _ = windows.CloseHandle(h) }()
|
||||
status, err := windows.WaitForSingleObject(h, 0)
|
||||
if err != nil {
|
||||
return true
|
||||
}
|
||||
return status != uint32(windows.WAIT_OBJECT_0)
|
||||
}
|
||||
|
|
@ -8,6 +8,7 @@ import (
|
|||
"os/exec"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
|
@ -85,7 +86,10 @@ func IsGitProtocolURL(url string) bool {
|
|||
// distinguishable (bd-6dnrw.33). A missing .dolt directory or repo_state.json
|
||||
// means "not a dolt repository here" and returns (nil, nil); an unreadable or
|
||||
// unparseable file returns an error so callers can tell "definitely none"
|
||||
// from "could not tell". Results are sorted by name.
|
||||
// from "could not tell". Results are sorted by name. Ref is the remote's
|
||||
// git_ref parameter when recorded, else empty; a git_ref that is present but
|
||||
// is not a string is unparseable in the same sense (see
|
||||
// storage.GitRefFromParams) and is reported as an error naming the remote.
|
||||
func PersistedRemotes(dbPath string) ([]storage.RemoteInfo, error) {
|
||||
path := filepath.Join(dbPath, ".dolt", "repo_state.json")
|
||||
data, err := os.ReadFile(path) // #nosec G304 -- repo-local dolt state file
|
||||
|
|
@ -97,7 +101,8 @@ func PersistedRemotes(dbPath string) ([]storage.RemoteInfo, error) {
|
|||
}
|
||||
var state struct {
|
||||
Remotes map[string]struct {
|
||||
URL string `json:"url"`
|
||||
URL string `json:"url"`
|
||||
Params map[string]any `json:"params"`
|
||||
} `json:"remotes"`
|
||||
}
|
||||
if err := json.Unmarshal(data, &state); err != nil {
|
||||
|
|
@ -105,7 +110,15 @@ func PersistedRemotes(dbPath string) ([]storage.RemoteInfo, error) {
|
|||
}
|
||||
remotes := make([]storage.RemoteInfo, 0, len(state.Remotes))
|
||||
for name, r := range state.Remotes {
|
||||
remotes = append(remotes, storage.RemoteInfo{Name: name, URL: r.URL})
|
||||
ref, err := storage.GitRefFromParams(r.Params)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("remote %s in %s: %w", name, path, err)
|
||||
}
|
||||
remotes = append(remotes, storage.RemoteInfo{
|
||||
Name: name,
|
||||
URL: r.URL,
|
||||
Ref: ref,
|
||||
})
|
||||
}
|
||||
sort.Slice(remotes, func(i, j int) bool { return remotes[i].Name < remotes[j].Name })
|
||||
return remotes, nil
|
||||
|
|
@ -114,6 +127,8 @@ func PersistedRemotes(dbPath string) ([]storage.RemoteInfo, error) {
|
|||
// ListCLIRemotes parses `dolt remote -v` output from the given database
|
||||
// directory. This is a read-only guard for deciding whether CLI push/pull/fetch
|
||||
// can safely run from that directory; remote mutation still goes through SQL.
|
||||
// Ref is not populated here (the params column is not parsed from the
|
||||
// listing); callers that need it use FindCLIRemoteRef.
|
||||
func ListCLIRemotes(dbPath string) ([]storage.RemoteInfo, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), listCLIRemotesTimeout(dbPath))
|
||||
defer cancel()
|
||||
|
|
@ -151,17 +166,71 @@ func RemoteURLsMatch(got, want string) bool {
|
|||
return false
|
||||
}
|
||||
|
||||
// ValidateGitDataRefArg checks a git data ref before it becomes a dolt
|
||||
// command-line argument. It refuses the bytes git's ref rules refuse, a
|
||||
// space, an ASCII control character, and DEL, plus a leading dash, so the
|
||||
// value can never be mistaken for an option. Dolt itself accepts all of them
|
||||
// through exec, so this is bd policy at the argv boundary, not a ref-name
|
||||
// check: a non-ASCII space such as U+00A0 is a valid ref byte for git and
|
||||
// for dolt and passes here like any other byte, and the full ref-name rules
|
||||
// belong to the command layer. An empty ref is valid and means Dolt's
|
||||
// default.
|
||||
func ValidateGitDataRefArg(ref string) error {
|
||||
if ref == "" {
|
||||
return nil
|
||||
}
|
||||
if strings.HasPrefix(ref, "-") {
|
||||
return fmt.Errorf("git data ref %q must not start with a dash", ref)
|
||||
}
|
||||
for _, r := range ref {
|
||||
if r < 0x20 || r == 0x7f || r == ' ' {
|
||||
return fmt.Errorf("git data ref %q contains a space or an ASCII control character", ref)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// SQLDoubleQuoted returns s as a MySQL string literal in double quotes, with
|
||||
// the backslash and the double quote escaped, for a statement that is shown
|
||||
// to a person to run rather than bound as a parameter.
|
||||
func SQLDoubleQuoted(s string) string {
|
||||
s = strings.ReplaceAll(s, `\`, `\\`)
|
||||
s = strings.ReplaceAll(s, `"`, `\"`)
|
||||
return `"` + s + `"`
|
||||
}
|
||||
|
||||
// AddCLIRemote adds a remote at the filesystem level via dolt CLI.
|
||||
// Remote mutation should normally go through SQL; this is reserved for the
|
||||
// local CLI mirror required by subprocess push/pull/fetch routing.
|
||||
//
|
||||
// It has no production caller: EnsureCLIRemote, its only one, now goes through
|
||||
// AddCLIRemoteWithRef. It is kept deliberately as the default-ref spelling for
|
||||
// tests and callers that have no ref to carry, so a reader changing
|
||||
// AddCLIRemoteWithRef does not have to re-derive that.
|
||||
func AddCLIRemote(dbPath, name, url string) error {
|
||||
return AddCLIRemoteWithRef(dbPath, name, url, "")
|
||||
}
|
||||
|
||||
// AddCLIRemoteWithRef is AddCLIRemote for a git-backed remote whose Dolt data
|
||||
// lives on the git ref ref (`dolt remote add --ref`). An empty ref is
|
||||
// AddCLIRemote.
|
||||
func AddCLIRemoteWithRef(dbPath, name, url, ref string) error {
|
||||
if err := remotecache.ValidateRemoteName(name); err != nil {
|
||||
return fmt.Errorf("invalid remote name: %w", err)
|
||||
}
|
||||
if err := remotecache.ValidateRemoteURL(url); err != nil {
|
||||
return fmt.Errorf("invalid remote URL: %w", err)
|
||||
}
|
||||
cmd := exec.Command("dolt", "remote", "add", name, url) // #nosec G204 -- validated argv
|
||||
ref = strings.TrimSpace(ref)
|
||||
if err := ValidateGitDataRefArg(ref); err != nil {
|
||||
return fmt.Errorf("invalid git data ref: %w", err)
|
||||
}
|
||||
args := []string{"remote", "add"}
|
||||
if ref != "" {
|
||||
args = append(args, "--ref", ref)
|
||||
}
|
||||
args = append(args, name, url)
|
||||
cmd := exec.Command("dolt", args...) // #nosec G204 -- validated argv
|
||||
cmd.Dir = dbPath
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
|
|
@ -199,35 +268,163 @@ func FindCLIRemote(dbPath, name string) string {
|
|||
return ""
|
||||
}
|
||||
|
||||
// EnsureCLIRemote makes the local CLI remote match the SQL-visible remote URL.
|
||||
// It is intentionally idempotent and only mutates the CLI surface when the
|
||||
// remote is absent or points somewhere else.
|
||||
func EnsureCLIRemote(dbPath, name, url string) error {
|
||||
// cliRefProbePrefix names the throwaway remote probeCLIRemoteRef adds.
|
||||
const cliRefProbePrefix = "bd-ref-probe-"
|
||||
|
||||
// ownCLIRefProbe names the throwaway remote this process's probe uses.
|
||||
func ownCLIRefProbe() string {
|
||||
return fmt.Sprintf("%s%d", cliRefProbePrefix, os.Getpid())
|
||||
}
|
||||
|
||||
// sweepLeftoverProbes removes probe remotes left behind by a run killed
|
||||
// between its add and its remove: a remote named exactly as probeCLIRemoteRef
|
||||
// names its probes whose pid is no longer running, and one under this
|
||||
// process's own name (see leftoverProbe). EnsureCLIRemote runs it before
|
||||
// every change to the mirror, whatever ref the change is onto, so a dead
|
||||
// run's leftover is removed at the next re-materialization of that mirror
|
||||
// where the platform has a liveness check (unix and windows; elsewhere only
|
||||
// the own name is swept). For another run's leftover that removal is
|
||||
// best-effort: a failure is warned about and tried again at the next change
|
||||
// rather than failing the caller's own mirror change, and a pid an unrelated
|
||||
// process has since taken keeps the leftover until that pid exits. A probe of
|
||||
// a live run is left alone, so two bd processes re-materializing the same
|
||||
// mirror at the same moment do not remove each other's probe. A leftover
|
||||
// under this process's own name must go, or the probe's add would fail and be
|
||||
// read as the server's refusal; its removal failing is reported as such.
|
||||
func sweepLeftoverProbes(dbPath string) error {
|
||||
own := ownCLIRefProbe()
|
||||
remotes, err := PersistedRemotes(dbPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("read the recorded remotes in %s before changing the CLI mirror: %w", dbPath, err)
|
||||
}
|
||||
for _, r := range remotes {
|
||||
if !leftoverProbe(r.Name, own) {
|
||||
continue
|
||||
}
|
||||
if err := RemoveCLIRemote(dbPath, r.Name); err != nil {
|
||||
if r.Name == own {
|
||||
return fmt.Errorf("remove leftover ref probe remote %s in %s: %w", own, dbPath, err)
|
||||
}
|
||||
// Best-effort, but never silent: an un-removable leftover stays
|
||||
// visible in `bd remote list` until some run succeeds, and without
|
||||
// this line nothing ever says why.
|
||||
fmt.Fprintf(os.Stderr,
|
||||
"Warning: could not remove the ref probe remote %s left in %s by an exited run (will retry at the next mirror change): %v\n",
|
||||
r.Name, dbPath, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// probeCLIRemoteRef adds and removes a throwaway remote with url and ref
|
||||
// before EnsureCLIRemote removes the real one: a dolt CLI proxied to a
|
||||
// running sql-server refuses remote parameters, and that refusal must come
|
||||
// before anything is deleted. The caller has swept leftovers already.
|
||||
func probeCLIRemoteRef(dbPath, url, ref string) error {
|
||||
if strings.TrimSpace(ref) == "" {
|
||||
return nil
|
||||
}
|
||||
probe := ownCLIRefProbe()
|
||||
if err := AddCLIRemoteWithRef(dbPath, probe, url, ref); err != nil {
|
||||
return fmt.Errorf("cannot record git data ref %s on the CLI mirror in %s (a dolt sql-server serving this directory refuses remote parameters over the CLI): %w", ref, dbPath, err)
|
||||
}
|
||||
if err := RemoveCLIRemote(dbPath, probe); err != nil {
|
||||
return fmt.Errorf("remove ref probe remote %s in %s: %w", probe, dbPath, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// leftoverProbe reports whether the remote called name may be swept as a
|
||||
// probe left by an earlier run: this process's own probe name, or the exact
|
||||
// shape probeCLIRemoteRef generates (the prefix followed by a pid in
|
||||
// canonical decimal, nothing else) for a pid that is not running. A remote
|
||||
// that merely shares the prefix, such as one a user named bd-ref-probe-backup,
|
||||
// and the probe of a run still alive are not leftovers.
|
||||
func leftoverProbe(name, own string) bool {
|
||||
if name == own {
|
||||
return true
|
||||
}
|
||||
digits, ok := strings.CutPrefix(name, cliRefProbePrefix)
|
||||
if !ok || digits == "" {
|
||||
return false
|
||||
}
|
||||
pid, err := strconv.Atoi(digits)
|
||||
if err != nil || pid <= 0 || strconv.Itoa(pid) != digits {
|
||||
return false
|
||||
}
|
||||
return !probeProcessAlive(pid)
|
||||
}
|
||||
|
||||
// FindCLIRemoteRef returns the git data ref recorded for the named remote in
|
||||
// dbPath's .dolt/repo_state.json, verbatim (compare with
|
||||
// storage.RemoteRefsMatch), or "" when the remote is absent or carries no
|
||||
// ref. A state file that cannot be read or parsed is an error, never the
|
||||
// default ref: treating it as the default would feed a remove-and-re-add of
|
||||
// a ref remote onto refs/dolt/data.
|
||||
func FindCLIRemoteRef(dbPath, name string) (string, error) {
|
||||
remotes, err := PersistedRemotes(dbPath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
for _, r := range remotes {
|
||||
if r.Name == name {
|
||||
return r.Ref, nil
|
||||
}
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// EnsureCLIRemote makes the local CLI remote match the SQL-visible remote URL
|
||||
// and git data ref (an empty ref is Dolt's default). It is intentionally
|
||||
// idempotent and only mutates the CLI surface when the remote is absent,
|
||||
// points somewhere else, or sits on a different ref; before it mutates, it
|
||||
// sweeps probe remotes left by interrupted runs.
|
||||
//
|
||||
// Reading the mirror's recorded ref comes before the no-op short-circuit, so a
|
||||
// state file that cannot be read fails the route for a default-ref remote too,
|
||||
// not only for a remote with a ref. That is deliberate: an unreadable file
|
||||
// cannot rule out a ref on the mirror, so a URL match alone does not prove the
|
||||
// mirror is already correct, and continuing would push onto whatever ref the
|
||||
// mirror is secretly pinned to. Failing is loud and mutates nothing.
|
||||
func EnsureCLIRemote(dbPath, name, url, ref string) error {
|
||||
if err := remotecache.ValidateRemoteName(name); err != nil {
|
||||
return fmt.Errorf("invalid remote name: %w", err)
|
||||
}
|
||||
if err := remotecache.ValidateRemoteURL(url); err != nil {
|
||||
return fmt.Errorf("invalid remote URL: %w", err)
|
||||
}
|
||||
if err := ValidateGitDataRefArg(strings.TrimSpace(ref)); err != nil {
|
||||
return fmt.Errorf("invalid git data ref: %w", err)
|
||||
}
|
||||
|
||||
lock := cliRemoteLock(dbPath)
|
||||
lock.Lock()
|
||||
defer lock.Unlock()
|
||||
|
||||
current := FindCLIRemote(dbPath, name)
|
||||
if RemoteURLsMatch(current, url) {
|
||||
currentRef, err := FindCLIRemoteRef(dbPath, name)
|
||||
if err != nil {
|
||||
return fmt.Errorf("read the recorded ref of CLI remote %q in %s: %w", name, dbPath, err)
|
||||
}
|
||||
if RemoteURLsMatch(current, url) && storage.RemoteRefsMatch(currentRef, ref) {
|
||||
return nil
|
||||
}
|
||||
if err := sweepLeftoverProbes(dbPath); err != nil {
|
||||
return err
|
||||
}
|
||||
if current != "" {
|
||||
if err := probeCLIRemoteRef(dbPath, url, ref); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := RemoveCLIRemote(dbPath, name); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if err := AddCLIRemote(dbPath, name, url); err != nil {
|
||||
if err := AddCLIRemoteWithRef(dbPath, name, url, ref); err != nil {
|
||||
if current == "" {
|
||||
return err
|
||||
}
|
||||
if restoreErr := AddCLIRemote(dbPath, name, current); restoreErr != nil {
|
||||
if restoreErr := AddCLIRemoteWithRef(dbPath, name, current, currentRef); restoreErr != nil {
|
||||
return fmt.Errorf("add replacement CLI remote failed: %w; additionally failed to restore previous URL %q: %v", err, current, restoreErr)
|
||||
}
|
||||
return fmt.Errorf("add replacement CLI remote failed; previous URL %q restored: %w", current, err)
|
||||
|
|
|
|||
|
|
@ -2,8 +2,11 @@ package db
|
|||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
"github.com/steveyegge/beads/internal/storage/domain"
|
||||
)
|
||||
|
||||
|
|
@ -28,6 +31,19 @@ func (r *remoteSQLRepositoryImpl) AddRemote(ctx context.Context, name, url strin
|
|||
return nil
|
||||
}
|
||||
|
||||
// AddRemoteWithRef passes a non-empty ref as DOLT_REMOTE's --ref, which Dolt
|
||||
// records as the remote's git_ref parameter and accepts for git-backed
|
||||
// remotes only.
|
||||
func (r *remoteSQLRepositoryImpl) AddRemoteWithRef(ctx context.Context, name, url, ref string) error {
|
||||
if ref = strings.TrimSpace(ref); ref == "" {
|
||||
return r.AddRemote(ctx, name, url)
|
||||
}
|
||||
if err := r.vc.Remote(ctx, "add", "--ref", ref, name, url); err != nil {
|
||||
return fmt.Errorf("db: AddRemoteWithRef %s: %w", name, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *remoteSQLRepositoryImpl) RemoveRemote(ctx context.Context, name string) error {
|
||||
if err := r.vc.Remote(ctx, "remove", name); err != nil {
|
||||
return fmt.Errorf("db: RemoveRemote %s: %w", name, err)
|
||||
|
|
@ -36,7 +52,7 @@ func (r *remoteSQLRepositoryImpl) RemoveRemote(ctx context.Context, name string)
|
|||
}
|
||||
|
||||
func (r *remoteSQLRepositoryImpl) ListRemotes(ctx context.Context) ([]domain.Remote, error) {
|
||||
rows, err := r.runner.QueryContext(ctx, "SELECT name, url FROM dolt_remotes")
|
||||
rows, err := r.runner.QueryContext(ctx, "SELECT name, url, params FROM dolt_remotes")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("db: ListRemotes: query: %w", err)
|
||||
}
|
||||
|
|
@ -45,9 +61,17 @@ func (r *remoteSQLRepositoryImpl) ListRemotes(ctx context.Context) ([]domain.Rem
|
|||
var remotes []domain.Remote
|
||||
for rows.Next() {
|
||||
var rem domain.Remote
|
||||
if err := rows.Scan(&rem.Name, &rem.URL); err != nil {
|
||||
var params sql.NullString
|
||||
if err := rows.Scan(&rem.Name, &rem.URL, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("db: ListRemotes: scan: %w", err)
|
||||
}
|
||||
if params.Valid {
|
||||
ref, err := storage.GitRefFromParamsJSON(params.String)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("db: ListRemotes: remote %s: %w", rem.Name, err)
|
||||
}
|
||||
rem.Ref = ref
|
||||
}
|
||||
remotes = append(remotes, rem)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
|
|
|
|||
75
internal/storage/domain/db/remote_test.go
Normal file
75
internal/storage/domain/db/remote_test.go
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
"regexp"
|
||||
"testing"
|
||||
|
||||
sqlmock "github.com/DATA-DOG/go-sqlmock"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// The proxied-server remote repository passes a ref as DOLT_REMOTE's --ref,
|
||||
// verbatim, for both ref shapes; without a ref the call is the same
|
||||
// three-argument add as before.
|
||||
func TestRemoteSQLRepositoryAddRemoteWithRef(t *testing.T) {
|
||||
const url = "git+https://example.com/repo.git"
|
||||
for _, ref := range []string{"refs/heads/issue-data", "refs/dolt/units/team-12542"} {
|
||||
t.Run(ref, func(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
require.NoError(t, err)
|
||||
defer db.Close()
|
||||
|
||||
mock.ExpectExec(regexp.QuoteMeta("CALL DOLT_REMOTE(?, ?, ?, ?, ?)")).
|
||||
WithArgs("add", "--ref", ref, "origin", url).
|
||||
WillReturnResult(sqlmock.NewResult(0, 1))
|
||||
|
||||
repo := NewRemoteSQLRepository(db)
|
||||
require.NoError(t, repo.AddRemoteWithRef(context.Background(), "origin", url, ref))
|
||||
require.NoError(t, mock.ExpectationsWereMet())
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("empty ref is a plain add", func(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
require.NoError(t, err)
|
||||
defer db.Close()
|
||||
|
||||
mock.ExpectExec(regexp.QuoteMeta("CALL DOLT_REMOTE(?, ?, ?)")).
|
||||
WithArgs("add", "origin", url).
|
||||
WillReturnResult(sqlmock.NewResult(0, 1))
|
||||
|
||||
repo := NewRemoteSQLRepository(db)
|
||||
require.NoError(t, repo.AddRemoteWithRef(context.Background(), "origin", url, " "))
|
||||
require.NoError(t, mock.ExpectationsWereMet())
|
||||
})
|
||||
}
|
||||
|
||||
// ListRemotes reads the ref back from the params column; remotes without
|
||||
// the parameter list the default.
|
||||
func TestRemoteSQLRepositoryListRemotesReadsRef(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
require.NoError(t, err)
|
||||
defer db.Close()
|
||||
|
||||
rows := sqlmock.NewRows([]string{"name", "url", "params"}).
|
||||
AddRow("plain", "file:///tmp/p", nil).
|
||||
AddRow("empty", "dolthub://org/repo", "{}").
|
||||
AddRow("branch", "git+https://example.com/repo.git", `{"git_ref":"refs/heads/issue-data"}`).
|
||||
AddRow("unit", "git+file:///srv/ledgers", `{"git_ref":"refs/dolt/units/team-12542"}`)
|
||||
mock.ExpectQuery(regexp.QuoteMeta("SELECT name, url, params FROM dolt_remotes")).WillReturnRows(rows)
|
||||
|
||||
remotes, err := NewRemoteSQLRepository(db).ListRemotes(context.Background())
|
||||
require.NoError(t, err)
|
||||
want := map[string]string{
|
||||
"plain": "",
|
||||
"empty": "",
|
||||
"branch": "refs/heads/issue-data",
|
||||
"unit": "refs/dolt/units/team-12542",
|
||||
}
|
||||
require.Len(t, remotes, len(want))
|
||||
for _, r := range remotes {
|
||||
require.Equal(t, want[r.Name], r.Ref, "remote %s", r.Name)
|
||||
}
|
||||
require.NoError(t, mock.ExpectationsWereMet())
|
||||
}
|
||||
|
|
@ -3,10 +3,18 @@ package domain
|
|||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage/doltutil"
|
||||
)
|
||||
|
||||
type DoltRemoteUseCase interface {
|
||||
CreateRemote(ctx context.Context, name, url string) error
|
||||
// CreateRemoteWithRef is CreateRemote for a git-backed remote whose Dolt
|
||||
// data lives on the git ref ref; an empty ref is CreateRemote. The ref is
|
||||
// checked against what the dolt argv boundary will accept, so a remote
|
||||
// cannot be recorded on a ref it could never be routed to.
|
||||
CreateRemoteWithRef(ctx context.Context, name, url, ref string) error
|
||||
UpdateRemote(ctx context.Context, name, url string) error
|
||||
DeleteRemote(ctx context.Context, name string) error
|
||||
ListRemotes(ctx context.Context) ([]Remote, error)
|
||||
|
|
@ -15,10 +23,14 @@ type DoltRemoteUseCase interface {
|
|||
type Remote struct {
|
||||
Name string
|
||||
URL string
|
||||
// Ref is the git ref a git-backed remote keeps its Dolt data on when one
|
||||
// was recorded; empty means Dolt's default, refs/dolt/data.
|
||||
Ref string
|
||||
}
|
||||
|
||||
type RemoteSQLRepository interface {
|
||||
AddRemote(ctx context.Context, name, url string) error
|
||||
AddRemoteWithRef(ctx context.Context, name, url, ref string) error
|
||||
RemoveRemote(ctx context.Context, name string) error
|
||||
ListRemotes(ctx context.Context) ([]Remote, error)
|
||||
}
|
||||
|
|
@ -46,6 +58,29 @@ func (u *doltRemoteUseCaseImpl) CreateRemote(ctx context.Context, name, url stri
|
|||
return nil
|
||||
}
|
||||
|
||||
func (u *doltRemoteUseCaseImpl) CreateRemoteWithRef(ctx context.Context, name, url, ref string) error {
|
||||
if name == "" {
|
||||
return fmt.Errorf("CreateRemoteWithRef: name must not be empty")
|
||||
}
|
||||
if url == "" {
|
||||
return fmt.Errorf("CreateRemoteWithRef: url must not be empty")
|
||||
}
|
||||
// The SQL write paths below bind the ref as a parameter and would happily
|
||||
// record one the dolt argv boundary later refuses, which would leave a
|
||||
// remote that can be created but never pushed, diagnosed only at push time.
|
||||
// This is the write seam for new remotes, so the recordable ref set is
|
||||
// narrowed to the routable one here, beside the other argument checks.
|
||||
// UpdateRemote deliberately does not re-check: it carries and restores refs
|
||||
// that are already recorded, and must not become unable to put one back.
|
||||
if err := doltutil.ValidateGitDataRefArg(strings.TrimSpace(ref)); err != nil {
|
||||
return fmt.Errorf("CreateRemoteWithRef %s: invalid git data ref: %w", name, err)
|
||||
}
|
||||
if err := u.remoteRepo.AddRemoteWithRef(ctx, name, url, ref); err != nil {
|
||||
return fmt.Errorf("CreateRemoteWithRef %s: %w", name, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (u *doltRemoteUseCaseImpl) UpdateRemote(ctx context.Context, name, url string) error {
|
||||
if name == "" {
|
||||
return fmt.Errorf("UpdateRemote: name must not be empty")
|
||||
|
|
@ -54,23 +89,34 @@ func (u *doltRemoteUseCaseImpl) UpdateRemote(ctx context.Context, name, url stri
|
|||
return fmt.Errorf("UpdateRemote: url must not be empty")
|
||||
}
|
||||
// Dolt has no atomic remote update, so this is remove-then-add. Capture
|
||||
// the old URL first so a failed add can restore the remote instead of
|
||||
// leaving it deleted (bd-6dnrw.44 P3).
|
||||
var oldURL string
|
||||
if remotes, err := u.remoteRepo.ListRemotes(ctx); err == nil {
|
||||
for _, rem := range remotes {
|
||||
if rem.Name == name {
|
||||
oldURL = rem.URL
|
||||
break
|
||||
}
|
||||
// the old URL and ref first so a failed add can restore the remote
|
||||
// instead of leaving it deleted (bd-6dnrw.44 P3), and so a URL change
|
||||
// keeps the remote's git data ref. A listing that fails stops the update
|
||||
// before anything is removed: without it the ref could not be kept.
|
||||
remotes, err := u.remoteRepo.ListRemotes(ctx)
|
||||
if err != nil {
|
||||
return fmt.Errorf("UpdateRemote %s: list remotes before replacing: %w", name, err)
|
||||
}
|
||||
var oldURL, oldRef string
|
||||
for _, rem := range remotes {
|
||||
if rem.Name == name {
|
||||
oldURL = rem.URL
|
||||
oldRef = rem.Ref
|
||||
break
|
||||
}
|
||||
}
|
||||
// A git data ref belongs to git-backed remotes only; Dolt refuses it on
|
||||
// any other scheme, so a move to one drops it.
|
||||
newRef := oldRef
|
||||
if !doltutil.IsGitProtocolURL(url) {
|
||||
newRef = ""
|
||||
}
|
||||
if err := u.remoteRepo.RemoveRemote(ctx, name); err != nil {
|
||||
return fmt.Errorf("UpdateRemote %s: remove: %w", name, err)
|
||||
}
|
||||
if err := u.remoteRepo.AddRemote(ctx, name, url); err != nil {
|
||||
if err := u.remoteRepo.AddRemoteWithRef(ctx, name, url, newRef); err != nil {
|
||||
if oldURL != "" {
|
||||
if restoreErr := u.remoteRepo.AddRemote(ctx, name, oldURL); restoreErr != nil {
|
||||
if restoreErr := u.remoteRepo.AddRemoteWithRef(ctx, name, oldURL, oldRef); restoreErr != nil {
|
||||
return fmt.Errorf("UpdateRemote %s: add: %w (restoring previous URL %s also failed: %v)", name, err, oldURL, restoreErr)
|
||||
}
|
||||
return fmt.Errorf("UpdateRemote %s: add: %w (previous URL %s restored)", name, err, oldURL)
|
||||
|
|
|
|||
143
internal/storage/domain/remote_test.go
Normal file
143
internal/storage/domain/remote_test.go
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
package domain
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type fakeRemoteRepo struct {
|
||||
remotes []Remote
|
||||
calls []string
|
||||
failAdd string
|
||||
listErr error
|
||||
}
|
||||
|
||||
func (f *fakeRemoteRepo) AddRemote(ctx context.Context, name, url string) error {
|
||||
return f.AddRemoteWithRef(ctx, name, url, "")
|
||||
}
|
||||
|
||||
func (f *fakeRemoteRepo) AddRemoteWithRef(_ context.Context, name, url, ref string) error {
|
||||
f.calls = append(f.calls, "add "+name+" "+url+" ref="+ref)
|
||||
if url == f.failAdd {
|
||||
return errors.New("refused")
|
||||
}
|
||||
f.remotes = append(f.remotes, Remote{Name: name, URL: url, Ref: ref})
|
||||
return nil
|
||||
}
|
||||
|
||||
func (f *fakeRemoteRepo) RemoveRemote(_ context.Context, name string) error {
|
||||
f.calls = append(f.calls, "remove "+name)
|
||||
kept := f.remotes[:0]
|
||||
for _, r := range f.remotes {
|
||||
if r.Name != name {
|
||||
kept = append(kept, r)
|
||||
}
|
||||
}
|
||||
f.remotes = kept
|
||||
return nil
|
||||
}
|
||||
|
||||
func (f *fakeRemoteRepo) ListRemotes(context.Context) ([]Remote, error) {
|
||||
if f.listErr != nil {
|
||||
return nil, f.listErr
|
||||
}
|
||||
return append([]Remote(nil), f.remotes...), nil
|
||||
}
|
||||
|
||||
// A remote's git data ref survives UpdateRemote: the replacement carries it,
|
||||
// and so does the restore after a failed replacement. Both ref shapes.
|
||||
func TestUpdateRemoteKeepsGitDataRef(t *testing.T) {
|
||||
for _, ref := range []string{"refs/heads/issue-data", "refs/dolt/units/team-12542"} {
|
||||
t.Run(ref, func(t *testing.T) {
|
||||
repo := &fakeRemoteRepo{remotes: []Remote{{Name: "origin", URL: "git+https://example.com/old.git", Ref: ref}}}
|
||||
uc := NewDoltRemoteUseCase(repo)
|
||||
if err := uc.UpdateRemote(context.Background(), "origin", "git+https://example.com/new.git"); err != nil {
|
||||
t.Fatalf("UpdateRemote: %v", err)
|
||||
}
|
||||
want := []string{"remove origin", "add origin git+https://example.com/new.git ref=" + ref}
|
||||
if !reflect.DeepEqual(repo.calls, want) {
|
||||
t.Fatalf("calls = %v, want %v", repo.calls, want)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("restore after a failed add keeps the ref", func(t *testing.T) {
|
||||
repo := &fakeRemoteRepo{
|
||||
remotes: []Remote{{Name: "origin", URL: "git+https://example.com/old.git", Ref: "refs/dolt/units/team-12542"}},
|
||||
failAdd: "git+https://example.com/new.git",
|
||||
}
|
||||
uc := NewDoltRemoteUseCase(repo)
|
||||
if err := uc.UpdateRemote(context.Background(), "origin", "git+https://example.com/new.git"); err == nil {
|
||||
t.Fatal("UpdateRemote should report the failed add")
|
||||
}
|
||||
if len(repo.remotes) != 1 || repo.remotes[0].Ref != "refs/dolt/units/team-12542" || repo.remotes[0].URL != "git+https://example.com/old.git" {
|
||||
t.Fatalf("restored remote = %+v, want the old URL on the old ref", repo.remotes)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("a failed listing stops the update before removal", func(t *testing.T) {
|
||||
repo := &fakeRemoteRepo{
|
||||
remotes: []Remote{{Name: "origin", URL: "git+https://example.com/old.git", Ref: "refs/dolt/units/team-12542"}},
|
||||
listErr: errors.New("server unreachable"),
|
||||
}
|
||||
uc := NewDoltRemoteUseCase(repo)
|
||||
if err := uc.UpdateRemote(context.Background(), "origin", "git+https://example.com/new.git"); err == nil {
|
||||
t.Fatal("UpdateRemote should fail when the listing fails")
|
||||
}
|
||||
if len(repo.calls) != 0 {
|
||||
t.Fatalf("nothing may be removed or added after a failed listing, got %v", repo.calls)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("a move to a non-git URL drops the ref", func(t *testing.T) {
|
||||
repo := &fakeRemoteRepo{remotes: []Remote{{Name: "origin", URL: "git+https://example.com/old.git", Ref: "refs/dolt/units/team-12542"}}}
|
||||
uc := NewDoltRemoteUseCase(repo)
|
||||
if err := uc.UpdateRemote(context.Background(), "origin", "dolthub://org/repo"); err != nil {
|
||||
t.Fatalf("UpdateRemote: %v", err)
|
||||
}
|
||||
if want := []string{"remove origin", "add origin dolthub://org/repo ref="}; !reflect.DeepEqual(repo.calls, want) {
|
||||
t.Fatalf("calls = %v, want %v", repo.calls, want)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("remote on the default ref stays on it", func(t *testing.T) {
|
||||
repo := &fakeRemoteRepo{remotes: []Remote{{Name: "origin", URL: "file:///tmp/old"}}}
|
||||
uc := NewDoltRemoteUseCase(repo)
|
||||
if err := uc.UpdateRemote(context.Background(), "origin", "file:///tmp/new"); err != nil {
|
||||
t.Fatalf("UpdateRemote: %v", err)
|
||||
}
|
||||
if want := []string{"remove origin", "add origin file:///tmp/new ref="}; !reflect.DeepEqual(repo.calls, want) {
|
||||
t.Fatalf("calls = %v, want %v", repo.calls, want)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestCreateRemoteWithRefValidatesArguments(t *testing.T) {
|
||||
repo := &fakeRemoteRepo{}
|
||||
uc := NewDoltRemoteUseCase(repo)
|
||||
if err := uc.CreateRemoteWithRef(context.Background(), "", "file:///tmp/x", "refs/heads/issue-data"); err == nil {
|
||||
t.Error("empty name should be refused")
|
||||
}
|
||||
if err := uc.CreateRemoteWithRef(context.Background(), "origin", "", "refs/heads/issue-data"); err == nil {
|
||||
t.Error("empty url should be refused")
|
||||
}
|
||||
// A ref the dolt argv boundary will refuse must not be recordable here: it
|
||||
// would create a remote that can never be pushed and only says so at push
|
||||
// time. Refused before the repository is touched, so nothing is recorded.
|
||||
for _, ref := range []string{"-x", "refs/heads/issue data", "refs/heads/issue\tdata"} {
|
||||
if err := uc.CreateRemoteWithRef(context.Background(), "origin", "git+file:///srv/ledgers", ref); err == nil {
|
||||
t.Errorf("CreateRemoteWithRef with ref %q should be refused", ref)
|
||||
}
|
||||
}
|
||||
if len(repo.calls) != 0 {
|
||||
t.Fatalf("a refused CreateRemoteWithRef reached the repository: calls = %v", repo.calls)
|
||||
}
|
||||
if err := uc.CreateRemoteWithRef(context.Background(), "origin", "git+file:///srv/ledgers", "refs/dolt/units/k"); err != nil {
|
||||
t.Fatalf("CreateRemoteWithRef: %v", err)
|
||||
}
|
||||
if want := []string{"add origin git+file:///srv/ledgers ref=refs/dolt/units/k"}; !reflect.DeepEqual(repo.calls, want) {
|
||||
t.Fatalf("calls = %v, want %v", repo.calls, want)
|
||||
}
|
||||
}
|
||||
26
internal/storage/embeddeddolt/git_ref_pin_test.go
Normal file
26
internal/storage/embeddeddolt/git_ref_pin_test.go
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
//go:build cgo
|
||||
|
||||
package embeddeddolt_test
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/dolthub/dolt/go/libraries/doltcore/dbfactory"
|
||||
"github.com/dolthub/dolt/go/store/blobstore"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
)
|
||||
|
||||
// The storage package pins two Dolt literals so that the binary does not
|
||||
// import Dolt outside the storage backends. A Dolt bump that renamed either
|
||||
// would make every ref remote read as the default and route to
|
||||
// refs/dolt/data silently; this keeps the pins honest against the module
|
||||
// the engine is built from.
|
||||
func TestGitRefPinsMatchDolt(t *testing.T) {
|
||||
if storage.GitRefParam != dbfactory.GitRefParam {
|
||||
t.Errorf("storage.GitRefParam = %q, dolt dbfactory.GitRefParam = %q", storage.GitRefParam, dbfactory.GitRefParam)
|
||||
}
|
||||
if storage.DefaultGitDataRef != blobstore.DoltDataRef {
|
||||
t.Errorf("storage.DefaultGitDataRef = %q, dolt blobstore.DoltDataRef = %q", storage.DefaultGitDataRef, blobstore.DoltDataRef)
|
||||
}
|
||||
}
|
||||
185
internal/storage/embeddeddolt/remote_git_ref_test.go
Normal file
185
internal/storage/embeddeddolt/remote_git_ref_test.go
Normal file
|
|
@ -0,0 +1,185 @@
|
|||
//go:build cgo
|
||||
|
||||
package embeddeddolt_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
"github.com/steveyegge/beads/internal/storage/doltutil"
|
||||
"github.com/steveyegge/beads/internal/storage/embeddeddolt"
|
||||
"github.com/steveyegge/beads/internal/storage/versioncontrolops"
|
||||
)
|
||||
|
||||
// runGit runs git in dir and fails the test on error.
|
||||
func runGit(t *testing.T, dir string, args ...string) string {
|
||||
t.Helper()
|
||||
cmd := exec.Command("git", args...)
|
||||
cmd.Dir = dir
|
||||
// Repository-local hooks path, so a developer's global core.hooksPath
|
||||
// cannot run inside the fixture repositories (engdocs/TESTING.md).
|
||||
cmd.Env = append(os.Environ(),
|
||||
"GIT_TERMINAL_PROMPT=0",
|
||||
"GIT_AUTHOR_NAME=bd-test", "GIT_AUTHOR_EMAIL=bd-test@example.com",
|
||||
"GIT_COMMITTER_NAME=bd-test", "GIT_COMMITTER_EMAIL=bd-test@example.com",
|
||||
"GIT_CONFIG_COUNT=1", "GIT_CONFIG_KEY_0=core.hooksPath", "GIT_CONFIG_VALUE_0="+filepath.Join(dir, ".git", "hooks"),
|
||||
)
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("git %s: %v\n%s", strings.Join(args, " "), err, out)
|
||||
}
|
||||
return string(out)
|
||||
}
|
||||
|
||||
// newBareGitRemote creates a bare repository with one commit on main, the
|
||||
// shape a git host presents before any Dolt data has been pushed to it.
|
||||
func newBareGitRemote(t *testing.T, name string) string {
|
||||
t.Helper()
|
||||
base := t.TempDir()
|
||||
bare := filepath.Join(base, name)
|
||||
runGit(t, base, "init", "--bare", "-b", "main", bare)
|
||||
seed := filepath.Join(base, "seed")
|
||||
if err := os.MkdirAll(seed, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
runGit(t, seed, "init", "-b", "main")
|
||||
runGit(t, seed, "commit", "--allow-empty", "-m", "init")
|
||||
runGit(t, seed, "push", bare, "main")
|
||||
return bare
|
||||
}
|
||||
|
||||
// lsRemoteRef returns the object a ref points at on the bare repository, or
|
||||
// "" when the ref does not exist there.
|
||||
func lsRemoteRef(t *testing.T, bare, ref string) string {
|
||||
t.Helper()
|
||||
return strings.TrimSpace(runGit(t, bare, "ls-remote", bare, ref))
|
||||
}
|
||||
|
||||
// The engine, not a mock, is the authority on the SQL shapes above: a remote
|
||||
// added with a ref lists that ref back from dolt_remotes and from
|
||||
// repo_state.json, for a branch and for a ref outside refs/heads/; a remote
|
||||
// added without one lists the default; Dolt refuses a ref on a non-git URL
|
||||
// and the refusal reaches the caller; a push lands on the custom ref and
|
||||
// never on refs/dolt/data; a clone with the ref records it on origin.
|
||||
func TestRemoteGitDataRefOnEngine(t *testing.T) {
|
||||
if _, err := exec.LookPath("git"); err != nil {
|
||||
t.Skip("git not available")
|
||||
}
|
||||
ctx := context.Background()
|
||||
te := newTestEnv(t, "gref")
|
||||
|
||||
shapes := map[string]string{
|
||||
"branch": "refs/heads/issue-data",
|
||||
"unit": "refs/dolt/units/team-12542",
|
||||
}
|
||||
bares := map[string]string{}
|
||||
for name, ref := range shapes {
|
||||
bares[name] = newBareGitRemote(t, name)
|
||||
if err := te.store.AddRemoteWithRef(ctx, name, "git+file://"+bares[name], ref); err != nil {
|
||||
t.Fatalf("AddRemoteWithRef(%s, %s): %v", name, ref, err)
|
||||
}
|
||||
}
|
||||
plainDir := t.TempDir()
|
||||
if err := te.store.AddRemote(ctx, "plain", "file://"+plainDir); err != nil {
|
||||
t.Fatalf("AddRemote(plain): %v", err)
|
||||
}
|
||||
|
||||
t.Run("listed from SQL and from disk", func(t *testing.T) {
|
||||
remotes, err := te.store.ListRemotes(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ListRemotes: %v", err)
|
||||
}
|
||||
assertRefs(t, "ListRemotes", remotes, shapes)
|
||||
|
||||
persisted, err := doltutil.PersistedRemotes(filepath.Join(te.dataDir, te.database))
|
||||
if err != nil {
|
||||
t.Fatalf("PersistedRemotes: %v", err)
|
||||
}
|
||||
assertRefs(t, "PersistedRemotes", persisted, shapes)
|
||||
})
|
||||
|
||||
t.Run("ref on a non-git remote is Dolt's error, not a silent default", func(t *testing.T) {
|
||||
err := te.store.AddRemoteWithRef(ctx, "bad", "file://"+t.TempDir(), "refs/heads/issue-data")
|
||||
if err == nil {
|
||||
t.Fatal("AddRemoteWithRef on file:// with a ref = nil, want error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "git remotes") {
|
||||
t.Errorf("error should be Dolt's --ref refusal, got: %v", err)
|
||||
}
|
||||
if ok, _ := te.store.HasRemote(ctx, "bad"); ok {
|
||||
t.Error("refused remote must not exist")
|
||||
}
|
||||
})
|
||||
|
||||
for name, ref := range shapes {
|
||||
t.Run("push and clone on "+ref, func(t *testing.T) {
|
||||
bare := bares[name]
|
||||
if err := te.store.PushRemote(ctx, name, false); err != nil {
|
||||
t.Fatalf("PushRemote(%s): %v", name, err)
|
||||
}
|
||||
if got := lsRemoteRef(t, bare, ref); got == "" {
|
||||
t.Fatalf("push did not create %s on the git remote", ref)
|
||||
}
|
||||
if got := lsRemoteRef(t, bare, storage.DefaultGitDataRef); got != "" {
|
||||
t.Errorf("push created %s (%s) although the remote was added with --ref %s", storage.DefaultGitDataRef, got, ref)
|
||||
}
|
||||
if got := lsRemoteRef(t, bare, "refs/heads/main"); got == "" {
|
||||
t.Error("refs/heads/main disappeared from the git remote")
|
||||
}
|
||||
|
||||
cloneDir := t.TempDir()
|
||||
boot, bootCleanup, err := embeddeddolt.OpenSQL(ctx, cloneDir, "", "")
|
||||
if err != nil {
|
||||
t.Fatalf("OpenSQL for clone: %v", err)
|
||||
}
|
||||
cloneErr := versioncontrolops.DoltCloneWithRef(ctx, boot, "git+file://"+bare, "cloned", "", ref)
|
||||
_ = bootCleanup()
|
||||
if cloneErr != nil {
|
||||
t.Fatalf("DoltCloneWithRef: %v", cloneErr)
|
||||
}
|
||||
persisted, err := doltutil.PersistedRemotes(filepath.Join(cloneDir, "cloned"))
|
||||
if err != nil {
|
||||
t.Fatalf("PersistedRemotes on clone: %v", err)
|
||||
}
|
||||
assertRefs(t, "clone repo_state.json", persisted, map[string]string{"origin": ref})
|
||||
|
||||
db, cleanup, err := embeddeddolt.OpenSQL(ctx, cloneDir, "cloned", "main")
|
||||
if err != nil {
|
||||
t.Fatalf("OpenSQL on clone: %v", err)
|
||||
}
|
||||
defer func() { _ = cleanup() }()
|
||||
listed, err := versioncontrolops.ListRemotes(ctx, db)
|
||||
if err != nil {
|
||||
t.Fatalf("ListRemotes on clone: %v", err)
|
||||
}
|
||||
assertRefs(t, "clone dolt_remotes", listed, map[string]string{"origin": ref})
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// assertRefs checks that every named remote in want lists exactly that ref
|
||||
// and that the remote named plain, when present, lists the default.
|
||||
func assertRefs(t *testing.T, where string, remotes []storage.RemoteInfo, want map[string]string) {
|
||||
t.Helper()
|
||||
got := map[string]string{}
|
||||
for _, r := range remotes {
|
||||
got[r.Name] = r.Ref
|
||||
}
|
||||
for name, ref := range want {
|
||||
if _, ok := got[name]; !ok {
|
||||
t.Errorf("%s: remote %s missing from %+v", where, name, remotes)
|
||||
continue
|
||||
}
|
||||
if got[name] != ref {
|
||||
t.Errorf("%s: remote %s Ref = %q, want %q", where, name, got[name], ref)
|
||||
}
|
||||
}
|
||||
if ref, ok := got["plain"]; ok && ref != "" {
|
||||
t.Errorf("%s: remote plain Ref = %q, want empty", where, ref)
|
||||
}
|
||||
}
|
||||
|
|
@ -198,9 +198,14 @@ func (s *EmbeddedDoltStore) CommitMergeResolution(ctx context.Context, message s
|
|||
}
|
||||
|
||||
func (s *EmbeddedDoltStore) AddRemote(ctx context.Context, name, url string) error {
|
||||
return s.AddRemoteWithRef(ctx, name, url, "")
|
||||
}
|
||||
|
||||
// AddRemoteWithRef adds a remote whose Dolt data lives on the git ref ref;
|
||||
// see storage.RemoteStore.
|
||||
func (s *EmbeddedDoltStore) AddRemoteWithRef(ctx context.Context, name, url, ref string) error {
|
||||
return s.withMutatingDBConn(ctx, func(db versioncontrolops.DBConn) error {
|
||||
_, err := db.ExecContext(ctx, "CALL DOLT_REMOTE('add', ?, ?)", name, url)
|
||||
return err
|
||||
return versioncontrolops.AddRemote(ctx, db, name, url, ref)
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -5,6 +5,10 @@ import "context"
|
|||
// RemoteStore provides remote management and push/pull/fetch operations.
|
||||
type RemoteStore interface {
|
||||
AddRemote(ctx context.Context, name, url string) error
|
||||
// AddRemoteWithRef is AddRemote for a git-backed remote whose Dolt data
|
||||
// lives on the git ref ref instead of DefaultGitDataRef. An empty ref is
|
||||
// AddRemote.
|
||||
AddRemoteWithRef(ctx context.Context, name, url, ref string) error
|
||||
RemoveRemote(ctx context.Context, name string) error
|
||||
HasRemote(ctx context.Context, name string) (bool, error)
|
||||
ListRemotes(ctx context.Context) ([]RemoteInfo, error)
|
||||
|
|
|
|||
69
internal/storage/remote_ref.go
Normal file
69
internal/storage/remote_ref.go
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
package storage
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// A git-backed Dolt remote keeps its data on one git ref of the remote
|
||||
// repository. Dolt records the ref as the remote parameter GitRefParam and
|
||||
// falls back to DefaultGitDataRef when the parameter is absent.
|
||||
const (
|
||||
GitRefParam = "git_ref"
|
||||
DefaultGitDataRef = "refs/dolt/data"
|
||||
)
|
||||
|
||||
// EffectiveGitDataRef returns ref, or DefaultGitDataRef when ref is empty.
|
||||
func EffectiveGitDataRef(ref string) string {
|
||||
if ref = strings.TrimSpace(ref); ref != "" {
|
||||
return ref
|
||||
}
|
||||
return DefaultGitDataRef
|
||||
}
|
||||
|
||||
// RemoteRefsMatch reports whether two git data refs name the same ref. An
|
||||
// empty ref stands for DefaultGitDataRef, so an unset ref and an explicit
|
||||
// default compare equal.
|
||||
func RemoteRefsMatch(a, b string) bool {
|
||||
return EffectiveGitDataRef(a) == EffectiveGitDataRef(b)
|
||||
}
|
||||
|
||||
// GitRefFromParamsJSON extracts GitRefParam from a dolt_remotes params value.
|
||||
// "", "null", an empty object, and an object without the key mean the
|
||||
// default ref; anything that is not a JSON object is an error rather than a
|
||||
// silent default.
|
||||
func GitRefFromParamsJSON(raw string) (string, error) {
|
||||
raw = strings.TrimSpace(raw)
|
||||
if raw == "" || raw == "null" {
|
||||
return "", nil
|
||||
}
|
||||
var params map[string]any
|
||||
if err := json.Unmarshal([]byte(raw), ¶ms); err != nil {
|
||||
return "", fmt.Errorf("parse dolt_remotes params %q: %w", raw, err)
|
||||
}
|
||||
return GitRefFromParams(params)
|
||||
}
|
||||
|
||||
// GitRefFromParams extracts GitRefParam from an already-decoded dolt_remotes
|
||||
// params object. It is the one reader of that key: GitRefFromParamsJSON uses
|
||||
// it for the SQL params column and doltutil.PersistedRemotes for the same
|
||||
// object in repo_state.json, so the two planes cannot drift.
|
||||
//
|
||||
// An absent key and a JSON null mean the default ref. A key that is present
|
||||
// but is not a string is an error, for the same reason a params value that is
|
||||
// not a JSON object is: a recorded ref bd cannot read could be a custom ref
|
||||
// the mirror is pinned to, and reading it as the default would push the
|
||||
// remote's data onto DefaultGitDataRef. Dolt only ever writes a string here,
|
||||
// so this is the fail-closed edge of that invariant, not a live shape.
|
||||
func GitRefFromParams(params map[string]any) (string, error) {
|
||||
value, ok := params[GitRefParam]
|
||||
if !ok || value == nil {
|
||||
return "", nil
|
||||
}
|
||||
ref, ok := value.(string)
|
||||
if !ok {
|
||||
return "", fmt.Errorf("remote parameter %s is %T, not a string", GitRefParam, value)
|
||||
}
|
||||
return strings.TrimSpace(ref), nil
|
||||
}
|
||||
114
internal/storage/remote_ref_test.go
Normal file
114
internal/storage/remote_ref_test.go
Normal file
|
|
@ -0,0 +1,114 @@
|
|||
package storage
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// An unset ref and Dolt's default name the same ref; any other value is
|
||||
// compared verbatim, so a branch and a ref outside refs/heads/ are distinct
|
||||
// from each other and from the default.
|
||||
func TestRemoteRefsMatch(t *testing.T) {
|
||||
tests := []struct {
|
||||
a, b string
|
||||
want bool
|
||||
}{
|
||||
{"", "", true},
|
||||
{"", DefaultGitDataRef, true},
|
||||
{DefaultGitDataRef, "", true},
|
||||
{" refs/dolt/data ", "", true},
|
||||
{"refs/heads/issue-data", "refs/heads/issue-data", true},
|
||||
{"refs/dolt/units/team-12542", "refs/dolt/units/team-12542", true},
|
||||
{"refs/heads/issue-data", "", false},
|
||||
{"refs/dolt/units/team-12542", "", false},
|
||||
{"refs/heads/issue-data", "refs/dolt/units/team-12542", false},
|
||||
{"refs/heads/issue-data", "refs/heads/issue-data-2", false},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
if got := RemoteRefsMatch(tt.a, tt.b); got != tt.want {
|
||||
t.Errorf("RemoteRefsMatch(%q, %q) = %v, want %v", tt.a, tt.b, got, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A params object carries the ref as a string or not at all. A present value
|
||||
// of any other JSON type is an error rather than the default ref: reading it as
|
||||
// the default is how a remote pinned to a custom ref gets pushed onto
|
||||
// refs/dolt/data, which is exactly the failure this parameter exists to
|
||||
// prevent. An absent key and an explicit null are the genuine "no ref" shapes.
|
||||
func TestGitRefFromParams(t *testing.T) {
|
||||
for _, tt := range []struct {
|
||||
name string
|
||||
params map[string]any
|
||||
want string
|
||||
}{
|
||||
{"absent", map[string]any{"aws-region": "us-east-1"}, ""},
|
||||
{"empty object", map[string]any{}, ""},
|
||||
{"nil map", nil, ""},
|
||||
{"null", map[string]any{GitRefParam: nil}, ""},
|
||||
{"empty string", map[string]any{GitRefParam: ""}, ""},
|
||||
{"trimmed", map[string]any{GitRefParam: " refs/dolt/units/k "}, "refs/dolt/units/k"},
|
||||
{"branch", map[string]any{GitRefParam: "refs/heads/issue-data"}, "refs/heads/issue-data"},
|
||||
} {
|
||||
got, err := GitRefFromParams(tt.params)
|
||||
if err != nil || got != tt.want {
|
||||
t.Errorf("GitRefFromParams(%s) = %q, %v; want %q, nil", tt.name, got, err, tt.want)
|
||||
}
|
||||
}
|
||||
|
||||
for _, tt := range []struct {
|
||||
name string
|
||||
value any
|
||||
}{
|
||||
{"array", []any{"refs/dolt/units/x"}},
|
||||
{"number", float64(7)},
|
||||
{"bool", true},
|
||||
{"object", map[string]any{"ref": "refs/dolt/units/x"}},
|
||||
} {
|
||||
got, err := GitRefFromParams(map[string]any{GitRefParam: tt.value})
|
||||
if err == nil {
|
||||
t.Errorf("GitRefFromParams(%s) = %q, nil; want an error, not the default ref", tt.name, got)
|
||||
continue
|
||||
}
|
||||
if !strings.Contains(err.Error(), GitRefParam) {
|
||||
t.Errorf("GitRefFromParams(%s) error %q does not name %s", tt.name, err, GitRefParam)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// GitRefFromParamsJSON reads the same key out of the SQL params column, so it
|
||||
// must refuse the same shapes: the two readers are one implementation.
|
||||
func TestGitRefFromParamsJSONRefusesNonStringRef(t *testing.T) {
|
||||
for _, raw := range []string{
|
||||
`{"git_ref":["refs/dolt/units/x"]}`,
|
||||
`{"git_ref":7}`,
|
||||
`{"git_ref":true}`,
|
||||
`{"git_ref":{"ref":"refs/dolt/units/x"}}`,
|
||||
} {
|
||||
if got, err := GitRefFromParamsJSON(raw); err == nil {
|
||||
t.Errorf("GitRefFromParamsJSON(%s) = %q, nil; want an error", raw, got)
|
||||
}
|
||||
}
|
||||
for _, tt := range []struct{ raw, want string }{
|
||||
{"", ""},
|
||||
{"null", ""},
|
||||
{`{}`, ""},
|
||||
{`{"git_ref":null}`, ""},
|
||||
{`{"aws-region":"us-east-1"}`, ""},
|
||||
{`{"git_ref":"refs/heads/issue-data"}`, "refs/heads/issue-data"},
|
||||
} {
|
||||
got, err := GitRefFromParamsJSON(tt.raw)
|
||||
if err != nil || got != tt.want {
|
||||
t.Errorf("GitRefFromParamsJSON(%s) = %q, %v; want %q, nil", tt.raw, got, err, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestEffectiveGitDataRef(t *testing.T) {
|
||||
if got := EffectiveGitDataRef(""); got != DefaultGitDataRef {
|
||||
t.Errorf("EffectiveGitDataRef(\"\") = %q, want %q", got, DefaultGitDataRef)
|
||||
}
|
||||
if got := EffectiveGitDataRef(" refs/dolt/units/k "); got != "refs/dolt/units/k" {
|
||||
t.Errorf("EffectiveGitDataRef trims but does not rewrite: got %q", got)
|
||||
}
|
||||
}
|
||||
|
|
@ -4,6 +4,7 @@ import (
|
|||
"context"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// DoltClone clones a Dolt database from a remote URL.
|
||||
|
|
@ -12,12 +13,27 @@ import (
|
|||
// If user is non-empty, authenticates with that user. Dolt reads the remote
|
||||
// password from DOLT_REMOTE_PASSWORD.
|
||||
func DoltClone(ctx context.Context, conn DBConn, remoteURL, database, user string) error {
|
||||
query := "CALL DOLT_CLONE(?, ?)"
|
||||
args := []any{remoteURL, database}
|
||||
return DoltCloneWithRef(ctx, conn, remoteURL, database, user, "")
|
||||
}
|
||||
|
||||
// DoltCloneWithRef is DoltClone for a git-backed remote whose Dolt data lives
|
||||
// on the git ref ref. A non-empty ref is passed as DOLT_CLONE's --ref option;
|
||||
// Dolt reads the data from that ref and records it on the clone's origin
|
||||
// remote, so later push and pull use it. An empty ref is DoltClone.
|
||||
func DoltCloneWithRef(ctx context.Context, conn DBConn, remoteURL, database, user, ref string) error {
|
||||
var parts []string
|
||||
var args []any
|
||||
if user != "" {
|
||||
query = "CALL DOLT_CLONE('--user', ?, ?, ?)"
|
||||
args = []any{user, remoteURL, database}
|
||||
parts = append(parts, "'--user', ?")
|
||||
args = append(args, user)
|
||||
}
|
||||
if ref = strings.TrimSpace(ref); ref != "" {
|
||||
parts = append(parts, "'--ref', ?")
|
||||
args = append(args, ref)
|
||||
}
|
||||
parts = append(parts, "?, ?")
|
||||
args = append(args, remoteURL, database)
|
||||
query := "CALL DOLT_CLONE(" + strings.Join(parts, ", ") + ")"
|
||||
|
||||
// GH#4272: the initial fetch runs git hooks just like push/pull; disable
|
||||
// them for the clone window too (see remotes.go for the full rationale).
|
||||
|
|
|
|||
|
|
@ -27,6 +27,49 @@ func TestDoltCloneWithoutUser(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// A ref reaches DOLT_CLONE as --ref, verbatim, for both ref shapes.
|
||||
func TestDoltCloneWithRef(t *testing.T) {
|
||||
for _, ref := range gitDataRefShapes {
|
||||
t.Run(ref, func(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatalf("sqlmock.New: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
mock.ExpectExec(regexp.QuoteMeta("CALL DOLT_CLONE('--ref', ?, ?, ?)")).
|
||||
WithArgs(ref, "git+https://example.com/repo.git", "beads").
|
||||
WillReturnResult(sqlmock.NewResult(0, 1))
|
||||
|
||||
if err := DoltCloneWithRef(context.Background(), db, "git+https://example.com/repo.git", "beads", "", ref); err != nil {
|
||||
t.Fatalf("DoltCloneWithRef: %v", err)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestDoltCloneWithUserAndRef(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatalf("sqlmock.New: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
mock.ExpectExec(regexp.QuoteMeta("CALL DOLT_CLONE('--user', ?, '--ref', ?, ?, ?)")).
|
||||
WithArgs("alice", "refs/dolt/units/team-12542", "git+https://example.com/repo.git", "beads").
|
||||
WillReturnResult(sqlmock.NewResult(0, 1))
|
||||
|
||||
if err := DoltCloneWithRef(context.Background(), db, "git+https://example.com/repo.git", "beads", "alice", "refs/dolt/units/team-12542"); err != nil {
|
||||
t.Fatalf("DoltCloneWithRef: %v", err)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDoltCloneWithUser(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
|
|
|
|||
|
|
@ -2,14 +2,19 @@ package versioncontrolops
|
|||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
)
|
||||
|
||||
// ListRemotes returns all configured Dolt remotes (name and URL).
|
||||
// ListRemotes returns all configured Dolt remotes: name, URL, and the git ref
|
||||
// a git-backed remote keeps its data on when one was set. The ref is the
|
||||
// git_ref key of the dolt_remotes params column, a JSON object of the
|
||||
// remote's parameters; a remote without the key lists an empty Ref.
|
||||
func ListRemotes(ctx context.Context, db DBConn) ([]storage.RemoteInfo, error) {
|
||||
rows, err := db.QueryContext(ctx, "SELECT name, url FROM dolt_remotes")
|
||||
rows, err := db.QueryContext(ctx, "SELECT name, url, params FROM dolt_remotes")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list remotes: %w", err)
|
||||
}
|
||||
|
|
@ -18,14 +23,35 @@ func ListRemotes(ctx context.Context, db DBConn) ([]storage.RemoteInfo, error) {
|
|||
var remotes []storage.RemoteInfo
|
||||
for rows.Next() {
|
||||
var r storage.RemoteInfo
|
||||
if err := rows.Scan(&r.Name, &r.URL); err != nil {
|
||||
var params sql.NullString
|
||||
if err := rows.Scan(&r.Name, &r.URL, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("scan remote: %w", err)
|
||||
}
|
||||
if params.Valid {
|
||||
ref, err := storage.GitRefFromParamsJSON(params.String)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("remote %s: %w", r.Name, err)
|
||||
}
|
||||
r.Ref = ref
|
||||
}
|
||||
remotes = append(remotes, r)
|
||||
}
|
||||
return remotes, rows.Err()
|
||||
}
|
||||
|
||||
// AddRemote adds a Dolt remote. A non-empty ref is passed as DOLT_REMOTE's
|
||||
// --ref option, which Dolt records as the remote's git_ref parameter and
|
||||
// accepts for git-backed remotes only; for any other scheme Dolt's own
|
||||
// refusal is returned unchanged.
|
||||
func AddRemote(ctx context.Context, db DBConn, name, url, ref string) error {
|
||||
if ref = strings.TrimSpace(ref); ref == "" {
|
||||
_, err := db.ExecContext(ctx, "CALL DOLT_REMOTE('add', ?, ?)", name, url)
|
||||
return err
|
||||
}
|
||||
_, err := db.ExecContext(ctx, "CALL DOLT_REMOTE('add', '--ref', ?, ?, ?)", ref, name, url)
|
||||
return err
|
||||
}
|
||||
|
||||
// RemoveRemote removes a configured Dolt remote.
|
||||
func RemoveRemote(ctx context.Context, db DBConn, name string) error {
|
||||
if _, err := db.ExecContext(ctx, "CALL DOLT_REMOTE('remove', ?)", name); err != nil {
|
||||
|
|
|
|||
132
internal/storage/versioncontrolops/remotes_test.go
Normal file
132
internal/storage/versioncontrolops/remotes_test.go
Normal file
|
|
@ -0,0 +1,132 @@
|
|||
package versioncontrolops
|
||||
|
||||
import (
|
||||
"context"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/DATA-DOG/go-sqlmock"
|
||||
)
|
||||
|
||||
// The two ref shapes every ref test in this change covers: a branch, for a
|
||||
// git host that only accepts pushes under refs/heads/, and a ref elsewhere
|
||||
// in the namespace, for several databases kept in one repository.
|
||||
var gitDataRefShapes = []string{"refs/heads/issue-data", "refs/dolt/units/team-12542"}
|
||||
|
||||
// Without a ref the add is the same two-argument DOLT_REMOTE call every
|
||||
// existing remote was created with.
|
||||
func TestAddRemoteWithoutRef(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatalf("sqlmock.New: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
mock.ExpectExec(regexp.QuoteMeta("CALL DOLT_REMOTE('add', ?, ?)")).
|
||||
WithArgs("origin", "git+https://example.com/repo.git").
|
||||
WillReturnResult(sqlmock.NewResult(0, 1))
|
||||
|
||||
if err := AddRemote(context.Background(), db, "origin", "git+https://example.com/repo.git", " "); err != nil {
|
||||
t.Fatalf("AddRemote: %v", err)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// With a ref the add carries --ref, and the value reaches Dolt verbatim for
|
||||
// both shapes.
|
||||
func TestAddRemoteWithRef(t *testing.T) {
|
||||
for _, ref := range gitDataRefShapes {
|
||||
t.Run(ref, func(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatalf("sqlmock.New: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
mock.ExpectExec(regexp.QuoteMeta("CALL DOLT_REMOTE('add', '--ref', ?, ?, ?)")).
|
||||
WithArgs(ref, "origin", "git+https://example.com/repo.git").
|
||||
WillReturnResult(sqlmock.NewResult(0, 1))
|
||||
|
||||
if err := AddRemote(context.Background(), db, "origin", "git+https://example.com/repo.git", ref); err != nil {
|
||||
t.Fatalf("AddRemote: %v", err)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// ListRemotes reads the ref back out of the params column. A NULL, the text
|
||||
// "null" a sql-server sends for a remote without parameters, an empty
|
||||
// object, and an object without git_ref all list as the default (empty Ref);
|
||||
// a recorded git_ref comes back verbatim.
|
||||
func TestListRemotesReadsGitRefParam(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatalf("sqlmock.New: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
rows := sqlmock.NewRows([]string{"name", "url", "params"}).
|
||||
AddRow("null-params", "file:///tmp/a", nil).
|
||||
AddRow("null-text", "file:///tmp/b", "null").
|
||||
AddRow("empty-params", "dolthub://org/repo", "{}").
|
||||
AddRow("other-params", "aws://bucket/repo", `{"aws-region":"us-east-1"}`).
|
||||
AddRow("branch", "git+https://example.com/repo.git", `{"git_ref":"refs/heads/issue-data"}`).
|
||||
AddRow("unit", "git+file:///srv/ledgers", `{"git_ref":"refs/dolt/units/team-12542"}`)
|
||||
mock.ExpectQuery(regexp.QuoteMeta("SELECT name, url, params FROM dolt_remotes")).WillReturnRows(rows)
|
||||
|
||||
remotes, err := ListRemotes(context.Background(), db)
|
||||
if err != nil {
|
||||
t.Fatalf("ListRemotes: %v", err)
|
||||
}
|
||||
want := map[string]string{
|
||||
"null-params": "",
|
||||
"null-text": "",
|
||||
"empty-params": "",
|
||||
"other-params": "",
|
||||
"branch": "refs/heads/issue-data",
|
||||
"unit": "refs/dolt/units/team-12542",
|
||||
}
|
||||
if len(remotes) != len(want) {
|
||||
t.Fatalf("ListRemotes returned %d remotes, want %d: %+v", len(remotes), len(want), remotes)
|
||||
}
|
||||
for _, r := range remotes {
|
||||
if r.Ref != want[r.Name] {
|
||||
t.Errorf("remote %s: Ref = %q, want %q", r.Name, r.Ref, want[r.Name])
|
||||
}
|
||||
}
|
||||
if remotes[0].URL != "file:///tmp/a" {
|
||||
t.Errorf("URL still scanned: got %q", remotes[0].URL)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// A params value that is not a JSON object is an error naming the remote,
|
||||
// never a silent fall-back to the default ref: routing a custom-ref remote
|
||||
// through the default would push its data to the wrong ref.
|
||||
func TestListRemotesRejectsMalformedParams(t *testing.T) {
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatalf("sqlmock.New: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
rows := sqlmock.NewRows([]string{"name", "url", "params"}).
|
||||
AddRow("origin", "git+https://example.com/repo.git", "not-json")
|
||||
mock.ExpectQuery(regexp.QuoteMeta("SELECT name, url, params FROM dolt_remotes")).WillReturnRows(rows)
|
||||
|
||||
_, err = ListRemotes(context.Background(), db)
|
||||
if err == nil {
|
||||
t.Fatal("ListRemotes with malformed params = nil error, want error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "origin") || !strings.Contains(err.Error(), "params") {
|
||||
t.Errorf("error should name the remote and the params column: %v", err)
|
||||
}
|
||||
}
|
||||
|
|
@ -39,6 +39,10 @@ type Conflict struct {
|
|||
type RemoteInfo struct {
|
||||
Name string `json:"name"` // Remote name (e.g., "town-beta")
|
||||
URL string `json:"url"` // Remote URL (e.g., "dolthub://org/repo")
|
||||
// Ref is the git ref a git-backed remote keeps its Dolt data on, when the
|
||||
// remote was added with one. Empty means DefaultGitDataRef; always empty
|
||||
// for remotes that are not git-backed.
|
||||
Ref string `json:"ref,omitempty"`
|
||||
}
|
||||
|
||||
// SyncStatus describes the synchronization state with a peer.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue