* fix(doltremote): treat s3:// as a native Dolt remote scheme s3:// URLs previously survived Normalize only by falling through, and the SCP heuristic treated any "@" in a URL as an SCP-style git remote, so an s3 URL with "@" in its path or query string would be rewritten into a bogus git+ssh remote. Add s3:// to NativeSchemes and anchor SCP detection to inputs without a "://" scheme that match a user@host:path grammar. * test(doltremote): pin the s3:// entry and the dotless host:pa@th case Follow-up to the #5948 review. Deleting "s3://" from NativeSchemes made no test fail, so TestNativeSchemesContainsEachNativeScheme now asserts every entry the list holds; removing the s3:// line fails its s3:// subtest. The base change also flipped a fourth behavior its description did not list: a dotless pre-colon token whose only "@" sits after the colon ("host:pa@th", "alias:repo@v1") used to classify as SCP-style on the bare strings.Contains(url, "@") check and normalize to git+ssh://host/pa@th. It now classifies false and passes through Normalize unchanged. Both inputs are pinned as rows in TestIsSCPStyleGitURLRejectsNonSCPInputs; against origin/main's remote.go they fail. Normalize's doc comment now lists s3:// with the other native schemes. * build(bazel): add go_test target for internal/doltremote The PR adds remote_test.go and remote_invariant_test.go to a package whose BUILD.bazel (added on main by #6798) declares only a go_library. Generated by bazel run //:gazelle; no hand-written lines. * fix(doltremote): guard FromGitURL's SCP split, cross-reference the SCP grammars Maintainer review fixes for the four non-gating findings in the iteration-1 synthesis (0 blockers, 0 majors; quality scorecard 986/approve). The PR's own fix is unchanged. Contract fidelity, the one gate-impact-minor finding: FromGitURL is exported and cmd/bd's gitURLToDoltRemote calls it directly, bypassing Normalize's classification. Its first-colon split still rebuilt s3://bucket/team@prod/db into git+ssh://s3///bucket/team@prod/db, the exact string this PR removes from Normalize. That split now obeys the same "://" rule as isSCPStyleGitURL. This takes the finding's second offered arm over its first. Gating on isSCPStyleGitURL(url) instead changes 17 of 31 inputs across the package's own corpus, against 9 for the "://" guard; the 8 extra are the SCP hosts that predicate deliberately declines to classify - IDN hosts, non-ASCII userinfo and dotless aliases - which FromGitURL converts correctly today. TestFromGitURLStillConvertsSCPForms pins them, and mutating to the rejected arm reds exactly those three rows while the new regression pin still passes. NativeSchemes: the finding's "add az:// and oci:// for consistency" arm is declined and the documentation arm taken instead. The PR body scopes this change to s3:// and defers az:// to #6227 explicitly; #6227 is open, and it pairs NativeSchemes with dolt.IsBackupURL - the site where the gap actually changes behavior - so adding the entry here would have overridden a tracked contributor decision while closing only half the bug. oci:// is in neither authority's native list. The list's doc comment now records that narrowing, points at remotecache.remoteSchemes as the fuller authority, and states that membership is inert: every entry contains "://", so isSCPStyleGitURL rejects it and Normalize's closing "return url" returns the same bytes listed or not. Dropping gs:// reds one census leaf and no behavioral test, which measures that claim rather than asserting it. The new FromGitURL table covers az:// and oci:// precisely because the guard is keyed on "://" and not on list membership. Duplicate SCP grammar: scpStyleGitURLPattern's first alternative is byte-identical to remotecache.gitSSHPattern, and the whole pattern is a superset by exactly the user-less dotted-host form - the form on which the two already disagree in production. Both regexes now cross-reference each other and the divergence is pinned from each side. Exporting a shared matcher was the other offered arm; it would have widened remotecache.IsRemoteURL's accept set, a behavior change in a second package for a non-gating finding. normalizeRemoteURL's doc comment restated Normalize's rules and had drifted twice: it never gained s3://, and it named gitURLToDoltRemote as the converter when the body calls doltremote.Normalize directly. Collapsed it to a reference so the surface cannot drift again. Codex's prescribed remedy for the hollow NativeSchemes census subtest is declined, as the synthesis directs: asserting Normalize(scheme+"bucket/x") per scheme passes against an empty list, for the same inertness reason above. The subtest's comment, which claimed to protect a fast path it cannot observe, is corrected instead. go test ./internal/doltremote/ ./internal/remotecache/ green; cmd/bd named selector 56 pass, 0 fail, 0 skip; go build ./..., go vet and gofmt clean; bazel test passes on both packages; gazelle and go_srcs.py --check report no BUILD drift. --------- Co-authored-by: Julian Knutsen <julian@gastownhall.com> Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
147 lines
5.7 KiB
Go
147 lines
5.7 KiB
Go
package main
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"os"
|
|
"regexp"
|
|
"strings"
|
|
|
|
"github.com/steveyegge/beads/internal/beads"
|
|
"github.com/steveyegge/beads/internal/config"
|
|
"github.com/steveyegge/beads/internal/doltremote"
|
|
)
|
|
|
|
// resolveSyncRemote returns the effective sync remote URL.
|
|
// Resolution order:
|
|
// 1. sync.remote (primary — any Dolt-compatible remote URL)
|
|
// 2. sync.git-remote (deprecated fallback)
|
|
// 3. "" (not configured)
|
|
func resolveSyncRemote() string {
|
|
if v := config.GetString("sync.remote"); v != "" {
|
|
return v
|
|
}
|
|
return config.GetString("sync.git-remote")
|
|
}
|
|
|
|
// resolveSyncRemoteFromDir is like resolveSyncRemote but reads from a
|
|
// specific beads directory's config.yaml. Used by context_cmd, doctor,
|
|
// and other paths that operate on a resolved beads dir rather than CWD.
|
|
func resolveSyncRemoteFromDir(beadsDir string) string {
|
|
if v := config.GetStringFromDir(beadsDir, "sync.remote"); v != "" {
|
|
return v
|
|
}
|
|
return config.GetStringFromDir(beadsDir, "sync.git-remote")
|
|
}
|
|
|
|
// commitBeadsConfig stages .beads/config.yaml and commits it.
|
|
// Silently no-ops if the file is clean or the commit fails (e.g. hooks,
|
|
// nothing to commit). Used by bd dolt remote add/remove to keep the
|
|
// working tree clean after persisting sync.remote.
|
|
func commitBeadsConfig(msg string) {
|
|
commitBeadsConfigForActiveRepo(context.Background(), msg)
|
|
}
|
|
|
|
func commitBeadsConfigForActiveRepo(ctx context.Context, msg string) {
|
|
rc, err := beads.GetRepoContext()
|
|
if err != nil {
|
|
return
|
|
}
|
|
addCmd := rc.GitCmd(ctx, "add", ".beads/config.yaml")
|
|
if err := addCmd.Run(); err != nil {
|
|
return
|
|
}
|
|
commitCmd := rc.GitCmd(ctx, "commit", "-m", msg)
|
|
if out, err := commitCmd.CombinedOutput(); err != nil {
|
|
if !strings.Contains(string(out), "nothing to commit") {
|
|
fmt.Fprintf(os.Stderr, "Warning: failed to commit config change: %v\n", err)
|
|
}
|
|
}
|
|
}
|
|
|
|
// normalizeRemoteURL converts a remote URL to a Dolt-compatible format.
|
|
// See doltremote.Normalize for the scheme rules - this wrapper adds nothing and
|
|
// exists only so call sites in this package read against one local name.
|
|
// Restating those rules here drifts: this comment still omitted s3:// after the
|
|
// source gained it, and named gitURLToDoltRemote as the converter when the call
|
|
// below goes to doltremote.Normalize directly.
|
|
func normalizeRemoteURL(url string) string {
|
|
return doltremote.Normalize(url)
|
|
}
|
|
|
|
// doltRemoteURL returns the form of remote that bd hands to Dolt — for
|
|
// DOLT_CLONE, for DOLT_REMOTE('add', ...), and therefore for DOLT_PUSH.
|
|
//
|
|
// A git-forge URL must reach Dolt in its git+ form. Dolt's dbfactory routes by
|
|
// scheme: raw http(s):// goes to the remotesapi client, which speaks Dolt's
|
|
// wire protocol at github.com and retries indefinitely (#4421), while
|
|
// git+https/git+ssh goes to the git remote factory, which shells out to git
|
|
// and fails cleanly.
|
|
//
|
|
// Everything else is returned byte-identical. That preserves GH#3339: a
|
|
// user-configured Dolt remotesapi endpoint (http://myserver:7007/mydb) must
|
|
// never be rewritten to git+http://, and it never classifies as a forge URL.
|
|
//
|
|
// This is the single owner of that routing rule. bd init and bd bootstrap must
|
|
// both call it: if they disagree about the URL derived from one committed
|
|
// sync.remote, that is the #5743 class of skew all over again.
|
|
func doltRemoteURL(remote string) string {
|
|
if isGitCodeRepoURL(remote) {
|
|
return normalizeRemoteURL(remote)
|
|
}
|
|
return remote
|
|
}
|
|
|
|
// redactRemoteURL strips credentials from a remote URL so it is safe to echo
|
|
// into errors, hints, logs and JSON. CI commonly configures sync.remote as
|
|
// https://x-access-token:<token>@github.com/org/repo.git, and the clone funnel
|
|
// already scrubs userinfo before reporting (versioncontrolops.sanitizeURL);
|
|
// diagnostics must not be the hole in that convention.
|
|
//
|
|
// HTTP(S) userinfo is transport credentials and is dropped whole. SSH userinfo
|
|
// selects the remote account (git@host is not a secret and is needed for the
|
|
// hint to be runnable), so only a password component is dropped.
|
|
func redactRemoteURL(raw string) string {
|
|
sep := strings.Index(raw, "://")
|
|
if sep < 0 {
|
|
// scp-style (git@host:path) or a bare path: no place for a password.
|
|
return raw
|
|
}
|
|
scheme, rest := raw[:sep], raw[sep+3:]
|
|
authority, tail := rest, ""
|
|
if slash := strings.Index(rest, "/"); slash >= 0 {
|
|
authority, tail = rest[:slash], rest[slash:]
|
|
}
|
|
at := strings.LastIndex(authority, "@")
|
|
if at < 0 {
|
|
return raw
|
|
}
|
|
userinfo, host := authority[:at], authority[at+1:]
|
|
if isDoltSSHScheme(scheme) {
|
|
if colon := strings.Index(userinfo, ":"); colon >= 0 {
|
|
userinfo = userinfo[:colon]
|
|
}
|
|
return scheme + "://" + userinfo + "@" + host + tail
|
|
}
|
|
return scheme + "://" + host + tail
|
|
}
|
|
|
|
func isDoltSSHScheme(scheme string) bool {
|
|
return scheme == "ssh" || scheme == "git+ssh"
|
|
}
|
|
|
|
// urlWithUserinfoRe matches the "scheme://userinfo@host" span of a URL embedded
|
|
// anywhere in free-form text — for example the git stderr line bootstrap folds
|
|
// into a probe error, which can echo the very remote (credentials and all) that
|
|
// git failed to reach. The userinfo run stops at the first "@" and the host run
|
|
// at the first "/" so a trailing path is left for redactRemoteURL to keep.
|
|
var urlWithUserinfoRe = regexp.MustCompile(`[a-zA-Z][a-zA-Z0-9+.-]*://[^@\s'"]*@[^/\s'"]*`)
|
|
|
|
// scrubURLCredentials redacts credential-bearing URLs embedded anywhere in s by
|
|
// running each match through redactRemoteURL, so the same rule applied to the
|
|
// plan's own URL fields (http(s) drops userinfo, ssh drops only a password) also
|
|
// covers URLs that arrive inside arbitrary text. Text with no "userinfo@" URL is
|
|
// returned unchanged, so credential-free git diagnostics keep their full detail.
|
|
func scrubURLCredentials(s string) string {
|
|
return urlWithUserinfoRe.ReplaceAllStringFunc(s, redactRemoteURL)
|
|
}
|