beads/cmd/bd/sync_remote.go
Jones 89348699b0
fix(doltremote): treat s3:// as a native Dolt remote scheme (#5948)
* fix(doltremote): treat s3:// as a native Dolt remote scheme

s3:// URLs previously survived Normalize only by falling through, and the
SCP heuristic treated any "@" in a URL as an SCP-style git remote, so an
s3 URL with "@" in its path or query string would be rewritten into a
bogus git+ssh remote. Add s3:// to NativeSchemes and anchor SCP detection
to inputs without a "://" scheme that match a user@host:path grammar.

* test(doltremote): pin the s3:// entry and the dotless host:pa@th case

Follow-up to the #5948 review. Deleting "s3://" from NativeSchemes made
no test fail, so TestNativeSchemesContainsEachNativeScheme now asserts
every entry the list holds; removing the s3:// line fails its s3://
subtest.

The base change also flipped a fourth behavior its description did not
list: a dotless pre-colon token whose only "@" sits after the colon
("host:pa@th", "alias:repo@v1") used to classify as SCP-style on the
bare strings.Contains(url, "@") check and normalize to
git+ssh://host/pa@th. It now classifies false and passes through
Normalize unchanged. Both inputs are pinned as rows in
TestIsSCPStyleGitURLRejectsNonSCPInputs; against origin/main's
remote.go they fail.

Normalize's doc comment now lists s3:// with the other native schemes.

* build(bazel): add go_test target for internal/doltremote

The PR adds remote_test.go and remote_invariant_test.go to a package whose
BUILD.bazel (added on main by #6798) declares only a go_library. Generated by
bazel run //:gazelle; no hand-written lines.

* fix(doltremote): guard FromGitURL's SCP split, cross-reference the SCP grammars

Maintainer review fixes for the four non-gating findings in the iteration-1
synthesis (0 blockers, 0 majors; quality scorecard 986/approve). The PR's own
fix is unchanged.

Contract fidelity, the one gate-impact-minor finding: FromGitURL is exported and
cmd/bd's gitURLToDoltRemote calls it directly, bypassing Normalize's
classification. Its first-colon split still rebuilt s3://bucket/team@prod/db
into git+ssh://s3///bucket/team@prod/db, the exact string this PR removes from
Normalize. That split now obeys the same "://" rule as isSCPStyleGitURL.

This takes the finding's second offered arm over its first. Gating on
isSCPStyleGitURL(url) instead changes 17 of 31 inputs across the package's own
corpus, against 9 for the "://" guard; the 8 extra are the SCP hosts that
predicate deliberately declines to classify - IDN hosts, non-ASCII userinfo and
dotless aliases - which FromGitURL converts correctly today.
TestFromGitURLStillConvertsSCPForms pins them, and mutating to the rejected arm
reds exactly those three rows while the new regression pin still passes.

NativeSchemes: the finding's "add az:// and oci:// for consistency" arm is
declined and the documentation arm taken instead. The PR body scopes this change
to s3:// and defers az:// to #6227 explicitly; #6227 is open, and it pairs
NativeSchemes with dolt.IsBackupURL - the site where the gap actually changes
behavior - so adding the entry here would have overridden a tracked contributor
decision while closing only half the bug. oci:// is in neither authority's
native list. The list's doc comment now records that narrowing, points at
remotecache.remoteSchemes as the fuller authority, and states that membership is
inert: every entry contains "://", so isSCPStyleGitURL rejects it and
Normalize's closing "return url" returns the same bytes listed or not. Dropping
gs:// reds one census leaf and no behavioral test, which measures that claim
rather than asserting it. The new FromGitURL table covers az:// and oci://
precisely because the guard is keyed on "://" and not on list membership.

Duplicate SCP grammar: scpStyleGitURLPattern's first alternative is
byte-identical to remotecache.gitSSHPattern, and the whole pattern is a superset
by exactly the user-less dotted-host form - the form on which the two already
disagree in production. Both regexes now cross-reference each other and the
divergence is pinned from each side. Exporting a shared matcher was the other
offered arm; it would have widened remotecache.IsRemoteURL's accept set, a
behavior change in a second package for a non-gating finding.

normalizeRemoteURL's doc comment restated Normalize's rules and had drifted
twice: it never gained s3://, and it named gitURLToDoltRemote as the converter
when the body calls doltremote.Normalize directly. Collapsed it to a reference
so the surface cannot drift again.

Codex's prescribed remedy for the hollow NativeSchemes census subtest is
declined, as the synthesis directs: asserting Normalize(scheme+"bucket/x") per
scheme passes against an empty list, for the same inertness reason above. The
subtest's comment, which claimed to protect a fast path it cannot observe, is
corrected instead.

go test ./internal/doltremote/ ./internal/remotecache/ green; cmd/bd named
selector 56 pass, 0 fail, 0 skip; go build ./..., go vet and gofmt clean; bazel
test passes on both packages; gazelle and go_srcs.py --check report no BUILD
drift.

---------

Co-authored-by: Julian Knutsen <julian@gastownhall.com>
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
2026-09-29 06:28:57 -07:00

147 lines
5.7 KiB
Go

package main
import (
"context"
"fmt"
"os"
"regexp"
"strings"
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/config"
"github.com/steveyegge/beads/internal/doltremote"
)
// resolveSyncRemote returns the effective sync remote URL.
// Resolution order:
// 1. sync.remote (primary — any Dolt-compatible remote URL)
// 2. sync.git-remote (deprecated fallback)
// 3. "" (not configured)
func resolveSyncRemote() string {
if v := config.GetString("sync.remote"); v != "" {
return v
}
return config.GetString("sync.git-remote")
}
// resolveSyncRemoteFromDir is like resolveSyncRemote but reads from a
// specific beads directory's config.yaml. Used by context_cmd, doctor,
// and other paths that operate on a resolved beads dir rather than CWD.
func resolveSyncRemoteFromDir(beadsDir string) string {
if v := config.GetStringFromDir(beadsDir, "sync.remote"); v != "" {
return v
}
return config.GetStringFromDir(beadsDir, "sync.git-remote")
}
// commitBeadsConfig stages .beads/config.yaml and commits it.
// Silently no-ops if the file is clean or the commit fails (e.g. hooks,
// nothing to commit). Used by bd dolt remote add/remove to keep the
// working tree clean after persisting sync.remote.
func commitBeadsConfig(msg string) {
commitBeadsConfigForActiveRepo(context.Background(), msg)
}
func commitBeadsConfigForActiveRepo(ctx context.Context, msg string) {
rc, err := beads.GetRepoContext()
if err != nil {
return
}
addCmd := rc.GitCmd(ctx, "add", ".beads/config.yaml")
if err := addCmd.Run(); err != nil {
return
}
commitCmd := rc.GitCmd(ctx, "commit", "-m", msg)
if out, err := commitCmd.CombinedOutput(); err != nil {
if !strings.Contains(string(out), "nothing to commit") {
fmt.Fprintf(os.Stderr, "Warning: failed to commit config change: %v\n", err)
}
}
}
// normalizeRemoteURL converts a remote URL to a Dolt-compatible format.
// See doltremote.Normalize for the scheme rules - this wrapper adds nothing and
// exists only so call sites in this package read against one local name.
// Restating those rules here drifts: this comment still omitted s3:// after the
// source gained it, and named gitURLToDoltRemote as the converter when the call
// below goes to doltremote.Normalize directly.
func normalizeRemoteURL(url string) string {
return doltremote.Normalize(url)
}
// doltRemoteURL returns the form of remote that bd hands to Dolt — for
// DOLT_CLONE, for DOLT_REMOTE('add', ...), and therefore for DOLT_PUSH.
//
// A git-forge URL must reach Dolt in its git+ form. Dolt's dbfactory routes by
// scheme: raw http(s):// goes to the remotesapi client, which speaks Dolt's
// wire protocol at github.com and retries indefinitely (#4421), while
// git+https/git+ssh goes to the git remote factory, which shells out to git
// and fails cleanly.
//
// Everything else is returned byte-identical. That preserves GH#3339: a
// user-configured Dolt remotesapi endpoint (http://myserver:7007/mydb) must
// never be rewritten to git+http://, and it never classifies as a forge URL.
//
// This is the single owner of that routing rule. bd init and bd bootstrap must
// both call it: if they disagree about the URL derived from one committed
// sync.remote, that is the #5743 class of skew all over again.
func doltRemoteURL(remote string) string {
if isGitCodeRepoURL(remote) {
return normalizeRemoteURL(remote)
}
return remote
}
// redactRemoteURL strips credentials from a remote URL so it is safe to echo
// into errors, hints, logs and JSON. CI commonly configures sync.remote as
// https://x-access-token:<token>@github.com/org/repo.git, and the clone funnel
// already scrubs userinfo before reporting (versioncontrolops.sanitizeURL);
// diagnostics must not be the hole in that convention.
//
// HTTP(S) userinfo is transport credentials and is dropped whole. SSH userinfo
// selects the remote account (git@host is not a secret and is needed for the
// hint to be runnable), so only a password component is dropped.
func redactRemoteURL(raw string) string {
sep := strings.Index(raw, "://")
if sep < 0 {
// scp-style (git@host:path) or a bare path: no place for a password.
return raw
}
scheme, rest := raw[:sep], raw[sep+3:]
authority, tail := rest, ""
if slash := strings.Index(rest, "/"); slash >= 0 {
authority, tail = rest[:slash], rest[slash:]
}
at := strings.LastIndex(authority, "@")
if at < 0 {
return raw
}
userinfo, host := authority[:at], authority[at+1:]
if isDoltSSHScheme(scheme) {
if colon := strings.Index(userinfo, ":"); colon >= 0 {
userinfo = userinfo[:colon]
}
return scheme + "://" + userinfo + "@" + host + tail
}
return scheme + "://" + host + tail
}
func isDoltSSHScheme(scheme string) bool {
return scheme == "ssh" || scheme == "git+ssh"
}
// urlWithUserinfoRe matches the "scheme://userinfo@host" span of a URL embedded
// anywhere in free-form text — for example the git stderr line bootstrap folds
// into a probe error, which can echo the very remote (credentials and all) that
// git failed to reach. The userinfo run stops at the first "@" and the host run
// at the first "/" so a trailing path is left for redactRemoteURL to keep.
var urlWithUserinfoRe = regexp.MustCompile(`[a-zA-Z][a-zA-Z0-9+.-]*://[^@\s'"]*@[^/\s'"]*`)
// scrubURLCredentials redacts credential-bearing URLs embedded anywhere in s by
// running each match through redactRemoteURL, so the same rule applied to the
// plan's own URL fields (http(s) drops userinfo, ssh drops only a password) also
// covers URLs that arrive inside arbitrary text. Text with no "userinfo@" URL is
// returned unchanged, so credential-free git diagnostics keep their full detail.
func scrubURLCredentials(s string) string {
return urlWithUserinfoRe.ReplaceAllStringFunc(s, redactRemoteURL)
}