beads/cmd/bd/init_policy_test.go
Julian Knutsen cba2037589
fix(bootstrap): probe git remotes for dolt data instead of rejecting; exit non-zero when bootstrap declines (#5743, #5663) (#6037)
* fix(bootstrap): probe git remotes for dolt data instead of rejecting them

`bd init` derives sync.remote from the git origin and persists it to the
git-tracked .beads/config.yaml — by design, because the first `bd dolt push`
creates refs/dolt/data on that same git remote. `bd bootstrap` then rejected
exactly that value: isGitCodeRepoURL classified any .git-suffixed or
known-forge URL as "not a Dolt remote" and returned Action=none *before* the
refs/dolt/data origin auto-detect that would have hydrated the clone. Every
fresh clone of such a repo got no database, and deleting the sync.remote line
by hand was the only way out (#5743, #5663).

The #4421 CPU storm this guard was added for is not a property of "the URL
points at a forge". Dolt's dbfactory routes by scheme: a raw http(s):// URL
goes to the remotesapi client, which speaks its wire protocol at github.com
and retries forever; a git+https/git+ssh URL goes to the git remote factory,
which shells out to git and fails cleanly. So the classifier is a good router
and a bad rejector.

Route on it instead: when sync.remote classifies as a forge URL, normalize it
to its git+ form and probe refs/dolt/data with the existing 10s-bounded
`git ls-remote` helper, then

  - data present: Action=sync from the git+ URL, never the raw form
  - definitively absent: note it and fall through to origin auto-detect,
    backup, issues.jsonl and fresh init, matching `bd init`'s own "remote has
    no Dolt data yet" behavior
  - probe error (UNKNOWN): Action=none with a new Blocked flag; never clone
    something we could not verify

Non-forge remotes (dolthub://, s3://, remotesapi http(s) endpoints,
self-hosted git+ssh) keep the trusted-as-is path untouched, so GH#3339 still
holds: normalization now happens only inside the forge branch, where a
remotesapi endpoint cannot appear.

Blocked/BlockedRemote are additive JSON fields; the exit-code half of the fix
follows in the next commit.

Refs #5743, #5663

* fix(bootstrap): exit non-zero when bootstrap declines to set up a database

Action=="none" was two different outcomes wearing one name: "a database
already exists, nothing to do" and "bootstrap refused and left you with no
database". printBootstrapPlan printed "✓ Database already exists" for both,
and RunE returned nil for both — so a rejected sync.remote reported success,
exited 0, and left the user with `bd list` saying "no beads database found"
(#5743, #5663).

The split was already encoded in the plan: every genuine no-op sets
HasExisting. Use it.

  - printBootstrapPlan reserves the success tick for HasExisting plans and
    otherwise reports the reason on stderr. Blocked plans additionally get a
    runnable `git ls-remote <url> refs/dolt/data` hint (the git+ prefix is
    stripped, as the probe itself strips it) and a pointer at the sync.remote
    line in config.yaml.
  - bootstrapPlanOutcome maps the plan to an exit status in one place, so
    text, --json and --dry-run all agree. JSON mode returns SilentExit, since
    the plan (carrying "blocked": true) is already on stdout.
  - The "Could not verify existing server database" branch is marked Blocked
    for the same reason: it also declined without confirming a database, and
    it also false-ticked.

Refs #5743, #5663

* fix(init): normalize git-forge sync.remote URLs before cloning

Init never had bootstrap's forge guard: a configured
`sync.remote: https://github.com/org/repo.git` was passed to DOLT_CLONE
verbatim, which Dolt routes to the remotesapi client — the same retry storm
(#4421) the bootstrap guard exists to prevent. Route it the same way
bootstrap now does: forge URLs are converted to their git+ form, which
selects the git remote factory, and everything else is left byte-identical so
a user-configured remotesapi endpoint (http://myserver:7007/mydb) is never
rewritten (GH#3339).

Also documents, at the two places someone would go looking after a bootstrap
failure, why init deliberately persists a plain git origin as sync.remote:
bootstrap probes refs/dolt/data to resolve it (#5743), so un-persisting here
would be the wrong fix and would strand every already-committed
.beads/config.yaml.

This commit is independent of the bootstrap fix and can be dropped from a
release cherry-pick if the minimum diff is wanted.

Refs #5743, #5663, #4421

* fix(sync): thread git's stderr into the refs/dolt/data probe error

The probe wrapped only the exec error, and exec.ExitError.Error() is just
"exit status 128" — git's actual complaint sits unread in ExitError.Stderr,
which cmd.Output() had already captured. A context timeout was worse still:
"signal: killed". Since bootstrap's blocked message is built from this error,
the fix's primary failure path told users nothing that distinguished bad
credentials from DNS from a deleted repository.

Now the error carries git's own first stderr line ("Permission denied
(publickey)", "Repository not found", "Invalid username or token") and names a
timeout as a timeout.

Also drops the remote and ref name from the wrapper. Every caller already
names the remote it probed, so wrapping here too produced messages that said
both twice: `... on sync.remote "X": probe refs/dolt/data on X: exit status
128`.

Refs #5743

* fix(bootstrap): derive the blocked verdict once; keep local recovery reachable

Review round on #6037. Every change below fixes a confirmed defect in this
PR's own new code.

State leak (the #5743 bug, reintroduced). existingBootstrapDBPlan's "none"
verdict was copied into the working plan, so every later branch inherited its
HasExisting/Blocked/Reason. A probe error on top of a seeded HasExisting=true
produced action=none + blocked=true + has_existing=true, which printed the
success tick and exited 0; a seeded server-verify error leaked blocked=true
into a *successful* sync plan's JSON, violating the field's own contract that
nothing clones from a blocked plan. The verdict is now held aside as an
explicit fallback instead of merged in, and Blocked is derived in one place
from the finished plan rather than hand-set by any branch.

Local recovery. A probe error used to return immediately, so an offline laptop
or credential-less CI runner was denied the backup/JSONL restore that the same
machine would have got if the probe had merely answered "no data". Restoring a
local backup touches no remote: the fail-closed rule now applies only to
remote actions, and the block is emitted at the tail, after every local source
has been tried.

Origin auto-detect is skipped once sync.remote is configured. sync.remote is
the authoritative remote, so hydrating from whatever git origin happens to be
— a fork, an old experiment, the code repo under the dedicated-data-repo
pattern — is not recovery, and finalizeSyncedBootstrap would then have
rewritten the team's committed sync.remote to point at it. This also removes
the second identical ls-remote in the flagship pre-first-push case (init
persists the git origin as sync.remote, so the two legs were probing the same
repository for the same ref), and makes the fall-through tests hermetic: the
origin leg is not behind the injectable probe seam.

Remote wiring. The fall-through made executeInitAction reachable with a
configured sync.remote for the first time, and it never registered the remote
as Dolt "origin". The promise that "the first bd dolt push creates
refs/dolt/data on that same git remote" then breaks: push either refuses to
adopt a remote without consent or adopts the git origin — a different
repository — and rewrites the committed sync.remote to match. Two machines
bootstrapping in the pre-push window would mint independent Dolt identities
and diverge. The provisioning actions now wire the routed remote, as the sync
action and bd init already did. bd init gets the same routing on its own
wiring call: its no-Dolt-data-yet path never clones, so a raw https forge URL
was stored verbatim in dolt_remotes and the *first* push took it to the
remotesapi client — #4421, one leg further down.

--dry-run always exits 0. bd doctor documents `bd bootstrap --dry-run` as the
safe inspection step, and a preview that aborts a set -e provisioning script
is a worse contract than a preview that merely reports. The blocked state is
still printed and still in the JSON, so the preview does predict what a real
run would refuse to do.

Credentials. sync.remote is commonly an https URL with an embedded CI token.
The reason, the hint and the blocked_remote JSON field all echoed it verbatim
into logs and agent transcripts; so did the sync plan's "Remote:" line. All
display paths now go through redactRemoteURL, which drops http(s) userinfo
whole and keeps only the account selector for ssh so the hint stays runnable.

Hints. The old remediation could name a config file that does not hold the key
(resolveSyncRemote reads a merged scope and a deprecated alias), and following
it was actively harmful: deleting sync.remote and re-running hits the same
auth failure on the git-origin probe, which swallows its error, so bootstrap
would "succeed" by creating a fresh database diverged from the team's history.
The hints now reproduce the probe, point at access, and offer `bd dolt remote
add` — never deletion.

One owner for the routing rule. doltRemoteURL (sync_remote.go) is now the
single definition of "classify, then normalize forge URLs only". It replaces
initCloneURL and the copy this PR had inlined into detectBootstrapAction: if
bd init and bd bootstrap ever disagree about the URL derived from one
committed sync.remote, that is the #5743 class of skew all over again.

Tests: blocked-with-HasExisting and blocked-leaking-into-sync (neither was
covered), probe-error-still-restores/imports, credential scrubbing on both the
plan and the printed output, dry-run exit status, the real probe's stderr
thread-through, and redactRemoteURL itself. The probe-error fixture now uses
the error shape production actually produces instead of a fabricated one, the
printer tests use the shared mutexed capture helpers, and the "#4421 pin"
comment is rescoped: it pins the routing, not the classifier, which still has
an unlisted-forge gap tracked as a follow-up.

Refs #5743, #5663, #4421

* fix(bootstrap): redact --json credentials, scrub git stderr, cut detect complexity

Maintainer fixup addressing PR #6037 review findings.

- Add BootstrapPlan.MarshalJSON so `bd bootstrap --json` (and
  `--dry-run --json`) redacts the credential-bearing sync_remote on a copy,
  matching the human print path. Closes the deterministic token leak to stdout
  while leaving the raw plan executeBootstrapPlan clones from untouched.
- Scrub credential-bearing URLs from the folded `git ls-remote` stderr at the
  source (scrubURLCredentials in gitLsRemoteProbeError) so a tokenized URL git
  echoes cannot reach plan.Reason, which flows to both stderr and JSON.
- Reduce detectBootstrapPlan cyclomatic 22->12 and cognitive 27->13 by
  extracting planConfiguredSyncRemote, gitOriginSyncPlan, and
  localFileRecoveryPlan. Detection order and plan-mutation semantics are
  preserved (a forge remote with no Dolt data still records SyncRemote so a
  local-recovery workspace stays wired to it).
- Drop the dead `plan.Blocked = true` in existingBootstrapDBPlan; the wrapper
  derives Blocked once from Action=="none" && !HasExisting.

Tests: assert the token is absent from the plan JSON (and MarshalJSON does not
mutate the source plan), and that tokenized git stderr is scrubbed while
credential-free diagnostics pass through unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 18:23:57 -07:00

252 lines
8.8 KiB
Go

package main
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
"github.com/steveyegge/beads/internal/config"
)
func TestRunInitRemoteClone(t *testing.T) {
t.Run("passes explicit HTTP URL through unchanged", func(t *testing.T) {
const remoteURL = "http://127.0.0.1:1/no-such-db"
var gotURL string
resolvedURL, source := resolveInitConfiguredSyncRemote(remoteURL, true, func() string {
t.Fatal("explicit remote must not resolve a configured fallback")
return ""
})
if resolvedURL != remoteURL {
t.Fatalf("resolved URL = %q, want %q", resolvedURL, remoteURL)
}
if source != initSyncRemoteExplicit {
t.Fatalf("source = %v, want explicit", source)
}
disposition, err := runInitRemoteClone(resolvedURL, func(url string) error {
gotURL = url
return nil
})
if err != nil {
t.Fatalf("runInitRemoteClone: %v", err)
}
if gotURL != remoteURL {
t.Fatalf("clone URL = %q, want %q", gotURL, remoteURL)
}
if disposition != initRemoteCloneBootstrapped {
t.Fatalf("disposition = %v, want bootstrapped", disposition)
}
})
t.Run("empty remote selects fresh initialization", func(t *testing.T) {
const remoteURL = "file:///empty"
resolvedURL, source := resolveInitConfiguredSyncRemote(remoteURL, true, func() string {
t.Fatal("explicit remote must not resolve a configured fallback")
return ""
})
if resolvedURL != remoteURL || source != initSyncRemoteExplicit {
t.Fatalf("resolved remote = (%q, %v), want (%q, explicit)", resolvedURL, source, remoteURL)
}
syncURLFromConfig := resolvedURL != "" && source != initSyncRemoteNone
called := false
disposition, err := runInitRemoteClone(resolvedURL, func(string) error {
called = true
return errors.New("remote contains no Dolt data")
})
if err != nil {
t.Fatalf("runInitRemoteClone: %v", err)
}
if !called {
t.Fatal("clone callback was not called")
}
if disposition != initRemoteCloneFresh {
t.Fatalf("disposition = %v, want fresh", disposition)
}
// The production call site clears syncFromRemote after this fresh
// fallback, but the explicit source remains a durable remote-wiring
// intent for both the Dolt remote and config.yaml.
if !shouldWireInitRemote(resolvedURL, false, syncURLFromConfig, false) {
t.Fatal("fresh fallback lost explicit remote wiring")
}
if !shouldWriteInitDoltRemote(false, resolvedURL, false, syncURLFromConfig, false, false) {
t.Fatal("fresh fallback lost explicit Dolt remote wiring")
}
beadsDir := filepath.Join(t.TempDir(), ".beads")
if err := os.MkdirAll(beadsDir, 0o750); err != nil {
t.Fatalf("mkdir beads dir: %v", err)
}
if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte("# Beads Config\n"), 0o600); err != nil {
t.Fatalf("write config.yaml: %v", err)
}
if err := persistInitSyncRemote(beadsDir, remoteURL, resolvedURL, false, syncURLFromConfig, false); err != nil {
t.Fatalf("persistInitSyncRemote: %v", err)
}
configBytes, err := os.ReadFile(filepath.Join(beadsDir, "config.yaml"))
if err != nil {
t.Fatalf("read config.yaml: %v", err)
}
if !strings.Contains(string(configBytes), remoteURL) {
t.Fatalf("config.yaml does not contain explicit remote %q:\n%s", remoteURL, configBytes)
}
})
t.Run("preserves non-empty clone failure", func(t *testing.T) {
want := errors.New("permission denied")
disposition, err := runInitRemoteClone("https://example.invalid/beads", func(string) error {
return want
})
if !errors.Is(err, want) {
t.Fatalf("error = %v, want %v", err, want)
}
if disposition != initRemoteCloneFailed {
t.Fatalf("disposition = %v, want failed", disposition)
}
})
}
func TestDetectInitRemoteHostConflict(t *testing.T) {
for _, tt := range []struct {
name string
configHost, envHost string
configPort, envPort string
want *initRemoteHostConflict
}{
{name: "port only", configPort: "3306"},
{
name: "remote config host",
configHost: "100.111.197.110",
want: &initRemoteHostConflict{host: "100.111.197.110", source: "config.yaml"},
},
{name: "localhost environment overrides remote config", configHost: "100.111.197.110", envHost: "localhost"},
{name: "loopback environment overrides remote config", configHost: "100.111.197.110", envHost: "127.0.0.1"},
{
name: "remote environment takes precedence",
configHost: "127.0.0.1",
envHost: "100.111.197.110",
envPort: "3307",
want: &initRemoteHostConflict{host: "100.111.197.110", source: "environment", includesPort: true},
},
} {
t.Run(tt.name, func(t *testing.T) {
got := detectInitRemoteHostConflict(tt.configHost, tt.envHost, tt.configPort, tt.envPort)
if got == nil || tt.want == nil {
if got != tt.want {
t.Fatalf("detectInitRemoteHostConflict() = %#v, want %#v", got, tt.want)
}
return
}
if *got != *tt.want {
t.Fatalf("detectInitRemoteHostConflict() = %#v, want %#v", got, tt.want)
}
})
}
}
func TestConfiguredImportJSONLPath(t *testing.T) {
t.Setenv("BEADS_TEST_IGNORE_REPO_CONFIG", "1")
config.ResetForTesting()
if err := config.Initialize(); err != nil {
t.Fatalf("config.Initialize: %v", err)
}
t.Cleanup(config.ResetForTesting)
const beadsDir = "/workspace/.beads"
for _, tt := range []struct {
name string
path string
want string
}{
{name: "default", want: filepath.Join(beadsDir, "issues.jsonl")},
{name: "configured", path: "beads.jsonl", want: filepath.Join(beadsDir, "beads.jsonl")},
} {
t.Run(tt.name, func(t *testing.T) {
config.Set("import.path", tt.path)
if got := configuredImportJSONLPath(beadsDir); got != tt.want {
t.Errorf("configuredImportJSONLPath(%q) = %q, want %q", beadsDir, got, tt.want)
}
})
}
}
func TestNormalizeIssuePrefixAndDatabaseName(t *testing.T) {
for _, tt := range []struct {
name string
raw string
want string
}{
{name: "dotted directory", raw: "MyPkg.jl", want: "MyPkg_jl"},
{name: "leading dot", raw: ".claude", want: "claude"},
{name: "trailing hyphen", raw: "project-", want: "project"},
{name: "numeric start", raw: "001", want: "bd_001"},
{name: "database hyphen", raw: "my-prefix", want: "my_prefix"},
} {
t.Run(tt.name, func(t *testing.T) {
got := dbNameFromPrefix(normalizeIssuePrefix(tt.raw))
if got != tt.want {
t.Errorf("dbNameFromPrefix(normalizeIssuePrefix(%q)) = %q, want %q", tt.raw, got, tt.want)
}
})
}
}
// TestDoltRemoteURL pins which URLs bd rewrites before handing them to Dolt —
// for DOLT_CLONE, for DOLT_REMOTE('add', ...) and therefore for DOLT_PUSH.
// Forge URLs must arrive in git+ form so Dolt uses the git remote factory
// rather than the remotesapi retry storm (#4421); a user-configured remotesapi
// endpoint must arrive byte-identical (GH#3339). bd init and bd bootstrap
// share this one helper so they can never disagree about the URL derived from
// a single committed sync.remote.
func TestDoltRemoteURL(t *testing.T) {
tests := []struct {
name string
in string
want string
}{
{"raw https forge", "https://github.com/org/repo.git", "git+https://github.com/org/repo.git"},
{"forge without .git", "https://github.com/org/repo", "git+https://github.com/org/repo"},
{"scp style forge", "git@github.com:org/repo.git", "git+ssh://git@github.com/org/repo.git"},
{"ssh forge", "ssh://git@gitlab.com/org/repo.git", "git+ssh://git@gitlab.com/org/repo.git"},
{"already normalized", "git+https://github.com/org/repo.git", "git+https://github.com/org/repo.git"},
{"remotesapi endpoint", "http://myserver:7007/mydb", "http://myserver:7007/mydb"},
{"remotesapi https endpoint", "https://doltremoteapi.dolthub.com/org/db", "https://doltremoteapi.dolthub.com/org/db"},
{"dolthub native", "dolthub://myorg/mydb", "dolthub://myorg/mydb"},
{"s3 native", "s3://bucket/path", "s3://bucket/path"},
{"self-hosted dolt over ssh", "git+ssh://my-dolt.example.com/org/db", "git+ssh://my-dolt.example.com/org/db"},
{"empty", "", ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := doltRemoteURL(tt.in); got != tt.want {
t.Errorf("doltRemoteURL(%q) = %q, want %q", tt.in, got, tt.want)
}
})
}
}
// TestRunInitRemoteCloneReceivesRoutedURL proves the routing survives the call
// into runInitRemoteClone — the seam the init RunE body actually uses.
func TestRunInitRemoteCloneReceivesRoutedURL(t *testing.T) {
for _, tt := range []struct{ in, want string }{
{"https://github.com/org/repo.git", "git+https://github.com/org/repo.git"},
{"http://myserver:7007/mydb", "http://myserver:7007/mydb"},
} {
t.Run(tt.in, func(t *testing.T) {
var gotURL string
if _, err := runInitRemoteClone(doltRemoteURL(tt.in), func(url string) error {
gotURL = url
return nil
}); err != nil {
t.Fatalf("runInitRemoteClone: %v", err)
}
if gotURL != tt.want {
t.Errorf("clone received %q, want %q", gotURL, tt.want)
}
})
}
}