feat(backend/http): public HTTP backend door, OSS registration, and bd connect (S6) (#7288)

* feat(backend/http): add the public HTTP backend door, cmd/bd registration, and bd connect

internal/httpclient (S1-S5) had no production caller for the per-open
backends.OpenOptions.Credential seam and no way for a workspace to actually
select it. This slice (S6) closes both gaps in OSS's own idiom, without
carrying over anything from bd-enterprise's gateway/box-login/storage-profile
machinery:

- backend/http (bdhttp) is the public, out-of-tree extension door: Register
  adds "http" to the backend registry and installs its transport, Open/
  Handshake dial a server directly with no workspace on disk, and the
  registered OpenWith hook threads a per-call credential through
  beads.OpenBestAvailableWith for a multi-tenant embedder.
- internal/httpclient/open_with.go is the backends.Backend.OpenWith hook
  itself: a fresh dial per open (opts.Credential/HTTPClient/UserAgent),
  distinct from the process-wide registered dialer Open/OpenReadOnly use.
- cmd/bd/http_backend.go registers the backend for the OSS binary, so a
  workspace whose metadata.json names "http" opens through the ordinary
  OpenBestAvailable path; any other unregistered backend name still refuses.
- cmd/bd/connect.go adds the minimal `bd connect <url>` command: a handshake
  that must succeed before anything is written, a per-user activation
  sidecar (.beads/http_target.json, never git-tracked), a plaintext-to-
  non-loopback refusal without --allow-plaintext, and a backend-switch
  refusal without --force. No credential is ever accepted on the command
  line or written to disk.
- cmd/bd/doctor/gitignore.go requires the new http_target.json /
  http_local_metadata.json sidecars be git-ignored, the same way the
  proxied-server client info already is.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(storage): keep external-dependency blocking over http unless the server enforces it

bd ready / bd close / claim hard-failed against the new http backend with
"GetAllDependencyRecords not supported by the http backend": cmd/bd's
storage chain wraps every opened store in the externaldeps policy
decorator, which needs either the narrow ExternalDependencyQueryStore
interface or the broad GetAllDependencyRecords fallback to resolve
blocking state, and the http client implemented neither.

The decorator now stays on for a remote store. It is skipped only when the
server's handshake advertises policy.external_dependencies (the server
enforces the policy before answering). Without that capability the client
resolves blocking itself from the listing and chunked dependency reads, so
an issue with an unresolvable external: dependency stays out of bd ready
exactly as it does on a local backend; it is never silently skipped
(DESIGN 3.6). Server-side enforcement is slice S10.

storage.RemoteBackendStore marks a DoltStorage that is a pure network
client of a remote bd serve; the http backend's Store satisfies it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(cmd/bd): suppress the CreatedBy stamp on create against a remote backend

bd create failed against the http backend with "Issue.CreatedBy: ... is not
expressible on the v0 wire": the http client's create role deliberately
refuses any POPULATED CreatedBy/CreatedAt (the creation stamp belongs to the
server's own journal entry, not a value the caller supplies), but
cmd/bd/create.go unconditionally stamped getActorWithGit() onto every
created issue regardless of backend. The refusal only fires on a non-zero
field, so suppressing the stamp for a registered remote backend is wire-
compatible by construction, not a workaround.

Add isRemoteBackendWorkspace()/creationActorStamp() to backend_support.go,
reusing the existing resolveCommandBeadsDir(dbPath) +
backendNameForErrorFraming(beadsDir) + backends.IsRemote(name) pattern
already used elsewhere in that file, and use creationActorStamp() at both
create.go call sites (the dry-run preview and the real create).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(backend/http): add an embedder linkage test over public packages only

Prove the one promise S6 makes to an out-of-tree embedder (gc is the
motivating case): importing nothing but github.com/steveyegge/beads,
.../backend, and .../backend/http is enough to register the http backend,
open a workspace through it with a per-call credential
(beads.OpenBestAvailableWith), and read through the role-accessor surface
(IssueReader().Ready) against a stand-in server — with no internal/...
import in the test file, which is itself the thing under test.

Written fresh for OSS beads S6; no bd-enterprise source copied.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: document the HTTP backend, bd connect, and the public backend/http door

Add a README section with bd connect usage, the credential ladder, and the
plaintext/force refusals, plus a CHANGELOG entry for the new cmd/bd
registration, bd connect command, and the public backend/http package.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(httpclient): address bee-ghosttrack review on #7288 (connect --clear restore, persisted --allow-plaintext)

Closes the CHANGES_REQUESTED items on the HTTP backend door PR:

- Must-fix: `bd connect --force` over a workspace with its own local
  (dolt) identity now has writes succeed instead of tripping "workspace
  identity mismatch detected" — validateWorkspaceIdentity (cmd/bd/main.go)
  and the sidecar (internal/httpclient/metadata.go) now compare against
  the http backend's own pinned project id. New e2e regression test
  TestE2E_ForceConnectWritesSucceedDespiteDifferingLocalProjectID
  (backend/http/e2e_cli_test.go), confirmed to fail without the fix.

- Should-fix 1: `bd connect --clear` now restores metadata.json's
  backend selection to whatever it was before this workspace connected,
  instead of leaving it pinned to "http" with no sidecar to dial.
  Target.PreviousBackend (internal/httpclient/target.go) is recorded by
  Attach (backend/http/attach.go) and round-tripped through the sidecar;
  connect.go reads it back on --clear. New tests in
  backend/http/attach_test.go and internal/httpclient/target_test.go.

- Should-fix 2: `bd connect --allow-plaintext` is now remembered in the
  sidecar (Target.AllowInsecureCredential), scoped to the target it was
  granted for, so every later ordinary command's dial honors the grant
  without also needing BEADS_HTTP_ALLOW_INSECURE=1 set
  (internal/httpclient/dial.go). Fixed BearerProvider.warnInsecure's
  wording (internal/httpclient/credential.go), which used to assert a
  token was being sent immediately above a refusal saying the opposite.
  New test TestDialWithWiresTargetAllowInsecureCredentialThroughToTheGuard.

- Should-fix 3: README no longer claims HTTP mode works "exactly as
  with a local database" — it lists what round-trips and points at the
  divergence ledger (engdocs/design/http-divergence-ledger.md) for what
  differs.

- Verified, no code change needed: Attach already handles an existing
  metadata.json project_id coherently (only cfg.Backend is touched); bd
  show's isNotFoundErr already matches a http 404 (issueops.ErrNotFound
  is storage.ErrNotFound, the same sentinel a local miss uses), so it
  reads as "not found" rather than an unsupported-operation error.

- Doc fix: guardInsecureCredential's doc comment and refusal message
  were stale against should-fix 2's own change — they named only
  DialOptions.AllowInsecureCredential and the env var as ways to get
  past the guard, not the newly-persisted target.AllowInsecureCredential
  or `bd connect --allow-plaintext` itself.

Gates run clean on this branch: go build/vet, go test over
internal/httpclient, backend/http (including the full e2e suite under
BEADS_TEST_EMBEDDED_DOLT=1) and cmd/bd, make api-check, make
ci-pr-lint, make bazel-sync-check, bazel run //:gazelle -- -mode=diff,
and internal/storage's TestEveryLegWiresEveryRoleContract.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(backend/http): give the e2e server the BatchGetter role #7248 added to the required set

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: exact http ready totals, connect capability row, doc corrections

Ready totals over http (review major). externaldeps drops externally
blocked rows client-side for a store that cannot express ExcludeIDs
(today httpclient.Store). dropBlockedIssuesWithCounts compacted the
fetched rows in place (`kept := rows[:0]`), and
readyWorkWithCountsAndTotal then re-scanned `rows` to count the blocked
rows it had dropped. By then the compaction had overwritten them, so
whenever a blocked row came back ahead of a kept one the re-scan
undercounted the drops: `bd ready --json`'s total and text-mode `bd
ready`'s count (CountReadyWork) overstated ready work against any server
that does not advertise policy.external_dependencies. Both drop helpers
now copy into a fresh slice, and dropBlockedIssuesWithCounts reports how
many rows it dropped, so nothing re-scans `rows`.

fakeStore gains an excludeIDsUnsupported switch that, like
httpclient.Store, refuses any ExcludeIDs. The new
TestReadyTotalsWhenStoreCannotExpressExcludeIDs puts the blocked issue
first and checks the page and total for an unlimited filter, a Limit
whose bumped window holds the whole set, and one the set outgrows, then
CountReadyWork. TestServedReadyExcludesUnsatisfiedExternalDependencyWithoutServerCapability
now also checks the total and count against a real in-process bd serve.
Against the previous decorator the unit test's unlimited case reports
total and count 3 beside a 2-row page, and the served test 2 beside a
1-row page.

Review minors, all wording:
- `bd connect --clear` pointed at `bd config set backend dolt`, which
  writes the database's config table rather than metadata.json and, in
  the state that prints the advice (metadata.json says http, sidecar
  gone), fails the pre-open identity check. It now says to reconnect, or
  to set "backend" in metadata.json.
- CHANGELOG described the superseded skip-at-wrap design; it now
  describes the always-wrap decorator with its capability probe and
  client-side drop.
- RemoteBackendStore's docs no longer name a consumer; nothing in-tree
  checks it.
- creationActorStamp's comment pointed at backend/http/lifecycle.go; the
  refusal lives in internal/httpclient/lifecycle.go.

Capability row. TestProxyCapabilityRegistryCoversCommandTree fails at
60b7128204: the command tree walk finds `connect`, which this PR adds,
with no row in cmd/bd/capability_registry.go ("1 command(s) have no
proxied-server capability row: connect"). The test predates the PR
(#6582 is an ancestor of its base) and passes at the base dabf20ef07.

connect opts out of store init (skipStoreAnnotation) and never reaches
the store factory, so it belongs in proxyPermittedPaths beside init and
setup. The row is runtime-neutral: validateProxyRegistryBeforeProvider
returns nil both for a permitted row and for a path with no row, so
this only records the policy decision the coverage test asks for.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(httpclient): regenerate the divergence ledger doc after the S5 rebase

The rebase onto feat/http-client-batch-apply's rebased head (S3/S4 merged to
main) left the ledger doc's summary line at a conflict-resolution placeholder
count. Regenerate it from internal/httpclient/encode.Ledger() so
TestDivergenceLedgerDocMatchesLedger passes against the merged ledger.go.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(backend/http): host-scope S6's test BEADS_HTTP_TOKEN values

Rebasing onto feat/http-client-batch-apply's rebased head pulled in main's
#7247 review-fix round, which made BEADS_HTTP_TOKEN/BEADS_HTTP_TOKEN_COMMAND
host-scoped (host[:port]=value, scopedEnvValue/parseScopedEnv in
internal/httpclient/credential.go) after S6's own tests were written against
the unscoped form. Five S6 tests across backend/http/open_test.go,
backend/http/register_test.go and
internal/httpclient/insecure_credential_guard_test.go set a bare token and
now hit the new ladder's "an unscoped token would be used for whatever
server..." refusal. Scope each to the test server's actual host
(127.0.0.1 for the httptest.Server-backed recording servers,
nonLoopbackTestHost for the insecure-credential-guard integration tests)
so they exercise the same ladder rung as before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): resync go_srcs/repo_files for the new backend/http and test/embedder packages

make bazel-sync-check failed after the rebase: tools/bazel/go_srcs.py's
managed repo_files filegroups for backend/http and test/embedder (new in
this slice) hadn't been generated, and the root BUILD.bazel / tools/bazel's
release_cross_build target list hadn't picked up the new package either.
Regenerate with `python3 tools/bazel/go_srcs.py`; `bazel run //:gazelle --
-mode=diff` reports no further drift.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(cmd/bd): route swarm's create paths through creationActorStamp after main's currentActor change

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: police http's served roles, refuse close --claim-next, exact totals

External-dependency policy over http (review blocker). cmd/bd wraps
every store, http included, in the externaldeps decorator, and its role
accessors rebuilt each role over legacy methods httpclient.Store refuses
(SearchIssuesWithCounts, GetDependencyTree, ClaimReadyIssue, IsBlocked,
GetDependencyRecordsForIssues, BatchCloserWithPolicy). Over http `bd
list`, `bd dep tree`, `bd ready --claim` and a policed `bd close`
therefore exited 1, even in a workspace with no external dependency.
Over a store that reports storage.RemoteBackendStore, each accessor now
starts from the inner store's served role and layers the policy around
it (remote_roles.go):

- IssueReader serves List and Get from the served reader; Ready and a
  ready-flagged List go through the policy reader.
- IssueClaimer refuses an externally blocked claim before the served
  one.
- ReadyClaimer uses the served claim-next while no issue, closed ones
  included, holds an unsatisfied external ref. Otherwise it claims down
  the policy's filtered ready page through the served Claimer, because
  claimNext cannot exclude ids.
- BlockingAnnotator and TreeWalker merge external blockers into, or add
  external leaves to, the served answer. They read only the named
  issues' edges through the served EdgeReader, 100 anchors per call.
- BatchCloser refuses externally blocked open items and sends the rest
  to the served closer as one request. --force keeps the base path, and
  a next claim is refused whole (below).

RemoteBackendStore now only chooses this composition. Whether the policy
runs is still ServerEnforcesExternalDependencyPolicy's answer (design
3.6), and the storage and httpclient docs say so. remote_roles_test.go
drives each role over a store that refuses the legacy methods.
TestServedRolesKeepTheExternalDependencyPolicy runs every role over a
real http server. TestE2E_BDOverHTTP adds `bd list` (JSON, text and
--deps), `bd dep tree`, `bd ready`, `bd list --ready`, `bd ready
--claim`, a refused `bd close --claim-next`, and a refused then forced
close of an externally blocked issue. At the fixup before the
composition, the role tests then written (bar one pin of the next-claim
refusal, since replaced) and the e2e's list, dep tree, claim and close
steps of that round were red.

Close with a next claim over http (review major). batchCloseIssues has
no member for a next claim, so httpclient refuses any ClaimNext before
the dial (ledger W-CloseBatchRequest.ClaimNext). Over http `bd close
--claim-next` therefore always exited 1 and closed nothing. With an
external ref anywhere in the workspace (a closed holder or --force
included) the decorator's base path failed first, with an opaque
`BatchCloserWithPolicy` unsupported. remoteBatchCloser now refuses a
next claim whole, before it reads or closes anything, and the error
says nothing closed: close without --claim-next, then run `bd ready
--claim`. A request that is invalid on every backend is still
ErrValidation first, and a server that enforces the policy itself gets
the request unchanged.

Composing close-then-claim was not taken. The role closes and claims in
one transaction and answers with an error or with outcomes, never both,
so a claim that failed after the closes committed could not report
them; httpclient refuses the same request for that reason. A CLI-level
close then claim that can report a partial result is the follow-up.
The review also suggested leaving closed holders out of the blocking
state. They stay in, as on every backend: a closed issue is never a
claim candidate, so they change no claim, and leaving them out would
cost a status read per holder because the wire has no batch read.

TestRemoteBatchCloserRefusesANextClaim covers an open holder, a closed
one, none, --force, and a server that enforces the policy; with the
previous remote_roles.go the first four are red.
TestRemoteBatchCloserValidatesANextClaimFirst keeps ErrValidation ahead
of the refusal. Dropping the filter check, or loading the blocking
state before refusing, is each caught. TestE2E_BDOverHTTP runs `bd
close --claim-next` while the holder is open and again after it is
closed; with the previous remote_roles.go both steps are red and print
`operation "BatchCloserWithPolicy" not supported`. Ledger F-close no
longer says --claim-next is served, and no longer names a
renderEscapedHTTPRefusal that does not exist. L14 says the decorator's
claim leg applies while any issue, closed or not, holds an unsatisfied
ref. The README and CHANGELOG mention the refusal, and the decorator's
IsBlocked, IsBlockedBatch and CloseIssueChecked say they refuse over a
store that serves its roles.

Ready totals with external blockers (review majors, both rounds).
CountReadyWork asks for Limit=0, and readyWorkWithCountsAndTotal took
any Limit<=0 to mean "the window held the whole ready set". Over http a
zero limit never reaches the wire: the server answers with its default
page (workapi.DefaultReadyLimit, 100 rows) while the bridge reports the
full total, so past 100 ready issues text-mode `bd ready` kept counting
externally blocked issues beyond the first page. Exactness is now judged
from the rows that came back (total <= len(rows)). A window that holds
only part of the set no longer subtracts every externally blocked id in
the workspace either: that set includes closed issues and ones the
filter excludes, none of them in the total, so the count came out low.
It refetches the filtered ready set, ids only, and subtracts the blocked
rows in it; the old bound is left as the fallback for a refetch that
comes back truncated too.

fakeStore gains the bridge's shape: a defaultPage for a zero Limit,
with GetReadyWorkWithCountsAndTotal still totalling the whole set.
TestReadyTotalsWhenAnUnlimitedFetchComesBackAsOnePage (a blocked issue
past the first page) and
TestReadyTotalsIgnoreExternallyBlockedIssuesOutsideTheReadySet (a
blocked issue outside the ready set) have served counterparts,
TestServedReadyCountExcludesAnExternallyBlockedIssuePastTheDefaultPage
and TestServedReadyTotalIgnoresAnExternallyBlockedIssueOutsideTheFilter
(an assignee filter). The first pair is red on the Limit<=0
condition, the second on the subtract-every-blocked-id branch.

Docs: ledger L8 now says which decorator paths take the chunked
GetAllDependencyRecords scan and which read only named edges, and L14
records the decorator's composed claim leg; http-divergence-ledger.md
is regenerated. The README names the verified commands, and the README
and CHANGELOG say how cross-repo dependencies behave over http.

Minors from the reviews:

- Drop creationActorStamp/isRemoteBackendWorkspace and restore the base
  stamps at all 11 sites. The arm guarded a refusal the http create
  role does not make: since #7247 (actorStampedCreateMember) it accepts
  a CreatedBy naming the actor. Because the arm keyed on the CWD
  workspace, a create routed from an http workspace into a local one
  (--repo) stored an empty created_by. TestE2E_BDOverHTTP checks
  created_by for a direct and a routed create; the routed check is red
  with the arm restored. The CHANGELOG sentence describing the arm goes
  with it.
- bdhttp.Attach decides a reconnect from the raw metadata.json backend.
  GetBackend() reads an unregistered "http" as the dolt default, so in
  an embedder that never called Register a reconnect recorded dolt as
  the backend --clear restores. The reconnect test now seeds postgres
  (TestAttachOnAReconnectKeepsTheEarlierPreviousBackend) and is red
  with the old check.
- README, CHANGELOG and `bd connect --help` document the host-scoped
  BEADS_HTTP_TOKEN=host[:port]=<token> form (and _COMMAND), and say why
  a bare value is refused.
- bdhttp.Attach also gitignores http_local_metadata.json, the per-user
  file the store writes on first use, so an embedder activating through
  Attach alone gets the coverage `bd connect` already had.
  TestAttachGitignoresBothPerUserFiles pins it.
- ServerSnapshot.WireRevision's doc names the handshake's actual check
  instead of a comparison `bd connect` never makes.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(backend/http): give the e2e server the LeaseReclaimer role #7412 added to the required set

main's #7412 made LeaseReclaimer part of httpapi's required role set and
updated main's own binders, but backend/http's e2eServeConfig predates it,
so every TestE2E_* that binds the in-process server failed with
"httpapi: no database source". Same shape as e63f7da36a for BatchGetter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(externaldeps): lead the --claim-next refusal with the wire fact; pin the L14 leg and connect --clear

Review i4 minors/nits on the stranded fixup, carried as a separate commit:

- remoteBatchCloser's --claim-next refusal now says the served batch close
  cannot carry a next claim (the real cause), rather than blaming
  externally blocked work a workspace may not have. The test asserts the
  reason, that nothing closed, and the remedy.
- GetDependencyTree's doc says a served store refuses its reads and that
  bd dep tree goes through remoteTreeWalker.
- README no longer gives `bd ready --claim` local semantics everywhere; it
  names when ledger row L14's two-step claim applies.
- TestE2E_BDOverHTTP claims once more after the external-ref holder is
  closed, pinning L14's composed leg (a mutant whose composed leg claims
  nothing turns only that step red).
- New TestE2E_ConnectClearRestoresThePreviousBackend: --clear restores
  the prior dolt backend with its identity intact, removes the sidecar,
  and a second --clear changes nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(backend/http): run the served e2e under bazel

//backend/http:http_test declared no bd binary, so under bazel the e2e
files (bazeltest.PrebuiltBD) failed on BEADS_TEST_BD_BINARY with
BEADS_HTTP_TEST_REQUIRED=1 and only "passed" by skipping otherwise. It now
takes //cmd/bd:bd_for_tests (visibility added), size large / timeout long:
the served cases run 4-5 minutes on the farm under BEADS_TEST_EMBEDDED_DOLT=1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(backend/http): run the e2e tier in the embedded lane as http_served_test

The previous commit gives //backend/http:http_test the bd binary, but no
lane sets BEADS_TEST_EMBEDDED_DOLT for it, so its TestE2E_* cases still
skipped in CI. http_served_test runs the same test binary through
//tools/bazel:go_test_variant.sh with BEADS_TEST_EMBEDDED_DOLT=1 and
BEADS_HTTP_TEST_REQUIRED=1 (a skip fails it). It is tagged embedded, so
the --config=embedded lane runs it and prcore/ci keep their current
shape, the same split as //internal/httpclient:httpclient_served_test.
Under plain --config=embedded all 8 TestE2E_* cases run, none skipped.

TestBazelRetiredLanesCannotBeNarrowed pins every embedded-lane target's
args and env; the new target is added to that pin list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(externaldeps): cap the ready rows kept, not the bumped http window

An http store cannot carry ExcludeIDs on listReadyWork, so readyExclusion
fetches the unfiltered ready set, widened by len(blocked) under a limit,
and drops the externally blocked rows client-side. The http ready bridge
enforced MaxRows on the rows it fetched rather than the rows kept. So
`bd ready --limit N --max-rows N` exited 2 whenever more than N issues
were ready and any workspace issue held an unsatisfied external blocker
(a closed one counts), even with no blocked row in the window, and a
blocked row in the page counted against the cap although the caller
never sees it. Locally neither can happen: issueops enforces the cap on
the filtered page it delivers, after the Limit trim
(finishReadyWorkWithCounts).

The client-side path now takes MaxRows off the inner query and enforces
the caller's cap on the rows it keeps, after the trim, in all three
listings. The inner query keeps the cap whenever nothing is dropped
client-side. MaxRows never rode the wire, so the fetch is the size it
was; refetchedReadyTotal already clears the cap on its own over-fetch.

TestServedReadyMaxRowsCountsTheRowsKept runs the decorator over a served
http store holding three free issues and one externally blocked one: a
cap equal to the limit, a cap under a wider limit, an unlimited listing,
and a kept page wider than the cap, which must still return
ErrTooManyRows with the caller's cap and source. Each case runs through
GetReadyWork, GetReadyWorkWithCounts and GetReadyWorkWithCountsAndTotal.
Leaving the cap on the inner query fails the first three cases in all
three methods, dropping the re-applied cap fails the fourth, and
enforcing it before the Limit trim fails the first.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
This commit is contained in:
Julian Knutsen 2026-10-10 07:19:18 +00:00 • committed by GitHub
commit 8dbbe78e9a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
55 changed files with 6472 additions and 82 deletions

View file

@ -299,6 +299,7 @@ filegroup(
) + [
"//backend:repo_doc_files",
"//backend/conformance:repo_doc_files",
"//backend/http:repo_doc_files",
"//beadserrors:repo_doc_files",
"//cmd/bd:repo_doc_files",
"//cmd/bd/doctor:repo_doc_files",
@ -442,6 +443,7 @@ filegroup(
"//scripts/workflowtags:repo_doc_files",
"//test/conformance:repo_doc_files",
"//test/docsync:repo_doc_files",
"//test/embedder:repo_doc_files",
"//test/testmainconvention:repo_doc_files",
"//tests/hook_timeout_backends:repo_doc_files",
"//tests/migration:repo_doc_files",
@ -530,6 +532,7 @@ filegroup(
) + [
"//backend:repo_go_srcs",
"//backend/conformance:repo_go_srcs",
"//backend/http:repo_go_srcs",
"//beadserrors:repo_go_srcs",
"//cmd/bd:repo_go_srcs",
"//cmd/bd/doctor:repo_go_srcs",
@ -673,6 +676,7 @@ filegroup(
"//scripts/workflowtags:repo_go_srcs",
"//test/conformance:repo_go_srcs",
"//test/docsync:repo_go_srcs",
"//test/embedder:repo_go_srcs",
"//test/testmainconvention:repo_go_srcs",
"//tests/hook_timeout_backends:repo_go_srcs",
"//tests/migration:repo_go_srcs",
@ -746,6 +750,7 @@ filegroup(
) + [
"//backend:repo_go_test_srcs",
"//backend/conformance:repo_go_test_srcs",
"//backend/http:repo_go_test_srcs",
"//beadserrors:repo_go_test_srcs",
"//cmd/bd:repo_go_test_srcs",
"//cmd/bd/doctor:repo_go_test_srcs",
@ -889,6 +894,7 @@ filegroup(
"//scripts/workflowtags:repo_go_test_srcs",
"//test/conformance:repo_go_test_srcs",
"//test/docsync:repo_go_test_srcs",
"//test/embedder:repo_go_test_srcs",
"//test/testmainconvention:repo_go_test_srcs",
"//tests/hook_timeout_backends:repo_go_test_srcs",
"//tests/migration:repo_go_test_srcs",
@ -971,6 +977,7 @@ filegroup(
) + [
"//backend:repo_other_files",
"//backend/conformance:repo_other_files",
"//backend/http:repo_other_files",
"//beadserrors:repo_other_files",
"//cmd/bd:repo_other_files",
"//cmd/bd/doctor:repo_other_files",
@ -1114,6 +1121,7 @@ filegroup(
"//scripts/workflowtags:repo_other_files",
"//test/conformance:repo_other_files",
"//test/docsync:repo_other_files",
"//test/embedder:repo_other_files",
"//test/testmainconvention:repo_other_files",
"//tests/hook_timeout_backends:repo_other_files",
"//tests/migration:repo_other_files",

View file

@ -135,6 +135,47 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
will not send, and value constraints (`maxLength`, `pattern` and the
like) are outside the digest: such changes need their own review against
`wire_revision`.
- `cmd/bd` registers the http client backend built on top of the v0 `bd
serve` wire (`internal/httpclient`), so a workspace whose
`.beads/metadata.json` selects `"backend": "http"` opens over HTTP through
the ordinary `OpenBestAvailable` path, exactly like a registered extension
backend does. A new `bd connect <url>` command performs the handshake
(`api_version`, wire revision, and — with `--expect-project-id` — workspace
identity) and writes nothing until it succeeds; it then records the
per-user activation sidecar (`.beads/http_target.json`, never
git-tracked — `cmd/bd/doctor/gitignore.go` now requires it and its local
metadata sidecar `http_local_metadata.json` be ignored) and sets
`metadata.json`'s backend to `"http"`. `bd connect` refuses a plain `http://`
URL to a non-loopback host (a bearer credential would cross the network in
the clear) unless `--allow-plaintext` is given, and refuses to switch a
workspace that already selects a different backend unless `--force`. A
credential is never accepted on the command line or written to disk; it
comes from the same ladder every http request already uses
(`BEADS_HTTP_TOKEN=host[:port]=<token>`, then
`BEADS_HTTP_TOKEN_COMMAND=host[:port]=<command>`, then the credentials
file, then none); both variables take only that host-scoped form, and a
bare value is refused, since it would be sent to whatever server the
workspace's `http_target.json` names. The external-dependency policy
decorator (`internal/storage/externaldeps`) still wraps an http store, and
stands down only when the server's handshake advertises
`policy.external_dependencies` (that server has already applied the policy
before answering); against any other server it applies the policy
client-side, around the http store's own served roles. Externally blocked
issues drop out of `bd ready`'s page, total, and count (the v0 wire cannot
express an id exclusion) and out of `bd ready --claim`, `bd close` refuses
them without `--force`, and `bd dep tree` shows their external leaves.
`bd close --claim-next` refuses over http before anything closes, since
the wire's batch close cannot carry the claim; its error says to close
without it and then run `bd ready --claim`.
- The public `backend/http` package (`bdhttp`) is the out-of-tree door onto
this backend for an embedder that links beads as a library rather than
running `cmd/bd`: `Register(Options)` adds `"http"` to the registry and
installs its transport, `Open`/`Handshake` dial a server directly with no
workspace on disk, and the existing public `beads.OpenBestAvailableWith`
takes a per-call `OpenOptions.Credential` through it — the case a
multi-tenant embedder serving many workspaces needs, where a
process-global credential cannot stand in for one tenant's own.
EXPERIMENTAL, pin an exact beads version.
### Changed

View file

@ -175,6 +175,76 @@ are unaffected.
the guard with a warning on stderr. Use this only if you know the forward
migrations are additive and safe for your specific workload.
## 🔌 HTTP Backend (experimental)
A workspace can also talk to a remote `bd serve` over HTTP instead of opening
a local Dolt database — useful for a thin client against a shared server, or
for embedding beads in another program as a library.
```bash
# Point this workspace at a server (verifies the server first; writes nothing on failure)
bd connect https://bd.example.com --expect-project-id my-project
# The everyday commands work the same way: create, show, list, update, close,
# dependencies (including `bd dep tree`), comments, ready/count queries, and
# batch apply all round-trip over the wire with the same semantics as a local
# database. So does `bd ready --claim`, except against an older server or while
# any issue holds an unsatisfied external ref: then it lists and claims in two
# steps rather than one transaction (ledger row L14, linked below).
bd create "Fix auth bug" -p 1 -t bug
bd ready --json
bd close bd-a1b2 "Fixed"
# Loopback needs no TLS
bd connect http://127.0.0.1:8080
# Forget the server and restore this workspace's previous backend selection
bd connect --clear
```
HTTP mode is not a byte-for-byte mirror of local mode, though: some
operations are knowingly refused (the wire has no way to express them) or
degrade to a different-but-not-misreadable behavior (for example, a few
unbounded listing shapes lose single-query snapshot isolation and instead
walk the server page by page). Every one of these is a deliberate, tested
row, never a silent gap — see
[`engdocs/design/http-divergence-ledger.md`](engdocs/design/http-divergence-ledger.md)
for the full, generated list of what differs and why before depending on an
HTTP-backed workspace for a workflow you haven't checked against it. One
refusal to know up front: `bd close --claim-next` closes nothing over HTTP,
because the wire's batch close cannot carry the claim. Close without it, then
run `bd ready --claim`.
[Cross-repo dependencies](docs/core-concepts/dependencies.md#cross-repo-dependencies)
keep their policy over HTTP. The client resolves each `external:` reference
against its own `external_projects` configuration. `bd dep tree` shows the
reference as a leaf, and while it is unsatisfied the issue stays out of
`bd ready` and `bd ready --claim`, and `bd close` refuses it without
`--force`. A server that advertises enforcing that policy itself is trusted
to do so instead.
`bd connect` writes two things on success: `.beads/metadata.json` gets
`"backend": "http"`, and a per-user, never-git-tracked sidecar
(`.beads/http_target.json`) records the server URL and pinned project id. It
never writes a credential to disk. A bearer credential, if the server
requires one, comes from `BEADS_HTTP_TOKEN=host[:port]=<token>`, then
`BEADS_HTTP_TOKEN_COMMAND=host[:port]=<command>` (a helper that prints a
token), then the credentials file, then no credential at all. Both variables
must name the server they are for (a pattern without a port matches that
host on any port); a bare value is refused, because it would be sent to
whatever server the workspace's `http_target.json` names. Connecting to a
non-loopback server over plain `http://` is refused by default (a bearer
credential would cross the network unencrypted) — pass `--allow-plaintext` to
override, or use `https://`. Switching a workspace that already selects a
different backend needs `--force`.
An embedding program that links beads as a library rather than running
`cmd/bd` registers this backend itself through the public
[`backend/http`](backend/http) package (`bdhttp.Register`), and can open a
workspace with a credential scoped to one caller via
`beads.OpenBestAvailableWith` — see that package's doc comment for the
multi-tenant case. This surface is EXPERIMENTAL: pin an exact beads version.
## 🌐 Community Tools
See [docs/community-tools.md](docs/community-tools.md) for a curated list of community-built UIs, extensions, and integrations—including terminal interfaces, web UIs, editor extensions, and native apps.

149
backend/http/BUILD.bazel Normal file
View file

@ -0,0 +1,149 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")
load("@rules_shell//shell:sh_test.bzl", "sh_test")
go_library(
name = "http",
srcs = [
"attach.go",
"credential.go",
"errors.go",
"http.go",
"target.go",
],
importpath = "github.com/steveyegge/beads/backend/http",
visibility = ["//visibility:public"],
deps = [
"//backend",
"//internal/configfile",
"//internal/gitignore",
"//internal/httpapi/apigen",
"//internal/httpclient",
"//internal/httpclient/wire",
"//internal/storage/backends",
],
)
go_test(
name = "http_test",
# The served e2e cases run ~4-5 min on the farm under
# BEADS_TEST_EMBEDDED_DOLT=1, past the medium 300s action limit.
size = "large",
timeout = "long",
srcs = [
"attach_test.go",
"e2e_cli_test.go",
"open_test.go",
"register_test.go",
],
# The e2e files exec the real bd (bazeltest.PrebuiltBD); they run under
# BEADS_TEST_EMBEDDED_DOLT=1 and skip otherwise.
data = ["//cmd/bd:bd_for_tests"],
env = {"BEADS_TEST_BD_BINARY": "$(rlocationpath //cmd/bd:bd_for_tests)"},
deps = [
":http",
"//backend",
"//internal/configfile",
"//internal/httpapi",
"//internal/httpclient",
"//internal/storage",
"//internal/storage/backends",
"//internal/storage/domain",
"//internal/storage/embeddeddolt",
"//internal/testutil/bazeltest",
],
)
# http_test again, with the e2e tier on: BEADS_TEST_EMBEDDED_DOLT=1 starts the
# in-process server over embedded Dolt and BEADS_HTTP_TEST_REQUIRED=1 turns a
# skip of that tier into a failure. http_test alone skips the TestE2E_* cases
# in every lane, because no lane sets those variables for it. It repeats
# http_test's `env` (go_test_variant.sh).
#
# Tags:
# embedded: the embedded-Dolt tier; excluded from --config=prcore/ci,
# selected by --config=embedded.
sh_test(
name = "http_served_test",
size = "large",
# -test.timeout=19m fires (with a goroutine dump) before Bazel's 1200s
# action limit (eternal under --config=embedded).
timeout = "eternal",
srcs = ["//tools/bazel:go_test_variant.sh"],
args = [
"$(rootpath :http_test)",
"-test.v",
"-test.count=1",
"-test.timeout=19m",
],
data = [
":http_test",
"//cmd/bd:bd_for_tests",
],
env = {
"BEADS_HTTP_TEST_REQUIRED": "1",
"BEADS_TEST_BD_BINARY": "$(rlocationpath //cmd/bd:bd_for_tests)",
"BEADS_TEST_EMBEDDED_DOLT": "1",
},
tags = ["embedded"],
)
# --- begin repo_files (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---
filegroup(
name = "repo_doc_files",
srcs = glob(
[
"**/*.md",
"**/*.mdx",
],
allow_empty = True,
exclude = ["**/node_modules/**"],
),
visibility = ["//:__subpackages__"],
)
filegroup(
name = "repo_go_srcs",
srcs = glob(
["**/*.go"],
allow_empty = True,
exclude = [
"**/*_test.go",
"**/node_modules/**",
],
),
visibility = ["//:__subpackages__"],
)
filegroup(
name = "repo_go_test_srcs",
srcs = glob(
["**/*_test.go"],
allow_empty = True,
exclude = ["**/node_modules/**"],
),
visibility = ["//:__subpackages__"],
)
filegroup(
name = "repo_other_files",
srcs = glob(
["**"],
exclude = [
"**/*.db",
"**/*.exe",
"**/*.go",
"**/*.md",
"**/*.mdx",
"**/*.out",
"**/*.prof",
"**/*.pyc",
"**/*.test",
"**/__pycache__/**",
"**/node_modules/**",
],
),
visibility = ["//:__subpackages__"],
)
# --- end repo_files ---

133
backend/http/attach.go Normal file
View file

@ -0,0 +1,133 @@
// Written fresh for OSS beads S6 (no bd-enterprise source copied): closes the
// S6 review's MED-8 finding, which asked for one public helper an embedder
// (or `bd connect`) can call to activate a workspace for this backend,
// instead of each caller hand-assembling the two-file write itself.
package bdhttp
import (
"context"
"fmt"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/gitignore"
"github.com/steveyegge/beads/internal/httpclient"
)
// Attach activates the http backend for the workspace at beadsDir: it writes
// the per-user activation sidecar (SaveTarget) pinning target's server and
// project, then sets the workspace's git-tracked backend selection in
// metadata.json to this backend, preserving every other configured field
// (database, project id, ...) and creating a fresh default metadata.json only
// when none exists yet. Both files are written atomically (temp file plus
// rename, 0600): see SaveTarget and configfile.Config.Save.
//
// Unless the caller already set target.PreviousBackend, Attach fills it in
// from the workspace's PRIOR backend selection, so `bd connect --clear` can
// restore it later; see that field's doc.
//
// Attach does NOT verify target itself — it has no network access and takes
// target on faith. A caller MUST call Handshake (or httpclient.Handshake)
// against target first and only call Attach once that succeeds; attaching an
// unverified target lets a workspace record a server that is wrong, hostile,
// or simply never answers. Connect, below, does both in the right order and
// is the easier door for a caller that does not need them split.
//
// Attach does not create beadsDir itself — MkdirAll is the caller's job, same
// as SaveTarget's own precondition. It DOES ensure both of this backend's
// per-user filenames are covered by beadsDir's .gitignore
// (gitignore.EnsurePatternIgnored): the sidecar, and
// httpclient.LocalMetadataFileName, which the store writes on first use. That
// is the same guarantee `bd connect` makes via doctor.EnsureGitignoreForBeadsDir
// — but through a small, independent implementation here rather than
// importing cmd/bd/doctor, whose package as a whole pulls in dolt and
// git-process dependencies that would break backend/http's promise of a
// minimal dependency footprint for an embedder. The two are complementary and
// idempotent together.
//
// It is the one place that writes BOTH of the files that activate beads/http
// (the sidecar and metadata.json's backend selection; the local metadata file
// is the store's own), so a caller that used to hand-assemble the pair
// (`bd connect`'s own connectCmd.RunE, test/embedder's linkage proof) can call
// this instead and stay correct across a future change to either file's shape.
func Attach(beadsDir string, target Target) error {
cfg, err := configfile.Load(beadsDir)
if err != nil {
return fmt.Errorf("reading %s: %w", configfile.ConfigFileName, err)
}
if cfg == nil {
cfg = configfile.DefaultConfig()
}
// target.PreviousBackend (bee-ghosttrack CHANGES_REQUESTED on #7288,
// should-fix 1) is what `bd connect --clear` restores metadata.json's
// backend selection to. A caller rarely sets it itself (connect.go
// doesn't), so fill it in here unless the workspace is ALREADY on this
// backend — a re-connect (a new URL, or re-pinning the same one) must
// not overwrite an earlier recorded previous backend with "http" itself,
// or a chain of reconnects would forget what to restore and --clear
// would land back on http with no sidecar. Carry the ALREADY-recorded
// value forward instead; a failed or missing read just leaves it "",
// the same as a workspace with nothing to restore. The check reads the
// raw field: in a process that never ran Register, GetBackend() reports an
// http workspace as the dolt default, and the reconnect would record that.
if target.PreviousBackend == "" {
if cfg.Backend == httpclient.Backend {
if prior, err := httpclient.LoadTarget(beadsDir); err == nil {
target.PreviousBackend = prior.PreviousBackend
}
} else {
target.PreviousBackend = cfg.GetBackend()
}
}
if err := SaveTarget(beadsDir, target); err != nil {
return fmt.Errorf("writing %s: %w", httpclient.TargetFileName, err)
}
// Both of this backend's per-user files: the sidecar just written, and
// the local metadata file the store itself writes on first use, which
// never passes through here and so must be covered in advance.
for _, name := range []string{httpclient.TargetFileName, httpclient.LocalMetadataFileName} {
if err := gitignore.EnsurePatternIgnored(beadsDir, name); err != nil {
return fmt.Errorf("ensuring %s is gitignored: %w", name, err)
}
}
// Only the backend SELECTION changes here. cfg.Database/cfg.ProjectID (if
// an existing metadata.json carried them for a different, prior backend)
// are left exactly as loaded — see connect.go's own comment on this same
// hazard: this backend's identity lives entirely in the sidecar just
// written above, and clobbering those fields would corrupt a future
// `bd connect --clear` back to a different backend.
cfg.Backend = httpclient.Backend
if err := cfg.Save(beadsDir); err != nil {
return fmt.Errorf("writing %s: %w", configfile.ConfigFileName, err)
}
return nil
}
// Connect is Handshake followed by Attach: it verifies target — the same
// server-identity check `bd connect` performs before it ever writes
// anything — and only then activates the workspace. It is the convenience
// door for a caller that wants Attach's ordering guarantee (never attach a
// workspace to a server this process has not handshaken) without composing
// Handshake and Attach itself.
//
// Mirroring `bd connect`'s own project-id pinning: if target.ExpectProjectID
// is empty, Connect pins it to whatever project the server answered with
// before calling Attach, so the workspace is tied to that project from this
// call on. If it is already set, Handshake itself refuses a mismatch and
// Connect returns that error without calling Attach at all. Connect does not
// replicate `bd connect`'s other CLI-level policy — --force re-pinning an
// already-attached workspace, or refusing to switch a workspace from a
// different backend — those stay the command's own job; Connect's contract
// is exactly "handshake, then attach".
func Connect(ctx context.Context, beadsDir string, target Target, opts Options) (*ServerSnapshot, error) {
snapshot, err := Handshake(ctx, target, opts)
if err != nil {
return nil, err
}
if target.ExpectProjectID == "" {
target.ExpectProjectID = snapshot.ProjectID
}
if err := Attach(beadsDir, target); err != nil {
return nil, err
}
return snapshot, nil
}

180
backend/http/attach_test.go Normal file
View file

@ -0,0 +1,180 @@
package bdhttp_test
// Written fresh for OSS beads S6: pins the bee-ghosttrack CHANGES_REQUESTED
// finding on #7288, should-fix 1 — `bd connect --clear` restoring a
// workspace's previous backend selection needs Attach to have recorded what
// that previous backend WAS, onto the sidecar, at connect time.
import (
"net/url"
"os"
"path/filepath"
"strings"
"testing"
bdhttp "github.com/steveyegge/beads/backend/http"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/httpclient"
)
func mustAttachTestURL(t *testing.T, raw string) *url.URL {
t.Helper()
u, err := url.Parse(raw)
if err != nil {
t.Fatalf("url.Parse(%q): %v", raw, err)
}
return u
}
// TestAttachRecordsThePriorBackendAsPreviousBackend is the first-connect
// case: a workspace that selected "dolt" before ever running `bd connect`
// must have Attach record "dolt" as PreviousBackend, so --clear can restore
// it later.
func TestAttachRecordsThePriorBackendAsPreviousBackend(t *testing.T) {
beadsDir := t.TempDir()
cfg := &configfile.Config{Backend: "dolt", Database: "beads.db"}
if err := cfg.Save(beadsDir); err != nil {
t.Fatalf("seed metadata.json: %v", err)
}
target := bdhttp.Target{BaseURL: mustAttachTestURL(t, "http://127.0.0.1:8080")}
if err := bdhttp.Attach(beadsDir, target); err != nil {
t.Fatalf("Attach: %v", err)
}
got, err := bdhttp.LoadTarget(beadsDir)
if err != nil {
t.Fatalf("LoadTarget: %v", err)
}
if got.PreviousBackend != "dolt" {
t.Errorf("PreviousBackend = %q, want %q", got.PreviousBackend, "dolt")
}
after, err := configfile.Load(beadsDir)
if err != nil {
t.Fatalf("reload metadata.json: %v", err)
}
// Asserts on the raw stored field, not GetBackend(): GetBackend() falls
// back to BackendDolt for any name backendnames.Has doesn't recognize
// (TestGetBackendAllowlist in internal/configfile), and "http" is only
// ever added there by bdhttp.Register/httpclient.Register — a
// once-per-process, panics-on-duplicate production wiring call this test
// package must not make. The raw field is exactly what Attach controls.
if after.Backend != httpclient.Backend {
t.Errorf("metadata.json backend = %q after Attach, want %q", after.Backend, httpclient.Backend)
}
}
// TestAttachOnAFreshWorkspaceRecordsTheImplicitDoltDefault covers a
// workspace with no metadata.json at all: GetBackend()'s own default
// ("dolt") is what a future --clear restores to, matching what every other
// backend-selection read in this codebase already treats as this
// workspace's backend.
func TestAttachOnAFreshWorkspaceRecordsTheImplicitDoltDefault(t *testing.T) {
beadsDir := t.TempDir()
target := bdhttp.Target{BaseURL: mustAttachTestURL(t, "http://127.0.0.1:8080")}
if err := bdhttp.Attach(beadsDir, target); err != nil {
t.Fatalf("Attach: %v", err)
}
got, err := bdhttp.LoadTarget(beadsDir)
if err != nil {
t.Fatalf("LoadTarget: %v", err)
}
if got.PreviousBackend != "dolt" {
t.Errorf("PreviousBackend = %q, want %q (configfile.BackendDolt)", got.PreviousBackend, "dolt")
}
}
// TestAttachOnAReconnectKeepsTheEarlierPreviousBackend covers a workspace
// ALREADY on http (a bare re-connect, or --force re-pinning the same
// backend): Attach must not overwrite an already-recorded PreviousBackend
// with "http" itself, or a chain of reconnects would forget what --clear is
// supposed to restore.
func TestAttachOnAReconnectKeepsTheEarlierPreviousBackend(t *testing.T) {
beadsDir := t.TempDir()
// Not dolt: an unregistered "http" reads back through GetBackend() as the
// dolt default, so a dolt prior could not tell carrying it forward from
// recording that default.
cfg := &configfile.Config{Backend: "postgres", Database: "beads.db"}
if err := cfg.Save(beadsDir); err != nil {
t.Fatalf("seed metadata.json: %v", err)
}
first := bdhttp.Target{BaseURL: mustAttachTestURL(t, "http://127.0.0.1:8080")}
if err := bdhttp.Attach(beadsDir, first); err != nil {
t.Fatalf("first Attach: %v", err)
}
// A second connect to a different url, same workspace, same backend
// ("http", already selected) — the shape `bd connect --force` to a new
// server produces for an already-http workspace.
second := bdhttp.Target{BaseURL: mustAttachTestURL(t, "http://127.0.0.1:9090")}
if err := bdhttp.Attach(beadsDir, second); err != nil {
t.Fatalf("second Attach: %v", err)
}
got, err := bdhttp.LoadTarget(beadsDir)
if err != nil {
t.Fatalf("LoadTarget: %v", err)
}
if got.PreviousBackend != "postgres" {
t.Errorf("PreviousBackend after a reconnect = %q, want the original %q carried forward, not the just-left \"http\"", got.PreviousBackend, "postgres")
}
if got.BaseURL.String() != "http://127.0.0.1:9090" {
t.Errorf("BaseURL = %q, want the second connect's url", got.BaseURL.String())
}
}
// TestAttachHonorsACallerSuppliedPreviousBackend covers Connect's doc
// contract: a caller that already set target.PreviousBackend itself (none
// of this package's own callers do today) must have that value win over
// Attach's own inference.
func TestAttachHonorsACallerSuppliedPreviousBackend(t *testing.T) {
beadsDir := t.TempDir()
cfg := &configfile.Config{Backend: "dolt", Database: "beads.db"}
if err := cfg.Save(beadsDir); err != nil {
t.Fatalf("seed metadata.json: %v", err)
}
target := bdhttp.Target{BaseURL: mustAttachTestURL(t, "http://127.0.0.1:8080"), PreviousBackend: "mysql"}
if err := bdhttp.Attach(beadsDir, target); err != nil {
t.Fatalf("Attach: %v", err)
}
got, err := bdhttp.LoadTarget(beadsDir)
if err != nil {
t.Fatalf("LoadTarget: %v", err)
}
if got.PreviousBackend != "mysql" {
t.Errorf("PreviousBackend = %q, want the caller-supplied %q to win over the inferred %q", got.PreviousBackend, "mysql", "dolt")
}
}
// TestAttachGitignoresBothPerUserFiles pins Attach's .gitignore guarantee for
// both files this backend keeps per user in beadsDir: the sidecar Attach
// writes, and the local metadata file the store writes later, on first use.
// `bd connect` also covers the second through doctor's required patterns,
// but an embedder activating through Attach alone has only this. A reconnect
// must not duplicate either line.
func TestAttachGitignoresBothPerUserFiles(t *testing.T) {
beadsDir := t.TempDir()
for _, raw := range []string{"http://127.0.0.1:8080", "http://127.0.0.1:9090"} {
if err := bdhttp.Attach(beadsDir, bdhttp.Target{BaseURL: mustAttachTestURL(t, raw)}); err != nil {
t.Fatalf("Attach(%s): %v", raw, err)
}
}
content, err := os.ReadFile(filepath.Join(beadsDir, ".gitignore"))
if err != nil {
t.Fatalf("read .gitignore after Attach: %v", err)
}
for _, name := range []string{httpclient.TargetFileName, httpclient.LocalMetadataFileName} {
count := 0
for _, line := range strings.Split(string(content), "\n") {
if strings.TrimSpace(line) == name {
count++
}
}
if count != 1 {
t.Errorf(".gitignore names %s %d time(s) after two Attaches, want exactly 1:\n%s", name, count, content)
}
}
}

View file

@ -0,0 +1,57 @@
package bdhttp
import (
"net/url"
"github.com/steveyegge/beads/internal/httpclient"
)
// CredentialProvider authorizes outbound requests to a bd serve.
//
// An implementation must be safe for concurrent use, must never log or
// persist a secret, and must FAIL CLOSED: a configured source that errors
// aborts the request rather than silently downgrading to an unauthenticated
// one.
type CredentialProvider = httpclient.CredentialProvider
const (
// TokenEnv carries the bearer token itself, the highest rung of the
// built-in ladder.
// #nosec G101 -- the NAME of an environment variable, not a credential.
TokenEnv = httpclient.TokenEnv
// TokenCommandEnv names a helper that prints a token, either bare or in
// the kubectl ExecCredential envelope {"token","expirationTimestamp"}.
// #nosec G101 -- the NAME of an environment variable, not a credential.
TokenCommandEnv = httpclient.TokenCommandEnv
)
// BearerProvider is the built-in credential ladder: TokenEnv, then
// TokenCommandEnv, then the credentials file's [host:port] section, then no
// credential at all (the tip OSS server's loopback-trust posture, a
// legitimate answer and not a failure).
type BearerProvider = httpclient.BearerProvider
// NewBearerProvider builds the built-in ladder for a server. Open and
// Handshake already dial with it; it is exported so an embedder that wants to
// WRAP the default (decorate it for one target, delegate to it for the rest)
// does not have to reimplement it.
func NewBearerProvider(base *url.URL) *BearerProvider {
return httpclient.NewBearerProvider(base)
}
// ProvidedCredential adapts a CredentialProvider into the opaque
// backends.Credential marker interface, so it can be carried through
// beads.OpenOptions / beads.OpenBestAvailableWith (and, directly, through
// backends.OpenOptions) to the registered "http" backend's OpenWith hook
// Register installs.
//
// A multi-tenant embedder (for example gc, Gas City: one process serving many
// workspaces, called "cities") that wants a distinct credential per workspace
// wraps that workspace's own provider in a ProvidedCredential rather than
// calling Open directly: the registered dialer Register installs is
// process-wide and cannot hold one credential per tenant, but the per-open
// OpenWith seam can, and this is the opaque wrapper type that seam
// recognizes. See httpclient.ResolveCredential and
// backends.OpenOptions.Credential's own doc comment for the single-tenant
// ambient-ladder default this exists to opt out of.
type ProvidedCredential = httpclient.ProvidedCredential

1059
backend/http/e2e_cli_test.go Normal file

File diff suppressed because it is too large Load diff

111
backend/http/errors.go Normal file
View file

@ -0,0 +1,111 @@
package bdhttp
import (
"github.com/steveyegge/beads/internal/httpclient"
"github.com/steveyegge/beads/internal/httpclient/wire"
)
// The refusal taxonomy, re-exported so an embedder can CLASSIFY what this
// backend answers rather than match its text.
//
// Every name here is an alias of the internal one, so errors.Is and
// errors.As reach the same values across the module boundary. Two audiences
// need that: the host application deciding whether a failure is the user's,
// the server's or the wiring's, and any surface that renders it, since these
// types carry the server URL, its release and its advertised capabilities
// precisely so a renderer parses nothing.
// ErrNoTransport reports a process that registered this backend without
// installing a transport. It is a build-wiring fault, not a user's: the
// workspace and the server may both be fine. Register cannot produce it — it
// installs the transport itself — so meeting it means something reached the
// registry another way (for example a test double that calls
// backends.Register directly).
var ErrNoTransport = httpclient.ErrNoTransport
// The store's own refusals. A method this backend does not serve returns
// UnsupportedError, which unwraps to the portable storage.ErrUnsupported, so
// the classification a caller already does for a local backend holds
// unchanged.
type (
// UnsupportedError is what every refused method of this backend returns: a
// STRUCT, reached with errors.As, carrying the server it refused on, that
// server's release and the capabilities it does advertise. It unwraps to
// the portable storage.ErrUnsupported, so a caller that only wants "this
// backend cannot do that" matches the same sentinel it would for a local
// store and needs none of these fields.
UnsupportedError = httpclient.ErrHTTPUnsupported
// InexpressibleError is a request carrying a field the v0 wire has no
// member for. It is both an unsupported-operation refusal and an encoder
// refusal, because two audiences classify it, and errors.As reaches either
// arm.
InexpressibleError = httpclient.InexpressibleError
// PartialIDSearchError answers a substring or prefix id lookup, which no
// server operation serves. The input may be a partial id OR a full id that
// does not exist, and the client cannot tell which, so the refusal names
// both outcomes.
PartialIDSearchError = httpclient.PartialIDSearchError
// WatchRefusedError refuses a `--watch`-style polling loop, which
// amplifies load on a shared server with no change detection to justify
// it.
WatchRefusedError = httpclient.WatchRefusedError
// RedactedSettingError reports a setting whose value the server withheld
// because its key marks it credential-bearing. It is not an unset key: one
// says the workspace stores nothing, the other says the workspace stores
// something this client may not see.
RedactedSettingError = httpclient.RedactedSettingError
)
// The class sentinels behind those types, for a caller that classifies
// without reaching for fields.
var (
ErrPartialIDSearch = httpclient.ErrPartialIDSearch
ErrWatchUnsupported = httpclient.ErrWatchUnsupported
ErrSettingRedacted = httpclient.ErrSettingRedacted
)
// The wire's own refusals, which reach a caller through Open, Handshake and
// every dispatch a store makes.
type (
// ProjectMismatchError is the wrong-server diagnostic: the server
// answered, and it owns a different workspace. It names both project ids
// AND the server's own database and repo root, because "the ids differ"
// does not say WHICH wrong server answered — and on a shared host that is
// the question.
ProjectMismatchError = wire.ProjectMismatchError
// APIVersionError reports a server on a path major this client cannot
// address. It names both versions, because neither alone says which side
// to move.
APIVersionError = wire.APIVersionError
// CapabilityError reports an operation the server does not advertise. It
// is how version skew is told apart from a missing entity: an unrouted
// path on an older server answers a bare 404, indistinguishable from
// not_found, so the advertised list is consulted before the dial.
CapabilityError = wire.CapabilityError
// ProblemError is a refusal the server described: its status, its
// problem code, and the typed discriminators it carries. Unwrap reaches
// the sentinel below that the code maps to.
ProblemError = wire.ProblemError
)
var (
ErrProjectMismatch = wire.ErrProjectMismatch
ErrAPIVersion = wire.ErrAPIVersion
ErrCapabilityAbsent = wire.ErrCapabilityAbsent
// ErrUnauthenticated is the credential the server refused. The error
// names which ladder rung held it — a provenance label, never the
// credential — so an operator knows what to rotate.
ErrUnauthenticated = wire.ErrUnauthenticated
ErrBadRequest = wire.ErrBadRequest
ErrInvalidCursor = wire.ErrInvalidCursor
ErrBusy = wire.ErrBusy
ErrDBUnavailable = wire.ErrDBUnavailable
ErrServerFault = wire.ErrServerFault
)
// ProjectMismatchRecovery is the one action that resolves a wrong-server
// mismatch: point the workspace back at the server that owns it (re-run
// `bd connect`). There is no local database to reconcile, so the repair
// tooling a local backend would name does not apply.
const ProjectMismatchRecovery = wire.ProjectMismatchRecovery

216
backend/http/http.go Normal file
View file

@ -0,0 +1,216 @@
// Package bdhttp is the public door to the http client store that ships
// inside this module (internal/httpclient) — the storage backend that speaks
// the v0 `bd serve` wire to a remote server — for embedders that link beads
// as a library rather than running cmd/bd.
//
// Such an embedder cannot reach the store any other way, for the reason its
// (hypothetical) postgres sibling would document: the registry facade's
// Register takes a backends.Backend value whose constructors live in an
// internal package, and the only other production wiring is cmd/bd's. Without
// this package a workspace whose .beads/metadata.json names "http" is a
// workspace an embedder can only tombstone.
//
// # Registration is two-step, and that is why Register takes Options
//
// Putting "http" in the registry is not enough to open anything: the store
// dials through a process-wide transport constructor that registration does
// not install on its own, so a binary that registered and stopped there would
// fail every open with httpclient.ErrNoTransport — a build-wiring fault, not a
// workspace or server problem. Register does both: it adds "http" to the
// registry AND installs the default dialer, and it installs a per-open
// OpenWith hook so a multi-tenant embedder (for example gc, Gas City — one
// process serving many workspaces, called "cities") can route a distinct
// credential through backends.OpenOptions / beads.OpenBestAvailableWith for
// each workspace it opens, without the process-wide dialer being asked to
// serve two tenants with one fixed credential.
//
// # Stability
//
// EXPERIMENTAL, on the same terms as the backend package this sits beside.
// Pin an exact beads version.
package bdhttp
import (
"context"
"net/http"
"github.com/steveyegge/beads/backend"
"github.com/steveyegge/beads/internal/httpapi/apigen"
"github.com/steveyegge/beads/internal/httpclient"
"github.com/steveyegge/beads/internal/storage/backends"
)
// UserAgentSuffix identifies this backend inside a stamped User-Agent, so a
// server log line says both which binary and which client spoke to it. An
// embedder that stamps its own build joins the two the way cmd/bd does:
// "myapp/1.4.2 " + bdhttp.UserAgentSuffix.
const UserAgentSuffix = httpclient.WireUserAgentSuffix
// Options configures the transport Register installs and Open/Handshake dial
// with.
//
// The zero value is usable and is what the CLI's own wiring amounts to: the
// built-in bearer credential ladder, this module's default User-Agent, and
// the standard http transport.
type Options struct {
// UserAgent identifies the embedding build on every request. Empty sends
// UserAgentSuffix alone, which names the backend but not the binary.
UserAgent string
// HTTPClient replaces the transport — a tuned timeout, a proxy, a custom
// TLS config, an httptest server in a test. Nil builds one. The client is
// used as given except for redirect handling, which the underlying wire
// client always forces off: a followed 30x would replay the Authorization
// header at whatever host the Location named.
HTTPClient *http.Client
// RequireCredential makes the registered backend's OpenWith seam refuse a
// registered open with no OpenOptions.Credential, rather than silently
// falling back to the ambient bearer ladder (BEADS_HTTP_TOKEN,
// BEADS_HTTP_TOKEN_COMMAND, the credentials file). It has no effect on
// Open/Handshake below, or on the plain Open/OpenReadOnly path a
// single-tenant CLI process uses: both keep the ambient ladder as their
// documented, legitimate posture.
//
// Set this when every caller through beads.OpenBestAvailableWith is a
// multi-tenant embedder for which ambient, process-global credential
// state can never safely stand in for one workspace's own — see
// httpclient.ResolveCredential and backends.OpenOptions.Credential's own
// doc comment.
RequireCredential bool
}
// Register adds the http store to the backend registry under the name "http"
// AND installs the transport it dials with, so a workspace whose
// metadata.json selects the backend opens instead of failing with
// httpclient.ErrNoTransport.
//
// The EMBEDDER calls this explicitly at process start, on the same terms as
// cmd/bd's own wiring: registration is a property of the distribution being
// built, not of the import graph. A plain `go build` of an OSS binary that
// imports this package only transitively, and never calls Register, gains no
// selectable "http" backend — metadata.json naming it still hard-fails with
// the registry's UnknownBackendError, exactly as before this package existed.
//
// Register panics on a duplicate registration, like any process-start wiring
// error, so it must have exactly one production call site. Call it once,
// before any concurrent store access.
func Register(opts Options) {
base := httpclient.DialOptions{UserAgent: opts.UserAgent, HTTPClient: opts.HTTPClient}
backends.Register(httpclient.Backend, backends.Backend{
Open: httpclient.NewFromConfig,
OpenReadOnly: httpclient.NewReadOnlyFromConfig,
WorkspaceIsBeadsDir: true,
Remote: true,
OpenWith: func(ctx context.Context, beadsDir string, bopts backends.OpenOptions) (backend.DoltStorage, error) {
return httpclient.OpenWith(ctx, beadsDir, bopts, base, opts.RequireCredential)
},
})
httpclient.RegisterDefaultDialer(base)
}
// Open dials target and returns a store for it, with no workspace on disk.
//
// It is the door for an embedder that already knows its server — from its own
// configuration, or minted per tenant — and would otherwise have to write a
// .beads directory for the sole benefit of a registry lookup. Registration is
// not required and does not affect it: Open dials fresh from the Options it is
// handed, using the same built-in bearer ladder Open/OpenReadOnly use (see
// Options.RequireCredential's doc for why a per-tenant credential belongs on
// the registry's OpenWith seam instead, via beads.OpenBestAvailableWith).
//
// The store carries no local metadata file, which is benign: the two
// per-user keys it would hold read as unset and write nowhere.
func Open(ctx context.Context, target Target, opts Options) (backend.DoltStorage, error) {
dialOpts := httpclient.DialOptionsForTarget(target, httpclient.DialOptions{UserAgent: opts.UserAgent, HTTPClient: opts.HTTPClient})
creds := httpclient.NewBearerProvider(target.BaseURL)
conn, err := httpclient.DialWith(target, creds, dialOpts)
if err != nil {
return nil, err
}
return httpclient.New(target, conn, nil), nil
}
// Handshake dials target once and returns its startup snapshot, applying the
// two gates a store applies: api_version equality and, when the target pins
// one, project identity.
//
// It is the pre-connect probe: `bd connect` verifies a server before it
// writes anything, and it has to do that without opening a store, because the
// workspace it is about to describe may not select this backend yet. A
// mismatch comes back as *ProjectMismatchError, which names both ids plus the
// server's own database and repo root.
//
// It dials with the same credentials Open would, so a probe can never verify
// a server the store then cannot reach.
func Handshake(ctx context.Context, target Target, opts Options) (*ServerSnapshot, error) {
dialOpts := httpclient.DialOptionsForTarget(target, httpclient.DialOptions{UserAgent: opts.UserAgent, HTTPClient: opts.HTTPClient})
// httpclient.Handshake dials via httpclient.Dial, which binds the same
// built-in bearer ladder (httpclient.NewBearerProvider) Open uses above —
// "the same credentials Open would use" holds without restating the dial
// here.
body, err := httpclient.Handshake(ctx, target, dialOpts)
if err != nil {
return nil, err
}
return newServerSnapshot(body), nil
}
// ServerSnapshot is what a server says about itself at handshake, curated for
// this door.
//
// It is a struct of its own rather than an alias of the wire document, and
// the omissions are the point. The wire type is GENERATED from the OpenAPI
// spec, so aliasing it would make a codegen bump — a field renamed, a member
// added — a breaking change to a published Go API, decided by a document
// nobody edits with that in mind. And the document carries facts an embedder
// has no business depending on: the server's own filesystem paths, its
// storage mode, its logical database name, the CLI's JSON schema version.
// Those are the operator's, and several are host paths a multi-tenant caller
// should not be holding at all.
//
// What remains is what a client actually decides with: Capabilities is the
// field it decides with (an operation is available because the server
// advertises its token, never because a version string looked new enough),
// and WireRevision is the revision the handshake already checked against
// this client's own compiled one before returning a snapshot at all.
type ServerSnapshot struct {
// APIVersion is the path major the server serves. The handshake already
// refused anything this client cannot address, so on a returned snapshot
// this is confirmation rather than a branch.
APIVersion string
// BdVersion is the release of the serving binary. Diagnostic and
// human-facing: branch on Capabilities, not on this.
BdVersion string
// WireRevision is the server's own wire_revision counter (see
// ContextResponse.WireRevision). The wire handshake itself refuses a
// server this client build predates (a wire_revision or
// min_client_wire_revision above wire.ClientWireRevision) before any
// snapshot is returned, so on a returned snapshot this is exposed for
// diagnostics, not a branch.
WireRevision int
// ProjectID is the workspace identity the server owns. When the Target
// pinned one, the handshake has already proved they match.
ProjectID string
// Capabilities are the operation tokens the server implements. The list
// grows additively and an operation never appears unless it is fully
// implemented, which is what makes membership the right question.
Capabilities []string
}
// newServerSnapshot maps the wire document onto the curated one.
//
// The capability slice is copied because the document it comes from is a
// pointer INTO the wire client's cached handshake, which every later dispatch
// reads to decide what the server can serve, and whose documented contract is
// that the caller treats it as read-only.
func newServerSnapshot(body *apigen.ContextResponse) *ServerSnapshot {
if body == nil {
return nil
}
return &ServerSnapshot{
APIVersion: body.ApiVersion,
BdVersion: body.BdVersion,
WireRevision: body.WireRevision,
ProjectID: body.ProjectId,
Capabilities: append([]string(nil), body.Capabilities...),
}
}

235
backend/http/open_test.go Normal file
View file

@ -0,0 +1,235 @@
package bdhttp_test
// Adapted from bd-enterprise backend/http/open_test.go under the MIT
// contribution: TestOpenNeedsNoWorkspaceOnDisk, the ServerSnapshot field-set
// drift guard, TestHandshakeReportsAWrongServer, and the activation sidecar
// round trip carry the same reasoning. Dropped or reshaped for OSS: the
// Credentials-hook tests (OSS Open/Handshake dial through the built-in bearer
// ladder only — see Options.RequireCredential's doc — there is no per-call
// hook to assert on here), ServerInfo / lazy ServerContext (the OSS Store
// does not expose that interface), and ServerSnapshot's field set itself,
// which follows DESIGN.txt exactly (WireRevision in place of the ORM-backed
// field bd-enterprise used). No bd-enterprise-specific wording carries over.
import (
"context"
"errors"
"net/http"
"net/url"
"path/filepath"
"reflect"
"testing"
bdhttp "github.com/steveyegge/beads/backend/http"
)
// TestOpenNeedsNoWorkspaceOnDisk is the programmatic door's reason to exist.
// An embedder that already knows the server — it read the URL from its own
// config, or minted a per-tenant one — has no .beads directory to point the
// registry at, and creating one just to open a store would be a file written
// for the benefit of a lookup.
func TestOpenNeedsNoWorkspaceOnDisk(t *testing.T) {
hermeticEnv(t)
t.Setenv(bdhttp.TokenEnv, "127.0.0.1=direct-token")
server := newRecordingServer(t)
store, err := bdhttp.Open(context.Background(), server.target(t), bdhttp.Options{HTTPClient: server.Client()})
if err != nil {
t.Fatalf("Open: %v", err)
}
t.Cleanup(func() { _ = store.Close() })
got, err := store.GetMetadata(context.Background(), "_project_id")
if err != nil {
t.Fatalf("GetMetadata: %v", err)
}
if got != projectID {
t.Errorf("project id = %q, want %q", got, projectID)
}
server.wantAuthorization(t, "Bearer direct-token")
}
// TestOpenWithoutRegistrationStillDials pins the independence of the two
// doors: Open dials from the Options it is handed, with no dependence on
// Register ever having run.
func TestOpenWithoutRegistrationStillDials(t *testing.T) {
hermeticEnv(t)
t.Setenv(bdhttp.TokenEnv, "127.0.0.1=unregistered")
server := newRecordingServer(t)
store, err := bdhttp.Open(context.Background(), server.target(t), bdhttp.Options{HTTPClient: server.Client()})
if err != nil {
t.Fatalf("Open without Register: %v", err)
}
t.Cleanup(func() { _ = store.Close() })
if _, err := store.GetMetadata(context.Background(), "_project_id"); err != nil {
t.Fatalf("GetMetadata: %v", err)
}
server.wantAuthorization(t, "Bearer unregistered")
}
// TestHandshakeVerifiesBeforeAnythingIsWritten is the connect path's probe. A
// caller about to record a server has to be able to check it first, and doing
// that by opening a store would mean describing a workspace before knowing
// whether the server behind it is the right one.
func TestHandshakeVerifiesBeforeAnythingIsWritten(t *testing.T) {
hermeticEnv(t)
server := newRecordingServer(t)
snapshot, err := bdhttp.Handshake(context.Background(), server.target(t), bdhttp.Options{HTTPClient: server.Client()})
if err != nil {
t.Fatalf("Handshake: %v", err)
}
if snapshot.ProjectID != projectID {
t.Errorf("project id = %q, want %q", snapshot.ProjectID, projectID)
}
if snapshot.BdVersion != "9.9.9" {
t.Errorf("bd_version = %q, want the server's own", snapshot.BdVersion)
}
if snapshot.APIVersion != "v0" {
t.Errorf("api_version = %q, want the major this client speaks", snapshot.APIVersion)
}
if snapshot.WireRevision != 1 {
t.Errorf("wire_revision = %d, want the server's own", snapshot.WireRevision)
}
if len(snapshot.Capabilities) != 2 || snapshot.Capabilities[0] != "issues.get" {
t.Errorf("capabilities = %v, want the server's advertised list", snapshot.Capabilities)
}
}
// TestServerSnapshotExportsNoHostFacts is the drift guard on the curated
// snapshot, and it is a guard rather than a comment because the thing it
// prevents arrives by accident.
//
// The wire document is GENERATED from the OpenAPI spec. Aliasing it, or
// widening this struct to match it, would put a codegen bump — a renamed
// field, a new member — on a published Go API, and would hand every embedder
// the server's own filesystem paths. A multi-tenant caller holding another
// tenant's repo root is not a field it chose to read; it is a field it could
// not avoid.
func TestServerSnapshotExportsNoHostFacts(t *testing.T) {
want := map[string]bool{
"APIVersion": true,
"BdVersion": true,
"WireRevision": true,
"ProjectID": true,
"Capabilities": true,
}
typ := reflect.TypeOf(bdhttp.ServerSnapshot{})
if typ.PkgPath() != "github.com/steveyegge/beads/backend/http" {
t.Fatalf("ServerSnapshot is declared in %q: it must be this package's own struct, not an alias of the generated wire type", typ.PkgPath())
}
for i := 0; i < typ.NumField(); i++ {
name := typ.Field(i).Name
if !want[name] {
t.Errorf("ServerSnapshot exports %q, which is not on the curated set: server host paths, storage mode, logical database and the CLI schema version stay behind this door", name)
continue
}
delete(want, name)
}
for name := range want {
t.Errorf("ServerSnapshot no longer carries %q", name)
}
}
// TestHandshakeReportsAWrongServer pins the gate that makes the probe worth
// running: the pinned identity is checked, and the refusal is typed, so a
// caller can tell "that is the wrong server" from "that server is down"
// without matching text.
func TestHandshakeReportsAWrongServer(t *testing.T) {
hermeticEnv(t)
server := newRecordingServer(t)
target := server.target(t)
target.ExpectProjectID = "proj-somewhere-else"
_, err := bdhttp.Handshake(context.Background(), target, bdhttp.Options{HTTPClient: server.Client()})
if !errors.Is(err, bdhttp.ErrProjectMismatch) {
t.Fatalf("Handshake against a wrong server = %v, want ErrProjectMismatch", err)
}
var mismatch *bdhttp.ProjectMismatchError
if !errors.As(err, &mismatch) {
t.Fatalf("the refusal is not a *ProjectMismatchError: %v", err)
}
}
// TestHandshakeUsesTheSameLadderOpenWouldUse keeps the two doors on one
// credential path: a probe that authorized differently from the store it
// precedes would verify a server the store then cannot reach.
func TestHandshakeUsesTheSameLadderOpenWouldUse(t *testing.T) {
hermeticEnv(t)
t.Setenv(bdhttp.TokenEnv, "127.0.0.1=probe-token")
server := newRecordingServer(t)
if _, err := bdhttp.Handshake(context.Background(), server.target(t), bdhttp.Options{HTTPClient: server.Client()}); err != nil {
t.Fatalf("Handshake: %v", err)
}
server.wantAuthorization(t, "Bearer probe-token")
}
// TestTheActivationSidecarRoundTrips covers the workspace half of the
// surface: an embedder that writes a connected workspace for a later `bd`
// invocation, or reads the one `bd connect` already wrote, goes through
// these.
func TestTheActivationSidecarRoundTrips(t *testing.T) {
beadsDir := t.TempDir()
if _, err := bdhttp.LoadTarget(beadsDir); !errors.Is(err, bdhttp.ErrNotConnected) {
t.Fatalf("LoadTarget on an unconnected workspace = %v, want ErrNotConnected", err)
}
if removed, err := bdhttp.RemoveTarget(beadsDir); err != nil || removed {
t.Fatalf("RemoveTarget on an unconnected workspace = (%v, %v), want (false, nil)", removed, err)
}
base, err := url.Parse("https://beads.example.com")
if err != nil {
t.Fatal(err)
}
want := bdhttp.Target{BaseURL: base, ExpectProjectID: projectID}
if err := bdhttp.SaveTarget(beadsDir, want); err != nil {
t.Fatalf("SaveTarget: %v", err)
}
if got := bdhttp.TargetPath(beadsDir); got != filepath.Join(beadsDir, bdhttp.TargetFileName) {
t.Errorf("TargetPath = %q, want the sidecar beside metadata.json", got)
}
got, err := bdhttp.LoadTarget(beadsDir)
if err != nil {
t.Fatalf("LoadTarget: %v", err)
}
if got.String() != want.String() || got.ExpectProjectID != want.ExpectProjectID {
t.Errorf("round trip = %+v, want %+v", got, want)
}
removed, err := bdhttp.RemoveTarget(beadsDir)
if err != nil {
t.Fatalf("RemoveTarget: %v", err)
}
if !removed {
t.Error("RemoveTarget did not report the sidecar it deleted")
}
if _, err := bdhttp.LoadTarget(beadsDir); !errors.Is(err, bdhttp.ErrNotConnected) {
t.Errorf("LoadTarget after RemoveTarget = %v, want ErrNotConnected", err)
}
}
// TestABearerProviderIsBuildableFromThePublicSurface proves the default
// ladder is nameable, which is what lets a caller wrap or inspect it without
// reimplementing it.
func TestABearerProviderIsBuildableFromThePublicSurface(t *testing.T) {
hermeticEnv(t)
t.Setenv(bdhttp.TokenEnv, "127.0.0.1=ladder-token")
server := newRecordingServer(t)
var provider bdhttp.CredentialProvider = bdhttp.NewBearerProvider(server.target(t).BaseURL)
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, server.URL, nil)
if err != nil {
t.Fatal(err)
}
if err := provider.Authorize(context.Background(), req); err != nil {
t.Fatalf("Authorize: %v", err)
}
if got := req.Header.Get("Authorization"); got != "Bearer ladder-token" {
t.Errorf("Authorization = %q, want the ladder's token", got)
}
}

View file

@ -0,0 +1,318 @@
package bdhttp_test
// Test helpers in this file are adapted from bd-enterprise
// backend/http/register_test.go under the MIT contribution: the registry
// double, the recording server/provider, and hermeticEnv's reasoning for
// taking the process environment out of credential decisions carry over
// directly. The Credentials-hook tests do not: OSS threads a per-open
// credential through backends.OpenOptions.Credential / httpclient.OpenWith
// instead of a bdhttp.Options.Credentials hook (no such hook exists on this
// path; see design D-OpenWith), so those tests are written fresh for this
// seam.
import (
"context"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"sync"
"testing"
"github.com/steveyegge/beads/backend"
bdhttp "github.com/steveyegge/beads/backend/http"
"github.com/steveyegge/beads/internal/storage/backends"
)
// The registry key is spelled literally, the way a workspace's metadata.json
// carries it: a test that read it from the code under test would stay green
// through a rename that orphans every existing connected workspace.
const backendName = "http"
// projectID is the workspace identity the stand-in server owns. Every target
// below pins it, so the handshake's identity gate is armed rather than
// skipped.
const projectID = "proj-embedder"
// TestRegisterPutsHTTPInTheRegistryAnEmbedderSees drives the reason this
// package exists: code holding nothing but the public surface can get "http"
// into the registry that store dispatch reads.
func TestRegisterPutsHTTPInTheRegistryAnEmbedderSees(t *testing.T) {
hermeticEnv(t)
if backend.Registered(backendName) {
t.Fatalf("%q was already registered before Register was called: linking this package must not register", backendName)
}
registerForTest(t, bdhttp.Options{})
registered, ok := backend.Lookup(backendName)
if !ok {
t.Fatalf("Lookup(%q) found nothing after Register", backendName)
}
if registered.Open == nil {
t.Error("registered http backend has no read-write Open")
}
if registered.OpenReadOnly == nil {
t.Error("registered http backend has no read-only Open")
}
if !registered.WorkspaceIsBeadsDir {
t.Error("an http workspace is the .beads directory alone; there is no local database to discover")
}
if !registered.Remote {
t.Error("the http backend is a pure network client with no local database; Remote must be set")
}
if registered.OpenWith == nil {
t.Error("registered http backend has no per-open OpenWith seam")
}
if !backend.WorkspaceIsBeadsDir(backendName) {
t.Error("backend.WorkspaceIsBeadsDir disagrees with the registration")
}
if !backend.IsRemote(backendName) {
t.Error("backend.IsRemote disagrees with the registration")
}
}
// TestRegisterTwicePanics pins the decision not to make this idempotent. A
// second call is a double-wired process start, and the registry's panic is
// the only place that ever says so.
func TestRegisterTwicePanics(t *testing.T) {
hermeticEnv(t)
registerForTest(t, bdhttp.Options{})
if !panics(func() { bdhttp.Register(bdhttp.Options{}) }) {
t.Fatal("a second Register returned instead of panicking: duplicate process-start wiring must not be absorbed")
}
if !backend.Registered(backendName) {
t.Fatalf("the refused duplicate took %q out of the registry", backendName)
}
}
// TestRegisterInstallsTheTransport is the whole reason Register takes
// Options: registration alone leaves the store's package-level dialer nil,
// and every open then fails with httpclient.ErrNoTransport — a two-step an
// embedder cannot complete, because the second step is an internal function.
func TestRegisterInstallsTheTransport(t *testing.T) {
hermeticEnv(t)
server := newRecordingServer(t)
registerForTest(t, bdhttp.Options{HTTPClient: server.Client()})
store := openWorkspace(t, connectedWorkspace(t, server))
got, err := store.GetMetadata(context.Background(), "_project_id")
if err != nil {
t.Fatalf("GetMetadata over the registered backend: %v", err)
}
if got != projectID {
t.Errorf("project id = %q, want %q: the store never reached the server", got, projectID)
}
}
// TestOpenWithHonorsAProvidedCredential pins the per-tenant seam Register
// installs: a caller that supplies backends.OpenOptions.Credential (wrapped
// in bdhttp.ProvidedCredential) reaches the server with THAT credential, not
// the ambient ladder, even though the registered backend was built with no
// credential of its own.
func TestOpenWithHonorsAProvidedCredential(t *testing.T) {
hermeticEnv(t)
server := newRecordingServer(t)
registerForTest(t, bdhttp.Options{HTTPClient: server.Client()})
beadsDir := connectedWorkspace(t, server)
registered, ok := backend.Lookup(backendName)
if !ok {
t.Fatal("the http backend is not registered")
}
provider := &recordingProvider{token: "per-tenant-token"}
store, err := registered.OpenWith(context.Background(), beadsDir, backends.OpenOptions{
Credential: bdhttp.ProvidedCredential{Provider: provider},
})
if err != nil {
t.Fatalf("OpenWith: %v", err)
}
t.Cleanup(func() { _ = store.Close() })
if _, err := store.GetMetadata(context.Background(), "_project_id"); err != nil {
t.Fatalf("GetMetadata: %v", err)
}
server.wantAuthorization(t, "Bearer per-tenant-token")
}
// TestOpenWithFallsBackToTheAmbientLadderByDefault pins the single-tenant
// default: a registered open with no Credential still authorizes, through
// the same built-in bearer ladder Open/OpenReadOnly use — RequireCredential
// is what opts OUT of this, not the zero value.
func TestOpenWithFallsBackToTheAmbientLadderByDefault(t *testing.T) {
hermeticEnv(t)
t.Setenv(bdhttp.TokenEnv, "127.0.0.1=ambient-token")
server := newRecordingServer(t)
registerForTest(t, bdhttp.Options{HTTPClient: server.Client()})
beadsDir := connectedWorkspace(t, server)
store := openWorkspace(t, beadsDir)
if _, err := store.GetMetadata(context.Background(), "_project_id"); err != nil {
t.Fatalf("GetMetadata: %v", err)
}
server.wantAuthorization(t, "Bearer ambient-token")
}
// TestRequireCredentialRefusesAnOpenWithNone is the fail-closed half: a
// multi-tenant embedder that sets RequireCredential must never have a
// workspace silently fall back to whatever the ambient process environment
// happens to hold, because that environment belongs to no one tenant.
func TestRequireCredentialRefusesAnOpenWithNone(t *testing.T) {
hermeticEnv(t)
server := newRecordingServer(t)
registerForTest(t, bdhttp.Options{HTTPClient: server.Client(), RequireCredential: true})
beadsDir := connectedWorkspace(t, server)
registered, ok := backend.Lookup(backendName)
if !ok {
t.Fatal("the http backend is not registered")
}
_, err := registered.OpenWith(context.Background(), beadsDir, backends.OpenOptions{})
if err == nil {
t.Fatal("OpenWith with RequireCredential and no Credential succeeded; want a refusal")
}
server.wantNoRequests(t)
}
func panics(fn func()) (panicked bool) {
defer func() { panicked = recover() != nil }()
fn()
return false
}
// registerForTest wires the backend for one test and gives the process-global
// registry back afterwards, the isolation seam backend.Deregister exists for.
func registerForTest(t *testing.T, opts bdhttp.Options) {
t.Helper()
bdhttp.Register(opts)
t.Cleanup(func() { backend.Deregister(backendName) })
}
// connectedWorkspace is a .beads directory attached to server, as `bd
// connect` would leave it.
func connectedWorkspace(t *testing.T, server *recordingServer) string {
t.Helper()
beadsDir := filepath.Join(t.TempDir(), ".beads")
if err := os.MkdirAll(beadsDir, 0o700); err != nil {
t.Fatalf("create workspace: %v", err)
}
if err := bdhttp.SaveTarget(beadsDir, server.target(t)); err != nil {
t.Fatalf("save the activation sidecar: %v", err)
}
return beadsDir
}
// openWorkspace opens beadsDir through the REGISTERED backend rather than
// through this package's own Open, so what it exercises is the dialer
// Register installed.
func openWorkspace(t *testing.T, beadsDir string) backend.DoltStorage {
t.Helper()
registered, ok := backend.Lookup(backendName)
if !ok {
t.Fatal("the http backend is not registered")
}
store, err := registered.Open(context.Background(), beadsDir)
if err != nil {
t.Fatalf("open the connected workspace: %v", err)
}
t.Cleanup(func() { _ = store.Close() })
return store
}
// hermeticEnv takes the process environment out of the credential decision.
// Every rung of the built-in ladder is read from the environment at dial
// time, so a developer machine that exports one would otherwise change which
// source a test exercises.
func hermeticEnv(t *testing.T) {
t.Helper()
t.Setenv(bdhttp.TokenEnv, "")
t.Setenv(bdhttp.TokenCommandEnv, "")
}
// recordingProvider is a CredentialProvider that signs with a fixed token.
type recordingProvider struct {
token string
mu sync.Mutex
calls int
}
func (p *recordingProvider) Authorize(_ context.Context, req *http.Request) error {
p.mu.Lock()
p.calls++
p.mu.Unlock()
req.Header.Set("Authorization", "Bearer "+p.token)
return nil
}
func (p *recordingProvider) Refresh(context.Context) (bool, error) { return false, nil }
// recordingServer is a bd serve stand-in that answers the v0 handshake and
// records what each request carried in its Authorization header.
type recordingServer struct {
*httptest.Server
mu sync.Mutex
auths []string
}
func newRecordingServer(t *testing.T) *recordingServer {
t.Helper()
s := &recordingServer{}
s.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
s.mu.Lock()
s.auths = append(s.auths, r.Header.Get("Authorization"))
s.mu.Unlock()
if r.URL.Path != "/v0/beads/context" {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"api_version":"v0","backend":"dolt","bd_version":"9.9.9",` +
`"beads_dir":"/srv/.beads","capabilities":["issues.get","issues.list"],` +
`"database":"beads","dolt_mode":"server","project_id":"` + projectID + `","repo_root":"/srv",` +
`"schema_version":1,"wire_revision":1,"min_client_wire_revision":1}`))
}))
t.Cleanup(s.Server.Close)
return s
}
func (s *recordingServer) target(t *testing.T) bdhttp.Target {
t.Helper()
base, err := url.Parse(s.URL)
if err != nil {
t.Fatalf("parse the test server URL: %v", err)
}
return bdhttp.Target{BaseURL: base, ExpectProjectID: projectID}
}
// wantAuthorization asserts that every request the server saw carried want.
// It is every request rather than any request because a credential that
// reaches only some of them is the same defect as one that reaches none.
func (s *recordingServer) wantAuthorization(t *testing.T, want string) {
t.Helper()
s.mu.Lock()
defer s.mu.Unlock()
if len(s.auths) == 0 {
t.Fatal("the server saw no requests at all")
}
for i, got := range s.auths {
if got != want {
t.Errorf("request %d carried Authorization %q, want %q", i, got, want)
}
}
}
// wantNoRequests is the fail-closed half: a refusal that still dialed would
// have sent whatever credential it fell back to, which is the outcome being
// refused.
func (s *recordingServer) wantNoRequests(t *testing.T) {
t.Helper()
s.mu.Lock()
defer s.mu.Unlock()
if len(s.auths) != 0 {
t.Errorf("the server saw %d requests, want none: the dial was supposed to fail closed", len(s.auths))
}
}

41
backend/http/target.go Normal file
View file

@ -0,0 +1,41 @@
package bdhttp
import "github.com/steveyegge/beads/internal/httpclient"
// Target is the resolved endpoint for one workspace: the server's mount root
// and, optionally, the workspace identity pinned when it was connected. A
// pinned id is compared against the server's own at every handshake, which is
// what turns "some server answered" into "the server that owns this workspace
// answered".
type Target = httpclient.Target
// TargetFileName is the per-user, untracked activation sidecar that names the
// server a workspace is attached to. metadata.json is git-tracked by design
// and selects the backend; the URL and the identity pin are per-user state
// and live beside it.
const TargetFileName = httpclient.TargetFileName
// ErrNotConnected reports a workspace that selects this backend but has no
// activation sidecar, so there is no server to dial. It is what an Open
// through the registry returns for a workspace nobody ran `bd connect` in.
var ErrNotConnected = httpclient.ErrNotConnected
// TargetPath is the sidecar's location for a workspace, given its .beads
// directory.
func TargetPath(beadsDir string) string { return httpclient.TargetPath(beadsDir) }
// LoadTarget reads the activation sidecar. A missing one is ErrNotConnected
// rather than a zero Target: an http workspace with no target cannot be
// opened at all.
func LoadTarget(beadsDir string) (Target, error) { return httpclient.LoadTarget(beadsDir) }
// SaveTarget writes the sidecar 0600, beside metadata.json. It is the encoder
// `bd connect` uses; the connect UX around it — gitignore coverage, identity
// verification, conversion consent — is the CLI's, so an embedder that writes
// a workspace this way owns those decisions itself.
func SaveTarget(beadsDir string, t Target) error { return httpclient.SaveTarget(beadsDir, t) }
// RemoveTarget deletes the activation sidecar, reporting whether one was
// there. Detaching is a per-user act, so it never touches the tracked
// metadata.json that selects the backend.
func RemoveTarget(beadsDir string) (bool, error) { return httpclient.RemoveTarget(beadsDir) }

View file

@ -62,6 +62,7 @@ go_library(
"config_show.go",
"config_side_effects.go",
"conflicts.go",
"connect.go",
"context.go",
"context_cmd.go",
"context_proxied_server.go",
@ -157,6 +158,7 @@ go_library(
"history.go",
"history_proxied_server.go",
"hooks.go",
"http_backend.go",
"human.go",
"human_proxied_server.go",
"if_revision.go",
@ -366,6 +368,7 @@ go_library(
visibility = ["//visibility:private"],
deps = [
"//:beads",
"//backend/http",
"//cmd/bd/doctor",
"//cmd/bd/doctor/fix",
"//cmd/bd/setup",
@ -392,6 +395,7 @@ go_library(
"//internal/gitlab",
"//internal/hooks",
"//internal/httpapi",
"//internal/httpclient",
"//internal/jira",
"//internal/linear",
"//internal/lockfile",
@ -493,6 +497,7 @@ go_binary(
embed = [":bd_lib"],
race = "off",
visibility = [
"//backend/http:__pkg__",
"//cmd/bd:__subpackages__",
"//internal/beads:__pkg__",
"//internal/storage/uow:__pkg__",

View file

@ -511,7 +511,7 @@ var proxyPermittedPaths = []string{
// workspace lifecycle and configuration
"bootstrap", "config apply", "config drift", "config get", "config list", "config set",
"config set-many", "config show", "config unset", "config validate", "context",
"config set-many", "config show", "config unset", "config validate", "connect", "context",
"init", "init-safety", "onboard", "preflight", "quickstart", "schema", "setup", "where",
"version", "worktree create", "worktree info", "worktree list", "worktree remove",

324
cmd/bd/connect.go Normal file
View file

@ -0,0 +1,324 @@
package main
import (
"context"
"errors"
"fmt"
"net/url"
"os"
"path/filepath"
"github.com/spf13/cobra"
bdhttp "github.com/steveyegge/beads/backend/http"
"github.com/steveyegge/beads/cmd/bd/doctor"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/httpclient"
"github.com/steveyegge/beads/internal/metrics"
)
// Written fresh for OSS beads S6 (no bd-enterprise source copied): richer
// connect flows (interactive login, server discovery, storage-profile
// prompts) are explicitly out of scope for this lift (see DESIGN.txt sec
// 3.1). This is the minimal OSS connect DESIGN.txt asks for: verify a
// server, then record it.
var (
connectExpectProjectID string
connectCAFile string
connectForce bool
connectAllowPlaintext bool
connectClear bool
)
var connectCmd = &cobra.Command{
Use: "connect [url]",
GroupID: "setup",
Short: "Attach this workspace to a remote bd serve over HTTP",
Long: `Attach this workspace to a remote bd serve over HTTP.
bd connect verifies the server first (a handshake: api_version, wire
revision, and — when --expect-project-id is given — workspace identity) and
writes nothing until that succeeds. On success it writes the per-user
activation sidecar (.beads/http_target.json, never git-tracked) and sets this
workspace's metadata.json to backend "http".
A credential, if the server requires one, is never read from a flag or
recorded on disk by this command. It comes from the same ladder every http
request does: the BEADS_HTTP_TOKEN environment variable, then
BEADS_HTTP_TOKEN_COMMAND (a helper that prints a token), then the credentials
file's [host:port] section, then no credential at all — which the tip OSS
server's loopback-trust posture answers legitimately, not as a failure. Both
variables name the server they are for, host[:port]=value (for example
BEADS_HTTP_TOKEN=bd.example.com=<token>); a bare value is refused, since it
would be sent to whatever server the workspace's http_target.json names.
--allow-plaintext IS recorded on disk, in the sidecar alongside the server
url: once granted here, a credential may cross this one server in the clear
on every later command too, not only this one, without re-passing the flag
or setting BEADS_HTTP_ALLOW_INSECURE=1 yourself. A later connect to a
DIFFERENT url starts that grant over; it is never carried forward to a
server it was not given for.
Examples:
bd connect http://127.0.0.1:8080 # loopback, no TLS needed
bd connect https://bd.example.com --expect-project-id p1 # pin workspace identity
bd connect https://bd.example.com --ca-file ./ca.pem # trust only this CA
bd connect --clear # forget the server, restoring the previous backend
`,
Args: cobra.MaximumNArgs(1),
SilenceUsage: true,
SilenceErrors: true,
// connect is how an http workspace gets its FIRST local state: there is
// no local database to find yet (none ever exists for this backend), so
// it opts out of PersistentPreRunE's store init the same way doctor,
// schema and worktree do, via skipStoreAnnotation rather than editing
// the central noDbCommands list (see that annotation's own doc comment).
Annotations: map[string]string{skipStoreAnnotation: "1"},
RunE: func(cmd *cobra.Command, args []string) error {
evt := metrics.NewCommandEvent("connect")
defer func() {
if c := metrics.Global(); c != nil {
c.CloseEventAndAdd(evt)
}
}()
// connect, like init, is how a workspace gets its FIRST local state:
// there may be no .beads directory yet anywhere above the CWD, so
// this resolves the same way init's own target does (BEADS_DIR, then
// the worktree fallback, then CWD/.beads) rather than
// beads.GetRepoContextAllowingNoGit(), which answers "no .beads
// directory found" with a hard error because every OTHER caller of
// it (bd context, the proxied context route) is read-only and has
// nothing to create.
beadsDir := resolveInitBeadsDir()
if beadsDir == "" {
return HandleError("cannot resolve a workspace directory for connect (set BEADS_DIR, or run from inside a writable directory)")
}
if connectClear {
if len(args) != 0 {
return HandleError("connect --clear takes no url argument")
}
// Read the sidecar's PreviousBackend BEFORE removing it: it is
// the backend metadata.json selected right before this
// workspace connected (bee-ghosttrack CHANGES_REQUESTED on
// #7288, should-fix 1). A missing sidecar (ErrNotConnected) or
// any other read failure just means there is nothing to
// restore, matching the "nothing was configured" case below.
priorTarget, loadErr := httpclient.LoadTarget(beadsDir)
removed, err := httpclient.RemoveTarget(beadsDir)
if err != nil {
return HandleError("clearing %s: %v", httpclient.TargetFileName, err)
}
restoredBackend := ""
if removed && loadErr == nil && priorTarget.PreviousBackend != "" {
if cfg, cfgErr := configfile.Load(beadsDir); cfgErr == nil && cfg != nil && cfg.GetBackend() == httpclient.Backend {
cfg.Backend = priorTarget.PreviousBackend
if saveErr := cfg.Save(beadsDir); saveErr == nil {
restoredBackend = priorTarget.PreviousBackend
}
// A save failure is not fatal to --clear: the sidecar is
// already gone (or never existed), which is the part
// that actually detaches the workspace from the
// server; metadata.json just keeps saying "http" and
// the manual fallback message below still applies.
}
}
if jsonOutput {
return outputJSON(map[string]any{"cleared": removed, "restored_backend": restoredBackend})
}
switch {
case restoredBackend != "":
fmt.Printf("Removed %s and restored this workspace's backend selection in metadata.json to %q.\n", httpclient.TargetFileName, restoredBackend)
case removed:
// Not `bd config set backend ...`: that writes the database's
// own config table, never metadata.json, and must first open
// the very store this workspace can no longer reach.
fmt.Printf("Removed %s. This workspace's backend selection in metadata.json is unchanged; run `bd connect <url>` to reconnect, or set \"backend\" in %s to pick a different backend.\n", httpclient.TargetFileName, filepath.Join(beadsDir, configfile.ConfigFileName))
default:
fmt.Println("No http target was configured; nothing to clear.")
}
return nil
}
if len(args) != 1 {
return HandleError("connect requires a <url> argument (or --clear)")
}
rawURL := args[0]
parsed, err := url.Parse(rawURL)
if err != nil || parsed.Host == "" {
// A parse failure has no *url.URL to redact, so there is nothing
// safer to print than the raw argument; a parse that succeeded
// but found no host (e.g. "http://user:pass@/path") does have
// one, and .Redacted() strips its userinfo before this message
// repeats it.
if err != nil {
return HandleError("invalid url %q", rawURL)
}
return HandleError("invalid url %q", parsed.Redacted())
}
// redactedURL is what every print/wrap of the connect target uses from
// here on: rawURL is the user's literal argument, which — unlike
// parsed.Redacted() — still carries a userinfo password verbatim
// (http://user:pass@host/...) if one was pasted into the URL. Nothing
// past this point should format rawURL again.
redactedURL := parsed.Redacted()
if parsed.Scheme != "http" && parsed.Scheme != "https" {
return HandleError("url %q must use http or https, got %q", redactedURL, parsed.Scheme)
}
// L1 (host safety, functional requirement 1): a bearer credential sent
// in the clear to anything but loopback is a token leaked to every
// network hop between here and the server. Refuse by default; require
// an explicit opt-in rather than a warning nobody reads at connect
// time, since connect time is the one moment a human is looking.
if parsed.Scheme == "http" && !configfile.IsLocalHostString(parsed.Hostname()) && !connectAllowPlaintext {
return HandleError(
"refusing to connect to %s over plain http: %s is not loopback, and any bearer credential this workspace sends would cross the network unencrypted; use https://, or pass --allow-plaintext to connect anyway",
redactedURL, parsed.Hostname())
}
existing, err := configfile.Load(beadsDir)
if err != nil {
return HandleError("reading %s: %v", configfile.ConfigFileName, err)
}
if existing != nil {
if b := existing.GetBackend(); b != "" && b != httpclient.Backend && !connectForce {
return HandleError("this workspace already selects backend %q; pass --force to switch it to %q (existing local data, if any, is left in place but will no longer be used)", b, httpclient.Backend)
}
}
// effectiveExpectedProjectID carries the PRIOR sidecar's pin forward
// into this run, so a bare re-connect (no --expect-project-id) to a
// DIFFERENT server still gets caught: without this, every connect
// built a brand-new Target starting from connectExpectProjectID alone
// (almost always "" on a re-run), which the "nothing was pinned yet"
// branch below would then adopt from whatever server just answered —
// silently re-pinning the workspace to a different project with no
// refusal at all. --force (regardless of --expect-project-id) drops
// the prior pin entirely and lets the new server's answer win, which
// is the explicit "I mean it" escape hatch this check otherwise has
// none of.
effectiveExpectedProjectID := connectExpectProjectID
if !connectForce {
priorTarget, loadErr := httpclient.LoadTarget(beadsDir)
if loadErr != nil && !errors.Is(loadErr, httpclient.ErrNotConnected) {
return HandleError("reading existing %s: %v", httpclient.TargetFileName, loadErr)
}
if loadErr == nil && priorTarget.ExpectProjectID != "" {
switch {
case effectiveExpectedProjectID == "":
effectiveExpectedProjectID = priorTarget.ExpectProjectID
case effectiveExpectedProjectID != priorTarget.ExpectProjectID:
return HandleError(
"this workspace is already connected to project %q; pass --force to re-pin it to %q, or pass --expect-project-id %q to confirm the existing pin",
priorTarget.ExpectProjectID, effectiveExpectedProjectID, priorTarget.ExpectProjectID)
}
}
}
var caFile string
dialOpts := httpclient.DialOptions{
UserAgent: "bd/" + Version + " " + httpclient.WireUserAgentSuffix,
// The L1 refusal above already gated this exact combination
// (plain http, non-loopback) on --allow-plaintext; carrying the
// flag through here keeps this command's own Handshake probe
// (which dials with the same ambient bearer ladder a credential
// might come from) from re-refusing what the flag already
// permitted. See DialOptions.AllowInsecureCredential.
AllowInsecureCredential: connectAllowPlaintext,
}
if connectCAFile != "" {
abs, err := filepath.Abs(connectCAFile)
if err != nil {
return HandleError("resolving --ca-file %q: %v", connectCAFile, err)
}
caFile = abs
// Verify against EXACTLY this file, bypassing BEADS_HTTP_CA_FILE
// entirely: a connect run with the env var set to a different CA
// must not verify against the env's CA while writing the flag's
// unverified one to the sidecar (see DialOptionsForFile's doc).
dialOpts, err = httpclient.DialOptionsForFile(caFile, dialOpts)
if err != nil {
return HandleError("--ca-file %q: %v", connectCAFile, err)
}
}
target := httpclient.Target{BaseURL: parsed, ExpectProjectID: effectiveExpectedProjectID, CAFile: caFile, AllowInsecureCredential: connectAllowPlaintext}
ctx := context.Background()
snapshot, err := httpclient.Handshake(ctx, target, dialOpts)
if err != nil {
return HandleError("connecting to %s: %v", redactedURL, err)
}
if target.ExpectProjectID == "" {
// Nothing was pinned (a genuine first connect, or --force
// dropped the prior pin above): record what the server said so
// this workspace is tied to THIS project from here on.
target.ExpectProjectID = snapshot.ProjectId
} else if target.ExpectProjectID != snapshot.ProjectId {
// effectiveExpectedProjectID is either an explicit
// --expect-project-id or the prior sidecar's pin carried
// forward above; either way the server disagrees, and --force
// was not given (the only way to reach this line with a
// mismatched prior pin AND --force would have cleared
// effectiveExpectedProjectID's carry-forward, not left it set).
return HandleError("server at %s owns project %q, not the pinned %q", redactedURL, snapshot.ProjectId, target.ExpectProjectID)
}
if err := os.MkdirAll(beadsDir, 0o700); err != nil {
return HandleError("creating %s: %v", beadsDir, err)
}
// A fresh .beads directory (the common case: this backend has no
// local database to have created one earlier) has no .gitignore yet.
// Ensure one before writing the sidecar, the same way bd init and
// doctor --fix do, so http_target.json (which carries a bearer
// credential's host and, soon, DialOptions) is never a candidate for
// `git add .` in the first commit after connect.
if err := doctor.EnsureGitignoreForBeadsDir(beadsDir); err != nil {
return HandleError("ensuring %s/.gitignore: %v", beadsDir, err)
}
// bdhttp.Attach writes both of this backend's files: the gitignored
// identity sidecar (SaveTarget — which server, which project; see
// NewFromConfig/NewReadOnlyFromConfig in internal/httpclient/store.go,
// which read identity from the sidecar alone, never from
// metadata.json) and metadata.json's backend SELECTION alone
// (git-tracked, the same file every clone of this repo shares). It
// leaves every other existing metadata.json field untouched — in
// particular cfg.Database/cfg.ProjectID, which belong to whatever
// backend this workspace selected before, and which a future
// `bd connect --clear` switching back to that backend (dolt) still
// needs intact.
if err := bdhttp.Attach(beadsDir, target); err != nil {
return HandleError("%v", err)
}
if jsonOutput {
return outputJSON(map[string]any{
"url": redactedURL,
"project_id": target.ExpectProjectID,
"bd_version": snapshot.BdVersion,
"api_version": snapshot.ApiVersion,
"wire_revision": snapshot.WireRevision,
"ca_file": caFile,
"capabilities": snapshot.Capabilities,
})
}
fmt.Printf("Connected %s to %s\n", beadsDir, redactedURL)
fmt.Printf(" project: %s\n", target.ExpectProjectID)
fmt.Printf(" bd serve: %s (api %s, wire revision %d)\n", snapshot.BdVersion, snapshot.ApiVersion, snapshot.WireRevision)
if caFile != "" {
fmt.Printf(" ca_file: %s\n", caFile)
}
return nil
},
}
func init() {
connectCmd.Flags().StringVar(&connectExpectProjectID, "expect-project-id", "", "pin the workspace identity the server must present at every handshake")
connectCmd.Flags().StringVar(&connectCAFile, "ca-file", "", "trust ONLY this PEM file for this server (resolved and recorded as an absolute path)")
connectCmd.Flags().BoolVar(&connectForce, "force", false, "allow switching a workspace that already selects a different backend")
connectCmd.Flags().BoolVar(&connectAllowPlaintext, "allow-plaintext", false, "allow a plain http connection to a non-loopback host; remembered in the sidecar for every later command against this server")
connectCmd.Flags().BoolVar(&connectClear, "clear", false, "remove this workspace's http activation sidecar and restore its previous backend selection in metadata.json")
rootCmd.AddCommand(connectCmd)
}

View file

@ -7,6 +7,7 @@ import (
"github.com/steveyegge/beads/internal/beads"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/doltserver"
"github.com/steveyegge/beads/internal/httpclient"
"github.com/steveyegge/beads/internal/metrics"
"github.com/steveyegge/beads/internal/storage/domain"
)
@ -135,7 +136,19 @@ Examples:
// while each held its own copy of `Backend: configfile.BackendDolt`.
func applyContextBackend(snapshot *domain.ContextInfo, beadsDir string, cfg *configfile.Config) error {
snapshot.SetBackendIdentity(cfg.GetBackend(), cfg.GetDoltMode(), cfg.GetDoltDatabase())
snapshot.ProjectID = cfg.ProjectID
if cfg.GetBackend() == httpclient.Backend {
// The http backend's identity lives in the per-user, gitignored
// sidecar (`bd connect`'s MED-6 fix stopped mirroring it into the
// git-tracked metadata.json), so `bd context` must read it from
// there too. A missing sidecar (ErrNotConnected) or any other read
// failure just leaves ProjectID empty rather than failing this
// otherwise-degraded-state-tolerant command.
if target, err := httpclient.LoadTarget(beadsDir); err == nil {
snapshot.ProjectID = target.ExpectProjectID
}
} else {
snapshot.ProjectID = cfg.ProjectID
}
if cfg.IsDoltServerMode() {
snapshot.ServerHost = cfg.GetDoltServerHost()

View file

@ -41,8 +41,12 @@ push-state.json
proxied_server_client_info.json
# http backend activation sidecar (per-user server URL and identity pin)
# HTTP backend activation sidecar and per-user local metadata (per-user state,
# never tracked — the server URL and any ca_file path are machine-specific,
# and the local metadata sidecar mirrors proxied_server_client_info.json's
# per-user precedent). Written by bd connect.
http_target.json
http_local_metadata.json
# Worktree redirect file (contains relative path to main repo's .beads/)
# Must not be committed as paths would be wrong in other clones
@ -143,6 +147,7 @@ var requiredPatterns = []string{
".beads-credential-key",
"proxied_server_client_info.json",
"http_target.json",
"http_local_metadata.json",
".local_version",
"backup/",
}

16
cmd/bd/http_backend.go Normal file
View file

@ -0,0 +1,16 @@
package main
import bdhttp "github.com/steveyegge/beads/backend/http"
// Registering the http backend is a property of the bd distribution being
// built, not of the import graph (see bdhttp.Register's own doc comment): OSS
// ships `bd serve`, so OSS `bd` should be able to talk to it too, and this is
// the one production call site. A workspace whose metadata.json names any
// OTHER unregistered backend still hard-fails with the registry's
// UnknownBackendError, exactly as before this file existed — only "http"
// becomes selectable, and only because this file registers it.
func init() {
bdhttp.Register(bdhttp.Options{
UserAgent: "bd/" + Version + " " + bdhttp.UserAgentSuffix,
})
}

View file

@ -30,6 +30,7 @@ import (
"github.com/steveyegge/beads/internal/debug"
"github.com/steveyegge/beads/internal/doltserver"
"github.com/steveyegge/beads/internal/hooks"
"github.com/steveyegge/beads/internal/httpclient"
"github.com/steveyegge/beads/internal/metrics"
"github.com/steveyegge/beads/internal/migration"
"github.com/steveyegge/beads/internal/molecules"
@ -2757,6 +2758,28 @@ func validateWorkspaceIdentity(ctx context.Context, s storage.DoltStorage, beads
return nil // No config, skip validation (fresh init)
}
configProjectID := cfg.ProjectID
if cfg.GetBackend() == httpclient.Backend {
// The http backend's identity pin lives in the per-user, gitignored
// sidecar, not in metadata.json: `bd connect` (MED-6) stopped
// mirroring project_id into the git-tracked config, so cfg.ProjectID
// here is whatever the workspace carried BEFORE connect — generally
// a different project's id than the server `bd connect --force`
// just pointed it at. Comparing that stale value against the live
// database's _project_id below would flag every write as drift.
// Read the sidecar instead, exactly as applyContextBackend
// (context_cmd.go) does for `bd context`. A missing sidecar
// (ErrNotConnected) or any other read failure falls through to the
// "no project_id" skip just below, the same as a pre-identity
// workspace — the wire handshake's own ExpectProjectID gate (design
// D6) is what actually refuses a mismatched server at dial time;
// this check only needs to avoid a FALSE mismatch once that gate has
// already passed.
target, err := httpclient.LoadTarget(beadsDir)
if err != nil {
return nil
}
configProjectID = target.ExpectProjectID
}
if configProjectID == "" {
return nil // No project_id in config (pre-identity era)
}

View file

@ -40,6 +40,17 @@ func wireStorageDecorators(store storage.DoltStorage, hookRunner *hooks.Runner,
// wireExternalDependencyPolicy applies only the read/guard policy. Routed
// stores must use this without inheriting the caller's hooks or telemetry.
//
// It ALWAYS wraps, including a registered Remote backend (design 3.6): a
// remote server that enforces this policy itself answers the decorator's
// storage.ExternalDependencyPolicyProber probe (implemented by
// httpclient.Store, keyed on the handshake's wire.CapExternalDependencies
// token) with true, so the decorator's own loadBlockingState is a cheap no-op
// for that server — but a server that does NOT advertise the capability must
// still get client-side enforcement, which requires wrapping it in the first
// place. Skipping the wrap here for every remote backend (the pre-S6-fix
// behavior) silently dropped the policy entirely against such a server; the
// capability-aware skip now lives inside the decorator, not at wrap time.
func wireExternalDependencyPolicy(store storage.DoltStorage) storage.DoltStorage {
if store == nil {
return nil

View file

@ -47,7 +47,7 @@ the behavior.
## Summary
264 rows: 205 refuse, 33 degrade, 26 retired.
264 rows: 204 refuse, 34 degrade, 26 retired.
## Whole-behavior divergences (D9 ledger)
@ -62,13 +62,13 @@ The L-rows the design's D9 table names: whole behaviors that differ over HTTP, e
| L5 | degrade | — | molecules are not auto-loaded pre-run | the loader writes, the write refuses, and the call site swallows the refusal to debug.Logf | D9 L5, D4 | TODO(S3): a pre-run degradation, observable only through a cmd/bd invocation; the subprocess parity corpus pins it |
| L6 | degrade | — | a pre-run auto-import ATTEMPT reaches an http workspace and fails as a best-effort stderr warning; nothing is imported | THE MECHANISM THIS ROW USED TO NAME DOES NOT EXIST. It said the path was "force-disabled through an import.auto=false env override", and there is no such override anywhere: shouldRunAutoImportJSONL (cmd/bd/main.go) reads the config key, whose default is true (internal/config/config.go), and no http path sets it false. WHAT ACTUALLY HAPPENS is that maybeAutoImportJSONL runs — on any non-read-only, non-global, non-server-mode command — and is stopped by its own guards rather than by a switch: a missing or empty issues.jsonl, the attempt stamp, or the emptiness guard, which is a GetStatistics the server serves. Where all three pass (an empty server beside a non-empty local JSONL) the import is attempted and fails, because the client is not a jsonlImporter and the fallback importer's first write — SetConfig, or CreateIssuesWithFullOptions — is on the unsupported allowlist. The function is documented best-effort, so the failure is an stderr warning and the command proceeds. The DEGRADATION is therefore that nothing imports, which is the outcome the design wanted for the right reason — importing a local JSONL into a shared server is not a pre-run side effect — reached by refusal rather than by configuration. It stays a degrade because the command proceeds and the answer is not narrower or wider than the caller asked for. The retirement path is a real skip on the http backend, so the warning stops being the mechanism | D9 L6, D4 | TODO(S3): a pre-run degradation, observable only through a cmd/bd invocation; the subprocess parity corpus pins it |
| L7 | degrade | — | the client-side status/type vocabulary is defaults plus the status.custom/types.custom/types.infra config keys; table-backed customs are invisible client-side | the custom_statuses/custom_types tables have no wire operation; server-side role validation still uses the true vocabulary | D9 L7, D4 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L8 | refuse | `--deps` | `bd list --deps` refuses, and the pretty/--format dependency-decoration arms render undecorated | GetAllDependencyRecords has no wire mapping in v1; listDependencies is an anchored read. --tree is NOT in this row: it defaults true and is `bd list`'s default text rendering | D9 L8, D4 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L8 | degrade | `--deps` | `bd list --deps` and the policy decorator's compatibility fallback for its whole-workspace blocking state (design 3.6) both work, but by a client-side chunked scan rather than a single wire call | GetAllDependencyRecords has no 1:1 wire mapping in v1 — listDependencies is anchored at <=100 ids per call (L12's maxEdgeCountAnchors) — so the client enumerates every issue id (fetchIssuePages) and issues it in chunks, grouping the results itself; this is slower and heavier than a local store's single query, not absent. --tree is NOT in this row: it defaults true and is `bd list`'s default text rendering. The decorator takes that scan for its whole-workspace answers: ready work and its counts, the ready claim, and the single-issue close guard. Its per-issue roles (blocking annotation, the dependency tree, claim, and a batch close without a next claim) build on the client's served roles instead and read only the edges of the issues they name, through the served EdgeReader in chunks under the same cap (internal/storage/externaldeps/remote_roles.go) | D9 L8, D4 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L9 | degrade | — | credential-bearing settings are unreadable client-side | the server redacts by omission and says so on the wire in Setting.redacted; the client answers absent-WITH-A-REASON rather than the empty string a caller would read as unset | D9 L9 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L10 | refuse | `--watch` | `bd list --watch` and `bd show --watch` refuse | the loops poll every 2s, each `bd list` tick is a full cursor walk (L2), per-tick errors spam or vanish, and the decoration is refused anyway (L8); N watchers against one server is a request storm its semaphore answers with 503s the loop does not back off from | D9 L10, D10 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L11 | degrade | — | hooks fire client-side after the server's commit with no shared transaction, on connected workspaces only | the seam sits below HookFiringStore in the client process; a --server-url ephemeral invocation has no hook directory and runs none | D9 L11, D1 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L12 | refuse | — | an inexpressible filter refuses, never drops — the class rule every E- row below is an instance of | a dropped filter widens a result set invisibly, and the server can only reject parameters it receives | D9 L12, D7 | TestEncoderHonorsEveryTableDisposition |
| L13 | degrade | — | pretty `bd ready` shows no parent-epic context | buildParentEpicMap swallows the GetDependencyRecordsForIssues refusal into a nil map; the retirement path is an anchored listDependencies plus per-parent getIssue | D9 L13, D4 | TODO(S3): a pre-run degradation, observable only through a cmd/bd invocation; the subprocess parity corpus pins it |
| L14 | degrade | — | against a server that does NOT advertise issues.claimNext, the composed ReadyClaimer leaves three residues: the ready-at-fetch/claim-at-dial window, a false empty under contention after the bounded refetch, and a claimed row whose CARDINALITIES are as of the listing rather than of the claiming transaction | claimIssue validates claimability, not readiness, so a listing plus a claim is not the local role's one transaction and cannot be made into one. THE ROW IS NARROWER THAN IT WAS. Upstream published POST /v0/beads/issues:claimNext (#5510) and client wave ga-jpywb dials it, so on any server that advertises the token NONE of these three residues exists: selection, the compare-and-set and the hydration share the server's own transaction, an empty front is a 200 with `claimed` absent rather than a lost-races error, and the counts describe the state the claim produced. The whole ReadyClaimer contract tier runs against that leg with nothing parked, which is the measurement that says the residues were the COMPOSITION's rather than the wire's. WHAT KEEPS THE ROW ALIVE is the DOWN-LEVEL leg, which survives on purpose: `bd ready --claim` worked against pre-#5510 servers before this port, and refusing it now would be a regression dressed as progress. It is the posture BatchCloser takes toward issues.batchClose, and it is why this row describes shipped behavior rather than history. The third residue is stated in its IMPLEMENTED form, which is not the one the design anticipated: the design expected a follow-up getIssue to hydrate the counts, and therefore a read that could fail AFTER the claim was durable. The composition takes the counts from the ready page it already fetched instead, so that failure mode does not exist — a claim changes no dependency, dependent or comment count, and what is left is staleness bounded by the same fetch-to-dial window residue (a) already owns. Retirement is no longer an upstream ask but a fleet fact: the row goes when no server this client may meet is older than #5510 | D9 L14, D8 row 4 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L14 | degrade | — | against a server that does NOT advertise issues.claimNext, or on the CLI's chain while any issue in the workspace, closed ones included, holds an unsatisfied external ref, a composed ReadyClaimer leaves three residues: the ready-at-fetch/claim-at-dial window, a false empty under contention after the bounded refetch, and a claimed row whose CARDINALITIES are as of the listing rather than of the claiming transaction | claimIssue validates claimability, not readiness, so a listing plus a claim is not the local role's one transaction and cannot be made into one. THE ROW IS NARROWER THAN IT WAS. Upstream published POST /v0/beads/issues:claimNext (#5510) and client wave ga-jpywb dials it, so on any server that advertises the token NONE of these three residues exists: selection, the compare-and-set and the hydration share the server's own transaction, an empty front is a 200 with `claimed` absent rather than a lost-races error, and the counts describe the state the claim produced. The whole ReadyClaimer contract tier runs against that leg with nothing parked, which is the measurement that says the residues were the COMPOSITION's rather than the wire's. WHAT KEEPS THE ROW ALIVE is the DOWN-LEVEL leg, which survives on purpose: `bd ready --claim` worked against pre-#5510 servers before this port, and refusing it now would be a regression dressed as progress. It is the posture BatchCloser takes toward issues.batchClose, and it is why this row describes shipped behavior rather than history. The third residue is stated in its IMPLEMENTED form, which is not the one the design anticipated: the design expected a follow-up getIssue to hydrate the counts, and therefore a read that could fail AFTER the claim was durable. The composition takes the counts from the ready page it already fetched instead, so that failure mode does not exist — a claim changes no dependency, dependent or comment count, and what is left is staleness bounded by the same fetch-to-dial window residue (a) already owns. A SECOND COMPOSED LEG exists on any server. On the CLI's chain the external-dependency decorator (design 3.6) hands the claim to this role only while no issue in the workspace holds an unsatisfied external ref, because claimNext has no parameter to exclude what that policy refuses. Otherwise it claims down its own filtered ready page through the served Claimer, so all three residues return, the second as a lost-races error (internal/storage/externaldeps/remote_roles.go). A closed holder counts, as it does in every backend's blocking state: it can never be a candidate, so the composed claim picks what the served one would, and leaving it out would cost a status read per holder. A server that advertises wire.CapExternalDependencies enforces the policy itself and never reaches that leg. `bd close --claim-next` takes no composed leg: the decorator refuses it rather than claim after the close commits (F-close). Retirement of the down-level leg is no longer an upstream ask but a fleet fact: it goes when no server this client may meet is older than #5510. The decorator's leg goes with an exclusion parameter on claimNext, or once every server enforces the policy itself | D9 L14, D8 row 4 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L15 | degrade | — | a --max-rows cap can fire over http on a query whose Limit is at or under the cap, on the `bd list` legs that still walk | locally the effective SQL LIMIT is min(Limit, MaxRows+1), so a window at or under the cap can never exceed it; over http a walk to exhaustion fetches far more than the window and the cap deliberately bounds WIRE ROWS FETCHED — safety wins over exit-code parity on exactly the unbounded walk the cap exists to bound. NARROWED by the same pushdown that narrowed L2: the pushdown leg asks for min(limit, MaxRows+1) rows in one request, which IS the local expression, so a cap at or above the limit can no longer fire there and a request that used to exit 2 now answers. That is a convergence toward the contract, and it is observable — the same command changes from a refusal to a page. THE NARROWING IS SCOPED TO THE ORDERS SQL CAN EXPRESS, and that scope is a correctness bound rather than an optimization left on the table. For a GO-SIDE sort — sqlbuild.IsGoSideSort, today `--sort id`, which needs the natural-numeric comparison (bd-9 before bd-10) no ORDER BY renders — workapi.SQLLimit pushes 0 down instead of the limit, so LOCALLY the window is MaxRows+1 whatever the limit is and the cap fires on the overage even under a limit at or below it. min(limit, MaxRows+1) is therefore NOT the local expression there, and a request bounded by the limit cannot see the overage at all. walkIssues' gate keeps a capped Go-side sort on the walk, where fetching MaxRows+1 wire rows reproduces that unbounded local window exactly and raises the same refusal — so on this one shape the walk leg is the CONVERGENT one and pushing down would have been the divergence. What keeps the row live is the walk legs (unlimited reads, keyset positions, down-level servers), where the old accounting is unchanged and correct, and `bd ready`, which has no pushdown at all | D9 L15, D12 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L16 | refuse | `--file` | `bd dep add --file` carrying more than 100 edges refuses, naming the wire bound; it is never chunked | maxAddDependencyEdges = 100 (internal/httpapi/dependency_edit.go); chunking is the one path that silently breaks the request's one-transaction/one-history-entry contract, where naive forwarding at least fails loudly | D9 L16, D7, D8 row 18 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| L-close-cap | refuse | — | a batch close naming more than 100 items refuses, naming the wire bound; it is never chunked | maxBatchCloseItems = 100 (internal/httpapi/batch_close.go); the item cap bounds how long one request may hold a write transaction. Chunking is the one path that silently breaks the request's one-transaction/one-history-entry contract — the same reason L16 refuses a bulk dependency add rather than splitting it — so a larger list refuses and the caller splits it, each request atomic on its own | D9 L-close-cap, D7 F-close, D8 row 17 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
@ -157,7 +157,7 @@ Whole commands, and the flag modes of served commands, that dispatch onto method
| F-blocked | refuse | — | `bd blocked` refuses | it is raw GetBlockedIssues, which is on the unsupported allowlist | D7 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| F-count | retired | — | RETIRED — `bd count` is SERVED whole | it said the command refuses because "issueops.Counter has no wire operation; a counts operation is a filed upstream ask", and that ask was DELIVERED: upstream #5508 published countIssues, client wave ga-icks1 wired the Counter accessor, and `count` moved from httpRefusedCommands into httpServedCommands. D8 row 20 and docs/reference/http-workspaces.md have said so since; this row was the last place that did not. Every flag the command publishes reaches a wire member — the whole predicate, the six instant bounds, the three emptiness checks, --include-infra, and the five --by-* flags as the one group_by parameter — so there is no refused mode left to name either. HOW IT ROTTED IS THE PART WORTH KEEPING. Its pin was TestClassifiedRefusedCommandsRefuseBeforeTheOSSPreRun, which walks httpRefusedCommands and drives every entry. The moment `count` left that table the pin stopped covering this row — it stayed LIVE BY NAME, so the well-formedness gate that resolves PinnedBy to a real test kept passing, and DEAD BY COVERAGE, so nothing anywhere asserted the row's claim. A ledger row can be false for a whole wave under a green pin, which is what TestFRowsAgreeWithTheCommandClassification now makes impossible | D7 (RETIRED), D8 row 20 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| F-serve | refuse | — | `bd serve` refuses against an http workspace | the OSS classifier would happily serve it from the opened store, and a bd serve re-serving a remote bd serve is a proxy chain nobody designed: run serve where the database is | D7 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| F-close | refuse | — | `bd close` is SERVED whole — single id, several ids and --claim-next — over issues:batchClose; it refuses only against a DOWN-LEVEL server that does not advertise issues.batchClose | the CLI's close front door routes every id, single-id included, through BatchCloser.CloseBatch, and Lifecycle.Close has zero CLI callers at tip. The wire now carries issues:batchClose, so the whole CloseBatchRequest — multi-item and the atomic ClaimNext included — is one wire call and one server-side transaction; nothing is composed or refused where the capability is present. Against a server too old to advertise issues.batchClose, servesBatchClose() reads the handshake snapshot and routes around the batch dial entirely, so no Preflight runs and no wire.ErrCapabilityAbsent is raised: the down-level leg's refuseUnservedCloseShape raises the STORE's own *ErrHTTPUnsupported (batchcloser.go), and cmd/bd's renderEscapedHTTPRefusal turns it into the capability-absent taxonomy (case 2) by scanning the served command's own capability list for the first token the server does not advertise — issues.batchClose, which is the token it then names to upgrade to. The single-item, no-ClaimNext shape still composes onto closeIssue in that down-level leg (see L18); the shapes that cannot compose refuse there. The item cap is L-close-cap | D8 row 17 (resolved, Wire), D9 L18, D9 L-close-cap | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| F-close | refuse | — | `bd close` is SERVED whole — single id and several ids — over issues:batchClose; it refuses against a DOWN-LEVEL server that does not advertise issues.batchClose, and --claim-next refuses against every server, before anything closes | the CLI's close front door routes every id, single-id included, through BatchCloser.CloseBatch, and Lifecycle.Close has zero CLI callers at tip. The wire now carries issues:batchClose, so a CloseBatchRequest of any size is one wire call and one server-side transaction; nothing is composed or refused where the capability is present. THE NEXT CLAIM IS THE EXCEPTION. batchCloseIssues has no member to carry it, so the client refuses a ClaimNext on both legs, after the claim's own validation and before the dial (W-CloseBatchRequest.ClaimNext). Closing and then claiming would split the role's one transaction in two, and a claim that failed after the closes committed could not report them, because the role answers with an error or with outcomes, never both. On the CLI's chain the external-dependency decorator (design 3.6) refuses first unless the server advertises wire.CapExternalDependencies, and its refusal says nothing closed and names the commands that do the same work: close without --claim-next, then `bd ready --claim` (internal/storage/externaldeps/remote_roles.go). Against a server too old to advertise issues.batchClose, servesBatchClose() reads the handshake snapshot and routes around the batch dial entirely, so no Preflight runs and no wire.ErrCapabilityAbsent is raised: the down-level leg's refuseUnservedCloseShape raises the STORE's own *ErrHTTPUnsupported (batchcloser.go), which names the refused shape and the server, and cmd/bd prints it against each id the batch carried. The single-item, no-ClaimNext shape still composes onto closeIssue in that down-level leg (see L18); the shapes that cannot compose refuse there. The item cap is L-close-cap | D8 row 17 (resolved, Wire), D9 L18, D9 L-close-cap | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
| F-partial-id | refuse | — | partial-id resolution refuses with its own taxonomy text | SearchIssueIDs has no wire operation, and the client cannot tell a partial id from a full id that does not exist — so the refusal text covers both outcomes rather than falling through to a raw search error | D11 | TODO(S3): store/dial-seam and cmd/bd conformance this package's own gates do not reach; see the in-repo divergence ledger, engdocs/design/http-divergence-ledger.md, for the discipline |
## Ready-request fields

View file

@ -2,14 +2,20 @@ load("@rules_go//go:def.bzl", "go_library", "go_test")
go_library(
name = "gitignore",
srcs = ["append.go"],
srcs = [
"append.go",
"ensure.go",
],
importpath = "github.com/steveyegge/beads/internal/gitignore",
visibility = ["//:__subpackages__"],
)
go_test(
name = "gitignore_test",
srcs = ["append_test.go"],
srcs = [
"append_test.go",
"ensure_test.go",
],
embed = [":gitignore"],
)

View file

@ -0,0 +1,42 @@
package gitignore
import (
"os"
"path/filepath"
"strings"
)
// EnsurePatternIgnored appends pattern to dir/.gitignore as a standalone line
// if it is not already present (matched exactly, after trimming whitespace,
// against each existing line — the same check
// cmd/bd/doctor.EnsureGitignoreForBeadsDir uses for its own required
// patterns). It creates the file (0600) when it does not exist yet.
//
// This is the minimal sibling of doctor.EnsureGitignoreForBeadsDir, not a
// move of it: that function also owns the full canonical .beads/.gitignore
// template, repairs permissions on an existing file, and covers every
// pattern bd's own on-disk state needs — and the cmd/bd/doctor package as a
// whole pulls in dolt and git-process dependencies through its other files.
// A caller like backend/http, which promises an embedder a minimal dependency
// footprint, cannot import cmd/bd/doctor just for this one line without
// breaking that promise. EnsurePatternIgnored exists so such a caller can
// still make the one guarantee it actually needs — this pattern is never
// offered to `git add` — on its own. The two are complementary and
// idempotent together: a workspace that goes through both ends up with the
// same file either way.
func EnsurePatternIgnored(dir, pattern string) error {
path := filepath.Join(dir, ".gitignore")
content, err := os.ReadFile(path) // #nosec G304 -- caller supplies the active beads dir
if os.IsNotExist(err) {
return os.WriteFile(path, AppendLines(nil, []string{pattern}), 0o600)
}
if err != nil {
return err
}
for _, line := range strings.Split(string(content), "\n") {
if strings.TrimSpace(line) == pattern {
return nil
}
}
return os.WriteFile(path, AppendLines(content, []string{pattern}), 0o600)
}

View file

@ -0,0 +1,64 @@
package gitignore
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestEnsurePatternIgnored(t *testing.T) {
t.Run("creates the file when missing", func(t *testing.T) {
dir := t.TempDir()
if err := EnsurePatternIgnored(dir, "http_target.json"); err != nil {
t.Fatalf("EnsurePatternIgnored: %v", err)
}
body, err := os.ReadFile(filepath.Join(dir, ".gitignore"))
if err != nil {
t.Fatalf("read .gitignore: %v", err)
}
if string(body) != "http_target.json\n" {
t.Errorf(".gitignore = %q, want %q", body, "http_target.json\n")
}
})
t.Run("appends when the pattern is missing from an existing file", func(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, ".gitignore")
if err := os.WriteFile(path, []byte("local/\n"), 0o600); err != nil {
t.Fatalf("seed .gitignore: %v", err)
}
if err := EnsurePatternIgnored(dir, "http_target.json"); err != nil {
t.Fatalf("EnsurePatternIgnored: %v", err)
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read .gitignore: %v", err)
}
if !strings.Contains(string(body), "local/") {
t.Errorf(".gitignore lost its existing content: %q", body)
}
if !strings.Contains(string(body), "http_target.json") {
t.Errorf(".gitignore = %q, want it to contain http_target.json", body)
}
})
t.Run("is a no-op when the pattern is already present", func(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, ".gitignore")
original := "local/\nhttp_target.json\n"
if err := os.WriteFile(path, []byte(original), 0o600); err != nil {
t.Fatalf("seed .gitignore: %v", err)
}
if err := EnsurePatternIgnored(dir, "http_target.json"); err != nil {
t.Fatalf("EnsurePatternIgnored: %v", err)
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read .gitignore: %v", err)
}
if string(body) != original {
t.Errorf(".gitignore changed on a no-op call: got %q, want unchanged %q", body, original)
}
})
}

View file

@ -24,6 +24,7 @@ go_library(
"dial.go",
"dispatch.go",
"errors.go",
"insecure_credential_guard.go",
"journal.go",
"leasereclaimer.go",
"lifecycle.go",
@ -32,6 +33,7 @@ go_library(
"metadata.go",
"metadatacas.go",
"offrole.go",
"open_with.go",
"order.go",
"readyclaimer.go",
"refusal.go",
@ -108,10 +110,12 @@ go_test(
"divergence_test.go",
"expected_revision_contract_test.go",
"helpers_test.go",
"insecure_credential_guard_test.go",
"lifecycle_create_test.go",
"lifecycle_update_test.go",
"list_sort_test.go",
"metadatacas_test.go",
"open_with_test.go",
"race_detector_off_test.go",
"race_detector_on_test.go",
"required_served_tier_test.go",
@ -131,6 +135,7 @@ go_test(
"served_detail_reads_test.go",
"served_dualrun_test.go",
"served_dualwrite_history_test.go",
"served_external_dependency_policy_test.go",
"served_graph_aux_test.go",
"served_graph_counter_test.go",
"served_harness_test.go",
@ -190,6 +195,7 @@ go_test(
"//internal/storage/batchfixtures",
"//internal/storage/domain",
"//internal/storage/embeddeddolt",
"//internal/storage/externaldeps",
"//internal/storage/issueops",
"//internal/storage/sqlbuild",
"//internal/types",

View file

@ -250,7 +250,6 @@ var legitimatelyUnsupported = map[string]string{
// refuse on these, which is why they are named in D7's refused class.
"DetectCycles": "raw read: role CycleDetector serves this shape",
"FindWispDependentsRecursive": "raw read: no recursive-dependents operation",
"GetAllDependencyRecords": "raw read: listDependencies is anchored (L8, `bd list --deps`)",
"GetBlockedIssues": "raw read: no blocked-set operation (`bd blocked`)",
"GetBlockingInfoForIssues": "raw read: role BlockingAnnotator serves this shape",
"GetDependencies": "raw read: role EdgeReader serves this shape",

View file

@ -206,16 +206,25 @@ func (p *BearerProvider) resolve(ctx context.Context) (token, source string, err
return cred.Value, cred.Source, nil
}
// warnInsecure fires once per provider, and only when a token is actually about
// to travel. bd serve has no TLS of its own, so a bearer bound anywhere but
// loopback crosses the network in the clear; that is an operator decision to
// make knowingly, in front of a reverse proxy that terminates TLS.
// warnInsecure fires once per provider, and only when a token is actually
// configured and about to be offered to Authorize's caller. bd serve has no
// TLS of its own, so a bearer bound anywhere but loopback crosses the network
// in the clear if it is sent at all; that is an operator decision to make
// knowingly, in front of a reverse proxy that terminates TLS.
//
// This fires BEFORE guardInsecureCredential (insecure_credential_guard.go)
// gets a chance to refuse the request outright on an unallowed target
// (bee-ghosttrack CHANGES_REQUESTED on #7288, should-fix 2's warning-text
// half): the two layers do not coordinate, so the wording here must stay
// true on EITHER outcome rather than asserting the token is being sent — a
// claim that used to read as misleading immediately above a refusal error
// saying the opposite.
func (p *BearerProvider) warnInsecure() {
if !p.insecure {
return
}
p.warnOnce.Do(func() {
fmt.Fprintf(p.warnTo, "Warning: sending a bearer token to %s over plain http; the token crosses the network in the clear. Put bd serve behind TLS, or bind it to loopback.\n", p.endpoint)
fmt.Fprintf(p.warnTo, "Warning: a bearer token is configured for %s, which is plain http and not loopback; the token would cross the network in the clear unless refused. Put bd serve behind TLS, bind it to loopback, or accept the risk knowingly (bd connect --allow-plaintext, or BEADS_HTTP_ALLOW_INSECURE=1).\n", p.endpoint)
})
}

View file

@ -72,6 +72,17 @@ type DialOptions struct {
// an external HTTPClient always goes through one of the two constructors
// above, or the normal env-aware resolution.
caOverride *resolvedCA
// AllowInsecureCredential opts THIS dial into sending a credential over
// plain http to a non-loopback target (MED-4, S6 review). Left false,
// DialWith refuses to let a credential cross such a target at all — see
// guardInsecureCredential. connect.go sets this from --allow-plaintext so
// its own Handshake probe honors the same opt-in the CLI flag already
// granted; every other caller defaults to false and relies on the
// process-wide BEADS_HTTP_ALLOW_INSECURE=1 escape hatch instead, since
// Open/OpenReadOnly/OpenWith's callers have no flag of their own to set
// this through.
AllowInsecureCredential bool
}
// Conn is the store's view of one wire client.
@ -193,6 +204,17 @@ func DialWith(target Target, creds CredentialProvider, opts DialOptions) (*Conn,
// refuse.
}
// target.AllowInsecureCredential (bee-ghosttrack CHANGES_REQUESTED on
// #7288, should-fix 2) is `bd connect --allow-plaintext`'s grant,
// persisted to the sidecar and scoped to the target it names: it is
// loaded back from there on every ordinary command's dial, not only
// connect's own Handshake probe, so a workspace that connected with the
// flag does not also need BEADS_HTTP_ALLOW_INSECURE=1 set for every
// later `bd` invocation. opts.AllowInsecureCredential stays OR'd in
// alongside it: connect.go sets that field for its own probe from the
// SAME flag before the sidecar carrying it even exists yet.
creds = guardInsecureCredential(target, creds, opts.AllowInsecureCredential || target.AllowInsecureCredential)
client, err := wire.New(target.BaseURL, creds, wire.Options{
HTTPClient: opts.HTTPClient,
UserAgent: opts.UserAgent,

View file

@ -181,6 +181,11 @@ func TestDialOptionsForFileBypassesEnvEntirely(t *testing.T) {
if err != nil {
t.Fatalf("DialOptionsForFile: %v", err)
}
// u names the server as "localhost" (so SNI is sent) even though
// startServer actually bound 127.0.0.1; without pinning the dial to
// tcp4, the default dual-stack dialer can race an unrelated ::1 listener
// and flake. See forceIPv4Loopback's doc for the full mechanism.
forceIPv4Loopback(t, opts.HTTPClient.Transport)
// Target.CAFile is deliberately left empty, mirroring cmd/bd's connect
// verification call.
target := Target{BaseURL: u}

View file

@ -320,9 +320,10 @@ func designRows() []Row {
PinnedBy: pinnedByS3Conformance,
},
{
ID: "L8", Kind: KindRefuse, Flag: "--deps",
What: "`bd list --deps` refuses, and the pretty/--format dependency-decoration arms render undecorated",
Why: "GetAllDependencyRecords has no wire mapping in v1; listDependencies is an anchored read. --tree is NOT in this row: it defaults true and is `bd list`'s default text rendering",
ID: "L8", Kind: KindDegrade, Flag: "--deps",
What: "`bd list --deps` and the policy decorator's compatibility fallback for its whole-workspace blocking state (design 3.6) both work, but by a client-side chunked scan rather than a single wire call",
Why: "GetAllDependencyRecords has no 1:1 wire mapping in v1 — listDependencies is anchored at <=100 ids per call (L12's maxEdgeCountAnchors) — so the client enumerates every issue id (fetchIssuePages) and issues it in chunks, grouping the results itself; this is slower and heavier than a local store's single query, not absent. --tree is NOT in this row: it defaults true and is `bd list`'s default text rendering. " +
"The decorator takes that scan for its whole-workspace answers: ready work and its counts, the ready claim, and the single-issue close guard. Its per-issue roles (blocking annotation, the dependency tree, claim, and a batch close without a next claim) build on the client's served roles instead and read only the edges of the issues they name, through the served EdgeReader in chunks under the same cap (internal/storage/externaldeps/remote_roles.go)",
SpecRow: "D9 L8, D4",
PinnedBy: pinnedByS3Conformance,
},
@ -368,12 +369,13 @@ func designRows() []Row {
},
{
ID: "L14", Kind: KindDegrade,
What: "against a server that does NOT advertise issues.claimNext, the composed ReadyClaimer leaves three residues: the ready-at-fetch/claim-at-dial window, a false empty under contention after the bounded refetch, and a claimed row whose CARDINALITIES are as of the listing rather than of the claiming transaction",
What: "against a server that does NOT advertise issues.claimNext, or on the CLI's chain while any issue in the workspace, closed ones included, holds an unsatisfied external ref, a composed ReadyClaimer leaves three residues: the ready-at-fetch/claim-at-dial window, a false empty under contention after the bounded refetch, and a claimed row whose CARDINALITIES are as of the listing rather than of the claiming transaction",
Why: "claimIssue validates claimability, not readiness, so a listing plus a claim is not the local role's one transaction and cannot be made into one. " +
"THE ROW IS NARROWER THAN IT WAS. Upstream published POST /v0/beads/issues:claimNext (#5510) and client wave ga-jpywb dials it, so on any server that advertises the token NONE of these three residues exists: selection, the compare-and-set and the hydration share the server's own transaction, an empty front is a 200 with `claimed` absent rather than a lost-races error, and the counts describe the state the claim produced. The whole ReadyClaimer contract tier runs against that leg with nothing parked, which is the measurement that says the residues were the COMPOSITION's rather than the wire's. " +
"WHAT KEEPS THE ROW ALIVE is the DOWN-LEVEL leg, which survives on purpose: `bd ready --claim` worked against pre-#5510 servers before this port, and refusing it now would be a regression dressed as progress. It is the posture BatchCloser takes toward issues.batchClose, and it is why this row describes shipped behavior rather than history. " +
"The third residue is stated in its IMPLEMENTED form, which is not the one the design anticipated: the design expected a follow-up getIssue to hydrate the counts, and therefore a read that could fail AFTER the claim was durable. The composition takes the counts from the ready page it already fetched instead, so that failure mode does not exist — a claim changes no dependency, dependent or comment count, and what is left is staleness bounded by the same fetch-to-dial window residue (a) already owns. " +
"Retirement is no longer an upstream ask but a fleet fact: the row goes when no server this client may meet is older than #5510",
"A SECOND COMPOSED LEG exists on any server. On the CLI's chain the external-dependency decorator (design 3.6) hands the claim to this role only while no issue in the workspace holds an unsatisfied external ref, because claimNext has no parameter to exclude what that policy refuses. Otherwise it claims down its own filtered ready page through the served Claimer, so all three residues return, the second as a lost-races error (internal/storage/externaldeps/remote_roles.go). A closed holder counts, as it does in every backend's blocking state: it can never be a candidate, so the composed claim picks what the served one would, and leaving it out would cost a status read per holder. A server that advertises wire.CapExternalDependencies enforces the policy itself and never reaches that leg. `bd close --claim-next` takes no composed leg: the decorator refuses it rather than claim after the close commits (F-close). " +
"Retirement of the down-level leg is no longer an upstream ask but a fleet fact: it goes when no server this client may meet is older than #5510. The decorator's leg goes with an exclusion parameter on claimNext, or once every server enforces the policy itself",
SpecRow: "D9 L14, D8 row 4",
PinnedBy: pinnedByS3Conformance,
},
@ -1102,9 +1104,10 @@ func commandRows() []Row {
"the OSS classifier would happily serve it from the opened store, and a bd serve re-serving a remote bd serve is a proxy chain nobody designed: run serve where the database is"),
{
ID: "F-close", Kind: KindRefuse, Command: "close", Capability: "issues.batchClose",
What: "`bd close` is SERVED whole — single id, several ids and --claim-next — over issues:batchClose; it refuses only against a DOWN-LEVEL server that does not advertise issues.batchClose",
Why: "the CLI's close front door routes every id, single-id included, through BatchCloser.CloseBatch, and Lifecycle.Close has zero CLI callers at tip. The wire now carries issues:batchClose, so the whole CloseBatchRequest — multi-item and the atomic ClaimNext included — is one wire call and one server-side transaction; nothing is composed or refused where the capability is present. " +
"Against a server too old to advertise issues.batchClose, servesBatchClose() reads the handshake snapshot and routes around the batch dial entirely, so no Preflight runs and no wire.ErrCapabilityAbsent is raised: the down-level leg's refuseUnservedCloseShape raises the STORE's own *ErrHTTPUnsupported (batchcloser.go), and cmd/bd's renderEscapedHTTPRefusal turns it into the capability-absent taxonomy (case 2) by scanning the served command's own capability list for the first token the server does not advertise — issues.batchClose, which is the token it then names to upgrade to. The single-item, no-ClaimNext shape still composes onto closeIssue in that down-level leg (see L18); the shapes that cannot compose refuse there. The item cap is L-close-cap",
What: "`bd close` is SERVED whole — single id and several ids — over issues:batchClose; it refuses against a DOWN-LEVEL server that does not advertise issues.batchClose, and --claim-next refuses against every server, before anything closes",
Why: "the CLI's close front door routes every id, single-id included, through BatchCloser.CloseBatch, and Lifecycle.Close has zero CLI callers at tip. The wire now carries issues:batchClose, so a CloseBatchRequest of any size is one wire call and one server-side transaction; nothing is composed or refused where the capability is present. " +
"THE NEXT CLAIM IS THE EXCEPTION. batchCloseIssues has no member to carry it, so the client refuses a ClaimNext on both legs, after the claim's own validation and before the dial (W-CloseBatchRequest.ClaimNext). Closing and then claiming would split the role's one transaction in two, and a claim that failed after the closes committed could not report them, because the role answers with an error or with outcomes, never both. On the CLI's chain the external-dependency decorator (design 3.6) refuses first unless the server advertises wire.CapExternalDependencies, and its refusal says nothing closed and names the commands that do the same work: close without --claim-next, then `bd ready --claim` (internal/storage/externaldeps/remote_roles.go). " +
"Against a server too old to advertise issues.batchClose, servesBatchClose() reads the handshake snapshot and routes around the batch dial entirely, so no Preflight runs and no wire.ErrCapabilityAbsent is raised: the down-level leg's refuseUnservedCloseShape raises the STORE's own *ErrHTTPUnsupported (batchcloser.go), which names the refused shape and the server, and cmd/bd prints it against each id the batch carried. The single-item, no-ClaimNext shape still composes onto closeIssue in that down-level leg (see L18); the shapes that cannot compose refuse there. The item cap is L-close-cap",
SpecRow: "D8 row 17 (resolved, Wire), D9 L18, D9 L-close-cap",
// The command is served; the down-level refusal is the capability gate's,
// exercised against a server without issues.batchClose.

View file

@ -0,0 +1,116 @@
// Written fresh for OSS beads S6 (no bd-enterprise source copied): closes
// the S6 review's MED-4 finding. BearerProvider.warnInsecure (credential.go)
// only ever WARNED that a bearer was bound for a non-loopback http:// target;
// `bd connect` (connect.go) separately REFUSES that combination, but only at
// connect time, and only for the ambient ladder it dials with there. Every
// ordinary open after that — Open, OpenReadOnly (the registered dialer) and
// OpenWith (the per-open seam backend/http's embedder door uses) — had no
// refusal at all: a sidecar or an embedder-supplied Target pinned at a
// non-loopback http:// host, however that happened, would have every later
// `bd` command send its bearer in the clear with nothing stopping it.
package httpclient
import (
"context"
"fmt"
"net/http"
"os"
"strings"
"github.com/steveyegge/beads/internal/configfile"
)
// AllowInsecureCredentialEnv opts an entire process into sending a credential
// over plain http to a non-loopback bd serve.
//
// It exists because Open/OpenReadOnly/OpenWith's callers — every bd command
// but connect — have no flag of their own to thread an opt-in through, unlike
// `bd connect --allow-plaintext`. Setting it is a deliberate, informed
// decision (the same one --allow-plaintext records at connect time): the
// operator trusts the network path, typically a reverse proxy terminating TLS
// in front of bd serve.
//
// #nosec G101 -- the NAME of an environment variable, not a credential.
const AllowInsecureCredentialEnv = "BEADS_HTTP_ALLOW_INSECURE"
func allowInsecureCredentialFromEnv() bool {
v := strings.TrimSpace(os.Getenv(AllowInsecureCredentialEnv))
return v == "1" || strings.EqualFold(v, "true")
}
// guardInsecureCredential wraps creds so that, once bound for a plain-http,
// non-loopback target, it refuses to let ANY credential header it adds reach
// the wire — unless allowed, which is true when EITHER opts.AllowInsecureCredential
// (DialOptions, which connect.go's own Handshake probe sets from
// --allow-plaintext before any sidecar exists) OR target.AllowInsecureCredential
// (the sidecar's persisted record of that same grant, read back on every
// LATER dial for this workspace — see Target.AllowInsecureCredential's own
// doc) is true, or BEADS_HTTP_ALLOW_INSECURE=1 is set in the environment.
//
// It wraps the resolved wire.CredentialProvider itself, never switching on
// its concrete type, so the refusal is uniform across the ambient
// BearerProvider ladder (Dial, Open, OpenReadOnly) AND an embedder-supplied
// ProvidedCredential (backends.OpenOptions.Credential, OpenWith's seam) — any
// implementation at all, including third-party ones this package has never
// heard of.
//
// A nil creds (the tip server's loopback-trust posture: no Authorization
// header is ever sent for an unauthenticated open) passes through untouched —
// there is nothing to guard, and refusing an unauthenticated plaintext dial
// would be a different, broader policy than the one this finding asks for
// ("refuse SENDING CREDENTIALS over http:// to non-loopback").
func guardInsecureCredential(target Target, creds CredentialProvider, allowed bool) CredentialProvider {
if creds == nil {
return nil
}
base := target.BaseURL
if base == nil || base.Scheme != "http" || configfile.IsLocalHostString(base.Hostname()) {
return creds
}
if allowed || allowInsecureCredentialFromEnv() {
return creds
}
return &insecureCredentialGuard{inner: creds, endpoint: base.Redacted()}
}
// insecureCredentialGuard is guardInsecureCredential's refusal. It lets the
// inner provider run first (so a provider that attaches nothing — an
// unconfigured ambient ladder — is never refused) and only turns the request
// into an error when Authorize actually added a header: the diff is taken on
// the header count rather than on any named header, deliberately, since
// CredentialProvider's own doc names more than one scheme ("Authorization:
// Bearer, DPoP, ...") and a third-party provider may add a header this
// package has never heard of either.
type insecureCredentialGuard struct {
inner CredentialProvider
endpoint string
}
func (g *insecureCredentialGuard) Authorize(ctx context.Context, req *http.Request) error {
before := len(req.Header)
if err := g.inner.Authorize(ctx, req); err != nil {
return err
}
if len(req.Header) > before {
return fmt.Errorf(
"refusing to send a credential to %s over plain http: %s is not loopback, and the credential would cross the network unencrypted; use https://, run `bd connect --allow-plaintext` to accept the risk knowingly for this server, or set %s=1 to override",
g.endpoint, req.URL.Hostname(), AllowInsecureCredentialEnv)
}
return nil
}
// Refresh delegates unchanged: a refused request never reaches a 401, so this
// guard has nothing of its own to add here.
func (g *insecureCredentialGuard) Refresh(ctx context.Context) (bool, error) {
return g.inner.Refresh(ctx)
}
// Source forwards the inner provider's provenance report, structurally
// satisfying wire's unexported credentialSourceReporter without importing it,
// so a 401 after an ALLOWED insecure dial still names the right ladder rung.
func (g *insecureCredentialGuard) Source() string {
if r, ok := g.inner.(interface{ Source() string }); ok {
return r.Source()
}
return ""
}

View file

@ -0,0 +1,251 @@
// Written fresh for OSS beads S6 (no bd-enterprise source copied). Tests
// guardInsecureCredential (insecure_credential_guard.go), the S6 review's
// MED-4 fix: Open/OpenReadOnly/OpenWith's shared choke point for refusing a
// credential bound for a plain-http, non-loopback target.
package httpclient
import (
"context"
"net/http"
"strings"
"testing"
)
const nonLoopbackTestHost = "198.51.100.1" // RFC 5737 TEST-NET-2: reserved, globally unroutable.
func newTestRequest(t *testing.T, rawURL string) *http.Request {
t.Helper()
req, err := http.NewRequest(http.MethodGet, rawURL, nil)
if err != nil {
t.Fatalf("new request: %v", err)
}
return req
}
// TestGuardInsecureCredentialPassesThroughNilCredential pins the loopback-
// trust posture: an unauthenticated open never had anything to guard, and
// must not be turned into a refusal by this fix.
func TestGuardInsecureCredentialPassesThroughNilCredential(t *testing.T) {
target := Target{BaseURL: mustParseURL(t, "http://"+nonLoopbackTestHost+"/")}
if got := guardInsecureCredential(target, nil, false); got != nil {
t.Fatalf("guardInsecureCredential(nil) = %v, want nil", got)
}
}
// TestGuardInsecureCredentialPassesThroughForLoopback: the finding is about
// a NON-loopback target; a loopback one (what the tip OSS server ships with)
// must be unaffected regardless of opts.
func TestGuardInsecureCredentialPassesThroughForLoopback(t *testing.T) {
target := Target{BaseURL: mustParseURL(t, "http://127.0.0.1:8080/")}
inner := stubProvider{tag: "loopback"}
got := guardInsecureCredential(target, inner, false)
if got != CredentialProvider(inner) {
t.Fatalf("guardInsecureCredential returned %T, want the inner provider unwrapped for a loopback target", got)
}
}
// TestGuardInsecureCredentialPassesThroughForHTTPS: the finding is about
// PLAIN http; a non-loopback https target is exactly what a credential should
// cross the network over, so it must be unaffected.
func TestGuardInsecureCredentialPassesThroughForHTTPS(t *testing.T) {
target := Target{BaseURL: mustParseURL(t, "https://"+nonLoopbackTestHost+"/")}
inner := stubProvider{tag: "https"}
got := guardInsecureCredential(target, inner, false)
if got != CredentialProvider(inner) {
t.Fatalf("guardInsecureCredential returned %T, want the inner provider unwrapped for an https target", got)
}
}
// TestGuardInsecureCredentialRefusesNonLoopbackPlainHTTP is MED-4's core
// case: a credential provider that DOES attach a header to a non-loopback
// plain-http target must be refused, by default, with no opt-in given.
func TestGuardInsecureCredentialRefusesNonLoopbackPlainHTTP(t *testing.T) {
target := Target{BaseURL: mustParseURL(t, "http://"+nonLoopbackTestHost+"/")}
inner := stubProvider{tag: "will-attach"}
guarded := guardInsecureCredential(target, inner, false)
if guarded == CredentialProvider(inner) {
t.Fatal("guardInsecureCredential returned the inner provider unwrapped for a non-loopback plain-http target")
}
req := newTestRequest(t, "http://"+nonLoopbackTestHost+"/v0/beads/context")
err := guarded.Authorize(context.Background(), req)
if err == nil {
t.Fatal("Authorize succeeded; want a refusal (plain http, non-loopback, no opt-in)")
}
if !strings.Contains(err.Error(), "refusing to send a credential") || !strings.Contains(err.Error(), AllowInsecureCredentialEnv) {
t.Errorf("Authorize error = %q, want it to name the refusal and %s", err.Error(), AllowInsecureCredentialEnv)
}
}
// TestGuardInsecureCredentialPassesThroughWhenNothingIsAttached: a provider
// that is consulted but attaches NOTHING (the ambient ladder with no token
// configured, the tip server's loopback-trust posture extended to a
// non-loopback target with no credential at all) must not be refused — the
// finding is specifically about a credential crossing the network, not about
// plain http to a non-loopback host in general.
func TestGuardInsecureCredentialPassesThroughWhenNothingIsAttached(t *testing.T) {
target := Target{BaseURL: mustParseURL(t, "http://"+nonLoopbackTestHost+"/")}
inner := noopProvider{}
guarded := guardInsecureCredential(target, inner, false)
req := newTestRequest(t, "http://"+nonLoopbackTestHost+"/v0/beads/context")
if err := guarded.Authorize(context.Background(), req); err != nil {
t.Fatalf("Authorize: %v, want no refusal since no credential was attached", err)
}
if req.Header.Get("Authorization") != "" {
t.Fatalf("Authorization header = %q, want none", req.Header.Get("Authorization"))
}
}
// TestGuardInsecureCredentialAllowedByOptIn proves DialOptions'
// AllowInsecureCredential=true (what connect.go sets from --allow-plaintext)
// lets the SAME combination through that the default test above refuses.
func TestGuardInsecureCredentialAllowedByOptIn(t *testing.T) {
target := Target{BaseURL: mustParseURL(t, "http://"+nonLoopbackTestHost+"/")}
inner := stubProvider{tag: "opted-in"}
guarded := guardInsecureCredential(target, inner, true)
if guarded != CredentialProvider(inner) {
t.Fatalf("guardInsecureCredential(allowed=true) returned %T, want the inner provider unwrapped", guarded)
}
}
// TestGuardInsecureCredentialAllowedByEnv proves the BEADS_HTTP_ALLOW_INSECURE
// escape hatch Open/OpenReadOnly/OpenWith's callers rely on (they have no
// flag of their own to set opts.AllowInsecureCredential through).
func TestGuardInsecureCredentialAllowedByEnv(t *testing.T) {
t.Setenv(AllowInsecureCredentialEnv, "1")
target := Target{BaseURL: mustParseURL(t, "http://"+nonLoopbackTestHost+"/")}
inner := stubProvider{tag: "env-opted-in"}
guarded := guardInsecureCredential(target, inner, false)
if guarded != CredentialProvider(inner) {
t.Fatalf("guardInsecureCredential(env opt-in) returned %T, want the inner provider unwrapped", guarded)
}
}
// TestGuardInsecureCredentialDelegatesRefreshAndSource proves the refusal
// wrapper stays transparent to the two capabilities a CredentialProvider may
// also offer, so a 401's rotation retry and its source-naming both still work
// across an ALLOWED insecure dial.
func TestGuardInsecureCredentialDelegatesRefreshAndSource(t *testing.T) {
target := Target{BaseURL: mustParseURL(t, "http://"+nonLoopbackTestHost+"/")}
inner := &reportingProvider{source: "test-rung"}
guarded := guardInsecureCredential(target, inner, true)
if guarded != CredentialProvider(inner) {
t.Fatalf("guardInsecureCredential(allowed=true) returned %T, want the inner provider unwrapped", guarded)
}
// With allowed=false the guard wraps, and must still forward both calls.
wrapped := guardInsecureCredential(target, inner, false)
retry, err := wrapped.Refresh(context.Background())
if err != nil || !retry {
t.Fatalf("Refresh = (%v, %v), want (true, nil)", retry, err)
}
reporter, ok := wrapped.(interface{ Source() string })
if !ok {
t.Fatal("wrapped provider does not implement Source() string")
}
if got := reporter.Source(); got != "test-rung" {
t.Fatalf("Source() = %q, want %q", got, "test-rung")
}
}
// TestDialWithWiresAllowInsecureCredentialThroughToTheGuard is the
// integration proof that DialWith actually applies guardInsecureCredential
// (not just that the helper is correct in isolation): a real ambient
// BearerProvider (BEADS_HTTP_TOKEN set) dialing a non-loopback plain-http
// target must be refused before the server ever sees the request, and the
// SAME dial must go through once AllowInsecureCredential is set — proving
// connect.go's --allow-plaintext -> DialOptions.AllowInsecureCredential wire
// actually reaches this guard.
func TestDialWithWiresAllowInsecureCredentialThroughToTheGuard(t *testing.T) {
clearCredentialEnvironment(t)
t.Setenv(TokenEnv, nonLoopbackTestHost+"=integration-token")
server := &contextServer{body: v0Context("proj-insecure-wiring")}
srv := server.start(t)
target := Target{BaseURL: mustParseURL(t, "http://"+nonLoopbackTestHost+"/")}
redirecting := &http.Client{Transport: &redirectToAddrTransport{addr: strings.TrimPrefix(srv.URL, "http://")}}
_, err := Handshake(context.Background(), target, DialOptions{HTTPClient: redirecting})
if err == nil {
t.Fatal("Handshake succeeded; want DialWith's insecure-credential guard to refuse")
}
if !strings.Contains(err.Error(), "refusing to send a credential") {
t.Errorf("Handshake error = %v, want the insecure-credential refusal", err)
}
if len(server.seen) != 0 {
t.Errorf("server saw %d request(s); want the guard to refuse before ever dialing", len(server.seen))
}
snap, err := Handshake(context.Background(), target, DialOptions{HTTPClient: redirecting, AllowInsecureCredential: true})
if err != nil {
t.Fatalf("Handshake with AllowInsecureCredential=true: %v", err)
}
if snap.ProjectId != "proj-insecure-wiring" {
t.Errorf("project_id = %q, want proj-insecure-wiring", snap.ProjectId)
}
}
// TestDialWithWiresTargetAllowInsecureCredentialThroughToTheGuard is
// TestDialWithWiresAllowInsecureCredentialThroughToTheGuard's companion for
// the PERSISTED grant (bee-ghosttrack CHANGES_REQUESTED on #7288,
// should-fix 2): `bd connect --allow-plaintext` now saves the opt-in onto
// the sidecar (Target.AllowInsecureCredential, round-tripped through
// SaveTarget/LoadTarget), so an ORDINARY command's dial — which builds
// DialOptions fresh with AllowInsecureCredential left false, unlike
// connect's own Handshake probe — must still get through on target's say
// alone. Without DialWith also consulting target.AllowInsecureCredential, a
// workspace connected with the flag would need BEADS_HTTP_ALLOW_INSECURE=1
// set for every later command too, which is exactly the gap this closes.
func TestDialWithWiresTargetAllowInsecureCredentialThroughToTheGuard(t *testing.T) {
clearCredentialEnvironment(t)
t.Setenv(TokenEnv, nonLoopbackTestHost+"=integration-token")
server := &contextServer{body: v0Context("proj-insecure-wiring-target")}
srv := server.start(t)
redirecting := &http.Client{Transport: &redirectToAddrTransport{addr: strings.TrimPrefix(srv.URL, "http://")}}
plainTarget := Target{BaseURL: mustParseURL(t, "http://"+nonLoopbackTestHost+"/")}
if _, err := Handshake(context.Background(), plainTarget, DialOptions{HTTPClient: redirecting}); err == nil {
t.Fatal("Handshake with no grant anywhere succeeded; want the guard to refuse")
}
grantedTarget := Target{BaseURL: mustParseURL(t, "http://"+nonLoopbackTestHost+"/"), AllowInsecureCredential: true}
snap, err := Handshake(context.Background(), grantedTarget, DialOptions{HTTPClient: redirecting})
if err != nil {
t.Fatalf("Handshake with target.AllowInsecureCredential=true (opts left false): %v", err)
}
if snap.ProjectId != "proj-insecure-wiring-target" {
t.Errorf("project_id = %q, want proj-insecure-wiring-target", snap.ProjectId)
}
}
// redirectToAddrTransport sends every request to addr instead of req.URL's
// own host, so a test can dial a real (loopback) httptest.Server while
// target.BaseURL carries a different, non-loopback-looking host — the shape
// needed to exercise the insecure-credential guard against a server that
// actually answers once the guard lets a request through.
type redirectToAddrTransport struct{ addr string }
func (r *redirectToAddrTransport) RoundTrip(req *http.Request) (*http.Response, error) {
clone := req.Clone(req.Context())
clone.URL.Host = r.addr
clone.Host = r.addr
return http.DefaultTransport.RoundTrip(clone)
}
// noopProvider never attaches anything — the ambient ladder's unconfigured
// state, or a third-party provider that only sometimes authorizes.
type noopProvider struct{}
func (noopProvider) Authorize(context.Context, *http.Request) error { return nil }
func (noopProvider) Refresh(context.Context) (bool, error) { return false, nil }
// reportingProvider is a minimal CredentialProvider that also implements the
// wire package's unexported credentialSourceReporter shape structurally, to
// prove insecureCredentialGuard forwards both Refresh and Source.
type reportingProvider struct{ source string }
func (p *reportingProvider) Authorize(context.Context, *http.Request) error { return nil }
func (p *reportingProvider) Refresh(context.Context) (bool, error) { return true, nil }
func (p *reportingProvider) Source() string { return p.source }

View file

@ -112,7 +112,10 @@ func (s *Store) SetLocalMetadata(_ context.Context, key, value string) error {
// A wrong-server handshake is the one error that must NOT collapse to that
// benign skip (ga-b8ddd.11, Option C): the mismatch carries the id the server
// actually owns, so handing that Got value back is what lets cmd/bd's
// validateWorkspaceIdentity compare it against metadata.json and FIRE the
// validateWorkspaceIdentity compare it against the workspace's expected
// project id — the sidecar's ExpectProjectID for this backend, not
// metadata.json (bee-ghosttrack CHANGES_REQUESTED on #7288: metadata.json's
// project_id is not kept current across `bd connect`) — and FIRE the
// identity check — the write is refused, exit 1. What this path does NOT surface
// is the wire ProjectMismatchError's own text: only the Got id survives this
// method, so the server URL, database, repo root and recovery clause it carries

View file

@ -0,0 +1,69 @@
// Written fresh for OSS beads S6 (no bd-enterprise source copied). A sibling
// enterprise implementation takes over credential selection with its own
// Options.Credentials hook; OSS instead threads the backends.OpenOptions.Credential
// seam S1 defined, which backend_credential.go's ResolveCredential (S2) was
// built to resolve but which, until this file, had no production caller.
// OpenWith is that caller.
package httpclient
import (
"context"
"fmt"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/backends"
)
// OpenWith is the backends.Backend.OpenWith hook backend/http (bdhttp)
// installs: the per-open seam that honors opts.Credential, opts.HTTPClient and
// opts.UserAgent for exactly THIS open, rather than the process-wide dialer
// Open/OpenReadOnly (NewFromConfig/NewReadOnlyFromConfig) use via the
// package-level registered dialer.
//
// That distinction is the whole reason this is a separate code path: the
// registered dialer is installed once, at process start, from one fixed
// DialOptions — exactly what a single-tenant CLI process wants, but useless
// to a multi-tenant embedder (gc, Gas City) that needs a different credential
// per workspace it opens in the same process. OpenWith dials fresh every call.
//
// base supplies the Register-time defaults (bdhttp.Options.UserAgent /
// HTTPClient) that a non-empty opts.UserAgent or non-nil opts.HTTPClient
// overrides for this one open.
//
// require is bdhttp.Options.RequireCredential: when true, a nil opts.Credential
// is ErrCredentialRequired rather than a silent fall-through to the ambient
// bearer ladder (BEADS_HTTP_TOKEN, BEADS_HTTP_TOKEN_COMMAND, the credentials
// file) — process-global ambient state a multi-tenant embedder cannot trust to
// name the right tenant. See ResolveCredential.
//
// Unlike Open/OpenReadOnly, OpenWith never consults BEADS_HTTP_CA_FILE: CA
// resolution is pinned to target.CAFile alone via DialOptionsForTarget,
// because an embedder juggling several targets in one process must not have
// one silently renarrowed (or widened) by an ambient value an unrelated open
// exported. See DialOptionsForTarget's own doc for why that rung is skipped
// here specifically.
func OpenWith(ctx context.Context, beadsDir string, opts backends.OpenOptions, base DialOptions, require bool) (storage.DoltStorage, error) {
target, err := LoadTarget(beadsDir)
if err != nil {
return nil, err
}
creds, err := ResolveCredential(opts, target.BaseURL, require)
if err != nil {
return nil, err
}
dialOpts := base
if opts.UserAgent != "" {
dialOpts.UserAgent = opts.UserAgent
}
if opts.HTTPClient != nil {
dialOpts.HTTPClient = opts.HTTPClient
}
dialOpts = DialOptionsForTarget(target, dialOpts)
conn, err := DialWith(target, creds, dialOpts)
if err != nil {
return nil, fmt.Errorf("dialing %s: %w", target, err)
}
s := New(target, conn, nil)
s.local = newLocalMetadata(beadsDir)
return s, nil
}

View file

@ -0,0 +1,67 @@
// Written fresh for OSS beads S6 (no bd-enterprise source copied).
package httpclient
import (
"context"
"net/http"
"net/url"
"path/filepath"
"strings"
"testing"
"github.com/steveyegge/beads/internal/storage/backends"
)
// literalSecretProvider is a CredentialProvider whose unexported field holds
// the literal secret value, the same shape a real token-bearing provider
// takes. It exists only so a test can grep for that literal in an error
// string: fmt's %+v prints unexported struct fields by value, so if OpenWith
// (or anything it calls) ever formats opts.Credential directly into an error
// — rather than formatting only the target and the wrapped dial error, as
// open_with.go's "dialing %s: %w" does today — this field's value leaks into
// logs and terminals, exactly what wire.CredentialProvider's own doc comment
// on Authorize forbids ("The error must not carry the credential").
type literalSecretProvider struct {
secret string
}
func (literalSecretProvider) Authorize(context.Context, *http.Request) error { return nil }
func (literalSecretProvider) Refresh(context.Context) (bool, error) { return false, nil }
// TestOpenWithDialErrorNeverFormatsTheCredential kills the mutation that
// widens open_with.go's dial-failure wrap from
// `fmt.Errorf("dialing %s: %w", target, err)` to one that also interpolates
// opts.Credential (e.g. "dialing %s cred=%+v: %w"). A credential a caller
// supplies through OpenOptions.Credential must never surface in an error this
// package hands back — the error travels into the embedder's own logs and
// terminals, same as the doc comment on CredentialProvider.Authorize says.
//
// The dial target names a CA file that does not exist, which DialWith (via
// dialTransport) refuses before opening any socket: a config-time error, not
// a network one, so the test stays fast and never reaches any real network
// — the standing safety rule for this suite — while still giving OpenWith a
// real, non-nil error from the exact call it wraps.
func TestOpenWithDialErrorNeverFormatsTheCredential(t *testing.T) {
beadsDir := t.TempDir()
base, err := url.Parse("http://127.0.0.1:1/")
if err != nil {
t.Fatalf("parse base url: %v", err)
}
target := Target{BaseURL: base, CAFile: filepath.Join(beadsDir, "no-such-ca.pem")}
if err := SaveTarget(beadsDir, target); err != nil {
t.Fatalf("SaveTarget: %v", err)
}
const secret = "KILL-ME-SECRET-1234"
opts := backends.OpenOptions{
Credential: ProvidedCredential{Provider: literalSecretProvider{secret: secret}},
}
_, err = OpenWith(context.Background(), beadsDir, opts, DialOptions{}, false)
if err == nil {
t.Fatal("OpenWith against an unreachable loopback port returned no error; cannot assert what its error does, or does not, carry")
}
if strings.Contains(err.Error(), secret) {
t.Errorf("OpenWith's error carries the literal credential secret %q: %v", secret, err)
}
}

View file

@ -274,6 +274,106 @@ func (s *Store) GetDependencyRecords(ctx context.Context, id string) ([]*types.D
// rather than chunking past it. See httpGraphCounter.CountEdges.
const maxEdgeCountAnchors = 100
// GetAllDependencyRecords answers storage.DoltStorage's broad, unfiltered
// dependency dump: every stored edge, grouped by its owning issue's id — the
// same shape issueops.GetAllDependencyRecordsInTx returns for a local
// backend.
//
// It overrides the unsupported_gen.go stub (design 3.6, HIGH-1): the
// external-deps decorator's loadBlockingState falls back to this method
// whenever the inner store implements neither
// storage.ExternalDependencyPolicyProber-reported server enforcement nor the
// narrower storage.ExternalDependencyQueryStore, which is exactly this
// backend's shape against a server that does NOT advertise
// wire.CapExternalDependencies. Refusing here (the old unsupported behavior)
// would make `bd ready`/`bd close`/claim hard-fail against such a server
// instead of correctly enforcing the policy client-side — the regression
// HIGH-1 reports.
//
// There is no single wire operation for "every dependency in the workspace":
// listDependencies is anchored at up to maxEdgeCountAnchors ids per call, so
// this walks every issue id over the full-exhaustion list (allIssueIDs) and
// reads its edges back in chunks of that size. It is O(issues/100) requests,
// which only this fallback path ever pays — a server that advertises the
// capability, or that implements the narrower query, never reaches this
// method.
func (s *Store) GetAllDependencyRecords(ctx context.Context) (map[string][]*types.Dependency, error) {
ids, err := s.allIssueIDs(ctx)
if err != nil {
return nil, fmt.Errorf("list all issue ids: %w", err)
}
result := make(map[string][]*types.Dependency)
for _, chunk := range chunkStrings(ids, maxEdgeCountAnchors) {
q := url.Values{}
for _, id := range chunk {
q.Add("issue_id", id)
}
var body apigen.DependencyEdges
if err := s.dispatch(ctx, wire.Request{
Op: wire.OpListDependencies,
Method: http.MethodGet,
Path: wire.PathDependencies,
Query: q,
}, &body); err != nil {
return nil, err
}
for i := range body.Items {
edge := &body.Items[i]
result[edge.IssueID] = append(result[edge.IssueID], edge)
}
}
return result, nil
}
// allIssueIDs enumerates every issue id in the workspace, regardless of
// status or kind: closed, templates, gates, infra and ephemeral/wisp issues
// are all included (the list operation's narrower defaults otherwise exclude
// several of those), because a dependency row can be owned by any of them and
// GetAllDependencyRecords promises the whole table, not the default `bd
// list` view of it.
//
// want=0, maxRows=0 walks fetchIssuePages to full exhaustion rather than one
// page.
func (s *Store) allIssueIDs(ctx context.Context) ([]string, error) {
q := url.Values{
"all": {"true"},
"include_templates": {"true"},
"include_gates": {"true"},
"include_infra": {"true"},
"include_ephemeral": {"true"},
}
rows, err := s.fetchIssuePages(ctx, q, true, 0, 0, "", nil)
if err != nil {
return nil, err
}
ids := make([]string, 0, len(rows))
for _, row := range rows {
if row == nil || row.Issue == nil || row.ID == "" {
continue
}
ids = append(ids, row.ID)
}
return ids, nil
}
// chunkStrings splits ids into slices of at most size, preserving order. size
// must be positive; the last chunk may be shorter.
func chunkStrings(ids []string, size int) [][]string {
if len(ids) == 0 {
return nil
}
chunks := make([][]string, 0, (len(ids)+size-1)/size)
for len(ids) > 0 {
n := size
if n > len(ids) {
n = len(ids)
}
chunks = append(chunks, ids[:n])
ids = ids[n:]
}
return chunks
}
// httpGraphCounter serves CountEdges from countDependencyEdges — the
// TWENTY-SECOND wire-backed accessor, and the only role on this surface whose
// answer is a number PER ANCHOR.

View file

@ -0,0 +1,492 @@
//go:build cgo
package httpclient
import (
"errors"
"fmt"
"slices"
"testing"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/externaldeps"
storageops "github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/types"
"github.com/steveyegge/beads/internal/workapi"
"github.com/steveyegge/beads/issueops"
)
// TestServedReadyExcludesUnsatisfiedExternalDependencyWithoutServerCapability
// reproduces the S6 review's HIGH-1 regression: cmd/bd's
// wireExternalDependencyPolicy used to skip wrapping ANY registered remote
// backend outright (storage.RemoteBackendStore), so an issue blocked by an
// external:<project>:<capability> dependency was listed by `bd ready` over
// http even though the identical workspace against a local backend would
// exclude it — the policy was silently skipped, not deferred to a server that
// claimed to enforce it (design 3.6).
//
// This pins the fix at the layer design 3.6 actually gates on: the decorator
// must still apply client-side enforcement whenever the server's handshake
// does not advertise wire.CapExternalDependencies. Today's OSS httpapi never
// advertises it (confirmed below), so this also exercises the http backend's
// GetAllDependencyRecords fallback (loadBlockingState's compatibility path)
// against a real served server, end to end.
func TestServedReadyExcludesUnsatisfiedExternalDependencyWithoutServerCapability(t *testing.T) {
env := newServedEnv(t, "hixd")
ctx := t.Context()
blockedID := env.prefix + "-blocked"
freeID := env.prefix + "-free"
if err := env.createIssue(ctx, &types.Issue{
ID: blockedID, Title: "blocked by an external capability",
Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask,
}, "tester"); err != nil {
t.Fatalf("seed blocked issue: %v", err)
}
if err := env.createIssue(ctx, &types.Issue{
ID: freeID, Title: "not blocked by anything",
Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask,
}, "tester"); err != nil {
t.Fatalf("seed free issue: %v", err)
}
if err := env.addDependency(ctx, &types.Dependency{
IssueID: blockedID,
DependsOnID: "external:otherproj:cap",
Type: types.DepBlocks,
}, "tester"); err != nil {
t.Fatalf("seed external dependency: %v", err)
}
// The premise this test pins: the server this harness serves does not
// advertise wire.CapExternalDependencies. If a future change makes OSS
// httpapi advertise it, this test's exclusion no longer proves the
// client-side path and must be re-pointed at a server that genuinely
// withholds the capability.
if enforced, err := env.subject.ServerEnforcesExternalDependencyPolicy(ctx); err != nil {
t.Fatalf("ServerEnforcesExternalDependencyPolicy: %v", err)
} else if enforced {
t.Fatal("served test server unexpectedly advertises wire.CapExternalDependencies; this test's no-server-enforcement premise no longer holds")
}
// externaldeps.New with nil locate/open funcs is the same fail-closed
// shape cmd/bd's wireExternalDependencyPolicy composes for an
// unconfigured external project: a reference to a project this workspace
// has not configured resolves to "unsatisfied", so the dependency keeps
// blocking rather than silently passing.
decorated := externaldeps.New(env.subject, nil, nil)
filter, err := workapi.BuildReadyFilter(issueops.ReadyRequest{})
if err != nil {
t.Fatalf("BuildReadyFilter: %v", err)
}
rows, err := decorated.GetReadyWorkWithCounts(ctx, filter)
if err != nil {
t.Fatalf("GetReadyWorkWithCounts over http: %v", err)
}
var ids []string
for _, row := range rows {
if row != nil && row.Issue != nil {
ids = append(ids, row.ID)
}
}
for _, id := range ids {
if id == blockedID {
t.Errorf("ready over http = %v, want %q excluded by its unsatisfied external:otherproj:cap dependency", ids, blockedID)
}
}
var sawFree bool
for _, id := range ids {
if id == freeID {
sawFree = true
}
}
if !sawFree {
t.Errorf("ready over http = %v, want unrelated issue %q present", ids, freeID)
}
// The total `bd ready --json` prints beside the page, and the count
// text-mode `bd ready` prints, must exclude the same issue. This store
// cannot express ExcludeIDs, so both come from the decorator's client-side
// drop, which is where the exclusion has to be subtracted.
_, total, err := decorated.GetReadyWorkWithCountsAndTotal(ctx, filter)
if err != nil {
t.Fatalf("GetReadyWorkWithCountsAndTotal over http: %v", err)
}
count, err := decorated.CountReadyWork(ctx, filter)
if err != nil {
t.Fatalf("CountReadyWork over http: %v", err)
}
if total != len(ids) || count != len(ids) {
t.Errorf("ready over http lists %v, but total = %d and CountReadyWork = %d; want both %d (%q excluded)", ids, total, count, len(ids), blockedID)
}
}
// TestServedReadyCountExcludesAnExternallyBlockedIssuePastTheDefaultPage pins,
// against a real served server, the shape the decorator once misread: an
// unlimited ready listing over http comes back as the server's default page
// (the wire omits a zero limit) while its total counts the whole ready set.
// Text-mode `bd ready` counts through CountReadyWork, which always fetches
// unlimited, so an externally blocked issue sorted past that page stayed in
// its count when the decorator took the zero limit for a window holding every
// row.
func TestServedReadyCountExcludesAnExternallyBlockedIssuePastTheDefaultPage(t *testing.T) {
env := newServedEnv(t, "hixp")
ctx := t.Context()
free := make([]*types.Issue, 0, workapi.DefaultReadyLimit+1)
for i := range workapi.DefaultReadyLimit + 1 {
free = append(free, &types.Issue{
ID: fmt.Sprintf("%s-free%03d", env.prefix, i), Title: "not blocked by anything",
Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask,
})
}
if err := env.reference.CreateIssues(ctx, free, "tester"); err != nil {
t.Fatalf("seed free issues: %v", err)
}
// Priority 4 sorts the blocked issue after every free one, past the
// default page.
blockedID := env.prefix + "-blocked"
if err := env.createIssue(ctx, &types.Issue{
ID: blockedID, Title: "blocked by an external capability",
Status: types.StatusOpen, Priority: 4, IssueType: types.TypeTask,
}, "tester"); err != nil {
t.Fatalf("seed blocked issue: %v", err)
}
if err := env.addDependency(ctx, &types.Dependency{
IssueID: blockedID,
DependsOnID: "external:otherproj:cap",
Type: types.DepBlocks,
}, "tester"); err != nil {
t.Fatalf("seed external dependency: %v", err)
}
countFilter, err := workapi.BuildReadyCountFilter(issueops.ReadyRequest{})
if err != nil {
t.Fatalf("BuildReadyCountFilter: %v", err)
}
// The premise this test pins: undecorated, the unlimited listing is the
// default page, the blocked issue is not on it, and the total counts it.
rows, total, err := env.subject.GetReadyWorkWithCountsAndTotal(ctx, countFilter)
if err != nil {
t.Fatalf("undecorated GetReadyWorkWithCountsAndTotal over http: %v", err)
}
if len(rows) != workapi.DefaultReadyLimit || total != workapi.DefaultReadyLimit+2 {
t.Fatalf("undecorated unlimited listing over http = %d rows, total %d; want the %d-row default page, total %d; this test's premise no longer holds",
len(rows), total, workapi.DefaultReadyLimit, workapi.DefaultReadyLimit+2)
}
for _, row := range rows {
if row != nil && row.Issue != nil && row.ID == blockedID {
t.Fatalf("%q is on the default page; this test needs it past the page", blockedID)
}
}
decorated := externaldeps.New(env.subject, nil, nil)
const want = workapi.DefaultReadyLimit + 1 // every free issue
count, err := decorated.CountReadyWork(ctx, countFilter)
if err != nil {
t.Fatalf("CountReadyWork over http: %v", err)
}
pageFilter, err := workapi.BuildReadyFilter(issueops.ReadyRequest{})
if err != nil {
t.Fatalf("BuildReadyFilter: %v", err)
}
pageRows, pageTotal, err := decorated.GetReadyWorkWithCountsAndTotal(ctx, pageFilter)
if err != nil {
t.Fatalf("GetReadyWorkWithCountsAndTotal over http: %v", err)
}
if count != want || pageTotal != want || len(pageRows) != workapi.DefaultReadyLimit {
t.Errorf("over http: CountReadyWork = %d; default page = %d rows, total %d; want count and total %d (%q excluded), %d rows",
count, len(pageRows), pageTotal, want, blockedID, workapi.DefaultReadyLimit)
}
}
// TestServedRolesKeepTheExternalDependencyPolicy pins, against a real served
// server, the roles the decorator composes over a remote store. Each one
// answers through httpclient.Store's served roles: the base accessors built
// them over legacy methods that store refuses, which failed `bd list`,
// `bd show`, `bd dep tree`, `bd ready --claim` and `bd close` even for an issue
// nothing blocks. And each still holds the externally blocked issue out of
// ready work, claims and closes. The served handler here binds its roles to
// the raw reference store, which knows nothing of the policy, so every
// exclusion below is the client's.
func TestServedRolesKeepTheExternalDependencyPolicy(t *testing.T) {
env := newServedEnv(t, "hixr")
ctx := t.Context()
const externalRef = "external:otherproj:cap"
blockedID := env.prefix + "-blocked"
freeID := env.prefix + "-free"
// Priority 1 sorts the blocked issue first, where a claim blind to the
// policy would take it.
for _, seed := range []*types.Issue{
{ID: blockedID, Title: "blocked by an external capability", Status: types.StatusOpen, Priority: 1, IssueType: types.TypeTask},
{ID: freeID, Title: "not blocked by anything", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask},
} {
if err := env.createIssue(ctx, seed, "tester"); err != nil {
t.Fatalf("seed %s: %v", seed.ID, err)
}
}
if err := env.addDependency(ctx, &types.Dependency{
IssueID: blockedID,
DependsOnID: externalRef,
Type: types.DepBlocks,
}, "tester"); err != nil {
t.Fatalf("seed external dependency: %v", err)
}
decorated := externaldeps.New(env.subject, nil, nil)
reader, err := decorated.IssueReader()
if err != nil {
t.Fatalf("IssueReader: %v", err)
}
listed, err := reader.List(ctx, issueops.ListRequest{})
if err != nil {
t.Fatalf("List over http: %v", err)
}
if ids := pageIDs(listed); !slices.Contains(ids, blockedID) || !slices.Contains(ids, freeID) {
t.Errorf("List over http = %v, want both issues: a plain listing claims nothing about readiness", ids)
}
readyListed, err := reader.List(ctx, issueops.ListRequest{ReadyFlag: true})
if err != nil {
t.Fatalf("List --ready over http: %v", err)
}
if ids := pageIDs(readyListed); !slices.Equal(ids, []string{freeID}) {
t.Errorf("List --ready over http = %v, want [%s]", ids, freeID)
}
ready, err := reader.Ready(ctx, issueops.ReadyRequest{})
if err != nil {
t.Fatalf("Ready over http: %v", err)
}
if ids := pageIDs(ready); !slices.Equal(ids, []string{freeID}) {
t.Errorf("Ready over http = %v, want [%s]", ids, freeID)
}
details, err := reader.Get(ctx, issueops.GetRequest{ID: blockedID})
if err != nil || details == nil || details.ID != blockedID {
t.Fatalf("Get %s over http = %v, %v", blockedID, details, err)
}
annotator, err := decorated.BlockingAnnotator()
if err != nil {
t.Fatalf("BlockingAnnotator: %v", err)
}
blocking, err := annotator.AnnotateBlocking(ctx, issueops.BlockingRequest{IDs: []string{blockedID, freeID}})
if err != nil {
t.Fatalf("AnnotateBlocking over http: %v", err)
}
for _, item := range blocking.Items {
if blockedBy := slices.Contains(item.BlockedBy, externalRef); blockedBy != (item.ID == blockedID) {
t.Errorf("AnnotateBlocking over http: %s blocked by %v", item.ID, item.BlockedBy)
}
}
walker, err := decorated.TreeWalker()
if err != nil {
t.Fatalf("TreeWalker: %v", err)
}
tree, err := walker.WalkTree(ctx, issueops.WalkTreeRequest{RootID: blockedID, MaxDepth: 50})
if err != nil {
t.Fatalf("WalkTree over http: %v", err)
}
var leaves int
for _, node := range tree.Nodes {
if node.ID == externalRef {
leaves++
}
}
if leaves != 1 {
t.Errorf("WalkTree over http shows %d %s leaves, want one", leaves, externalRef)
}
claimer, err := decorated.IssueClaimer()
if err != nil {
t.Fatalf("IssueClaimer: %v", err)
}
if _, err := claimer.Claim(ctx, issueops.ClaimRequest{Actor: "worker", IssueID: blockedID}); !errors.Is(err, storage.ErrCloseBlocked) {
t.Errorf("Claim %s over http = %v, want ErrCloseBlocked", blockedID, err)
}
readyClaimer, err := decorated.ReadyClaimer()
if err != nil {
t.Fatalf("ReadyClaimer: %v", err)
}
next, err := readyClaimer.ClaimNext(ctx, issueops.ClaimNextRequest{Actor: "worker"})
if err != nil || next.Claimed == nil || next.Claimed.ID != freeID {
t.Fatalf("ClaimNext over http = %+v, %v; want %s", next.Claimed, err, freeID)
}
closer, err := decorated.BatchCloser()
if err != nil {
t.Fatalf("BatchCloser: %v", err)
}
closed, err := closer.CloseBatch(ctx, issueops.CloseBatchRequest{
Actor: "worker",
Items: []issueops.BatchCloseItem{{IssueID: blockedID}, {IssueID: freeID}},
})
if err != nil || len(closed.Outcomes) != 2 {
t.Fatalf("CloseBatch over http = %+v, %v; want two outcomes", closed.Outcomes, err)
}
if !errors.Is(closed.Outcomes[0].Err, storage.ErrCloseBlocked) || closed.Outcomes[1].Err != nil {
t.Fatalf("CloseBatch over http: %s = %v, %s = %v; want the first refused, the second closed",
blockedID, closed.Outcomes[0].Err, freeID, closed.Outcomes[1].Err)
}
assertServedStatus(t, env, blockedID, types.StatusOpen)
assertServedStatus(t, env, freeID, types.StatusClosed)
forced, err := closer.CloseBatch(ctx, issueops.CloseBatchRequest{
Actor: "worker", Force: true,
Items: []issueops.BatchCloseItem{{IssueID: blockedID}},
})
if err != nil || len(forced.Outcomes) != 1 || forced.Outcomes[0].Err != nil {
t.Fatalf("CloseBatch --force over http = %+v, %v; want it closed", forced.Outcomes, err)
}
assertServedStatus(t, env, blockedID, types.StatusClosed)
}
// TestServedReadyTotalIgnoresAnExternallyBlockedIssueOutsideTheFilter pins the
// count an assignee-filtered `bd ready` prints over http. Past the server's
// default page the decorator used to subtract every externally blocked issue
// in the workspace from the filtered total, so bob's blocked issue, which the
// filter never counted, took one off alice's.
func TestServedReadyTotalIgnoresAnExternallyBlockedIssueOutsideTheFilter(t *testing.T) {
env := newServedEnv(t, "hixa")
ctx := t.Context()
mine := make([]*types.Issue, 0, workapi.DefaultReadyLimit+1)
for i := range workapi.DefaultReadyLimit + 1 {
mine = append(mine, &types.Issue{
ID: fmt.Sprintf("%s-alice%03d", env.prefix, i), Title: "alice's ready work",
Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask, Assignee: "alice",
})
}
if err := env.reference.CreateIssues(ctx, mine, "tester"); err != nil {
t.Fatalf("seed alice's issues: %v", err)
}
blockedID := env.prefix + "-bob"
if err := env.createIssue(ctx, &types.Issue{
ID: blockedID, Title: "bob's externally blocked work",
Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask, Assignee: "bob",
}, "tester"); err != nil {
t.Fatalf("seed blocked issue: %v", err)
}
if err := env.addDependency(ctx, &types.Dependency{
IssueID: blockedID,
DependsOnID: "external:otherproj:cap",
Type: types.DepBlocks,
}, "tester"); err != nil {
t.Fatalf("seed external dependency: %v", err)
}
decorated := externaldeps.New(env.subject, nil, nil)
const want = workapi.DefaultReadyLimit + 1 // every one of alice's issues
request := issueops.ReadyRequest{Assignee: "alice"}
countFilter, err := workapi.BuildReadyCountFilter(request)
if err != nil {
t.Fatalf("BuildReadyCountFilter: %v", err)
}
count, err := decorated.CountReadyWork(ctx, countFilter)
if err != nil {
t.Fatalf("CountReadyWork over http: %v", err)
}
pageFilter, err := workapi.BuildReadyFilter(request)
if err != nil {
t.Fatalf("BuildReadyFilter: %v", err)
}
rows, total, err := decorated.GetReadyWorkWithCountsAndTotal(ctx, pageFilter)
if err != nil {
t.Fatalf("GetReadyWorkWithCountsAndTotal over http: %v", err)
}
if count != want || total != want || len(rows) != workapi.DefaultReadyLimit {
t.Errorf("over http, alice's ready work: CountReadyWork = %d; page = %d rows, total %d; want count and total %d, %d rows",
count, len(rows), total, want, workapi.DefaultReadyLimit)
}
}
// TestServedReadyMaxRowsCountsTheRowsKept pins `bd ready --limit N
// --max-rows N` over http, which failed with exit 2 once more than N issues
// were ready and any issue held an unsatisfied external blocker. The decorator
// fetches a window wider than the limit to make room for the rows it drops,
// and the http bridge caps the rows it fetched, so the cap counted rows the
// caller was never handed. A SQL store caps the page it delivers, and so must
// this path.
func TestServedReadyMaxRowsCountsTheRowsKept(t *testing.T) {
env := newServedEnv(t, "hixm")
ctx := t.Context()
free := []string{env.prefix + "-a", env.prefix + "-b", env.prefix + "-c"}
blockedID := env.prefix + "-z"
// The priorities fix the ready order: the three free issues, then the
// blocked one, so a window of the limit plus one can leave it out and
// still hold more rows than the limit.
for i, id := range append(slices.Clone(free), blockedID) {
if err := env.createIssue(ctx, &types.Issue{
ID: id, Title: id, Status: types.StatusOpen, Priority: i + 1, IssueType: types.TypeTask,
}, "tester"); err != nil {
t.Fatalf("seed %s: %v", id, err)
}
}
if err := env.addDependency(ctx, &types.Dependency{
IssueID: blockedID,
DependsOnID: "external:otherproj:cap",
Type: types.DepBlocks,
}, "tester"); err != nil {
t.Fatalf("seed external dependency: %v", err)
}
decorated := externaldeps.New(env.subject, nil, nil)
for _, tc := range []struct {
name string
limit, maxRows int
want []string // nil: the cap fires
}{
// The window is 2+1 rows, all free; the cap counts the page of two.
{name: "cap equals the limit", limit: 2, maxRows: 2, want: free[:2]},
// The window holds all four rows, three of them kept.
{name: "cap below the limit", limit: 5, maxRows: 3, want: free},
// The server's default page, again all four.
{name: "unlimited", limit: 0, maxRows: 3, want: free},
// The three rows kept exceed the caller's cap, which still fires.
{name: "kept rows exceed the cap", limit: 5, maxRows: 2},
} {
t.Run(tc.name, func(t *testing.T) {
limit := tc.limit
filter, err := workapi.BuildReadyFilter(issueops.ReadyRequest{Limit: &limit})
if err != nil {
t.Fatalf("BuildReadyFilter: %v", err)
}
filter.MaxRows, filter.MaxRowsSource = tc.maxRows, "--max-rows"
check := func(method string, ids []string, err error) {
t.Helper()
var tooMany *storageops.ErrTooManyRows
switch {
case tc.want == nil:
if !errors.As(err, &tooMany) || tooMany.Cap != tc.maxRows || tooMany.Source != "--max-rows" {
t.Errorf("%s over http = %v, %v; want the --max-rows cap of %d to fire on the rows kept", method, ids, err, tc.maxRows)
}
case err != nil:
t.Errorf("%s over http: %v; want %v, which a cap of %d admits", method, err, tc.want, tc.maxRows)
case !slices.Equal(ids, tc.want):
t.Errorf("%s over http = %v, want %v (%q excluded)", method, ids, tc.want, blockedID)
}
}
issues, err := decorated.GetReadyWork(ctx, filter)
check("GetReadyWork", issueIDs(issues), err)
rows, err := decorated.GetReadyWorkWithCounts(ctx, filter)
check("GetReadyWorkWithCounts", rowIDs(rows), err)
rows, total, err := decorated.GetReadyWorkWithCountsAndTotal(ctx, filter)
check("GetReadyWorkWithCountsAndTotal", rowIDs(rows), err)
if err == nil && total != len(free) {
t.Errorf("GetReadyWorkWithCountsAndTotal over http: total %d, want %d (%q excluded)", total, len(free), blockedID)
}
})
}
}
func assertServedStatus(t *testing.T, env *servedEnv, id string, want types.Status) {
t.Helper()
issue, err := env.getIssue(t.Context(), id)
if err != nil || issue == nil {
t.Fatalf("read %s back: %v, %v", id, issue, err)
}
if issue.Status != want {
t.Errorf("%s status = %s, want %s", id, issue.Status, want)
}
}

View file

@ -18,9 +18,11 @@ import (
"context"
"errors"
"fmt"
"slices"
"sync"
"github.com/steveyegge/beads/internal/httpapi/apigen"
"github.com/steveyegge/beads/internal/httpclient/wire"
"github.com/steveyegge/beads/internal/storage"
)
@ -73,6 +75,58 @@ type Store struct {
// interfaces here would assert a maintenance contract no OSS caller checks.
var _ storage.DoltStorage = (*Store)(nil)
// IsRemoteBackendStore satisfies storage.RemoteBackendStore. cmd/bd's storage
// chain wraps this store in the external-dependency policy decorator like any
// other. The decorator reads this marker only to build its roles on this
// store's served roles rather than on the legacy methods it refuses
// (internal/storage/externaldeps/remote_roles.go). It decides whether to stand
// down through ServerEnforcesExternalDependencyPolicy below, never through
// this marker.
func (s *Store) IsRemoteBackendStore() bool { return true }
var _ storage.RemoteBackendStore = (*Store)(nil)
// ServerEnforcesExternalDependencyPolicy satisfies
// storage.ExternalDependencyPolicyProber (design 3.6): it answers whether the
// server this store dials already enforces bd's external-dependency policy
// itself, by checking the handshake for wire.CapExternalDependencies.
//
// It propagates a handshake error rather than swallowing it to false, the
// same asymmetry servesBatchClose documents: this probe gates whether the
// external-deps DECORATOR runs its own enforcement at all, so an unresolved
// "does the server already enforce this" has to fail the caller, not silently
// resolve to "no, so enforce here" (which happens to be safe for THIS probe's
// specific true/false meaning but would still hide a transport failure behind
// a policy decision) or to "yes, so skip" (which would be the actual bug this
// interface exists to prevent — the policy must never be silently skipped).
func (s *Store) ServerEnforcesExternalDependencyPolicy(ctx context.Context) (bool, error) {
snap, err := s.snapshot(ctx)
if err != nil {
return false, err
}
if snap == nil {
return false, nil
}
return slices.Contains(snap.Capabilities, wire.CapExternalDependencies), nil
}
var _ storage.ExternalDependencyPolicyProber = (*Store)(nil)
// ExcludeIDsUnsupported satisfies storage.ExcludeIDsUnsupportedStore: the v0
// wire's listReadyWork and countReadyWork operations publish no id-exclusion
// parameter, so encode.ReadyBridgeParams / ReadyBridgeCountParams refuse any
// types.WorkFilter carrying ExcludeIDs outright (D8/D9, L12) rather than
// silently widening the result. Reporting true here — unconditionally, since
// the gap is in the wire shape, not in any per-call state — lets the
// external-deps decorator fall back to filtering its OWN additional
// exclusions (unsatisfied external:<project>:<capability> blockers) client
// side instead of folding them into the filter it hands this store, so design
// 3.6's policy still runs against a server that does not advertise
// wire.CapExternalDependencies rather than tripping this store's refusal.
func (s *Store) ExcludeIDsUnsupported() bool { return true }
var _ storage.ExcludeIDsUnsupportedStore = (*Store)(nil)
// New builds the store around an already-resolved target and transport.
//
// snapshot pre-seeds the handshake context, or is nil to let D6 fetch it lazily

View file

@ -30,6 +30,26 @@ type Target struct {
// skips the check. It is compared against ContextResponse.project_id at
// handshake (D6).
ExpectProjectID string
// PreviousBackend records the backend selection metadata.json carried
// right before THIS connect (bee-ghosttrack CHANGES_REQUESTED on #7288,
// should-fix 1): `bd connect --clear` reads it back to restore
// metadata.json's backend selection, so detaching from http returns the
// workspace to whatever it was attached to before, rather than leaving
// it pinned to "http" with no sidecar to dial. "" means there was
// nothing to restore (a workspace's first-ever connect, or a value
// Attach deliberately left alone — see Attach's own doc).
PreviousBackend string
// AllowInsecureCredential records `bd connect --allow-plaintext`'s grant
// for THIS target (bee-ghosttrack CHANGES_REQUESTED on #7288,
// should-fix 2): once connected with the flag, every later dial for this
// workspace — not only connect's own Handshake probe — carries the same
// opt-in, so an operator who accepted the risk once at connect time does
// not also need BEADS_HTTP_ALLOW_INSECURE=1 set for every ordinary `bd`
// command afterward. It is scoped to the sidecar it is saved beside: a
// `bd connect` to a DIFFERENT url without the flag writes a fresh
// sidecar with this false, so the grant never silently carries over to a
// server it was never given for. See guardInsecureCredential.
AllowInsecureCredential bool
// CAFile names a PEM file that becomes the ENTIRE trusted root pool for
// this target — not an addition to the system store — set by
// `bd connect --ca-file <path>`. "" leaves this target on the system
@ -81,6 +101,14 @@ type targetFile struct {
// CAFile mirrors Target.CAFile; see that field's doc for the trust model
// and TransportFor for the env-vs-sidecar precedence.
CAFile string `json:"ca_file,omitempty"`
// PreviousBackend mirrors Target.PreviousBackend; see that field's doc.
PreviousBackend string `json:"previous_backend,omitempty"`
// AllowPlaintext mirrors Target.AllowInsecureCredential; see that field's
// doc. Named differently on the wire (matching the CLI flag's own
// spelling) than the Go field (matching DialOptions.AllowInsecureCredential),
// deliberately: this is the one record of what a human typed, and the
// JSON key should read that way on disk.
AllowPlaintext bool `json:"allow_plaintext,omitempty"`
}
// TargetPath is the sidecar's location for a workspace.
@ -122,7 +150,13 @@ func LoadTarget(beadsDir string) (Target, error) {
if err := checkCAFileAbsolute(f.CAFile); err != nil {
return Target{}, err
}
return Target{BaseURL: u, ExpectProjectID: f.ExpectProjectID, CAFile: f.CAFile}, nil
return Target{
BaseURL: u,
ExpectProjectID: f.ExpectProjectID,
CAFile: f.CAFile,
PreviousBackend: f.PreviousBackend,
AllowInsecureCredential: f.AllowPlaintext,
}, nil
}
// checkNoUserinfo refuses a url carrying userinfo ("user:secret@host").
@ -155,7 +189,10 @@ func checkCAFileAbsolute(caFile string) error {
// and the connect command share one encoder; the connect UX itself (gitignore
// coverage, identity verification, conversion consent) is not here. It refuses
// a url carrying userinfo and a relative CAFile before writing anything, with
// LoadTarget's own messages.
// LoadTarget's own messages. The write is atomic (temp file in the same
// directory, then rename): a reader racing this write — LoadTarget, or
// another process entirely — must never observe a truncated or partial
// sidecar.
func SaveTarget(beadsDir string, t Target) error {
if err := checkNoUserinfo(t.BaseURL); err != nil {
return err
@ -163,7 +200,13 @@ func SaveTarget(beadsDir string, t Target) error {
if err := checkCAFileAbsolute(t.CAFile); err != nil {
return err
}
f := targetFile{ExpectProjectID: t.ExpectProjectID, API: "v0", CAFile: t.CAFile}
f := targetFile{
ExpectProjectID: t.ExpectProjectID,
API: "v0",
CAFile: t.CAFile,
PreviousBackend: t.PreviousBackend,
AllowPlaintext: t.AllowInsecureCredential,
}
if t.BaseURL != nil {
f.URL = t.BaseURL.String()
}
@ -171,12 +214,38 @@ func SaveTarget(beadsDir string, t Target) error {
if err != nil {
return fmt.Errorf("marshaling %s: %w", TargetFileName, err)
}
if err := os.WriteFile(TargetPath(beadsDir), data, 0o600); err != nil {
if err := writeFileAtomic(TargetPath(beadsDir), data, 0o600); err != nil {
return fmt.Errorf("writing %s: %w", TargetFileName, err)
}
return nil
}
// writeFileAtomic writes data to a temp file in path's directory and renames
// it over path, so a concurrent reader never sees a truncated or partial
// file. Mirrors internal/configfile's own helper of the same name and shape;
// not shared directly because that one is package-private to configfile.
func writeFileAtomic(path string, data []byte, perm os.FileMode) error {
dir := filepath.Dir(path)
tmp, err := os.CreateTemp(dir, filepath.Base(path)+".tmp-*")
if err != nil {
return err
}
tmpName := tmp.Name()
defer os.Remove(tmpName) //nolint:errcheck // no-op after successful rename
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
return err
}
if err := tmp.Chmod(perm); err != nil {
_ = tmp.Close()
return err
}
if err := tmp.Close(); err != nil {
return err
}
return os.Rename(tmpName, path)
}
// RemoveTarget deletes the activation sidecar, reporting whether one was there.
// It is `bd connect --clear`'s whole write: detaching is a per-user act, so it
// must never touch the tracked metadata.json that selects the backend.

View file

@ -193,3 +193,64 @@ func TestLoadTargetAcceptsAbsoluteCAFile(t *testing.T) {
t.Errorf("CAFile = %q, want %q", got.CAFile, abs)
}
}
// TestSaveTargetRoundTripsPreviousBackendAndAllowInsecureCredential pins the
// bee-ghosttrack CHANGES_REQUESTED findings on #7288 (should-fix 1 and 2):
// `bd connect --clear` needs PreviousBackend back to restore metadata.json's
// backend selection, and an ordinary command's dial needs
// AllowInsecureCredential back to honor a plaintext grant made at connect
// time. Neither is useful if the sidecar does not actually carry it through
// a save/load round trip.
func TestSaveTargetRoundTripsPreviousBackendAndAllowInsecureCredential(t *testing.T) {
dir := t.TempDir()
target := Target{
BaseURL: mustParseURL(t, "http://127.0.0.1:8080"),
PreviousBackend: "dolt",
AllowInsecureCredential: true,
}
if err := SaveTarget(dir, target); err != nil {
t.Fatalf("SaveTarget: %v", err)
}
got, err := LoadTarget(dir)
if err != nil {
t.Fatalf("LoadTarget: %v", err)
}
if got.PreviousBackend != "dolt" {
t.Errorf("PreviousBackend = %q, want %q", got.PreviousBackend, "dolt")
}
if !got.AllowInsecureCredential {
t.Error("AllowInsecureCredential = false after a round trip, want true")
}
raw, err := os.ReadFile(TargetPath(dir))
if err != nil {
t.Fatalf("reading sidecar: %v", err)
}
if !strings.Contains(string(raw), `"previous_backend": "dolt"`) {
t.Errorf("sidecar does not carry previous_backend: %s", raw)
}
if !strings.Contains(string(raw), `"allow_plaintext": true`) {
t.Errorf("sidecar does not carry allow_plaintext: %s", raw)
}
}
// TestSaveTargetOmitsPreviousBackendAndAllowInsecureCredentialWhenUnset keeps
// the common case's sidecar exactly as small as before this fix: a workspace
// with nothing to restore and no plaintext grant must not gain two new
// always-present keys.
func TestSaveTargetOmitsPreviousBackendAndAllowInsecureCredentialWhenUnset(t *testing.T) {
dir := t.TempDir()
if err := SaveTarget(dir, testTarget(t)); err != nil {
t.Fatalf("SaveTarget: %v", err)
}
raw, err := os.ReadFile(TargetPath(dir))
if err != nil {
t.Fatalf("reading sidecar: %v", err)
}
if strings.Contains(string(raw), "previous_backend") {
t.Errorf("sidecar carries previous_backend with nothing to restore: %s", raw)
}
if strings.Contains(string(raw), "allow_plaintext") {
t.Errorf("sidecar carries allow_plaintext with no grant: %s", raw)
}
}

View file

@ -227,11 +227,6 @@ func (unsupportedDoltStorage) ForcePush(_ context.Context) (err error) {
return
}
func (unsupportedDoltStorage) GetAllDependencyRecords(_ context.Context) (_ map[string][]*types.Dependency, err error) {
err = errUnsupported("GetAllDependencyRecords")
return
}
func (unsupportedDoltStorage) GetAllEventsSince(_ context.Context, _ time.Time) (_ []*types.Event, err error) {
err = errUnsupported("GetAllEventsSince")
return
@ -742,4 +737,4 @@ func (unsupportedDoltStorage) VersionReconciler() (_ issueops.VersionReconciler,
return
}
// NOTE: partial shell (145 of 200 methods generated; 55 left to this package's hand-written set).
// NOTE: partial shell (144 of 200 methods generated; 56 left to this package's hand-written set).

View file

@ -240,18 +240,16 @@ const CapIssuesCreateDefaultPriority = "issues.create.defaultPriority"
// through the policy layer, so httpapi.Capabilities() — the build-level list
// the parity gate compares against — never contains it.
//
// UNCONSUMED since S3 reconciliation (2026-10): THIS token is unread by this
// client, not because OSS lacks external-dependency policy enforcement — the
// externaldeps decorator (internal/storage/externaldeps) exists in OSS and is
// wired unconditionally into the local storage chain (cmd/bd/storage_chain.go),
// so a local backend already applies the policy itself. What this http
// client specifically lacks is a client-side COMPOSITION of that same
// decorator around a storage.ExternalDependencyQueryStore-backed remote
// store, because a remote server advertising this capability applies the
// policy on its own side before answering — there is nothing left for the
// client to decorate. The token stays declared, because the server-side
// capability and its wire spelling are real and S10 schedules the client half
// that reads it.
// Consumed since S6 (design 3.6): the externaldeps decorator
// (internal/storage/externaldeps) wraps the http store the same as any local
// backend (cmd/bd/storage_chain.go wires it unconditionally), and consults
// storage.ExternalDependencyPolicyProber — which httpclient.Store implements
// by checking for this token in the handshake — before deciding whether to
// also enforce the policy client-side. A server that advertises this token
// has already applied the policy itself, so the decorator's own check is
// redundant there. A server that does NOT advertise it gets the ordinary
// client-side enforcement: the policy is never silently skipped merely
// because the store is remote.
const CapExternalDependencies = "policy.external_dependencies"
// ClientWireRevision is the wire shape this client was built to speak and

View file

@ -5,6 +5,7 @@ go_library(
srcs = [
"batch_closer.go",
"decorator.go",
"remote_roles.go",
"resolver.go",
"role_accessors.go",
"uow_decorator.go",
@ -30,6 +31,7 @@ go_test(
srcs = [
"decorator_test.go",
"ready_wake_test.go",
"remote_roles_test.go",
"uow_decorator_test.go",
],
embed = [":externaldeps"],

View file

@ -10,7 +10,12 @@ import (
)
// BatchCloser preserves external policy on bd close, including its next claim.
// Over a store that serves its roles, remoteBatchCloser refuses the next claim
// instead.
func (s *Store) BatchCloser() (issueops.BatchCloser, error) {
if s.servesRoles() {
return &remoteBatchCloser{policy: s}, nil
}
return &batchCloser{policy: s}, nil
}

View file

@ -119,6 +119,19 @@ type blockingState struct {
}
func (s *Store) loadBlockingState(ctx context.Context) (blockingState, error) {
// Design 3.6: a remote server that already enforces this policy itself
// (advertised as the handshake's wire.CapExternalDependencies capability,
// probed here through storage.ExternalDependencyPolicyProber) has already
// applied it before answering, so this decorator's own pass would be
// redundant — skip to an empty blocking state. A store that does not
// implement the prober, or that implements it and reports false (a remote
// server silent on the capability included), falls through to the
// ordinary client-side enforcement below: the policy is never silently
// skipped merely because the inner store is remote.
if enforced, err := s.serverEnforcesPolicy(ctx); err != nil || enforced {
return blockingState{}, err
}
queryStore, ok := storage.UnwrapStore(s.inner).(storage.ExternalDependencyQueryStore)
var allDeps map[string][]*types.Dependency
var err error
@ -136,6 +149,20 @@ func (s *Store) loadBlockingState(ctx context.Context) (blockingState, error) {
return s.blockingStateFromRecords(ctx, allDeps)
}
// serverEnforcesPolicy asks the inner store's ExternalDependencyPolicyProber,
// if it has one, whether its server already enforced this policy.
func (s *Store) serverEnforcesPolicy(ctx context.Context) (bool, error) {
prober, ok := storage.UnwrapStore(s.inner).(storage.ExternalDependencyPolicyProber)
if !ok {
return false, nil
}
enforced, err := prober.ServerEnforcesExternalDependencyPolicy(ctx)
if err != nil {
return false, fmt.Errorf("external dependencies: probe server policy: %w", err)
}
return enforced, nil
}
func (s *Store) blockingStateFromRecords(ctx context.Context, allDeps map[string][]*types.Dependency) (blockingState, error) {
refs := make([]reference, 0)
refsByIssue := make(map[string][]string)
@ -176,8 +203,16 @@ func (s *Store) GetReadyWork(ctx context.Context, filter types.WorkFilter) ([]*t
if err != nil {
return nil, err
}
filter = withExternalExclusions(filter, state.refsByIssue)
return s.inner.GetReadyWork(ctx, filter)
queryFilter, blockedIDs := s.readyExclusion(filter, state.refsByIssue)
issues, err := s.inner.GetReadyWork(ctx, queryFilter)
if err != nil {
return nil, err
}
kept := dropBlockedIssues(issues, blockedIDs, filter.Limit)
if err := capKeptRows(len(kept), filter, blockedIDs); err != nil {
return nil, err
}
return kept, nil
}
// GetReadyWorkWithCounts is the counts-bearing equivalent of GetReadyWork.
@ -186,8 +221,23 @@ func (s *Store) GetReadyWorkWithCounts(ctx context.Context, filter types.WorkFil
if err != nil {
return nil, err
}
filter = withExternalExclusions(filter, state.refsByIssue)
return s.inner.GetReadyWorkWithCounts(ctx, filter)
return s.readyWorkWithCounts(ctx, filter, state)
}
// readyWorkWithCounts is GetReadyWorkWithCounts' body against an
// already-loaded blockingState, which remoteReadyClaimer reuses across its
// passes.
func (s *Store) readyWorkWithCounts(ctx context.Context, filter types.WorkFilter, state blockingState) ([]*types.IssueWithCounts, error) {
queryFilter, blockedIDs := s.readyExclusion(filter, state.refsByIssue)
rows, err := s.inner.GetReadyWorkWithCounts(ctx, queryFilter)
if err != nil {
return nil, err
}
kept, _ := dropBlockedIssuesWithCounts(rows, blockedIDs, filter.Limit)
if err := capKeptRows(len(kept), filter, blockedIDs); err != nil {
return nil, err
}
return kept, nil
}
// GetReadyWorkWithCountsAndTotal applies the same external exclusions as
@ -199,8 +249,106 @@ func (s *Store) GetReadyWorkWithCountsAndTotal(ctx context.Context, filter types
if err != nil {
return nil, 0, err
}
filter = withExternalExclusions(filter, state.refsByIssue)
return s.inner.GetReadyWorkWithCountsAndTotal(ctx, filter)
return s.readyWorkWithCountsAndTotal(ctx, filter, state)
}
// readyExclusion resolves how this decorator applies its OWN additional
// exclusions (issues with an unsatisfied external blocker) against s.inner.
//
// A store that can express types.WorkFilter.ExcludeIDs over its own
// transport (every first-party SQL-backed store) gets them folded straight
// into the filter, exactly as before: the WHERE clause does the exclusion.
//
// A store that cannot (storage.ExcludeIDsUnsupportedStore — today only
// httpclient.Store: design 3.6 / L12 leaves listReadyWork with no
// id-exclusion parameter on the v0 wire at all) instead gets a bumped Limit
// and the exclusions applied client-side by the caller. Requesting
// Limit+len(refsByIssue) rows from the UNFILTERED ready set is provably
// enough headroom to still surface Limit genuinely-unblocked rows whenever
// that many exist: refsByIssue cannot hold more entries than the number of
// issues anywhere with an unsatisfied external blocker, so at most
// len(refsByIssue) of the bumped window's extra rows can be ones this
// decorator has to drop. Whichever path is taken, the policy itself is never
// skipped — only where the exclusion happens differs.
//
// The client-side path also takes the MaxRows cap off the inner query. That
// store caps the rows it fetched, which the bump lets outnumber Limit, so
// `--limit N --max-rows N` failed with exit 2 as soon as more than N issues
// were ready, refusing a page that could never exceed N. The caller enforces
// the cap on the rows it keeps instead (capKeptRows), after the Limit trim,
// which is where issueops enforces it on the rows a SQL store delivers
// (finishReadyWorkWithCounts).
func (s *Store) readyExclusion(filter types.WorkFilter, refsByIssue map[string][]string) (queryFilter types.WorkFilter, blockedIDs map[string]bool) {
if len(refsByIssue) == 0 {
return filter, nil
}
if unsupported, ok := storage.UnwrapStore(s.inner).(storage.ExcludeIDsUnsupportedStore); !ok || !unsupported.ExcludeIDsUnsupported() {
return withExternalExclusions(filter, refsByIssue), nil
}
blockedIDs = make(map[string]bool, len(refsByIssue))
for issueID := range refsByIssue {
blockedIDs[issueID] = true
}
queryFilter = filter
if queryFilter.Limit > 0 {
queryFilter.Limit += len(blockedIDs)
}
queryFilter.MaxRows = 0
queryFilter.MaxRowsSource = ""
return queryFilter, blockedIDs
}
// capKeptRows enforces the MaxRows cap readyExclusion took off the inner
// query, on the kept rows the caller is handed. With nothing to drop
// client-side the inner query kept the cap and has already enforced it.
func capKeptRows(kept int, filter types.WorkFilter, blockedIDs map[string]bool) error {
if len(blockedIDs) == 0 {
return nil
}
return issueops.EnforceMaxRowsCap(kept, filter.MaxRows, filter.MaxRowsSource)
}
// dropBlockedIssues is readyExclusion's client-side half: it drops the rows
// blockedIDs names and trims what remains to limit. It copies into a fresh
// slice rather than compacting issues in place, so the slice s.inner returned
// still holds every row it fetched.
func dropBlockedIssues(issues []*types.Issue, blockedIDs map[string]bool, limit int) []*types.Issue {
if len(blockedIDs) == 0 {
return issues
}
kept := make([]*types.Issue, 0, len(issues))
for _, issue := range issues {
if issue != nil && blockedIDs[issue.ID] {
continue
}
kept = append(kept, issue)
}
if limit > 0 && len(kept) > limit {
kept = kept[:limit]
}
return kept
}
// dropBlockedIssuesWithCounts is dropBlockedIssues for counts-bearing rows. It
// also reports how many rows it dropped from the whole fetched window (before
// the limit trim), which is the figure readyWorkWithCountsAndTotal subtracts
// from the inner store's total.
func dropBlockedIssuesWithCounts(rows []*types.IssueWithCounts, blockedIDs map[string]bool, limit int) (kept []*types.IssueWithCounts, dropped int) {
if len(blockedIDs) == 0 {
return rows, 0
}
kept = make([]*types.IssueWithCounts, 0, len(rows))
for _, row := range rows {
if row != nil && row.Issue != nil && blockedIDs[row.ID] {
dropped++
continue
}
kept = append(kept, row)
}
if limit > 0 && len(kept) > limit {
kept = kept[:limit]
}
return kept, dropped
}
func withExternalExclusions(filter types.WorkFilter, refsByIssue map[string][]string) types.WorkFilter {
@ -216,6 +364,80 @@ func withExternalExclusions(filter types.WorkFilter, refsByIssue map[string][]st
return filter
}
// readyWorkWithCountsAndTotal is GetReadyWorkWithCountsAndTotal's body, split
// out so CountReadyWork's fallback (below) can reuse it against an
// already-loaded blockingState instead of loading it a second time.
func (s *Store) readyWorkWithCountsAndTotal(ctx context.Context, filter types.WorkFilter, state blockingState) ([]*types.IssueWithCounts, int, error) {
queryFilter, blockedIDs := s.readyExclusion(filter, state.refsByIssue)
rows, total, err := s.inner.GetReadyWorkWithCountsAndTotal(ctx, queryFilter)
if err != nil {
return nil, 0, err
}
kept, blockedSeen := dropBlockedIssuesWithCounts(rows, blockedIDs, filter.Limit)
if len(blockedIDs) == 0 {
return kept, total, nil
}
if err := capKeptRows(len(kept), filter, blockedIDs); err != nil {
return nil, 0, err
}
var adjustedTotal int
if total <= len(rows) {
// The fetch's own reported total says the window already held the
// entire ready set, so every blocked row the window contains is every
// blocked row the ready set contains: the subtraction is exact. This
// is judged from the rows that came back, not from the Limit asked
// for: a zero Limit means every row to a SQL store, but the http wire
// omits it and the server answers with its default page
// (workapi.DefaultReadyLimit), so an unlimited fetch can come back
// truncated too.
adjustedTotal = total - blockedSeen
} else {
// The ready set is bigger than the window that came back (the bumped
// window, or the server's default page for an unlimited fetch), so
// the window cannot say how many of its unseen rows are blocked. Nor
// can refsByIssue: it names every workspace issue with an unsatisfied
// external blocker, closed ones and ones the filter excludes included,
// and only those in the filtered ready set were ever in total. Size
// that set instead. bd ready's own page (above) is exact either way.
adjustedTotal, err = s.refetchedReadyTotal(ctx, filter, total, blockedIDs)
if err != nil {
return nil, 0, err
}
}
if adjustedTotal < len(kept) {
adjustedTotal = len(kept)
}
if adjustedTotal < 0 {
adjustedTotal = 0
}
return kept, adjustedTotal, nil
}
// refetchedReadyTotal sizes the externally filtered ready set when the first
// window held only part of it: it refetches the whole filtered set, ids only,
// and subtracts the blocked rows actually in it. A refetch that comes back
// truncated too (the set grew in between, or the server bounded the page)
// falls back to subtracting every blocked id, the bound least likely to
// overstate how much ready work remains.
func (s *Store) refetchedReadyTotal(ctx context.Context, filter types.WorkFilter, total int, blockedIDs map[string]bool) (int, error) {
full := filter
full.Offset = 0
full.Limit = total + len(blockedIDs)
full.MaxRows = 0
full.MaxRowsSource = ""
full.Lite = true
rows, fullTotal, err := s.inner.GetReadyWorkWithCountsAndTotal(ctx, full)
if err != nil {
return 0, err
}
if fullTotal > len(rows) {
return total - len(blockedIDs), nil
}
_, blocked := dropBlockedIssuesWithCounts(rows, blockedIDs, 0)
return fullTotal - blocked, nil
}
// CountReadyWork reports the externally filtered ready count.
func (s *Store) CountReadyWork(ctx context.Context, filter types.WorkFilter) (int, error) {
filter.Limit = 0
@ -224,15 +446,14 @@ func (s *Store) CountReadyWork(ctx context.Context, filter types.WorkFilter) (in
if err != nil {
return 0, err
}
filter = withExternalExclusions(filter, state.refsByIssue)
if counter, ok := storage.UnwrapStore(s.inner).(storage.ReadyWorkCounter); ok {
return counter.CountReadyWork(ctx, filter)
return counter.CountReadyWork(ctx, withExternalExclusions(filter, state.refsByIssue))
}
issues, err := s.inner.GetReadyWork(ctx, filter)
if err != nil {
return 0, err
}
return len(issues), nil
// s.inner has no indexed counter (httpclient.Store among others): reuse
// the counts-and-total path above, which already copes with a store whose
// transport cannot express ExcludeIDs at all, nor this zero Limit.
_, total, err := s.readyWorkWithCountsAndTotal(ctx, filter, state)
return total, err
}
// ReadyCounter sizes the ready set through CountReadyWork above, so the total
@ -399,6 +620,8 @@ func matchesParentFilter(issueID string, parentID *string, allDeps map[string][]
}
// IsBlocked includes explicit unsatisfied external blockers in the close guard.
// A store that serves its roles refuses the reads this makes, so over one the
// per-issue roles in remote_roles.go answer instead.
func (s *Store) IsBlocked(ctx context.Context, issueID string) (bool, []string, error) {
blocked, blockers, err := s.inner.IsBlocked(ctx, issueID)
if err != nil {
@ -429,7 +652,8 @@ func (s *Store) IsBlocked(ctx context.Context, issueID string) (bool, []string,
// IsBlockedBatch preserves the external blocker invariant for batch callers.
// The embedded Dolt implementation promotes this method from the wrapped
// store, so it must be declared explicitly here rather than relying on
// IsBlocked alone.
// IsBlocked alone. Like IsBlocked, it refuses over a store that serves its
// roles.
func (s *Store) IsBlockedBatch(ctx context.Context, issueIDs []string) (map[string]bool, error) {
blocked, err := s.inner.IsBlockedBatch(ctx, issueIDs)
if err != nil {
@ -463,7 +687,9 @@ func (s *Store) IsBlockedBatch(ctx context.Context, issueIDs []string) (map[stri
// CloseIssueChecked applies the external guard before the atomic local close.
// The local store cannot see foreign capability state, so promoting its method
// would allow an externally blocked issue to close without --force.
// would allow an externally blocked issue to close without --force. A store
// that serves its roles refuses this method; bd close goes through
// remoteBatchCloser (remote_roles.go) there.
func (s *Store) CloseIssueChecked(ctx context.Context, issueID, actor string, opts storage.CloseIssueOptions) (storage.CloseIssueResult, error) {
if !opts.Force {
issue, err := s.inner.GetIssue(ctx, issueID)
@ -484,7 +710,9 @@ func (s *Store) CloseIssueChecked(ctx context.Context, issueID, actor string, op
}
// GetDependencyTree appends external refs as synthetic leaf nodes because no
// local issue row exists for the normal graph hydrator to return.
// local issue row exists for the normal graph hydrator to return. A store that
// serves its roles refuses the reads this makes; bd dep tree goes through
// remoteTreeWalker (remote_roles.go) there.
func (s *Store) GetDependencyTree(ctx context.Context, issueID string, maxDepth int, showAllPaths bool, reverse bool) ([]*types.TreeNode, error) {
tree, err := s.inner.GetDependencyTree(ctx, issueID, maxDepth, showAllPaths, reverse)
if err != nil || reverse || len(tree) == 0 {

View file

@ -3,6 +3,7 @@ package externaldeps
import (
"context"
"errors"
"fmt"
"slices"
"testing"
"time"
@ -15,22 +16,48 @@ import (
type fakeStore struct {
storage.DoltStorage
ready []*types.Issue
blocked []*types.BlockedIssue
tree []*types.TreeNode
deps map[string][]*types.Dependency
labels map[string][]*types.Issue
labelErr error
claimed []string
isBlocked bool
blockerIDs []string
closed []string
lifecycle publicops.Lifecycle
batch *fakeBatchCloser
ready []*types.Issue
blocked []*types.BlockedIssue
tree []*types.TreeNode
deps map[string][]*types.Dependency
labels map[string][]*types.Issue
labelErr error
claimed []string
isBlocked bool
blockerIDs []string
closed []string
lifecycle publicops.Lifecycle
batch *fakeBatchCloser
enforced bool
enforcedErr error
excludeIDsUnsupported bool
// defaultPage, when positive, is how many rows a zero-Limit ready listing
// returns, while GetReadyWorkWithCountsAndTotal's total still counts the
// whole set: httpclient.Store's shape, whose wire omits a zero limit so
// the server answers with its default page. Zero keeps the SQL stores'
// meaning, every row.
defaultPage int
}
func (f *fakeStore) IssueLifecycle() (publicops.Lifecycle, error) { return f.lifecycle, nil }
// ServerEnforcesExternalDependencyPolicy makes every fakeStore satisfy
// storage.ExternalDependencyPolicyProber. Defaulting to (false, nil) keeps
// every existing test's raw store on today's client-side-enforcement path
// unchanged; enforced:true is set explicitly only by the test below that
// pins loadBlockingState's skip branch.
func (f *fakeStore) ServerEnforcesExternalDependencyPolicy(_ context.Context) (bool, error) {
return f.enforced, f.enforcedErr
}
// ExcludeIDsUnsupported likewise makes every fakeStore satisfy
// storage.ExcludeIDsUnsupportedStore. Defaulting to false keeps every existing
// test's exclusions folded into WorkFilter.ExcludeIDs; excludeIDsUnsupported:
// true models httpclient.Store, whose ready listings refuse ExcludeIDs (see
// GetReadyWork below), so the decorator must drop blocked rows client-side.
func (f *fakeStore) ExcludeIDsUnsupported() bool { return f.excludeIDsUnsupported }
// BatchCloser without BatchCloserWithPolicy models a backend that predates
// storage.PolicyBatchCloserSource.
func (f *fakeStore) BatchCloser() (publicops.BatchCloser, error) { return f.batch, nil }
@ -59,13 +86,29 @@ func (f *fakeLifecycle) Update(context.Context, publicops.UpdateRequest) (public
}
func (f *fakeStore) GetReadyWork(_ context.Context, filter types.WorkFilter) ([]*types.Issue, error) {
candidates, err := f.readyCandidates(filter)
if err != nil {
return nil, err
}
limit := filter.Limit
if limit <= 0 {
limit = f.defaultPage
}
return page(candidates, filter.Offset, limit), nil
}
// readyCandidates is the whole ready set filter admits, before paging.
func (f *fakeStore) readyCandidates(filter types.WorkFilter) ([]*types.Issue, error) {
if f.excludeIDsUnsupported && len(filter.ExcludeIDs) > 0 {
return nil, errors.New("fakeStore: ExcludeIDs cannot be expressed")
}
candidates := make([]*types.Issue, 0, len(f.ready))
for _, issue := range f.ready {
if !slices.Contains(filter.ExcludeIDs, issue.ID) {
candidates = append(candidates, issue)
}
}
return page(candidates, filter.Offset, filter.Limit), nil
return candidates, nil
}
func (f *fakeStore) GetReadyWorkWithCounts(ctx context.Context, filter types.WorkFilter) ([]*types.IssueWithCounts, error) {
@ -85,9 +128,7 @@ func (f *fakeStore) GetReadyWorkWithCountsAndTotal(ctx context.Context, filter t
if err != nil {
return nil, 0, err
}
unbounded := filter
unbounded.Limit, unbounded.Offset = 0, 0
all, err := f.GetReadyWork(ctx, unbounded)
all, err := f.readyCandidates(filter)
if err != nil {
return nil, 0, err
}
@ -242,7 +283,7 @@ func externalDep(source, ref string, depType types.DependencyType) *types.Depend
return &types.Dependency{IssueID: source, DependsOnID: ref, Type: depType}
}
func testStore(raw, foreign *fakeStore, configured bool) *Store {
func testStore(raw storage.DoltStorage, foreign *fakeStore, configured bool) *Store {
return New(
raw,
func(project ProjectName) (string, bool) {
@ -321,6 +362,46 @@ func TestGetReadyWorkFailsClosedForUnconfiguredProject(t *testing.T) {
}
}
// TestLoadBlockingStateSkipsClientSideEnforcementWhenServerReportsItEnforced
// pins design 3.6's skip branch in loadBlockingState (internal/storage/
// externaldeps/decorator.go) on its CURRENT shape: a remote store
// implementing storage.ExternalDependencyPolicyProber that reports
// enforced=true must stop this decorator from applying its own client-side
// exclusion at all. The S6 review's M5 mutation targeted an earlier, cruder
// "skip whenever the backend is remote" implementation that the HIGH-1 fix
// replaced with this probe-based design, so M5's exact mutation target no
// longer exists; this test gives the probe's own skip branch equivalent
// mutation coverage. Nothing else in the suite exercises enforced==true: see
// internal/httpclient/served_external_dependency_policy_test.go, which
// documents exercising only enforced==false against a real served server
// (OSS httpapi never advertises the capability).
//
// configured=false on the foreign-project locator reproduces the fail-closed
// path from TestGetReadyWorkFailsClosedForUnconfiguredProject: if the
// `if enforced { return blockingState{}, nil }` skip were removed or
// inverted, this would fall through to ordinary client-side enforcement,
// which cannot resolve "remote" and would exclude be-a. Seeing be-a survive
// is therefore proof the skip branch fired.
func TestLoadBlockingStateSkipsClientSideEnforcementWhenServerReportsItEnforced(t *testing.T) {
a := issue("be-a")
raw := &fakeStore{
ready: []*types.Issue{a},
deps: map[string][]*types.Dependency{
a.ID: {externalDep(a.ID, "external:remote:payments", types.DepBlocks)},
},
enforced: true,
}
store := testStore(raw, &fakeStore{}, false)
got, err := store.GetReadyWork(t.Context(), types.WorkFilter{})
if err != nil {
t.Fatalf("GetReadyWork: %v", err)
}
if ids := issueIDs(got); !slices.Equal(ids, []string{a.ID}) {
t.Fatalf("ready IDs = %v, want [%s] (a server-enforced probe should skip this decorator's own client-side exclusion)", ids, a.ID)
}
}
func TestGetReadyWorkFailsClosedWhenForeignReadFails(t *testing.T) {
a := issue("be-a")
raw := &fakeStore{ready: []*types.Issue{a}, deps: map[string][]*types.Dependency{
@ -464,6 +545,183 @@ func TestReadyTotalsAgreeAcrossOutputModes(t *testing.T) {
}
}
// TestReadyTotalsWhenStoreCannotExpressExcludeIDs pins the path httpclient.Store
// takes: the decorator bumps Limit and drops externally blocked rows
// client-side, and the page, its total, and CountReadyWork must all still
// exclude them. be-a comes back first, ahead of the rows kept — the order in
// which compacting the fetched rows in place once overwrote be-a before the
// total's blocked-row count reached it, so the total came out 3, not 2.
func TestReadyTotalsWhenStoreCannotExpressExcludeIDs(t *testing.T) {
a, b, c := issue("be-a"), issue("be-b"), issue("be-c")
raw := &fakeStore{
ready: []*types.Issue{a, b, c},
deps: map[string][]*types.Dependency{
a.ID: {externalDep(a.ID, "external:remote:payments", types.DepBlocks)},
},
excludeIDsUnsupported: true,
}
store := testStore(raw, &fakeStore{}, true)
for _, tc := range []struct {
name string
limit int
wantIDs []string
}{
{name: "unlimited", limit: 0, wantIDs: []string{b.ID, c.ID}},
// The bumped window (2+1) holds the whole ready set: the total is exact.
{name: "window holds the set", limit: 2, wantIDs: []string{b.ID, c.ID}},
// The ready set outgrows the bumped window (1+1): the total is the
// inner total less every externally blocked issue.
{name: "set outgrows the window", limit: 1, wantIDs: []string{b.ID}},
} {
t.Run(tc.name, func(t *testing.T) {
filter := types.WorkFilter{Limit: tc.limit}
issues, err := store.GetReadyWork(t.Context(), filter)
if err != nil {
t.Fatalf("GetReadyWork: %v", err)
}
if ids := issueIDs(issues); !slices.Equal(ids, tc.wantIDs) {
t.Fatalf("GetReadyWork IDs = %v, want %v", ids, tc.wantIDs)
}
rows, total, err := store.GetReadyWorkWithCountsAndTotal(t.Context(), filter)
if err != nil {
t.Fatalf("GetReadyWorkWithCountsAndTotal: %v", err)
}
ids := make([]string, 0, len(rows))
for _, row := range rows {
ids = append(ids, row.ID)
}
if !slices.Equal(ids, tc.wantIDs) || total != 2 {
t.Fatalf("page = %v, total = %d; want %v, total 2 (be-a is externally blocked)", ids, total, tc.wantIDs)
}
})
}
got, err := store.CountReadyWork(t.Context(), types.WorkFilter{})
if err != nil {
t.Fatalf("CountReadyWork: %v", err)
}
if got != 2 {
t.Fatalf("ready count = %d, want 2 (be-a is externally blocked)", got)
}
}
// TestReadyTotalsWhenAnUnlimitedFetchComesBackAsOnePage pins the shape
// httpclient.Store has and an every-row Limit==0 fake hid: the wire omits a
// zero Limit, the server answers with its default page, and the total still
// counts the whole ready set. CountReadyWork, text-mode bd ready's total,
// always fetches with Limit 0, so be-z (ready, externally blocked, and past
// that page) once stayed in its total because the decorator took a zero Limit
// for a window that held every row.
func TestReadyTotalsWhenAnUnlimitedFetchComesBackAsOnePage(t *testing.T) {
const defaultPage = 100 // workapi.DefaultReadyLimit, the server's page for an omitted limit
ready := make([]*types.Issue, 0, defaultPage+2)
for i := range defaultPage + 1 {
ready = append(ready, issue(fmt.Sprintf("be-%03d", i)))
}
z := issue("be-z")
raw := &fakeStore{
ready: append(ready, z),
deps: map[string][]*types.Dependency{
z.ID: {externalDep(z.ID, "external:remote:payments", types.DepBlocks)},
},
excludeIDsUnsupported: true,
defaultPage: defaultPage,
}
store := testStore(raw, &fakeStore{}, true)
const want = defaultPage + 1 // every ready issue but be-z
counter, err := store.ReadyCounter()
if err != nil {
t.Fatalf("ReadyCounter: %v", err)
}
text, err := counter.CountReady(t.Context(), publicops.ReadyRequest{})
if err != nil {
t.Fatalf("CountReady: %v", err)
}
if text.Total != want {
t.Fatalf("text total = %d, want %d (be-z is externally blocked)", text.Total, want)
}
// The JSON total must agree, whether the page is unlimited (the server's
// default page comes back) or the CLI's default limit (bumped by one).
for _, limit := range []int{0, defaultPage} {
rows, total, err := store.GetReadyWorkWithCountsAndTotal(t.Context(), types.WorkFilter{Limit: limit})
if err != nil {
t.Fatalf("GetReadyWorkWithCountsAndTotal(Limit %d): %v", limit, err)
}
if len(rows) != defaultPage || total != want {
t.Fatalf("Limit %d: %d rows, total %d; want %d rows, total %d (be-z is externally blocked)", limit, len(rows), total, defaultPage, want)
}
}
}
// TestReadyTotalsIgnoreExternallyBlockedIssuesOutsideTheReadySet pins the
// total of a ready set bigger than the window that came back. refsByIssue
// names every workspace issue with an unsatisfied external blocker, closed
// ones and ones the filter excludes included, but only those in the filtered
// ready set were ever in the inner total, so subtracting all of them once
// reported 100 of 101 ready issues as soon as an issue outside the ready set
// held an external blocker.
func TestReadyTotalsIgnoreExternallyBlockedIssuesOutsideTheReadySet(t *testing.T) {
const defaultPage = 100 // workapi.DefaultReadyLimit, the server's page for an omitted limit
for _, tc := range []struct {
name string
// readyBlocked adds be-z: ready, externally blocked, and past the page.
readyBlocked bool
// outside is how many externally blocked issues are not ready.
outside int
}{
{name: "one blocked issue outside the ready set", outside: 1},
{name: "blocked issues inside and outside the ready set", readyBlocked: true, outside: 20},
} {
t.Run(tc.name, func(t *testing.T) {
ready := make([]*types.Issue, 0, defaultPage+2)
for i := range defaultPage + 1 {
ready = append(ready, issue(fmt.Sprintf("be-%03d", i)))
}
deps := make(map[string][]*types.Dependency)
if tc.readyBlocked {
z := issue("be-z")
ready = append(ready, z)
deps[z.ID] = []*types.Dependency{externalDep(z.ID, "external:remote:payments", types.DepBlocks)}
}
for i := range tc.outside {
id := fmt.Sprintf("be-done-%02d", i)
deps[id] = []*types.Dependency{externalDep(id, "external:remote:payments", types.DepBlocks)}
}
raw := &fakeStore{
ready: ready,
deps: deps,
excludeIDsUnsupported: true,
defaultPage: defaultPage,
}
store := testStore(raw, &fakeStore{}, true)
const want = defaultPage + 1 // every ready issue but be-z
counter, err := store.ReadyCounter()
if err != nil {
t.Fatalf("ReadyCounter: %v", err)
}
text, err := counter.CountReady(t.Context(), publicops.ReadyRequest{})
if err != nil {
t.Fatalf("CountReady: %v", err)
}
if text.Total != want {
t.Fatalf("text total = %d, want %d", text.Total, want)
}
for _, limit := range []int{0, defaultPage} {
rows, total, err := store.GetReadyWorkWithCountsAndTotal(t.Context(), types.WorkFilter{Limit: limit})
if err != nil {
t.Fatalf("GetReadyWorkWithCountsAndTotal(Limit %d): %v", limit, err)
}
if len(rows) != defaultPage || total != want {
t.Fatalf("Limit %d: %d rows, total %d; want %d rows, total %d", limit, len(rows), total, defaultPage, want)
}
}
})
}
}
func TestGetBlockedIssuesAddsUnsatisfiedExternalRefs(t *testing.T) {
a, c := issue("be-a"), issue("be-c")
raw := &fakeStore{

View file

@ -0,0 +1,412 @@
package externaldeps
import (
"context"
"errors"
"fmt"
"github.com/steveyegge/beads/internal/storage"
storageissueops "github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/types"
"github.com/steveyegge/beads/internal/workapi"
"github.com/steveyegge/beads/issueops"
)
// Remote role composition.
//
// The base accessors (role_accessors.go, batch_closer.go) rebuild each role
// over this store's legacy methods, which a SQL store implements in full. A
// remote store does not: httpclient.Store serves its roles over the wire and
// refuses most of the legacy methods those builds call (SearchIssuesWithCounts,
// IsBlocked, GetDependencyRecordsForIssues, GetDependencyTree,
// ClaimReadyIssue, BatchCloserWithPolicy), so the base accessors would fail
// `bd list`, `bd dep tree`, `bd ready --claim` and a policed close even in a
// workspace with no external dependency at all. Over such a store each
// accessor starts from the inner store's own role instead and layers the
// policy around it, before or after the served call.
//
// storage.RemoteBackendStore chooses that composition and nothing else.
// Whether the policy runs is still serverEnforcesPolicy's question (design
// 3.6), asked on every path that loads blocking state.
// servesRoles reports whether the inner store serves its roles rather than
// building them over its legacy methods.
func (s *Store) servesRoles() bool {
remote, ok := storage.UnwrapStore(s.inner).(storage.RemoteBackendStore)
return ok && remote.IsRemoteBackendStore()
}
// servedEdgeAnchors is the most anchors one served edge read names: the wire
// caps listDependencies' issue_id at 100 items, and the client passes a
// request through without chunking it.
const servedEdgeAnchors = 100
// servedEdges reads the stored outgoing edges of ids through the inner store's
// EdgeReader, the served stand-in for GetDependencyRecordsForIssues. A missing
// anchor contributes no edges.
func (s *Store) servedEdges(ctx context.Context, ids []string) (map[string][]*types.Dependency, error) {
anchors := make([]string, 0, len(ids))
named := make(map[string]bool, len(ids))
for _, id := range ids {
if id != "" && !named[id] {
named[id] = true
anchors = append(anchors, id)
}
}
deps := make(map[string][]*types.Dependency, len(anchors))
if len(anchors) == 0 {
return deps, nil
}
reader, err := s.inner.EdgeReader()
if err != nil {
return nil, err
}
for start := 0; start < len(anchors); start += servedEdgeAnchors {
end := min(start+servedEdgeAnchors, len(anchors))
result, err := reader.ReadEdges(ctx, issueops.EdgeReadRequest{IDs: anchors[start:end]})
if err != nil {
return nil, err
}
for _, anchor := range result.Anchors {
deps[anchor.ID] = append(deps[anchor.ID], anchor.Edges...)
}
}
return deps, nil
}
// blockingStateFor is loadBlockingState narrowed to ids, for a role that names
// its issues and so needs only their edges.
func (s *Store) blockingStateFor(ctx context.Context, ids []string) (blockingState, error) {
if enforced, err := s.serverEnforcesPolicy(ctx); err != nil || enforced {
return blockingState{}, err
}
deps, err := s.servedEdges(ctx, ids)
if err != nil {
return blockingState{}, fmt.Errorf("external dependencies: list blocking records: %w", err)
}
return s.blockingStateFromRecords(ctx, deps)
}
// plainDownTree reports whether req is the walk the policy decorates. Reverse
// walks do not follow a source's dependencies, and the status- or row-bounded
// variants keep the backend's own traversal semantics.
func plainDownTree(req issueops.WalkTreeRequest) bool {
return (req.Direction == "" || req.Direction == issueops.TreeDown) && req.Status == "" && req.MaxRows == 0
}
// isLostClaimRace classifies the refusals that mean someone else took a
// candidate first. ErrNotFound joins them, as in httpclient's own composed
// claim: a ready row deleted before it was claimed is the same situation for
// a caller that wants the next piece of work.
func isLostClaimRace(err error) bool {
return errors.Is(err, issueops.ErrAlreadyClaimed) ||
errors.Is(err, issueops.ErrNotClaimable) ||
errors.Is(err, issueops.ErrNotFound)
}
// remoteReader serves List and Get from the inner store's reader: this store
// overrides no search or detail method, so neither loses any policy. Ready and
// a ready-flagged List go through the policy reader, whose ready listing drops
// externally blocked work.
type remoteReader struct {
served issueops.Reader
policy issueops.Reader
}
func (r *remoteReader) Ready(ctx context.Context, req issueops.ReadyRequest) (issueops.IssuePage, error) {
return r.policy.Ready(ctx, req)
}
func (r *remoteReader) List(ctx context.Context, req issueops.ListRequest) (issueops.IssuePage, error) {
if req.ReadyFlag {
return r.policy.List(ctx, req)
}
return r.served.List(ctx, req)
}
func (r *remoteReader) Get(ctx context.Context, req issueops.GetRequest) (*issueops.IssueDetails, error) {
return r.served.Get(ctx, req)
}
// remoteClaimer refuses an externally blocked claim before the served one.
// Local blockers are the served claim's own answer: IsBlocked is a refused
// legacy method on a remote store, and the server checks its own graph.
type remoteClaimer struct {
served issueops.Claimer
policy *Store
}
func (c *remoteClaimer) Claim(ctx context.Context, req issueops.ClaimRequest) (issueops.ClaimResult, error) {
state, err := c.policy.blockingStateFor(ctx, []string{req.IssueID})
if err != nil {
return issueops.ClaimResult{}, err
}
if refs := state.refsByIssue[req.IssueID]; len(refs) > 0 {
return issueops.ClaimResult{}, fmt.Errorf("%w: %s is blocked by %v", storage.ErrCloseBlocked, req.IssueID, refs)
}
return c.served.Claim(ctx, req)
}
// remoteClaimPage is how many externally unblocked candidates one pass of
// remoteReadyClaimer reads, matching httpclient's composed ready claim.
const remoteClaimPage = 25
// remoteReadyClaimer hands the claim to the served ReadyClaimer when no issue
// in the workspace holds an unsatisfied external ref. Otherwise the served
// claim cannot be told what to skip — the wire's claim-next has no exclusion
// parameter — so it reads the policy's filtered ready page and claims down it
// through the served Claimer, as httpclient's own composition does for an
// older server. The same window applies: a row that gains a blocker between
// the listing and its claim can still be claimed.
//
// A closed holder counts too, as it does in every backend's blocking state.
// It can never be a candidate, so the composed claim takes the same issue the
// served one would. Leaving it out would cost a status read per holder, since
// the wire has no batch form of one.
type remoteReadyClaimer struct{ policy *Store }
func (c *remoteReadyClaimer) ClaimNext(ctx context.Context, req issueops.ClaimNextRequest) (issueops.ClaimNextResult, error) {
if err := storageissueops.ValidateClaimNextRequest(req); err != nil {
return issueops.ClaimNextResult{}, err
}
filter, err := workapi.BuildReadyFilter(req.Filter)
if err != nil {
return issueops.ClaimNextResult{}, err
}
state, err := c.policy.loadBlockingState(ctx)
if err != nil {
return issueops.ClaimNextResult{}, err
}
if len(state.refsByIssue) == 0 {
served, err := c.policy.inner.ReadyClaimer()
if err != nil {
return issueops.ClaimNextResult{}, err
}
return served.ClaimNext(ctx, req)
}
claimer, err := c.policy.inner.IssueClaimer()
if err != nil {
return issueops.ClaimNextResult{}, err
}
// The candidates a local claim-next selects (ClaimReadyIssueInTx): open,
// unassigned, with no row cap on a scan that delivers one row.
filter.Status = types.StatusOpen
filter.Unassigned = true
filter.Assignee = nil
filter.MaxRows = 0
filter.MaxRowsSource = ""
filter.Limit = remoteClaimPage
// A second pass only after a full page was lost: claimed rows leave the
// unassigned set, so it reads the candidates behind them.
var lost int
for range 2 {
rows, err := c.policy.readyWorkWithCounts(ctx, filter, state)
if err != nil {
return issueops.ClaimNextResult{}, err
}
if len(rows) == 0 {
return issueops.ClaimNextResult{}, nil
}
for _, row := range rows {
if row == nil || row.Issue == nil {
continue
}
res, err := claimer.Claim(ctx, issueops.ClaimRequest{Actor: req.Actor, IssueID: row.ID})
switch {
case err == nil:
return issueops.ClaimNextResult{Claimed: &types.IssueWithCounts{
Issue: res.Issue,
DependencyCount: row.DependencyCount,
DependentCount: row.DependentCount,
CommentCount: row.CommentCount,
Parent: row.Parent,
}}, nil
case isLostClaimRace(err):
lost++
default:
return issueops.ClaimNextResult{}, err
}
}
if len(rows) < remoteClaimPage {
break
}
}
return issueops.ClaimNextResult{}, fmt.Errorf("claim ready: lost %d races for externally unblocked ready work; re-run to take the next one", lost)
}
// remoteBlockingAnnotator merges the external blockers the policy resolves
// into the served annotation's BlockedBy.
type remoteBlockingAnnotator struct {
served issueops.BlockingAnnotator
policy *Store
}
func (a *remoteBlockingAnnotator) AnnotateBlocking(ctx context.Context, req issueops.BlockingRequest) (issueops.BlockingResult, error) {
result, err := a.served.AnnotateBlocking(ctx, req)
if err != nil || len(result.Items) == 0 {
return result, err
}
ids := make([]string, 0, len(result.Items))
for _, item := range result.Items {
ids = append(ids, item.ID)
}
state, err := a.policy.blockingStateFor(ctx, ids)
if err != nil {
return issueops.BlockingResult{}, err
}
for i := range result.Items {
for _, ref := range state.refsByIssue[result.Items[i].ID] {
result.Items[i].BlockedBy = appendUnique(result.Items[i].BlockedBy, ref)
}
}
return result, nil
}
// remoteTreeWalker adds external leaves to the served down-tree. A leaf the
// server already rendered is kept as it came (appendTreeExternalReferences
// skips an id the tree holds), and like GetDependencyTree the leaves are
// rendered whether or not the server enforces the policy: a leaf is display,
// not enforcement.
type remoteTreeWalker struct {
served issueops.TreeWalker
policy *Store
}
func (t *remoteTreeWalker) WalkTree(ctx context.Context, req issueops.WalkTreeRequest) (issueops.TreeResult, error) {
result, err := t.served.WalkTree(ctx, req)
if err != nil || !plainDownTree(req) || len(result.Nodes) == 0 {
return result, err
}
ids := make([]string, 0, len(result.Nodes))
for _, node := range result.Nodes {
if node != nil && !isExternalReference(node.ID) {
ids = append(ids, node.ID)
}
}
deps, err := t.policy.servedEdges(ctx, ids)
if err != nil {
return issueops.TreeResult{}, fmt.Errorf("external dependencies: load tree edges: %w", err)
}
nodes, err := t.policy.appendTreeExternalReferences(ctx, result.Nodes, deps, req.MaxDepth, false)
if err != nil {
return issueops.TreeResult{}, err
}
result.Nodes = nodes
return result, nil
}
// remoteBatchCloser refuses each externally blocked, still-open item the way
// the local policy close does, then sends the rest to the served closer as one
// request, so they still close in one server transaction. The external answer
// is read before that request rather than inside it, the same window
// remoteReadyClaimer has.
//
// A batch that also claims the next issue is refused whole, before anything
// closes. The local closer claims inside the closing transaction with the
// policy's exclusions in its filter. A served close has no parameter to carry
// them, and claiming after the close commits would split the role's one
// transaction in two. The refusal does not read the workspace, so the command
// fails the same way whether or not anything in it is externally blocked.
type remoteBatchCloser struct{ policy *Store }
func (c *remoteBatchCloser) CloseBatch(ctx context.Context, request issueops.CloseBatchRequest) (issueops.CloseBatchResult, error) {
if request.ClaimNext != nil {
return c.closeAndClaimNext(ctx, request)
}
if request.Force {
// Force waives the policy, so the base closer reaches the served one
// with an empty policy.
return (&batchCloser{policy: c.policy}).CloseBatch(ctx, request)
}
if err := storageissueops.ValidateCloseBatchRequest(request); err != nil {
return issueops.CloseBatchResult{}, err
}
ids := make([]string, 0, len(request.Items))
for _, item := range request.Items {
ids = append(ids, item.IssueID)
}
state, err := c.policy.blockingStateFor(ctx, ids)
if err != nil {
return issueops.CloseBatchResult{}, err
}
served, err := c.policy.inner.BatchCloser()
if err != nil {
return issueops.CloseBatchResult{}, err
}
if len(state.refsByIssue) == 0 {
return served.CloseBatch(ctx, request)
}
policy := storage.NewBatchClosePolicy(state.refsByIssue)
outcomes := make([]issueops.CloseOutcome, len(request.Items))
forward := request
forward.Items = make([]issueops.BatchCloseItem, 0, len(request.Items))
forwarded := make([]int, 0, len(request.Items))
for i, item := range request.Items {
if refusal := policy.CheckClose(item.IssueID, false); refusal != nil {
// As in the local close: only a live target is refused, so an
// idempotent re-close and a not-found keep their own answers.
issue, err := c.policy.inner.GetIssue(ctx, item.IssueID)
if err != nil {
outcomes[i] = issueops.CloseOutcome{IssueID: item.IssueID, Err: err}
continue
}
if issue != nil && issue.Status != types.StatusClosed {
outcomes[i] = issueops.CloseOutcome{IssueID: item.IssueID, Err: refusal}
continue
}
}
forward.Items = append(forward.Items, item)
forwarded = append(forwarded, i)
}
if len(forward.Items) == 0 {
return issueops.CloseBatchResult{Outcomes: outcomes}, nil
}
result, err := served.CloseBatch(ctx, forward)
if err != nil {
return issueops.CloseBatchResult{}, err
}
if len(result.Outcomes) != len(forward.Items) {
return issueops.CloseBatchResult{}, fmt.Errorf("close batch: served closer answered %d outcomes for %d items", len(result.Outcomes), len(forward.Items))
}
for j, outcome := range result.Outcomes {
outcomes[forwarded[j]] = outcome
}
return issueops.CloseBatchResult{Outcomes: outcomes}, nil
}
// closeAndClaimNext answers a batch with a next claim. A request that is
// invalid on every backend is still ErrValidation. A server that enforces the
// policy itself gets the request as it came; over the v0 wire its closer
// refuses a next claim too (ledger row W-CloseBatchRequest.ClaimNext).
func (c *remoteBatchCloser) closeAndClaimNext(ctx context.Context, request issueops.CloseBatchRequest) (issueops.CloseBatchResult, error) {
if err := storageissueops.ValidateCloseBatchRequest(request); err != nil {
return issueops.CloseBatchResult{}, err
}
if _, err := workapi.BuildReadyFilter(*request.ClaimNext); err != nil {
return issueops.CloseBatchResult{}, err
}
enforced, err := c.policy.serverEnforcesPolicy(ctx)
if err != nil {
return issueops.CloseBatchResult{}, err
}
if !enforced {
return issueops.CloseBatchResult{}, fmt.Errorf(
"close with a next claim: the served batch close cannot carry a next claim, so nothing was closed; close without --claim-next, then run `bd ready --claim` (%w)",
&storage.ErrUnsupported{Op: "CloseBatchRequest.ClaimNext", Backend: fmt.Sprintf("%T", storage.UnwrapStore(c.policy.inner))})
}
served, err := c.policy.inner.BatchCloser()
if err != nil {
return issueops.CloseBatchResult{}, err
}
return served.CloseBatch(ctx, request)
}
var (
_ issueops.Reader = (*remoteReader)(nil)
_ issueops.Claimer = (*remoteClaimer)(nil)
_ issueops.ReadyClaimer = (*remoteReadyClaimer)(nil)
_ issueops.BlockingAnnotator = (*remoteBlockingAnnotator)(nil)
_ issueops.TreeWalker = (*remoteTreeWalker)(nil)
_ issueops.BatchCloser = (*remoteBatchCloser)(nil)
)

View file

@ -0,0 +1,636 @@
package externaldeps
import (
"context"
"errors"
"fmt"
"slices"
"strings"
"testing"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/types"
publicops "github.com/steveyegge/beads/issueops"
)
// remoteStore models httpclient.Store for remote_roles.go. Each role it hands
// out is served over the embedded fakeStore's rows, the way a server that
// knows nothing of the policy answers it, and each legacy method a base
// accessor builds on is refused, so a composition that reaches one fails the
// test instead of passing on a fake that answers it. Like httpclient.Store it
// cannot express ExcludeIDs and has no BatchCloserWithPolicy.
type remoteStore struct {
*fakeStore
// served logs each served role call, in order.
served []string
// closes logs the item ids of each served batch close.
closes [][]string
// edgeReads logs how many anchors each served edge read named.
edgeReads []int
// scans counts the whole-workspace dependency scans read through it.
scans int
// localBlockers is the served annotation's own BlockedBy, per id.
localBlockers map[string][]string
// claimErr, when set, answers every served Claim.
claimErr error
}
var errLegacyRefused = errors.New("remoteStore: legacy method refused")
// wireEdgeAnchorCap is the most anchors the wire's listDependencies accepts.
const wireEdgeAnchorCap = 100
func newRemoteStore(ready ...*types.Issue) *remoteStore {
return &remoteStore{fakeStore: &fakeStore{
ready: ready,
deps: make(map[string][]*types.Dependency),
excludeIDsUnsupported: true,
}}
}
func (r *remoteStore) IsRemoteBackendStore() bool { return true }
// httpclient.Store serves the vocabulary a listing loads before its search.
func (r *remoteStore) GetCustomStatusesDetailed(context.Context) ([]types.CustomStatus, error) {
return nil, nil
}
func (r *remoteStore) GetCustomTypes(context.Context) ([]string, error) { return nil, nil }
func (r *remoteStore) GetInfraTypes(context.Context) map[string]bool { return nil }
func (r *remoteStore) SearchIssuesWithCounts(context.Context, string, types.IssueFilter) ([]*types.IssueWithCounts, error) {
return nil, errLegacyRefused
}
func (r *remoteStore) IsBlocked(context.Context, string) (bool, []string, error) {
return false, nil, errLegacyRefused
}
func (r *remoteStore) GetDependencyRecordsForIssues(context.Context, []string) (map[string][]*types.Dependency, error) {
return nil, errLegacyRefused
}
func (r *remoteStore) GetDependencyTree(context.Context, string, int, bool, bool) ([]*types.TreeNode, error) {
return nil, errLegacyRefused
}
func (r *remoteStore) ClaimReadyIssue(context.Context, types.WorkFilter, string) (*types.Issue, error) {
return nil, errLegacyRefused
}
func (r *remoteStore) GetAllDependencyRecords(ctx context.Context) (map[string][]*types.Dependency, error) {
r.scans++
return r.fakeStore.GetAllDependencyRecords(ctx)
}
func (r *remoteStore) GetExternalBlockingDependencyRecords(ctx context.Context) (map[string][]*types.Dependency, error) {
r.scans++
return r.fakeStore.GetExternalBlockingDependencyRecords(ctx)
}
func (r *remoteStore) IssueReader() (publicops.Reader, error) { return servedReader{r}, nil }
func (r *remoteStore) IssueClaimer() (publicops.Claimer, error) { return servedClaimer{r}, nil }
func (r *remoteStore) ReadyClaimer() (publicops.ReadyClaimer, error) {
return servedReadyClaimer{r}, nil
}
func (r *remoteStore) TreeWalker() (publicops.TreeWalker, error) { return servedWalker{r}, nil }
func (r *remoteStore) EdgeReader() (publicops.EdgeReader, error) { return servedEdgeReader{r}, nil }
func (r *remoteStore) BatchCloser() (publicops.BatchCloser, error) { return servedCloser{r}, nil }
func (r *remoteStore) BlockingAnnotator() (publicops.BlockingAnnotator, error) {
return servedAnnotator{r}, nil
}
type servedReader struct{ s *remoteStore }
func (r servedReader) Ready(context.Context, publicops.ReadyRequest) (publicops.IssuePage, error) {
return publicops.IssuePage{}, errors.New("served Ready: ready work is the policy reader's to answer")
}
func (r servedReader) List(context.Context, publicops.ListRequest) (publicops.IssuePage, error) {
r.s.served = append(r.s.served, "List")
items := make([]*types.IssueWithCounts, 0, len(r.s.ready))
for _, issue := range r.s.ready {
items = append(items, &types.IssueWithCounts{Issue: issue})
}
return publicops.IssuePage{Items: items}, nil
}
func (r servedReader) Get(ctx context.Context, req publicops.GetRequest) (*publicops.IssueDetails, error) {
r.s.served = append(r.s.served, "Get "+req.ID)
issue, err := r.s.GetIssue(ctx, req.ID)
if err != nil {
return nil, err
}
if issue == nil {
return nil, publicops.ErrNotFound
}
return &publicops.IssueDetails{Issue: *issue}, nil
}
type servedClaimer struct{ s *remoteStore }
func (c servedClaimer) Claim(ctx context.Context, req publicops.ClaimRequest) (publicops.ClaimResult, error) {
c.s.served = append(c.s.served, "Claim "+req.IssueID)
if c.s.claimErr != nil {
return publicops.ClaimResult{}, c.s.claimErr
}
if err := c.s.ClaimIssue(ctx, req.IssueID, req.Actor); err != nil {
return publicops.ClaimResult{}, err
}
issue, err := c.s.GetIssue(ctx, req.IssueID)
return publicops.ClaimResult{Issue: issue, Changed: true}, err
}
// servedReadyClaimer claims the first unassigned ready row, blind to the
// policy as a claim-next with no exclusion parameter is.
type servedReadyClaimer struct{ s *remoteStore }
func (c servedReadyClaimer) ClaimNext(ctx context.Context, req publicops.ClaimNextRequest) (publicops.ClaimNextResult, error) {
c.s.served = append(c.s.served, "ClaimNext")
for _, issue := range c.s.ready {
if issue.Assignee != "" {
continue
}
if err := c.s.ClaimIssue(ctx, issue.ID, req.Actor); err != nil {
return publicops.ClaimNextResult{}, err
}
return publicops.ClaimNextResult{Claimed: &types.IssueWithCounts{Issue: issue}}, nil
}
return publicops.ClaimNextResult{}, nil
}
type servedAnnotator struct{ s *remoteStore }
func (a servedAnnotator) AnnotateBlocking(_ context.Context, req publicops.BlockingRequest) (publicops.BlockingResult, error) {
a.s.served = append(a.s.served, "AnnotateBlocking")
items := make([]publicops.IssueBlocking, 0, len(req.IDs))
for _, id := range req.IDs {
items = append(items, publicops.IssueBlocking{ID: id, BlockedBy: slices.Clone(a.s.localBlockers[id])})
}
return publicops.BlockingResult{Items: items}, nil
}
type servedWalker struct{ s *remoteStore }
func (w servedWalker) WalkTree(context.Context, publicops.WalkTreeRequest) (publicops.TreeResult, error) {
w.s.served = append(w.s.served, "WalkTree")
return publicops.TreeResult{Nodes: slices.Clone(w.s.tree)}, nil
}
type servedEdgeReader struct{ s *remoteStore }
func (e servedEdgeReader) ReadEdges(_ context.Context, req publicops.EdgeReadRequest) (publicops.EdgeReadResult, error) {
if len(req.IDs) > wireEdgeAnchorCap {
return publicops.EdgeReadResult{}, fmt.Errorf("served ReadEdges: %d anchors, past the wire's %d", len(req.IDs), wireEdgeAnchorCap)
}
e.s.edgeReads = append(e.s.edgeReads, len(req.IDs))
anchors := make([]publicops.AnchorEdges, 0, len(req.IDs))
for _, id := range req.IDs {
anchors = append(anchors, publicops.AnchorEdges{ID: id, Edges: e.s.deps[id]})
}
return publicops.EdgeReadResult{Anchors: anchors}, nil
}
type servedCloser struct{ s *remoteStore }
// errServedClaimNextRefused is the served closer's answer to a next claim.
// httpclient refuses one on every server: the v0 wire's batch close has no
// member to carry it (ledger row W-CloseBatchRequest.ClaimNext).
var errServedClaimNextRefused = &storage.ErrUnsupported{Op: "BatchCloser.CloseBatch", Backend: "http"}
func (c servedCloser) CloseBatch(ctx context.Context, req publicops.CloseBatchRequest) (publicops.CloseBatchResult, error) {
if req.ClaimNext != nil {
c.s.served = append(c.s.served, "CloseBatch ClaimNext")
return publicops.CloseBatchResult{}, errServedClaimNextRefused
}
ids := make([]string, 0, len(req.Items))
outcomes := make([]publicops.CloseOutcome, 0, len(req.Items))
for _, item := range req.Items {
ids = append(ids, item.IssueID)
result, err := c.s.CloseIssueChecked(ctx, item.IssueID, req.Actor, storage.CloseIssueOptions{})
outcomes = append(outcomes, publicops.CloseOutcome{IssueID: item.IssueID, Changed: err == nil && !result.Unchanged, Err: err})
}
c.s.closes = append(c.s.closes, ids)
return publicops.CloseBatchResult{Outcomes: outcomes}, nil
}
const paymentsRef = "external:remote:payments"
func blockOnPayments(raw *remoteStore, ids ...string) {
for _, id := range ids {
raw.deps[id] = []*types.Dependency{externalDep(id, paymentsRef, types.DepBlocks)}
}
}
func closeItems(ids ...string) []publicops.BatchCloseItem {
items := make([]publicops.BatchCloseItem, 0, len(ids))
for _, id := range ids {
items = append(items, publicops.BatchCloseItem{IssueID: id})
}
return items
}
// TestRemoteRolesAnswerWithoutTheLegacyMethods pins the blocker the base
// accessors had over httpclient.Store: built over legacy methods a remote
// store refuses, they failed bd list, bd show, bd dep tree, bd ready --claim
// and bd close in a workspace where nothing had an external dependency.
func TestRemoteRolesAnswerWithoutTheLegacyMethods(t *testing.T) {
a, b := issue("be-a"), issue("be-b")
raw := newRemoteStore(a, b)
raw.tree = []*types.TreeNode{{Issue: *a}}
store := testStore(raw, &fakeStore{}, true)
ctx := t.Context()
reader, err := store.IssueReader()
if err != nil {
t.Fatalf("IssueReader: %v", err)
}
if page, err := reader.List(ctx, publicops.ListRequest{}); err != nil || len(page.Items) != 2 {
t.Fatalf("List = %d items, %v; want 2", len(page.Items), err)
}
if details, err := reader.Get(ctx, publicops.GetRequest{ID: a.ID}); err != nil || details.ID != a.ID {
t.Fatalf("Get = %v, %v; want %s", details, err, a.ID)
}
if page, err := reader.Ready(ctx, publicops.ReadyRequest{}); err != nil || len(page.Items) != 2 {
t.Fatalf("Ready = %d items, %v; want 2", len(page.Items), err)
}
claimer, err := store.IssueClaimer()
if err != nil {
t.Fatalf("IssueClaimer: %v", err)
}
if _, err := claimer.Claim(ctx, publicops.ClaimRequest{Actor: "worker", IssueID: a.ID}); err != nil {
t.Fatalf("Claim: %v", err)
}
readyClaimer, err := store.ReadyClaimer()
if err != nil {
t.Fatalf("ReadyClaimer: %v", err)
}
if next, err := readyClaimer.ClaimNext(ctx, publicops.ClaimNextRequest{Actor: "worker"}); err != nil || next.Claimed == nil || next.Claimed.ID != b.ID {
t.Fatalf("ClaimNext = %+v, %v; want %s", next.Claimed, err, b.ID)
}
annotator, err := store.BlockingAnnotator()
if err != nil {
t.Fatalf("BlockingAnnotator: %v", err)
}
if result, err := annotator.AnnotateBlocking(ctx, publicops.BlockingRequest{IDs: []string{a.ID, b.ID}}); err != nil || len(result.Items) != 2 {
t.Fatalf("AnnotateBlocking = %+v, %v; want 2 items", result.Items, err)
}
walker, err := store.TreeWalker()
if err != nil {
t.Fatalf("TreeWalker: %v", err)
}
if tree, err := walker.WalkTree(ctx, publicops.WalkTreeRequest{RootID: a.ID, MaxDepth: 50}); err != nil || len(tree.Nodes) != 1 {
t.Fatalf("WalkTree = %d nodes, %v; want the served root alone", len(tree.Nodes), err)
}
closer, err := store.BatchCloser()
if err != nil {
t.Fatalf("BatchCloser: %v", err)
}
result, err := closer.CloseBatch(ctx, publicops.CloseBatchRequest{Actor: "worker", Items: closeItems(a.ID, b.ID)})
if err != nil || len(result.Outcomes) != 2 {
t.Fatalf("CloseBatch = %+v, %v; want 2 outcomes", result.Outcomes, err)
}
if want := []string{"List", "Get be-a", "Claim be-a", "ClaimNext", "AnnotateBlocking", "WalkTree"}; !slices.Equal(raw.served, want) {
t.Fatalf("served calls = %v, want %v", raw.served, want)
}
if want := [][]string{{a.ID, b.ID}}; !slices.EqualFunc(raw.closes, want, slices.Equal[[]string]) {
t.Fatalf("served closes = %v, want %v", raw.closes, want)
}
}
// TestRemoteReaderPolicesReadyWorkOnly: an externally blocked issue leaves the
// ready listing and stays in a plain one, which claims nothing about
// readiness.
func TestRemoteReaderPolicesReadyWorkOnly(t *testing.T) {
a, b := issue("be-a"), issue("be-b")
raw := newRemoteStore(a, b)
blockOnPayments(raw, a.ID)
reader, err := testStore(raw, &fakeStore{}, true).IssueReader()
if err != nil {
t.Fatalf("IssueReader: %v", err)
}
ready, err := reader.Ready(t.Context(), publicops.ReadyRequest{})
if err != nil {
t.Fatalf("Ready: %v", err)
}
if len(ready.Items) != 1 || ready.Items[0].ID != b.ID {
t.Fatalf("Ready = %d items, want %s alone (%s is externally blocked)", len(ready.Items), b.ID, a.ID)
}
listed, err := reader.List(t.Context(), publicops.ListRequest{})
if err != nil {
t.Fatalf("List: %v", err)
}
if len(listed.Items) != 2 {
t.Fatalf("List = %d items, want both", len(listed.Items))
}
}
func TestRemoteClaimerRefusesExternallyBlockedWork(t *testing.T) {
for _, enforced := range []bool{false, true} {
t.Run(fmt.Sprintf("server enforces %v", enforced), func(t *testing.T) {
a := issue("be-a")
raw := newRemoteStore(a)
raw.enforced = enforced
blockOnPayments(raw, a.ID)
claimer, err := testStore(raw, &fakeStore{}, true).IssueClaimer()
if err != nil {
t.Fatalf("IssueClaimer: %v", err)
}
_, err = claimer.Claim(t.Context(), publicops.ClaimRequest{Actor: "worker", IssueID: a.ID})
if enforced {
// The server already applied the policy, so the claim is its
// to answer.
if err != nil || !slices.Equal(raw.served, []string{"Claim be-a"}) {
t.Fatalf("Claim = %v, served %v; want the served claim", err, raw.served)
}
return
}
if !errors.Is(err, storage.ErrCloseBlocked) || !strings.Contains(err.Error(), paymentsRef) {
t.Fatalf("Claim = %v, want ErrCloseBlocked naming %s", err, paymentsRef)
}
if len(raw.served) != 0 {
t.Fatalf("served calls = %v, want none for a refused claim", raw.served)
}
})
}
}
// TestRemoteReadyClaimerSkipsExternallyBlockedWork: the served claim-next
// cannot be told what to skip, so with anything externally blocked the claim
// walks the policy's ready page through the served Claimer instead. be-a sorts
// first, where the served claim-next would take it.
func TestRemoteReadyClaimerSkipsExternallyBlockedWork(t *testing.T) {
a, b := issue("be-a"), issue("be-b")
raw := newRemoteStore(a, b)
blockOnPayments(raw, a.ID)
claimer, err := testStore(raw, &fakeStore{}, true).ReadyClaimer()
if err != nil {
t.Fatalf("ReadyClaimer: %v", err)
}
next, err := claimer.ClaimNext(t.Context(), publicops.ClaimNextRequest{Actor: "worker"})
if err != nil {
t.Fatalf("ClaimNext: %v", err)
}
if next.Claimed == nil || next.Claimed.ID != b.ID || b.Assignee != "worker" {
t.Fatalf("ClaimNext claimed %+v, want %s for worker", next.Claimed, b.ID)
}
if want := []string{"Claim be-b"}; !slices.Equal(raw.served, want) {
t.Fatalf("served calls = %v, want %v", raw.served, want)
}
}
// TestRemoteReadyClaimerReportsLostRaces: a front that had work but lost every
// candidate is an error, not the empty answer that means there is no work.
func TestRemoteReadyClaimerReportsLostRaces(t *testing.T) {
a, b := issue("be-a"), issue("be-b")
raw := newRemoteStore(a, b)
blockOnPayments(raw, a.ID)
raw.claimErr = storage.ErrAlreadyClaimed
claimer, err := testStore(raw, &fakeStore{}, true).ReadyClaimer()
if err != nil {
t.Fatalf("ReadyClaimer: %v", err)
}
next, err := claimer.ClaimNext(t.Context(), publicops.ClaimNextRequest{Actor: "worker"})
if err == nil || !strings.Contains(err.Error(), "lost 1 races") || next.Claimed != nil {
t.Fatalf("ClaimNext = %+v, %v; want a lost-races error", next.Claimed, err)
}
}
// TestRemoteBlockingAnnotatorMergesExternalBlockers: the served annotation's
// own blockers stay, the external ones join them, and the edges behind them
// are read in chunks the wire accepts.
func TestRemoteBlockingAnnotatorMergesExternalBlockers(t *testing.T) {
issues := make([]*types.Issue, 0, 150)
ids := make([]string, 0, 150)
for i := range 150 {
issues = append(issues, issue(fmt.Sprintf("be-%03d", i)))
ids = append(ids, issues[i].ID)
}
raw := newRemoteStore(issues...)
blockOnPayments(raw, "be-000", "be-149")
raw.localBlockers = map[string][]string{"be-000": {"be-local"}}
annotator, err := testStore(raw, &fakeStore{}, true).BlockingAnnotator()
if err != nil {
t.Fatalf("BlockingAnnotator: %v", err)
}
result, err := annotator.AnnotateBlocking(t.Context(), publicops.BlockingRequest{IDs: ids})
if err != nil {
t.Fatalf("AnnotateBlocking: %v", err)
}
blockedBy := make(map[string][]string, len(result.Items))
for _, item := range result.Items {
blockedBy[item.ID] = item.BlockedBy
}
if want := []string{"be-local", paymentsRef}; !slices.Equal(blockedBy["be-000"], want) {
t.Errorf("be-000 blocked by %v, want %v", blockedBy["be-000"], want)
}
if want := []string{paymentsRef}; !slices.Equal(blockedBy["be-149"], want) {
t.Errorf("be-149 blocked by %v, want %v", blockedBy["be-149"], want)
}
if len(blockedBy["be-001"]) != 0 {
t.Errorf("be-001 blocked by %v, want nothing", blockedBy["be-001"])
}
if want := []int{100, 50}; !slices.Equal(raw.edgeReads, want) {
t.Errorf("served edge reads named %v anchors, want %v", raw.edgeReads, want)
}
}
// TestRemoteTreeWalkerAddsExternalLeaves: the served down-tree gains the leaf
// its stored edges name, a leaf the server rendered is not repeated, and any
// other walk is the served answer as it came.
func TestRemoteTreeWalkerAddsExternalLeaves(t *testing.T) {
root := &types.TreeNode{Issue: *issue("be-a")}
served := &types.TreeNode{Issue: types.Issue{ID: paymentsRef, Title: "served leaf"}, Depth: 1, ParentID: "be-a"}
for _, tc := range []struct {
name string
req publicops.WalkTreeRequest
tree []*types.TreeNode
wantIDs []string
}{
{name: "down walk", req: publicops.WalkTreeRequest{RootID: "be-a", MaxDepth: 50}, tree: []*types.TreeNode{root}, wantIDs: []string{"be-a", paymentsRef}},
{name: "server rendered the leaf", req: publicops.WalkTreeRequest{RootID: "be-a", MaxDepth: 50}, tree: []*types.TreeNode{root, served}, wantIDs: []string{"be-a", paymentsRef}},
{name: "up walk", req: publicops.WalkTreeRequest{RootID: "be-a", MaxDepth: 50, Direction: publicops.TreeUp}, tree: []*types.TreeNode{root}, wantIDs: []string{"be-a"}},
} {
t.Run(tc.name, func(t *testing.T) {
raw := newRemoteStore()
raw.tree = tc.tree
blockOnPayments(raw, "be-a")
walker, err := testStore(raw, &fakeStore{}, true).TreeWalker()
if err != nil {
t.Fatalf("TreeWalker: %v", err)
}
result, err := walker.WalkTree(t.Context(), tc.req)
if err != nil {
t.Fatalf("WalkTree: %v", err)
}
ids := make([]string, 0, len(result.Nodes))
for _, node := range result.Nodes {
ids = append(ids, node.ID)
}
if !slices.Equal(ids, tc.wantIDs) {
t.Fatalf("tree = %v, want %v", ids, tc.wantIDs)
}
if leaf := result.Nodes[len(result.Nodes)-1]; tc.name == "down walk" {
if leaf.Status != types.StatusOpen || leaf.ParentID != "be-a" || leaf.Depth != 1 || leaf.Title != "○ payments" {
t.Fatalf("leaf = %+v, want the open payments leaf under be-a", leaf)
}
}
})
}
}
// TestRemoteBatchCloserRefusesExternallyBlockedItems: a still-open externally
// blocked item is refused the way the local policy close refuses it and the
// rest close in one served request. An item already closed keeps the served
// answer, Force waives the policy, and a server that enforces the policy gets
// the whole batch.
func TestRemoteBatchCloserRefusesExternallyBlockedItems(t *testing.T) {
for _, tc := range []struct {
name string
force bool
enforced bool
wantRefused bool
}{
{name: "policy", wantRefused: true},
{name: "force", force: true},
{name: "server enforces", enforced: true},
} {
t.Run(tc.name, func(t *testing.T) {
blocked, free, done := issue("be-blocked"), issue("be-free"), issue("be-done")
done.Status = types.StatusClosed
raw := newRemoteStore(blocked, free, done)
raw.enforced = tc.enforced
blockOnPayments(raw, blocked.ID, done.ID)
closer, err := testStore(raw, &fakeStore{}, true).BatchCloser()
if err != nil {
t.Fatalf("BatchCloser: %v", err)
}
request := publicops.CloseBatchRequest{Actor: "worker", Force: tc.force, Items: closeItems(blocked.ID, free.ID, done.ID)}
result, err := closer.CloseBatch(t.Context(), request)
if err != nil {
t.Fatalf("CloseBatch: %v", err)
}
if len(result.Outcomes) != len(request.Items) {
t.Fatalf("CloseBatch answered %d outcomes for %d items", len(result.Outcomes), len(request.Items))
}
for i, outcome := range result.Outcomes {
if outcome.IssueID != request.Items[i].IssueID {
t.Fatalf("outcome %d is %s's, want %s's", i, outcome.IssueID, request.Items[i].IssueID)
}
}
wantServed := []string{blocked.ID, free.ID, done.ID}
if tc.wantRefused {
refusal := result.Outcomes[0].Err
want := "cannot close blocked issue: be-blocked is blocked by [" + paymentsRef + "]"
if !errors.Is(refusal, storage.ErrCloseBlocked) || refusal.Error() != want {
t.Fatalf("be-blocked outcome = %v, want %q", refusal, want)
}
wantServed = []string{free.ID, done.ID}
} else if result.Outcomes[0].Err != nil {
t.Fatalf("be-blocked outcome = %v, want it closed", result.Outcomes[0].Err)
}
if result.Outcomes[1].Err != nil || result.Outcomes[2].Err != nil {
t.Fatalf("be-free, be-done outcomes = %v, %v; want the served answers", result.Outcomes[1].Err, result.Outcomes[2].Err)
}
if want := [][]string{wantServed}; !slices.EqualFunc(raw.closes, want, slices.Equal[[]string]) {
t.Fatalf("served closes = %v, want %v", raw.closes, want)
}
})
}
}
// TestRemoteBatchCloserRefusesANextClaim: the served batch close cannot carry
// a next claim, so a batch with one is refused whole, the same way whether the
// workspace holds an open holder of an external ref, only a closed one, or
// none, and with Force too. The refusal leads with that wire fact, reads
// nothing, closes nothing and names the commands that do the same work. A
// server that enforces the policy itself gets the request as it came, and its
// closer answers.
func TestRemoteBatchCloserRefusesANextClaim(t *testing.T) {
for _, tc := range []struct {
name string
holder types.Status
force bool
enforced bool
wantServed []string
}{
{name: "open holder", holder: types.StatusOpen},
{name: "closed holder", holder: types.StatusClosed},
{name: "no holder"},
{name: "force", holder: types.StatusOpen, force: true},
{name: "server enforces", holder: types.StatusOpen, enforced: true, wantServed: []string{"CloseBatch ClaimNext"}},
} {
t.Run(tc.name, func(t *testing.T) {
free, holder := issue("be-free"), issue("be-holder")
raw := newRemoteStore(free, holder)
raw.enforced = tc.enforced
if tc.holder != "" {
holder.Status = tc.holder
blockOnPayments(raw, holder.ID)
}
closer, err := testStore(raw, &fakeStore{}, true).BatchCloser()
if err != nil {
t.Fatalf("BatchCloser: %v", err)
}
_, err = closer.CloseBatch(t.Context(), publicops.CloseBatchRequest{
Actor: "worker", Force: tc.force, Items: closeItems(free.ID), ClaimNext: &publicops.ReadyRequest{},
})
if tc.enforced {
if !errors.Is(err, errServedClaimNextRefused) {
t.Fatalf("CloseBatch with ClaimNext = %v, want the served closer's answer", err)
}
} else {
var unsupported *storage.ErrUnsupported
if !errors.As(err, &unsupported) || unsupported.Op != "CloseBatchRequest.ClaimNext" {
t.Fatalf("CloseBatch with ClaimNext = %v, want the CloseBatchRequest.ClaimNext refusal", err)
}
if msg := err.Error(); !strings.Contains(msg, "cannot carry a next claim") || !strings.Contains(msg, "nothing was closed") || !strings.Contains(msg, "then run `bd ready --claim`") {
t.Fatalf("refusal %q does not say why, that nothing closed, and what to run instead", msg)
}
}
if !slices.Equal(raw.served, tc.wantServed) {
t.Fatalf("served calls = %v, want %v", raw.served, tc.wantServed)
}
if len(raw.closes) != 0 || len(raw.edgeReads) != 0 || raw.scans != 0 {
t.Fatalf("closes %v, edge reads %v, scans %d; want none", raw.closes, raw.edgeReads, raw.scans)
}
})
}
}
// TestRemoteBatchCloserValidatesANextClaimFirst: a next claim that is invalid
// on every backend is ErrValidation, not the refusal.
func TestRemoteBatchCloserValidatesANextClaimFirst(t *testing.T) {
for name, claim := range map[string]*publicops.ReadyRequest{
"offset": {Offset: 1},
"bad sort": {Sort: "bogus"},
} {
t.Run(name, func(t *testing.T) {
raw := newRemoteStore(issue("be-free"))
closer, err := testStore(raw, &fakeStore{}, true).BatchCloser()
if err != nil {
t.Fatalf("BatchCloser: %v", err)
}
_, err = closer.CloseBatch(t.Context(), publicops.CloseBatchRequest{
Actor: "worker", Items: closeItems("be-free"), ClaimNext: claim,
})
var unsupported *storage.ErrUnsupported
if !errors.Is(err, storage.ErrValidation) || errors.As(err, &unsupported) {
t.Fatalf("CloseBatch with ClaimNext %+v = %v, want ErrValidation", claim, err)
}
})
}
}

View file

@ -14,8 +14,19 @@ import (
// IssueReader builds the read role on the policy store. In particular, Ready
// and List(ReadyFlag) must call this store's filtered ready methods rather than
// promoted methods on the undecorated store.
func (s *Store) IssueReader() (issueops.Reader, error) { return storereader.New(s) }
// promoted methods on the undecorated store. Over a remote store the served
// reader answers everything else (remote_roles.go).
func (s *Store) IssueReader() (issueops.Reader, error) {
policy, err := storereader.New(s)
if err != nil || !s.servesRoles() {
return policy, err
}
served, err := s.inner.IssueReader()
if err != nil {
return nil, err
}
return &remoteReader{served: served, policy: policy}, nil
}
// IssueClaimer rejects a direct claim of externally blocked work before the
// backend's atomic claim operation. ReadyClaimer below handles selection among
@ -25,6 +36,9 @@ func (s *Store) IssueClaimer() (issueops.Claimer, error) {
if err != nil {
return nil, err
}
if s.servesRoles() {
return &remoteClaimer{served: inner, policy: s}, nil
}
return &issueClaimer{inner: inner, policy: s}, nil
}
@ -47,6 +61,9 @@ func (c *issueClaimer) Claim(ctx context.Context, req issueops.ClaimRequest) (is
// ReadyClaimer keeps external blockers out of the ready-claim selection used
// by HTTP serving. The local compare-and-swap remains inside ClaimReadyIssue.
func (s *Store) ReadyClaimer() (issueops.ReadyClaimer, error) {
if s.servesRoles() {
return &remoteReadyClaimer{policy: s}, nil
}
return &readyClaimer{policy: s}, nil
}
@ -86,6 +103,9 @@ func (s *Store) BlockingAnnotator() (issueops.BlockingAnnotator, error) {
if err != nil {
return nil, err
}
if s.servesRoles() {
return &remoteBlockingAnnotator{served: inner, policy: s}, nil
}
return &blockingAnnotator{inner: inner, policy: s}, nil
}
@ -119,6 +139,9 @@ func (s *Store) TreeWalker() (issueops.TreeWalker, error) {
if err != nil {
return nil, err
}
if s.servesRoles() {
return &remoteTreeWalker{served: inner, policy: s}, nil
}
return &treeWalker{inner: inner, policy: s}, nil
}
@ -128,7 +151,7 @@ type treeWalker struct {
}
func (t *treeWalker) WalkTree(ctx context.Context, req issueops.WalkTreeRequest) (issueops.TreeResult, error) {
if (req.Direction != "" && req.Direction != issueops.TreeDown) || req.Status != "" || req.MaxRows != 0 {
if !plainDownTree(req) {
return t.inner.WalkTree(ctx, req)
}
nodes, err := t.policy.GetDependencyTree(ctx, req.RootID, req.MaxDepth, false, false)

View file

@ -1022,6 +1022,55 @@ type ExternalDependencyQueryStore interface {
GetExternalBlockingDependencyRecords(ctx context.Context) (map[string][]*types.Dependency, error)
}
// RemoteBackendStore is implemented by a DoltStorage that is a pure network
// client of a remote bd serve process (a registered backend whose
// backends.Backend.Remote is true — see internal/storage/backends). It is
// metadata about the store's transport, not a policy decision. Its one in-tree
// consumer is the external-deps decorator, which wraps a remote store like any
// other and reads this marker only to choose how it composes its roles: over a
// remote store it builds each role on the store's own served role instead of
// the legacy methods such a store refuses. Whether the decorator skips its
// client-side enforcement is ExternalDependencyPolicyProber's question.
type RemoteBackendStore interface {
IsRemoteBackendStore() bool
}
// ExternalDependencyPolicyProber is implemented by a DoltStorage that can
// answer whether the remote server it talks to already enforces bd's
// external-dependency policy itself (design 3.6, "External-dependency server
// policy": upstream's storage.ServerEnforcedPolicy / PolicyEnforcedByServer).
// The external-deps decorator consults this — never RemoteBackendStore, which
// only shapes how it composes its roles — before skipping its own client-side
// enforcement: a remote store whose
// server advertises the capability (httpapi's policy.external_dependencies)
// has already enforced the policy before answering, so a second client-side
// pass would be redundant. A store that does not implement this interface, or
// that implements it and reports false — including a remote store whose
// server is silent on the capability — gets the ordinary client-side
// enforcement. The policy is never silently skipped merely because the store
// is remote.
type ExternalDependencyPolicyProber interface {
ServerEnforcesExternalDependencyPolicy(ctx context.Context) (bool, error)
}
// ExcludeIDsUnsupportedStore is implemented by a DoltStorage whose ready-work
// reads cannot express types.WorkFilter.ExcludeIDs over their own transport.
// Today that is exactly httpclient.Store: the v0 wire's listReadyWork and
// countReadyWork operations publish no id-exclusion parameter, so the http
// bridge refuses rather than silently widening the result set whenever a
// filter carries any (design 3.6 / L12, "every field it cannot express
// refuses").
//
// The external-deps decorator consults this before deciding how to apply its
// OWN additional exclusions (issues blocked by an unsatisfied
// external:<project>:<capability> dependency): a store that answers true here
// gets those exclusions applied client-side in Go instead of folded into the
// filter handed down, so the policy still runs — design 3.6 says it is never
// silently skipped — without tripping the wire's own refusal.
type ExcludeIDsUnsupportedStore interface {
ExcludeIDsUnsupported() bool
}
// Transaction provides atomic multi-operation support within a single database transaction.
//
// The Transaction interface exposes a subset of storage methods that execute within

View file

@ -898,6 +898,10 @@ func TestBazelRetiredLanesCannotBeNarrowed(t *testing.T) {
[]string{"$(rootpath :served_pinned_shards.txt)", "$(rootpath :httpclient_test)", "-test.v", "-test.count=1", "-test.timeout=19m"},
map[string]string{"BEADS_HTTP_TEST_REQUIRED": "1", "BEADS_TEST_EMBEDDED_DOLT": "1"},
},
"//backend/http:http_served_test": {
[]string{"$(rootpath :http_test)", "-test.v", "-test.count=1", "-test.timeout=19m"},
map[string]string{"BEADS_HTTP_TEST_REQUIRED": "1", "BEADS_TEST_BD_BINARY": "$(rlocationpath //cmd/bd:bd_for_tests)", "BEADS_TEST_EMBEDDED_DOLT": "1"},
},
"//cmd/bd:bd_proxied_test": {
[]string{"$(rootpath //:.github/scripts/proxied-test-shard.sh)", "BEADS_TEST_CMD_BINARY", "$(rootpath :bd_test)"},
map[string]string{

73
test/embedder/BUILD.bazel Normal file
View file

@ -0,0 +1,73 @@
load("@rules_go//go:def.bzl", "go_test")
go_test(
name = "embedder_test",
srcs = ["embedder_test.go"],
deps = [
"//:beads",
"//backend",
"//backend/http",
"//issueops",
],
)
# --- begin repo_files (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---
filegroup(
name = "repo_doc_files",
srcs = glob(
[
"**/*.md",
"**/*.mdx",
],
allow_empty = True,
exclude = ["**/node_modules/**"],
),
visibility = ["//:__subpackages__"],
)
filegroup(
name = "repo_go_srcs",
srcs = glob(
["**/*.go"],
allow_empty = True,
exclude = [
"**/*_test.go",
"**/node_modules/**",
],
),
visibility = ["//:__subpackages__"],
)
filegroup(
name = "repo_go_test_srcs",
srcs = glob(
["**/*_test.go"],
allow_empty = True,
exclude = ["**/node_modules/**"],
),
visibility = ["//:__subpackages__"],
)
filegroup(
name = "repo_other_files",
srcs = glob(
["**"],
exclude = [
"**/*.db",
"**/*.exe",
"**/*.go",
"**/*.md",
"**/*.mdx",
"**/*.out",
"**/*.prof",
"**/*.pyc",
"**/*.test",
"**/__pycache__/**",
"**/node_modules/**",
],
),
visibility = ["//:__subpackages__"],
)
# --- end repo_files ---

View file

@ -0,0 +1,253 @@
// Package embedder_test proves the one promise S6 makes to an out-of-tree
// embedder (gc, Gas City is the motivating case): linking nothing but PUBLIC
// doors — github.com/steveyegge/beads, .../backend, and .../backend/http — is
// enough to register the http backend and open a workspace through it with a
// per-call credential, via beads.OpenBestAvailableWith.
//
// Written fresh for OSS beads S6 (no bd-enterprise source copied). This
// package imports ONLY those three import paths (plus the standard library
// and net/http/httptest for the stand-in server): no internal/... package
// appears in this file's imports, which is itself the thing under test — an
// internal import here would compile today and break silently the day
// internal/httpclient's shape changes, defeating the whole point of a public
// door.
package embedder_test
import (
"context"
"encoding/json"
"go/parser"
"go/token"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"testing"
beads "github.com/steveyegge/beads"
"github.com/steveyegge/beads/backend"
bdhttp "github.com/steveyegge/beads/backend/http"
"github.com/steveyegge/beads/issueops"
)
// TestNoInternalImports is the mechanical, enforced form of this file's doc
// comment promise above: nothing in this package may import a
// github.com/steveyegge/beads/internal/... package, directly or otherwise.
// That import would compile fine today and only break silently the day some
// internal package's shape changes — exactly what a public-door embedder
// test exists to catch before it ships. It replaces an earlier, separate
// go.mod for this directory: a second module cannot see internal/... either,
// but it is heavier machinery for the same guarantee, and nothing stops a
// future edit from deleting a go.mod quietly, whereas this test fails loudly
// in the same `go test` run as everything else.
//
// It only inspects this package's own files' import declarations
// (go/parser, ImportsOnly — no type-checking, so it stays fast and adds no
// dependency beyond the standard library) and does not recurse into what
// those imports themselves pull in; that transitive check is exactly what
// go build/go vet already do on every change.
func TestNoInternalImports(t *testing.T) {
fset := token.NewFileSet()
pkgs, err := parser.ParseDir(fset, ".", nil, parser.ImportsOnly)
if err != nil {
t.Fatalf("ParseDir: %v", err)
}
const forbidden = "github.com/steveyegge/beads/internal"
for _, pkg := range pkgs {
for filename, file := range pkg.Files {
for _, imp := range file.Imports {
path, err := strconv.Unquote(imp.Path.Value)
if err != nil {
t.Fatalf("%s: unquoting import %s: %v", filename, imp.Path.Value, err)
}
if path == forbidden || strings.HasPrefix(path, forbidden+"/") {
t.Errorf("%s imports %q: test/embedder must link only public doors, never an internal/... package", filename, path)
}
}
}
}
}
const embedderProjectID = "proj-gc-tenant"
// embedderReadyIssueID is the one issue the stand-in server's /v0/beads/ready
// page carries, so the test has something distinctive to assert on besides
// "the call did not error".
const embedderReadyIssueID = "gc-1"
// backendName is spelled literally, the way a workspace's metadata.json
// carries it, rather than read back from a re-exported constant: bdhttp
// deliberately exports no Backend name constant (see backend/http/http.go),
// so a rename of the registry key here would be caught by every CONNECTED
// workspace left unopenable, which is the fact this spells out explicitly
// instead of hiding behind a symbol that does not exist.
const backendName = "http"
// TestEmbedderOpensTheHTTPBackendThroughThePublicDoorsOnly is the linkage
// proof: Register from the public backend/http door, a workspace described
// only by the public Target/SaveTarget helpers, and
// beads.OpenBestAvailableWith from the public root package — no package under
// internal/... in the call chain this test writes.
func TestEmbedderOpensTheHTTPBackendThroughThePublicDoorsOnly(t *testing.T) {
server := newStandInServer(t)
bdhttp.Register(bdhttp.Options{HTTPClient: server.Client()})
t.Cleanup(func() { backend.Deregister(backendName) })
beadsDir := filepath.Join(t.TempDir(), ".beads")
// SaveTarget, like the package's other low-level helpers, writes into an
// already-existing directory; creating it is the caller's job (cmd/bd's
// connect command does the same MkdirAll before calling SaveTarget).
if err := os.MkdirAll(beadsDir, 0o700); err != nil {
t.Fatalf("mkdir %s: %v", beadsDir, err)
}
base, err := url.Parse(server.URL)
if err != nil {
t.Fatalf("parse stand-in server URL: %v", err)
}
target := bdhttp.Target{BaseURL: base, ExpectProjectID: embedderProjectID}
// Attach is the one public call that activates a workspace for this
// backend: it writes the per-user sidecar (what SaveTarget alone used to
// leave to this test) AND metadata.json's {"backend": "http"} selection —
// the fact that tells OpenBestAvailableWith to consult the registry at
// all — in one call, the same pair `bd connect` itself now calls through
// (cmd/bd/connect.go). Before Attach existed this test hand-wrote
// metadata.json's JSON shape directly, since there was no public door
// onto that second file's writer; asserting the on-disk shape below
// keeps this test proving the SAME canary (a rename of the registry key
// breaks every connected workspace) without reaching into
// internal/configfile for it.
if err := bdhttp.Attach(beadsDir, target); err != nil {
t.Fatalf("Attach: %v", err)
}
metadataBytes, err := os.ReadFile(filepath.Join(beadsDir, "metadata.json"))
if err != nil {
t.Fatalf("reading metadata.json after Attach: %v", err)
}
var metadata struct {
Backend string `json:"backend"`
}
if err := json.Unmarshal(metadataBytes, &metadata); err != nil {
t.Fatalf("parsing metadata.json after Attach: %v", err)
}
if metadata.Backend != backendName {
t.Fatalf("metadata.json backend = %q, want %q", metadata.Backend, backendName)
}
// A multi-tenant embedder's reason to call OpenBestAvailableWith rather
// than OpenBestAvailable: this credential is THIS caller's, not whatever
// the ambient process environment happens to hold, which belongs to no
// one tenant in a process serving many.
provider := &tenantCredential{token: "gc-tenant-token"}
store, err := beads.OpenBestAvailableWith(context.Background(), beadsDir, beads.OpenOptions{
Credential: bdhttp.ProvidedCredential{Provider: provider},
})
if err != nil {
t.Fatalf("OpenBestAvailableWith: %v", err)
}
t.Cleanup(func() { _ = store.Close() })
// The read itself goes through the role accessor, not a backend-specific
// method: IssueReader().Ready is issueops.Reader's own door, the same one
// `bd ready` calls against ANY backend. Ready is the simplest operation to
// stand a fake server in for — a single GET with no pagination contract
// and no handshake capability gate (it is a baseline v0 operation) — so
// this proves the public linkage without reimplementing the wire's
// capability negotiation in a throwaway test server.
reader, err := store.IssueReader()
if err != nil {
t.Fatalf("IssueReader: %v", err)
}
page, err := reader.Ready(context.Background(), issueops.ReadyRequest{})
if err != nil {
t.Fatalf("Ready: %v", err)
}
if len(page.Items) != 1 || page.Items[0].ID != embedderReadyIssueID {
t.Errorf("ready page = %+v, want one item %q", page.Items, embedderReadyIssueID)
}
server.wantAuthorization(t, "Bearer gc-tenant-token")
}
// tenantCredential is a bdhttp.CredentialProvider scoped to one embedded
// tenant, built from nothing but the public CredentialProvider interface.
type tenantCredential struct{ token string }
func (c *tenantCredential) Authorize(_ context.Context, req *http.Request) error {
req.Header.Set("Authorization", "Bearer "+c.token)
return nil
}
func (c *tenantCredential) Refresh(context.Context) (bool, error) { return false, nil }
// standInServer is a minimal bd serve stand-in: just enough of the v0
// handshake response for Open/GetMetadata to succeed, recording the
// Authorization header every request carried.
type standInServer struct {
*httptest.Server
mu sync.Mutex
auths []string
}
func newStandInServer(t *testing.T) *standInServer {
t.Helper()
s := &standInServer{}
s.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
s.mu.Lock()
s.auths = append(s.auths, r.Header.Get("Authorization"))
s.mu.Unlock()
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/v0/beads/context":
// Open/OpenWith dial lazily and never fetch this on their own
// (the handshake is cached per-process on the first POST-BASELINE
// dispatch — see internal/httpclient.Store.snapshot) — Ready
// below is baseline and preflights without one. This handler
// stays in place anyway: it is the one other request a workspace
// opened through this backend could legitimately make, and a
// stand-in server that 404s it would be a trap for the next
// person who extends this test to a non-baseline operation.
_ = json.NewEncoder(w).Encode(map[string]any{
"api_version": "v0",
"backend": "dolt",
"bd_version": "9.9.9",
"beads_dir": "/srv/.beads",
"capabilities": []string{"issues.get", "issues.list"},
"database": "beads",
"dolt_mode": "server",
"project_id": embedderProjectID,
"repo_root": "/srv",
"schema_version": 1,
"wire_revision": 1,
"min_client_wire_revision": 1,
})
case "/v0/beads/ready":
_ = json.NewEncoder(w).Encode(map[string]any{
"has_more": false,
"items": []map[string]any{
{"id": embedderReadyIssueID, "title": "embedder linkage proof", "priority": 2},
},
})
default:
http.NotFound(w, r)
}
}))
t.Cleanup(s.Server.Close)
return s
}
func (s *standInServer) wantAuthorization(t *testing.T, want string) {
t.Helper()
s.mu.Lock()
defer s.mu.Unlock()
if len(s.auths) == 0 {
t.Fatal("the stand-in server saw no requests at all")
}
for i, got := range s.auths {
if got != want {
t.Errorf("request %d carried Authorization %q, want %q", i, got, want)
}
}
}

View file

@ -177,6 +177,7 @@ release_cross_build(
":pure_bd_has_no_cgo_only_deps",
"//:beads",
"//backend/conformance:conformance",
"//backend/http:http",
"//backend:backend",
"//beadserrors:beadserrors",
"//cmd/bd/doctor/fix:fix",