* feat(backend/http): add the public HTTP backend door, cmd/bd registration, and bd connect internal/httpclient (S1-S5) had no production caller for the per-open backends.OpenOptions.Credential seam and no way for a workspace to actually select it. This slice (S6) closes both gaps in OSS's own idiom, without carrying over anything from bd-enterprise's gateway/box-login/storage-profile machinery: - backend/http (bdhttp) is the public, out-of-tree extension door: Register adds "http" to the backend registry and installs its transport, Open/ Handshake dial a server directly with no workspace on disk, and the registered OpenWith hook threads a per-call credential through beads.OpenBestAvailableWith for a multi-tenant embedder. - internal/httpclient/open_with.go is the backends.Backend.OpenWith hook itself: a fresh dial per open (opts.Credential/HTTPClient/UserAgent), distinct from the process-wide registered dialer Open/OpenReadOnly use. - cmd/bd/http_backend.go registers the backend for the OSS binary, so a workspace whose metadata.json names "http" opens through the ordinary OpenBestAvailable path; any other unregistered backend name still refuses. - cmd/bd/connect.go adds the minimal `bd connect <url>` command: a handshake that must succeed before anything is written, a per-user activation sidecar (.beads/http_target.json, never git-tracked), a plaintext-to- non-loopback refusal without --allow-plaintext, and a backend-switch refusal without --force. No credential is ever accepted on the command line or written to disk. - cmd/bd/doctor/gitignore.go requires the new http_target.json / http_local_metadata.json sidecars be git-ignored, the same way the proxied-server client info already is. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(storage): keep external-dependency blocking over http unless the server enforces it bd ready / bd close / claim hard-failed against the new http backend with "GetAllDependencyRecords not supported by the http backend": cmd/bd's storage chain wraps every opened store in the externaldeps policy decorator, which needs either the narrow ExternalDependencyQueryStore interface or the broad GetAllDependencyRecords fallback to resolve blocking state, and the http client implemented neither. The decorator now stays on for a remote store. It is skipped only when the server's handshake advertises policy.external_dependencies (the server enforces the policy before answering). Without that capability the client resolves blocking itself from the listing and chunked dependency reads, so an issue with an unresolvable external: dependency stays out of bd ready exactly as it does on a local backend; it is never silently skipped (DESIGN 3.6). Server-side enforcement is slice S10. storage.RemoteBackendStore marks a DoltStorage that is a pure network client of a remote bd serve; the http backend's Store satisfies it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(cmd/bd): suppress the CreatedBy stamp on create against a remote backend bd create failed against the http backend with "Issue.CreatedBy: ... is not expressible on the v0 wire": the http client's create role deliberately refuses any POPULATED CreatedBy/CreatedAt (the creation stamp belongs to the server's own journal entry, not a value the caller supplies), but cmd/bd/create.go unconditionally stamped getActorWithGit() onto every created issue regardless of backend. The refusal only fires on a non-zero field, so suppressing the stamp for a registered remote backend is wire- compatible by construction, not a workaround. Add isRemoteBackendWorkspace()/creationActorStamp() to backend_support.go, reusing the existing resolveCommandBeadsDir(dbPath) + backendNameForErrorFraming(beadsDir) + backends.IsRemote(name) pattern already used elsewhere in that file, and use creationActorStamp() at both create.go call sites (the dry-run preview and the real create). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * test(backend/http): add an embedder linkage test over public packages only Prove the one promise S6 makes to an out-of-tree embedder (gc is the motivating case): importing nothing but github.com/steveyegge/beads, .../backend, and .../backend/http is enough to register the http backend, open a workspace through it with a per-call credential (beads.OpenBestAvailableWith), and read through the role-accessor surface (IssueReader().Ready) against a stand-in server — with no internal/... import in the test file, which is itself the thing under test. Written fresh for OSS beads S6; no bd-enterprise source copied. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs: document the HTTP backend, bd connect, and the public backend/http door Add a README section with bd connect usage, the credential ladder, and the plaintext/force refusals, plus a CHANGELOG entry for the new cmd/bd registration, bd connect command, and the public backend/http package. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(httpclient): address bee-ghosttrack review on #7288 (connect --clear restore, persisted --allow-plaintext) Closes the CHANGES_REQUESTED items on the HTTP backend door PR: - Must-fix: `bd connect --force` over a workspace with its own local (dolt) identity now has writes succeed instead of tripping "workspace identity mismatch detected" — validateWorkspaceIdentity (cmd/bd/main.go) and the sidecar (internal/httpclient/metadata.go) now compare against the http backend's own pinned project id. New e2e regression test TestE2E_ForceConnectWritesSucceedDespiteDifferingLocalProjectID (backend/http/e2e_cli_test.go), confirmed to fail without the fix. - Should-fix 1: `bd connect --clear` now restores metadata.json's backend selection to whatever it was before this workspace connected, instead of leaving it pinned to "http" with no sidecar to dial. Target.PreviousBackend (internal/httpclient/target.go) is recorded by Attach (backend/http/attach.go) and round-tripped through the sidecar; connect.go reads it back on --clear. New tests in backend/http/attach_test.go and internal/httpclient/target_test.go. - Should-fix 2: `bd connect --allow-plaintext` is now remembered in the sidecar (Target.AllowInsecureCredential), scoped to the target it was granted for, so every later ordinary command's dial honors the grant without also needing BEADS_HTTP_ALLOW_INSECURE=1 set (internal/httpclient/dial.go). Fixed BearerProvider.warnInsecure's wording (internal/httpclient/credential.go), which used to assert a token was being sent immediately above a refusal saying the opposite. New test TestDialWithWiresTargetAllowInsecureCredentialThroughToTheGuard. - Should-fix 3: README no longer claims HTTP mode works "exactly as with a local database" — it lists what round-trips and points at the divergence ledger (engdocs/design/http-divergence-ledger.md) for what differs. - Verified, no code change needed: Attach already handles an existing metadata.json project_id coherently (only cfg.Backend is touched); bd show's isNotFoundErr already matches a http 404 (issueops.ErrNotFound is storage.ErrNotFound, the same sentinel a local miss uses), so it reads as "not found" rather than an unsupported-operation error. - Doc fix: guardInsecureCredential's doc comment and refusal message were stale against should-fix 2's own change — they named only DialOptions.AllowInsecureCredential and the env var as ways to get past the guard, not the newly-persisted target.AllowInsecureCredential or `bd connect --allow-plaintext` itself. Gates run clean on this branch: go build/vet, go test over internal/httpclient, backend/http (including the full e2e suite under BEADS_TEST_EMBEDDED_DOLT=1) and cmd/bd, make api-check, make ci-pr-lint, make bazel-sync-check, bazel run //:gazelle -- -mode=diff, and internal/storage's TestEveryLegWiresEveryRoleContract. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * test(backend/http): give the e2e server the BatchGetter role #7248 added to the required set Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: exact http ready totals, connect capability row, doc corrections Ready totals over http (review major). externaldeps drops externally blocked rows client-side for a store that cannot express ExcludeIDs (today httpclient.Store). dropBlockedIssuesWithCounts compacted the fetched rows in place (`kept := rows[:0]`), and readyWorkWithCountsAndTotal then re-scanned `rows` to count the blocked rows it had dropped. By then the compaction had overwritten them, so whenever a blocked row came back ahead of a kept one the re-scan undercounted the drops: `bd ready --json`'s total and text-mode `bd ready`'s count (CountReadyWork) overstated ready work against any server that does not advertise policy.external_dependencies. Both drop helpers now copy into a fresh slice, and dropBlockedIssuesWithCounts reports how many rows it dropped, so nothing re-scans `rows`. fakeStore gains an excludeIDsUnsupported switch that, like httpclient.Store, refuses any ExcludeIDs. The new TestReadyTotalsWhenStoreCannotExpressExcludeIDs puts the blocked issue first and checks the page and total for an unlimited filter, a Limit whose bumped window holds the whole set, and one the set outgrows, then CountReadyWork. TestServedReadyExcludesUnsatisfiedExternalDependencyWithoutServerCapability now also checks the total and count against a real in-process bd serve. Against the previous decorator the unit test's unlimited case reports total and count 3 beside a 2-row page, and the served test 2 beside a 1-row page. Review minors, all wording: - `bd connect --clear` pointed at `bd config set backend dolt`, which writes the database's config table rather than metadata.json and, in the state that prints the advice (metadata.json says http, sidecar gone), fails the pre-open identity check. It now says to reconnect, or to set "backend" in metadata.json. - CHANGELOG described the superseded skip-at-wrap design; it now describes the always-wrap decorator with its capability probe and client-side drop. - RemoteBackendStore's docs no longer name a consumer; nothing in-tree checks it. - creationActorStamp's comment pointed at backend/http/lifecycle.go; the refusal lives in internal/httpclient/lifecycle.go. Capability row. TestProxyCapabilityRegistryCoversCommandTree fails at 60b7128204: the command tree walk finds `connect`, which this PR adds, with no row in cmd/bd/capability_registry.go ("1 command(s) have no proxied-server capability row: connect"). The test predates the PR (#6582 is an ancestor of its base) and passes at the base dabf20ef07. connect opts out of store init (skipStoreAnnotation) and never reaches the store factory, so it belongs in proxyPermittedPaths beside init and setup. The row is runtime-neutral: validateProxyRegistryBeforeProvider returns nil both for a permitted row and for a path with no row, so this only records the policy decision the coverage test asks for. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(httpclient): regenerate the divergence ledger doc after the S5 rebase The rebase onto feat/http-client-batch-apply's rebased head (S3/S4 merged to main) left the ledger doc's summary line at a conflict-resolution placeholder count. Regenerate it from internal/httpclient/encode.Ledger() so TestDivergenceLedgerDocMatchesLedger passes against the merged ledger.go. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(backend/http): host-scope S6's test BEADS_HTTP_TOKEN values Rebasing onto feat/http-client-batch-apply's rebased head pulled in main's #7247 review-fix round, which made BEADS_HTTP_TOKEN/BEADS_HTTP_TOKEN_COMMAND host-scoped (host[:port]=value, scopedEnvValue/parseScopedEnv in internal/httpclient/credential.go) after S6's own tests were written against the unscoped form. Five S6 tests across backend/http/open_test.go, backend/http/register_test.go and internal/httpclient/insecure_credential_guard_test.go set a bare token and now hit the new ladder's "an unscoped token would be used for whatever server..." refusal. Scope each to the test server's actual host (127.0.0.1 for the httptest.Server-backed recording servers, nonLoopbackTestHost for the insecure-credential-guard integration tests) so they exercise the same ladder rung as before. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * build(bazel): resync go_srcs/repo_files for the new backend/http and test/embedder packages make bazel-sync-check failed after the rebase: tools/bazel/go_srcs.py's managed repo_files filegroups for backend/http and test/embedder (new in this slice) hadn't been generated, and the root BUILD.bazel / tools/bazel's release_cross_build target list hadn't picked up the new package either. Regenerate with `python3 tools/bazel/go_srcs.py`; `bazel run //:gazelle -- -mode=diff` reports no further drift. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(cmd/bd): route swarm's create paths through creationActorStamp after main's currentActor change Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: police http's served roles, refuse close --claim-next, exact totals External-dependency policy over http (review blocker). cmd/bd wraps every store, http included, in the externaldeps decorator, and its role accessors rebuilt each role over legacy methods httpclient.Store refuses (SearchIssuesWithCounts, GetDependencyTree, ClaimReadyIssue, IsBlocked, GetDependencyRecordsForIssues, BatchCloserWithPolicy). Over http `bd list`, `bd dep tree`, `bd ready --claim` and a policed `bd close` therefore exited 1, even in a workspace with no external dependency. Over a store that reports storage.RemoteBackendStore, each accessor now starts from the inner store's served role and layers the policy around it (remote_roles.go): - IssueReader serves List and Get from the served reader; Ready and a ready-flagged List go through the policy reader. - IssueClaimer refuses an externally blocked claim before the served one. - ReadyClaimer uses the served claim-next while no issue, closed ones included, holds an unsatisfied external ref. Otherwise it claims down the policy's filtered ready page through the served Claimer, because claimNext cannot exclude ids. - BlockingAnnotator and TreeWalker merge external blockers into, or add external leaves to, the served answer. They read only the named issues' edges through the served EdgeReader, 100 anchors per call. - BatchCloser refuses externally blocked open items and sends the rest to the served closer as one request. --force keeps the base path, and a next claim is refused whole (below). RemoteBackendStore now only chooses this composition. Whether the policy runs is still ServerEnforcesExternalDependencyPolicy's answer (design 3.6), and the storage and httpclient docs say so. remote_roles_test.go drives each role over a store that refuses the legacy methods. TestServedRolesKeepTheExternalDependencyPolicy runs every role over a real http server. TestE2E_BDOverHTTP adds `bd list` (JSON, text and --deps), `bd dep tree`, `bd ready`, `bd list --ready`, `bd ready --claim`, a refused `bd close --claim-next`, and a refused then forced close of an externally blocked issue. At the fixup before the composition, the role tests then written (bar one pin of the next-claim refusal, since replaced) and the e2e's list, dep tree, claim and close steps of that round were red. Close with a next claim over http (review major). batchCloseIssues has no member for a next claim, so httpclient refuses any ClaimNext before the dial (ledger W-CloseBatchRequest.ClaimNext). Over http `bd close --claim-next` therefore always exited 1 and closed nothing. With an external ref anywhere in the workspace (a closed holder or --force included) the decorator's base path failed first, with an opaque `BatchCloserWithPolicy` unsupported. remoteBatchCloser now refuses a next claim whole, before it reads or closes anything, and the error says nothing closed: close without --claim-next, then run `bd ready --claim`. A request that is invalid on every backend is still ErrValidation first, and a server that enforces the policy itself gets the request unchanged. Composing close-then-claim was not taken. The role closes and claims in one transaction and answers with an error or with outcomes, never both, so a claim that failed after the closes committed could not report them; httpclient refuses the same request for that reason. A CLI-level close then claim that can report a partial result is the follow-up. The review also suggested leaving closed holders out of the blocking state. They stay in, as on every backend: a closed issue is never a claim candidate, so they change no claim, and leaving them out would cost a status read per holder because the wire has no batch read. TestRemoteBatchCloserRefusesANextClaim covers an open holder, a closed one, none, --force, and a server that enforces the policy; with the previous remote_roles.go the first four are red. TestRemoteBatchCloserValidatesANextClaimFirst keeps ErrValidation ahead of the refusal. Dropping the filter check, or loading the blocking state before refusing, is each caught. TestE2E_BDOverHTTP runs `bd close --claim-next` while the holder is open and again after it is closed; with the previous remote_roles.go both steps are red and print `operation "BatchCloserWithPolicy" not supported`. Ledger F-close no longer says --claim-next is served, and no longer names a renderEscapedHTTPRefusal that does not exist. L14 says the decorator's claim leg applies while any issue, closed or not, holds an unsatisfied ref. The README and CHANGELOG mention the refusal, and the decorator's IsBlocked, IsBlockedBatch and CloseIssueChecked say they refuse over a store that serves its roles. Ready totals with external blockers (review majors, both rounds). CountReadyWork asks for Limit=0, and readyWorkWithCountsAndTotal took any Limit<=0 to mean "the window held the whole ready set". Over http a zero limit never reaches the wire: the server answers with its default page (workapi.DefaultReadyLimit, 100 rows) while the bridge reports the full total, so past 100 ready issues text-mode `bd ready` kept counting externally blocked issues beyond the first page. Exactness is now judged from the rows that came back (total <= len(rows)). A window that holds only part of the set no longer subtracts every externally blocked id in the workspace either: that set includes closed issues and ones the filter excludes, none of them in the total, so the count came out low. It refetches the filtered ready set, ids only, and subtracts the blocked rows in it; the old bound is left as the fallback for a refetch that comes back truncated too. fakeStore gains the bridge's shape: a defaultPage for a zero Limit, with GetReadyWorkWithCountsAndTotal still totalling the whole set. TestReadyTotalsWhenAnUnlimitedFetchComesBackAsOnePage (a blocked issue past the first page) and TestReadyTotalsIgnoreExternallyBlockedIssuesOutsideTheReadySet (a blocked issue outside the ready set) have served counterparts, TestServedReadyCountExcludesAnExternallyBlockedIssuePastTheDefaultPage and TestServedReadyTotalIgnoresAnExternallyBlockedIssueOutsideTheFilter (an assignee filter). The first pair is red on the Limit<=0 condition, the second on the subtract-every-blocked-id branch. Docs: ledger L8 now says which decorator paths take the chunked GetAllDependencyRecords scan and which read only named edges, and L14 records the decorator's composed claim leg; http-divergence-ledger.md is regenerated. The README names the verified commands, and the README and CHANGELOG say how cross-repo dependencies behave over http. Minors from the reviews: - Drop creationActorStamp/isRemoteBackendWorkspace and restore the base stamps at all 11 sites. The arm guarded a refusal the http create role does not make: since #7247 (actorStampedCreateMember) it accepts a CreatedBy naming the actor. Because the arm keyed on the CWD workspace, a create routed from an http workspace into a local one (--repo) stored an empty created_by. TestE2E_BDOverHTTP checks created_by for a direct and a routed create; the routed check is red with the arm restored. The CHANGELOG sentence describing the arm goes with it. - bdhttp.Attach decides a reconnect from the raw metadata.json backend. GetBackend() reads an unregistered "http" as the dolt default, so in an embedder that never called Register a reconnect recorded dolt as the backend --clear restores. The reconnect test now seeds postgres (TestAttachOnAReconnectKeepsTheEarlierPreviousBackend) and is red with the old check. - README, CHANGELOG and `bd connect --help` document the host-scoped BEADS_HTTP_TOKEN=host[:port]=<token> form (and _COMMAND), and say why a bare value is refused. - bdhttp.Attach also gitignores http_local_metadata.json, the per-user file the store writes on first use, so an embedder activating through Attach alone gets the coverage `bd connect` already had. TestAttachGitignoresBothPerUserFiles pins it. - ServerSnapshot.WireRevision's doc names the handshake's actual check instead of a comparison `bd connect` never makes. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(backend/http): give the e2e server the LeaseReclaimer role #7412 added to the required set main's #7412 made LeaseReclaimer part of httpapi's required role set and updated main's own binders, but backend/http's e2eServeConfig predates it, so every TestE2E_* that binds the in-process server failed with "httpapi: no database source". Same shape as e63f7da36a for BatchGetter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(externaldeps): lead the --claim-next refusal with the wire fact; pin the L14 leg and connect --clear Review i4 minors/nits on the stranded fixup, carried as a separate commit: - remoteBatchCloser's --claim-next refusal now says the served batch close cannot carry a next claim (the real cause), rather than blaming externally blocked work a workspace may not have. The test asserts the reason, that nothing closed, and the remedy. - GetDependencyTree's doc says a served store refuses its reads and that bd dep tree goes through remoteTreeWalker. - README no longer gives `bd ready --claim` local semantics everywhere; it names when ledger row L14's two-step claim applies. - TestE2E_BDOverHTTP claims once more after the external-ref holder is closed, pinning L14's composed leg (a mutant whose composed leg claims nothing turns only that step red). - New TestE2E_ConnectClearRestoresThePreviousBackend: --clear restores the prior dolt backend with its identity intact, removes the sidecar, and a second --clear changes nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(backend/http): run the served e2e under bazel //backend/http:http_test declared no bd binary, so under bazel the e2e files (bazeltest.PrebuiltBD) failed on BEADS_TEST_BD_BINARY with BEADS_HTTP_TEST_REQUIRED=1 and only "passed" by skipping otherwise. It now takes //cmd/bd:bd_for_tests (visibility added), size large / timeout long: the served cases run 4-5 minutes on the farm under BEADS_TEST_EMBEDDED_DOLT=1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(backend/http): run the e2e tier in the embedded lane as http_served_test The previous commit gives //backend/http:http_test the bd binary, but no lane sets BEADS_TEST_EMBEDDED_DOLT for it, so its TestE2E_* cases still skipped in CI. http_served_test runs the same test binary through //tools/bazel:go_test_variant.sh with BEADS_TEST_EMBEDDED_DOLT=1 and BEADS_HTTP_TEST_REQUIRED=1 (a skip fails it). It is tagged embedded, so the --config=embedded lane runs it and prcore/ci keep their current shape, the same split as //internal/httpclient:httpclient_served_test. Under plain --config=embedded all 8 TestE2E_* cases run, none skipped. TestBazelRetiredLanesCannotBeNarrowed pins every embedded-lane target's args and env; the new target is added to that pin list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(externaldeps): cap the ready rows kept, not the bumped http window An http store cannot carry ExcludeIDs on listReadyWork, so readyExclusion fetches the unfiltered ready set, widened by len(blocked) under a limit, and drops the externally blocked rows client-side. The http ready bridge enforced MaxRows on the rows it fetched rather than the rows kept. So `bd ready --limit N --max-rows N` exited 2 whenever more than N issues were ready and any workspace issue held an unsatisfied external blocker (a closed one counts), even with no blocked row in the window, and a blocked row in the page counted against the cap although the caller never sees it. Locally neither can happen: issueops enforces the cap on the filtered page it delivers, after the Limit trim (finishReadyWorkWithCounts). The client-side path now takes MaxRows off the inner query and enforces the caller's cap on the rows it keeps, after the trim, in all three listings. The inner query keeps the cap whenever nothing is dropped client-side. MaxRows never rode the wire, so the fetch is the size it was; refetchedReadyTotal already clears the cap on its own over-fetch. TestServedReadyMaxRowsCountsTheRowsKept runs the decorator over a served http store holding three free issues and one externally blocked one: a cap equal to the limit, a cap under a wider limit, an unlimited listing, and a kept page wider than the cap, which must still return ErrTooManyRows with the caller's cap and source. Each case runs through GetReadyWork, GetReadyWorkWithCounts and GetReadyWorkWithCountsAndTotal. Leaving the cap on the inner query fails the first three cases in all three methods, dropping the re-applied cap fails the fourth, and enforcing it before the Limit trim fails the first. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
226 lines
8.7 KiB
Go
226 lines
8.7 KiB
Go
// Contributed by gascity from bd-enterprise (internal/enterprise/httpstore/dial_test.go@49d1df2f6)
|
|
// to OSS beads under the MIT license.
|
|
package httpclient
|
|
|
|
import (
|
|
"context"
|
|
"net/http"
|
|
"net/http/httptrace"
|
|
"sync"
|
|
"sync/atomic"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/steveyegge/beads/internal/httpapi/apigen"
|
|
"github.com/steveyegge/beads/internal/httpclient/wire"
|
|
)
|
|
|
|
// TestBaselineTransportRaisesMaxIdleConnsPerHost is finding 9: the ceiling
|
|
// TransportFor already applies to a CA-scoped target must ALSO apply to an
|
|
// ordinary target with no CA configured, since the RTT cost of an
|
|
// under-pooled host is the same either way.
|
|
func TestBaselineTransportRaisesMaxIdleConnsPerHost(t *testing.T) {
|
|
rt := baselineTransport()
|
|
transport, ok := rt.(*http.Transport)
|
|
if !ok {
|
|
t.Fatalf("baselineTransport returned %T, want *http.Transport", rt)
|
|
}
|
|
if transport.MaxIdleConnsPerHost != maxIdleConnsPerHost {
|
|
t.Errorf("MaxIdleConnsPerHost = %d, want %d", transport.MaxIdleConnsPerHost, maxIdleConnsPerHost)
|
|
}
|
|
}
|
|
|
|
// TestBaselineTransportIsSharedAcrossCalls confirms baselineTransport is built
|
|
// once and reused (sync.Once), not cloned per dial: a long-lived process must
|
|
// not leak a fresh transport, and its connection pool, on every dial of a
|
|
// target that never configured a CA.
|
|
func TestBaselineTransportIsSharedAcrossCalls(t *testing.T) {
|
|
first := baselineTransport()
|
|
second := baselineTransport()
|
|
if first != second {
|
|
t.Error("baselineTransport returned a different instance on a second call; it should be built once and reused")
|
|
}
|
|
}
|
|
|
|
// TestDialTransportUsesBaselineWhenNoCAConfigured is finding 9's unit test on
|
|
// the exact seam DialWith uses when the caller leaves HTTPClient nil: an
|
|
// unconfigured resolvedCA must produce the shared baselineTransport, not a
|
|
// freshly cloned one, and it must carry the raised ceiling.
|
|
func TestDialTransportUsesBaselineWhenNoCAConfigured(t *testing.T) {
|
|
rt, err := dialTransport(resolvedCA{})
|
|
if err != nil {
|
|
t.Fatalf("dialTransport: %v", err)
|
|
}
|
|
if rt != baselineTransport() {
|
|
t.Error("dialTransport did not return the shared baselineTransport for an unconfigured CA")
|
|
}
|
|
}
|
|
|
|
// TestDialWithNoCARaisesMaxIdleConnsPerHost is finding 9's end-to-end case: a
|
|
// target with no CA configured, dialed through DialWith with HTTPClient left
|
|
// nil, still reaches a real server through the raised-ceiling baseline
|
|
// transport.
|
|
func TestDialWithNoCARaisesMaxIdleConnsPerHost(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
server := &contextServer{body: v0Context("proj-baseline")}
|
|
srv := server.start(t)
|
|
target := Target{BaseURL: mustParseURL(t, srv.URL)}
|
|
|
|
snap, err := Handshake(context.Background(), target, DialOptions{})
|
|
if err != nil {
|
|
t.Fatalf("Handshake: %v", err)
|
|
}
|
|
if snap.ProjectId != "proj-baseline" {
|
|
t.Errorf("project_id = %q, want proj-baseline", snap.ProjectId)
|
|
}
|
|
}
|
|
|
|
// TestDialWithSuppliedClientNilTransportNoCARaisesMaxIdleConnsPerHost covers
|
|
// the OTHER shape DialWith's own doc comment promises the raised ceiling for
|
|
// ("Every dial also gets the raised MaxIdleConnsPerHost ceiling... whether or
|
|
// not a CA is configured"): a caller that supplies its own *http.Client with
|
|
// a nil Transport, dialed against a target with no CA configured. The
|
|
// opts.HTTPClient == nil branch above (TestDialWithNoCARaisesMaxIdleConnsPerHost)
|
|
// does not exercise this branch at all — a caller-supplied client with a nil
|
|
// Transport used to fall through to http.DefaultTransport's own unraised
|
|
// MaxIdleConnsPerHost of 2 instead, silently exempting this one shape from
|
|
// the doc's "every dial" promise, and forcing a fresh connection (a fresh TLS
|
|
// handshake, over a real network) on most of a concurrent burst even once the
|
|
// pool is warm.
|
|
//
|
|
// This asserts it end to end, through wire.Client.Do rather than by reaching
|
|
// into the transport this package's *Conn hands wire.New (wire.Client keeps
|
|
// it unexported, and rightly so): a burst of concurrent requests through the
|
|
// resulting Conn, traced with httptrace the way
|
|
// TestTransportForRaisesMaxIdleConnsPerHostForBursts traces a raw
|
|
// *http.Transport, must settle into reusing every connection once the pool
|
|
// has gone idle — which the default ceiling of 2 cannot do at this burst
|
|
// size.
|
|
func TestDialWithSuppliedClientNilTransportNoCARaisesMaxIdleConnsPerHost(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
server := &contextServer{body: v0Context("proj-supplied-client")}
|
|
srv := server.start(t)
|
|
target := Target{BaseURL: mustParseURL(t, srv.URL)}
|
|
|
|
supplied := &http.Client{}
|
|
conn, err := DialWith(target, nil, DialOptions{HTTPClient: supplied})
|
|
if err != nil {
|
|
t.Fatalf("DialWith: %v", err)
|
|
}
|
|
if supplied.Transport != nil {
|
|
t.Error("DialWith mutated the caller's own *http.Client in place; it must inject onto a copy")
|
|
}
|
|
|
|
const burst = 8
|
|
req := wire.Request{Op: "test.burst", Method: http.MethodGet, Path: "/v0/test-burst"}
|
|
|
|
runBurst := func() int64 {
|
|
var wg sync.WaitGroup
|
|
var newConns int64
|
|
for i := 0; i < burst; i++ {
|
|
wg.Add(1)
|
|
go func() {
|
|
defer wg.Done()
|
|
reused := make(chan bool, 1)
|
|
trace := &httptrace.ClientTrace{
|
|
GotConn: func(info httptrace.GotConnInfo) { reused <- info.Reused },
|
|
}
|
|
ctx := httptrace.WithClientTrace(context.Background(), trace)
|
|
if err := conn.Do(ctx, req, &apigen.ContextResponse{}); err != nil {
|
|
t.Errorf("Do: %v", err)
|
|
return
|
|
}
|
|
if !<-reused {
|
|
atomic.AddInt64(&newConns, 1)
|
|
}
|
|
}()
|
|
}
|
|
wg.Wait()
|
|
return newConns
|
|
}
|
|
|
|
if warm := runBurst(); warm == 0 {
|
|
t.Fatal("first burst reused connections that could not have existed yet")
|
|
}
|
|
|
|
var steady int64
|
|
for attempt := 0; attempt < 5; attempt++ {
|
|
time.Sleep(50 * time.Millisecond)
|
|
steady = runBurst()
|
|
if steady == 0 {
|
|
break
|
|
}
|
|
}
|
|
if steady != 0 {
|
|
t.Errorf("repeat burst dialed %d/%d fresh connections after warm-up; the supplied-client-nil-Transport branch is not getting the raised MaxIdleConnsPerHost ceiling", steady, burst)
|
|
}
|
|
}
|
|
|
|
// --- finding 2: DialOptionsForFile ---
|
|
|
|
// TestDialOptionsForFileBypassesEnvEntirely is finding 2's core unit: a
|
|
// DialOptions built from an explicit file must verify against EXACTLY that
|
|
// file, never BEADS_HTTP_CA_FILE, even when the env is set to a DIFFERENT,
|
|
// otherwise-valid CA for the same host. This is what lets `bd connect
|
|
// --ca-file X` check X for itself before writing it, regardless of what the
|
|
// env currently resolves to.
|
|
func TestDialOptionsForFileBypassesEnvEntirely(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
right := newTestCA(t)
|
|
wrong := newTestCA(t)
|
|
h := &caContextHandler{body: v0Context("proj-connect-check")}
|
|
u := wrong.startServer(t, h.handler())
|
|
|
|
// The env is set to a CA that would NOT verify this server; if
|
|
// DialOptionsForFile consulted it, the handshake below would fail for the
|
|
// wrong reason, or a matching-host env could mask the flag's own file
|
|
// entirely.
|
|
t.Setenv(CAFileEnv, u.Host+"="+right.writePEM(t))
|
|
|
|
opts, err := DialOptionsForFile(wrong.writePEM(t), DialOptions{})
|
|
if err != nil {
|
|
t.Fatalf("DialOptionsForFile: %v", err)
|
|
}
|
|
// u names the server as "localhost" (so SNI is sent) even though
|
|
// startServer actually bound 127.0.0.1; without pinning the dial to
|
|
// tcp4, the default dual-stack dialer can race an unrelated ::1 listener
|
|
// and flake. See forceIPv4Loopback's doc for the full mechanism.
|
|
forceIPv4Loopback(t, opts.HTTPClient.Transport)
|
|
// Target.CAFile is deliberately left empty, mirroring cmd/bd's connect
|
|
// verification call.
|
|
target := Target{BaseURL: u}
|
|
snap, err := Handshake(context.Background(), target, opts)
|
|
if err != nil {
|
|
t.Fatalf("Handshake verifying exactly the flag's CA file: %v", err)
|
|
}
|
|
if snap.ProjectId != "proj-connect-check" {
|
|
t.Errorf("project_id = %q, want proj-connect-check", snap.ProjectId)
|
|
}
|
|
}
|
|
|
|
// TestDialOptionsForFileRefusesTheWrongCAEvenWithAGoodEnv is finding 2's exact
|
|
// regression scenario: "With the env var set to the good CA and --ca-file set
|
|
// to a wrong CA, connect succeeded" (the bug). DialOptionsForFile must make
|
|
// the handshake fail here, since --ca-file's own file is the wrong one.
|
|
func TestDialOptionsForFileRefusesTheWrongCAEvenWithAGoodEnv(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
right := newTestCA(t)
|
|
wrong := newTestCA(t)
|
|
h := &caContextHandler{body: v0Context("proj-should-not-verify")}
|
|
u := right.startServer(t, h.handler())
|
|
|
|
t.Setenv(CAFileEnv, u.Host+"="+right.writePEM(t))
|
|
|
|
opts, err := DialOptionsForFile(wrong.writePEM(t), DialOptions{})
|
|
if err != nil {
|
|
t.Fatalf("DialOptionsForFile: %v", err)
|
|
}
|
|
target := Target{BaseURL: u}
|
|
_, err = Handshake(context.Background(), target, opts)
|
|
if err == nil {
|
|
t.Fatal("Handshake succeeded with --ca-file's own wrong CA, despite a good env CA for the same host; DialOptionsForFile must check exactly the flag's file")
|
|
}
|
|
if h.sawRequest() {
|
|
t.Error("server saw a request; TLS verification against the wrong CA should have failed first")
|
|
}
|
|
}
|