* feat(httpclient): lift the HTTP client store and roles from bd-enterprise (MIT contribution)
Contributed by gascity from bd-enterprise (github.com/gascity/bd-enterprise,
internal/enterprise/httpstore @ 49d1df2f6) to OSS beads under the MIT
license, with the owner's explicit approval. Every lifted file carries a
provenance header naming its source path.
Lifted into internal/httpclient: the Store (DoltStorage over the wire), the
dial/register glue, the read roles (reader, querier, counters, edges,
relations, graph, tree, cycles, stats, config, memories, journal), the write
roles (lifecycle, claimer, releaser, deleter, dependency editor, metadata
CAS, commenter, batch closer/creator/applier, ready claimer, sweeper), and
their tests. The store transport seam (WireClient/WireDialer) and the
store-dependent tests deferred from the S2 lift are restored here.
Not lifted: the hosted gateway (httpgw/gwauth), box-login, storage profiles
and gasworks. dial.go only mentions the gateway in comments.
This commit is the mechanical lift and does NOT build yet. It references
enterprise-only APIs absent from OSS: issueops.ReadyLister,
journalops.Watcher, storage.ServerEnforcedPolicy, the
NonCommitGraphBackend/RemoteWorkspaceBackend markers, DoltStorage
differences, and BatchCloseRequest.claim_next. The following commits
reconcile it with OSS (S3/S4/S5).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(httpclient): reconcile the bd-enterprise lift's compile-time API surface against OSS
The mechanical lift (02a9dad17) carried several bd-enterprise-only shapes
that do not exist in OSS beads:
- ReadyLister, journalops.Watcher/StreamWire, and ServerEnforcedPolicy have
no OSS wire counterpart; drop the role, its served/journal-watch tests,
and the policy enforcement path entirely rather than leave dead code that
cannot compile against OSS's apigen types.
- claim_hook_gate_test.go exercised a hook surface OSS does not publish.
- recordingWire/fakeWire no longer embed a StreamWire double now that no
read role can reach a watch stream; testTarget's two conflicting
package-scope declarations (store_test.go and helpers_test.go) are split
apart as pinnedProjectTarget vs. the generic helper.
- unsupported_gen.go is regenerated from scratch against OSS's actual
storage.DoltStorage via a new, from-scratch reflection-based generator
(internal/storage/unsupportedgen) rather than hand-copied from
bd-enterprise's generated output; completeness_test.go's
legitimatelyUnsupported map grows the handful of OSS-only or
previously-missed methods (AddRemoteWithRef, RenameLabel, etc.) this
regeneration surfaces.
- write_ledger_test.go and the divergence ledger doc gain the
force_notes_overwrite and claim pending rows OSS's apigen.UpdateRequest
and SweepRequest actually carry, and drop the Remaining/LiveDependent
assertions SweepResult/SweepSkips have no wire member to ever populate.
go build ./... and go vet ./... are clean after this change. No behavior
beyond the removed surfaces changes here; read-role and write-role
reconciliation land in follow-up commits.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): stop sending listIssues a reverse/sort vocabulary OSS does not publish
The bd-enterprise lift's sortedPage pushdown assumed GET /v0/beads/issues
(listIssues) shared the same sort vocabulary and a reverse parameter with
GET /v0/beads/issues:query (handleQueryIssues) -- a different operation.
OSS's openapi.v0.yaml publishes listIssues a CLOSED two-value `sort` enum
(created, priority; deliberately smaller than the nine values :query
takes) and no `reverse` parameter at all. Sending either would earn an
unknown-parameter or invalid_value refusal from the real server, not a
silently-wrong answer -- which is why this only surfaces against a served
backend, not a Go-side unit test of the request builder.
Add listSortPushdownEligible to gate the pushdown fast path to the two
keyset orders listIssues can actually page, unconditionally refusing any
reversed request (listIssues has no parameter that names a reversed
direction, so a reversed request of any order walks and sorts
client-side). Drop the unconditional `reverse` send from sortedPage.
served_list_test.go's three affected cases encoded the wrong,
enterprise-only wire shape (expecting reverse=false on the fallback path,
expecting a reversed order to push down with sort+reverse, and expecting a
Go-side/SQL-side mix in listIssues' sort enum); corrected to match OSS's
actual, provable wire contract rather than left red. list_sort_test.go's
publishedParamEnum helper now reads the OpenAPI document via the
go:embed'd spec.OpenAPIV0() accessor instead of a relative os.ReadFile, so
it no longer depends on the test's working directory (this also fixes the
same test failing under Bazel's sandbox, which does not expose arbitrary
relative-path files).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): reconcile write-role tests against OSS's BatchClose/Update shapes
OSS's apigen.BatchCloseRequest and BatchCloseResponse publish no
claim_next/claimed_next member at all (ledger row
W-CloseBatchRequest.ClaimNext) -- bd-enterprise's claim-folded-into-close
shape has no OSS wire counterpart. Replace the three tests that exercised
decoding a served claim (TestBatchCloseRefusesAServedClaimCarryingNoIssue,
TestBatchCloseRefusesAServedClaimTheRequestNeverEarned,
TestBatchCloseKeepsTheClaimItEarned) with
TestBatchCloseRefusesClaimNextUnconditionally, which pins the actual OSS
behavior: a CloseBatchRequest naming ClaimNext refuses before any dial, on
both the unserved and served legs, regardless of what a server would
otherwise answer. Drop the now-dead "claim-next" case and helper claim
variable from TestBatchCloseServesTheSingleItemShapeAndRefusesTheRest.
Also:
- Add the ForceNotesOverwrite case to
TestUpdateRefusesEveryMemberTheWireExcludes's hand-maintained member
map -- a ledgered-pending UpdateRequest member (see
W-UpdateRequest.ForceNotesOverwrite) the lift's test left uncovered.
- Remove write_roles_test.go's duplicate `ptr[T]` generic helper, which
redeclared the one already in helpers_test.go and never compiled.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(conformance): pin post-write RowVersion propagation across Update/Close/Reopen
types.Issue.RowVersion is json:"-", so an implementation that forgot to
populate it on the way back from storage would still compile and still
pass every existing content assertion in lifecycle_close_reopen_contract.go
-- none of them read RowVersion. Add
RunLifecycleResultsCarryThePostWriteRowVersion, which chains each write's
result token into the next write's ExpectedVersion (Update -> Close ->
Reopen) with no intervening read, so a dropped or stale token fails closed
on the next write's optimistic-concurrency guard rather than passing
silently.
Register it in role_bundle_cases.go's roleContractCases dispatch table at
its declaration-order position -- TestRoleContractCasesCoverEveryContractCase
was failing because the function existed with no dispatcher entry.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* chore(bazel): sync BUILD files via gazelle for the S3 reconciliation
bazel run //:gazelle after the compile/read-role/write-role reconciliation
commits:
- internal/httpclient/BUILD.bazel: regenerate srcs/deps for the files
added, removed and newly imported above (drops the deleted
policy/journal-watch/readylister files, adds the spec dep for
list_sort_test.go's embedded OpenAPI read). Manually add
`data = glob(["*.go"])` to the go_test rule, matching the idiom already
used by //internal/httpclient/encode: completeness_test.go and
divergence_citation_gate_test.go read their own package's .go sources at
runtime (a regex scan of unsupported_gen.go and an AST walk of the
package directory), and Bazel's go_test sandbox only copies in files a
rule declares as `data` -- `srcs` are compiled away, not left readable.
A `# keep` directive protects the attribute across future gazelle runs
(confirmed gazelle does not manage `data` on this rule).
- internal/storage/BUILD.bazel,
internal/storage/unsupportedgen/BUILD.bazel: pick up the new
from-scratch unsupportedgen generator package.
bazel test //internal/httpclient/... //backend/conformance/... now passes
all 4 targets (previously 4 of the package's tests failed only under
Bazel's sandbox: TestInterfaceCompleteness,
TestTheCountGroupVocabularyMatchesTheSharedOne,
TestFlaglessListSortIsTheSameOrderNamed,
TestDivergenceParkCitationsResolveToLiveLedgerRows).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(conformance): close two pre-existing leg-wiring gaps found while registering http
- RunLifecycleResultsCarryThePostWriteRowVersion (added in 536561124) was
never wired into dolt/embeddeddolt/uow's own lifecycle_close_reopen test
tables, only into the http leg's. TestEveryLegWiresEveryRoleContract
catches this once a fourth leg forces the test to actually run again;
wire the missing t.Run into all three.
- The three StatsReporter "suppressed rows" cases (GateIssues/TemplateIssues
breakdown) were never added to served_graph_aux_test.go even though the
wire already carries both fields end-to-end (types.Statistics is shared
verbatim by apigen.StatsResponse and the client's decode). Wire them; all
three pass over HTTP with no server or client change needed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(conformance): register the http leg in the role-contract lock
Registers "http" (wiringRoot internal/httpclient) in contractLegRegistry.
The served-surface harness (served_harness_test.go + 16 served_*_test.go
files) already wired 495 of 576 role-contract entrypoints from the
bd-enterprise lift; this change:
- Waives the 3 RunCycleDetectorIncludeTracks* cases: issueops.
DetectCyclesRequest.IncludeTracks has no plumbing on either side of the
wire today (cycles.go always calls DetectCycles with a zero-value
request; the client's DetectCycles ignores its request parameter). No
slice through S13 adds one, so the waiver says a new slice is needed
rather than pointing at an existing one.
- Sets adoptionCeiling: 75 for the remaining, not-yet-wired write-role and
internal-plumbing entrypoints (lifecycle writes, DependencyEditor,
MetadataCAS, Lease, Sweeper, VersionReconciler, Bootstrapper/
InitVerifier, cross-record invariants) that S4 and later write slices
pick up and ratchet down.
- Adds //internal/httpclient:go_srcs to storage_test's data: the leg's
wiring root lives outside internal/storage, and the Bazel sandbox has no
other reason to materialize those sources for a test that reads them at
runtime.
TestEveryLegWiresEveryRoleContract, TestEveryRoleMethodHasAContractCase,
and RunUnsupportedContract (already wired pre-lift in
internal/httpclient/completeness_test.go) all pass for the http leg.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): close review's HIGH 2-5 gaps and reconcile their test doubles
HIGH 2 (BatchCloser), HIGH 3 (created_by stamping, absent-parent 404/400,
force_notes_overwrite carriage) and HIGH 4 (CapabilityError now unwraps to a
typed *storage.ErrUnsupported with Capability) were implemented in prior
commits on this branch; this change finishes HIGH 5 and the verification the
review asked for:
- role_reader.go's Reader.Get and bridge.go's raw GetIssue both now stitch
Revision back onto Issue.RowVersion via parseRevision, so a guarded write
built off either read carries the row's real token instead of a decoded
zero. Both are pinned by round-trip served tests that guard a write with
the read's token and require a stale-token replay to refuse.
- Fixed create_test.go's stale assertion that created_by must stay empty
(contradicted the HIGH 3 stamping fix it predated).
- Fixed recordingWire's canned IssueDetails{} response (roles_test.go): it
omitted Revision, which is a required field on every getIssue response
per the OpenAPI spec, and the HIGH 5 stitch now reads it unconditionally.
- Synced internal/httpclient/wire/BUILD.bazel via gazelle: handshake_test.go's
new internal/storage import had no corresponding dep.
- gofmt'd write_roles_test.go's stubWire struct tags.
Verified: served HTTP conformance tier (BEADS_TEST_EMBEDDED_DOLT=1
BEADS_HTTP_TEST_REQUIRED=1 go test ./internal/httpclient/, -timeout=40m)
600 PASS / 0 FAIL / 72 SKIP (all ledgered known-divergence waivers). One
TLS-handshake-timing test failed in that run and passed 3/3 in isolation
(unrelated file, pre-existing flake under machine load, not reproducible).
Also green: default go test (minus the pre-existing, unrelated
TestEngdocsAndRootMarkdownLinks doc-link drift and the environmental
internal/storage/uow dolt-server-spawn hang), bazel build/test for the
touched packages, make api-check, and gofmt/vet.
Live-mutated and reverted 3 guards to confirm their tests are load-bearing:
role_reader.go's RowVersion stitch, bridge.go's RowVersion stitch, and
lifecycle.go's force_notes_overwrite wiring.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(ci): add served HTTP conformance as Tier 3 of the conformance gate
The served tier (internal/httpclient's served_*_test.go corpus) never ran
in CI, so HIGH 2-5's regressions (and any future ones) would merge
undetected. Add it as Tier 3 of scripts/conformance.sh, the existing
required entrypoint conformance.yml already runs on every PR/push/merge_group
(not a new workflow, not wired through ci-gate.sh's aggregation -- this
repo's convention for this gate is a single standalone job):
CGO_ENABLED=1 BEADS_TEST_EMBEDDED_DOLT=1 BEADS_HTTP_TEST_REQUIRED=1 \
go test -timeout=40m ./internal/httpclient/
Left unsharded: Tier 1 and Tier 2 are themselves single unsharded
invocations, and Tier 3 is one Go package, not a multi-package corpus the
repo's existing package-sharding scripts apply to. No Bazel-lane addition:
bazel-coverage's embedded/dolt_server outputs only ever substitute for
main.yml/pr-risk.yml jobs, never for conformance.yml's own job, so there is
no existing Bazel-lane relationship for this gate to extend.
Bumped the job's timeout-minutes 45 -> 100 (30m + 10m + 40m budgeted
sub-timeouts plus checkout/setup-go overhead) and updated the three policy
tests that pin this script/workflow byte-for-byte
(TestConformanceScriptUsesExplicitTimeoutBudgets,
TestConformanceScriptPropagatesGoTestFailures,
TestConformanceWorkflowHasOuterTimeoutBudget), re-included
internal/httpclient/** in both path filters (push and pull_request, kept
identical) since served_*_test.go files are the suite itself and would
otherwise be excluded by the shared !**_test.go negation, and documented the
new filter entry in ci_f7c_advisory_test.go's own-entries map.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): root-cause and fix the Happy-Eyeballs TLS flake in store_dependent_test.go
TestTransportForSucceedsWithEnvCAFile (and 3 Handshake-based siblings that
assert on a successful dial through a configured CA: SidecarCAFile,
CAFileEnvAgreeingWithSidecarSucceeds, DialWithNilTransportHTTPClientGetsCAInjectedOnACopy)
flaked once with "tls: first record does not look like a TLS handshake".
Root cause: our code, same class S2 already fixed for TransportFor's direct
callers (forceIPv4Loopback in ca_test.go/ca_wrong_ca_test.go). Every server
these tests dial is addressed by hostname "localhost" (ca.startServer
rewrites it there deliberately, so SNI is actually sent), which Go's
default dialer races a tcp6 dial to ::1:<port> against the real tcp4 dial to
127.0.0.1:<port> for (RFC 6555). An unrelated local process already bound to
a colliding ephemeral port on ::1 can occasionally win that race before the
real dial completes, and the client speaks TLS to that unrelated process
instead -- the misleading handshake error, not a real hang or a product bug.
These four tests go through Handshake with DialOptions{} (HTTPClient left
nil deliberately, since that is itself the nil-HTTPClient branch of DialWith
under test), so they never get a *http.Transport back to mutate the way
forceIPv4Loopback's existing callers do. Fixed via warmIPv4OnlyCATransport:
pre-build the target's CA-scoped transport through the same public
TransportFor the product code itself calls (so it lands in the exact
(path, content)-keyed cache transportForFile's cache hit returns), force
tcp4-only on that cached pointer, and let Handshake's own DialWith run
unchanged -- same branch under test, same cached transport, no sleeps, no
retries. Verified with -race -count=50 on all four: 200/200 pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): refuse DetectCycles(IncludeTracks=true) instead of silently ignoring it
roles_graph.go's httpCycleDetector.DetectCycles took its request as `_`,
forwarding nothing: a caller asking for the widened (tracks-following) walk
silently got the narrower default walk back with no error. listDependencyCycles
has no include_tracks wire parameter at all, so there is no way to honor the
field — only to refuse it loudly or drop it silently. Switch to the typed,
ledger-cited refusal (encode.RefusedError via refuse(), citing new row
"L-cycles-tracks") that CountEdges' own wire bound already uses, add the
ledger row and its Op, regenerate the auto-generated ledger doc, and pin the
behavior with a unit test (no served-tier case needed: the refusal is
pre-dial by design, same reasoning as the existing edge-count bound pin).
Also fix leg_contract_wiring_test.go's cycleTracksWireGapWaiverReason, which
had gone stale: it described the http leg as silently forwarding nothing for
this field, which was true before this change and is not after. The waiver
itself (the three RunCycleDetectorIncludeTracks* contract cases stay unwired
for the http leg) is unaffected — they were never wired, and still are not —
only the reason text needed updating to describe a typed refusal instead of a
silent ignore.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(storage): name the http leg's permanent contract gaps instead of budgeting them
The http leg's role-contract adoption ceiling (internal/storage/contract_leg_registry_test.go)
sat at a flat 75, with a prose "adopting" note that bundled every skip into
one vague bucket — including MetadataCAS, which is in fact fully wired (0
missing cases), a false claim left over from an earlier wiring pass.
Twenty-six of the 75 skips are not S4-and-later work at all: they are
PERMANENT gaps that no future slice closes, because the contract itself has
no publicops role for an HTTP client to route through:
- Importer (4 cases): uow is Importer's one accessor, same precedent the
dolt and embeddeddolt legs already carry (importerOneAccessorWaiverReason).
- RetentionFixture/EpochFixture (15 cases) and CrossRecordInvariantFixture
(7 cases): both fixtures carry raw storage hooks (Resolve, Erase, Mint,
CurrentEpoch, GuardedWrite, EnforceCrossRecordInvariant, ...) with no
publicops field at all. dolt, embeddeddolt and uow wire these because
they ARE the storage engine this address/epoch/retention and
cross-record transaction bookkeeping belongs to; the http leg is a
remote client with none of its own to report. There is no wire surface
for this to ever route through short of a new role being designed from
scratch — a new slice, not a missing test line.
These 26 are now named, reasoned waivers in unwiredContractEntrypoints
(two new reason constants: retentionEpochNoRoleWaiverReason and
crossRecordInvariantNoRoleWaiverReason), and the ceiling drops from 75 to 49
to match. The "adopting" note is rewritten to correctly name every remaining
category under budget (lifecycle writes, DependencyEditor, DualWrite,
Sweeper, VersionReconciler, Bootstrapper/InitVerifier, ExpectedRevision/
Refusal) and to drop the false MetadataCAS claim.
Also documents, at conformanceEntrypointsWiredBy, why a leg wiring a
conformance.RunX reference with every hook left nil would read as "adopted"
by this lock's static source grep while actually running as a skip —
matching the loophole internal/httpclient's divergence_citation_gate_test.go
already closes for skipKnownDivergence call sites. No leg currently does
this (verified by inspection); the note sets the review expectation for
future wiring rather than papering over the gap with no record of the policy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(licensing): scrub internal program/org naming and a leaked private ID from OSS-visible text
MEDIUM 8 of the S3 ship review: remove licensing-hygiene leaks found by a
read-only repo-wide survey, while leaving every legitimate "Contributed by
gascity from bd-enterprise ... to OSS beads under the MIT license." provenance
header and every genuine public gascity/gastownhall self-reference (the real
OSS org, beads.gascity.com docs domain, RBE infra) untouched:
- Delete release-gates/ga-7psli2-3-1-baseline-cmd-bd-fix-gate.md: the only
file in release-gates/ keyed by a private `ga-` tracker ID instead of the
directory's established public `be-` convention (30 other files), and the
only one leaking internal filesystem paths (/home/jaword/...,
gc-management/.gc/artifacts) and internal routing terms (mayor/mpr,
"deployer session"). Not referenced from anywhere else in the tree.
- .golangci.yml: drop two private `ga-2ltro.12`/`ga-26w10` history citations
used as nolint-comment justifications; reworded to the same explanatory
content without naming a private tracker ID.
- Reword 19 "gc native-program" occurrences (14 files, comments and one
design doc) to drop the internal program name while keeping the "S3
reconciliation (2026-10)" dating they were attached to.
- cmd/bd/if_revision.go, if_revision_test.go, update.go,
update_proxied_server.go: drop literal citations of gascity's private
bdstore_conditional.go classifier (including a test function name,
TestIfRevisionEnvelopeGascityContract -> ...DownstreamContract) while
keeping the substantive explanation of why this envelope's shape is
load-bearing for a downstream consumer.
- internal/httpclient/lifecycle.go: drop a citation of gascity's private
internal/beads/bdstore.go.
- internal/httpclient/dial.go: drop the internal/enterprise/httpgw package
path, keeping "a gateway dialer outside this repo".
- internal/httpclient/register.go: reword away from naming "the enterprise
distribution" while keeping the build-tag-opt-in behavior it documents.
Verified: go build ./..., go vet ./internal/httpclient/... ./cmd/bd/...,
go test ./internal/httpclient/... and ./cmd/bd/... -run TestIfRevision all
pass; TestDivergenceLedgerDocMatchesLedger still passes (ledger.go and its
doc were edited in lockstep). gofmt -l reports no diffs on any touched file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient/wire): correct the CapExternalDependencies doc's implied false claim
MEDIUM 9 of the S3 ship review. The old comment on CapExternalDependencies
said this client "has no external-dependency policy layer to gate" and that
"OSS publishes no storage.ExternalDependencyQueryStore-backed client policy
... to compose" immediately after naming bd-enterprise's removed policy.go --
easily misread as "OSS has no external-dependency decorator at all." It does:
internal/storage/externaldeps's decorator is unconditional (no build tag) and
is wired into every local storage chain by cmd/bd/storage_chain.go.
Reworded to say precisely what is and is not true: the decorator exists and
runs for local backends; what this http client lacks is a client-side
composition of that decorator around a remote store, which is moot anyway
because a server advertising this capability already applies the policy
itself before answering. (Rolled in the same "gc native-program" wording
removal from the MEDIUM 8 commit, since both edits touch this comment block.)
A second claim from the same review item -- that OSS is said somewhere not to
serve events:watch -- could not be located despite an exhaustive repo-wide
search (this file, engdocs/, docs/reference/events-journal.md,
SERVE_RUNBOOK.md, the divergence ledger, CHANGELOG.md, and a dedicated
read-only research pass). internal/httpapi/events_watch.go,
routes.go's events.watch route entry, and internal/httpclient/wire/stream.go
all confirm OSS serves it unconditionally (gated only per-workspace by the
events journal setting, not by build/distribution). No corresponding false
claim exists in this worktree to fix; noting this rather than editing
something that already reads correctly.
Verified: go build ./internal/httpclient/..., go vet, go test
./internal/httpclient/wire/... pass. gofmt -l clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(unsupportedgen): add a freshness check for the generated shell
LOW item from the S3 ship review: internal/storage/unsupportedgen had no
tests of its own, so the generator could drift from the checked-in
internal/httpclient/unsupported_gen.go (a storage.DoltStorage interface
change, or a hand-written method added/removed on Store) with nothing to
catch it short of a human noticing the diff. TestInterfaceCompleteness
(completeness_test.go) already pins which methods belong in the shell by
name, but not whether the checked-in bytes match what the generator would
produce today.
TestGeneratedShellIsFresh runs the same scan-then-generate pipeline main()
runs for the registered DoltStorage/httpclient/Store consumer and diffs the
result against the checked-in file -- mirrors the existing
TestDivergenceLedgerDocMatchesLedger pattern in
internal/httpclient/encode/ledger_render_test.go. Added
TestScanHandWrittenMethodsExcludesOwnOutput alongside it to pin the
correctness property the freshness check depends on (scanning must not see
the generator's own prior output as hand-written).
BUILD.bazel: added the go_test rule by hand (gazelle cannot infer the
cross-package `data` dependency a relative-path os.ReadFile needs under the
Bazel sandbox) with data = ["//internal/httpclient:go_srcs"], following the
existing cross-package data pattern in
internal/httpclient/encode/BUILD.bazel. Placed go_test before the
go_srcs.py-managed filegroup block, which `make bazel-sync-check` requires.
Verified: `bazel run //:gazelle` leaves the rule untouched, `bazel test
//internal/storage/unsupportedgen:unsupportedgen_test` passes, `make
bazel-sync-check` is clean, go build/vet/test ./... all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient/encode): scrub a private tracker-ID cluster from the shipped divergence-ledger doc
Follow-up to the MEDIUM 8 licensing scrub: internal/httpclient/encode's
ledger_render.go header and ledger.go's comments/pinnedElsewhere constant/two
Why strings cited "ga-b8ddd.12/.19/.23/.30" by name. These feed directly into
engdocs/design/http-divergence-ledger.md -- a shipped, generated OSS design
doc, the most visible surface this private ID cluster reached (Fork A's
MEDIUM 8 survey found ~40 occurrences of "ga-b8ddd" across the httpclient
package; this commit addresses the subset inside package encode that is
checked-in prose/doc text, not test-fixture or assertion values). Reworded to
keep the same substantive explanation (what closed, what's still deferred,
which dual run resolved L17) without naming the private ID.
The remaining ~30 occurrences of "ga-b8ddd" outside package encode
(offrole.go, metadata.go, store.go, roles_test.go, wire/*_test.go,
served_*_test.go) are left for a follow-up pass: scrubbing them individually
needs the same per-file verification this commit did, and is a larger,
separate blast radius than was practical in this pass.
Regenerated engdocs/design/http-divergence-ledger.md via
`go test ./internal/httpclient/encode -run TestDivergenceLedgerDocMatchesLedger
-update-ledger-doc`; diff confirmed only the ga-b8ddd text changed. go build
./..., go vet ./internal/httpclient/..., and go test ./internal/httpclient/...
all pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(httpclient): drop a private host reference from a test comment
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* revert(licensing): keep the scrub to the contributed client code
The previous scrub also edited upstream files this PR doesn't otherwise
touch (.golangci.yml, cmd/bd --if-revision files from #7203) and deleted an
upstream release-gate record. Restore them; the provenance scrub applies only
to the code contributed from bd-enterprise.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(httpclient): wire S8 count-scope fields on the client
Encode ParentID/NoParent/ExcludeTypes/ExcludeStatus onto countIssues
(table.go + encode.go), add issues.count.scope to the client capability
table (wire/handshake.go, wire/vocabulary_test.go), and refuse locally
before dialing when the server hasn't advertised the token and any scope
field is set (counter.go's refuseUnservedScope), returning a typed
*storage.ErrUnsupported{Capability:"issues.count.scope"} rather than
dropping fields or taking a guaranteed-400 round trip.
Adds encoding-parity coverage (counts_test.go), a masked-token skew test
proving the pre-dial refusal and zero dials (counts_test.go), roundtrip
and skew-table fixes for the newly-encoded fields (encode/roundtrip_test.go,
encode/skew.go), and wires S8's nine shared Counter conformance cases into
the served http leg (served_counter_test.go), including AddDependency so
the wisp/compose cases run instead of skipping. List stays nil on this
fixture since listTable() has pre-existing, unrelated permanent refusals
for ListRequest.NoParent/ExcludeTypes, so the three *MatchesListCardinality
cases skip here by design.
internal/storage's TestEveryLegWiresEveryRoleContract/http ceiling check
remains red (53 skipped vs ceiling 49) after this change, but strictly
improved: before this change the inherited baseline was 62 skipped (13
over ceiling), all nine of S8's new entrypoints unwired. This change wires
all nine, closing exactly that gap; the remaining 4-over are named
RunBootstrapper*/RunDependencyEditor* cases unrelated to count-scope,
pre-existing debt from other already-merged slices whose ceiling was
never bumped for them. Left the ceiling constant alone rather than
raising it to paper over debt this change doesn't own.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(storage): close the http leg's role-contract ceiling regression
TestEveryLegWiresEveryRoleContract/http went red after the rebase (62
skipped, 13 past its ceiling of 49) before any work in this session; the
prior commit closed 9 of those (S8's Counter cases). The remaining 4-over
were RunBootstrapper* (9 entries) and RunDependencyEditor* (2 entries),
both newly-merged contracts this leg had never classified.
RunBootstrapper*: internal/httpclient/accessors.go already names
Bootstrapper PERMANENTLY UNSERVABLE over http (bd init resolves its
substrate identity entirely client-side, before any server exists to
dial) — add a new named waiver, bootstrapperPermanentlyUnservableWaiverReason,
covering all nine entrypoints. VersionReconciler and InitVerifier are
named permanently unservable by the same accessors.go comment but are
left under the ceiling for their own reviewed change, out of scope here.
RunDependencyEditor*: the OSS server already serves add/remove and the
http leg already wires nearly all of DependencyEditor's contract. The two
missing cases (SameTypeReAddWithChangedMetadataMintsOneVersion,
SameTypeReAddWithIdenticalMetadataIsANoOp) only needed
SetVersionedHistoryEnabled wired onto the fixture, the same way
SetJournalEnabled already is — wired it for real; both pass served.
With Bootstrapper waived and DependencyEditor fully wired, the leg's
unwaived-missing count drops from 53 to 42, so adoptionCeiling moves
from 49 to 42 to keep the ratchet exact, with the adopting prose updated
to match.
TestEveryLegWiresEveryRoleContract is green. Served tier (real, CGO +
embedded dolt): 611 PASS, 0 FAIL, 72 SKIP (up from 609). bazel build/test
and make api-check pass for the touched targets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37307836753
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
* fix(conformance): pass -tags gms_pure_go to Tier 3's served httpclient run
scripts/conformance.sh:61 ran a bare `go test -timeout=40m ./internal/httpclient/`
for Tier 3 (served HTTP client/role conformance), which `make ci-pr-policy`'s
build-tag policy check refuses ("bare go test without -tags=gms_pure_go").
Pass the same -tags Tier 1 already does, and update
TestConformanceScriptUsesExplicitTimeoutBudgets' pinned argv to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(bazel): give //internal/httpclient:httpclient_test room to run
The package takes ~350s under --config=ci (BEADS_TEST_SKIP=dolt, no
BEADS_TEST_EMBEDDED_DOLT) — mostly wall-clock wait on the CA tests'
dialer/keep-alive timeouts, not CPU — but the go_test had no size/timeout
override, so it inherited the medium default's 300s and timed out
intermittently under Bazel (and took "Go test equivalence" down with it).
Set size = "large", timeout = "long", matching
//internal/storage/embeddeddolt's go_test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): fix data race in caContextHandler test helper
TestTransportForRaisesMaxIdleConnsPerHostForBursts drives caContextHandler
with 8 concurrent requests; handler() wrote requestSeen/host/sni from every
server goroutine with no synchronization, and the other nine callers (all
single-request) never exercised the race. go test -race reproduced it as
3x DATA RACE across repeated runs.
Guard the fields with a mutex and expose them only through accessor methods
(sawRequest/seenHost/seenSNI), updating every call site in
store_dependent_test.go accordingly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(unsupportedgen): canonicalize reflect-visible stdlib aliases
TestGeneratedShellIsFresh fails on Go >= 1.27: encoding/json.RawMessage
became an alias of encoding/json/jsontext.Value, and reflect only ever sees
an alias's underlying type, so formatType rendered
storage.DoltStorage.MergeMetadata's signature with the new, non-portable
name even though the signature itself did not change. A contributor on
1.27 following the freshness test's own advice ("go generate
./internal/httpclient") would commit a shell that does not compile on the
1.26.7 SDK this repo pins.
Add reflectAliasCanonical, a table mapping a reflect-visible alias target
back to the name a human wrote in source, consulted by namedType. Add an
entry here for any future stdlib alias this generator's output needs to
stay stable across.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): claimOnlyUpdate propagates the claim's RowVersion
bridge.go's GetIssue already stitches details.Revision onto
hydrated.RowVersion when claimOnlyUpdate rehydrates labels/CreatedBy onto
the claim response, but the claim response itself (issueops.Claimer's
bare-row contract) carries no revision at all, and the copy-back only
touched Labels and CreatedBy. So Lifecycle.Update({Actor, IssueID, Claim:
true}) — `bd update X --claim` — handed back Issue.RowVersion == 0 over
http where the local leg returns the real token, and feeding that into the
next write's ExpectedVersion got a spurious mismatch instead of none.
Copy RowVersion back too. Extend
RunLifecycleResultsCarryThePostWriteRowVersion (shared lifecycle/close/reopen
conformance contract) with a claim-shaped first link in the chain: claim,
then guard the next write with the claim result's own token, proving the
claim-only path populates the real version rather than a nonzero constant.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): stitch RowVersion on SearchIssues' exact-id fast path
resolve.go's getIssuesByExactID decodes the same IssueDetails shape
bridge.go's GetIssue does, but never parsed details.Revision the way
GetIssue's own stitch does, so SearchIssues(ctx, "<exact id>") answered
RowVersion 0 — a one-line gap: Issue.RowVersion is json:"-", so the decode
left it at zero, and only the explicit parseRevision stitch recovers it from
the response's own Revision string member.
Add TestSearchIssuesExactIDPopulatesRowVersionForAGuardedWrite, pinning the
same guarded-write round trip TestBridgeGetIssuePopulatesRowVersionForAGuardedWrite
already pins for the plain GetIssue path.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): route every write role's refusals through inexpressible
Review finding 7: write-side refusals were not uniformly *ErrUnsupported.
Read roles already wrap encoder refusals through Store.inexpressible, which
reaches both encode.RefusedError and encode.CapabilityError; the write side's
refuse() returned a bare *encode.RefusedError instead, so a caller doing
errors.As(err, &unsupported) caught the count-scope refusal but missed
Sweeper.Sweep(Limit), DetectCycles(IncludeTracks), ReadyClaimer.ClaimNext,
BatchApplier.ApplyBatch, BatchCloser.CloseBatch, Deleter.Delete,
DependencyEditor.AddDependencies, and BatchCreator.CreateBatch.
Give InexpressibleError a detail field that preserves the original error's
composite text (an "items[%d]..." or "Issue.%s:" prefix a deeper encoder
helper already adds) instead of collapsing the message to the bare ledger
row text, and wrap every remaining write role's exported method in a single
named-return defer calling store.inexpressible, matching the pattern
lifecycle.go's Create/Update/Reopen already use.
Add TestWriteRoleRefusalsMatchTheReadShape, pinning three write-side
refusals (Sweeper.Sweep Limit, CycleDetector.DetectCycles IncludeTracks,
ReadyClaimer.ClaimNext) now classify via errors.As(*storage.ErrUnsupported)
the same way a read-side refusal does.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): stop spelling out Status: StatusOpen on a markdown batch create
Review nit 8: batchcreator.go's refuseUnwirableIssue refuses a populated
Status (apigen.BatchCreateItem has no wire slot for it), and
buildMarkdownBatchRequest always set Status: types.StatusOpen explicitly, so
`bd create --file` refused on every call against an http-backed store.
PreparePublicCreateRequest already defaults an empty Status to StatusOpen on
every backend, including the http client store's own server, so the explicit
assignment here was never a different outcome - only a populated member the
wire can't carry. Leave it unset; the created issue is identical on every
backend, and the http leg's role-level accounting for why Status and
CreatedBy are excluded from its allowlists is documented on
batchCreateCarriedIssueMembers (internal/httpclient/batchcreator.go).
CreatedBy is left as-is: that member genuinely has no wire slot AND the
server does not yet stamp it from Actor (a gap already tracked in this PR's
Follow-ups), so its refusal is correct, not a bug to route around.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(httpapi): correct create.go's CreatedBy comment to caller-asserted
Review nit 8: the comment claimed CreatedBy is stamped from the
"authenticated actor," but actor is parsed straight from the request body
(s.bodyActor) and never authenticated - exactly as this file's own doc
comment states a few lines above. Reword to "caller-asserted actor" and
cross-reference that doc comment so the two stay consistent. No behavior
change: the stamp itself and its test coverage were already correct.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(httpclient): raise MaxIdleConnsPerHost for a supplied client's nil Transport
Review nit 8: DialWith's doc comment promises the raised MaxIdleConnsPerHost
ceiling on every dial, "whether or not a CA is configured," but the
opts.HTTPClient.Transport == nil arm only injected a transport when the
target's CA was configured (resolved.configured()). A caller-supplied
*http.Client with a nil Transport and no CA fell through untouched, so it
defaulted to http.DefaultTransport's own ceiling of 2 at request time instead
- silently exempting this one shape from the doc's promise and forcing a
fresh TLS handshake on most of a concurrent burst even once the pool is warm.
Route this arm through the same dialTransport helper the opts.HTTPClient ==
nil arm above already uses (the CA-scoped transport when a CA is configured,
the shared baseline otherwise), injected onto a copy so the caller's own
*http.Client fields (Timeout, CheckRedirect, Jar) stay untouched.
Add TestDialWithSuppliedClientNilTransportNoCARaisesMaxIdleConnsPerHost: an
end-to-end burst through wire.Client.Do, traced with httptrace, asserting the
connection pool settles into reuse at a burst size the default ceiling of 2
cannot sustain.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs(httpclient): stop citing nonexistent design docs from new source files
Review nit 8: several comments added by this lift cited
engdocs/design/http-client-backend.md, "D8 row N", and beads-design/slices.json
- none of which exist in this repository (the design lived in the
bd-enterprise-internal source this code was lifted from).
Point each citation at what actually ships in-repo instead:
engdocs/design/http-divergence-ledger.md (the generated, human-readable
rendering of the divergence ledger in encode/ledger.go) for the D7/D8/D9
discipline citations in encode.go, ledger.go, bijection_test.go,
write_ledger_test.go, wire/credential.go and store.go's package doc; and a
direct description of the wire-gap it records, rather than a slices.json
scope reference, in leg_contract_wiring_test.go's cycle-tracks waiver
rationale. Regenerate http-divergence-ledger.md from ledger_render.go (which
had the same stale citation) via
`go test ./internal/httpclient/encode -run TestDivergenceLedgerDocMatchesLedger -update-ledger-doc`
so the shipped doc and its generator agree.
No behavior change; comment and generated-doc text only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(httpclient): implement issueops.BatchGetter for the HTTP client leg
The rebase onto origin/main (item 9 of PR #7247's review) brought in
upstream #7248's new issueops.BatchGetter role on storage.DoltStorage,
breaking the HTTP client leg's interface satisfaction. This wires it up
as a read role (no actor, no version guard, writes nothing — dialed as
a POST only because MaxGetManyIDs ids cannot ride a GET query string),
mirroring Counter/Querier rather than the WriteWire-wrapped roles.
- wire/paths.go: add PathIssuesBatchGet ("/v0/beads/issues:batchGet").
- batchgetter.go: httpBatchGetter.GetMany dials the server and stitches
each resolved issue's RowVersion from apigen.BatchGetIssue.Revision
(the role's only wire spelling of the row token, since RowVersion is
json:"-"). checkGetManyIDs pre-validates the cap and blank-id rules,
in the same order internal/storage/issueops.ValidateGetManyRequest
applies them, because batch_get.go's failBatchGetErr collapses every
validation failure into one generic, code-less InvalidArgument the
client could never decode back into *issueops.TooManyIDsError.
- accessors.go: add the BatchGetter() accessor (26th wire-backed role).
- unsupported_gen.go: regenerated, dropping the now-implemented stub.
- completeness_test.go: move BatchGetter into wireBackedAccessors and
bump the asserted accessor-matrix total from 28 to 29.
- served_test.go / served_harness_test.go: both package-local "by hand"
httpapi.Config builders (serveRoles, serveConfig) now wire
cfg.BatchGetter, fixing ~56 served-tier tests that broke once
BatchGetter joined httpapi.Listen's required role set.
- served_batch_getter_test.go: wire the HTTP leg into all 14
backend/conformance.RunBatchGetterXxx cases, following
served_graph_counter_test.go's pattern. This surfaced a real bug:
GetMany built Missing via append([]string(nil), body.Missing...),
which stays nil when the server reports no misses, violating the
role's "Missing is never nil on success" contract — fixed to build
via make([]string, len(body.Missing)) + copy.
Verified: go build/vet clean; internal/httpclient suite green
(including the embedded-dolt served tier); TestEveryLegWiresEveryRoleContract
holds the http leg's adoption ceiling at exactly 42 with no change needed;
TestInterfaceCompleteness and TestAccessorMatrixMatchesDesign pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* build(bazel): auto-sync BUILD files
Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37450731721
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.
* test(httpclient): lock contextServer.seen against concurrent handler goroutines
TestDialWithSuppliedClientNilTransportNoCARaisesMaxIdleConnsPerHost drives
the stub with bursts of 8 concurrent requests; the handler appended to
seen without a lock. Readers only look at seen after their requests return.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(bazel): sync go_srcs managed blocks for unsupportedgen after the main merge
The PR adds internal/storage/unsupportedgen on a base that predates #7320,
which gave every Bazel package a managed repo_files filegroup. On the merge,
`python3 tools/bazel/go_srcs.py --check` (TestBazelGoSrcsBlocksCurrent) found
3 stale blocks: the package's own repo_files block, its entry in the root
BUILD.bazel repo_files list, and its entry in tools/bazel/BUILD.bazel.
`make bazel-sync` regenerated exactly those; gazelle changed nothing else.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(httpclient): refuse DepAddItem.HasSpawner and ThreadID on applyBatch after the main merge
Main's #7209 gave issueops.DepAddItem two members, HasSpawner and
ThreadID, and a conformance case, RunBatchApplyStampsSpawnerIDOnlyWhenNamed.
ApplyDepAddItem on the v0 wire publishes source, target, type and
metadata only, so on the merge the http leg dropped both silently and
two of the PR's gates went red:
- TestEveryWireExcludedWriteMemberCarriesALedgerRow: the reflective
census found two wire-excluded DepAddItem members with no ledger row.
- TestEveryLegWiresEveryRoleContract/http: the new case left the leg
at 43 unwired cases against its exact ceiling of 42.
applyDepAddItemBody now refuses each member where the role would store
something the wire cannot say, under two new KindRefuse rows,
W-DepAddItem.HasSpawner and W-DepAddItem.ThreadID. The spawner flag
refuses only on a waits-for edge, because the role ignores it on every
other type and stores the same row either way. The ledger doc is
regenerated (264 rows).
The served tier wires the new case as a park on W-DepAddItem.HasSpawner,
owned by the family's existing bead, which brings the http leg back to
its ceiling. With the skip removed the case fails on the client's
refusal; with the refusal also removed it fails on the drop the row
names: spawner_id "" where the role stamps the resolved target.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(httpclient): address #7247 relaunch review iteration 1
Review iteration 1 of the relaunched adoption raised six majors. This
commit fixes all six. Five are the majors the first adoption's
iteration 1 raised. Its fixup, 696803f250 (tag
adopt-pr/bd-f5illd/stranded-fixup-696803f250), fixed them but never
reached the branch. That fix is ported here onto the contributor's
current head, with its reasoning unchanged; the sixth is new.
Creates refused what the CLI sends. Every `bd create` carries
CreatedBy, a workspace whose config.yaml names a prefix also carries
IDPrefix, and the client refused both.
- The server stamps created_by from the actor on all three create
shapes: batch_create.go and batch_apply.go now match create.go.
- The client sends a CreatedBy that names the actor and refuses any
other value, which the server's stamp would silently replace.
createIssue and the apply create item share
refuseUnwirableIssueMembers.
- IDPrefix refuses only where the role would act on it: an explicit,
unforced id.
- `bd create --file` stays refused over http. Dropping its Status was
safe but not enough: the batch create still refuses its Provenance
and Owner. The markdown.go and batchcreator.go comments that called
it servable now say so.
keysetFilter ignored AfterPriority. It now compares priority first,
mirroring sqlbuild's KeysetPriorityCreatedAtIDPredicate, and a new
served case resumes a priority keyset position. AfterID or AfterPriority
without AfterCreatedAt no longer reads as a zero-instant position that
discarded every row (the AfterID minor).
Claim with a patch was refused on a false ledger claim. `claim`,
`force_assignee_transfer` and `force_close_policy` now ride updateBody
like `force_notes_overwrite`, so a claim with a patch or a version guard
is one updateIssue call. A claim ALONE sent to a server that predates
the member (#6890) falls back to claimIssue. The three ledger rows are
retired, and the served cases parked on them now run.
OpenReadOnly returns a writable store. It stays writable, as a
documented exception: backends.Backend.OpenReadOnly also serves the
root pre-run's classified reads, and `bd ready --claim` writes through
that store. The exception is named in the backends.Backend contract,
the store_factory invariant and the store doc, and
TestReadOnlyOpenServesTheClaimBdReadyMakes pins it. An OpenNonMutating
backend hook is the follow-up.
Security surface.
- http_target.json is in the doctor gitignore template and its required
patterns.
- BEADS_HTTP_TOKEN and BEADS_HTTP_TOKEN_COMMAND are host-scoped with
BEADS_HTTP_CA_FILE's "host[:port]=value" syntax. A value scoped to
another server falls through to the next rung, and a command scoped
elsewhere never runs.
- LoadTarget and SaveTarget refuse a url that carries userinfo, and
Target.String drops userinfo.
The served Tier 3 had no CI home. The new target
//internal/httpclient:httpclient_served_test is an `embedded`-tagged
sh_test. It reruns httpclient_test's binary through
tools/bazel/go_test_variant.sh with the script's
BEADS_TEST_EMBEDDED_DOLT=1 and BEADS_HTTP_TEST_REQUIRED=1, in 24 shards
on the embedded lane's race build. bazel.yml's bazel-embedded job
already runs `bazel test //... --config=embedded`, so no workflow
changes.
- -test.timeout=19m, not the script's 40m: Go's own timeout then fires,
and dumps every goroutine, before the action's 1200s limit.
- TestBazelRetiredLanesCannotBeNarrowed pins its args and env. The new
TestConformanceServedTierRunsUnderBazel ties the script's Tier 3
switches to the rule.
- The new untagged TestServedTierIsLinkedWhenRequired fails a required
run built without cgo, which drops every served file (the sentinel
minor).
- A first 12-shard run found two test defects; both are fixed.
TestServedBatchApplyBoundsTheItemCount's 1000-item apply outlasts the
client's 60s per-request timeout under -race, so a
race_detector_{on,off}_test.go pair scales its "at bound" half to 150
there, as embeddeddolt's TestBatchApplyContract does. And
TestTheCompositionCatchesAWrongParameter's dropped-filter case leaned
on rows earlier tests had seeded: run alone, or first in its shard,
it failed. It now seeds its own row outside the scope.
- conformance.sh, its script test and
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md name the target.
Deviations from the review's suggested fixes:
- A bare BEADS_HTTP_TOKEN(_COMMAND) fails closed rather than falling
back, and the error does not echo the value: a bare token split at an
"=" would put token text into the pattern half. Dropping the bare
syntax is safe today: the backend has no Register call site and no
docs/ page names the variables.
- Target.String drops userinfo instead of using url.URL.Redacted, which
masks only the password, so a token used as the username would print.
- The pre-#6890 claimIssue fallback leaves RowVersion unset instead of
borrowing the follow-up read's token. That read is a second snapshot,
so its token could postdate a write this claim never saw and arm a
guard that should miss; a guard built from the unset token fails
closed. Against a current server the claim rides updateIssue, whose
`revision` is stitched as for any update.
Also fixed: the nit about the completeness failure message, which now
says to regenerate the shell, and the offrole.go tombstone comment that
named a skip list which no longer exists. Left for follow-up (all
latent, since httpclient.Register has no production caller): vocabOnce
latching a failed fetch, snapshot holding s.mu across the handshake
dial, unchunked graph anchors, private tracker IDs in park owners, and
the bounded journal limit above 10000.
Validation:
- git diff --check; gofmt -l; go vet over internal/httpclient/... and
scripts
- go test -tags gms_pure_go over internal/httpapi/...,
internal/httpclient/..., internal/storage/backends/... and
backend/conformance/...; the doctor gitignore tests; the cmd/bd
events-journal construction guard; ./scripts
- bazel test --config=embedded
//internal/httpclient:httpclient_served_test on the lane's remote
executors: 24 shards, 702 top-level tests, 632 pass, 70 skip (69
ledgered KNOWN DIVERGENCE, plus
TestCAFileHygieneRefusesGroupWritableFile, which finds no
supplementary group on the executor), 0 fail, 0 data races; slowest
shard 283s. tools/bazel/check_testcases.py --bep passes.
- each of the 702 tests run alone (go test -run '^Name$', without
-race): 0 fail
- the whole tier under go test without -race, with the script's
switches and -timeout=40m, before the two test fixes: 633 pass, 69
skip, 0 fail
- removing BEADS_HTTP_TEST_REQUIRED from the rule fails both policy
tests
- bazel test --config=ci over //internal/httpclient/...,
//internal/httpapi/..., //scripts/..., //test/docsync/...,
//internal/storage/backends/..., //backend/conformance/... and
//cmd/bd/doctor/...: 32 tests pass
- make ci-pr-lint: nogo native plus windows/darwin, passes
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
1335 lines
48 KiB
Go
1335 lines
48 KiB
Go
// Contributed by gascity from bd-enterprise (internal/enterprise/httpstore/ca_test.go@49d1df2f6)
|
|
// to OSS beads under the MIT license.
|
|
package httpclient
|
|
|
|
import (
|
|
"bytes"
|
|
"context"
|
|
"crypto/ecdsa"
|
|
"crypto/elliptic"
|
|
"crypto/rand"
|
|
"crypto/sha256"
|
|
"crypto/tls"
|
|
"crypto/x509"
|
|
"crypto/x509/pkix"
|
|
"encoding/json"
|
|
"encoding/pem"
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
"math/big"
|
|
"net"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"net/url"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"sync"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/steveyegge/beads/internal/httpapi/apigen"
|
|
)
|
|
|
|
// clearCAEnvironment puts BEADS_HTTP_CA_FILE into its not-configured state, the
|
|
// CA equivalent of clearCredentialEnvironment, so a test that names only the
|
|
// sidecar rung cannot pass because of an operator's own exported override.
|
|
func clearCAEnvironment(t *testing.T) {
|
|
t.Helper()
|
|
t.Setenv(CAFileEnv, "")
|
|
}
|
|
|
|
// testCA is a self-signed CA plus one server leaf it has signed, built fresh
|
|
// per call so a "wrong CA" scenario never accidentally shares key material
|
|
// with the "right" one.
|
|
type testCA struct {
|
|
pem []byte
|
|
pool *x509.CertPool
|
|
leaf tls.Certificate
|
|
}
|
|
|
|
// newTestCA builds a CA and a leaf for "localhost" / 127.0.0.1, the two forms
|
|
// an httptest server's URL can take.
|
|
func newTestCA(t *testing.T) testCA {
|
|
t.Helper()
|
|
|
|
caKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
|
if err != nil {
|
|
t.Fatalf("generate CA key: %v", err)
|
|
}
|
|
caTmpl := &x509.Certificate{
|
|
SerialNumber: big.NewInt(1),
|
|
Subject: pkix.Name{CommonName: "bdent-e2 test CA " + t.Name()},
|
|
NotBefore: time.Now().Add(-time.Hour),
|
|
NotAfter: time.Now().Add(time.Hour),
|
|
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
|
|
IsCA: true,
|
|
BasicConstraintsValid: true,
|
|
}
|
|
caDER, err := x509.CreateCertificate(rand.Reader, caTmpl, caTmpl, &caKey.PublicKey, caKey)
|
|
if err != nil {
|
|
t.Fatalf("create CA cert: %v", err)
|
|
}
|
|
caCert, err := x509.ParseCertificate(caDER)
|
|
if err != nil {
|
|
t.Fatalf("parse CA cert: %v", err)
|
|
}
|
|
caPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: caDER})
|
|
|
|
leafKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
|
if err != nil {
|
|
t.Fatalf("generate leaf key: %v", err)
|
|
}
|
|
leafTmpl := &x509.Certificate{
|
|
SerialNumber: big.NewInt(2),
|
|
Subject: pkix.Name{CommonName: "localhost"},
|
|
NotBefore: time.Now().Add(-time.Hour),
|
|
NotAfter: time.Now().Add(time.Hour),
|
|
KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageKeyEncipherment,
|
|
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
|
|
DNSNames: []string{"localhost"},
|
|
IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
|
|
}
|
|
leafDER, err := x509.CreateCertificate(rand.Reader, leafTmpl, caCert, &leafKey.PublicKey, caKey)
|
|
if err != nil {
|
|
t.Fatalf("create leaf cert: %v", err)
|
|
}
|
|
leafPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: leafDER})
|
|
leafKeyDER, err := x509.MarshalECPrivateKey(leafKey)
|
|
if err != nil {
|
|
t.Fatalf("marshal leaf key: %v", err)
|
|
}
|
|
leafKeyPEM := pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: leafKeyDER})
|
|
|
|
leaf, err := tls.X509KeyPair(leafPEM, leafKeyPEM)
|
|
if err != nil {
|
|
t.Fatalf("X509KeyPair: %v", err)
|
|
}
|
|
|
|
pool := x509.NewCertPool()
|
|
pool.AddCert(caCert)
|
|
|
|
return testCA{pem: caPEM, pool: pool, leaf: leaf}
|
|
}
|
|
|
|
// secureTempDir returns a fresh temp directory chmod'd to 0700, the same
|
|
// hygiene checkCAFilePermissions (finding 7) requires of a CA file's parent
|
|
// directory. t.TempDir() defaults to a group-writable mode in some sandboxes
|
|
// (observed here: 0775), which that check correctly refuses — the check is
|
|
// doing its job, so the fix belongs in the test fixture, not in the product
|
|
// code. Every test that writes a CA (or CA-adjacent) file must create it
|
|
// under a directory this helper cleaned, not directly under a bare
|
|
// t.TempDir().
|
|
func secureTempDir(t *testing.T) string {
|
|
t.Helper()
|
|
dir := t.TempDir()
|
|
if err := os.Chmod(dir, 0o700); err != nil {
|
|
t.Fatalf("chmod temp dir: %v", err)
|
|
}
|
|
return dir
|
|
}
|
|
|
|
// writePEM writes the CA's own certificate (never the server leaf) to a file a
|
|
// test can point ca_file / BEADS_HTTP_CA_FILE at.
|
|
func (ca testCA) writePEM(t *testing.T) string {
|
|
t.Helper()
|
|
return writeCAFile(t, ca.pem)
|
|
}
|
|
|
|
// writeCAFile writes data to a fresh, hygiene-clean temp file; see
|
|
// secureTempDir.
|
|
func writeCAFile(t *testing.T, data []byte) string {
|
|
t.Helper()
|
|
path := filepath.Join(secureTempDir(t), "ca.pem")
|
|
if err := os.WriteFile(path, data, 0o600); err != nil {
|
|
t.Fatalf("write CA pem: %v", err)
|
|
}
|
|
return path
|
|
}
|
|
|
|
// startServer starts an httptest TLS server whose leaf this CA signed, and
|
|
// returns a URL that names the host "localhost" (a hostname, not an IP
|
|
// literal) so SNI is actually sent — httptest's default 127.0.0.1 URL would
|
|
// make the SNI/Host assertions vacuous, since crypto/tls never sends SNI for
|
|
// an IP-literal ServerName.
|
|
func (ca testCA) startServer(t *testing.T, handler http.HandlerFunc) *url.URL {
|
|
t.Helper()
|
|
srv := httptest.NewUnstartedServer(handler)
|
|
srv.TLS = &tls.Config{Certificates: []tls.Certificate{ca.leaf}}
|
|
srv.StartTLS()
|
|
t.Cleanup(srv.Close)
|
|
|
|
u, err := url.Parse(srv.URL)
|
|
if err != nil {
|
|
t.Fatalf("parse server URL: %v", err)
|
|
}
|
|
_, port, err := net.SplitHostPort(u.Host)
|
|
if err != nil {
|
|
t.Fatalf("split host:port: %v", err)
|
|
}
|
|
u.Host = net.JoinHostPort("localhost", port)
|
|
return u
|
|
}
|
|
|
|
// forceIPv4Loopback makes rt (the concrete *http.Transport TransportFor and
|
|
// TransportForFile always return) dial exclusively over tcp4, never tcp6.
|
|
//
|
|
// Every TLS fixture in this file addresses its server by the hostname
|
|
// "localhost" rather than the IP literal startServer actually bound to
|
|
// (127.0.0.1), specifically so SNI is sent. But net/http's default dialer
|
|
// does RFC 6555 Happy Eyeballs for a hostname that resolves to more than one
|
|
// address family, racing a tcp6 dial to ::1:<port> against the real tcp4
|
|
// dial to 127.0.0.1:<port>. On a host that already has other, unrelated
|
|
// processes bound to high ports on ::1 (observed here: JVM tooling), that
|
|
// race can occasionally "win" against a listener that has nothing to do
|
|
// with this test, before the real tcp4 dial completes. The client then
|
|
// speaks TLS to that unrelated process instead of the test server and the
|
|
// handshake fails with a misleading "first record does not look like a TLS
|
|
// handshake" rather than exercising the CA logic under test at all —
|
|
// the flake this helper exists to remove. Restricting the network to
|
|
// "tcp4" removes the race outright; SNI and the request's Host are
|
|
// unaffected, since crypto/tls and net/http derive both from the URL's
|
|
// host, never from which address family actually carried the bytes.
|
|
func forceIPv4Loopback(t *testing.T, rt http.RoundTripper) {
|
|
t.Helper()
|
|
transport, ok := rt.(*http.Transport)
|
|
if !ok {
|
|
t.Fatalf("forceIPv4Loopback: RoundTripper is %T, want *http.Transport", rt)
|
|
}
|
|
dialer := &net.Dialer{Timeout: 30 * time.Second, KeepAlive: 30 * time.Second}
|
|
transport.DialContext = func(ctx context.Context, _, addr string) (net.Conn, error) {
|
|
return dialer.DialContext(ctx, "tcp4", addr)
|
|
}
|
|
}
|
|
|
|
// caContextHandler answers the handshake like contextServer, and additionally
|
|
// records the SNI name and Host header the request actually carried, so a test
|
|
// can prove TransportFor never overrides either. Some callers (e.g. the
|
|
// MaxIdleConnsPerHost burst test) drive this handler with several concurrent
|
|
// requests, so every field is behind mu: both the writes in handler() and the
|
|
// reads in the accessor methods below.
|
|
type caContextHandler struct {
|
|
body apigen.ContextResponse
|
|
|
|
mu sync.Mutex
|
|
sni string
|
|
host string
|
|
requestSeen bool
|
|
}
|
|
|
|
func (h *caContextHandler) handler() http.HandlerFunc {
|
|
return func(w http.ResponseWriter, r *http.Request) {
|
|
h.mu.Lock()
|
|
h.requestSeen = true
|
|
h.host = r.Host
|
|
if r.TLS != nil {
|
|
h.sni = r.TLS.ServerName
|
|
}
|
|
h.mu.Unlock()
|
|
w.Header().Set("Content-Type", "application/json")
|
|
_ = json.NewEncoder(w).Encode(h.body)
|
|
}
|
|
}
|
|
|
|
// sawRequest reports whether the handler has observed at least one request.
|
|
func (h *caContextHandler) sawRequest() bool {
|
|
h.mu.Lock()
|
|
defer h.mu.Unlock()
|
|
return h.requestSeen
|
|
}
|
|
|
|
// seenHost returns the Host header the most recent request carried.
|
|
func (h *caContextHandler) seenHost() string {
|
|
h.mu.Lock()
|
|
defer h.mu.Unlock()
|
|
return h.host
|
|
}
|
|
|
|
// seenSNI returns the TLS ServerName the most recent request carried.
|
|
func (h *caContextHandler) seenSNI() string {
|
|
h.mu.Lock()
|
|
defer h.mu.Unlock()
|
|
return h.sni
|
|
}
|
|
|
|
func TestTransportForNoCAConfiguredIsUnchanged(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
rt, err := TransportFor(Target{})
|
|
if err != nil {
|
|
t.Fatalf("TransportFor with no CA configured: %v", err)
|
|
}
|
|
if rt != nil {
|
|
t.Errorf("TransportFor returned a non-nil transport with no CA configured: %#v", rt)
|
|
}
|
|
}
|
|
|
|
func TestTransportForMissingFileRefusesClearly(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
absent := filepath.Join(t.TempDir(), "does-not-exist.pem")
|
|
rt, err := TransportFor(Target{CAFile: absent})
|
|
if err == nil {
|
|
t.Fatal("TransportFor accepted a missing ca_file")
|
|
}
|
|
if rt != nil {
|
|
t.Error("TransportFor returned a non-nil transport on error")
|
|
}
|
|
if !strings.Contains(err.Error(), absent) {
|
|
t.Errorf("error %q does not name the missing path %q", err, absent)
|
|
}
|
|
if !strings.Contains(err.Error(), "ca_file") {
|
|
t.Errorf("error %q does not say which setting (ca_file) is at fault", err)
|
|
}
|
|
}
|
|
|
|
func TestTransportForUnreadableFileRefusesClearly(t *testing.T) {
|
|
if os.Geteuid() == 0 {
|
|
t.Skip("root ignores file permissions")
|
|
}
|
|
clearCAEnvironment(t)
|
|
path := filepath.Join(secureTempDir(t), "unreadable.pem")
|
|
if err := os.WriteFile(path, []byte("whatever"), 0o000); err != nil {
|
|
t.Fatalf("write unreadable file: %v", err)
|
|
}
|
|
_, err := TransportFor(Target{CAFile: path})
|
|
if err == nil {
|
|
t.Fatal("TransportFor accepted an unreadable ca_file")
|
|
}
|
|
}
|
|
|
|
func TestTransportForGarbageFileRefusesClearly(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
path := filepath.Join(secureTempDir(t), "garbage.pem")
|
|
if err := os.WriteFile(path, []byte("this is not a PEM certificate\n"), 0o600); err != nil {
|
|
t.Fatalf("write garbage file: %v", err)
|
|
}
|
|
rt, err := TransportFor(Target{CAFile: path})
|
|
if err == nil {
|
|
t.Fatal("TransportFor accepted a non-PEM ca_file")
|
|
}
|
|
if rt != nil {
|
|
t.Error("TransportFor returned a non-nil transport on error")
|
|
}
|
|
if !strings.Contains(err.Error(), "PEM") {
|
|
t.Errorf("error %q does not say the file has no PEM certificates", err)
|
|
}
|
|
}
|
|
|
|
// TestTransportForEnvLabelsErrorsWithTheEnvName checks the refusal names
|
|
// BEADS_HTTP_CA_FILE, not "ca_file", when the env rung is the one that is
|
|
// broken — a refusal has to point at the thing to fix.
|
|
func TestTransportForEnvLabelsErrorsWithTheEnvName(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
absent := filepath.Join(t.TempDir(), "absent.pem")
|
|
u, err := url.Parse("https://example.com:8443")
|
|
if err != nil {
|
|
t.Fatalf("parse url: %v", err)
|
|
}
|
|
t.Setenv(CAFileEnv, u.Host+"="+absent)
|
|
_, err = TransportFor(Target{BaseURL: u})
|
|
if err == nil {
|
|
t.Fatal("TransportFor accepted a missing BEADS_HTTP_CA_FILE")
|
|
}
|
|
if !strings.Contains(err.Error(), CAFileEnv) {
|
|
t.Errorf("error %q does not name %s", err, CAFileEnv)
|
|
}
|
|
}
|
|
|
|
// TestTransportForEnvMalformedValueRefuses is finding 1(b)'s "refuse a
|
|
// malformed value" requirement: a bare path, with no host[:port]= prefix, is
|
|
// no longer accepted at all — the old, unscoped syntax this replaces.
|
|
func TestTransportForEnvMalformedValueRefuses(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
ca := newTestCA(t)
|
|
t.Setenv(CAFileEnv, ca.writePEM(t)) // bare path, no "host=" prefix
|
|
_, err := TransportFor(Target{})
|
|
if err == nil {
|
|
t.Fatal("TransportFor accepted a bare-path BEADS_HTTP_CA_FILE with no host scope")
|
|
}
|
|
if !strings.Contains(err.Error(), CAFileEnv) {
|
|
t.Errorf("error %q does not name %s", err, CAFileEnv)
|
|
}
|
|
}
|
|
|
|
// --- finding 5 (round 2 review): parseCAFileEnv split point and absolute path ---
|
|
|
|
// TestParseCAFileEnvSplitsOnFirstEquals proves a path containing "=" (legal
|
|
// in a POSIX filename, if unusual) parses by splitting on the FIRST "=", not
|
|
// the last: splitting on the last would silently fold the "=" into the host
|
|
// pattern instead of the path, mis-parsing rather than refusing.
|
|
func TestParseCAFileEnvSplitsOnFirstEquals(t *testing.T) {
|
|
dir := t.TempDir()
|
|
weird := filepath.Join(dir, "ca=file.pem")
|
|
host, path, err := parseCAFileEnv("example.com=" + weird)
|
|
if err != nil {
|
|
t.Fatalf("parseCAFileEnv: %v", err)
|
|
}
|
|
if host != "example.com" {
|
|
t.Errorf("host = %q, want example.com", host)
|
|
}
|
|
if path != weird {
|
|
t.Errorf("path = %q, want %q", path, weird)
|
|
}
|
|
}
|
|
|
|
// TestParseCAFileEnvRequiresAbsolutePath is finding 5: a relative path's
|
|
// meaning would depend on the process's current directory at resolution
|
|
// time, which the sidecar and --ca-file paths never do — refuse it outright
|
|
// rather than resolve it against an ambient cwd.
|
|
func TestParseCAFileEnvRequiresAbsolutePath(t *testing.T) {
|
|
_, _, err := parseCAFileEnv("example.com=relative/ca.pem")
|
|
if err == nil {
|
|
t.Fatal("parseCAFileEnv accepted a relative path")
|
|
}
|
|
if !strings.Contains(err.Error(), "absolute") {
|
|
t.Errorf("error %q does not say the path must be absolute", err)
|
|
}
|
|
}
|
|
|
|
// TestParseCAFileEnvRefusesEmptyPort is finding 7 (round 3 review): a host
|
|
// pattern that ends in a bare ":" with nothing after it (or the IPv6 literal
|
|
// equivalent) is malformed — neither of the two meanings a colon carries
|
|
// here (absent entirely: any port; host:port: exactly one) is a reasonable
|
|
// guess for what an empty port after it was meant to say.
|
|
func TestParseCAFileEnvRefusesEmptyPort(t *testing.T) {
|
|
dir := t.TempDir()
|
|
path := filepath.Join(dir, "ca.pem")
|
|
|
|
for _, raw := range []string{
|
|
"h.example:=" + path,
|
|
"[::1]:=" + path,
|
|
} {
|
|
t.Run(raw, func(t *testing.T) {
|
|
_, _, err := parseCAFileEnv(raw)
|
|
if err == nil {
|
|
t.Fatalf("parseCAFileEnv(%q) accepted a host pattern with an empty port", raw)
|
|
}
|
|
if !strings.Contains(err.Error(), "no port") {
|
|
t.Errorf("error %q does not explain the empty-port refusal", err)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// --- finding 5 (round 2 review): caHostMatches normalization ---
|
|
|
|
func TestCAHostMatchesTable(t *testing.T) {
|
|
mustURL := func(t *testing.T, raw string) *url.URL {
|
|
t.Helper()
|
|
u, err := url.Parse(raw)
|
|
if err != nil {
|
|
t.Fatalf("parse %q: %v", raw, err)
|
|
}
|
|
return u
|
|
}
|
|
|
|
cases := []struct {
|
|
name string
|
|
pattern string
|
|
target string
|
|
want bool
|
|
wantErr bool
|
|
}{
|
|
{"exact match", "example.com", "https://example.com/", true, false},
|
|
{"case-insensitive", "Example.COM", "https://example.com/", true, false},
|
|
{"trailing dot on pattern", "example.com.", "https://example.com/", true, false},
|
|
{"trailing dot on target host", "example.com", "https://example.com./", true, false},
|
|
{"no-port pattern matches any target port", "example.com", "https://example.com:8443/", true, false},
|
|
{"explicit port matches default https port implicit in target", "example.com:443", "https://example.com/", true, false},
|
|
{"explicit port matches default http port implicit in target", "example.com:80", "http://example.com/", true, false},
|
|
{"explicit port mismatch refused", "example.com:443", "https://example.com:8443/", false, false},
|
|
{"different host refused", "example.com", "https://other.example.com/", false, false},
|
|
{"ipv6 literal exact", "[::1]:8443", "https://[::1]:8443/", true, false},
|
|
{"ipv6 literal default port", "[::1]", "https://[::1]/", true, false},
|
|
{"ipv6 literal host mismatch", "[::1]", "https://[::2]/", false, false},
|
|
// finding 7 (round 3 review): IDNA/punycode normalization, both
|
|
// directions, and a hostname that plain ASCII lowercasing could
|
|
// never make agree with its punycode form.
|
|
{"idna: unicode pattern matches punycode target", "café.example", "https://xn--caf-dma.example/", true, false},
|
|
{"idna: punycode pattern matches unicode target", "xn--caf-dma.example", "https://café.example/", true, false},
|
|
{"idna: unicode both sides, exact match", "café.example", "https://café.example/", true, false},
|
|
{"idna: unicode host mismatch still refused", "café.example", "https://other.example/", false, false},
|
|
}
|
|
for _, tc := range cases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
target := Target{BaseURL: mustURL(t, tc.target)}
|
|
got, err := caHostMatches(tc.pattern, target)
|
|
if tc.wantErr {
|
|
if err == nil {
|
|
t.Errorf("caHostMatches(%q, %q) returned no error, want one", tc.pattern, tc.target)
|
|
}
|
|
return
|
|
}
|
|
if err != nil {
|
|
t.Fatalf("caHostMatches(%q, %q) returned unexpected error: %v", tc.pattern, tc.target, err)
|
|
}
|
|
if got != tc.want {
|
|
t.Errorf("caHostMatches(%q, %q) = %v, want %v", tc.pattern, tc.target, got, tc.want)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// --- finding 5 (round 2 review): sameCAFile ---
|
|
|
|
func TestSameCAFile(t *testing.T) {
|
|
dir := t.TempDir()
|
|
abs := filepath.Join(dir, "ca.pem")
|
|
if err := os.WriteFile(abs, []byte("x"), 0o600); err != nil {
|
|
t.Fatalf("write file: %v", err)
|
|
}
|
|
other := filepath.Join(dir, "other.pem")
|
|
if err := os.WriteFile(other, []byte("y"), 0o600); err != nil {
|
|
t.Fatalf("write file: %v", err)
|
|
}
|
|
|
|
if !sameCAFile(abs, abs) {
|
|
t.Error("sameCAFile(x, x) = false, want true")
|
|
}
|
|
if !sameCAFile(abs, filepath.Join(dir, ".", "ca.pem")) {
|
|
t.Error("sameCAFile did not treat an unclean-but-equivalent path as the same file")
|
|
}
|
|
if sameCAFile(abs, other) {
|
|
t.Error("sameCAFile treated two distinct files as the same")
|
|
}
|
|
if sameCAFile("", "") == false {
|
|
t.Error("sameCAFile('', '') = false, want true (both unset)")
|
|
}
|
|
if sameCAFile("", abs) {
|
|
t.Error("sameCAFile('', abs) = true, want false")
|
|
}
|
|
}
|
|
|
|
// TestTransportForReplacesRootPoolNotExtends is the mutation-check on the
|
|
// REPLACE requirement: the returned transport's RootCAs must be EXACTLY a
|
|
// fresh pool holding only the configured CA — never the system pool plus an
|
|
// append, which is what would let the Gas City Beads Serve CA (no name
|
|
// constraints) vouch for hosts far outside the one target it was scoped to.
|
|
func TestTransportForReplacesRootPoolNotExtends(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
ca := newTestCA(t)
|
|
target := Target{CAFile: ca.writePEM(t)}
|
|
|
|
rt, err := TransportFor(target)
|
|
if err != nil {
|
|
t.Fatalf("TransportFor: %v", err)
|
|
}
|
|
transport, ok := rt.(*http.Transport)
|
|
if !ok {
|
|
t.Fatalf("TransportFor returned %T, want *http.Transport", rt)
|
|
}
|
|
if transport.TLSClientConfig == nil {
|
|
t.Fatal("TLSClientConfig is nil")
|
|
}
|
|
if transport.TLSClientConfig.ServerName != "" {
|
|
t.Errorf("ServerName = %q, want empty: SNI must come from the request URL, not be pinned here", transport.TLSClientConfig.ServerName)
|
|
}
|
|
got := transport.TLSClientConfig.RootCAs
|
|
if got == nil {
|
|
t.Fatal("RootCAs is nil")
|
|
}
|
|
if !got.Equal(ca.pool) {
|
|
t.Error("RootCAs is not exactly {the configured CA}; a mutation merging it into the system pool (or any other pool) would still pass a looser check but fails this one")
|
|
}
|
|
}
|
|
|
|
// TestTransportForFileCachesByPathAndContent is finding 6's cache-hit case: two
|
|
// calls naming the SAME (absolute path, content) pair must return the exact
|
|
// same *http.Transport, so a long-lived process reuses one connection pool
|
|
// instead of building (and leaking) a fresh transport on every dial.
|
|
func TestTransportForFileCachesByPathAndContent(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
ca := newTestCA(t)
|
|
path := ca.writePEM(t)
|
|
|
|
first, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile (first): %v", err)
|
|
}
|
|
second, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile (second): %v", err)
|
|
}
|
|
if firstT, ok := first.(*http.Transport); !ok || second.(*http.Transport) != firstT {
|
|
t.Error("TransportForFile built a new transport for an unchanged file; it should have reused the cached one")
|
|
}
|
|
}
|
|
|
|
// TestTransportForFileRotatesOnContentChange is finding 6's rotation case: a
|
|
// changed file at the SAME path must produce a NEW transport (a stale
|
|
// transport would keep the old certificate's TLSClientConfig.RootCAs forever),
|
|
// and the superseded transport's idle connections must be closed rather than
|
|
// leaked.
|
|
func TestTransportForFileRotatesOnContentChange(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
oldCA := newTestCA(t)
|
|
newCA := newTestCA(t)
|
|
dir := secureTempDir(t)
|
|
path := filepath.Join(dir, "ca.pem")
|
|
if err := os.WriteFile(path, oldCA.pem, 0o600); err != nil {
|
|
t.Fatalf("write initial ca file: %v", err)
|
|
}
|
|
|
|
before, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile (before rotation): %v", err)
|
|
}
|
|
beforeTransport, ok := before.(*http.Transport)
|
|
if !ok {
|
|
t.Fatalf("TransportForFile returned %T, want *http.Transport", before)
|
|
}
|
|
// Prove it is really the cached instance for this path.
|
|
if again, err := TransportForFile(path); err != nil || again.(*http.Transport) != beforeTransport {
|
|
t.Fatalf("expected a cache hit before rotation, got %v, %v", again, err)
|
|
}
|
|
|
|
if err := os.WriteFile(path, newCA.pem, 0o600); err != nil {
|
|
t.Fatalf("rewrite ca file for rotation: %v", err)
|
|
}
|
|
|
|
after, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile (after rotation): %v", err)
|
|
}
|
|
afterTransport, ok := after.(*http.Transport)
|
|
if !ok {
|
|
t.Fatalf("TransportForFile returned %T, want *http.Transport", after)
|
|
}
|
|
if afterTransport == beforeTransport {
|
|
t.Fatal("TransportForFile kept the stale transport after the file's content changed")
|
|
}
|
|
if !afterTransport.TLSClientConfig.RootCAs.Equal(newCA.pool) {
|
|
t.Error("the rotated transport's RootCAs is not exactly the new CA's pool")
|
|
}
|
|
}
|
|
|
|
// TestTransportForFileRotationClosesSupersededIdleConnections is finding 7's
|
|
// mutation-kill test: a mutant that removed the outgoing transport's
|
|
// CloseIdleConnections() call on rotation would leak its idle connection
|
|
// rather than closing it, but TestTransportForFileRotatesOnContentChange
|
|
// above would still pass (it only checks that the returned *http.Transport
|
|
// pointer changed). This proves the close actually happens, observed
|
|
// through the server's own ConnState callback rather than any internal
|
|
// accounting on the transport.
|
|
func TestTransportForFileRotationClosesSupersededIdleConnections(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
oldCA := newTestCA(t)
|
|
newCA := newTestCA(t)
|
|
h := &caContextHandler{body: v0Context("proj-rotate-close")}
|
|
|
|
var mu sync.Mutex
|
|
closedStates := 0
|
|
srv := httptest.NewUnstartedServer(h.handler())
|
|
srv.TLS = &tls.Config{Certificates: []tls.Certificate{oldCA.leaf}}
|
|
srv.Config.ConnState = func(_ net.Conn, state http.ConnState) {
|
|
if state == http.StateClosed {
|
|
mu.Lock()
|
|
closedStates++
|
|
mu.Unlock()
|
|
}
|
|
}
|
|
srv.StartTLS()
|
|
t.Cleanup(srv.Close)
|
|
|
|
u, err := url.Parse(srv.URL)
|
|
if err != nil {
|
|
t.Fatalf("parse server URL: %v", err)
|
|
}
|
|
_, port, err := net.SplitHostPort(u.Host)
|
|
if err != nil {
|
|
t.Fatalf("split host:port: %v", err)
|
|
}
|
|
u.Host = net.JoinHostPort("localhost", port)
|
|
|
|
dir := secureTempDir(t)
|
|
path := filepath.Join(dir, "ca.pem")
|
|
if err := os.WriteFile(path, oldCA.pem, 0o600); err != nil {
|
|
t.Fatalf("write initial ca file: %v", err)
|
|
}
|
|
|
|
before, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile (before rotation): %v", err)
|
|
}
|
|
forceIPv4Loopback(t, before)
|
|
client := &http.Client{Transport: before}
|
|
resp, err := client.Get(u.String())
|
|
if err != nil {
|
|
t.Fatalf("GET before rotation: %v", err)
|
|
}
|
|
_, _ = io.Copy(io.Discard, resp.Body)
|
|
_ = resp.Body.Close()
|
|
// Let the just-finished connection settle into the idle pool before the
|
|
// rotation below closes it.
|
|
time.Sleep(100 * time.Millisecond)
|
|
|
|
if err := os.WriteFile(path, newCA.pem, 0o600); err != nil {
|
|
t.Fatalf("rewrite ca file for rotation: %v", err)
|
|
}
|
|
if _, err := TransportForFile(path); err != nil {
|
|
t.Fatalf("TransportForFile (after rotation): %v", err)
|
|
}
|
|
|
|
deadline := time.Now().Add(2 * time.Second)
|
|
for {
|
|
mu.Lock()
|
|
n := closedStates
|
|
mu.Unlock()
|
|
if n > 0 {
|
|
break
|
|
}
|
|
if time.Now().After(deadline) {
|
|
t.Fatal("rotating the CA file did not close the superseded transport's idle connection (CloseIdleConnections not called, or not effective)")
|
|
}
|
|
time.Sleep(10 * time.Millisecond)
|
|
}
|
|
}
|
|
|
|
// --- finding 6 (round 2 review): bounded LRU cache ---
|
|
|
|
// resetTransportCacheForTest empties the process-wide transport cache for the
|
|
// duration of one test and restores whatever was there afterward, so this
|
|
// test's assertions about cache SIZE are not polluted by every other test in
|
|
// this file that has already populated it, and vice versa.
|
|
func resetTransportCacheForTest(t *testing.T) {
|
|
t.Helper()
|
|
transportCacheMu.Lock()
|
|
savedCache := transportCache
|
|
savedOrder := transportCacheOrder
|
|
transportCache = map[string]*cachedTransport{}
|
|
transportCacheOrder = nil
|
|
transportCacheMu.Unlock()
|
|
t.Cleanup(func() {
|
|
transportCacheMu.Lock()
|
|
transportCache = savedCache
|
|
transportCacheOrder = savedOrder
|
|
transportCacheMu.Unlock()
|
|
})
|
|
}
|
|
|
|
// TestTransportCacheEvictsLeastRecentlyUsed is finding 6: the cache must be
|
|
// bounded, and an entry pushed out past the bound must have its idle
|
|
// connections closed rather than merely dropped from the map.
|
|
func TestTransportCacheEvictsLeastRecentlyUsed(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
resetTransportCacheForTest(t)
|
|
|
|
dir := secureTempDir(t)
|
|
firstCA := newTestCA(t)
|
|
firstPath := filepath.Join(dir, "ca-0.pem")
|
|
if err := os.WriteFile(firstPath, firstCA.pem, 0o600); err != nil {
|
|
t.Fatalf("write ca file: %v", err)
|
|
}
|
|
first, err := TransportForFile(firstPath)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile: %v", err)
|
|
}
|
|
firstTransport := first.(*http.Transport) //nolint:errcheck // asserted by construction
|
|
|
|
for i := 1; i <= maxCachedTransports; i++ {
|
|
ca := newTestCA(t)
|
|
path := filepath.Join(dir, fmt.Sprintf("ca-%d.pem", i))
|
|
if err := os.WriteFile(path, ca.pem, 0o600); err != nil {
|
|
t.Fatalf("write ca file %d: %v", i, err)
|
|
}
|
|
if _, err := TransportForFile(path); err != nil {
|
|
t.Fatalf("TransportForFile %d: %v", i, err)
|
|
}
|
|
}
|
|
|
|
absFirst, err := filepath.Abs(firstPath)
|
|
if err != nil {
|
|
t.Fatalf("Abs: %v", err)
|
|
}
|
|
transportCacheMu.Lock()
|
|
_, stillCached := transportCache[absFirst]
|
|
size := len(transportCache)
|
|
transportCacheMu.Unlock()
|
|
|
|
if stillCached {
|
|
t.Error("the least-recently-used entry was not evicted once the cache exceeded maxCachedTransports")
|
|
}
|
|
if size != maxCachedTransports {
|
|
t.Errorf("cache size = %d, want %d", size, maxCachedTransports)
|
|
}
|
|
// The evicted entry's idle connections must have been closed, the same
|
|
// as an ordinary rotation — CloseIdleConnections is idempotent, so
|
|
// calling it again here would be a no-op if it already ran; this at
|
|
// least proves eviction does not skip the call entirely by asserting the
|
|
// transport is unusable as a live pool going forward is not directly
|
|
// observable without a real server, so this documents the intent that
|
|
// evictOverCapacityLocked always calls it (see the source).
|
|
_ = firstTransport
|
|
}
|
|
|
|
// TestTransportForFileDoesNotClobberNewerEntryWithStaleRead is finding 6's
|
|
// stale-write guard, updated for the seq-based ordering finding 1 (round 3)
|
|
// requires: a concurrent reader whose read of the file STARTED before one
|
|
// that already landed in the cache must not overwrite that newer entry just
|
|
// because its content hash differs from it. This primes the cache with a
|
|
// synthetic "newer" entry (a seq far ahead of the counter) and then drives a
|
|
// real read of the actual file at the same path through transportForFile,
|
|
// asserting the synthetic newer entry survives untouched. mtime is
|
|
// deliberately NOT how this is simulated: F1 is exactly that mtime cannot be
|
|
// trusted to order two reads (same-second writes via cp -p, rsync -t, tar),
|
|
// so this test primes the ordering the same way production now establishes
|
|
// it — a sequence number claimed under transportCacheMu.
|
|
func TestTransportForFileDoesNotClobberNewerEntryWithStaleRead(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
resetTransportCacheForTest(t)
|
|
|
|
ca := newTestCA(t)
|
|
path := filepath.Join(secureTempDir(t), "ca.pem")
|
|
if err := os.WriteFile(path, ca.pem, 0o600); err != nil {
|
|
t.Fatalf("write ca file: %v", err)
|
|
}
|
|
abs, err := filepath.Abs(path)
|
|
if err != nil {
|
|
t.Fatalf("Abs: %v", err)
|
|
}
|
|
|
|
newerCA := newTestCA(t)
|
|
newerPool, err := parseCAPool(newerCA.pem, abs, "ca_file")
|
|
if err != nil {
|
|
t.Fatalf("parseCAPool: %v", err)
|
|
}
|
|
newerTransport := http.DefaultTransport.(*http.Transport).Clone() //nolint:errcheck
|
|
newerTransport.TLSClientConfig = &tls.Config{RootCAs: newerPool}
|
|
|
|
transportCacheMu.Lock()
|
|
futureSeq := transportReadSeq + 1_000_000
|
|
transportCache[abs] = &cachedTransport{
|
|
hash: sha256.Sum256(newerCA.pem),
|
|
seq: futureSeq,
|
|
transport: newerTransport,
|
|
}
|
|
transportCacheOrder = []string{abs}
|
|
transportCacheMu.Unlock()
|
|
|
|
// The real file on disk has DIFFERENT content (ca.pem, not newerCA.pem)
|
|
// than what was just primed into the cache above, and TransportForFile's
|
|
// own read of it will claim a seq far below futureSeq — the exact shape
|
|
// of a reader that started before, and finished after, a read that
|
|
// already landed a newer revision.
|
|
got, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile: %v", err)
|
|
}
|
|
if got.(*http.Transport) != newerTransport { //nolint:errcheck
|
|
t.Error("a stale read clobbered a newer cache entry instead of losing the race to it")
|
|
}
|
|
}
|
|
|
|
// TestTransportForFileRotatesEvenWithUnchangedMtime is finding 1 (round 3):
|
|
// a genuine rotation whose new content lands with the SAME mtime as the file
|
|
// it replaces — cp -p, rsync -t, tar, or simply two writes inside the same
|
|
// filesystem-clock second — must still be picked up. The prior mtime-based
|
|
// guard treated "not strictly newer" as "stale reader, keep the old
|
|
// transport", which silently kept serving a revoked CA in exactly this
|
|
// shape; ordering now comes from a monotonic sequence number, never mtime.
|
|
func TestTransportForFileRotatesEvenWithUnchangedMtime(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
resetTransportCacheForTest(t)
|
|
|
|
oldCA := newTestCA(t)
|
|
newCA := newTestCA(t)
|
|
path := filepath.Join(secureTempDir(t), "ca.pem")
|
|
if err := os.WriteFile(path, oldCA.pem, 0o600); err != nil {
|
|
t.Fatalf("write initial ca file: %v", err)
|
|
}
|
|
|
|
before, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile (before rotation): %v", err)
|
|
}
|
|
beforeTransport, ok := before.(*http.Transport)
|
|
if !ok {
|
|
t.Fatalf("TransportForFile returned %T, want *http.Transport", before)
|
|
}
|
|
|
|
fi, err := os.Stat(path)
|
|
if err != nil {
|
|
t.Fatalf("stat before rewrite: %v", err)
|
|
}
|
|
mtime := fi.ModTime()
|
|
|
|
if err := os.WriteFile(path, newCA.pem, 0o600); err != nil {
|
|
t.Fatalf("rewrite ca file for rotation: %v", err)
|
|
}
|
|
// Force the mtime to stay EXACTLY what it was before the rewrite — the
|
|
// same-second-write shape cp -p, rsync -t, and tar can all produce.
|
|
if err := os.Chtimes(path, mtime, mtime); err != nil {
|
|
t.Fatalf("Chtimes: %v", err)
|
|
}
|
|
if got, err := os.Stat(path); err != nil {
|
|
t.Fatalf("stat after rewrite: %v", err)
|
|
} else if !got.ModTime().Equal(mtime) {
|
|
t.Fatalf("mtime changed despite Chtimes (got %v, want %v); this test's premise depends on the filesystem preserving it", got.ModTime(), mtime)
|
|
}
|
|
|
|
after, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile (after rotation): %v", err)
|
|
}
|
|
afterTransport, ok := after.(*http.Transport)
|
|
if !ok {
|
|
t.Fatalf("TransportForFile returned %T, want *http.Transport", after)
|
|
}
|
|
|
|
if afterTransport == beforeTransport {
|
|
t.Fatal("TransportForFile kept serving the old transport after a same-mtime rotation; the new CA was silently ignored")
|
|
}
|
|
if !afterTransport.TLSClientConfig.RootCAs.Equal(newCA.pool) {
|
|
t.Error("the rotated transport's RootCAs is not exactly the new CA's pool: the new CA is not actually used")
|
|
}
|
|
if afterTransport.TLSClientConfig.RootCAs.Equal(oldCA.pool) {
|
|
t.Error("the rotated transport's RootCAs still equals the old CA's pool: the old (possibly revoked) CA was not refused")
|
|
}
|
|
}
|
|
|
|
// --- finding 4 (round 3 review): transport provenance survives eviction,
|
|
// re-validates against the current file, and does not match by path alone
|
|
// ---
|
|
|
|
// TestMatchesCachedTransportSurvivesLRUEviction is finding 4's availability
|
|
// half: a transport a caller obtained from TransportForFile and is still
|
|
// holding must keep matching even after enough OTHER paths have been built
|
|
// to push its own cache entry out of the bounded, LRU-evicted
|
|
// transportCache — because transportProvenance (keyed by the transport's own
|
|
// pointer, never evicted) still records what it was built from, and the
|
|
// file it names has not changed.
|
|
func TestMatchesCachedTransportSurvivesLRUEviction(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
resetTransportCacheForTest(t)
|
|
|
|
dir := secureTempDir(t)
|
|
heldCA := newTestCA(t)
|
|
heldPath := filepath.Join(dir, "held.pem")
|
|
if err := os.WriteFile(heldPath, heldCA.pem, 0o600); err != nil {
|
|
t.Fatalf("write held ca file: %v", err)
|
|
}
|
|
held, err := TransportForFile(heldPath)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile(held): %v", err)
|
|
}
|
|
|
|
for i := 0; i < maxCachedTransports+5; i++ {
|
|
ca := newTestCA(t)
|
|
path := filepath.Join(dir, fmt.Sprintf("churn-%d.pem", i))
|
|
if err := os.WriteFile(path, ca.pem, 0o600); err != nil {
|
|
t.Fatalf("write churn ca file %d: %v", i, err)
|
|
}
|
|
if _, err := TransportForFile(path); err != nil {
|
|
t.Fatalf("TransportForFile(churn %d): %v", i, err)
|
|
}
|
|
}
|
|
|
|
absHeld, err := filepath.Abs(heldPath)
|
|
if err != nil {
|
|
t.Fatalf("Abs: %v", err)
|
|
}
|
|
transportCacheMu.Lock()
|
|
_, stillCached := transportCache[absHeld]
|
|
transportCacheMu.Unlock()
|
|
if stillCached {
|
|
t.Fatal("test setup did not actually evict the held path's cache entry")
|
|
}
|
|
|
|
matched, err := matchesCachedTransport(held, heldPath)
|
|
if err != nil {
|
|
t.Fatalf("matchesCachedTransport returned an error after eviction: %v", err)
|
|
}
|
|
if !matched {
|
|
t.Error("matchesCachedTransport refused a held transport solely because its cache entry was LRU-evicted")
|
|
}
|
|
}
|
|
|
|
// TestMatchesCachedTransportRefusedAfterRotation is finding 4's stale-trust
|
|
// half: a transport built for a path must stop matching once that path's
|
|
// file content changes — the point of re-hashing the CURRENT file rather
|
|
// than trusting the transport's cache-map membership alone.
|
|
func TestMatchesCachedTransportRefusedAfterRotation(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
resetTransportCacheForTest(t)
|
|
|
|
path := filepath.Join(secureTempDir(t), "ca.pem")
|
|
oldCA := newTestCA(t)
|
|
if err := os.WriteFile(path, oldCA.pem, 0o600); err != nil {
|
|
t.Fatalf("write ca file: %v", err)
|
|
}
|
|
transport, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile: %v", err)
|
|
}
|
|
|
|
newCA := newTestCA(t)
|
|
if err := os.WriteFile(path, newCA.pem, 0o600); err != nil {
|
|
t.Fatalf("rewrite ca file: %v", err)
|
|
}
|
|
|
|
matched, err := matchesCachedTransport(transport, path)
|
|
if matched {
|
|
t.Error("matchesCachedTransport accepted a transport whose file has been rotated since it was built")
|
|
}
|
|
if !errors.Is(err, errCAFileChangedSinceTransportBuilt) {
|
|
t.Errorf("error = %v, want errCAFileChangedSinceTransportBuilt", err)
|
|
}
|
|
}
|
|
|
|
// TestMatchesCachedTransportRefusesUnrelatedTransportForSamePath is finding
|
|
// 4's M5 survivor: priming the cache via TransportFor(target) for a path,
|
|
// then presenting a DIFFERENT, unrelated *http.Transport for that exact same
|
|
// path, must be refused — matching may never fall back to "some transport is
|
|
// cached for this path", only to "this EXACT transport is the one this
|
|
// package built and still vouches for".
|
|
func TestMatchesCachedTransportRefusesUnrelatedTransportForSamePath(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
resetTransportCacheForTest(t)
|
|
|
|
path := filepath.Join(secureTempDir(t), "ca.pem")
|
|
ca := newTestCA(t)
|
|
if err := os.WriteFile(path, ca.pem, 0o600); err != nil {
|
|
t.Fatalf("write ca file: %v", err)
|
|
}
|
|
target := Target{BaseURL: mustParseURL(t, "https://example.com/"), CAFile: path}
|
|
if _, err := TransportFor(target); err != nil {
|
|
t.Fatalf("TransportFor: %v", err)
|
|
}
|
|
|
|
unrelated := http.DefaultTransport.(*http.Transport).Clone() //nolint:errcheck
|
|
|
|
matched, err := matchesCachedTransport(unrelated, path)
|
|
if matched {
|
|
t.Error("matchesCachedTransport accepted an unrelated *http.Transport merely because a transport is cached for the same path")
|
|
}
|
|
if err != nil {
|
|
t.Errorf("matchesCachedTransport returned an error for an unrecognized transport, want a plain false: %v", err)
|
|
}
|
|
}
|
|
|
|
// --- finding 2: TOCTOU between the permission check and the read ---
|
|
|
|
// TestReadCAFileRefusesFileSwappedAfterPermissionCheck exercises the
|
|
// caFileReadHookForTest seam to swap the CA file's content for a DIFFERENT
|
|
// (but equally well-permissioned) regular file in the exact window between
|
|
// checkCAFilePermissions succeeding and readCAFile's own open — the TOCTOU
|
|
// gap finding 2 flags. No permission check run BEFORE the swap could ever
|
|
// see it; only comparing what was actually opened (via os.SameFile) against
|
|
// what was checked catches it.
|
|
func TestReadCAFileRefusesFileSwappedAfterPermissionCheck(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
dir := secureTempDir(t)
|
|
path := filepath.Join(dir, "ca.pem")
|
|
|
|
original := newTestCA(t)
|
|
if err := os.WriteFile(path, original.pem, 0o600); err != nil {
|
|
t.Fatalf("write original ca file: %v", err)
|
|
}
|
|
|
|
swapped := newTestCA(t)
|
|
swapPath := filepath.Join(dir, "swapped.pem")
|
|
if err := os.WriteFile(swapPath, swapped.pem, 0o600); err != nil {
|
|
t.Fatalf("write swap-in ca file: %v", err)
|
|
}
|
|
|
|
restore := caFileReadHookForTest
|
|
t.Cleanup(func() { caFileReadHookForTest = restore })
|
|
caFileReadHookForTest = func(real string) {
|
|
// Replace path's directory entry with a different, but equally
|
|
// well-permissioned, file — AFTER checkCAFilePermissions already
|
|
// examined (and returned the FileInfo for) the original one.
|
|
if err := os.Rename(swapPath, real); err != nil {
|
|
t.Fatalf("rename swap-in file over checked path: %v", err)
|
|
}
|
|
}
|
|
|
|
_, err := readCAFile(path, "ca_file")
|
|
if err == nil {
|
|
t.Fatal("readCAFile accepted a file swapped out from under it between the permission check and the read")
|
|
}
|
|
if !strings.Contains(err.Error(), "changed between the permission check and the read") {
|
|
t.Errorf("error %q does not explain the TOCTOU refusal", err)
|
|
}
|
|
}
|
|
|
|
// TestReadCAFileRefusesSymlinkSwappedInAfterPermissionCheck is the sibling
|
|
// TOCTOU case: the checked path is replaced with a SYMLINK, rather than
|
|
// another regular file, in the same window. openCAFileNoFollow's O_NOFOLLOW
|
|
// must refuse this outright.
|
|
func TestReadCAFileRefusesSymlinkSwappedInAfterPermissionCheck(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
dir := secureTempDir(t)
|
|
path := filepath.Join(dir, "ca.pem")
|
|
|
|
original := newTestCA(t)
|
|
if err := os.WriteFile(path, original.pem, 0o600); err != nil {
|
|
t.Fatalf("write original ca file: %v", err)
|
|
}
|
|
|
|
elsewhere := newTestCA(t)
|
|
elsewhereDir := secureTempDir(t)
|
|
elsewherePath := filepath.Join(elsewhereDir, "elsewhere.pem")
|
|
if err := os.WriteFile(elsewherePath, elsewhere.pem, 0o600); err != nil {
|
|
t.Fatalf("write elsewhere ca file: %v", err)
|
|
}
|
|
|
|
restore := caFileReadHookForTest
|
|
t.Cleanup(func() { caFileReadHookForTest = restore })
|
|
caFileReadHookForTest = func(real string) {
|
|
if err := os.Remove(real); err != nil {
|
|
t.Fatalf("remove checked path before symlink swap: %v", err)
|
|
}
|
|
if err := os.Symlink(elsewherePath, real); err != nil {
|
|
t.Fatalf("symlink swap-in over checked path: %v", err)
|
|
}
|
|
}
|
|
|
|
_, err := readCAFile(path, "ca_file")
|
|
if err == nil {
|
|
t.Fatal("readCAFile accepted a symlink swapped in after the permission check, in violation of O_NOFOLLOW")
|
|
}
|
|
}
|
|
|
|
// --- finding 4: proxy handling ---
|
|
|
|
// TestCAAwareProxyRefusesHTTPSProxy is finding 4's refusal half: a
|
|
// CA-scoped transport must not reach its target through an https:// proxy,
|
|
// because the CONNECT to the proxy itself would be verified with the narrow
|
|
// CA pool, and Proxy-Authorization would ride across that connection.
|
|
func TestCAAwareProxyRefusesHTTPSProxy(t *testing.T) {
|
|
t.Setenv("HTTPS_PROXY", "https://proxy.example.com:443")
|
|
t.Setenv("HTTP_PROXY", "")
|
|
t.Setenv("NO_PROXY", "")
|
|
req, err := http.NewRequest(http.MethodGet, "https://target.example.com/", nil)
|
|
if err != nil {
|
|
t.Fatalf("NewRequest: %v", err)
|
|
}
|
|
_, err = caAwareProxy(req)
|
|
if err == nil {
|
|
t.Fatal("caAwareProxy accepted an https:// proxy while a CA is configured")
|
|
}
|
|
if !strings.Contains(err.Error(), "https") {
|
|
t.Errorf("error %q does not mention the https proxy", err)
|
|
}
|
|
}
|
|
|
|
// TestCAAwareProxyAllowsHTTPProxy confirms the documented safe case: a plain
|
|
// http:// proxy is passed through unchanged, because the destination's TLS is
|
|
// still verified end to end against the configured CA through it.
|
|
func TestCAAwareProxyAllowsHTTPProxy(t *testing.T) {
|
|
t.Setenv("HTTPS_PROXY", "http://proxy.example.com:8080")
|
|
t.Setenv("HTTP_PROXY", "")
|
|
t.Setenv("NO_PROXY", "")
|
|
req, err := http.NewRequest(http.MethodGet, "https://target.example.com/", nil)
|
|
if err != nil {
|
|
t.Fatalf("NewRequest: %v", err)
|
|
}
|
|
proxyURL, err := caAwareProxy(req)
|
|
if err != nil {
|
|
t.Fatalf("caAwareProxy refused a plain http:// proxy: %v", err)
|
|
}
|
|
if proxyURL == nil || proxyURL.Scheme != "http" {
|
|
t.Errorf("proxyURL = %v, want an http:// URL", proxyURL)
|
|
}
|
|
}
|
|
|
|
// TestCAAwareProxyNoProxyConfiguredPassesThrough confirms the no-proxy-set
|
|
// case is unaffected: nil, nil, exactly like http.ProxyFromEnvironment.
|
|
func TestCAAwareProxyNoProxyConfiguredPassesThrough(t *testing.T) {
|
|
t.Setenv("HTTPS_PROXY", "")
|
|
t.Setenv("HTTP_PROXY", "")
|
|
t.Setenv("NO_PROXY", "")
|
|
req, err := http.NewRequest(http.MethodGet, "https://target.example.com/", nil)
|
|
if err != nil {
|
|
t.Fatalf("NewRequest: %v", err)
|
|
}
|
|
proxyURL, err := caAwareProxy(req)
|
|
if err != nil {
|
|
t.Fatalf("caAwareProxy: %v", err)
|
|
}
|
|
if proxyURL != nil {
|
|
t.Errorf("proxyURL = %v, want nil with no proxy configured", proxyURL)
|
|
}
|
|
}
|
|
|
|
// --- finding 7: CA file hygiene ---
|
|
|
|
// TestCAFileHygieneRefusesOversizedFile caps the size at 1 MiB: a file past
|
|
// that ceiling is almost certainly the wrong file, not a deliberate CA
|
|
// bundle. This is finding 7's mutation-kill test: the file is built from
|
|
// many VALID, repeated CERTIFICATE PEM blocks (the real CA's own PEM,
|
|
// repeated past 1 MiB), not garbage bytes — so a mutant that removed the
|
|
// size check (or swapped LimitReader for a plain read) would otherwise
|
|
// happily parse the whole thing into a working pool and this test would
|
|
// still need the size gate specifically to catch it. Garbage bytes would
|
|
// also be refused by parseCAPool alone, which would let that mutant survive.
|
|
func TestCAFileHygieneRefusesOversizedFile(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
ca := newTestCA(t)
|
|
path := filepath.Join(secureTempDir(t), "huge.pem")
|
|
|
|
var buf bytes.Buffer
|
|
for buf.Len() <= maxCAFileBytes {
|
|
buf.Write(ca.pem)
|
|
}
|
|
if err := os.WriteFile(path, buf.Bytes(), 0o600); err != nil {
|
|
t.Fatalf("write huge file: %v", err)
|
|
}
|
|
_, err := TransportForFile(path)
|
|
if err == nil {
|
|
t.Fatal("TransportForFile accepted a file larger than the size cap")
|
|
}
|
|
if !strings.Contains(err.Error(), fmt.Sprintf("%d", maxCAFileBytes)) {
|
|
t.Errorf("error %q does not name the size cap; want it to be refused BY THE SIZE CHECK, not a PEM-parse failure", err)
|
|
}
|
|
}
|
|
|
|
// chownToNonPrivateGroup chowns path's group to one of the running process's
|
|
// own real supplementary groups — a group that genuinely exists on this
|
|
// host, and that the process is genuinely a member of, but whose NAME is not
|
|
// the running user's own username, so isUserPrivateGroup correctly refuses
|
|
// it (finding 4's "genuinely shared group" case, as opposed to the
|
|
// Debian/OpenSSH user-private-group convention). Skips the test when the
|
|
// process belongs to no supplementary group at all (a minimal container).
|
|
func chownToNonPrivateGroup(t *testing.T, path string) {
|
|
t.Helper()
|
|
groups, err := os.Getgroups()
|
|
if err != nil {
|
|
t.Skipf("Getgroups: %v", err)
|
|
}
|
|
primary := os.Getgid()
|
|
for _, gid := range groups {
|
|
if gid == primary {
|
|
continue
|
|
}
|
|
if err := os.Chown(path, -1, gid); err == nil {
|
|
return
|
|
}
|
|
}
|
|
t.Skip("process belongs to no supplementary group usable for this test")
|
|
}
|
|
|
|
// TestCAFileHygieneRefusesGroupWritableFile refuses a CA file that is
|
|
// group-writable by a GENUINELY shared group — one of the running process's
|
|
// real supplementary groups, not its own user-private group — proving
|
|
// finding 4's exception is scoped to the private-group case only.
|
|
func TestCAFileHygieneRefusesGroupWritableFile(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
ca := newTestCA(t)
|
|
path := filepath.Join(secureTempDir(t), "ca.pem")
|
|
if err := os.WriteFile(path, ca.pem, 0o660); err != nil {
|
|
t.Fatalf("write group-writable ca file: %v", err)
|
|
}
|
|
// WriteFile's mode passes through the process umask: under umask 022 the
|
|
// file lands 0640, not group-writable at all, and the refusal under test
|
|
// is never reached. Set the mode explicitly.
|
|
if err := os.Chmod(path, 0o660); err != nil {
|
|
t.Fatalf("chmod ca file 0660: %v", err)
|
|
}
|
|
chownToNonPrivateGroup(t, path)
|
|
_, err := TransportForFile(path)
|
|
if err == nil {
|
|
t.Fatal("TransportForFile accepted a ca file writable by a genuinely shared group")
|
|
}
|
|
if !strings.Contains(err.Error(), "writable") {
|
|
t.Errorf("error %q does not explain the hygiene failure", err)
|
|
}
|
|
}
|
|
|
|
// TestCAFileHygieneRefusesWorldWritableParentDir refuses a CA file whose
|
|
// parent directory anyone can write to — SSH's known_hosts / authorized_keys
|
|
// hygiene rule — since replacing the file entirely bypasses the file's own
|
|
// permission bits.
|
|
func TestCAFileHygieneRefusesWorldWritableParentDir(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
ca := newTestCA(t)
|
|
dir := t.TempDir()
|
|
if err := os.Chmod(dir, 0o777); err != nil {
|
|
t.Fatalf("chmod dir world-writable: %v", err)
|
|
}
|
|
path := filepath.Join(dir, "ca.pem")
|
|
if err := os.WriteFile(path, ca.pem, 0o600); err != nil {
|
|
t.Fatalf("write ca file: %v", err)
|
|
}
|
|
_, err := TransportForFile(path)
|
|
if err == nil {
|
|
t.Fatal("TransportForFile accepted a ca file inside a world-writable directory")
|
|
}
|
|
if !strings.Contains(err.Error(), "directory") {
|
|
t.Errorf("error %q does not name the directory as the problem", err)
|
|
}
|
|
}
|
|
|
|
// TestCAFileHygieneRefusesNonCertificateBlock refuses a PEM file containing
|
|
// any non-CERTIFICATE block, even if another block in the same file would
|
|
// have parsed fine — accepting the file anyway would trust a pool the
|
|
// operator never actually reviewed.
|
|
func TestCAFileHygieneRefusesNonCertificateBlock(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
ca := newTestCA(t)
|
|
keyBlock := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: []byte("not-really-a-key")})
|
|
mixed := append(append([]byte{}, ca.pem...), keyBlock...)
|
|
path := filepath.Join(secureTempDir(t), "mixed.pem")
|
|
if err := os.WriteFile(path, mixed, 0o600); err != nil {
|
|
t.Fatalf("write mixed pem: %v", err)
|
|
}
|
|
_, err := TransportForFile(path)
|
|
if err == nil {
|
|
t.Fatal("TransportForFile accepted a file with a non-CERTIFICATE PEM block")
|
|
}
|
|
if !strings.Contains(err.Error(), "CERTIFICATE") {
|
|
t.Errorf("error %q does not name the offending block type", err)
|
|
}
|
|
}
|
|
|
|
// TestCAFileHygieneRefusesUnparsableCertificateBlock refuses a PEM file whose
|
|
// CERTIFICATE-typed block does not actually parse as a certificate, rather
|
|
// than silently ignoring it the way x509.CertPool.AppendCertsFromPEM would.
|
|
func TestCAFileHygieneRefusesUnparsableCertificateBlock(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
ca := newTestCA(t)
|
|
badBlock := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: []byte("not-a-real-der-certificate")})
|
|
mixed := append(append([]byte{}, ca.pem...), badBlock...)
|
|
path := filepath.Join(secureTempDir(t), "unparsable.pem")
|
|
if err := os.WriteFile(path, mixed, 0o600); err != nil {
|
|
t.Fatalf("write unparsable pem: %v", err)
|
|
}
|
|
_, err := TransportForFile(path)
|
|
if err == nil {
|
|
t.Fatal("TransportForFile accepted a file with an unparsable CERTIFICATE block")
|
|
}
|
|
}
|
|
|
|
// TestCAFileHygieneAcceptsLeafCertificateAsPin documents that a single leaf
|
|
// certificate (no CA:true) is a valid, if narrow, way to pin a target,
|
|
// per parseCAPool's doc.
|
|
func TestCAFileHygieneAcceptsLeafCertificateAsPin(t *testing.T) {
|
|
clearCAEnvironment(t)
|
|
ca := newTestCA(t)
|
|
leafDER := ca.leaf.Certificate[0]
|
|
leafPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: leafDER})
|
|
path := filepath.Join(secureTempDir(t), "leaf.pem")
|
|
if err := os.WriteFile(path, leafPEM, 0o600); err != nil {
|
|
t.Fatalf("write leaf-only pem: %v", err)
|
|
}
|
|
rt, err := TransportForFile(path)
|
|
if err != nil {
|
|
t.Fatalf("TransportForFile with a leaf-only pem: %v", err)
|
|
}
|
|
if transport, ok := rt.(*http.Transport); !ok || transport.TLSClientConfig.RootCAs == nil {
|
|
t.Errorf("TransportForFile returned %T (%v), want a *http.Transport with RootCAs set", rt, rt)
|
|
}
|
|
}
|
|
|
|
func mustParseOneCert(t *testing.T, pemBytes []byte) *x509.Certificate {
|
|
t.Helper()
|
|
block, _ := pem.Decode(pemBytes)
|
|
if block == nil {
|
|
t.Fatal("no PEM block")
|
|
}
|
|
cert, err := x509.ParseCertificate(block.Bytes)
|
|
if err != nil {
|
|
t.Fatalf("ParseCertificate: %v", err)
|
|
}
|
|
return cert
|
|
}
|