* feat(http): serve list/show counts/dep tree/ping, close+reopen under a session, one-lookup not-found (S6b/S6c/S6d) S6c. The external-dependency decorator rebuilt its role views from the legacy DoltStorage seam (storereader over itself, IsBlocked, GetDependencyTree, ClaimReadyIssue), which a remote store only partly serves: `bd list`/`bd children` refused on the SearchIssuesWithCounts stub, `bd show --json` printed zero counts, the default `bd dep tree` refused, `bd ready --claim` and `bd close` of an externally blocked issue reached unsupported paths. For a storage.RemoteBackendStore each view now passes the inner role through and lays the policy over it in role terms (externaldeps/remote_roles.go): ready reads and the ready claim exclude the workspace-wide blocked set over a widened window; per-issue checks (claim, close, batch close, list annotations, tree leaves) read only the named issues' edges through EdgeReader instead of every edge in the workspace. A server advertising policy.external_dependencies (MC's does) gets a plain passthrough. `bd ping` uses a new storage.Pinger, which the http store answers with its authenticated context read. S6d. A store reporting storage.CallerAttributionLimitedStore (http) cannot carry a Provenance label or a patch's closed_by_session. The library still refuses both; the CLI omits them where nothing is lost: `bd reopen`'s fixed "bd: reopen <id>" label (the server labels the same entry), and the closed_by_session of `bd update -s closed` (attribution only, usually the ambient CLAUDE_SESSION_ID; an explicit --session prints a notice; `bd close` still records it). See cmd/bd/remote_backend.go. S6b. utils.ExactIDLookupStore: the resolver stops at exact lookups for a store with no substring search, so a missing prefixed id is a not-found (beadserrors.ErrNotFound, "no issue found matching") after one getIssue. The general resolver no longer repeats an identical normalized probe. The http SearchIssueIDs answers D11's partial-id refusal, which now unwraps to ErrNotFound. Contributor auto-routing for a remote workspace reads local config only, so the not-found fallback pays no settings round trip. `bd update <absent> --claim` (gc's hook claim) answers a gc-classifiable not-found in at most two requests. Served e2e (backend/http/e2e_remote_roles_test.go): every gc list shape and `bd children` against embedded, `show --json` count parity, default dep tree parity, external blocking still enforced, close/reopen under CLAUDE_SESSION_ID, missing-id request counts, absent-id claim against the current server and the MC handshake mask, policy passthrough under the MC mask, ping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(externaldeps): role-level remote reader and claimer cases bd ready and bd update --claim reach the policy through the legacy seam and the lifecycle guard, so the remote Reader.Ready exclusion and the remote Claimer guard (what bd serve and library callers reach) are pinned at the role level with a fake remote store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(http): bd serve advertises policy.external_dependencies; clients pass through bd serve composes every ready, claim, claim-next, close, blocking and tree role through the external-dependency policy, but never said so. Every client therefore ran the policy a second time, with the CLIENT machine's external-project config, and claim-next became a non-atomic read-then-claim loop on the client. The server now advertises the token exactly when the roles it serves were composed through the policy: httpapi.Config.ExternalDependencyPolicy, which bd serve derives from its actual chain (externaldeps.Applied for the store arm, externaldeps.AppliedToProvider for the provider arm). A server handed raw roles still withholds it, so the token stays truthful. Against a server that advertises it, every remote role view hands the request to the server's own role untouched: no client validation, no edge reads, claim-next is the server's one atomic call, and the ready count is the server's countReadyWork (new remoteReadyCounter). The client-side policy remains only for servers that do not advertise the token. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(create): default created_by to the request's actor in the create role created_by was stamped in two front doors and never in the library: the CLI (creationActorStamp, with a remote-workspace special case) and the three httpapi create handlers. A library caller of Lifecycle.Create or BatchCreator with an empty CreatedBy stored "" locally and the actor over http. PreparePublicCreateRequest, which every create shape prepares through (Lifecycle.Create, BatchCreator items, batch-apply create items), now defaults an empty CreatedBy to request.Actor; an explicit value is kept. The handlers no longer stamp, `bd create` and the state event create pass only the actor, and creationActorStamp/isRemoteBackendWorkspace are gone. The CLI sites that still create through the raw CreateIssue seam (swarm, todo, gate, the create form) keep stamping currentActor(): that seam is shared with import, where defaulting would re-attribute imported rows, and the http store refuses it anyway, so the remote special case was never reachable there. Conformance: the lifecycle-create, batch-creator and batch-apply contracts now pin created_by == actor for an item that names none, on every leg. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(httpclient): GetIssue answers a miss with storage.ErrNotFound, like the local stores The http store's raw GetIssue answered a missing id with (nil, nil), where DoltStore and the embedded store return a wrapped storage.ErrNotFound. A caller written against the backend contract read the http miss as a hit: `bd create --graph` refused every explicit-id plan over http (patched in cmd/bd on feat/http-mc-corpus; that workaround is reverted there). GetIssue now returns the miss (and the empty id) as storage.ErrNotFound, spelled as issueops.GetIssueInTx spells it. GetLabels, which reads through GetIssue, keeps the local read's "no labels" answer for a missing id; the `bd list --parent` walk reads the typed miss as "parent issue not found". The served bridge test now holds the local reference store and the http store to the same miss: ErrNotFound with the same text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(createbatchequiv): re-record the apply golden for the created_by default c2866ea7fe made the create role default an empty CreatedBy to the request's actor, so every create item of the apply scenario's batch (actor "apply-writer") now stores created_by "apply-writer" instead of "". TestCreateBatchFastPathsMatchPerRow_*/apply compared the stored rows to a digest recorded before that change and failed on exactly the three tables the column reaches: issues and wisps (the row) and events (the update event's old-value snapshot). Fast and per-row bodies still agree; the batch-apply conformance contract pins the same value. Bisected with bazel: passes at be92efe40f's integ twin 696f7acc8e, fails from the created_by commit on. Only apply.json's three digests move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * style: gofmt ledger.go and the externaldeps remote-roles test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cmd/bd): bd create reads routing settings local-only on a remote backend too determineAutoRoutedRepoPath skipped a remote backend's settings document (routing targets are local paths that one shared document cannot name for every client), but getRoutingConfigValue, which `bd create` and migrate-personal read routing.* and contributor.* through, still asked it. The same workspace could route a create one way and a lookup another, and every create paid a settings round trip per key. Both now go through routingSettingsStore: nil for a remote backend, so routing comes from config.yaml / the environment only there, and a local store is still consulted. A unit test holds both reads to the rule with a store that counts settings reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cmd/bd): say so when update -s closed drops an ambient CLAUDE_SESSION_ID Over http `bd update -s closed` cannot carry closed_by_session, so the CLI drops it rather than refusing the close. An explicit --session printed a notice; the ambient CLAUDE_SESSION_ID every agent under Claude Code exports was dropped without a word, a silent data difference from a local backend (the served corpus's mc-update-meta-status-closed). The drop is now named on stderr in both cases, once per command, pointing at `bd close`, whose wire operation records the session. The served e2e asserts the notice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(backend/http): hold bd list --limit 2 to each side's own unlimited order TestE2E_ListWithGCFiltersMatchesEmbedded compared the --limit 2 rows of the served and embedded workspaces as sets. The seed has four P2 rows, so the limit cuts a priority tie whose created_at order differs between two workspaces seeded seconds apart: a test-only flake, the order itself being total. Each side's two rows must now be, in order, the first two of that same side's --limit 0 listing; the unlimited listings are still compared across sides by the shape loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: unit-tier pins for the created_by default, the policy token and the GetIssue miss Three review mutations were caught only by a heavier tier (the uow conformance legs, the served httpclient tier) or by one package's test alone. Each now has a unit test where its rule lives: - issueops: PreparePublicCreateRequest defaults an empty CreatedBy to the actor and keeps an explicit one, without mutating the caller's request. - externaldeps: the policy layer AppliedToProvider finds becomes the handshake's policy.external_dependencies through httpapi.Listen, the composition bd serve performs, and a raw provider's handshake lacks it. - httpclient (no embedded engine): GetIssue over the real wire client against a stub server answers a 404 not_found, and the empty id without dialing, as storage.ErrNotFound spelled like the local stores. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): S6b/c/d entries match the code The S6 entry still said `bd create` suppresses created_by on a remote workspace and that the external-dependency policy runs client-side. Since S6b/c/d, created_by defaults to the actor in the library on every backend, and every bd serve composed through the policy advertises policy.external_dependencies, so clients pass those operations through; the client-side policy (now in role terms) runs only against a server that does not advertise it. New entries: the advertised policy token, bd ping over http, the created_by library default, closed_by_session dropped by an http `bd update -s closed` (with a notice, typed or ambient), reopen without a Provenance label over http, local-only routing.* / contributor.* on a remote backend, GetIssue's miss as storage.ErrNotFound, and list/children/show counts/dep tree over http. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(externaldeps): keep S6's list --ready, --claim-next refusal and lost-race error on the S6c composition The re-adopted S6 head (#7288) carries the review fixup's remote role composition; S6c replaces it with its own (rolesAreRemote, the widened ready window, the policy passthrough). Two behaviours S6 shipped and its reviews pinned would otherwise go: - bd close --claim-next against a server that does not apply the policy is refused whole, before anything is read or closed, with "the served batch close cannot carry a next claim, so nothing was closed; close without --claim-next, then run `bd ready --claim`". S6c passed the batch to the http closer, whose W-CloseBatchRequest.ClaimNext refusal drops the remedy, so TestE2E_BDOverHTTP's two refuseClaimNext steps went red. A request invalid on every backend is still ErrValidation first, and a server that applies the policy still gets the request untouched. - remoteReadyClaimer.ClaimNext reports a page on which every candidate was lost to another actor as "claim ready: lost N races ...", not as the empty answer that means there is no work. - `bd list --ready` (List with ReadyFlag) is answered by the policy store's own reader over the served ready reads, as S6 does. S6c passed it to the served listing, which cannot carry ReadyFlag (httpclient ledger row E-ListRequest.ReadyFlag), so it exited 1 over http with or without external refs; TestE2E_BDOverHTTP and TestServedRolesKeepTheExternalDependencyPolicy were red on it. TestRemoteReaderAnswersAReadyListingWithoutTheWireFlag pins it, with the fake's served List refusing ReadyFlag as the wire does. S6's composition tests now run against S6c as remote_roles_composition_test.go. Their fake serves Ready the way a server does (the S6 composition read ready work through the legacy policy reader instead), and two call logs gain that Ready. TestRemoteBatchCloserRefusesANextClaim, TestRemoteBatchCloserValidatesANextClaimFirst and TestRemoteReadyClaimerReportsLostRaces were red on S6c before this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(externaldeps): unassigned claim-next and fail-closed close reads Review fixes for the S6c composition against a server that does not advertise policy.external_dependencies: the two majors of review attempt 1, plus the minors on the same code. - ReadyClaimer: the composed claim-next reads only open, unassigned ready rows (Unassigned=true, Assignee cleared), the candidates the local ClaimReadyIssueInTx selects. A row another actor holds, or one already assigned to the caller, is no longer taken or counted as a lost race. Each pass reads 25 candidates past the blocked rows, as httpclient's composed claim does, and a full window lost end to end is read once more: claimed rows leave the unassigned set. Only a claim that tried candidates and won none is an error. - BatchCloser: an externally blocked item travels to the server only when its status read finds it closed or not found (liveRefusal). Any other read error is that item's outcome, so a transient or auth failure no longer forwards a refused close to a server that will not refuse it. - TreeWalker: a failed capability probe renders the external leaves over the walked tree instead of failing a walk the server already answered. appendTreeExternalReferences skips an id the tree holds, so an enforcing server's own leaves are not doubled. - remoteEdges names each anchor once and skips an empty one, which the http EdgeReader refuses with ErrValidation. The composition fakes now honor Unassigned/Assignee as workapi.BuildReadyFilter does, model a rival claim landing first and a failing GetIssue, and refuse an empty edge anchor as httpclient does. New: TestRemoteReadyClaimerTakesOnlyUnassignedWork, TestRemoteReadyClaimerWalksAContendedFront, TestRemoteBatchCloserAnswersAFailedReadWithIt, TestRemoteEdgesNamesEachAnchorOnce, and the probe-failure cases of TestRemoteTreeWalkerAddsExternalLeaves. Each is red with its fix reverted. From review attempt 2, three nearby cleanups: - TestServedRolesKeepTheExternalDependencyPolicy seeds open rows held by another actor and by the claimant ahead of the free work, then claims with and without an assignee filter. The unassigned-only claim is now pinned over a real served server, not only through the fakes: red with the pre-fixup remote_roles.go, with filter.Unassigned dropped, and (the filtered claim) with an explicit assignee honored. - The capabilities description in openapi.v0.yaml (and the regenerated types.gen.go comment) no longer calls the whole list build-level: policy.external_dependencies depends on how the server composed its roles, as the paragraph above it says. - createbatchequiv's golden note names issueops.PreparePublicCreateRequest instead of a commit id that is not on this branch. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(backend/http): count a not_found read as reaching the remote in #7469's credential doors main's #7469 added TestExplicitCredentialIgnoresTheEnvPlaintextGrant. Its OpenWith and Open doors read bd-1 from plaintextRemote, which answers every issue read not_found, and the granted leg needs that read to return a nil error. That held while the http client answered a miss with (nil, nil). This branch's "fix(httpclient): GetIssue answers a miss with storage.ErrNotFound, like the local stores" makes the same read return storage.ErrNotFound. So on main plus this branch both doors fail at per_open_credential_test.go:392 ("with Target.AllowInsecureCredential: not found: issue bd-1") under plain go test. The merge is textually clean, so only a build of the merge shows it. The two store doors now count storage.ErrNotFound as success, since that answer came off the wire. The assertions are unchanged. The refused leg still needs the "refusing to send a credential" error and no request on the transport. The granted leg still needs exactly one request carrying "Bearer explicit-token": a granted read that never reaches the wire still fails at :408. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
493 lines
16 KiB
Go
493 lines
16 KiB
Go
//go:build cgo
|
|
|
|
package main
|
|
|
|
import (
|
|
"bufio"
|
|
"bytes"
|
|
"encoding/json"
|
|
"errors"
|
|
"io"
|
|
"net/http"
|
|
"os/exec"
|
|
"slices"
|
|
"strings"
|
|
"sync"
|
|
"syscall"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/steveyegge/beads/internal/httpapi"
|
|
)
|
|
|
|
// End-to-end lifecycle for `bd serve`, driven as a subprocess exactly as an
|
|
// operator runs it: bound address parsed off stdout, requests over real TCP,
|
|
// and a signal to shut it down.
|
|
|
|
// serveProcess is a running `bd serve` subprocess plus the two streams the
|
|
// contract makes promises about.
|
|
type serveProcess struct {
|
|
cmd *exec.Cmd
|
|
addr string
|
|
stderr *syncBuffer
|
|
client *http.Client
|
|
|
|
waitOnce sync.Once
|
|
waitErr error
|
|
}
|
|
|
|
type syncBuffer struct {
|
|
mu sync.Mutex
|
|
b bytes.Buffer
|
|
}
|
|
|
|
func (s *syncBuffer) Write(p []byte) (int, error) {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
return s.b.Write(p)
|
|
}
|
|
|
|
func (s *syncBuffer) String() string {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
return s.b.String()
|
|
}
|
|
|
|
// startServe launches `bd serve` in dir and waits for the bound address it
|
|
// prints. Under the ephemeral default that line is the ONLY way to find the
|
|
// port, which is why it is stdout's first line and why this parses it rather
|
|
// than guessing.
|
|
//
|
|
// It takes the environment rather than a proxied project because the same
|
|
// lifecycle is the contract in every mode serve supports: the server-mode
|
|
// integration test drives this exact harness with a server-mode env.
|
|
func startServe(t *testing.T, bd, dir string, env []string, args ...string) *serveProcess {
|
|
t.Helper()
|
|
|
|
cmd := exec.Command(bd, append([]string{"serve", "--addr", "127.0.0.1:0"}, args...)...)
|
|
cmd.Dir = dir
|
|
cmd.Env = env
|
|
|
|
stdout, err := cmd.StdoutPipe()
|
|
if err != nil {
|
|
t.Fatalf("stdout pipe: %v", err)
|
|
}
|
|
stderr := &syncBuffer{}
|
|
cmd.Stderr = stderr
|
|
|
|
if err := cmd.Start(); err != nil {
|
|
t.Fatalf("start bd serve: %v", err)
|
|
}
|
|
|
|
sp := &serveProcess{
|
|
cmd: cmd,
|
|
stderr: stderr,
|
|
client: &http.Client{Timeout: 30 * time.Second},
|
|
}
|
|
t.Cleanup(func() {
|
|
if sp.cmd.ProcessState == nil {
|
|
_ = sp.cmd.Process.Kill()
|
|
sp.wait()
|
|
}
|
|
})
|
|
|
|
lines := make(chan string, 1)
|
|
go func() {
|
|
scanner := bufio.NewScanner(stdout)
|
|
for scanner.Scan() {
|
|
select {
|
|
case lines <- scanner.Text():
|
|
default:
|
|
}
|
|
}
|
|
close(lines)
|
|
}()
|
|
|
|
select {
|
|
case line, ok := <-lines:
|
|
if !ok {
|
|
t.Fatalf("bd serve exited before printing the bound address\nstderr:\n%s", stderr.String())
|
|
}
|
|
addr, found := strings.CutPrefix(line, "bd serve: listening on http://")
|
|
if !found {
|
|
t.Fatalf("first stdout line %q does not announce the bound address\nstderr:\n%s", line, stderr.String())
|
|
}
|
|
sp.addr = strings.TrimSpace(addr)
|
|
case <-time.After(90 * time.Second):
|
|
t.Fatalf("bd serve did not print a bound address\nstderr:\n%s", stderr.String())
|
|
}
|
|
|
|
return sp
|
|
}
|
|
|
|
func (sp *serveProcess) wait() error {
|
|
sp.waitOnce.Do(func() { sp.waitErr = sp.cmd.Wait() })
|
|
return sp.waitErr
|
|
}
|
|
|
|
func (sp *serveProcess) url(path string) string { return "http://" + sp.addr + path }
|
|
|
|
// assertServedCapabilities checks a decoded context body against the capability
|
|
// set this build's route table implies. Every topology asserts the same thing,
|
|
// because a mode changes where the data lives and never what the handshake
|
|
// advertises.
|
|
//
|
|
// THE EXPECTATION IS DERIVED, and it used to be a literal spelled out here for
|
|
// a reason that did not survive contact with the third operation to be added.
|
|
// That reason was: "comparing the server against its own derivation would pass
|
|
// for any surface at all, and the point of this list is that adding an
|
|
// operation costs a line here."
|
|
//
|
|
// The first half is true and is answered elsewhere. The capability CONTENT is
|
|
// pinned by TestSpecCapabilityVocabularyMatchesTheRouteTable, which compares
|
|
// the derived set against the `capabilities` prose in openapi.v0.yaml in both
|
|
// directions — a paragraph a human writes for the operation being added, which
|
|
// is an independent source in a way a second test literal never was.
|
|
//
|
|
// The second half is the part that inverted. The line this list cost was paid
|
|
// in the WRONG package: an operation is added in internal/httpapi, its own
|
|
// gates go green, and the copy that fails is out here in a proxied shard the
|
|
// author's pre-merge checks do not run. That is precisely what happened to the
|
|
// slice that added `issues.count` — this file went red on a list nothing about
|
|
// counting had changed. And the failure mode the old comment itself describes,
|
|
// a copy silently stuck at four entries while the surface grew, is not a bug
|
|
// derivation makes harder to catch: it is a bug derivation makes impossible to
|
|
// write.
|
|
//
|
|
// What the assertion still buys, and what no in-package test can see: each
|
|
// TOPOLOGY's subprocess really binds, really answers /v0/beads/context, and
|
|
// really publishes the whole set rather than a truncated one.
|
|
func assertServedCapabilities(t *testing.T, body map[string]any) {
|
|
t.Helper()
|
|
caps, ok := body["capabilities"].([]any)
|
|
if !ok {
|
|
t.Fatalf("capabilities = %#v, want an array", body["capabilities"])
|
|
}
|
|
got := make([]string, 0, len(caps))
|
|
for i, c := range caps {
|
|
token, ok := c.(string)
|
|
if !ok {
|
|
t.Fatalf("capabilities[%d] = %#v, want a string", i, c)
|
|
}
|
|
got = append(got, token)
|
|
}
|
|
want := httpapi.Capabilities()
|
|
if len(want) == 0 {
|
|
t.Fatal("this build derives no capabilities; the assertion below would pass against a server that advertises nothing")
|
|
}
|
|
// Every bd serve topology composes its roles through the external-dependency
|
|
// policy, so every one advertises the conditional policy token too.
|
|
want = append(want, httpapi.CapExternalDependencies)
|
|
slices.Sort(want)
|
|
if !slices.Equal(got, want) {
|
|
t.Errorf("capabilities = %v, want %v", got, want)
|
|
}
|
|
}
|
|
|
|
func (sp *serveProcess) get(t *testing.T, path string) (int, map[string]any, http.Header) {
|
|
t.Helper()
|
|
resp, err := sp.client.Get(sp.url(path))
|
|
if err != nil {
|
|
t.Fatalf("GET %s: %v\nstderr:\n%s", path, err, sp.stderr.String())
|
|
}
|
|
defer func() { _ = resp.Body.Close() }()
|
|
body, err := io.ReadAll(resp.Body)
|
|
if err != nil {
|
|
t.Fatalf("read %s: %v", path, err)
|
|
}
|
|
var m map[string]any
|
|
if len(body) > 0 {
|
|
if err := json.Unmarshal(body, &m); err != nil {
|
|
t.Fatalf("decode %s body %q: %v", path, body, err)
|
|
}
|
|
}
|
|
return resp.StatusCode, m, resp.Header
|
|
}
|
|
|
|
// shutdown sends the signal an operator or a supervisor sends, and requires a
|
|
// clean drain and exit 0.
|
|
func (sp *serveProcess) shutdown(t *testing.T) {
|
|
t.Helper()
|
|
if err := sp.cmd.Process.Signal(syscall.SIGTERM); err != nil {
|
|
t.Fatalf("SIGTERM: %v", err)
|
|
}
|
|
|
|
done := make(chan error, 1)
|
|
go func() { done <- sp.wait() }()
|
|
select {
|
|
case err := <-done:
|
|
if err != nil {
|
|
t.Fatalf("bd serve exited %v after SIGTERM, want a clean exit\nstderr:\n%s", err, sp.stderr.String())
|
|
}
|
|
case <-time.After(60 * time.Second):
|
|
_ = sp.cmd.Process.Kill()
|
|
t.Fatalf("bd serve did not exit after SIGTERM\nstderr:\n%s", sp.stderr.String())
|
|
}
|
|
}
|
|
|
|
func TestProxiedServerServeLifecycle(t *testing.T) {
|
|
requireSharedProxiedServer(t)
|
|
t.Parallel()
|
|
bd := buildEmbeddedBD(t)
|
|
p := newSharedProxiedProject(t, bd, "srvl")
|
|
|
|
sp := startServe(t, bd, p.dir, bdProxiedEnv(p.dir))
|
|
|
|
// Startup lines: what the operator needs before the first request, and the
|
|
// only record of which workspace this process is serving.
|
|
startup := sp.awaitLogLine(t, "event=startup")
|
|
for _, want := range []string{"addr=" + sp.addr, "mode=", "workspace=", "beads_dir="} {
|
|
if !strings.Contains(startup, want) {
|
|
t.Errorf("startup line is missing %q:\n%s", want, startup)
|
|
}
|
|
}
|
|
limits := sp.awaitLogLine(t, "event=limits")
|
|
for _, want := range []string{"max_inflight=", "max_conns=", "sem_wait=", "deadline=", "pool_max_open="} {
|
|
if !strings.Contains(limits, want) {
|
|
t.Errorf("limits line is missing %q:\n%s", want, limits)
|
|
}
|
|
}
|
|
|
|
t.Run("healthz", func(t *testing.T) {
|
|
status, body, header := sp.get(t, "/healthz")
|
|
if status != http.StatusOK {
|
|
t.Fatalf("status = %d, want 200", status)
|
|
}
|
|
if body["status"] != "ok" {
|
|
t.Errorf("body = %v", body)
|
|
}
|
|
if got := header.Get("Cache-Control"); got != "no-store" {
|
|
t.Errorf("Cache-Control = %q, want no-store", got)
|
|
}
|
|
})
|
|
|
|
t.Run("context", func(t *testing.T) {
|
|
status, body, _ := sp.get(t, "/v0/beads/context")
|
|
if status != http.StatusOK {
|
|
t.Fatalf("status = %d, want 200", status)
|
|
}
|
|
if body["api_version"] != "v0" {
|
|
t.Errorf("api_version = %v, want v0", body["api_version"])
|
|
}
|
|
if body["dolt_mode"] != "proxied-server" {
|
|
t.Errorf("dolt_mode = %v, want proxied-server", body["dolt_mode"])
|
|
}
|
|
if body["database"] != p.database {
|
|
t.Errorf("database = %v, want %q", body["database"], p.database)
|
|
}
|
|
if body["bd_version"] == "" || body["bd_version"] == nil {
|
|
t.Error("bd_version is empty")
|
|
}
|
|
assertServedCapabilities(t, body)
|
|
// The allowlist is enforced in internal/httpapi; this is the end-to-end
|
|
// half — a real workspace's real config, over a real socket.
|
|
for _, forbidden := range []string{"sync_remote", "server_host", "server_port", "data_dir", "proxied_dir", "role"} {
|
|
if _, present := body[forbidden]; present {
|
|
t.Errorf("context body carries %q: %v", forbidden, body)
|
|
}
|
|
}
|
|
})
|
|
|
|
t.Run("a read operation answers from the database", func(t *testing.T) {
|
|
status, body, header := sp.get(t, "/v0/beads/ready")
|
|
if status != http.StatusOK {
|
|
t.Fatalf("status = %d, want 200: %v", status, body)
|
|
}
|
|
if got := header.Get("Content-Type"); !strings.HasPrefix(got, "application/json") {
|
|
t.Errorf("Content-Type = %q, want json", got)
|
|
}
|
|
if _, ok := body["items"].([]any); !ok {
|
|
t.Errorf("items = %#v, want an array (never null)", body["items"])
|
|
}
|
|
if _, ok := body["has_more"].(bool); !ok {
|
|
t.Errorf("has_more = %#v, want a boolean", body["has_more"])
|
|
}
|
|
})
|
|
|
|
t.Run("foreign Host is refused", func(t *testing.T) {
|
|
req, err := http.NewRequest(http.MethodGet, sp.url("/healthz"), nil)
|
|
if err != nil {
|
|
t.Fatalf("new request: %v", err)
|
|
}
|
|
req.Host = "evil.example"
|
|
resp, err := sp.client.Do(req)
|
|
if err != nil {
|
|
t.Fatalf("GET with foreign Host: %v", err)
|
|
}
|
|
defer func() { _ = resp.Body.Close() }()
|
|
if resp.StatusCode != http.StatusBadRequest {
|
|
t.Fatalf("status = %d, want 400 (DNS-rebinding defense)", resp.StatusCode)
|
|
}
|
|
var body map[string]any
|
|
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
|
|
t.Fatalf("decode: %v", err)
|
|
}
|
|
if body["code"] != "invalid_argument" || body["param"] != "Host" {
|
|
t.Errorf("body = %v, want invalid_argument on param Host", body)
|
|
}
|
|
// The rebinding defense has to leave a trace naming what it refused,
|
|
// or an actual probe is invisible server-side.
|
|
sp.awaitLogLine(t, "refused=evil.example")
|
|
})
|
|
|
|
t.Run("unrouted paths keep the error shape", func(t *testing.T) {
|
|
status, body, header := sp.get(t, "/v0/beads/nope")
|
|
if status != http.StatusNotFound {
|
|
t.Fatalf("status = %d, want 404", status)
|
|
}
|
|
if got := header.Get("Content-Type"); !strings.HasPrefix(got, "application/problem+json") {
|
|
t.Errorf("Content-Type = %q, want problem+json", got)
|
|
}
|
|
if body["code"] != "not_found" {
|
|
t.Errorf("code = %v", body["code"])
|
|
}
|
|
})
|
|
|
|
// One line per request, with the fields that separate "the database is
|
|
// slow" from "the server is saturated" from "the handler is broken".
|
|
requests := []struct{ path, op, status string }{
|
|
{"/healthz", "health", "200"},
|
|
{"/v0/beads/context", "getContext", "200"},
|
|
{"/v0/beads/ready", "listReadyWork", "200"},
|
|
}
|
|
for _, want := range requests {
|
|
line := sp.awaitLogLine(t, "path="+want.path+" ")
|
|
for _, field := range []string{
|
|
"request_id=", "op=" + want.op, "method=GET", "status=" + want.status,
|
|
"duration_ms=", "sem_wait_ms=", "uow_ms=",
|
|
// The connection gauge and the caller. The cap is enforced by the
|
|
// listener, so this line is where an operator watches it climb;
|
|
// on loopback the remote port names the calling process.
|
|
"conns=", "remote_addr=127.0.0.1:",
|
|
} {
|
|
if !strings.Contains(line, field) {
|
|
t.Errorf("request log line for %s is missing %q:\n%s", want.path, field, line)
|
|
}
|
|
}
|
|
}
|
|
|
|
sp.shutdown(t)
|
|
|
|
// Wait has returned, so the stream copiers are done: read directly.
|
|
stderr := sp.stderr.String()
|
|
for _, want := range []string{"event=shutdown_start", "event=shutdown_complete"} {
|
|
if !strings.Contains(stderr, want) {
|
|
t.Errorf("shutdown log is missing %q:\n%s", want, stderr)
|
|
}
|
|
}
|
|
// The listener is released, which is what makes "the TCP bind is the mutual
|
|
// exclusion" true rather than aspirational.
|
|
if _, err := http.Get(sp.url("/healthz")); err == nil {
|
|
t.Error("server still answering after shutdown")
|
|
}
|
|
}
|
|
|
|
// TestProxiedServerServeModeGate covers the refusals an operator can actually
|
|
// hit from the command line.
|
|
//
|
|
// The dolt-server-mode staged refusal is NOT driven end to end here: this
|
|
// harness provisions proxied and embedded workspaces, not a `bd init --server`
|
|
// one, and building that topology for a message assertion would trade a lot of
|
|
// setup for no extra coverage. The typed error and its wording — including that
|
|
// it does not read as permanent — are pinned in TestServeRefusalsPromiseNothing
|
|
// (cmd/bd/serve_test.go), which is pure and runs in the PR workflow's
|
|
// unconditional Go test job.
|
|
func TestProxiedServerServeModeGate(t *testing.T) {
|
|
requireProxiedServerEnv(t)
|
|
t.Parallel()
|
|
bd := buildEmbeddedBD(t)
|
|
|
|
t.Run("embedded workspace is refused permanently", func(t *testing.T) {
|
|
t.Parallel()
|
|
dir, _, _ := bdInit(t, bd)
|
|
|
|
cmd := exec.Command(bd, "serve")
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
stdout, stderr, err := runCommandBuffers(t, cmd)
|
|
if err == nil {
|
|
t.Fatalf("bd serve succeeded in an embedded workspace\nstdout:\n%s", stdout.String())
|
|
}
|
|
assertServeExitCode(t, err, 1)
|
|
|
|
out := stderr.String()
|
|
if !strings.Contains(out, "not supported by the embedded-dolt backend") {
|
|
t.Errorf("refusal is not the typed ErrUnsupported message:\n%s", out)
|
|
}
|
|
if !strings.Contains(out, "embedded Dolt") {
|
|
t.Errorf("refusal does not name the workspace's backend:\n%s", out)
|
|
}
|
|
})
|
|
|
|
t.Run("non-loopback bind needs the opt-in", func(t *testing.T) {
|
|
t.Parallel()
|
|
dir, _, _ := bdInit(t, bd)
|
|
|
|
cmd := exec.Command(bd, "serve", "--addr", "10.0.0.5:8080")
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
_, stderr, err := runCommandBuffers(t, cmd)
|
|
if err == nil {
|
|
t.Fatal("bd serve accepted a non-loopback address without --allow-non-loopback")
|
|
}
|
|
assertServeExitCode(t, err, 1)
|
|
// Checked before anything about the workspace, so the refusal is the
|
|
// same in every mode — including this embedded one, which would
|
|
// otherwise be refused first for a different reason.
|
|
if !strings.Contains(stderr.String(), "--allow-non-loopback") {
|
|
t.Errorf("refusal does not name the opt-in flag:\n%s", stderr.String())
|
|
}
|
|
})
|
|
|
|
t.Run("a hostname is not an address", func(t *testing.T) {
|
|
t.Parallel()
|
|
dir, _, _ := bdInit(t, bd)
|
|
|
|
cmd := exec.Command(bd, "serve", "--addr", "localhost:0")
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
_, stderr, err := runCommandBuffers(t, cmd)
|
|
if err == nil {
|
|
t.Fatal("bd serve accepted a hostname as --addr")
|
|
}
|
|
assertServeExitCode(t, err, 1)
|
|
if !strings.Contains(stderr.String(), "numeric IP literal") {
|
|
t.Errorf("refusal does not explain the numeric-IP rule:\n%s", stderr.String())
|
|
}
|
|
})
|
|
}
|
|
|
|
// assertServeExitCode reads a SUBPROCESS exit status. The package's
|
|
// assertExitCode unwraps the in-process *exitError instead, which a forked bd
|
|
// never produces.
|
|
func assertServeExitCode(t *testing.T, err error, want int) {
|
|
t.Helper()
|
|
var exit *exec.ExitError
|
|
if !errors.As(err, &exit) {
|
|
t.Fatalf("error %v is not a subprocess exit status", err)
|
|
}
|
|
if got := exit.ExitCode(); got != want {
|
|
t.Errorf("exit code = %d, want %d", got, want)
|
|
}
|
|
}
|
|
|
|
// awaitLogLine returns the first stderr line containing needle, waiting for it
|
|
// to arrive. os/exec copies the subprocess's stderr on its own goroutine, so a
|
|
// line the server has already written is not necessarily one this process has
|
|
// already read — polling is the difference between a test and a flaky test.
|
|
func (sp *serveProcess) awaitLogLine(t *testing.T, needle string) string {
|
|
t.Helper()
|
|
deadline := time.Now().Add(30 * time.Second)
|
|
for {
|
|
log := sp.stderr.String()
|
|
for _, line := range strings.Split(log, "\n") {
|
|
if strings.Contains(line, needle) {
|
|
return line
|
|
}
|
|
}
|
|
if time.Now().After(deadline) {
|
|
t.Fatalf("no log line containing %q in:\n%s", needle, log)
|
|
}
|
|
time.Sleep(20 * time.Millisecond)
|
|
}
|
|
}
|